Encoding Standard

Time 2020-07-22 17:05:52

Web Name: Encoding Standard

WebSite: http://encoding.spec.whatwg.org

ID:31935

Keywords:

Encoding,Standard,

Description:

1. Preface The UTF-8 encoding is the most appropriate encoding for interchange of Unicode, theuniversal coded character set. Therefore for new protocols and formats, as well asexisting formats deployed in new contexts, this specification requires (and defines) theUTF-8 encoding. The other (legacy) encodings have been defined to some extent in the past. However,user agents have not always implemented them in the same way, have not always used thesame labels, and often differ in dealing with undefined and former proprietary areas ofencodings. This specification addresses those gaps so that new user agents do not have toreverse engineer encoding implementations and existing user agents can converge. In particular, this specification defines all those encodings, their algorithms to gofrom bytes to scalar values and back, and their canonical names and identifying labels.This specification also defines an API to expose part of the encoding algorithms toJavaScript. User agents have also significantly deviated from the labels listed in the IANA Character Sets registry.To stop spreading legacy encodings further, this specification is exhaustive about theaforementioned details and therefore has no need for the registry. In particular, thisspecification does not provide a mechanism for extending any aspect of encodings. 2. Security background There is a set of encoding security issues when the producer and consumer do not agreeon the encoding in use, or on the way a given encoding is to be implemented. For instance,an attack was reported in 2011 where a Shift_JIS lead byte 0x82 was used to“mask” a 0x22 trail byte in a JSON resource of which an attacker could control some field.The producer did not see the problem even though this is an illegal byte combination. Theconsumer decoded it as a single U+FFFD and therefore changed the overall interpretation asU+0022 is an important delimiter. Decoders of encodings that use multiple bytes for scalarvalues now require that in case of an illegal byte combination, a scalar value in therange U+0000 to U+007F, inclusive, cannot be “masked”. For the aforementioned sequence theoutput would be U+FFFD U+0022. This is a larger issue for encodings that map anything that is an ASCII byte tosomething that is not an ASCII code point, when there is no lead byte present. Theseare “ASCII-incompatible” encodings and other than ISO-2022-JP, UTF-16BE,and UTF-16LE, which are unfortunately required due to deployed content, they are notsupported. (Investigation is ongoing whether more labels of other such encodings can be mapped to the replacement encoding, rather than the unknown encoding fallback.) An example attack is injectingcarefully crafted content into a resource and then encouraging the user to override theencoding, resulting in e.g. script execution. Encoders used by URLs found in HTML and HTML’s form feature can also result in slightinformation loss when an encoding is used that cannot represent all scalar values. E.g.when a resource uses the windows-1252 encoding a server will not be able todistinguish between an end user entering “

TAGS:Encoding Standard 

<<< Thank you for your visit >>>

Hot Websites