To encode an XML document using UTF-8

Google Sitemap Service requires that all site maps published must be UTF-8 encoded in Unicode. Google does not even allow other Unicode encodings (such as UTF-16), let alone iso-8859-1 such a non-Unicode encoding. Technically, this means that Google uses a nonstandard XML parser because XML recommendation specifically requires that "all XML handlers must accept the UTF-8 and UTF-16 encoding of Unicode 3.1", but is this really a big problem?

Everyone can use UTF-8.

Universality is the first and most persuasive reason for choosing UTF-8. It can handle every type of text currently used in the world. Although there are a few gaps, but increasingly not obvious, is gradually filled. Text that is not included is usually not implemented in any other character set, even if it is not used in XML. In the best case, these literals are transferred to Latin-1 such as the single-byte character set through font borrowing. The true support for this type of rare text may come first from Unicode and may only be supported by Unicode.

But this is only one reason to use Unicode. Why choose UTF-8 instead of UTF-16 or other Unicode encodings? One of the most direct reasons is the extensive tool support. Basically all the major editors that are likely to be used in XML can handle UTF-8, including JEdit, BBEdit, Eclipse, Emacs, and even Notepad. In XML and non-XML tools, no other UNICODE encoding has such a wide range of tool support.

Some of these editors, such as BBEdit and Eclipse,utf-8, are not the default character sets. It is now necessary to change the default settings, and all tools should choose UTF-8 as the default encoding when they leave the factory. Unless that happens, when files are passed across borders, platforms, and languages, we are stuck in a quagmire that cannot interoperate. However, it is also easy to modify the default settings yourself before all programs have UTF-8 as the default encoding. For example, in Eclipse, the "General/editors" preferences panel shown in Figure 1 allows you to specify that all files use UTF-8. You may notice that Eclipse expects the default value to be Macroman, but if so, it will not compile if you pass the file to a programmer using microsoft®windows® or to a computer outside the United States and Western Europe.

Figure 1. Change Eclipse's default character set

Of course, for UTF-8 to work, the files that developers exchange must also use UTF-8, but that's not a problem. Unlike Macroman, UTF-8 is not limited to a few words or individual platforms. Anyone can use UTF-8. Macroman, Latin-1, Sjis and other legacy national character sets are not possible.

UTF-8 works well in tools that do not support multibyte data. Other Unicode formats, such as UTF-16, often contain a lot of 0 bytes. Many tools interpret these bytes as the end of a file or some other special kind of delimiter, creating unintended, unpredictable, and often unpleasant results. For example, if the UTF-16 data is loaded into the C string, the string may be truncated from the second byte of the first ASCII character. The UTF-8 file contains null only where NULL is actually represented. Of course, you should not choose such naïve tools to process XML documents. However, documents in legacy systems often end up in strange places, and no one really realizes or understands those sequences of characters that are just old bottles of new wine. Compared to UTF-16 or other Unicode encodings, UTF-8 is less likely to cause problems for systems that do not support Unicode and XML.