Jump to content

GB 18030

From Wikipedia, the free encyclopedia
(Redirected from Code page 54936)
GB 18030
GB 18030 encoding layout. "Half codes" indicates codes used in pairs as four-byte codes.
MIME / IANAGB18030
Alias(es)Code page 54936
Language(s)International, but primarily meant for Chinese
StandardGB 18030-2022, GB 18030-2005, GB 18030-2000
ClassificationUnicode Transformation Format, extended ASCII,[a] variable-width encoding, CJK encoding
ExtendsEUC-CN, GBK
Transforms / EncodesISO 10646 (Unicode)
Preceded byGBK, GB2312
  1. ^ Not in the strictest sense of the term, as ASCII bytes can appear as trail bytes.

GB 18030 is a Chinese government standard, described as Information Technology — Chinese coded character set and defines the required language and character support necessary for software in China. GB18030 is the registered Internet name for the official character set of the People's Republic of China (PRC) superseding GB2312.[1] As a Unicode Transformation Format[a] (i.e. an encoding of all Unicode code points), GB18030 supports both simplified and traditional Chinese characters. It is also compatible with legacy encodings including GB/T 2312, CP936,[b] and GBK 1.0.

The Unicode Consortium has warned implementers that the latest version of this Chinese standard, GB 18030-2022, introduces what they describe as "disruptive changes" from the previous version GB 18030-2005 "involving 33 different characters and 55 code positions".[2] GB 18030-2022 was enforced from 1 August 2023.[3] It has been implemented in ICU 73.2; and in Java 21,[4] and backported to older Java 8, 11, 17 (LTS releases) and 20.0.2.[5]

In addition to the encoding method, this standard contains requirements about which additional scripts and languages should be represented, and to whom this standard is applicable.[6] This standard however does not define the official character forms for the Chinese characters; this is standardised in List of Commonly Used Standard Chinese Characters.

History

[edit]
See also: GB 2312, GBK (character encoding), and Code page 936 (Microsoft Windows)

The GB18030 character set is formally called "Chinese National Standard GB 18030-2005: Information Technology—Chinese coded character set". GB abbreviates Guójiā Biāozhǔn (国家标准), which means national standard in Chinese. The standard was published by the China Standard Press, Beijing, 8 November 2005. Only a portion of the standard is mandatory.[6] Since 1 May 2006, support for the mandatory subset is officially required for all software products sold in the PRC.

Different Unicode mappings between GB 18030 versions
GB byte
sequence		 Unicode code point
GB 18030-2000 GB 18030-2005
A8 BC (ḿ) U+E7C7 U+1E3F ḿ LATIN SMALL LETTER M WITH ACUTE
81 35 F4 37 U+1E3F ḿ LATIN SMALL LETTER M WITH ACUTE U+E7C7

An older version of the standard, known as "Chinese National Standard GB 18030-2000: Information Technology—Chinese ideograms coded character set for information interchange—Extension for the basic set", was published on March 17, 2000. The encoding scheme stays the same in the new version, and the only difference in GB-to-Unicode mapping is that GB 18030-2000 mapped the character A8 BC (ḿ) to a private use code point U+E7C7, and character 81 35 F4 37 (without specifying any glyph) to U+1E3F (ḿ), whereas GB 18030-2005 swaps these two mapping assignments.[7]: 534  More code points are now associated with characters due to update of Unicode, especially the appearance of CJK Unified Ideographs Extension B. Some characters used by ethnic minorities in China, such as Mongolian characters and Tibetan characters (GB 16959-1997 and GB/T 20542-2006), have been added as well, which accounts for the renaming of the standard.

Compared with its ancestors, GB 18030's mapping to Unicode has been modified for the 81 characters that were provisionally assigned a Unicode Private Use Area code point (U+E000–F8FF) in GBK 1.0 and that have later been encoded in Unicode.[8] This is specified in Appendix E of GB 18030.[7]: 534 [9]: 499  There are 24 characters in GB 18030-2005 that are still mapped to Unicode PUA.[10]

In the GB 18030-2022 update, the requirements for characters to be mapped to PUA has been lifted completely and all characters should be mapped to their standard Unicode codepoints. Of these, 18 mappings were updated by position-swapping similar to what happened between GBK and GB 18030. The remaining six kept the two-byte PUA mappings, so that a change to the 4-byte sequence is needed to follow the non-PUA preference.[11]

Private use characters in GB-to-Unicode mappings
GB byte
sequence		 Unicode code point [a]
GBK 1.0[12][7]: 534  GB 18030-2005[10] Unicode 4.1 GB 18030-2022[11]
A6 D9[13]: 108  U+E78D U+FE10 ︐ PRESENTATION FORM FOR VERTICAL COMMA
A6 DA U+E78E U+FE12 ︒ PRESENTATION FORM FOR VERTICAL IDEOGRAPHIC FULL STOP
A6 DB U+E78F U+FE11 ︑ PRESENTATION FORM FOR VERTICAL IDEOGRAPHIC COMMA
A6 DC U+E790 U+FE13 ︓ PRESENTATION FORM FOR VERTICAL COLON
A6 DD U+E791 U+FE14 ︔ PRESENTATION FORM FOR VERTICAL SEMICOLON
A6 DE U+E792 U+FE15 ︕ PRESENTATION FORM FOR VERTICAL EXCLAMATION MARK
A6 DF U+E793 U+FE16 ︖ PRESENTATION FORM FOR VERTICAL QUESTION MARK
A6 EC U+E794 U+FE17 ︗ PRESENTATION FORM FOR VERTICAL LEFT WHITE LENTICULAR BRACKET
A6 ED U+E795 U+FE18 ︘ PRESENTATION FORM FOR VERTICAL RIGHT WHITE LENTICULAR BRAKCET
A6 F3 U+E796 U+FE19 ︙ PRESENTATION FORM FOR VERTICAL HORIZONTAL ELLIPSIS
A8 BC U+E7C7 U+1E3F ḿ LATIN SMALL LETTER M WITH ACUTE
A8 BF U+E7C8 U+01F9 ǹ LATIN SMALL LETTER N WITH GRAVE
A9 89 U+E7E7 U+303E 〾 IDEOGRAPHIC VARIATION INDICATOR
A9 8A U+E7E8 U+2FF0 ⿰ IDEOGRAPHIC DESCRIPTION CHARACTER LEFT TO RIGHT
A9 8B U+E7E9 U+2FF1 ⿱ IDEOGRAPHIC DESCRIPTION CHARACTER ABOVE TO BELOW
A9 8C U+E7EA U+2FF2 ⿲ IDEOGRAPHIC DESCRIPTION CHARACTER LEFT TO MIDDLE AND RIGHT
A9 8D U+E7EB U+2FF3 ⿳ IDEOGRAPHIC DESCRIPTION CHARACTER ABOVE TO MIDDLE AND BELOW
A9 8E U+E7EC U+2FF4 ⿴ IDEOGRAPHIC DESCRIPTION CHARACTER FULL SURROUND
A9 8F U+E7ED U+2FF5 ⿵ IDEOGRAPHIC DESCRIPTION CHARACTER SURROUND FROM ABOVE
A9 90 U+E7EE U+2FF6 ⿶ IDEOGRAPHIC DESCRIPTION CHARACTER SURROUND FROM BELOW
A9 91 U+E7EF U+2FF7 ⿷ IDEOGRAPHIC DESCRIPTION CHARACTER SURROUND FROM LEFT
A9 92 U+E7F0 U+2FF8 ⿸ IDEOGRAPHIC DESCRIPTION CHARACTER SURROUND FROM UPPER LEFT
A9 93 U+E7F1 U+2FF9 ⿹ IDEOGRAPHIC DESCRIPTION CHARACTER SURROUND FROM UPPER RIGHT
A9 94[13]: 173  U+E7F2 U+2FFA ⿺ IDEOGRAPHIC DESCRIPTION CHARACTER SURROUND FROM LOWER LEFT
A9 95 U+E7F3 U+2FFB ⿻ IDEOGRAPHIC DESCRIPTION CHARACTER OVERLAID
FE 50 U+E815 U+2E81 ⺁ CJK RADICAL CLIFF
FE 51 U+E816 U+20087 𠂇 CJK UNIFIED IDEOGRAPH-20087[b] U+E816
FE 52 U+E817 U+20089 𠂉 CJK UNIFIED IDEOGRAPH-20089[c] U+E817
FE 53 U+E818 U+200CC 𠃌 CJK UNIFIED IDEOGRAPH-200CC[d] U+E818
FE 54 U+E819 U+2E84 ⺄ CJK RADICAL SECOND THREE
FE 55 U+E81A U+3473 㑳 CJK UNIFIED IDEOGRAPH-3473
FE 56 U+E81B U+3447 㑇 CJK UNIFIED IDEOGRAPH-3447
FE 57 U+E81C U+2E88 ⺈ CJK RADICAL KNIFE ONE
FE 58 U+E81D U+2E8B ⺋ CJK RADICAL SEAL
FE 59 U+E81E U+9FB4 龴 CJK UNIFIED IDEOGRAPH-9FB4
FE 5A U+E81F U+359E 㖞 CJK UNIFIED IDEOGRAPH-359E
FE 5B U+E820 U+361A 㘚 CJK UNIFIED IDEOGRAPH-361A
FE 5C U+E821 U+360E 㘎 CJK UNIFIED IDEOGRAPH-360E
FE 5D U+E822 U+2E8C ⺌ CJK RADICAL SMALL ONE
FE 5E U+E823 U+2E97 ⺗ CJK RADICAL HEART TWO
FE 5F U+E824 U+396E 㥮 CJK UNIFIED IDEOGRAPH-396E
FE 60 U+E825 U+3918 㤘 CJK UNIFIED IDEOGRAPH-3918
FE 61 U+E826 U+9FB5 龵 CJK UNIFIED IDEOGRAPH-9FB5
FE 62 U+E827 U+39CF 㧏 CJK UNIFIED IDEOGRAPH-39CF
FE 63 U+E828 U+39DF 㧟 CJK UNIFIED IDEOGRAPH-39DF
FE 64 U+E829 U+3A73 㩳 CJK UNIFIED IDEOGRAPH-3A73
FE 65 U+E82A U+39D0 㧐 CJK UNIFIED IDEOGRAPH-39D0
FE 66 U+E82B U+9FB6 龶 CJK UNIFIED IDEOGRAPH-9FB6
FE 67 U+E82C U+9FB7 龷 CJK UNIFIED IDEOGRAPH-9FB7
FE 68 U+E82D U+3B4E 㭎 CJK UNIFIED IDEOGRAPH-3B4E
FE 69 U+E82E U+3C6E 㱮 CJK UNIFIED IDEOGRAPH-3C6E
FE 6A U+E82F U+3CE0 㳠 CJK UNIFIED IDEOGRAPH-3CE0
FE 6B U+E830 U+2EA7 ⺧ CJK RADICAL COW
FE 6C U+E831 U+215D7 𡗗 CJK UNIFIED IDEOGRAPH-215D7[e] U+E831
FE 6D U+E832 U+9FB8 龸 CJK UNIFIED IDEOGRAPH-9FB8
FE 6E U+E833 U+2EAA ⺪ CJK RADICAL BOLT OF CLOTH
FE 6F U+E834 U+4056 䁖 CJK UNIFIED IDEOGRAPH-4056
FE 70 U+E835 U+415F 䅟 CJK UNIFIED IDEOGRAPH-415F
FE 71 U+E836 U+2EAE ⺮ CJK RADICAL BAMBOO
FE 72 U+E837 U+4337 䌷 CJK UNIFIED IDEOGRAPH-4337
FE 73 U+E838 U+2EB3 ⺳ CJK RADICAL NET THREE
FE 74 U+E839 U+2EB6 ⺶ CJK RADICAL SHEEP
FE 75 U+E83A U+2EB7 ⺷ CJK RADICAL RAM
FE 76 U+E83B U+2298F 𢦏 CJK UNIFIED IDEOGRAPH-2298F[f] U+E83B
FE 77 U+E83C U+43B1 䎱 CJK UNIFIED IDEOGRAPH-43B1
FE 78 U+E83D U+43AC 䎬 CJK UNIFIED IDEOGRAPH-43AC
FE 79 U+E83E U+2EBB ⺻ CJK RADICAL BRUSH TWO
FE 7A U+E83F U+43DD 䏝 CJK UNIFIED IDEOGRAPH-43DD
FE 7B U+E840 U+44D6 䓖 CJK UNIFIED IDEOGRAPH-44D6
FE 7C U+E841 U+4661 䙡 CJK UNIFIED IDEOGRAPH-4661
FE 7D U+E842 U+464C 䙌 CJK UNIFIED IDEOGRAPH-464C
FE 7E U+E843 U+9FB9 龹 CJK UNIFIED IDEOGRAPH-9FB9
FE 80 U+E844 U+4723 䜣 CJK UNIFIED IDEOGRAPH-4723
FE 81 U+E845 U+4729 䜩 CJK UNIFIED IDEOGRAPH-4729
FE 82 U+E846 U+477C 䝼 CJK UNIFIED IDEOGRAPH-477C
FE 83 U+E847 U+478D 䞍 CJK UNIFIED IDEOGRAPH-478D
FE 84 U+E848 U+2ECA ⻊ CJK RADICAL FOOT
FE 85 U+E849 U+4947 䥇 CJK UNIFIED IDEOGRAPH-4947
FE 86 U+E84A U+497A 䥺 CJK UNIFIED IDEOGRAPH-497A
FE 87 U+E84B U+497D 䥽 CJK UNIFIED IDEOGRAPH-497D
FE 88 U+E84C U+4982 䦂 CJK UNIFIED IDEOGRAPH-4982
FE 89 U+E84D U+4983 䦃 CJK UNIFIED IDEOGRAPH-4983
FE 8A U+E84E U+4985 䦅 CJK UNIFIED IDEOGRAPH-4985
FE 8B U+E84F U+4986 䦆 CJK UNIFIED IDEOGRAPH-4986
FE 8C U+E850 U+499F 䦟 CJK UNIFIED IDEOGRAPH-499F
FE 8D U+E851 U+499B 䦛 CJK UNIFIED IDEOGRAPH-499B
FE 8E U+E852 U+49B7 䦷 CJK UNIFIED IDEOGRAPH-49B7
FE 8F U+E853 U+49B6 䦶 CJK UNIFIED IDEOGRAPH-49B6
FE 90 U+E854 U+9FBA 龺 CJK UNIFIED IDEOGRAPH-9FBA
FE 91 U+E855 U+241FE 𤇾 CJK UNIFIED IDEOGRAPH-241FE[g] U+E855
FE 92 U+E856 U+4CA3 䲣 CJK UNIFIED IDEOGRAPH-4CA3
FE 93 U+E857 U+4C9F 䲟 CJK UNIFIED IDEOGRAPH-4C9F
FE 94 U+E858 U+4CA0 䲠 CJK UNIFIED IDEOGRAPH-4CA0
FE 95 U+E859 U+4CA1 䲡 CJK UNIFIED IDEOGRAPH-4CA1
FE 96 U+E85A U+4C77 䱷 CJK UNIFIED IDEOGRAPH-4C77
FE 97 U+E85B U+4CA2 䲢 CJK UNIFIED IDEOGRAPH-4CA2
FE 98 U+E85C U+4D13 䴓 CJK UNIFIED IDEOGRAPH-4D13
FE 99 U+E85D U+4D14 䴔 CJK UNIFIED IDEOGRAPH-4D14
FE 9A U+E85E U+4D15 䴕 CJK UNIFIED IDEOGRAPH-4D15
FE 9B U+E85F U+4D16 䴖 CJK UNIFIED IDEOGRAPH-4D16
FE 9C U+E860 U+4D17 䴗 CJK UNIFIED IDEOGRAPH-4D17
FE 9D U+E861 U+4D18 䴘 CJK UNIFIED IDEOGRAPH-4D18
FE 9E U+E862 U+4D19 䴙 CJK UNIFIED IDEOGRAPH-4D19
FE 9F U+E863 U+4DAE 䶮 CJK UNIFIED IDEOGRAPH-4DAE
FE A0 U+E864 U+9FBB 龻 CJK UNIFIED IDEOGRAPH-9FBB
Notes
a.^ Blue indicates private use area
b.^ U+20087 𠂇 CJK UNIFIED IDEOGRAPH-20087 mapped to 0x95329031 in GB 18030-2022
c.^ U+20089 𠂉 CJK UNIFIED IDEOGRAPH-20089 mapped to 0x95329033 in GB 18030-2022
d.^ U+200CC 𠃌 CJK UNIFIED IDEOGRAPH-200CC mapped to 0x95329730 in GB 18030-2022
e.^ U+215D7 𡗗 CJK UNIFIED IDEOGRAPH-215D7 mapped to 0x9536B937 in GB 18030-2022
f.^ U+2298F 𢦏 CJK UNIFIED IDEOGRAPH-2298F mapped to 0x9630BA35 in GB 18030-2022
g.^ U+241FE 𤇾 CJK UNIFIED IDEOGRAPH-241FE mapped to 0x9635B630 in GB 18030-2022

As a national standard

[edit]

The first version of GB 18030, designated GB 18030-2000 Information Technology—Chinese coded character set for information interchange — Extension for the basic set, consists of 1-byte and 2-byte encodings, together with 4-byte encoding for CJK Unified Ideographs Extension A matching those in Unicode 3.0. The corresponding Unicode code points of this subset, including provisional private assignments, lie entirely in the BMP. These parts are fully mandatory in GB 18030-2000.[6]: 2  Most major computer companies had already standardized on some version of Unicode as the primary format for use in their binary formats and OS calls. However, they mostly had only supported code points in the BMP originally defined in Unicode 1.0, which supported only 65,536 codepoints and was often encoded in 16 bits as UCS-2. This standard is basically an extension based on GBK with additional characters in CJK Unified Ideographs Extension A.

The second version designated GB 18030-2005 Information Technology—Chinese coded character set has the same mandatory subset as GB 18030-2000 of 1-, 2- and 4-byte encodings.[7]: 3  This version also includes the full CJK Unified Ideographs Extension B in the 4-byte encoding section which is outside the BMP[10] as a suggestion support requirement.[14] However, as the inclusion of CJK Unified Ideographs Extension B in a 4-byte region is required to be maintained during information processing, software can no longer get away with treating characters as 16-bit fixed width entities (UCS-2). Therefore, they must either process the data as a variable-width format (as with UTF-8 or UTF-16), which is the most common choice, or move to a larger fixed-width format (i.e. UTF-32). Microsoft made the change from UCS-2 to UTF-16 with Windows 2000. This version matches with Unicode 3.1, and also provided support for Hangul (Korean), Mongolian (including Manchu, Clear script, Sibe hergen, Galik), Tai Nuea, Tibetan, Uyghur/Kazakh/Kyrgyz and Yi.

The third and latest version, GB 18030-2022 Information Technology—Chinese coded character set, mandates the suggestion support part of CJK Unified Ideographs Extension B in GB 18030-2005, along with updates up to Unicode 11.0 including Kangxi Radicals and CJK Unified Ideographs URO, Extension C, D, E and F. Additional languages are also recognized by GB 18030-2022 such as part of Arabic, Tai Le, New Tai Lue, Tai Tham, Lisu, and Miao. GB 18030-2022 also introduces three implementation levels, with the requirement of "all products using this standard should implements Implementation Level 1" that includes 66 new BMP characters in the 4-byte encoding region that were added between Unicode 3.1 and Unicode 11.0. Implementation Level 2 requires the support of List of Commonly Used Standard Chinese Characters, and Implementation Level 3 requires all other specified regions in the standard.[11]

Main article: CJK Unified Ideographs Extension I

From late 2022 to 2023, drafts of a further amendment are to be made to GB 18030-2022 available for public consultation. The current draft updates up to Unicode 15.1 on Ideographic Description Characters, CJK Unified Ideographs URO, Extension A, B, C, G, H and I.[15][16][17] Originally, in late 2022, it would have placed 897 new sinographic characters in Plane 10 (hexadecimal: 0A), a yet-untitled astral Unicode plane, for citizen real-name certification in China, but eventually the repertoire (reduced to 622 characters after expert review) was fast-tracked into Unicode 15.1 in September 2023, as the CJK Unified Ideographs Extension I block.[18] Following this, the amendment draft was modified to use the Extension I code points.[17]

Mapping

[edit]

GB 18030 defines a one (ASCII), two (extended GBK), or four-byte (UTF) encoding. The two-byte codes are defined in a lookup table, while the four-byte codes are defined sequentially (hence algorithmically) to fill otherwise unencoded parts in UCS. GB 18030 inherits the bad aspects of GBK, most notably needing special code to safely find ASCII characters in a GB18030 sequence.

GB 18030 encoding[7]: 3 [9]: 252 [19]
GB 18030 code points[c] Unicode
byte 1 (MSB) byte 2 byte 3 byte 4
00 – 7F 128 0000 – 007F
80 — invalid[d]
81 – FE 40 – FE except 7F[e] 23940 0080 – FFFF except D800 – DFFF[f]
81 – 84 30 – 39 81 – FE 30 – 39 39420
85 — (12600) reserved for future character extension
86 – 8F — (126000) reserved for future ideographic extension
unassigned — D800 – DFFF[g]
90 – E3 30 – 39 81 – FE 30 – 39 1048576 10000 – 10FFFF
E4 – FC — (315000) reserved for future standard extension
FD – FE — (25200) user-defined
FF — invalid
Total 1112064

The one- and two-byte code points are essentially GBK with the euro sign, PUA mappings for unassigned/user-defined points, and vertical punctuations. The four byte scheme can be thought of as consisting of two units, each of two bytes. Each unit has a similar format to a GBK two byte character but with a range of values for the second byte of 0x30–0x39 (the ASCII codes for decimal digits). The first byte has the range 0x81 to 0xFE, as before. This means that a string-search routine that is safe for GBK should also be reasonably[clarification needed] safe for GB18030 (in much the same way that a basic byte-oriented search routine is reasonably safe for EUC).

This gives a total of 1,587,600 (126×10×126×10) possible 4 byte sequences, which is easily sufficient to cover Unicode's 1,112,064 (17×65536 − 2048 surrogates) assigned, reserved, and noncharacter code points.

Unfortunately, to further complicate matters there are no simple rules to translate between a 4 byte sequence and its corresponding code point. Instead, codes are allocated sequentially (with the first byte containing the most significant part and the last the least significant part) only to Unicode code points that are not mapped in any other manner.[h] For example: 

U+00DE (Þ) → 81 30 89 37
U+00DF (ß) → 81 30 89 38
U+00E0 (à) → A8 A4
U+00E1 (á) → A8 A2
U+00E2 (â) → 81 30 89 39
U+00E3 (ã) → 81 30 8A 30

An offset table is used in the WHATWG and W3C version of GB 18030 to efficiently translate code points.[20] ICU[19] and glibc use similar range definitions to avoid wasting space on large sequential blocks.

Support

[edit]

Encoding

[edit]

GB 18030 has been supported on Windows since the release of Windows 95, as code page 54936.[21] Windows 2000 and XP offer a GB18030 Support Package.[22] The open source PostgreSQL database supports GB18030 through its full support for UTF-8, i.e. by converting it to and from UTF-8. Similarly Microsoft SQL Server supports GB18030 by conversion to and from UTF-16.

More specifically, supporting the GB18030 encoding on Windows means that Code Page 54936 is supported by MultiByteToWideChar and WideCharToMultiByte. Due to the backward compatibility of the mapping, many files in GB18030 can be actually opened successfully as the legacy Code Page 936, that is GBK, even if the Code Page 54936 is not supported. However, that is only true if the file in question contains only GBK characters. Loading will fail or cause corrupted result if the file contains characters that do not exist in GBK (see § Technical details for examples).

GNU glibc's gconv, the character codec library used on most Linux distributions, supports GB 18030-2000 since 2.2,[23] and GB 18030-2005 since 2.14;[24] glibc notably includes non-PUA mappings for GB 18030-2005 in order to achieve round-trip conversion.[25] GNU libiconv, an alternative iconv implementation frequently used on non-glibc UNIX-like environments like Cygwin, supports GB 18030 since version 1.4.[26]

As of 2022, "supporting non-Chinese scripts continues to be optional"[27] (presumably for display/font support only; and in China, since the encoding is a full UTF). The standard is known to support English/ASCII and the "following non-Chinese scripts are recognized by GB 18030-2022: Arabic, Tibetan, Mongolian, Tai Le, New Tai Lue, Tai Tham, Yi, Lisu, Hangul (Korean), and Miao."[27]

Fonts

[edit]

The GB18030 Support Package for Windows contains SimSun18030.ttc, a TrueType font collection file which combines two Chinese fonts, SimSun-18030 and NSimSun-18030. The SimSun 18030 font includes all the characters[clarification needed] in Unicode 2.1 plus new characters found in the Unicode CJK Unified Ideographs Extension A block although, despite its name, it does not contain glyphs for all characters encoded by GB 18030, as all (about a million) Unicode code points up to U+10FFFF can be encoded as GB 18030. GB 18030 compliance certification only requires correct handling and recognition of glyphs in the mandatory (two-byte, and CJK Ext. A) Chinese part.[6]: 4  Nevertheless, the requirement of PUA characters in the standard have hampered this implementation.[28]

Microsoft YaHei and DengXian provided by Microsoft are updated in 2023 to match GB 18030-2022 implementation level 2, and SimSun is updated to match implementation level 3.[29]

Source Han Sans (and its counterpart Noto Sans CJK) are already compliant with GB 18030-2022 implementation level 2 when the standard update for GB 18030 is announced as of November 2022. Source Han Serif (and its counterpart Noto Serif CJK) however is not compliant at the time, and an update is provided to ensure the font is compliant to implementation level 2. Similarly Microsoft YaHei and PingFang (Apple) require a small number of URO additions that are associated with implementation level 1 in order to become compliant with GB 18030-2022 implementation level 2.[27]

Other CJK font families like HAN NOM[30] and Hanazono Mincho[31] provide wider coverage for Unicode CJK Extension blocks than SimSun-18030 or even SimSun (Founder Extended), but they don't support all code points defined in GB 18030.

See also

[edit]

Notes

[edit]
  1. ^ Note that GB18030 omits surrogates; see #Mapping.
  2. ^ The euro sign is an exception which is given a single byte code of 0x80 in Microsoft's later versions of CP936/GBK and a two byte code of A2 E3 in GB18030.
  3. ^ The code points include the 66 Unicode noncharacters.
  4. ^ ICU seems to erroneously consider this code point valid, which is in neither versions of the published standards. WHATWG assigns this byte to U+20AC (GBK euro sign) in its universal gb2312-gbk-gb18030 decoder.
  5. ^ For a finer division of this range, see GBK (character encoding) § Encoding.
  6. ^ Some code points are encoded with two bytes (upper row), the others with four bytes (lower row). U+FFFF is encoded as 84 31 A4 39 on page 239 of the 2005 standard, although the standard gives as far as 84 39 FE 39 for BMP mapping.
  7. ^ These are surrogate code points; they have no meaning outside of UTF-16 encoding.
  8. ^ Furthermore, due to the encodings of U+E7C7 and U+1E3F having been swapped, U+E7C7 is encoded in the 2005 edition of the standard as 81 35 F4 37, between U+1E3E (81 35 F4 36) and U+1E40 (81 35 F4 38). Hence, only the 2000 edition is entirely sequential in allocating the four-byte codes to otherwise unmapped code points.

References

[edit]
  1. ^ Anthony Fok (2002-03-15). "Application of IANA Charset Registration for GB18030". IANA Character Set Registrations. Retrieved 2016-12-05.
  2. ^ "Disruptive Changes in GB 18030-2022" (PDF). www.unicode.org. Retrieved 2024-02-12.
  3. ^ "[JDK-8301119] Support for GB18030-2022 - Java Bug System". bugs.openjdk.org. Retrieved 2023-08-14.
  4. ^ "JDK 21 Release Notes". jdk.java.net. Retrieved 2023-08-14.
  5. ^ "[JDK-8307340] Release Note: Support for GB18030-2022 - Java Bug System". bugs.openjdk.org. Retrieved 2023-08-30.
  6. ^ a b c d CESI (2009-07-08). "GB18030 符合性问与答" [GB18030 compliance FAQ]. CESI Certification Center. Archived from the original on 2016-09-28. Retrieved 2016-10-12. Page 4 同时达到以下两个要求的产品，为符合GB 18030-2005强制部分的产品：①产品可以正确输入、输出、处理GB 18030-2005强制部分规定的全部汉字字符；②产品可以正确识别GB 18030-2005强制性部分规定的全部汉字字符对应的编码。 [A product compliant with the mandatory part of GB 18030 must be able to correctly a) input, output and process all Chinese characters defined in the mandatory set; b) recognize encodings for characters in the mandatory set.] Alt URL
  7. ^ a b c d e Standardization Administration of China (SAC) (2005-11-18). GB 18030-2005: Information Technology—Chinese coded character set.
  8. ^ "Unicode FAQ on GB 18030". ICU Project. Retrieved 10 September 2016.
  9. ^ a b GB 18030-2000: Information Technology—Chinese ideograms coded character set for information interchange—Extension for the basic set. 2000-03-17. {{cite book}}: |work= ignored (help)
  10. ^ a b c Lunde, Ken (2006). "L2/06-394 Update on GB 18030:2005". Unicode Technical Committee Document Registry. Retrieved 28 September 2016.
  11. ^ a b c Lunde, Ken (4 August 2022). "The GB 18030-2022 Standard". Medium. Retrieved 7 August 2022.
  12. ^ "Group:GBK外字". GlyphWiki. Retrieved 11 September 2016.
  13. ^ a b Lunde, Ken (December 2008). CJKV Information Processing. O'Reilly Media, Inc. ISBN 978-0-596-51447-1. Retrieved 11 September 2016.
  14. ^ CESI (2009-07-08). "GB18030 符合性问与答" [GB18030 compliance FAQ]. CESI Certification Center. Archived from the original on 2016-09-28. Retrieved 2016-10-12. Page 4 同时达到以下两个要求的产品，为符合GB 18030-2005强制部分的产品：①产品可以正确输入、输出、处理GB 18030-2005强制部分规定的全部汉字字符；②产品可以正确识别GB 18030-2005强制性部分规定的全部汉字字符对应的编码。 [A product compliant with the mandatory part of GB 18030 must be able to correctly a) input, output and process all Chinese characters defined in the mandatory set; b) recognize encodings for characters in the mandatory set.]
  15. ^ Lunde, Dr Ken (2023-11-12). "The First Amendment". Medium. Retrieved 2024-09-09.
  16. ^ China National Body (2023-04-26). "GB 18030—2022《信息技术 中文编码字符集》国家标准第１号修改单（征求意见稿）" (PDF). UTC L2/23-113.
  17. ^ a b China National Body (2023-10-13). "IRG #61 Activity Report" (PDF). ISO/IEC JTC1/SC2/WG2/IRG N2623; UTC L2/23-240.
  18. ^ United States National Body (May 1, 2023). "USNB Comments on Draft 2 of GB 18030-2022 Amendment 1 and recommendation for ISO/IEC 10646:2020 Amendment 2" (PDF). ISO/IEC JTC1/SC2 N4852, WG2 N5222; UTC L2/23-115.
  19. ^ a b Authoritative mapping table between GB18030-2000 and Unicode. ICU – International Components for Unicode. 2001-02-21. Accessed 2016-09-04.
  20. ^ "Encoding Standard # gb18030-index". WHATWG. Retrieved 2016-09-24.
  21. ^ Bridge, Karl (13 October 2021). "MultiByteToWideChar function (stringapiset.h) - Win32 apps". learn.microsoft.com. Retrieved 2022-11-01.
  22. ^ Microsoft. "GB18030 Support Package". Microsoft. Archived from the original on 2012-06-05.
  23. ^ Drepper, Ulrich. "GB18030 iconv module for glibc". glibc git. Retrieved 29 November 2016.
  24. ^ Drepper, Ulrich. "Update GB18030 to 2005 version". glibc git. Retrieved 29 November 2016.
  25. ^ Weimer, Florian; O'Donell, Carlos. "Status of GB18030 tables (#19575)". Sourceware Bugzilla. Retrieved 29 November 2016.
  26. ^ "NEWS - libiconv.git - libiconv". git.savannah.gnu.org. Retrieved 2016-10-13.
  27. ^ a b c Lunde, Ken (2022-08-16). "The GB 18030-2022 Standard". Medium. Retrieved 2022-11-01.
  28. ^ Lunde, Ken. "If gb18030 is revised, consider aligning the Encoding Standard · Issue #27 · whatwg/encoding". GitHub. Besides, supporting PUA code points in the context of the Noto CJK and Source Han fonts is a total non-starter, mainly because they are Pan-CJK typefaces, and PUA usage is extremely dangerous in such contexts.[...] One of my friends at CESI shared with me the text from the final draft a few days ago. This confirmed that the PUA requirement for the 24 characters is being lifted.
  29. ^ "July 11, 2023—KB5028171 (OS Build 20348.1850) - Microsoft Support". support.microsoft.com. Microsoft. Retrieved 25 March 2024.
  30. ^ VietUnicode. "/hannom". sourceforge.net. Retrieved 2016-10-13.
  31. ^ "Hanazono fonts". fonts.jp. Archived from the original on 2010-04-12. Retrieved 2016-10-13.
[edit]
Retrieved from "https://en.wikipedia.org/w/index.php?title=GB_18030&oldid=1247444557#Encoding"