The early Internet was largely limited to the ASCII character set. As we have seen elsewhere, this has had dramatic implications on the development of a multilingual Internet for both the operations and content of the Internet. The ability to transmit and display content in other scripts – to support the internationalisation of the Internet – started in the 1990’s. However, the Domain Name System (DNS) lagged behind.
As first standardised, names in the DNS were limited to a subset of ASCII characters including the letters a-z, the numbers 0-9 and the hyphen character (“-“). All registrations in the DNS were, for a long time, limited to these characters. Support for a more diverse character set was first provided in a set of standards published by the Internet Engineering Task Force (IETF) in 2003. However, just because the standard was published, did not mean they were widely available or usable.
While this research reports on the availability and trends for registrations of those IDNs, this section focuses on how usable those IDNs are once they are registered and resolvable.
Universal Acceptance (UA) is a metric. It is a measure of how well IDN domain names are accepted, displayed, stored and processed by the Internet’s applications and infrastructure. In previous research we have said that UA is a measure of how ‘usable’ an IDN is. UA is a metric that measures the ability of IDNs to be used in the same way as older, more limited domain names. Another definition might be: UA is the state where an IDN can appear and be used anywhere an ASCII domain name appears, with predictable, reliable and appropriate results.
The DNS is a fundamental yet evolving part of the Internet’s infrastructure. The ability to use a domain name as part of a query for other information is a part of the Internet we often take for granted. The ubiquity of the DNS has been a source of the UA challenges for IDNs. For some time, application developers for the Internet presumed, erroneously, that all domain names ended with two or three characters – and that those characters where always ASCII. That misunderstanding has led to the principal problem for Universal Acceptance: while it may be possible to register and resolve IDNs, if software does not accept, process, display and use those IDNs correctly, they remain unable to fulfil the potential of a truly rich, multilingual Internet.
What is the Source of the Problem?
To support IDNs, the Internet changed the DNS so that it supported non-ASCII characters. To do this, the DNS evolved to support the Universal Coded Character Set, known as Unicode (or UTF8). Since the DNS was only built to support ASCII characters, there needed to be a translation from Unicode to ASCII strings – as well as a translation in the other direction. The Unicode-to-ASCII translation is called Punycode and results in a translation for every IDN to a string called an “A-label.” The A-label for IDNs is easy to recognise because is always starts with the ASCII characters “XN—”
Older parts of the Internet would have no problem processing, supporting and displaying A-labels because they are simply parts of an ASCII domain name. The problem for IDNs emerges when the Unicode characters are used, stored or displayed by the Internet’s infrastructure. Older software and applications do not have the ability to store, process and display the Unicode characters properly. As we have seen, even newer software fails to take the Unicode labels into account. The result is that the IDNs, while registered in the DNS and available for resolution, do not work like the older ASCII domain names. The result is that IDNs face a barrier not faced by other domain names.
Universal Acceptance is a problem that is not limited to IDNs. With the expansion of the DNS root zone starting in 2013, many new top-level domain names appeared in the public Internet. Many have characteristics (especially string-length) that make previous assumptions about top-level domain names fail. As we have seen in previous years, this is a problem that significantly affects user account creation and validation.