Brian Krebs writes on how browsers choose to display IDN. The issue here is of course spoofing valid URLs with visually similar letters. You probably would notice the lame attempt in the department line but some of the international characters are very similar or indeed identical. Depending on your personal preferences it might be a good idea to use punycode instead. Could save you a headache later.
https://krebsonsecurity.com/2018/03/look-alike-domains-and-visual-confusion/
Here are some of the applicable RFCs:
- RFC 3490 - Internationalizing Domain Names in Applications (IDNA)
- RFC 3491 - Nameprep: A Stringprep Profile for Internationalized Domain Names (IDN)
- RFC 3492 - Punycode: A Bootstring encoding of Unicode for Internationalized Domain Names in Applications (IDNA)
- RFC 3986 - Uniform Resource Identifier (URI): Generic Syntax
- RFC 4690 - Review and Recommendations for Internationalized Domain Names (IDNs)
- RFC 5890 - Internationalized Domain Names for Applications (IDNA): Definitions and Document Framework
- RFC 5891 - Internationalized Domain Names in Applications (IDNA): Protocol
- RFC 5892 - The Unicode Code Points and Internationalized Domain Names for Applications (IDNA)
- RFC 5893 - Right-to-Left Scripts for Internationalized Domain Names for Applications (IDNA)
- RFC 5894 - Internationalized Domain Names for Applications (IDNA): Background, Explanation, and Rationale
This discussion has been archived.
No new comments can be posted.
Web Browsers and International Domain Names
|
Log In/Create an Account
| Top
| 74 comments
| Search Discussion
The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.
(1)
(1)
(Score: 3, Interesting) by martyb on Sunday March 11 2018, @11:13AM (31 children)
As the person who performed the testing of the implementation of UTF-8 and Unicode support on SoylentNews, I am curious what experiences other Soylentils may have in this area.
How did you perform your testing?
What tools did you find helpful?
What test data or even test suites did you use?
Besides the RFCs, what other documents did you find helpful or instructive?
Wit is intellect, dancing.
(Score: 3, Troll) by FatPhil on Sunday March 11 2018, @11:44AM (25 children)
Great minds discuss ideas; average minds discuss events; small minds discuss people; the smallest discuss themselves
(Score: 2, Troll) by FatPhil on Sunday March 11 2018, @12:11PM (11 children)
Modding me "troll" for simply stating my *entirely justifiable* opinion is cowardly.
The fact that punycode exists is all the proof you need that DNS was never intended to support non-ASCII. The second someone mentioned the idea of expanding the alphabet the wiser thinkers said "you'll get spoofing if you do that" - that was decades ago. We, or they, didn't listen, and now we've got an unresolvable mess, just because some PC types wanted to be "inclusive". Fuck inclusivity - which bit of "<letter> ::= any one of the 52 alphabetic characters A through Z in upper case and a through z in lower case" do you fail to understand?
Great minds discuss ideas; average minds discuss events; small minds discuss people; the smallest discuss themselves
(Score: 4, Touché) by c0lo on Sunday March 11 2018, @01:04PM (2 children)
"Fuck everyone" at the beginning of of what it is supposed to be an argumentation one can agree or disagree with?
Well, fuck you.
If you want a discussion, be civil. Show a minimum level of respect necessary to have a discussion - otherwise all I can get is "If you disagree with me, fuck you. Mind you, I'll fuck you even if you agree with me".
There. It is spelled clear enough for you to get it?
https://www.youtube.com/watch?v=aoFiw2jMy-0 https://soylentnews.org/~MichaelDavidCrawford
(Score: 1, Insightful) by khallow on Sunday March 11 2018, @03:01PM
What an interesting idea. I think it's a bit crazy for the internets though.
(Score: 2, Troll) by FatPhil on Sunday March 11 2018, @03:59PM
Great minds discuss ideas; average minds discuss events; small minds discuss people; the smallest discuss themselves
(Score: 4, Interesting) by requerdanos on Sunday March 11 2018, @02:58PM (4 children)
Pues, la parte en que en la idioma española, y en otras idiomas tambien, hay mas que veintiseis letras, por ejemplo. En eso solo refiere a idiomas escritas en letras latinas; idiomas como ruso y griego tienes letras--si, son letras--que son fuera du tu idea de lo que son las letras. Hay miles de milliones de gente que no usan el alfabeto que usas. No se puede decir 'de alfa á omega' sin letras que tu no reconoces.
ясно, что алфавит - это не то, что вы думаете.
有些語言甚至不使用字母。
Even given the depth of your inability to define the word "letter" in a broadly useful way, however, Unicode is a very poor answer because of all the characters which are frankly visual duplicates, and many more that are inexact duplicates but duplicates nonetheless. A good answer would allow visiting something like española.com or 那些愚蠢的西方人.net or избирательные-хакеры.org with each glyph being not only unique, but mapped to a unique code point.
Unicode isn't anything like that, and using it is a gaping security hole that enables sophisticated-seeming but dead-simple spoofing. Unicode-enabled fake domain + letsencrypt would have an undetectability factor of something like 90%. It's vulnerability by design.
(Score: 3, Insightful) by coolgopher on Monday March 12 2018, @12:50AM (3 children)
Unicode lost its way a long time ago. Unfortunately. It's still better than having to deal with the old "code page" approach though I think. Or is it? I didn't get emojis in my code pages at least...
(Score: 0) by Anonymous Coward on Monday March 12 2018, @03:54AM
Dude. Please, stop. Some sort of warning before posting that stuff.
Too many years of dealing with "cockup pages"
(Score: 0) by Anonymous Coward on Monday March 12 2018, @09:42AM (1 child)
Actually, I think in *this* case - DNS - the code page idea is not that far off. But instead of the overlapping code pages we used to have, have code pages map onto the unicode space, and let the user select which code pages to show as unicode letters, with the rest being shown as punycode. Of course with sensible defaults depending on the system language.
That way, a Russian would be able to see domain names in Cyrillic, and a Chinese would be able to see domain names in Chinese, but the Russian would see Chinese domain names (which he can't read anyway) as punycode.
(Score: 2) by requerdanos on Monday March 12 2018, @02:38PM
Believe it or not, reading more than one alphabet is really not uncommon. (Heck, I am from the linguistically and geographically ignorant United States of America, and I can read more than one alphabet--Latin, Cyrillic, some Greek--imagine how much more so someone from countries where "literate" implies at least "bilingual", and that's most of them...)
Take your Russian friend in your example. China is his neighbor to the south. Even if he isn't fluent in Chinese, he can still pick out some Chinese here and there. He can tell the difference between the similar "人" and "入", for example.
Which punycode kills, stone cold dead. If you start obfuscating domains with punycode, then suddenly all the Chinese domains--all the non-Latin, non-Cyrillic domains--look alike and he can't readily tell one from another, even though he could before. That makes things arguably worse, not better.
Plus, there is no punycode for Latin characters, many of which are dead ringers for his native Cyrillic ones. "да.example" and "дa.example" might look the same, but one's all Cyrillic and the other is a mixed Cyrillic and Latin phishing site. The only way punycode would show this difference would be if he's looking at Cyrillic-alphabet sites in punycode ("xn--80ah.example" vs. "xn--a-gtb.example"), which would be insane (he would like to be able to read the address bar) and additionally, unhelpful, because neither of those is legible and so both register as "X N dash dash gibberish." So this approach makes him feel warm, fuzzy, and protected, while leaving him wide open to alphabet attacks Latin vs. Cyrillic. That makes things arguably worse, not better.
Only knowing, recognizing, or speaking one language or alphabet is not a condition most people are in, even if you and/or most of your neighbors may be.
(Score: 2, Troll) by realDonaldTrump on Monday March 12 2018, @09:00AM (1 child)
We only use 26 letters for Internet, for the Web addresses in Internet, the domain names. I put big letters all the time. But Browser always changes them to small. Very difficult & confusing if DONALDJTRUMP.COM was a different site from DonaldJTrump.com! And there would be A LOT of Fake & Hoax sites!
(Score: 1, Troll) by realDonaldTrump on Monday March 12 2018, @04:13PM
One of the Moderators doesn't know Internet. Doesn't know the alphabet. And doesn't want anyone else to learn. That's OK, I love poorly educated people.
(Score: 2, Informative) by realDonaldTrump on Monday March 12 2018, @04:25PM
WRONG! 26 letters in the Internet alphabet. Not 52.
(Score: 2) by maxwell demon on Sunday March 11 2018, @12:16PM (1 child)
Well, the biggest problem with inventing your own internet is getting other people build and to use it. I think I'll finish my work on a time machine first. ;-)
The Tao of math: The numbers you can count are not the real numbers.
(Score: 3, Informative) by requerdanos on Monday March 12 2018, @12:32AM
Use it, sure; adoption would be a problem until a critical mass was reached.
But build it? Most devices capable of operating on an internetwork have the hardware (ethernet, wifi, etc.) and software (networking stack that can perform tcp/ip) built right in or easily available.
Instead of connecting your devices to the Internet, connect them instead to your internet. The infrastructure (the links between nodes, not the routing, dns, etc.) is going to be largely the same; besides leased lines between locations, you could even tunnel links across another network (such as the Internet).
I would hope that a person starting his or her own internetwork would start with something like ipv6 (and not, not, not ipv4) but I expect the opposite.
(Score: 3, Interesting) by Anonymous Coward on Sunday March 11 2018, @02:17PM (9 children)
Ok.
Uh... gimme a second.
The fucking fact that you're too fucking stupid to learn a fucking foreign language shouldn't fucking doom the rest of the fucking world to keep fucking using fucking ancient, US-only fucking standards. If you don't fucking like fucking non-ASCII parts of the fucking Internet, you're more than fucking welcome to keep the fuck away from them. Fuckity fuck.
... There. I think that was the level of discourse you wanted. If you're willing to participate in a polite discussion instead, please say so. Thank you.
(Score: 2) by requerdanos on Sunday March 11 2018, @03:15PM (1 child)
As kind as that was of you to adapt to circumstances and offer your flexibility and patience, I have never found such an effort, nor such an offer, to be properly appreciated. Know that at least one person appreciates it.
(Score: 2) by MichaelDavidCrawford on Sunday March 11 2018, @05:27PM
Jhhdj uffjhh ddfggg
Yes I Have No Bananas. [gofundme.com]
(Score: 2) by FatPhil on Sunday March 11 2018, @04:03PM
You're a presumptive arse - I'm simply being *practical*.
Great minds discuss ideas; average minds discuss events; small minds discuss people; the smallest discuss themselves
(Score: 3, Interesting) by HiThere on Sunday March 11 2018, @07:52PM (5 children)
I think his, poorly stated, point was that URLs should only contain ASCII-7 characters. I'm not, however, certain. If I'm correct as to what he meant, then there are valid arguments in favor of it. E.g., it not only avoids ambiguities, it allows significant URL compression when compared to the alternatives. And ambiguous URLs are dangerous.
That said, an alternative that answers some of the objections would be to specify a font in which there were no ambiguous URLs to be used for the display of URLs. Unfortunately, the only ones I've encountered do something like display a numeric code for many valid URL codes. Also that would negate the possibility of compression, though admittedly URLs are generally short enough that this wouldn't be very significant in most circumstances. But if you knew that the codes were ASCII-7 alphanumerics you could use a byte for each character, with one bit for parity. And there would be several unused characters that could be used for control codes. This gives almost-optimal compression.
So there is a clear case of the requirement that URLs should contain only ASCII-7 characters, and mainly alphanumerics. And there are arguments against allowing a fuller unicode implementation, as that would, at minimum, mean you could no longer specify parity. And it also provides techniques to allow spoofing.
N.B.: This is not a claim that there is not a valid counter-argument, but rather that I haven't encountered one that impressed me.
Javascript is what you use to allow unknown third parties to run software you have no idea about on your computer.
(Score: 2) by FatPhil on Monday March 12 2018, @09:16AM (4 children)
The internet was internetting first, if the rest of the world wants to play, it should adapt to the internet, not have the internet adapt to it.
Great minds discuss ideas; average minds discuss events; small minds discuss people; the smallest discuss themselves
(Score: 2) by massa on Monday March 12 2018, @07:28PM (3 children)
You do realize the "rest of the world" internet is far bigger than the USofA internet, don't you?
(Score: 2) by FatPhil on Tuesday March 13 2018, @07:37AM (2 children)
As I said initially - let them invent their own internet.
Great minds discuss ideas; average minds discuss events; small minds discuss people; the smallest discuss themselves
(Score: 2) by massa on Tuesday March 20 2018, @02:21PM (1 child)
We did. And we even let you USofAns in :-)
(Score: 2) by FatPhil on Tuesday March 20 2018, @04:34PM
You see there's no hypocrisy in my statement - I happily promote the American Standard Code for Information Interchange as being what the internet was built around despite not being American.
Great minds discuss ideas; average minds discuss events; small minds discuss people; the smallest discuss themselves
(Score: 2, Troll) by realDonaldTrump on Monday March 12 2018, @08:44AM
So true! China, VERY WEAK country when Internet was invented. And in China they have VERY SPECIAL writing, they don't use letters, they use little pictures. And they use our numbers, those are the same. We use our letters & numbers for Internet because the Internet is American. Invented in America, belonged to America for a long time. Until Obama VERY STUPIDLY decided to turn over our Internet to foreigners.
China is becoming very powerful. While America has become VERY WEAK. Before, the Chinese Internet used numbers, because they couldn't use their picture writing. 163.com for example, very big site in China, right? But now they want to use their picture writing for the domain names -- people don't know this, a domain name is just the name of an Internet site. Very important part of the Web address. In America we don't use the picture writing, very hard for us to do that. We try to look at the Chinese Internet, very difficult for us, maybe, probably we have to get a Chinese typewriter. But Chinese typewriters are always made in........China!!!! Money leaving our Country, JOBS leaving our Country, bigger trade deficit.
Folks, we need to TAKE BACK OUR INTERNET. The U.S. should not turn control of the Internet over to the United Nations and the international community. America First!
(Score: 2) by c0lo on Sunday March 11 2018, @11:56AM (2 children)
You asking for means other than dick niggers?
https://www.youtube.com/watch?v=aoFiw2jMy-0 https://soylentnews.org/~MichaelDavidCrawford
(Score: 0) by Anonymous Coward on Monday March 12 2018, @03:59AM (1 child)
Psst! YOU FORGOT TO TICK THE POST ANON BOX
Now everyone knows you are one of the ducks who posts about black people using unPC terminology
(Score: 2) by requerdanos on Monday March 12 2018, @02:49PM
That's not a post about people of any particular color--that's the self-given name of a troll who made several racially charged, obscenity-laden almost-spam* troll posts here. Once the messages started to get filtered administratively, the troll used various alternate spellings and alternate characters to post the same message for a while in evasion of the filters. The admins won, the troll lost, and the episode was a learning experience similar to how to international domain name problem is also a learning experience.
You don't need to be anonymous to know that any of this happened; it doesn't help in any way. Remembering a troll's tactics does not make you that troll.
-----
* I say almost-spam because the troll would often devote a few words of an otherwise invariant troll post to the topic of the article being trolled. It was odd. The posts were frankly more bizarre than offensive, despite their inflammatory language.
(Score: 4, Insightful) by requerdanos on Sunday March 11 2018, @03:48PM
Well, comments like this one [soylentnews.org], below, could be incorporated into a test suite. Something's definitely very broken there.
What, for example, is "привет" and what does it have to do with "привет" or "privyet" or "xn--b1agh1afp"?
(Score: 2) by driverless on Sunday March 11 2018, @10:47PM
In security terms it's not actually that bad, none of the major browsers are vulnerable, the only one that is is a minor also-ran clone of Chrome.
(Score: 1, Troll) by c0lo on Sunday March 11 2018, @11:53AM
Oh, the fucking "Dι¢κ Иї99εrs"... they moved away from S/N and onto domain names, didn't they?
https://www.youtube.com/watch?v=aoFiw2jMy-0 https://soylentnews.org/~MichaelDavidCrawford
(Score: 5, Informative) by Runaway1956 on Sunday March 11 2018, @01:52PM (12 children)
Perfect example of idiots bending over backward to appease the politically correct crowd - and exposing people to exploits.
If you're using a Mozilla product, the fix is easy.
(Score: 3, Insightful) by requerdanos on Sunday March 11 2018, @03:44PM (3 children)
Yes, that is a pretty perfect example, and yes, it exposes people to trivial exploits. But the "fix" fixes one problem and introduces another.
"Punycode" is a method to translate non-ascii but perfectly readable text into illegible gibberish. Since I can read words in more than one alphabet--I'd wager that most people in the world do this, though admittedly I don't know if that translates to most Internet users--that's no fix. It creates its own class of problem: Now I would be expecting that perfect-readable-words.com would display as "xn--gibberish2389.com" and I wouldn't be disappointed, whether I was at the legitimate site or at some spoofed alternative.
Not being able to read the domain name (punycode prevents this) but rather being shown a mathematically coded representation puts me at a disadvantage because now I can't quickly and easily spot obvious spoof-fakes. Don't even know what site is loaded except through context clues.
(Score: 2) by driverless on Sunday March 11 2018, @10:50PM (2 children)
I've always felt that punycode should be called wtfcode, because, WTF? It has exactly the problem you mention, is incredibly complex to process, the code to do so is probably prone to all sorts of vulns because of its complexity, and all it's doing is taking something that's a problem and turning it into an even worse problem.
(Score: 3, Touché) by coolgopher on Monday March 12 2018, @01:02AM (1 child)
Sounds like its true name then is XML... ;)
(Score: 0) by Anonymous Coward on Monday March 12 2018, @04:01AM
XML is like violence...
(Score: 2) by isj on Sunday March 11 2018, @03:56PM (7 children)
It's a perfect example of firefox considering the larger implications.
You comment seems to indicate that you haven't understood the purpose of punycode and why people would want to use their non-english script.
(Score: 2) by Runaway1956 on Sunday March 11 2018, @04:45PM (6 children)
At this point in time, English is the leading language used on the internet, with Chinese trailing a respectable second place, and Spanish a very distant third. https://www.internetworldstats.com/stats7.htm [internetworldstats.com]
It would seem reasonable to use punycode by default, especially when packaged for use in primarily English speaking countries. If you've read TFA, then you already know that other browsers will render the address into meaningful gibberish by default. Why does IE, Edge, Opera, Safari, and Chrome all get it right, but Mozilla does not?
You and requerdanos both make cases for using a different default configuration in countries most affected by punycode.
I can't speak for eastern European users, and certainly not for Asians - but relatively few Americans are savvy enough to understand this language issue. Default configuration for downloads in the US really should offer this protection.
Funny thing about Pale Moon, though. When I went to the test page, my address bar looked odd. Before the little green padlock, I had the xn--80a7a.com/ clearly visible, and after the lock it showed address the same as the TFA. Firefox, however, only displays the green lock, and https://www.са.com/ [www.Ñа.com]
It appears that Palemoon has it right. I get my warning that maybe ca.com isn't really ca.com, but it will render the script so that you, who uses other languages and script, can see what it really is.
(Score: 2) by requerdanos on Sunday March 11 2018, @05:51PM
Only, as you mention, if your native language doesn't use non-ascii characters; otherwise punycode turns readable domains into machine-readable but human-unreadable gibberish, effectively removing the feature of "The domain of the current site appears in the address bar" for all sites whose domains are in your native language.
The gibberish may be "meaningful," but a string of "meaningful numbers and letters" in lieu of a human-readable name does not go easy on the eyes. It could arguably make spoofing easier; if someone is checking to make sure that the site is displayed as punycode gibberish, then they are not going to recognize a spoof site which also shows up as gibberish unless they compare character-for-character the punycode itself. That isn't really a part of anyone's workflow, and a kludge that requires that is a workaround, but no solution.
(Score: 3, Interesting) by isj on Sunday March 11 2018, @05:51PM (2 children)
It appears that Mozilla changed their algorithm since I last checked (which admittedly was a decade ago): https://wiki.mozilla.org/IDN_Display_Algorithm [mozilla.org]
Mozillas updated algorithm detects mixed scripts, but that doesn't help for са.com where the each component does not mix scripts. I don't know which algorithm chrome/safari/opera/... use. I quick test show that chrome does not show punycode for some ccTLDs that allow non-english characters. So my guess is that mozilla allows non-english characters in .com while the other browsers do not.
I think that defaulting to show raw punycode for non-english letters is an arrogant attitude. That ignores approximately 85% of the world population which natively use non-english letters. If you disagree then please stop using those fake letters Y, W, X, J - they are not part of the Latin alphabet :-)
I think the browsers should consider not only the TLD but also the user's capabilities. Limiting the non-punycode display characters for US downloads to A-Z (and possibly Ñ and Ç too - there are quite a lot of Spanish speakers in the US) sounds like a reasonable idea. Or use the language preferences to infer which characters the user knows (although setting language preferences in the browser is quite rare for normal users). Combine that with IDN whitelists for ccTLD.
Then there is the problem that the generic TLDs are being used for multiple scripts. That is where the main problem is. If the generic TLDs didn't allow mixing scripts with similarly-looking glyphs then there would not be a problem. But they do, so we have a mess. I'm in favor of abandoning the generic TLDs and only use ccTLDs - but that is not going to happen.
(Score: 0) by Anonymous Coward on Sunday March 11 2018, @11:19PM (1 child)
Make it keyboard-dependant (as tfa hints at).
If the url could be typed without using the compose key on the current keyboard, display as characters. Otherwise, display as punycode.
(Score: 2) by requerdanos on Monday March 12 2018, @03:06PM
I have two keyboards connected, one US layout (It's a Microsoft Natural keyboard, the only Microsoft product that I am fond of), one Russian. All the time.
Your solution makes no sense. Even if we say "the current keyboard layout" instead of "the keyboard", typists generally only change layouts in preparation for typing something, not reading something, not for following a link. Most web browsing doesn't use keyboard input.
On a deeper level, there is no way to define "THE language" someone speaks, because people speak/read many languages to a certain degree. Even most Americans can pick out a handful of foreign words, which by definition is "reading another language."
In the same way, it's very poor engineering indeed to design an information system that assumes "the keyboard" or "the monitor" or "the mouse" or "the printer" for the simple reason that any given system may have zero or more of those things. In past decades, these lessons were learned and applied, and now all your major operating systems properly allow for more than one of each with no special hoops to jump through. Deliberately un-learning such a lesson doesn't seem like progress.
My system I'm typing on right now has the two (very different) keyboards, three monitors, a three-button mouse, a five-button mouse, and a Wacom pad also providing mouse input. The machines to its immediate right and left are headless no-keyboard no-mouse machines that I use via ssh. All of them use the same two laser printers, one color, one black and white.
(Score: 2, Funny) by Anonymous Coward on Sunday March 11 2018, @06:43PM
Ich spreche Deutsch, du insensible Klumpen!
(Score: 0) by Anonymous Coward on Monday March 12 2018, @04:05AM
WTF
On a mobile browser that is www.ca.com
Highlight the url and it shows what looks like encoding.
Why doesn't it detect that the local language is not the url language and provide warning
The opportunities for spoofing are surreal
(Score: 4, Interesting) by opinionated_science on Sunday March 11 2018, @02:19PM (12 children)
How about simply changing the non-ascii codes, to red say?
I'm willing to wager most of the "fool you with similar character" URL are directed specifically at the "I expect ASCII" crowd (aka me!).
Logically this must be the case - everyone else gets ASCII by default....
Does this sound too hard? Display a URL with character class/colors etc...
(Score: 2) by requerdanos on Sunday March 11 2018, @03:35PM
Let's imagine a site in the Russian-language world called привет.com (привет ~= "privyet" ~= "hi"). (This exists, with only a parking page at http://привет.com/.) [пÑивеÑ.com]
If we replace the Cyrillic "в" with the latin "B", or worse, replace the Cyrillic "е" and "р" with latin "e" and "p", we get lots of variants that look identical, or almost identical, to our hypothetical original.
So perhaps it's not the sophisticated world directing "fool you with similar" attacks at the "I Expect ASCIIs" but the "criminal element" directing "fool with similar" attacks at all-and-sundry.
Changing color based on which Unicode page a character is from would admittedly reveal this just as well, But Ivan Pa-Russki and many others would have to put up with their address bar being an ugly error-red indicating "normal" and friendly ordinary black meaning "someone is trying to fool you."
Punycode probably isn't a universal answer--the friendly "привет.com" becomes "xn--b1agh1afp.com" in punycode (Blag one a fop? Blog one a fip?). Which would be sort of like "google.com" always showing up as "qz--jkl2h298398j.com" for us ASCII folks. I.e. similar problem to the red-coding, but worse because instead of turning letters red, it renders them unreadable.
(Score: 3, Interesting) by isj on Sunday March 11 2018, @03:44PM (10 children)
What about domains in .ru and .au? Why should the Russians/Ukranians be punished for using their own script?
ASCII is English-centric. The only languages that I know of that can be correctly represented in ASCII are English, Dutch and Greenlandic.
Even the languages that use the latin script (and extensions thereof) most of them cannot be represented correctly in ASCII: French, German, Norwegian, Spanish, Italian, Polish, …
The underlying problems are:
1: that the non-ccTLDs exist. Eg .com should have been transformed into .com.us a long time ago.
2: some ccTLD registars have inadequate or no rules for which scripts can be used.
Some ccTLD registrars have the rule that you can only use scripts and characters that are commonly known in the languages used in that country.
With the non-ccTLDs .com and all the new non-specific domains (.xyz, .guru, ...) we have the problem that multiple scripts can be used in domain names. The article doesn't cover the details of Firefox' IDN checks. Firefox has a list of TLDs where the registrar has sensible rules. So eg. .dk is whitelisted because the rules there only allow the latin script with the extensions æ/ø/å.
The mixed-script checks that the article talks about can only be applied on each domain component separately, so that doesn't help with eg.асе.com where no mixing occurs within each component.
I'm working on related problems (eg. a blog on diacritics [privacore.com] here), but there we have sort of the opposite problem: users type something simple and we have to extend the meaning to include other glyphs/codepoints (but we don't mix latin with cyrillic because that wouldn't make sense).
(Score: 4, Interesting) by opinionated_science on Sunday March 11 2018, @06:36PM (6 children)
that's not punishment, they simply have a different language.
I speak multiple languages and not all the characters are ascii.
But guess what if I go to "www.amazon.com" , I know it should be ascii, and so *any* URL that has a NON-standard character set should show this based on my LANG preference.
I am sitting here with declare -x LANG="en_US.UTF-8"
Someone in Russia would have (I imagine, someone correct me!) "LANG=ru_RU.utf8"
Hence, the hierarchy of languages is clear. I'm not saying it should be mandated, but an option for "show non standard chars" would go a long way to combating click-jacking as the majority of languages are non-ascii.
Is this so unreasonable?
(Score: 3, Interesting) by isj on Sunday March 11 2018, @07:29PM (5 children)
I mostly agree with you.
Note: My use of .ru TLD was a bad choice. The domains in .ru are using transliterated Russian letters, while the real Russian TLD is .рф
I think it is reasonable to require that if an average Russian goes to the президент.рф site which has cyrillic letters in it then the cyrillic letters are shown as they should - not as raw punycode.
And if an average German goes to www.bücher.de which has latin-1 letters in it then the latin-1 letters are shown as they should - not as raw punycode.
Now, if an average American goes to the президент.рф site? Well, since the TLD has a strict script policy (only cyrillic is allowed) it would be okay to show the cyrillic letters. Or the raw punycode. Either would be fine IMHO.
What about са.com (or any other TLD with loose script policy) ? This is where the idea of showing what the user should be familiar with as fine glyphs, and the unfamiliar stuff as punycode seems like a good idea. It would as you put it go a long way against click-jacking. The average American would see xn--80a7a.com while the average Russian would see са.com.
But then you have a nasty problem: The opposite case (plain ascii ca.com) the average Russian would see uhm... (you can't punycode-encode plain a-z) some clear indication that it is not cyrillic. But that would be silly because it is quite common. Are Russians tricked by cyrillic-looking glyphs, or are they just more aware of it? Inquiring minds want to know...
(Score: 3, Touché) by FatPhil on Sunday March 11 2018, @08:36PM
Great minds discuss ideas; average minds discuss events; small minds discuss people; the smallest discuss themselves
(Score: 2) by requerdanos on Monday March 12 2018, @12:41AM (1 child)
Looking at one, I see "president dot R F", and looking at the other, I would see "X N dash dash meaningless gibberish".
Sure, I know tastes vary, but--I can read one of those and can't read the other. Regardless of what the machine might be able to read.
(Score: 2) by isj on Monday March 12 2018, @12:58AM
My imperfect phrasing. What I meant was that I can see pros and cons of each approach in this particular unusual case and I don't have a strong opinion on that.
(Score: 2) by requerdanos on Monday March 12 2018, @12:47AM (1 child)
I do know that I've made the odd Russian-language post on this very site, to make a point (a sad tendency I have that sometimes casts my maturity in doubt), and been rebuffed by the lameness filter *unless* I substituted Latin characters for a certain percentage of the Cyrillic ones. The look the same, read the same, and though I am no Russian, they would sure fool me.
As a side note, it is amazing to me how much more slowly I type while using a Russian keyboard/keyboard layout than I do while using US-International layout. Is it just me?
(Score: 2) by isj on Monday March 12 2018, @01:22AM
I'm hoping that some Russians will chime in. I have no idea if there are the reverse phishing attacks using latin letters against cyrillic users.
Regarding keyboard layout: I imagine that it depends on what you type and how familiar you are with the keyboard layout. If you have been programming for a while then I imagine using any non-latin keyboard would be much slower due to lack of muscle memory. Typing on a french keyboard is no fun either if it is not your primary keyboard layout. It once took me 8 tries to type my password correctly on that abomination.
(Score: 0) by Anonymous Coward on Sunday March 11 2018, @11:24PM (2 children)
As i wrote above: match the current keyboard.
If all characters in the url map to keys of the current keyboard, then just site the characters.
This assumes that the user can tell the keys on his keyboard apart - quite reasonable imho.
Otherwise: punycode.
(Score: 2) by requerdanos on Monday March 12 2018, @01:12AM
On my desk is an en-us keyboard. In the keyboard drawer is an ru-ru keyboard. Both connected by the magic of USB. I switch among keyboard layouts with the scroll lock key--and the "current keyboard layout" is on a per-window basis, with the layout often switched when I want to type something, not "as soon as I start reading something."
I only have two different keyboard layouts plugged in, but I have lots of free USB ports.
No matter what keyboard I plug in or use, punycode is readable by no one.
(Score: 2) by isj on Monday March 12 2018, @01:12AM
If the user's keyboard uses a different script (latin, cyrillic, arabic, hebrew, thai, ...) than the URL then that could work for some cases. However, you are overlooking the reverse scenario:a Russian with a cyrillic keyboard visiting www.example.com. In that case the whole URL should be shown as punycode, but that sounds a bit silly.
(Score: 2, Insightful) by cocaine overdose on Sunday March 11 2018, @04:08PM (8 children)
And here we are. After years of appealing to mouth-breathers who couldn't remember 5 sets of 1-3 characters each (or 4 sets of 4 characters, woe is me), and facing the consequences, we're now at another absolutely avoidable exploit (it's an old one, Apple users were phished with it last year). The only two things domain names do well is: make it easier for retards to remember the Pinterest IP and make it easier for sysadmins to play hot potato with service providers. There's also the metadata aspect, but you can get past that with a little bit of finger grease.
Besides those, I cannot think of any other reason to use domain names (except making name squatters and "registrats" a.k.a licensed-squatters loads of dosh). But I can think of many reasons not to:
Abolish domain names. IPv6 addresses are the future.
(Score: 3, Informative) by MichaelDavidCrawford on Sunday March 11 2018, @05:10PM (1 child)
Back in the day the only way to host multiple sites on just one box required that box to have multiple IP addresses
Everyone knew that hilarity would soon ensue so HTTP 1.1 enables multiple sites by putting the domain in the header:
GET /hello.jpg goatse.cx
Or something like that
Yes I Have No Bananas. [gofundme.com]
(Score: 2, Informative) by cocaine overdose on Sunday March 11 2018, @05:41PM
(Score: 2) by requerdanos on Monday March 12 2018, @01:01AM (5 children)
With what shall we replace them?
a. "On AOL go to keyword 'twitter'"
b. Google it... Uh, I mean, remember a google ip address and "number it" (we remember 8.8.8.8 for dns, ironically, so why not e.g. 1.2.3.4 for web search?)
c. DHT or the equivalent
d. Call us at 1-800-toll-free or visit our website at https://[2001:0db8:0a0b:12f0:0000:0000:0000:0001] [2001:0db8:0a0b:12f0:0000:0000:0000]
e. Other, please specify
I ask mostly out of curiosity. I have given namecheap a pretty good chunk of money over the years.
(Score: 2, Interesting) by cocaine overdose on Monday March 12 2018, @03:11AM (4 children)
(Score: 0) by Anonymous Coward on Monday March 12 2018, @02:37PM
So someone at Google writes a script to request new keysigns until they get google again? And then discards the ready, or just hangs on to them for other purposes?
(Score: 2) by requerdanos on Monday March 12 2018, @03:11PM (2 children)
Well, onion domains are almost good enough.
Almost means "not".
(Score: 1) by cocaine overdose on Monday March 12 2018, @03:43PM (1 child)
(Score: 2) by requerdanos on Monday March 12 2018, @04:41PM
Purpose of DNS [business.com]: Translate unique easy-to-remember words or phrases to harder-to-remember IP addresses.
Number of onion domains that are easy-to-remember words or phrases: ~= 0 [imgur.com]
Number of onion domains that are indisputably easier to remember than arbitrary IP addresses: ~= 0 [imgur.com]
Scores (Any "No" means "Fails to provide functionality of DNS"):
Do onion domains provide unique mapping? Yes, the mapping is unambiguous.
Do onion domains provide easily memorable words/phrases? No, although onion domains may contain words or phrases as components, the domains themselves are either gibberish, gibberish+word(s), or word(s)+gibberish.
Do onion domains translate the domain to an IP address or other appropriate record type? Yes, but the IP address may well be easier to remember.
∴ Onion domains provide functionality of DNS? No. If there were no DNS and the world had to use onion or nothing, it's debatable which would win out. Slight edge to onion for potential to support things like MX records which bare IP addresses don't address; but then you'd be edging back into DNS territory.
Analysis: Like the unreadable punycode-gibberish solution, an unreadable onion-gibberish solution proposes to replace readable names of sites with gibberish, arguably with the goal of of improving the system, but unable to do so because of the fatal flaw of being made of unintelligible gibberish by design. Our current system has serious problems, but introducing additional problems such as removing human-readability is no improvement nor solution.
(Score: 2, Insightful) by MichaelDavidCrawford on Sunday March 11 2018, @05:01PM
That international domains are so complex doubtlessly give rise to many zero-days that we don't know about yet
Yes I Have No Bananas. [gofundme.com]
(Score: 2) by darkfeline on Sunday March 11 2018, @10:50PM (4 children)
Why is this an issue? The only things I can think of are
1. Phishing. You do use a browser integrated password manager which checks the domain name, right? A password manager isn't going to be tricked by visually similar characters.
2. You unconditionally trust the fake/real news on the site with the domain foxnews/cnn/whatever and you end up unconditionally trusting the real/fake news on the imposter site. Eh, tough luck.
And all of those require 3. You followed a link instead of navigating to the URL yourself.
It's not like this is significantly different from typo squatting, e.g. solyentnews.org. Best practices that worked then still work now.
Join the SDF Public Access UNIX System today!
(Score: 2) by DavePolaschek on Monday March 12 2018, @02:12PM
It's an issue because companies like BIGCO use BIGCOSYSTEMS.com and BIGCONEWPRODUCT.com for outgoing marketing and websites (because Brands!), and thus have trained users to use their BIGCOID username and password on all sorts of sites that aren't BIGCO.com.
Yeah, best practices work, but marketing departments seldom follow best practices.
(Score: 2) by requerdanos on Monday March 12 2018, @03:25PM (2 children)
We'd like to make it hard, not easy, to fool people and take their money and/or trust for nefarious purposes.
Yes and no. For some pages, the password manager recognizes the initial login that you normally navigate to from your bookmarks or typing in a site manually, but the different login pages that you might encounter from being auto-logged out ("your session has timed out"), etc. are different pages and thus might not be recognized. This conditions people to type their credentials into unfamiliar-but-probably-genuine forms, and makes them ripe for phishing *because* of the behavior of a password manager.
People do things like turn over money and reveal personal information on the strength of trust. If the trust is misplaced, bad things happen. We want to avoid that where possible.
That's how everyone from Tim Berners-Lee on down navigates the web, yourself included.
You do not examine every link you come to and then hand-type it into an address bar, and even if you did, that would introduce more errors than it would correct.
Saying that doesn't make either one of them not a bad thing. However, typosquatting is much more obvious and therefore less nefarious.
http://trustme.example/ [trustme.example] and http://trustame.example/ [trustame.example] can be differentiated if you're paying attention.
That's less true of http://trustworthy.example/ [trustworthy.example] and http://trustwоrthy.example/ [trustwоrthy.example] (which are different sites with identical-appearing names).*
----
* At the time of writing, soylentnews.org is calling one of these "[trustworthy.example]" and the other "[trustwоrthy.example]"--which does differentiate them, but in an interestingly nonstandard way.
(Score: 2) by darkfeline on Monday March 12 2018, @04:38PM (1 child)
>We'd like to make it hard, not easy, to fool people and take their money and/or trust for nefarious purposes.
Sure, but a fool and his money are still easily parted.
>This conditions people to type their credentials into unfamiliar-but-probably-genuine forms, and makes them ripe for phishing *because* of the behavior of a password manager.
PEBCAK, and requires the premise that the user is visiting poorly designed sites that suffer this issue, so it assumes the user is already putting themselves at risk before the event. Even my bank's crappy website combined with Chromium's built in password manager doesn't suffer from this issue.
>People do things like turn over money and reveal personal information on the strength of trust. If the trust is misplaced, bad things happen. We want to avoid that where possible.
The problem isn't trust, the problem is people trusting untrusted things. Sure, we can try to protect the fool, but fools are clever in their ability to avoid protections. If you unconditionally trust anything on the Internet, well, you're already in a bad spot.
>That's how everyone from Tim Berners-Lee on down navigates the web, yourself included.
Not really, no. I don't log in to pages that I have followed from a link, this is best practices 101. I don't unconditionally trust the claims on a page that I have followed from a link (or from a typed URL either).
>That's less true of ...
When I mouse over the link, I get http://xn--trustwrthy-jvi.example/, [trustwоrthy.example] so no, not really. Even so, I would not enter my credentials on either page.
Join the SDF Public Access UNIX System today!
(Score: 4, Interesting) by requerdanos on Monday March 12 2018, @05:16PM
While true, that's not justification for the technically literate failing to take reasonable precautions on behalf of the not.
My bank's website + firefox or pale moon does suffer from this issue. "The user is visiting poorly designed sites" is a guarantee, not merely a required premise.
We should try to protect the fools. Those who insist on being fools who are fooled will be, but that won't be because we didn't try.
Educating people not to do this is part of protecting fools from foolishness, but it hasn't got very far. That's no justification for not taking other prudent measures.
According to Nielsen Norman Group research [nngroup.com], "complex" tasks like "navigation across pages and applications" or tasks that "involve multiple steps and operators" (basically, tasks that require thinking and not just blindly following steps by rote) are not within the abilities of over 2/3 of the adult population of first-world countries.
Look, as you know, you're not wrong. But the arguably simple things you cite above are still demonstrated to be "too complicated" for most, and the challenge for the 5% of the population that is technically proficient (as measured by the referenced study) is to design a system navigable by the other 95% without undue risks.
The 2016 article referred to above (which I urge anyone interested in this discussion to read), titled "The Distribution of Users’ Computer Skills: Worse Than You Think" and written by Jakob Nielsen, is summarized as "Across 33 rich countries, only 5% of the population has high computer-related abilities, and only a third of people can complete medium-complexity tasks." Over 200,000 people between the ages of 16 and 65 inclusive were tested on computer-related tasks. This is the best research available to us, and its results are that people are less capable than we usually assume. That's a hard lesson to internalize and plan from, but for those of us like you and me, who are in the top 5% and able to do arbitrary tasks on a computer that require thought and decision making, I would submit that it's our responsibility to do it because literally no one else is capable of doing so.
If, in spite of our best efforts, fools and their money/personal info/good reputation are still parted, then so be it.
But if we don't make that effort, then we share in responsibility for that parting. No need for that! The fools* can do it on their own.
-----
* My own foolish behavior has put me into this category more than once. I am saying "us", not "they."