Quantity, gender, and case in internationalization. It's not easy. It's actually a great deal more difficult than it would appear at first glance. Sifting through SoylentNews' #rss-bot feed for 2015-01-15 I came upon a link to this gem of an article from 1999: A Localization Horror Story: It Could Happen To You.
The author provides a simple example of the challenges arising from trying to internationalize phrases detailing how many files and directories were found during a search. Just when it seemed like the problem was solved, another language presented yet another challenge that upended that erstwhile solution. The author identifies challenges unique to English [US and UK], Chinese, Italian, Russian, Arabic, and more.
Both entertaining in writing style and illuminating in content, this article gave me a whole new perspective on internationalization and the variations present in human languages. I'm so glad to have come upon it and hope others find it interesting, too!
(Score: 3, Funny) by Marand on Friday January 16 2015, @04:23AM
The author identifies challenges unique to English [US and UK], Chinese, Italian, Russian, Arabic, and more.
Both entertaining in writing style and illuminating in content, this article gave me a while new perspective on internationalization and the variations present in human languages.
A typo while talking about challenges in English; that's a hell of a time to make a mistake, isn't it? ;)
(Score: 3, Interesting) by Anonymous Coward on Friday January 16 2015, @04:42AM
Submitting articles is tough. There's a lot to get right - facts, interest, grammar and spelling, links (and make sure they're not paywalled, which is not always obvious), avoiding duplication with previous stories at SN and Slashdot, fairness and balance - and you aren't getting paid so you feel guilty about spending more than 20 minutes pulling it together. It's a LOT harder to do than writing even a long comment post in response to a summary.
But a good submission usually takes more than 20 minutes. Even so, I've submitted lots of articles to Slashdot and SN and I've made a glaring mistake in almost every single one.
(Score: 3, Interesting) by Marand on Friday January 16 2015, @04:53AM
I agree, it's hard to make a good submission, and once you spend any length of time reading the submission your brain starts to actively fight you by filling in the blanks when proofreading, making it even harder to avoid mistakes. I spent far longer than I probably should have (and definitely more than twenty minutes) on the submissions I sent in before, double- and triple-checking everything, from writing to links, and I probably still screwed something up on them, too.
That's why I try to point out mistakes with a joke. That way, they can be corrected by an editor without any hostility or hurt feelings. If somebody gets a laugh out of it that way, even better.
(Score: 1) by anubi on Friday January 16 2015, @05:31AM
I can certainly understand.
I just spent a week making sure a hand-routed PCB was just perfect before sending it out.
No sooner I submit the board, it comes back with 1/8 inch of trace missing I flat missed putting in. *snort*
Absolute perfection seems like absolute zero, the closer you try to come, the more time and expense invested.
Sometimes the perfect point isn't the optimal point.
I have submitted a couple of stories here, however I am not much good at getting the wrapping on straight. The editors here have done a wonderful job of dressing up interesting tidbits I found while nosing around the web. If what I submitted went up as I had provided it, I would look like quite an ignoramus.
Knowing how many times I have overlooked something, despite my very best intention, I will be damned to hold stuff like that over anybody else. Every time I point the finger, I have three of 'em pointing right back at me.
"Prove all things; hold fast that which is good." [KJV: I Thessalonians 5:21]
(Score: 1) by arulatas on Friday January 16 2015, @04:23PM
Wish I had mod points.
----- 10 turns around
(Score: 2) by Joe Desertrat on Saturday January 17 2015, @02:50AM
Wish I had mod points.
I wish you had typed "I wish you had mood points".
[I had a typo in that, but I caught it on preview.]
[Dang, in that too!]
(Score: 0) by Anonymous Coward on Friday January 16 2015, @11:36AM
once you spend any length of time reading the submission your brain starts to actively fight you by filling in the blanks when proofreading, making it even harder to avoid mistakes.
Read it out loud. Don't subvocalize, don't mumble, do it like you are reading it to another person. It engages a different part of your brain and it is like you are reading it for the first time.
(Score: 0) by Anonymous Coward on Friday January 16 2015, @04:48PM
What is SN? I'm on the lookout for good geek news sites and SN doesn't ring a bell. Thanks!
(Score: 0) by Anonymous Coward on Friday January 16 2015, @05:52PM
Never mind!! I just figured out that I'm already here at Soylent News *smacks forehead*! I'm new to SN so I managed to overlook it. However, I'm still on the hunt for great geek news sites, so your suggestions are welcome.
(Score: 2) by el_oscuro on Saturday January 17 2015, @05:35PM
I once submitted an interesting story to slashdot about an Army project which was switched to Linux. After it was published, I noticed something about the story which appear on the home page of another tech site: It was over a year old. I didn't notice it, the editors didn't notice it, and only a few of 300 comments pointed it out.
SoylentNews is Bacon! [nueskes.com]
(Score: 2) by frojack on Friday January 16 2015, @05:26AM
A typo while talking about challenges in English; that's a hell of a time to make a mistake, isn't it? ;)
No that's par for the course. Muphry's Law [wikipedia.org] I believe.
No, you are mistaken. I've always had this sig.
(Score: 2, Interesting) by Anonymous Coward on Friday January 16 2015, @04:30AM
This reminds me of the American auto industry, which basically didn't care about fit and finish (doors closing without rattling, inside handles staying attached for more than six months, etc) for several decades after the Second World War because they were breaking sales records almost every year. Oh, the Germans sold better cars, but those were expensive here, and everyone knew that getting them serviced in America was *really* expensive.
Then the series of OPEC oil shocks hit in the '70s, and consumers became more interested in smaller cars, which the Japanese specialized in. After they started buying them, they noticed that the Japanese had spent a long time working on fit and finish - with help from the American statistician W. Edwards Deming and his peers, who were ignored in America but were very highly respected in Japan.
I think we're at the cusp of an "OPEC" event with regards to localized UI and error messages, and correct handling of substitution parameters (even in English). Right now, customers might not care that much because they can still get their work done. But at some point, suddenly it's going to matter, and those who can't provide polished product will be left in the dust.
(Score: 2) by Common Joe on Friday January 16 2015, @06:16AM
I don't know. Living here in Germany, the majority of technical stuff is done in English, but content is for locals. I.e., the programmers can read and write English (even if they can't speak it) so they understand how to code, but the website content is in German.
I've never done localization, but it seems very difficult for both the programmers and the translators. Difficult = $$$. (Or, difficult = €€€ in my case.) I think only the larger businesses will invest the time and money into this kind of stuff.
In theory, localization should be straight forward, but how does one thoroughly test a complicated program (with tough to find error messages) in multiple languages to make sure it all came out ok? My local trains can't even get the English right over the loud speakers asking us to change trains. ("This trains ends at the next stop. All change please." My wife and I joke that we need to keep a spare set of clothes in our backpacks so we can change.) I can say that the quality of translations that my wife and I have seen in every day life have been extremely poor. Localization of programs is an order of magnitude harder.
(Score: 2) by tangomargarine on Friday January 16 2015, @08:18AM
If that's the worst example I could find, I'd count myself extremely lucky.
"Is that really true?" "I just spent the last hour telling you to think for yourself! Didn't you hear anything I said?"
(Score: 2) by Common Joe on Friday January 16 2015, @10:18AM
[Chuckling] No, no... the train example is far from the worst. It's just something I've heard very often for over a year. It's so simple that it shouldn't be a problem, but it is still a problem even after they updated some of the voices. The idea was that if simple things can't be done right, the more complicated things will be that much more likely to be with error or cost a lot more. That was all I was trying to say.
(Score: 3, Interesting) by sudo rm -rf on Friday January 16 2015, @08:35AM
As a sidenote, i think some projects / manufacturers take i18n too far, e.g. Excel and Libre Office Calc translate the function names to your locale settings by default[1]. My Linux machine with culture settings de_AT and my Windows machine with de_DE both translate
=SUM to =SUMME
AND to UND
and so on. Because most online resources/tutorials/examples are in English, it can give you a hard time to search the internet for help. Luckily there are dictionaries like this [excelnova.org].
[1] I know I can change the setting of how to display the function names, but it doesn't help when having a german speaking customer on the line discussing an xlsx.
(Score: 0) by Anonymous Coward on Friday January 16 2015, @12:28PM
Difficult = $$$. (Or, difficult = €€€ in my case.)
Difficult = ¥€$
(Score: 3, Insightful) by Marand on Friday January 16 2015, @04:45AM
Maybe it's because of missing details, but it seems like the author could have redefined the problem and avoided the complex situation and solution of the story. He even touched on a way to do it at one point, when he mentioned that it's much easier to translate static phrases with unchanging elements, and that adding changing elements complicates the translation greatly. Instead of trying to make dynamic sentences, it could have been possible to reword the text in a way that separated the numbers from the translated strings. Example:
Directories scanned: %g
Matching directories: %g
Matching files: %g
(There's probably a better way to word it, but for the sake of example this is sufficient.)
Sometimes you get so hung up on solving the problem in front of you, instead of redefining the problem to be more manageable, that you make more work for yourself.
(Score: 2) by CirclesInSand on Friday January 16 2015, @05:33AM
Especially if you are using it for console output, it writing a quick regex that pipes in the stdout of the other program much easier this way. For a GUI, on the other hand, and for a program that is for sale, you might have a boss that expects prose.
(Score: 0) by Anonymous Coward on Friday January 16 2015, @10:29AM
Of course if you want to use it for regex matching, you don't want to apply any locale at all (or equivalently, apply the "C" locale), because even with that form, your regex /Maching (directories|files):/ will match neither Passende Verzeichnisse: nor Passende Dateien:.
But you could make this "itemized form" the base form, and the "prose form" ("I scanned %g directories." + singular/plural form) the English locale. Which would have the advantage that you'd get less scripts which forget "LANG=C" before calling tools whose output shall be parsed, because people tend to forget that not everyone has a non-English locale, so your code already breaking with English locale is an advantage.
In other words: Make the "C" locale equivalent to "machine readable English" and the "en" locale equivalent to "pretty English prose for humans".
(Score: 0) by Anonymous Coward on Friday January 16 2015, @10:33AM
This should, of course, have read: "… that not everyone has an English locale".
(Score: 2) by frojack on Friday January 16 2015, @05:33AM
Exactly. And most languages have such constructs.
You can forget about the plural/singular forms when you use this construct, because it really doesn't matter. Unless you are trying to write conversational text, any time wasted on this sort of problem really isn't worth it.
No, you are mistaken. I've always had this sig.
(Score: 2) by MrNemesis on Friday January 16 2015, @05:08PM
I was going to chip in with the same thing - I do the same sort of stuff myself in shell scripts (because I'm a terrible coder and formulating sentences from variables is frequently hard even in a single language) but it's a perfectly valid approach.
However, I get the impression that this was tended as an overly simplistic example just to demonstrate some of the hideous hidden complexity inherent in translation; "if translating two very simple statements and some numbers is hard, imagine trying to do this with a whole X".
Fascinating article by the way and even though I don't code much perl and don't know what gettext and maketext are I actually read the whole thing. As a native and solely english speaker who didn't have any languages other than french to study at school it's a brilliant glimpse of another world.
"To paraphrase Nietzsche, I have looked into the abyss and been sick in it."
(Score: 2) by c0lo on Friday January 16 2015, @05:12AM
gettext has ways which handle the pluralisation for quite a while.
Next time, RTFM [gnu.org] - as a translator, you may also want to read the tramslating plurals [gnu.org] section.
For your hair-raising pleasurable experience (assuming you still have hair to raise):
https://www.youtube.com/@ProfSteveKeen https://soylentnews.org/~MichaelDavidCrawford
(Score: 1, Informative) by Anonymous Coward on Friday January 16 2015, @06:09AM
I haven't used gettext() before, but I have used the Qt toolkit, and it too has a fairly easy-to-use translation system (QtLinguist), which is also plural-aware [digia.com].
Some interesting facts about plurals from the Qt documentation:
(Score: 2) by Common Joe on Friday January 16 2015, @06:26AM
This is one of the best intros to localization I've ever read. It's actually easier to understand than just about every other localization doc I've seen to date. (From a beginner's point of view.) I wouldn't recommend the GNU articles suggested. That is difficult reading.
(Score: 2, Informative) by martyb on Friday January 16 2015, @06:44AM
Yes, traMslating is very challenging! =)
From the linked story:
And, to my reading, provided the background and justification for their writing the Locale::Maketext [cpan.org] CPAN module. Nice to see that gettext has been updated since the linked story was published back in 1999.
My hair supply is rather limited and not up to the challenge (pun intended!) Your example agrees well with this part of the linked story (emphasis added):
So, yes, that makes the English "directory" vs "directories" distinction look like child's play, by comparison; thanks for the illuminating example! (I hesitate to submit this as it is now 1:30 AM and I'm tempting Muphry's Law [wikipedia.org]; oh well!)
(Score: 2) by c0lo on Friday January 16 2015, @10:49AM
Indeed (especially after a sleepless night; I should refrain from commenting in such cases).
https://www.youtube.com/@ProfSteveKeen https://soylentnews.org/~MichaelDavidCrawford
(Score: 2) by dcollins on Friday January 16 2015, @07:15AM
That part of the article seemed much less shocking than was intended, to me, because it seems broadly analogous to the English issue of ending an ordinal number (1st, 2nd, etc.):
x1 st
x2 nd
x3 rd
x4-0 th
except
x11-13 th
(Score: 0) by Anonymous Coward on Friday January 16 2015, @11:42AM
You mean ó?
(Score: 1) by fritsd on Friday January 16 2015, @01:52PM
ISO latin-2 is sóó old-fashioned! ;-)
(yes that was a jǫke)
(Score: 1) by cliffjumper222 on Friday January 16 2015, @07:06PM
I write plugins for Minecraft and volunteers have translated the locale.yml (and other) files into Chinese, Korean, Polish, French, German, etc. However, normally the admin of the server translates the file, or comes up with their own server-specific text. locale.yml is a lexicon file with all the strings of the whole game written out with variables marked by tokens, e.g., [playername] or [cost], etc. Those tokens are replaced by the values when the string is shown. Color codes are also available. The advantage of this is that the translator can put those values anywhere they like in their translation. Sometimes in a sentence, sometimes not. This works for simple, non-professional situations.
What this article taught me was that in some languages (Russian sounds daunting!), you need to display different grammar depending on the data that is to be shown. In essence, you need a little scripting capability. This is actually not that difficult to do. I can add an option to add Javascript (or similar), into the locale.yml file for each element. This would be executed at run time if it exists. I'm quite excited about this because it could be really useful and extensible. Admins would be able to get really creative with the responses shown to commands based on the output rather than using the stock output!
As with all the translations, if anything fails or is absent the default English translation is shown. This speaks to the "inheritance" part he was talking about. It also alerts the admin (who usually makes the locale.yml file themselves) to where they need to translate, which is not always done in one go.
He who dares wins, Rodney
(Score: 2) by darkfeline on Friday January 16 2015, @11:18PM
You do it by writing the semantic tree of what you want to say instead of writing it in one language and then translating it to another language.
For example, instead of
gettext("Your query matched 10 files in 2 directories.")
you do (in Lispy grammar)
(gettext '(matched (your query) (10 files) (in 2 directories))
The idea is you design a semantic grammar tree that can then be automatically compiled into whatever language you want. The semantics don't change, only how each language expresses each semantic construct. Note that the preferred semantic structure may vary between languages (e.g., language A says (eat me apple), language B (me (eat apple)), but these are logical transformations that can be done easily and reversibly.
Here's a post that talks about a similar concept: http://felesatra.moe/posts/2015/01/01/the-babel-meta-language/ [felesatra.moe]
Join the SDF Public Access UNIX System today!