Brian Krebs reports today on the biggest global information security freak-out since Heartbleed (2014). Or not -- I'm not sure.
Virtually all compilers -- programs that transform human-readable source code into computer-executable machine code -- are vulnerable to an insidious attack in which an adversary can introduce targeted vulnerabilities into any software without being detected, new research released today warns. The vulnerability disclosure was coordinated with multiple organizations, some of whom are now releasing updates to address the security weakness.
TL/DR: Because of Unicode string processing in all editors and compilers, and specifically how RTL [Right-to-Left][*] and LTR [Left-to-Right][*] control codes are supposed to affect the ordering of all characters regardless of whether or not they belong to an LTR language... Any source code processed by a Unicode-aware compiler is subject to hidden meaning where what's rendered in your editor or terminal is not what is actually read by the compiler. Re-ordering the display of characters in a block of code can change the meaning of comparison statements, string or number constants, and comments.
Krebs cites a paper (PDF) from researchers at the University of Cambridge, which contains some nifty code examples including changing "User is not in Admin group" to render as logic for "User is in Admin group" in every source control tool or editor you might use. This sort of supply chain attack can be inserted by anyone with commit access to the code you use from upstream sources -- disgruntled employees, open source contributors; virtually all software you use now could be a target.
Is this the end of the world, or just another Monday?
This post was written in pure ASCII, just to be safe.
[*] https://en.wikipedia.org/wiki/Right-to-left_mark.
(Score: 5, Funny) by Anonymous Coward on Tuesday November 02 2021, @03:36AM (2 children)
Both.
(Score: 0) by Anonymous Coward on Tuesday November 02 2021, @04:47AM
just another manic monday
(Score: 0) by Anonymous Coward on Tuesday November 02 2021, @04:58AM
(Score: 3, Touché) by drussell on Tuesday November 02 2021, @03:39AM (30 children)
Who the fork-a-frick-a-frack edits their code in an environment that even understands unicode?!
I'm pretty sure I'm safe with vi on a serial terminal or fully-text-only console...
(Score: 5, Insightful) by FatPhil on Tuesday November 02 2021, @04:20AM (3 children)
The malware writers would like to thank you for your blind cooperation.
Great minds discuss ideas; average minds discuss events; small minds discuss people; the smallest discuss themselves
(Score: 5, Insightful) by tangomargarine on Tuesday November 02 2021, @04:43AM
Plenty of languages handle Unicode, but at least some of them require you escape it, which seems like the more reasonable path, so you can save your source as plain ASCII or whatever.
Understanding Unicode is one thing; displaying it as-is is another. Especially when you're talking about coding, which really should have no reason to "pretty print" things. Do it like emacs does and display it as flagrantly a special character like "\233" or something.
"Is that really true?" "I just spent the last hour telling you to think for yourself! Didn't you hear anything I said?"
(Score: -1, Troll) by Anonymous Coward on Tuesday November 02 2021, @06:00AM
Oh, great, now I suspect FatPhil as being one of the malware authors! And he is right here on SoylentNews!!!
(Score: 5, Funny) by driverless on Tuesday November 02 2021, @09:13AM
This is why I edit all my code as comments on Slashdot, that doesn't even know about latin-1 so I'm fully protected.
(Score: 5, Insightful) by tangomargarine on Tuesday November 02 2021, @04:39AM (13 children)
Probably the same people who think that Python using indentation to compile was a good idea. Because what could possibly go wrong?
I did a little bit of Python for work and it's got some nice ideas, but seriously, the indentation thing is just crazy.
"Is that really true?" "I just spent the last hour telling you to think for yourself! Didn't you hear anything I said?"
(Score: 2, Troll) by PiMuNu on Tuesday November 02 2021, @06:01AM (11 children)
> the indentation thing is just crazy.
Why?Ormorespecificallydoyoubelievethatpeopleshouldbefreeto
put in whitespace
whereverthey like?
(Score: 0, Flamebait) by Anonymous Coward on Tuesday November 02 2021, @06:56AM (5 children)
The issue is that whitespace has semantic meaning above and beyond token separation in python.
Why you gotta lie and distract on purpose?
(Score: 0, Informative) by Anonymous Coward on Tuesday November 02 2021, @09:02AM
Whitespace has semantic meaning above and beyond token separation in most languages. Its just that python has it for more than the human compiler too.
(Score: 2) by PiMuNu on Tuesday November 02 2021, @01:39PM (3 children)
Fair point, but every sane programmer I have seen since the 1980s uses indentation to delineate different levels of conditional/loop. So it is not too bad to make it a requirement of the language (or whenever fortran abandoned 7 white spaces with column 3 for comments). The only bad thing is that they let folks use tab indentation (which is a crime against humanity).
(Score: 1, Touché) by Anonymous Coward on Tuesday November 02 2021, @01:44PM (1 child)
(Score: 2) by PiMuNu on Wednesday November 03 2021, @11:23AM
Saves space in the source file.
Laughs.
(Score: 1) by shrewdsheep on Wednesday November 03 2021, @10:37AM
I am solidly in the tab-using camp using it for all languages including python. I used to be derogatory about the python white space handling but I have gotten over it some time ago. All languages have deep flaws and the lack of insight of the designers into certain aspects are showing blatantly. Use the right language for the task, write clear code and let other bicker about white space.
What I have not seen in text editors but could help in these discussions is a way to reformat coding styles on the fly, i.e. re-format into your preferred style on loading and save into another style. This is not trivial as full parsing would be required but doable IMO.
(Score: 0) by Anonymous Coward on Tuesday November 02 2021, @07:06AM
FTFY
(Score: 5, Insightful) by tangomargarine on Tuesday November 02 2021, @02:12PM (3 children)
I like that thing in some IDEs where there's a key combination to auto-indent your code based on braces and various other things. You accidentally put a tab in the wrong place, and Python won't even compile, let alone manage to auto-indent (because the information to do so doesn't exist). So you can get in the situation where you can't be sure you're actually fixing the indentation correctly in code you're unfamiliar with, which is much less likely (although not impossible) with braces.
"Is that really true?" "I just spent the last hour telling you to think for yourself! Didn't you hear anything I said?"
(Score: 2) by PiMuNu on Tuesday November 02 2021, @02:25PM (2 children)
(Score: 2) by PiMuNu on Tuesday November 02 2021, @02:36PM (1 child)
if (a != b); {
do some stuff
}
(Score: 0) by Anonymous Coward on Tuesday November 02 2021, @03:24PM
look closer:
if( a!= b ) ; // note semicolon
{
do some stuff
}
(Score: 0) by Anonymous Coward on Tuesday November 02 2021, @06:51PM
my first lang with this behavior has been nim and it has caused zero issues for me. I like it in fact as it's actually easier to see what the hell is going on.
(Score: 5, Informative) by janrinok on Tuesday November 02 2021, @08:06AM (5 children)
We don't all speak in ASCII you know. Look outside, there is a big wide world out there all speaking different languages.
(Score: 0) by Anonymous Coward on Tuesday November 02 2021, @01:40PM (2 children)
It's code, not a haiku or poetry.
I'm in favour of speaking more than one language, but code is code - KEEP IT SIMPLE, SHITHEAD. No accented characters, no crdilas, no circumflexes, no umlauts, nothing. Just look at how scammers have been using unicode to typosquat. North Korea takes in more than 2 billion a year from cyberscams. Boycott unicode - you're helping finance terrorists.
(Score: 3, Insightful) by janrinok on Tuesday November 02 2021, @04:04PM (1 child)
So, for people who don't happen to use your language, how do they write meaningful function and variable names? Or do you believe that your ascii should be enforced upon others?
The fault lies in the compilers. The article says that "some of whom are now releasing updates to address the security weakness". It can be fixed, so you leave your arrogant "Not my problem" out of this. You are the problem. The compiler can be changed - you, on the other hand, not so easily.
(Score: 1, Insightful) by Anonymous Coward on Wednesday November 03 2021, @04:23AM
English is the #1 language in the world. If you throw in everyone who understands it as a second language, it's the majority of tge world's population. So again, for source code, 7 bit clean is good enough for the majority.
After all, you STILL have to use the english keywords (if, switch, case, default, break, etc) or it won't compile anyway.
Back in the 80s I wrote a program that changed the in-memory tokens in the BASIC interpreter to french, so you could actually use french keywords; saving code in binary format still used the english keyword tokens, which allowed for automatic translation between the two languages. Demoed it, it was pretty cool, but french users still preferred to code in english - out of habit and so as not to have to learn a second set of keywords. So I dropped it, having learned the lesson that people prefer one set of standards for things like keywords.
And that's still true today. There is zero justification for including unicode in source code. Just explicitly escape it if you need to use it (example: a user message). For actual logic? Don't be illogical.
(Score: 2) by epitaxial on Tuesday November 02 2021, @02:07PM (1 child)
I use EBCDIC.
(Score: 0) by Anonymous Coward on Wednesday November 03 2021, @02:34AM
Our mainframes support Unicode in EBCDIC. You can have the best (worst?) of both worlds over there.
(Score: 3, Interesting) by bradley13 on Tuesday November 02 2021, @11:45AM (2 children)
Um, how about "almost everybody". If I understand TFA correctly, this applies to essentially every major IDE out there: IntelliJ, NetBeans, VSCode, Eclipse, etc, etc.. No, most people do not program in emacs or vi. Having IDE support makes most programmers more efficient. Also, for anyone working in a non-English-speaking environment, some text (for example, comments) may be written in languages that require something beyond the ASCII table.
That said, code is always written left-to-right. It would be entirely reasonable for IDEs to flag LTR/RTL (and any other non-printing characters) in code as errors. Whether they should be acceptable in quoted text blocks or comments is a more difficult question.
Everyone is somebody else's weirdo.
(Score: 0) by Anonymous Coward on Tuesday November 02 2021, @01:46PM
(Score: 1, Interesting) by Anonymous Coward on Tuesday November 02 2021, @04:59PM
There are projects working on right-to-left code to go along with spoken languages that work that way. As far as I know they don't have much traction yet, but since hundreds of millions of people speak languages like that I expect it to take off eventually.
(Score: 2) by DannyB on Tuesday November 02 2021, @03:35PM (2 children)
Java programmers and their various IDEs. (Integrated Development
EmbarrassmentEnvironment)The Narns and Klingons will be very offended if you do not support their native languages with Unicode.
Don't you realize all of the amazing power tools on a modern IDE? With just a right click and a few menu selections, you can screw up source code with much greater speed and efficiency than you would ever dream of with primitive tools like "vi".
Some people don't like the noise and complexity of IDEs. Just as some prefer to dig a ditch with a shovel instead of the noise and complexity of a backhoe.
The Java compiler understands Unicode.
// constants
public final double π = Math.PI; // Pi
public final double τ = 2.0 * π; // Tau
public final double ℯ = Math.E; // e
private final int £€$æøå = 42;
// a function
public void £€$æøåσΣ() { }
Here is a fun practical example:
/**
* Sigma function.
* @param fn a Java 8 Function that takes an integer an returns an integer.
* @param first
* @param last
* @return Sum of calling fn over range from first to last, incremented by one.
*/
public int Σ( java.util.function.Function fn, int first, int last ) { . . . }
To transfer files: right-click on file, pick Copy. Unplug mouse, plug mouse into other computer. Right-click, paste.
(Score: 0) by Anonymous Coward on Wednesday November 03 2021, @04:33AM (1 child)
As for IDEs, Borland c++ 3.1 with dual monitor support was good enough to be productive without all the crap IDEs like eclipse shovel onto us.
Simplicity forces you to plan your shit. But then again, Java was so badly "optimised" that it couldn't even handle multiple inheritance - a basic flaw.
(Score: 0) by Anonymous Coward on Thursday November 04 2021, @06:56AM
A flaw? Maybe it just forces you to plan your shit.
(Score: 5, Informative) by owl on Tuesday November 02 2021, @03:46AM (6 children)
Where have these researchers been since 1984? Have they never, ever, read Ken Thompson's paper "Reflections on Trusting Trust [win.tue.nl]"?
Because if they had, then this exploit would:
(Score: 5, Insightful) by FatPhil on Tuesday November 02 2021, @04:44AM (5 children)
Great minds discuss ideas; average minds discuss events; small minds discuss people; the smallest discuss themselves
(Score: 4, Interesting) by maxwell demon on Tuesday November 02 2021, @07:42AM (4 children)
The LANG= will do nothing here because cat doesn't interpret the data; it just copies the bytes from input to output. It's your terminal that interprets the bytes as Unicode for display (and yes, I just tested it). On the other hand, if you display it with less (which you'd likely want to do anyway) it will work, because unless instructed otherwise, less will write byte values for all bytes outside the ASCII range.
However, at least on my system, the terminal ignores the RTL character anyway.
On the other hand, you don't need Unicode support to make your code look different to your eyes than to the compiler; pure ASCII suffices. Just try echo -e 'Test\b\b\boast' on the console. And no, less won't help with that; not even with LANG=
However every decent editor should show you the special characters in some way in both cases.
The Tao of math: The numbers you can count are not the real numbers.
(Score: 2) by maxwell demon on Tuesday November 02 2021, @07:46AM (1 child)
Sorry for the self-reply; I noticed only after submission that my comment is misleading on the behaviour of less. Of course the “outside the ASCII range” only applies with LANG=.
The Tao of math: The numbers you can count are not the real numbers.
(Score: 2) by FatPhil on Tuesday November 02 2021, @11:39AM
-r or --raw-control-chars
Causes "raw" control characters to be displayed. The default
is to display control characters using the caret notation
-R or --RAW-CONTROL-CHARS
Like -r, but only ANSI "color" escape sequences are output in
"raw" form.
-u or --underline-special
Causes backspaces and carriage returns to be treated as print‐
able characters
-U or --UNDERLINE-SPECIAL
Causes backspaces, tabs and carriage returns to be treated as
control characters; that is, they are handled as specified by
the -r option.
Great minds discuss ideas; average minds discuss events; small minds discuss people; the smallest discuss themselves
(Score: 2) by FatPhil on Tuesday November 02 2021, @11:31AM (1 child)
Great minds discuss ideas; average minds discuss events; small minds discuss people; the smallest discuss themselves
(Score: 4, Insightful) by maxwell demon on Tuesday November 02 2021, @12:01PM
Ah yes, “trivial” last-minute changes … who doesn't have experience with those :-)
The Tao of math: The numbers you can count are not the real numbers.
(Score: 3, Funny) by dltaylor on Tuesday November 02 2021, @03:48AM (16 children)
C, at least, has never needed one. When I need UNICODE in a program, those strings are compiled in their own module(s).
What computer languages require UNICODE processing to create the source? If there is not one, then what kind of idiot (oh, yeah, some GenZ fashionista) would use such a tool? If there is one, it's probably the same kind of doofus that invented it.
(Score: 0) by Anonymous Coward on Tuesday November 02 2021, @04:24AM (4 children)
I don't know of any programming language whose keywords are not in 7-bit ASCII code.
The problem is decoding multi-byte Unicode characters/escape sequences, mixing up keywords and strings, causing ambiguities.
(Score: 1, Troll) by FatPhil on Tuesday November 02 2021, @04:59AM (3 children)
Sure, there was a mass-extinction event, but nothing removed the incentive to evolve in that direction again, so naturally we repeated the same old mistakes.
Great minds discuss ideas; average minds discuss events; small minds discuss people; the smallest discuss themselves
(Score: 1, Touché) by Anonymous Coward on Tuesday November 02 2021, @05:19AM (1 child)
You are drunk. That's Ok, so am I.
Me, whiskey and beer. You?
(Score: 2) by FatPhil on Tuesday November 02 2021, @03:19PM
Great minds discuss ideas; average minds discuss events; small minds discuss people; the smallest discuss themselves
(Score: 0) by Anonymous Coward on Tuesday November 02 2021, @01:51PM
(Score: 2) by FatPhil on Tuesday November 02 2021, @04:38AM (4 children)
Great minds discuss ideas; average minds discuss events; small minds discuss people; the smallest discuss themselves
(Score: 1, Insightful) by Anonymous Coward on Tuesday November 02 2021, @09:13AM (3 children)
This stupid notion is precisely the root of all evil. "Idle hands are the devil’s workshop"
Things not needed, should NOT be done. Period. Full stop.
(Score: 5, Touché) by maxwell demon on Tuesday November 02 2021, @11:27AM (1 child)
Adding “Full Stop.” at the end was clearly not needed. Therefore according to your own rule, you should't have done it. In other words, you are breaking your own rules.
Anyway, was your comment needed at all? I doubt so. Indeed, in the grand scheme of things it probably didn't make any significant difference. Therefore according to your rule, you shouldn't have written it.
The Tao of math: The numbers you can count are not the real numbers.
(Score: 0) by Anonymous Coward on Tuesday November 02 2021, @02:24PM
When attempting to appear smart, extra verbosity does not help. Do not strain yourself in vain.
(Score: 2) by FatPhil on Tuesday November 02 2021, @03:23PM
Idealism is nice, but nothing but a pipe dream. Stop pretending that idealism is what actually happens.
Great minds discuss ideas; average minds discuss events; small minds discuss people; the smallest discuss themselves
(Score: 3, Interesting) by krishnoid on Tuesday November 02 2021, @04:52AM (2 children)
Any programming language created by/for native speakers of something other than Latin-1 [sic]?
(Score: 2) by FatPhil on Tuesday November 02 2021, @05:08AM
Any Englishman demanding nothing more than a pound sign supports breaking away from ASCII.
I say this as an Englishman who has never used a UK keyboard mapping, only ever a US one. I am not part of the problem - this is a xenophobia I *can* get behind.
Great minds discuss ideas; average minds discuss events; small minds discuss people; the smallest discuss themselves
(Score: 1, Informative) by Anonymous Coward on Wednesday November 03 2021, @07:06AM
APL
(Score: 4, Insightful) by PiMuNu on Tuesday November 02 2021, @06:17AM (1 child)
> oh, yeah, some GenZ fashionista
Someone who doesn't have a first language that uses Roman character set? You know, 2/3 of the world...
(Score: 0) by Anonymous Coward on Tuesday November 02 2021, @09:22AM
Do you know of the uncounted attempts in those "2/3 of the world" to create "nativized" programming languages, none of which ever went anywhere? Stupid to insert yourself into this discussion if you do not; even stupider if you do.
Politicking has no place in tech. The trouble is, the machinators pushing it there, are not who have to suffer the consequences.
(Score: 3, Insightful) by bradley13 on Tuesday November 02 2021, @11:50AM
The world is not English. Even though most programmers write their code in something resembling English, they may well want at least the comments to be in their native language. I know some people who prefer to name variables in their native language. Non-English languages almost always include letters outside a-z/A-Z, and modern IDEs allow this with no problems.
tl;dr: it's not at all unreasonable to expect Unicode support when coding, if you live in an area where English is not the main language.
Everyone is somebody else's weirdo.
(Score: 1, Touché) by Anonymous Coward on Tuesday November 02 2021, @03:55AM
Not so readable for "regular" humans - e.g., Lisp codes, or even Perl.
(Score: 1, Informative) by Anonymous Coward on Tuesday November 02 2021, @04:07AM (4 children)
There are bunch Unicode code points that look visually identical in fonts but are distinct characters.
I guess the possibility of introducing vulnerability using this Unicode "feature" is minimal, but you never know.
The point is, Unicode needs to be reigned in. That thing went crazy.
(Score: 1, Interesting) by Anonymous Coward on Tuesday November 02 2021, @07:01AM
Our red team did a test of this today. Diffs were more obvious than I thought they would be, but that could maybe have passed inspection in a crunch or with a bit more effort. However that doesn't really matter because our already-existing checks we use caught it without any config changes. It also looked suspicious with syntax highlighting enabled. While I'm glad that the various tools out there will probably catch that so everyone can benefit from catching them, seems that anyone already aware of Unicode risks or using many public tools to lint/check their software were already safe.
(Score: 2) by driverless on Tuesday November 02 2021, @09:27AM
The Schneier blog has replies from people who inserted backdoors/other tricks without resorting to Unicode, so it's quite possible.
(Score: 2) by tangomargarine on Tuesday November 02 2021, @02:15PM (1 child)
reign = to rule over
rein = that thing you use with a horse
"Is that really true?" "I just spent the last hour telling you to think for yourself! Didn't you hear anything I said?"
(Score: 0) by Anonymous Coward on Tuesday November 02 2021, @05:51PM
rein = whip
At least according to Brandon when discussing mounted immigration agents.
(Score: 5, Funny) by fustakrakich on Tuesday November 02 2021, @05:04AM
Well, at least the green site is safe
La politica e i criminali sono la stessa cosa..
(Score: 3, Interesting) by maxwell demon on Tuesday November 02 2021, @07:58AM
This is just another instance of a known problem: What you see is not always what you get. [thejh.net]
The Tao of math: The numbers you can count are not the real numbers.
(Score: 0) by Anonymous Coward on Tuesday November 02 2021, @09:17AM
This one's both trivial to catch, at least now we know about it. Cue a new default option to add some visual indication when RTL/LTR mode is flipped in your terminal emulator.
And also - keep in mind anything malicious done in a widely-shared git repo becomes evidence that can't be deleted.
That's sort of the point of Git. Edit history however you like - until you share it. At least we'll know what *was* compromised, if anything is. Honestly it's probably much worse news for the closed-source guys.
It's a bit insidious, but as pointed out, doesn't hold a candle to 'Trusting Trust', OR the much more relevant work more recently by Christopher Domas.
Probably by this point all mainstream CPU's have a separate hidden core for each major spy agency - all quite possibly unaware of each other.
It's not like it's possible to ensure that the PC's used to design new chips are perfectly secure. So in principle, a 'trusting trust'-like attack on the HDL compilers themselves could be inserting such trojan cores into mainstream new chip designs without the HDL software tool or chip manufacturers having any clue.
It's just not possible to have a human go over the whole design at layout time and look for anything that ought not to be there.
Especially if any tool they might use to automate such a task could likewise be compromised.
The designs are too big, with too many transistors, and too many layers of automation between the top level that engineers see, and what is actually fabricated.
Just reverse engineering a single existing chip would likely take years, and noone is bothering. That horse has long bolted.
If there's ever another world war, you can probably expect that all mass-manufactured CPU's will be knocked out first, via a coordinated attack likely involving such cores.
(Score: 1, Informative) by Anonymous Coward on Tuesday November 02 2021, @09:36AM (3 children)
One grep to solve it all.
(Score: 3, Interesting) by maxwell demon on Tuesday November 02 2021, @11:51AM (2 children)
?gnihtyreve sehctac taht erus uoy erA
Try it with this comment, for example.
The Tao of math: The numbers you can count are not the real numbers.
(Score: 0, Touché) by Anonymous Coward on Tuesday November 02 2021, @01:32PM (1 child)
stdin is a very different beast than the paste buffer.
(Score: 3, Interesting) by FatPhil on Tuesday November 02 2021, @04:20PM
luser@spaz:/home/luser$ od -tc crap
0000000 342 200 256 ? g n i h t y r e v e s
0000020 e h c t a c t a h t e r u s
0000040 u o y e r A 342 200 255 \n
0000054
luser@spaz:/home/luser$ grep -P '\x{2005}' crap
luser@spaz:/home/luser$ grep -P '\x{202E}' crap
?gnihtyreve sehctac taht erus uoy erA
To be honest, I thought SN stripped these characters, I'd support it so doing.
Great minds discuss ideas; average minds discuss events; small minds discuss people; the smallest discuss themselves
(Score: 3, Interesting) by istartedi on Tuesday November 02 2021, @05:11PM
If I were worried about this, I'd come up with some tool that scans the source for non-ASCII characters outside of quotes. I'd make that a pre-commit requirement (I haven't admined source control, but I'm given to understand you can do that). Not sure about other languages, but C, C++ and similar languages are ASCII and if you're handling Unicode in the actual code it's isolated to strings constants. This seems like a relatively straightforward problem to address (famous last words in security, I know).
Appended to the end of comments you post. Max: 120 chars.
(Score: 0) by Anonymous Coward on Wednesday November 03 2021, @10:34AM
Breaking News! Idiots finally discovered unicode!