https://www.wired.com/story/jia-tan-xz-backdoor/
The Wired article linked above is a good high level overview. For those interested in the low level how does it work, how was it hidden details then this web page is a good read: The xz attack shell script
Quote from Wired article:
The scourge of software supply chain attacks—an increasingly common hacking technique that hides malicious code in a widely used legitimate program—can take many forms. Hackers can penetrate an update server to seed out their malware, or even break into the network where the software was developed to corrupt it at the source. Or, in the case of one particularly insidious software supply chain attacker known as Jia Tan, they can spend two years politely and enthusiastically volunteering to help.
Over the weekend, the cybersecurity and open source software community was shocked by the news that a relatively new, experimental version of XZ Utils—a compression utility integrated into many popular distributions of Linux—contained a backdoor that would have allowed hackers in possession of a specific private key to connect to the backdoored system and run their own commands as an administrator. Only some chance detective work carried out by a lone Microsoft engineer, Andres Freund—who'd detected a strange delay in how the remote connection protocol SSH was running in a version of the Linux variant Debian—caught the spy trick before it ended up in many millions of systems worldwide.
(Score: 1, Insightful) by Anonymous Coward on Monday April 08, @05:07AM
(Score: 5, Interesting) by krishnoid on Monday April 08, @05:11AM (3 children)
When I read this strip [schlockmercenary.com], I thought it was a little far-fetched for fiction. Truth appears to have proved me wrong.
(Score: 4, Interesting) by Subsentient on Tuesday April 09, @04:35AM (2 children)
It's no use trying to hide from the three letter agencies nowadays if they're interested. The best you can do is be as irritating as possible if they try to spy on you.
Our technology is absolute swiss cheese, with deliberate backdoors like this, baked-into-silicon backdoors [arstechnica.com], extremely severe, suspiciously widespread vulnerabilities in every UEFI machine. [arstechnica.com], and just about every type of undiscovered (or undisclosed) speculative execution attack imaginable.
I use a lot of my own stuff. It's not quantum-safe (working on that, I'm close), it's not peer-to-peer, but it's obscure with undisclosed source code, and it's designed to be as irritating as possible for anyone trying to snoop. More trouble than it's worth, so to say.
Other than being a young white dude, I don't really have any affiliations or beliefs that would typically concern them, but after Roe V Wade was overturned, I knew this government was no longer to be trusted to any length. I never encrypted my drives before Roe V Wade was overturned.
I believe I started researching how to encrypt them the same day.
It signalled to me that the legal system can no longer be held to be stable, and that it would be wise to minimize my footprint in case of future political etc persecution by an even-less-scrupulous US Government.
And, if you do encrypt your drives, don't use the TPM. Just, don't. I have a feeling some real ugly shit about that will come out eventually.
"It is no measure of health to be well adjusted to a profoundly sick society." -Jiddu Krishnamurti
(Score: 2) by maxwell demon on Tuesday April 09, @05:48PM (1 child)
Well, if the shit hits the fan, your use of non-standard encryption will be used as evidence of you hiding something malicious. Because, after all, why would you put that much effort into it? No decryption of your content needed.
The Tao of math: The numbers you can count are not the real numbers.
(Score: 2) by Subsentient on Wednesday April 10, @02:40AM
Yeah, I know. They did that in France to some guys using Signal I think. They hadn't done anything, but using decent encryption was enough to get them raided and arrested.
It's not about that for me at this point, it's more of a "I see what you're doing, eat sh*t" kind of thing.
"It is no measure of health to be well adjusted to a profoundly sick society." -Jiddu Krishnamurti
(Score: 3, Informative) by vali.magni on Monday April 08, @05:23AM (9 children)
The surname Tan rang some bells, and with Cheong thrown into the mix, this points to Southeast Asia, specifically Malaysia, Singapore, Indonesia and some neighbouring countries where these surnames are common among the Chinese diaspora. You're more likely to find a Tan or a Cheong in Southeast Asia and places outside China, a Cheung in HK, and often 张 (Zhang) in mainland China.
(Score: 0) by Anonymous Coward on Monday April 08, @05:56AM (1 child)
What IP addresses were the commits and comments from? Tor/VPN IPs? bot farm IPs?
(Score: 2) by kazzie on Tuesday April 09, @05:19AM
A VPN in Singapore, per TFA. So the other end of the tunnel could be anywhere, but that's where they wanted to appear like they were coming from.
(Score: 2, Interesting) by shrewdsheep on Monday April 08, @10:57AM
Maybe it's worthwhile to look through all available email/chat/comments to see whether, apart by name, there is any indication that the backdoor authors were actually Asian. I could well imaging a false flag operation.
I was surprised that one bug came through in 5.6.0. On a state level, I would have expected internal testing and review, as any required fix would make discovery more likely. Certainly, those oversights can happen even at that level.
(Score: 3, Insightful) by loonycyborg on Monday April 08, @12:40PM (1 child)
Chinese names can't be even accurately reproduced in latin letters since this alphabet doesn't support tone. There are many transliteration traditions and each particular person can follow any of them or even invent their own.
(Score: 2) by RamiK on Monday April 08, @06:47PM
It parses as gwoyeu but the resulting pinyin (jiā qiǒng tān) isn't Mandarin (a mix of Hokkien-Taiwanese-Singaporean surnames?): https://ctext.org/pinyin.pl?if=en&text=Jia+Cheong+Tan&from=gwoyeu&to=pinyin [ctext.org]
Disclaimer: I don't know any Chinese.
compiling...
(Score: 0) by Anonymous Coward on Monday April 08, @03:01PM
Cheong is almost certainly a romanization of Korean, while Jia is probably Chinese, and Tan could be from anywhere.
(Score: 4, Touché) by owl on Monday April 08, @04:58PM (1 child)
And, of course, if you were John Smith, employed by the NSA,and given the task of "add a backdoor to sshd" with the sub-task of "and make it look like China is the culprit", you might pick a fake name of "Jia Tan" in order to attempt to deflect blame onto the geographical location you were told to deflect blame onto.
Because we have no information on the actual veracity of the name "Jia Tan" we can speculate that it is a south-east Asian name all we want, but that maybe just exactly what the real attacker wanted us to do.
(Score: 1) by khallow on Tuesday April 09, @03:53AM
(Score: 0) by Anonymous Coward on Tuesday April 09, @01:48PM
I'm sorry. You believe this person never thought to use a fake name?
(Score: 5, Interesting) by Anonymous Coward on Monday April 08, @05:38AM (5 children)
It's not a supply chain attack when your "suppliers" are literally volunteers (or, in this case, malicious actors pretending to be volunteers) with whom you have literally no business relationship whatsoever. You are in essence just using stuff you found for free on the internet. Big businesses go crying that their "supply chain" was compromised in the hope that they can convince volunteers to do even more work for free.
This sort of thing is exactly why basically all free software on the internet is distributed with text similar to this:
The biggest takeaway from this I think is that volunteer maintainers should be more direct at politely telling choosing beggars to please fuck off. You can do this even if you aren't directly involved in maintenance. Anyone on the xz-devel mailing list could have told "Jigar Kumar" or "Dennis Ens" to kindly go fuck themselves when they posted shit like:
or
But nobody did.
(Score: 3, Interesting) by sjames on Monday April 08, @08:07AM (2 children)
The problem I have with all of this is novel-itus. XZ is is a data compressor. At least the core functionality seems to be mature and by all rights should be seeing a change rate somewhere near glacial. That's a GOOD thing! That's also why in Debian, only Sid included the backdoored version. Testing and stable included a version prior to the back door. Similar for other major distros I know of.
(Score: 3, Interesting) by owl on Monday April 08, @05:12PM (1 child)
The problem is that the Microsofts and Googles and play stores and app stores of the worlld has mistrained a whole generation of people that software which is not receiving a constant stream of useless changes to the lipstick is somehow bad and a 'problem' to be fixed.
Meanwhile, you are quite right, for a compression library, one would expect at some point for it to be "complete" and not need any changes for a very long time. And even then, the changes would likely be on the order of "fix compile issue with new GCC 7.2.4" and not anything directly related to the library itself.
In this case, however, the general internet consensus is that the pushy users were just Jia Tan sock puppets created for the express purpose of being pushy to try to social engineer the original developer into granting Jia Tan commit privileges.
(Score: 0) by Anonymous Coward on Tuesday April 09, @06:16PM
While that is probably the case in this instance, this sort of discourse is unfortunately not unusual on public forums for free software projects run by volunteers. The behaviour of "Dennis Ens" and "Jigar Kumar" would not have raised any eyebrows before this incident. Other times they might be sock puppets / employees for big businesses trying to get volunteers to work for free. Or they might just be assholes with an internet connection and no other agenda. It doesn't matter. People posting this sort of shit publically should be publically told to shove it.
(Score: 3, Insightful) by RedGreen on Monday April 08, @12:31PM (1 child)
"The biggest takeaway from this I think is that volunteer maintainers should be more direct at politely telling choosing beggars to please fuck off. You can do this even if you aren't directly involved in maintenance. Anyone on the xz-devel mailing list could have told "Jigar Kumar" or "Dennis Ens" to kindly go fuck themselves when they posted shit like:"
Then you get the up in arms thought police of open source outraged and on the maintainers back for being rude or anti-social by telling some moron to pound sand up his ass. If you have not noticed that disease is everywhere in Linux/Open Source now a days they sit and wait to pounce on anyone not up to their "standards" of behaviour. They call it a Code of Conduct for most of them projects.
"I modded down, down, down, and the flames went higher." -- Sven Olsen
(Score: 3, Touché) by owl on Monday April 08, @05:14PM
Which is exactly the problem when a code of conduct infects a project.
Instead those who want the code of conduct should be told, in direct language, exactly where they can shove their code of conduct. If they leave because of that truth, well then, they probably were never going to be worth having around anyway.
(Score: 1, Insightful) by Anonymous Coward on Monday April 08, @05:53AM
https://www.reddit.com/r/linuxmemes/comments/1bref3b/updating_xkcd_dependency/ [reddit.com]
p.s. might be more than one backdoor...
(Score: 4, Interesting) by Mojibake Tengu on Monday April 08, @08:26AM (4 children)
In good ancient times, when me young became an administrator of a mainframe, with all those undisciplined console operators, fancy users and all kind of strangers bringing in their batch jobs on punch cards feeding up the reader, my boss (who had a bigger mainframe to administrate), taught me the first lesson of system administration:
You are never paranoid enough!
If I was an established Linux distro dev, with a good name, and asked by some random state agency to introduce a backdoor in my work for their power and glory (which is inevitable in times of peace, the absolutely certain in time of war), I'd rather invent a fake person to delegate such disreputable task on.
And that's what I, classically paranoid, expect as really happened.
Respect Authorities. Know your social status. Woke responsibly.
(Score: 5, Interesting) by Thexalon on Monday April 08, @11:45AM (3 children)
The actual story looks a bit different: Apparently, this was code that a dedicated volunteer had been maintaining, thanklessly, for something like 35 years. He wanted to step away from that because he's retiring. He put out a call to find people to take over for him, and encountered someone who looked promising. They worked side-by-side on it for 2 years, starting with the new volunteer giving him patches to merge, then both of them having merge privileges, and then the dedicated volunteer stepping away. And it was the moment the dedicated volunteer stepped away that the attack was done.
So whichever agency did it ran a very long game. And yeah, it turned out that guy wasn't paranoid enough.
The only thing that stops a bad guy with a compiler is a good guy with a compiler.
(Score: 3, Touché) by PiMuNu on Monday April 08, @12:05PM (2 children)
In closed source world, I guess it is equivalent to embedding an employee in M$/Apple and then sitting for a few years. Of course, all the code in closed source world is code reviewed a trillion times and no darkness can ever enter (that was a joke).
Given the steaming nature of, for example, M$ cloud offerings one may assume that they are compromised. Many Western governments sit on top of M$ cloud offerings (exchange, Sh**point, etc).
(Score: 2) by choose another one on Monday April 08, @03:53PM (1 child)
Yeah... but no (or maybe, yeah... but maybe).
In closed source corporate world it's pretty darn rare for anyone to be left working alone on one project for years with no one else working on, or up-to-date on, the code, even rarer to be able to target and assign such a role for oneself. There's business continuity planning for a start - the "what if X got hit by a bus" scenario. Also large parts of the closed-source world (both suppliers and customers) place more value generally in support and maintenance because as a customer having invested $$$ in software you want it to be maintained and as a supplier it's a potential profit centre - that support $ pays for people, roles, hours, eyes, redundancy (Note: even if your SLA response times are as long as several days, you still _need_ more than one person on every bit of code because employees get holidays, unlike OS maints...). All a long long long way from fool-proof or invulnerable, but more redundancy and checks and balances than tend to be there in large parts of the open-source world IMO.
Sadly, with open-source the customer usually invested zero in the actual product so the percentage of purchase price that they're prepared to invest in maintaining their investment doesn't matter, it's still zero. Maintainers end up unsupported themselves, unloved and yet held responsible when issues surface (which they will, eventually, even in the most mature code, because the environment code is used in changes - look at shellshock). As to the bad-actor cuckoo-maintainer - TBH I'm just surprised it's taken so long*, more likely it's happened already but not been spotted.
Closed source OSes/stacks are now almost all vast piles of known-to-be-legacy code which the maintaining corporate struggles to keep enough paid teams au-fait with and capable of maintaining whilst (because users hat hate hate it when you bust backwards compatibility) being unable to throw much of it away.
Open source OS/stacks meanwhile are now vast piles of nested dependencies with no one having a clue what is legacy or not let alone whether there is anyone current on each piece (and I reckon there is much that is, de jure or de facto, unmaintained).
*[maybe best to leave out the stuff about whether Poettering is a saint for taking ensuring so many unmaintained projects are now cared for and updated or is in fact the devil-incarnate most evil cuckoo-maintainer ever...]
(Score: 2) by PiMuNu on Monday April 08, @04:17PM
I sort of agree. I think that your "struggles to keep enough paid teams au-fait with" statement is a bit optimistic IMHO. We just don't know about it because it's closed.
(Score: 2, Insightful) by DadaDoofy on Monday April 08, @01:29PM (1 child)
You'd have to be incredibly naive to not believe for each exploit like this that gets exposed, ten more are chugging happily along undetected.
(Score: 0) by Anonymous Coward on Monday April 08, @02:42PM
So, how do you avoid these back doors? OpenBSD?
(Score: 5, Funny) by Rosco P. Coltrane on Monday April 08, @05:13PM (1 child)
And he's none other than...
Andres Freund himself! The perp is hiding in plain sight 🙂
How do I know that? Well, re-read that sentence again: whoever heard of a competent Microsoft programmer? A man this good and this thorough would have left to work for a worthier employer a long time ago.
(Score: 0) by Anonymous Coward on Tuesday April 09, @01:20AM
Nah he's probably working on Linux or Linux related stuff (e.g. Microsoft Azure Linux stuff).
I suspect more and more Microsoft employees are no longer using Windows much (esp after Windows 7).
Just look at Windows 11 - it looks like it was designed by noobs rejected by Apple, developed by noobs rejected by everywhere and mainly tested by the public.
They deprecate/discontinue stuff that large organizations would continue to buy/use Windows for. They remove GUI methods of doing stuff and force CLI only.
So maybe he and others are moles but they are more likely to be wanna be Linux/Apple moles working in Microsoft.
Dunno how true this is, but it's plausible enough to me: https://news.ycombinator.com/item?id=30019307 [ycombinator.com]
These bunch have lost power: https://devblogs.microsoft.com/oldnewthing/20061106-01/?p=29123 [microsoft.com]
(Score: 5, Insightful) by stormreaver on Monday April 08, @06:48PM
This is a strength of Open Source. One person, not even related to the project, with the necessary skills was able to find the trojan by reading the source. No obtuse reverse engineering or blackbox hacking was required. He noticed a problem and was able to find the root cause because he was able to read the source code.
(Score: 1) by khallow on Tuesday April 09, @04:16AM (2 children)
(Score: 3, Insightful) by maxwell demon on Tuesday April 09, @05:27AM
I'd say this project was a target because it had several characteristics that played into the aattack:
Those two points combined meant that it was likely not examined by too many people. Yet, many measures were done that those who do examine it will miss the backdoor.
This means it is perfect for sneaking in an exploit which no one expects.
That is, it was a good target for social engineering.
And the majority of servers in the world is using this. In other words, had this backdoor not been detected at the last moment, the attacker would basically have had root access to close to 100% of all IT infrastructure.
The Tao of math: The numbers you can count are not the real numbers.
(Score: 3, Insightful) by kazzie on Tuesday April 09, @05:34AM
One thing I picked up from reading the linked articles:
It's a compression library, and ships with test files to check for correct decompression of clean and corrupted archives. An intentionally corrupted file is a great place to hide malicious code.
The other bits, like obfuscated code and abusing vulnerabilities in other programs/libraries could be done anywhere. It's the handy garbled "test" files, and the opportunity to dislodge the previous maintainer, that made this project a target.