Submitted via IRC for SoyCow5743
The Slovak National Security Office (NBU) has identified ten malicious Python libraries uploaded on PyPI — Python Package Index — the official third-party software repository for the Python programming language.
NBU experts say attackers used a technique known as typo-squatting to upload Python libraries with names similar to legitimate packages — e.g.: "urlib" instead of "urllib."
The PyPI repository does not perform any types of security checks or audits when developers upload new libraries to its index, so attackers had no difficulty in uploading the modules online.
Developers who mistyped the package name loaded the malicious libraries in their software's setup scripts.
"These packages contain the exact same code as their upstream package thus their functionality is the same, but the installation script, setup.py, is modified to include a malicious (but relatively benign) code," NBU explained.
[...] Indicators of compromise are available in the NBU security alert.
[...] On a side note, and unrelated to the attack vector, NBU also advises Python developers to avoid using "pip" — a Python package installer — when downloading Python libraries, as pip does not support cryptographic signatures.
(Score: 3, Insightful) by Grishnakh on Monday September 18 2017, @02:20PM (3 children)
NBU experts say attackers used a technique known as typo-squatting to upload Python libraries with names similar to legitimate packages — e.g.: "urlib" instead of "urllib."
It seems to me that many things like this, and especially instances where someone takes advantage of cases where letters and numbers look nearly identical (1 vs. l, 0 vs. O) could have been avoided if we, as a society, had never abandoned serifed fonts.
(Score: 0) by Anonymous Coward on Monday September 18 2017, @07:21PM
Meh, I've seen serifed 1's and l's often look similar.
(Score: 2) by FakeBeldin on Wednesday September 20 2017, @12:36PM (1 child)
The impersonating libraries, with the originals (bold highlights the differences):
A different font would not have helped at all against this attack or any attack like this attack. This is not a case of mistaking characters for other characters, this was more of a phishing attack.
(Score: 2) by Grishnakh on Wednesday September 20 2017, @02:40PM
I disagree on a couple of them: acqusition, and urlib3. With these sans-serif fonts where the l and i characters are so thin that you can barely see them (esp. when they're doubled, or not, as in the case of urlib3 (vs. urllib3)), it's easy to not notice the discrepancy if you're not looking for it. Even back in the good-ol' days of non-proportional terminal fonts, these things would be a lot easier to spot because each letter takes up the same amount of horizontal space (in fact, this might be an argument for non-proportional fonts rather than merely serifed ones).
(Score: 3, Interesting) by pvanhoof on Monday September 18 2017, @03:02PM (9 children)
just remove those packages? How did they get accepted in the first place? If I wanted to apt-get install thisisavirus on my Debian, I sure would like the people maintaining the Debian repositories to make it clear to me what the software is going to do. And I sure would like them to verify it at least a little bit. The guy who proposed to add urlib to the package index uses the exact same description as existing package urllib and this didn't get rejected by a responsible individual who cares about the pip repositories or index?
The hell?!
(Score: 2) by JNCF on Monday September 18 2017, @03:32PM (8 children)
They did. As to why they were initially accepted, probably just a lack of human review. I saw internal discussion about another package manager potentially making near matches to existing names off-limits, but obviously there are legitimate cases that would be caught, so maybe near matches should just get flagged for human review... policing this stuff is a thankless job; nobody notices until something goes wrong. I could see perfect description matches happening by happenstance, too, but maybe near matches for both name and description would be rare enough to have a human sort through.
I think the blame should fall on the individual authors who included the wrong package. Sometimes tpyos don't really matter, but this is not one of those times. If the blame is placed on the programmers, and not the maintainer of the package manager, we can eventually cut out the package manager as a point of weakness entirely (think Namecoin).
(Score: 2) by Grishnakh on Monday September 18 2017, @04:34PM (4 children)
policing this stuff is a thankless job
Perhaps, but you don't see this shit happening with Debian, and you do see a very serious commitment there to making sure every package has a proper maintainer and is verified to not be some kind of garbage.
I think the blame should fall on the individual authors who included the wrong package.
It should also fall on the entire culture Python has for having a repository loaded with garbage with no policing at all, and authors including dependency packages for everything imaginable, even if it's some ultra-simple thing that could be done in a few lines of code. As I said before, you don't see this shit with Debian.
(Score: 2) by JNCF on Monday September 18 2017, @05:03PM
Debian is targeting a certain crowd a users, and I don't fault them for that. I don't think every distro should vet their packages by hand, though; a decentralised, uncensorable repository sounds lovely. Our disagreement is ideological.
(Score: 0) by Anonymous Coward on Monday September 18 2017, @05:04PM
this is what happens when you are lousey with mac using hipsters. they were too busy twisting their handlebar mustaches to care about security.
(Score: 4, Informative) by LoRdTAW on Monday September 18 2017, @07:21PM (1 child)
Yet somehow pulseaudio AND systemd somehow made their way into the codebase. (ducks)
(Score: 2) by pvanhoof on Monday September 18 2017, @08:40PM
Well, pulseaudio and systemd's package maintainers at least provide security updates for the malware their upstreams created and get discovered. The package maintainer of bzip in pip will probably not respond with a security update of the package now that malicious code has been discovered. So it has to be removed. Hmm. I guess that would make systemd and pulseaudio haters more happy? Fine then, maintain your local distro with pip instead of apt-get.
(Score: 2) by pvanhoof on Monday September 18 2017, @07:00PM (2 children)
I read the article and noticed that indeed they did. However, you mention that there are obviously legitimate cases that would be caught: any person doing minimal research or review should at the very least have been suspicious about a package proposal for a package named bzip, to be malicious.
If as a Python software developer you don't understand the significance of a package named bzip or the danger when such a package is malicious, then I don't know where to start explaining how wrong your processes will be.
I speculate that the amount of human review on package proposals was roughly equal to zero point zero.
You think the blame should fall on the individual authors who included the wrong package, when malicious pip packages are called bzip? I'm afraid we disagree there.
Isn't the very point of a system like pip to allow Python software developers to rapidly start using various dependencies within the Python eco-system? Why then must they check out all the source code of each and every package, even the ones called fscking bzip, to verify them against not having malicious code? Doesn't that defeat the purpose of pip? If I wanted or had time to check all the code of all dependencies I use within a software development eco-system, wouldn't I also have time to just git clone it from github and compile it all myself?
(Score: 2) by JNCF on Monday September 18 2017, @07:23PM (1 child)
Reading all of the code is usually way more time intensive than gathering some basic meta data about a package (who wrote it, how many contributors there are, how responsive to bug reports the maintainer is, etc.). I don't think this sort of research is unreasonable to ask of the person including the code in their program, but I want the tool that automates inclusion of code to allow me to shoot myself in the foot if I tell it to, and I don't want volunteered hours from people I don't really trust anyway to be the bottle-neck that prevents an ecosystem from rapidly scaling. There are still good reasons to use a package manager rather than manually finding the correct version of an already-vetted module which will work for your current project and compiling it.
(Score: 2) by pvanhoof on Monday September 18 2017, @07:47PM
I suppose Ubuntu's Launchpad is for you, then. It allows individual software developers to easily provide you with a repository upon which their packages and made available. You can add their launchpad to your list of trusted repositories. I guess a system that publishes popularity of such repositories could serve as a somewhat automated way of providing trust.
ie. A Launchpad that contains a pip package called bzip containing malicious code would after this news-item have lost almost all of its trust. You could then quite easily let your scripts or other tools threshold at a certain level of community-assigned trust. Or you could use another source of trust when multiple such sources are made available.
A single repository with no human review whatsoever, however, deserves equal amounts of trust: zero. If that's what pip is, then pip cannot be used. Plus pip apparently has no cryptographic verification either. That sucks for networks that can be MiTM'ed. So that sucks for the vast majority of networks.
(Score: 2) by JNCF on Monday September 18 2017, @03:33PM
(Score: 2) by fritsd on Monday September 18 2017, @03:38PM
That's funny, because just yesterday I came across an article about Python setuptools (didn't know there were so many different ones)
and how they all depend on each other's integrity:
How to bootstrap setuptools from source #980 [github.com]
That issue doesn't show the year (just "Feb") so I don't know if this has been resolved in the last three years or so.
This gives me even more respect for the GCC authors that thought of this bootstrapping problem many years ago.
(Score: 2) by fritsd on Monday September 18 2017, @03:43PM (1 child)
That issue, that I just referenced in my previous comment, talked about using something called "distutils" to bootstrap "setuptools".
Can anyone more familiar with Python (i.e. anyone) explain how I can install distutils from source? Do I need pip, wheel, python-six, python-setuptools, python3-setuptools and python-nose-of-the-camel-in-the-tent pre-installed??
I've got python2.7 and python3.5, if that helps.
(Score: 0) by Anonymous Coward on Monday September 18 2017, @04:12PM
Distutils and ensurepip are in the standard library. If you install python, then both are installed by default. If you want to install distutils from source, then all you need to do is install CPython from source (or copy the .py files into the path, I suppose). Also of note, when you install CPython from source or using official installers, it will automatically run ensurepip, which uses vendorized wheels to install setuptools and pip into the site-packages using distutils.
(Score: 1, Insightful) by Anonymous Coward on Monday September 18 2017, @04:03PM
aka the windoze model
Well done, well done indeed. I hope your new servitude will please you well. (somehow I doubt it)
(Score: 1) by terrab0t on Monday September 18 2017, @04:35PM (1 child)
The same thing recently happened to Node Package Manager (NPM) [scmagazine.com]. They were looking at automated ways of detecting these kinds of similarly named packages.
I think they have the same problem as PyPl. No one has time to manually audit each package submission.
(Score: -1, Redundant) by Anonymous Coward on Monday September 18 2017, @05:37PM
whitelist