More than 75% of all vulnerabilities reside in indirect dependencies:
The vast majority of security vulnerabilities in open-source projects reside in indirect dependencies rather than directly and first-hand loaded components.
"Aggregating the numbers from all ecosystems, we found more than three times as many vulnerabilities in indirect dependencies than we did direct dependencies," Alyssa Miller, Application Security Advocate at Snyk, told ZDNet in an interview discussing Snyk's State of Open Source Security for 2020 study.
The report looked at how vulnerabilities impacted the JavaScript (npm), Ruby (RubyGems), Java (MavenCentral), PHP (Packagist), and Python (PyPI) ecosystems.
Snyk said that 86% of the JavaScript security bugs, 81% of the Ruby bugs, and 74% of the Java ones impacted libraries that were dependencies of the primary components loaded inside a project.
[...] Snyk argues that companies scanning their primary dependencies for security issues without exploring their full dependency tree multiple levels down would release or end up running products that were vulnerable to unforeseen bugs.
So dear Soylentils, how do you track vulnerabilities in libraries that you use in your projects and do you scan beyond direct dependencies?
Previously:
(2020-05-16) Nine in Ten Biz Applications Harbor Out-of-Date, Unsupported, Insecure Open-Source Code, Study Shows
(Score: 3, Insightful) by bradley13 on Friday July 03 2020, @11:27AM (26 children)
Don't use external libraries. Seriously, avoid them whenever possible.
For one application, I used little, stand-alone library published by Google. Then came an update, and the little library pulled in a big library, which pulled in 5 or 6 more dependencies. Who knows what all that code did, who knows what security holes it had, and anyway: bloat. There was only one possible answer to this: throw out the library and write my own code for the needed functionality.
Unless a library or framework is doing something you cannot reasonably replicate, avoid it. Write your own solution. It will be smaller, faster, and you will know that it does exactly and only what you need it to do.
Examples of libraries you should use: anything built into the language, encryption libraries, front-end frameworks (JavaFX, React, etc.), database connectors.
Examples of libraries you should avoid: Pretty much everything else
Everyone is somebody else's weirdo.
(Score: 2, Insightful) by Anonymous Coward on Friday July 03 2020, @11:45AM (17 children)
"Write your own solution."
That is a two edged sword.
The good news is that yours is smaller so less chance for bugs.
The bad news is only one set of eyes looking for the bugs.
Maybe a better solution is to encourage small, common, well tested libraries.
That would provide your answer's benefit (low attack surface) but mine also (many eyes looking for bugs).
So how would one encourage such a thing?
Perhaps make it easy to rate how much cruft a web page takes when it loads?
(Score: 1, Insightful) by Anonymous Coward on Friday July 03 2020, @11:54AM
As always, a mixture of opinions is ideal. Everyone writing their own means slower development, less eyes, less maintainability. Everyone collaborating on one project brings a monoculture and a single point of failure.
(Score: 3, Insightful) by HiThere on Friday July 03 2020, @01:50PM (14 children)
The real problem is "dynamic linking" where the particular library that you are using can be updated after you write the code. No language that does that can safely use external libraries. This is one of my problems with go and rust...those languages are designed to be dependent on the net. If you're going to use an external library it should at least match a checksum.
Javascript is what you use to allow unknown third parties to run software you have no idea about on your computer.
(Score: 2) by RS3 on Friday July 03 2020, @03:02PM
Yes, exactly, and I commented on this above.
My thought was to download the needed library code and host it yourself, if licensing allows.
(Score: 3, Informative) by The Mighty Buzzard on Friday July 03 2020, @03:14PM (9 children)
Rust does not update library versions for a project unless you foolishly use wildcards in your library version numbers. And even then, once you've built it once on that box you have to explicitly tell it to update the libraries every time you want to. This still doesn't protect you from any system-wide shared libraries being updated by the operating system though.
My rights don't end where your fear begins.
(Score: 2) by HiThere on Friday July 03 2020, @10:48PM (8 children)
OK, I'm not rust expert, and that's not the way I understood cargo as working. The objection still stands in a weaker form, but in that form it applies to all systems libraries. If your system needs to be secure you should include all the libraries you need statically linked. This can get bulky.
Javascript is what you use to allow unknown third parties to run software you have no idea about on your computer.
(Score: 2) by The Mighty Buzzard on Saturday July 04 2020, @04:13AM
Their library handling is actually pretty sweet even if the libraries themselves are extremely unpolished and constantly breaking compatibility in the quest to use the shiny new coding thing. Have a look into it if you're actually interested.
My rights don't end where your fear begins.
(Score: 2) by fyngyrz on Saturday July 04 2020, @01:36PM (6 children)
Static linking, absolutely.
Bulk: RAM is cheap. Mass storage is cheap. CPUs can address very large amounts of RAM and easily access significant amounts of storage across multiple, very large storage devices. Even very large software can be loaded from mass storage very fast now.
The time to worry about executable size for desktop machines (and coming soonish for palmtops) is past, IMO. The remaining important goals to reach for these days are:
OPC, with the single exception of well-written and carefully validated public domain code, tends to impede those goals one way or another. And lawyers. Lawyers pollute everything they touch. Shakespeare's Dick the Butcher [wikipedia.org] had it exactly right, if for perhaps the wrong reasons, depending on how you read the intent.
--
Government: Designed to provide you with "service" and...
...the Media: Designed to provide you with Vaseline.
(Score: 2) by The Mighty Buzzard on Saturday July 04 2020, @02:59PM (4 children)
Okay, roll yourself a statically linked Linux distro. I'm curious to see how enormous it comes in at and if it'll even boot on a machine with less than 32GB of ram.
My rights don't end where your fear begins.
(Score: 0) by Anonymous Coward on Sunday July 05 2020, @01:46AM (3 children)
This made me do the math on one of our test machines. Based on the output of lsof, grepping for "\.so", stating the results and multiplying by the allocation units, the resulting disk space used by shared libraries is 256,081,920 bytes allocated on disk and 31,531,820 bytes of memory. And this is for a machine that is basically running one service and 25 processes total including the counting process, ttys, and ssh or system ones. It'd probably be interesting to see how much bigger that would be on a machine running X/Wayland or general purpose. However, the various elimination optimizations would cut down on some of the needed space.
(Score: 2) by The Mighty Buzzard on Sunday July 05 2020, @10:46AM (2 children)
du -hs reports 698MB for /lib on this box. How much each binary would have added without shared libraries is another matter entirely though.
My rights don't end where your fear begins.
(Score: 0) by Anonymous Coward on Monday July 06 2020, @05:12AM (1 child)
To be clear count I provided would be the total counting all repeated libraries multiple times to somewhat simulate them being included in each static executable based on lsof output. I also understand that the optimizers would remove unused parts of the code for each executable. Your comment just made me curious as to what some sort of ballpark might be on even a stripped down system. Perhaps if I am bored one day, I'll put the static USE flag and the static* flags in the build chain on one of the systems to see what comes out. I'll then stand there staring at the screen like Oppenheimer wondering what sort of monster I have unleashed.
I do find it somewhat interesting that the du -hs on the box is 83MB, just under 12% of your size. And most of that is kernel modules. I'm assuming you box has X11, generic kernels, and other incidental software on it as well, which means your /usr/lib and /usr/*/lib probably add up to a lot more than the 166 MB on the test machine here.
(Score: 2) by The Mighty Buzzard on Monday July 06 2020, @01:51PM
Daily driver desktop box, yeah. Can't cut out as much if you want to be able to hook random kit up without having to rebuild the kernel every time.
My rights don't end where your fear begins.
(Score: 2) by Pino P on Sunday July 05 2020, @02:24AM
Cellular data transfer quota to redownload the package after a security update is not cheap. Nor is mass storage cheap if it is already full and soldered to the mainboard of the paid-for appliance or phone. People are still using 8 GB Android phones where more than half the space is taken by the operating system and two copies of each application: the version that came with the operating system (for use after a factory reset) and the updated version downloaded from the device's app store. And the operating system tends to be incapable of moving a lot of applications to removable storage.
(Score: 2) by darkfeline on Friday July 03 2020, @11:01PM (2 children)
I find the first half of your post reasonable and the second half absurd or ignorant. Go is not only statically linked, but all dependencies are pinned via checksum and minimum specified version (rather than maximum specified version like many language dependency managers) and Go provides both global and private checksum servers and package proxies. It is the most reproducible and privately controllable language dependency manager that I know of, other than vendoring all code directly.
Join the SDF Public Access UNIX System today!
(Score: 2) by HiThere on Saturday July 04 2020, @02:05PM (1 child)
The checksum is good. Minimum specified version is not. It requires that you trust later versions without knowing anything about them. Which is it?
Yes, I'm relatively ignorant of go. There are lots of languages out there, and I'm not expert in most of them, perhaps any of them depending on what you mean by expert. I read the go documentation and decided that code that was written was too subject to change by people supplying the libraries. Perhaps I was wrong, but a minimum specified version is *not* reassuring. Later versions being malicious is one of the things that have gotten Javascript programs in trouble. Also, I believe, Ruby. IIRC, it once even bit Debian, though they vet the changes pretty strictly. You need to be able to specify the exact version of the library AND the checksum to be secure. Any changes need the same testing as the original version. Well, that's a bit extreme, but it's close to correct. You need a good suite of tests, and changed libraries need to pass those tests without change (including without fixing any mistakes that your test answers depend on).
There are decent arguments that all code that uses non-system libraries should be run inside of a jail, but usually need for security isn't strong enough to justify that.
All that said, this is not an ideal world, and one must make do. But there's no reason to make things worse by increasing remote dependencies. Sometimes you must, but for that to be reasonable you need to ensure that they won't change in the future. There are clear advantages to dynamic linking, but they come at the cost of lessened security. When it's links to system libraries one can say "well, if the system is 0wn3d, then there's nothing one can do", but the same doesn't apply to remote code.
Javascript is what you use to allow unknown third parties to run software you have no idea about on your computer.
(Score: 2) by darkfeline on Saturday July 04 2020, @11:18PM
>Minimum specified version is not. It requires that you trust later versions without knowing anything about them.
Minimum specified version means it uses the oldest version that satisfies the requirements. It does the exact opposite of what you seem to be talking about. Most dep managers like pip will use use the latest version available if you specify >=2.1. If you specify "require 2.1", Go will use the minimum specified version, i.e., 2.1, unless either your package or another explicitly specifies a later version, in which case Go will use the minimum specified version that satisfies that requirement.
Join the SDF Public Access UNIX System today!
(Score: 2, Insightful) by Anonymous Coward on Friday July 03 2020, @05:39PM
But then again, the good news is probably no set of eyes looking for the exploits. When yours is the only project in all the wide Internet, that uses a specific bit of code, it need be a pretty juicy target to recoup the time spent hacking it. With the plethora of low-hanging fruit everywhere, yours being singled out is quite unlikely without a VERY good reason.
(Score: 3, Informative) by driverless on Friday July 03 2020, @01:12PM
That's why djb writes all of his own library routines, he replaces whatever the system provides with djb-created minimalist secure ones. Downside is that you then get a slightly different djbmemcpy() and djbprintf() in every bit of code of his that you use.
(Score: 3, Insightful) by The Mighty Buzzard on Friday July 03 2020, @03:36PM (5 children)
All of the above assumes you're A) Capable of writing every library you need without adding your own bugs or huge performance degradations and B) Have the time to do so. I dunno about you but I'm just not interested in reading up enough to competently write my own Vulkan, H.265, NVIDIA driver, or PCRE implementation. I don't even really care to write my own database or stdio libs. I'd much rather be writing the bits that are the actual purpose of the project in the first place.
Which is to say, this kind of thing is fine for projects with trivial functionality, enormous manpower, or extremely constrained requirements but ludicrous for most anything else.
And this is coming from a guy who's been writing zero-library code for industrial equipment as his latest "How little work can I do and still afford fishing gear?" project.
My rights don't end where your fear begins.
(Score: 2) by RS3 on Friday July 03 2020, @06:40PM (2 children)
All good points. I broke the rules and read TFA, and my comments in this discussion, and maybe bradley13's too, are in context of the problem areas:
ftfa:
I don't think TFA refers to drivers, or really anything else written in c or compiled languages, stdio, etc. They mentioned php and Python as having some of the problem, but very low percentage so far.
Oddly, they made no mention of perl... :)
(Score: 3, Insightful) by The Mighty Buzzard on Saturday July 04 2020, @04:20AM (1 child)
That's because Perl guys are all old and wise enough to spot someone spouting ideology instead of good sense. There are plenty of libraries that I use on a regular basis simply because I'm not willing to spend twice as long coding a single library as I do coding the core application. There's just no way I'm writing an entire IRC library from scratch every time I decide to rewrite my bot MrPlow in a new language to (re)familiarize myself with it. It's not conscientious, it's bloody stupid.
My rights don't end where your fear begins.
(Score: 0) by Anonymous Coward on Saturday July 04 2020, @07:23AM
One thing Perl has is thousands of insecure libraries that no one really looks at. But they also have thousands of secure packages that are well-vetted and well-used. Instead of a fractured NIH-RTW ecology, there are a good number of modules on CPAN and elsewhere that are a de facto standard library. Everyone uses them for a particular problem, they are simultaneously stable and maintained, and professionals of various beard length watch over them rather than rolling their own.
(Score: 2) by Common Joe on Saturday July 04 2020, @10:34AM (1 child)
I had to drop a "insightful" point upon you because you're right on the money.
I would add that frameworks -- a key cornerstone to a lot of applications -- depend on many libraries too.
(Score: 2) by The Mighty Buzzard on Sunday July 05 2020, @10:58AM
It honestly never even occurred to me not to just write the code for most things frameworks bring to the table. I mean, it's not like keystrokes are what takes up most of a coder's time and most of the shit frameworks do is stuff I could code with a wicked hangover and a bunch of noisy fuckers in the room.
My rights don't end where your fear begins.
(Score: 0) by Anonymous Coward on Friday July 03 2020, @06:17PM
Yeah, this. It's been a while since I've looked into these things, but it seems like everybody's C library has its own way to handle strings to make up for C's issues. I bet there are hundreds of string libraries running on my machine, all accomplishing the same thing in subtly different ways.
People blame C for that, but I think it's human nature.
So you're going to solve this problem by rolling your own, eh?
You know what? The developers of the library you pulled in had the same problem. They solved it by rolling their own. Now their solution is your problem.
Aside from some dictator forcing us to use the One True Library for any given functionality, this doesn't seem like a problem that can be fixed, and I wouldn't want it to be fixed that way. The shark infested waters are where innovation happens. Some of it is just redundant; but some of it is progress. We're still in the early days of computing.