I am really astonished by the capabilities of static code analysis. The tool surprised me the other day as it turned out to be smarter and more attentive than I am. I found I must be careful when working with static analysis tools. Code reported by the analyzer often looks fine and I'm tempted to discard the warning as a false positive and move on. I fell into this trap and failed to spot bugs...Even I, one of the PVS-Studio developers.
So, appreciate and use static code analyzers! They will help save your time and nerve cells.
[Ed note: I debated running this story as there was an element of self-promotion (aka Bin Spam), but the submitter has been with the site for a while and has posted informative comments. Besides, I know there have been far too many times when I've seen a compiler complain about some section of my code and I'm thinking there is nothing wrong with it — and then I, finally, see my mistake. Anyone have samples of code where you just knew the compiler or static analyzer was wrong, only to find out otherwise? --martyb]
(Score: 5, Insightful) by TheRaven on Monday October 23 2017, @08:23AM (6 children)
I disagree. Static analysis should be something that you integrate into your development flow and run even before you run a test binary. It doesn't catch all of the bugs, but bugs caught by static analysis are typically the easiest to fix. First fix any compiler warnings, then fix any static analyser warnings, then fix test failures, then generate more tests with a fuzzer and fix the failing ones. That gives you a workflow where each round of fixes has an increasing effort:reward ratio.
sudo mod me up
(Score: 4, Informative) by tibman on Monday October 23 2017, @04:24PM (4 children)
I'd rather have the test suite. Then when people are "fixing the issues" they aren't introducing logic bugs. Tests to check for bad input are nice and all but the real value is in proving (and protecting) the business logic. My opinion of course : )
SN won't survive on lurkers alone. Write comments.
(Score: 2) by TheRaven on Monday October 23 2017, @04:54PM (2 children)
sudo mod me up
(Score: 0) by Anonymous Coward on Monday October 23 2017, @05:20PM (1 child)
You need a better test suite than that. If you haven't already spent 10x the cost of PVS studio developing your test suite, your priorities need a rethink.
Static analysis of the sort this tool does is just a bullshit lint pass, supported by the agglomeration of various C++ updates. They shit on other languages because they don't have a product for them. They don't have a product because the compilers already ship a comprehensive lint pass.
(Score: 2) by TheRaven on Tuesday October 24 2017, @08:47AM
That's my point. The cost of developing a good test suite is high. The cost of running a static analyser (particularly something like the Clang analyser, rather than something like PVS, or Coverity which is free for open source projects) is very cheap. The bug reports that you get from a static analyser are much easier to fix, because they already show you the chain of flow control that leads to the possible failure case, whereas failing tests just tell you that something is broken and require developer time to track down exactly why it's broken.
The correct time to run a static analyser is early on in development, more or less the same time as fixing compiler warnings. They give you the low-hanging fruit. If static analysis can find a bug, it can do so with a more helpful report and with a lot less effort than either a robust test suite or a fuzzing tool. If you have finite resources, that leaves you with more time to find the difficult bugs with a test suite.
sudo mod me up
(Score: 3, Informative) by JoeMerchant on Monday October 23 2017, @09:43PM
I prefer to have the requirements for the business logic written in something other than vague non-techie speak before embarking upon a coding up a test suite.
Unfortunately, it seems in my world that the only way to get the vague non-techies to stop with their self-contradictory waffle-speak is to show them a running prototype.
🌻🌻🌻 [google.com]
(Score: 2) by driverless on Tuesday October 24 2017, @07:46AM
The problem with static analysers is that they fall into two classes, really good and commercial, or somewhat basic and free. Then the commercial ones fall into three price classes, fucking expensive, outrageously expensive, and eyewateringly expensive.
PREfast is the closest you can get to free + reasonably good, but it suffers from way too many FPs and the need to annotate your code extensively to get decent results. Oh, and it's Windows-only.