Editorial Note: I originally wrote this post for the NDepend blog. You can check out the original here, at the NDepend site. If you’re a fan of static analysis tools, do yourself a favor and take a look at the NDepend offering while you’re there.
A common complaint and source of resistance to the adoption of static analysis is the idea of false positives. And I can understand this. It requires only one case of running a tool on your codebase and seeing 27,834 warnings to color your view on such things forever.
There are any number of ways to react to such a state of affairs, though there are two that I commonly see. These are as follows.
- Sheepish, rueful acknowledgement: “yeah, we’re pretty hopeless…”
- Defensive, indignant resistance: “look, we’re not perfect, but any tool that says our code is this bad is nitpicking to an insane degree.”
In either case, the idea of false positives carries the day. With the first reaction, the team assumes that the tool’s results are, by and large, too advanced to be of interest. In the second case, the team assumes that the results are too fussy. In both of these, and in the case of other varying reactions as well, the tool is providing more information than the team wants or can use at the moment. “False positive” becomes less a matter of “the tool detected what it calls a violation but the tool is wrong” and more a matter of “the tool accurately detected a violation, but I don’t care right now.” (Of course, the former can happen as well, but the latter seems more frequently to serve as a barrier to adoption and what I’m interested in discussing today).
Is this a reason to skip the tool altogether? Of course not. And, when put that way, I doubt many people would argue that it is. But that doesn’t stop people form hesitating or procrastinating when it comes to adoption. After all, no one is thinking, “I want to add 27,834 things to the team’s to do list.” Nor should they — that’s clearly not a good outcome.
With that in mind, let’s take a look at some common sources of false positives and the ways to address them. How can you ratchet up the signal to noise ratio of a static analysis tool so that is valuable, rather than daunting?