Last friday,
Eric Jarvi blogged about the
Color scheme used by the code coverage tools of Visual Studio Team System. His post made me think about code coverage another time. Beyond the fact that people tend to misunderstand the green color, there is maybe a more general misunderstanding of code coverage. The code coverage rate can easily be overestimated; Badly covered code is of course also badly tested code, well tested code sure has a good coverage rate. But well covered code is not automatically well tested.
If coverage rate is at 100%, this means that every piece of code was executed
at least once. So with one pass, you get good coverage, but a good test requires more passes with different sets of input. To extensively cover your code, it is necessary to test it with valid data, invalid data, and make sure to have tested border cases. This sure requires more than only one pass and relativates the coverage rate as a means for rating your tests.
So maybe it's not only the green color that makes people think their covered code is "good" code - maybe it is also the overestimation of the coverage rate.