Striving for 100 percent
August 22nd, 2007
If it wasn’t obvious from my last two posts I’ve been doing a lot of testing. In fact, I spent almost the entire day yesterday writing tests. This got me thinking about the value of testing.
With the testing I’ve done recently, I feel like I’ve uncovered several problems that would have made it through to production. This is good. Testing first forces you to think about what you’ll be developing before you ever develop it. I think this has helped me to write better code and, to me, this has value.
However, when I consider the quest for attaining 100% code coverage I start to waffle on the value of that activity. It seems like a blind generalization such as “we must have 100% code coverage” is neither practical nor valuable. When it comes to coverage, it seems that high profile functionality should be well tested, probably 100% covered. High profile functionality being functionality that is executed easily and often, functionality with high consequences, etc. But does it really make sense to spend a lot of time trying to get code coverage on something that will probably never happen and where, even if it does, it has little to no impact on the system? To me, this doesn’t seem like a valuable activity. If it’s easy to test, then go for it. But if it starts to become a time-sink then it might be time to think about whether you’re really adding value.
If testing should be as easy to do as not, then should 100% really be the goal? Is not striving for 100% a cop-out?
August 24th, 2007
07:13 PM
I would say the goal of testing is not to get a 100% badge. But at the same I would say if a function is easy to write, it should be easy to test. If it isn’t, was the function created really that easy. Were all the nuances taken into account? Could you be confident that it would work 100% of the time? And maybe a question to the customer would be what level of uncertainty do they risk by not having a good test suite? It’s something that I think over time it’s easier to see the value than right off the bat.
August 24th, 2007
07:42 PM
@heidmotron, thanks for the comment.
I agree that functions should be easy to test, but certain aspects of testing can be pretty tricky. This is especially true when dealing with negative path cases.
I also don’t necessarily equate 100% coverage with 100% tested, although that occurs sometimes. IMO, coverage should not really be used as the sole measure of quality. Coverage merely lets you know that a line of code was executed during a test. You’re not assured that line of code was executed with all permutations and therefore cannot be sure it works for any cases other than those tested, despite the results of your code coverage utility.
Support for mocks and stubs in Rspec is certainly making it easier to get good coverage, but there are still some tricky cases that take more time to get coverage for. Perhaps as I become more versed and seasoned with Rspec I will overcome those challenges more easily.