Code Coverage Goals
I haven't talked about it yet on this blog, but I am a big fan of unit testing. If you care about the quality of your system, all code should always be tested. I hear this excuse and that from my fellow developers about why certain code wasn't covered. Maybe they don't have time. Maybe they don't believe it is worth the hassle. Maybe it was "too hard". The bottom line is that, for whatever reason, unit testing just wasn't a priority.
Enter management. In their infinite wisdom, they decide to inspire developers to start writing unit tests by creating a performance metric based on code coverage. This will surely fix the problem, right? Now that the test coverage is tied to the year-end bonuses, the developers will definitely make writing them a priority!
Why Code Coverage Goals are Bad for Your System
You need to be very careful when you make performance metrics for code quality. If you make a goal around executing lines of code in unit tests, then that is exactly what you will get. Instead of writing tests based on the requirements or what the code is supposed to do, they will be based on the gaps in the coverage reports.
Developers will start executing code left and right but that doesn't mean that there were actually any tests or that any of this code is verified in any way. What it means is that it was executed. There is no guarantee that any asserts were included or meaningful. There is no guarantee that exceptions aren't caught and ignored. There is no guarantee whatsoever that any of that code was, in fact, tested in any way.
This is considerably worse than not having test coverage in the first place. At least when the code isn't covered, you know exactly what was tested and what wasn't. When everything is being executed, you no longer have any clue what is actually being verified. You are right back to square one when you had no testing whatsoever.
Why Code Coverage Goals are Bad for Your Staff
For starters, motivating people with money is easy, but not effective. If your employees work at your company for the money, then their loyalty is to the money and you will have a very tenuous relationship with them. Not only that, but when someone else offers them more money, they will jump ship in a heartbeat. Oh, and you don't have the best pay in town. You may think your package is "highly competitive", but there is always someone who will pay more than you do and you can rest assured that once your employees figure out who that is you will lose them.
I read a great post about this on the Venture Hacks blog called We don’t pay you to work here. It profiles a book called Hidden Value that goes into great depth about this. I would highly recommend you at least read their post on it. Ultimately it boils down to the fact that money only motivates people to quest for more money.
So What Should We Do?
The root of the problem is not that the system is not covered by unit tests, but that your developers don't believe in unit testing and hold it as priority #1. Installing monetary goals around coverage numbers will do little for your application and only serve to annoy your developers. If you need a measurable goal, a better one might be how many defects are found by your QA department and remind people that if the code is properly unit tested, the defect numbers will be considerably lower, if not zero.
The only solution to the problem, however, is to get your developers to believe that it is essential. Changing people's beliefs is a hard thing to do. The first step is to make it clear that it is important to the company and not just by saying it, but by acting on it.
My recommendation: check the project plan. It needs to be considered the flagship part of the development life cycle. If unit testing isn't a major part at the start of development, then you are sending a clear message that it isn't really that important to the company after all.
