Almost daily, in some form or another, I find myself listening to things like "the code should have a very high test coverage", or "we should go for 85% test coverage", or "our test coverage % is very low". I tremble whenever I hear these sentences. I will try to summarize something that makes sense to me, but might not make sense to a lot of other people (namely the kind of people who say sentences like that): Test coverage percentage means shit. There, I said it, and it feels good to say something that my brain can relate to. Now to explain it, with a bit of background: Unit tests, like everybody knows, are meant to test small parts/modules/components/<insert_your_favorite_application_separation_designation_here> of the code in isolation. Then we have those that are usually called integration tests, that usually test the, hey, integration between these smaller modules. One thing in common with all types of tests is that (*duh*) they are supposed to make sure that the module/application behaves the way it was designed/thought of. Does this mean that making sure that every line of code is currently being tested? Hell no! Should we make sure that the data access layer retrieves data properly and fails properly? Absolutely! But does that mean that we should test, say, hibernate retrieving something from the database, when we know that hibernate - being a proper open source product - already contains it's own set of tests? I believe not. And this does not apply only to external, tested, libraries used by our applications; The point is also that, what matters in the end, is to make sure that the code does what it is supposed to do, not to make sure that line X in your code gets executed in the correct order (think easymock.expect(Y) here), or some other barbaric things done in the name of test coverage percentage.

Posted by email from nocivus (posterous)