I’ve seen a few Rake tasks for Rcov that work OK, but which fail in an interesting way (if you care about coverage): they give your coverage metrics an unexpected boost if you have 0% coverage in one or more source files.
Huh? Exactly. If you have 500 source files, and your test suite only requires one of them, then you get a free ride on those 499 files that have 0% coverage. Theoretically you could get 100% coverage in your report even though 499 source files are not touched at all. D’oh!
The reason for this is that rcov isn’t responsible for finding all of your source files. It just measures what portion of the files which you loaded were executed. The Rake tasks that people have written just kick off the test code, which also have no need to load all of your application’s source files. They just load whatever they need, and so that’s what rcov is aware of. But that’s not answering the question you thought you were asking, which is “how much of my application is being tested?”
You have to explicitly say that you want to see coverage numbers for all the files your tests need, plus all of the files that your tests did not touch. Put this in your test/test_helper.rb:
|
1 2 3 4 5 6 7 8 9 |
# require the entire app if we're running under coverage testing, # so we measure 0% covered files in the report coverage_testing_active = defined?(Rcov) if coverage_testing_active all_app_files = Dir.glob('{app,lib}/**/*.rb').grep( /^(?!lib\\/(scheduled_tasks|template_optimizer)\\/)/) all_app_files.unshift('app/controllers/application.rb') all_app_files.each{|rb| require rb} end |
You’ll probably want to customize that regexp to weed out any .rb files that you don’t want to load during the test suite (or which aren’t your own code).
Because of all this require-ing, you might want to apply the snippet in my prior post, which eliminates duplicate required source files if they’re under your RAILS_ROOT. Otherwise you may see evidence of repeated loading of the same source files.
With this snippet in place, you’ll still never see 0% coverage in a file. If there are zero lines of executable code in the file, it will not be listed. But you’ll see really low figures (4% etc.) for files that are loaded but not executed. Hopefully that will help you see where you’ve completely overlooked some code in your tests that users might still be able to get to.
Thank you! This is something that’s been bugging me about rcov for a while, but I just haven’t taken time to chase down. I look forward to putting this in place.