I subscribe to dozens of tech blogs (including but not limited to Ruby and Rails), and although I’ve seen quite a lot of commentary about TDD and BDD, I’m not sold on either of them yet. TDD is interesting, but BDD seems like a waste of time. But I am completely sold on automated testing in general. I write lots and lots of tests and make sure using rcov that my tests cover all of the code.
Getting 100% coverage isn’t easy. In general it means you definitely can’t just write a single test case per method and declare victory when it passes. When the number of possible combinations of inputs (method arguments and/or mock objects) and expected outputs (return values, exceptions, and side effects) becomes large, then the potential for copy-and-paste errors in your test code becomes large, and legibility becomes an issue. This is the point at which I find the recent fascination with writing tests in a near-natural-language DSL to be a distraction. It’s orthogonal to the problem I’m dealing with, which is how to comprehensively test the code. In other words, the problem is not making a small number of tests more readable, but concisely expressing a large number of input/output combinations and a large number of different tests, and making it all readable.
I assume that there are others working on this problem too, so I’ll describe some of the things I’ve come up with, and hopefully if you have any good ideas you’ll post them in the comments section.
Continue reading “Techniques for Exhaustively Testing a Rails App”