googletesting: August 2010

An Ingredients List for Testing - Part Two

Friday, August 27, 2010

never

An Ingredients List for Testing - Part One

Friday, August 20, 2010

Ingredient 1: Product expertise

Test Driven Code Review

Monday, August 2, 2010

By Philip Zembrod
In my quest to explore TDD I recently found another propery of TDD-written code that I hadn't expected: When reviewing or just reading such code, it's often best to first read the tests.
When I look at new code or a code change, I ask: What is this about? What is it supposed to do? Questions that tests often have a good answer for. They expose interfaces and state use cases. This is cool, I thought, and decided to establish test-first reading as my code-reviewing routine. Of course this just applies the specification aspect of tests: Reading the specs before reading the code.
Only it didn't always work. From some tests I just failed to learn the point and intention of the tested code. Often, though not always, these were tests that were heavy with mocks and mock expectations.
Mocks aren't always a helpful tool, was my first conclusion. The phrase "Good mocks, bad mocks" popped up in my mind. I began to appreciate fakes again - and the people who write them. But soon I realized that this was about more than mocks vs. fakes vs. dummies vs. other Friends You Can Depend On. I was really looking at how well tests fulfill their role as specification.
TDD teaches that tests are a better specification than prose. Tests are automatically enforced, and get stale less easily. But not all tests work equally well as specification! That's what test driven code reviewing taught me.
I began to call them well-specifying tests and poorly-specifying tests. And the specification aspect isn't just some additional benefit, it's a really crucial property of tests. The more I thought about it, the more I saw: It is connected to a lot of things that first weren't obvious to me:

If tests are poorly-specifying, then possibly the tested product is poorly specified or documented. After all, it's the tests that really make sure how a product behaves. If they don't clearly state what they test, then it's less clear how the product works. That's a problem.

Well-specifying tests are more robust. If a test just does and verifies things of which the architect or product manager will readily say "yes, we need that" then the test will survive refactorings or new features. Simply because "yes, we need that." The test's use case is needed, its conditions must hold. It needn't be adapted to new code, new code must pass it. False positives are less likely.

Corollary: Well-specifying tests have higher authority. If a test fails, a natural reaction is to ask "is this serious?" If a test is poorly-specifying, if you don't really understand what it is testing, then you may say "well, maybe it's nothing". And you may even be right! If a test is well-specifying, you'll easily see that its failing is serious. And you'll make sure the code gets fixed.
I'm now thinking about an authority rank between 0 and 1 as a property of tests. It could be used to augment test coverage metrics. Code that is just covered by poorly-specifying tests would have poor authority coverage, even if the coverage is high. Quantifying an authority rank would be a conceptual challenge, of course, but part of it could be how well test driven code reviewing works with a given test.
P.S. If anyone suspects that I'm having some fun inventing terms beginning with "test driven," I'll plead guilty as charged. :-)

Testing Blog

An Ingredients List for Testing - Part Two

An Ingredients List for Testing - Part One

Test Driven Code Review

Labels

Archive

Feed

Company-wide

Products

Developers