googletesting: Burning Test Questions at Google

Burning Test Questions at Google

Tuesday, June 9, 2009

One of the best parts about change is meeting new people and I've met a lot of Googlers in Mountain View and Kirkland over the past two weeks. There are many burning questions we've discussed but one has surfaced so much that it has to take top billing: manual vs. automated testing.
Google, I've learned, has come full circle on this issue. When the company was new and Search was king most testing was manual and like a lot of startups there was little focus on actual QA. In recent years the pendulum has swung to automation with developers writing a lot of unit tests and testers creating automation frameworks prolifically.
And now with my recent work on manual testing making the rounds what will throwing me into this mix produce?
Actually, I'd like to put the manual vs. automation debate to bed. Instead, I think the issue is test design versus doing something because we can. How much automation is written not because there is a clear need but because we can? Or, perhaps, because we think we must? Hmm.
Before you cry bias, how much manual testing is seat of the pants versus intentional, purposeful testing?
See the point? Whether you are talking about manual or automated testing, it's test design ... identifiying what needs testing and what is the best way to test it ... that has to take center stage. Too often we are solution focused and marry ourselves to our automation rather than the product we are testing. Whenever I see testers that seem more vested in their automation than in the product they are shipping I worry the pendulum has swung too far. Automation suffers from this because there is a product - the tool - that can become the object of obession. Manual testers don't really produce such a physical baby they can fuss over or they probably would fuss just as much as the automaters. Ignore the tool, focus on the problem it should solve!
Besides, I think fundamentally that manual and automated testing are irreversibly linked. All good automation begins it's life as manual test cases. And to create really good automation, you have to be good at manual testing. It's during manual testing that a tester gets good at test design, identifying important testing problems and crafting the solution approach. Any tester who isn't good at test design should think twice before they automate.
Let's shift the debate to good test design. Whether those tests ultimately end up being executed manually or via a tool is a moot point, let's just make them good ones that solve real testing problems.
How many octomoms does the testing world need anyway?