Michael Feathers defines the qualities of a good unit test as: “they run fast, they help us localize problems.” This can be hard to accomplish when your code accesses a database, hits another server, is time-dependent, etc.
By substituting custom objects for some of your module's dependencies, you can thoroughly test your code, increase your coverage, and still run in less than a second. You can even simulate rare scenarios like database failures and test your error handling code.
A variety of different terms are used to refer to these “custom objects”. In an effort to clarify the vocabulary, Gerard Meszaros provides the following definitions:
For example, to test a simple method like getIdPrefix() in the IdGetter class:
public class IdGetter { // Constructor omitted. public String getIdPrefix() { try { String s = db.selectString("select id from foo"); return s.substring(0, 5); } catch (SQLException e) { return ""; } }}
You could write:
db.execute("create table foo (id varchar(40))"); // db created in setUp(). db.execute("insert into foo (id) values ('hello world!')"); IdGetter getter = new IdGetter(db); assertEquals("hello", getter.getIdPrefix());
The test above works but takes a relatively long time to run (network access), can be unreliable (db machine might be down), and makes it hard to test for errors. You can avoid these pitfalls by using stubs:
public class StubDbThatReturnsId extends Database { public String selectString(String query) { return "hello world"; } } public class StubDbThatFails extends Database { public String selectString(String query) throws SQLException { throw new SQLException("Fake DB failure"); } } public void testReturnsFirstFiveCharsOfId() throws Exception { IdGetter getter = new IdGetter(new StubDbThatReturnsId()); assertEquals("hello", getter.getIdPrefix()); } public void testReturnsEmptyStringIfIdNotFound() throws Exception { IdGetter getter = new IdGetter(new StubDbThatFails()); assertEquals("", getter.getIdPrefix()); }
Remember to download this episode of Testing on the Toilet and post it in your office.
setTimeout()
jsUnit
Clock.tick()
function showProgress(status) { status.message = "Loading"; for (var time = 1000; time <= 3000; time += 1000) { // Append a '.' to the message every second for 3 secs. setTimeout(function() { status.message += "."; }, time); } setTimeout(function() { // Special case for the 4th second. status.message = "Done"; }, 4000);}
function testUpdatesStatusMessageOverFourSeconds() { Clock.reset(); // Clear any existing timeout functions on the event queue. var status = {}; showProgress(status); // Call our function. assertEquals("Loading", status.message); Clock.tick(2000); // Call any functions on the event queue that have // been scheduled for the first two seconds. assertEquals("Loading..", status.message); Clock.tick(2000); // Same thing again, for the next two seconds. assertEquals("Done", status.message);}
Clock
setInterval()
clearTimeout()
clearInterval()
jsUnitMockTimeout.js
jsUnitCore.js
Software testing is tough. It can be exhausting, and there is rarely enough time to find all the important bugs. Wouldn't it be nice to have a staff of tireless servants working day and night to make you look good? Well, those days are here. On Thursday, March 22, I'll give a lunchtime presentation titled "How to Build Your Own Robot Army" for the Quality Assurance SIG of the Software Association of Oregon.
Two decades ago, machine time was expensive, so test suites had to run as quickly and efficiently as possible. Today, CPUs are cheap, so it becomes reasonable to move test creation to the shoulders of a test machine army. But we're not talking about the run-of-the-mill automated scripts that only do what you explicitly told them. We're talking about programs that create and execute tests you never thought of to find bugs you never dreamed of. From Orcs to Zergs to Droids to Cyborgs, this presentation will show how to create a robot test army using tools lying around on the Web. Most importantly, it will cover how to take appropriate credit for your army's work!
The first-ever industry Developer-Tester/Tester-Developer Summit was held at the Mountain View Googleplex on Saturday, February 24th. Hosted by Elisabeth Hendrickson and Chris McMahon, the all-day workshop consisted of experience reports and lightning talks including:
Al Snow - Form Letter Generator Technique
Chris McMahon – Emulating User Actions in Random and Deterministic Modes
Dave Liebreich – Test Mozilla
David Martinez – Tk-Acceptance
Dave W. Smith – System Effects of Slow Tests
Harry Robinson – Exploratory Automation
Jason Reid – Not Trusting Your Developers
Jeff Brown – MBUnit
Jeff Fry – Generating Methods on the Fly
Keith Ray – ckr_spec
Kurman Karabukaev – Whitebox testing using Watir
Mark Striebeck – How to Get Developers and Tester to Work Closer Together
Sergio Pinon – UI testing + Cruise Control
There were also brainstorming exercises and discussions on the benefits that DT/TDs can bring to organizations and the challenges they face. Several participants have blogged about the Summit. The discussions continue at http://groups.google.com/group/td-dt-discuss.
If you spend your days coding and testing, try this opening exercise from the Summit. Imagine that:
is a spectrum that has "Tester" at one end and "Developer" at the other. Where would you put yourself, and why?
Posted by Allen Hutchison, Engineering Manager
