googletesting

Survival techniques for acceptance tests of web applications (Part 1)

Tuesday, April 28, 2009

By Julian HartyHow to create acceptance testsAcceptance tests must meet the needs of several groups, including the users and the developers. Long-lived tests must be written in the language of each group, using terms users will recognize and a programming language and style in which the developers are competent.
We create tests by modelling the purpose of a test from the user’s perspective: send a message, order a book, etc. Each test is decomposed into individual actions: to send a message, a user must be logged in, select the compose message icon, specify one or more recipients, type a minimum of either a subject or a message, then select Send. From this list of actions, create a skeleton in the programming language of choice and create a method name that reflects each action. Show these to both the users and programmers and ask them to tell you what they think each step represents. Now is a great time to refine the names and decide which methods are appropriate: before you’ve invested too much time in the work. If you wait until later, your natural protective instincts will make it harder for you to accept good suggestions and make useful changes.
For each method, we need to work out how to implement it in code. How could an automated test select the compose message icon? Do alternative ways exist? An understanding of HTML, CSS, and JavaScript will help you if you plan to use browser automation tools. All the visible elements of a web application are reflected in the Document Object Model (DOM) in HTML, and they can be addressed in various ways: the directions from the root of the document to the element using xpath; unique identifiers; or characteristics possessed by the elements, such as class names, attributes, or link text. Some examples of these addressing options are shown in the Navigation Options illustration below. (Notes: navigation using xpath is much slower than using IDs; and IDs should be unique.)

Some actions can be initiated using JavaScript running in the browser. For devices such as the iPhone, changes in orientation when the phone is rotated are triggered this way (see Handling Orientation Events in the Safari Reference Library).
Typically, automated web application tests use JavaScript, either directly or indirectly, to interact with the web application being tested.
Utilities such as recording tools can help reduce the effort required to discover how to interact with the web application. The open-source test automation tool Selenium (http://seleniumhq.org/) includes a simple IDE record and playback tool that runs in the Firefox browser. Recorded scripts can help bootstrap your automated tests. However, don’t be tempted to consider the recorded scripts as automated tests: they’re unlikely to be useful for long. Instead, plan to design and implement your test code properly, using good software design techniques. Read on to learn how to use the PageObject design pattern to design your test code.
Two of the tools I find most useful are Firebug (http://getfirebug.com/), a Swiss Army knife for the Web Browser, and Wireshark (http://www.wireshark.org/), a network protocol analysis tool with a distinguished pedigree. Firebug is extremely useful when learning how to interact with a web application or debug mysterious problems with your tests when they seem to be misbehaving. I encourage you to persist when learning to use these tools – it took me a while to get used to their foibles, but I wouldn’t be without either of them these days.Homogenous languages and toolsSeveral years of experience across multiple project teams have taught us that the tests are more likely to survive when they’re familiar and close to the developers. Use their programming language, put them in their codebase, use their test automation framework (and even their operating system). We need to reduce the effort of maintaining the tests to a minimum. Get the developers to review the automated tests (whether they write them or you do) and actively involve them when designing and implementing the tests.
Typically, our acceptance tests use the xUnit framework; for example, JUnit for Java projects (see http://www.junit.org/). A good source of inspiration for creating effective tests is Gerard Meszaros’ work (see http://www.xunitpatterns.com).
Effective test designsBy using effective test designs, we can make tests easier to implement and maintain. The initial investment is minor compared to the benefits. One of my favourite designs is called Page Objects (see PageObjects on the Google Code site). A PageObject represents part or all of a page in a web application – something a user would interact with. A PageObject provides services to your test automation scripts and encapsulates the nitty-gritty details of how these services are performed. By encapsulating the nitty-gritty stuff, many changes to the web application, such as the reordering or renaming of elements, can be reflected in one place in your tests. A well-designed PageObject separates the ‘what’ from the ‘how’.givenwhenthen
// Given I have a valid user account and am at the login page,

// When I enter the account details and select the Enter button,

// Then I expect the inbox to be displayed with the most recent email selected.
The previous code consists of three programming comments that are easy for users to read. The actual programming code is entered immediately below each comment. Programming concepts such as literate programming are intended to make the code almost as readable as the textual comments.
Isolate things that change from those that don’t. For example, separate user account data from your test code. The separation makes changes easier, faster, and safer to implement, compared to making updates in the code for each test.Gaining SkillsWriting automated tests may be easy for some of you. In my case, I started with some simple example tests and tweaked them to suit my needs. I received boosts from working with more experienced practitioners who were able to correct my course and educate me in how to use various tools effectively. I recommend pairing with one of the developers of the software to be tested when you face a new testing requirement. Their intimate knowledge of the code and your understanding of the tests can form a potent combination. For instance, by working with one of the developers on a recent project, we were able to implement bi-directional injection of JSON messages and capture the responses from the server to test a key interaction between the server and client that was causing problems in production.
I encourage you to try out examples, tweak them, experiment, and plunge in to writing your first automated tests. Learn about AJAX – it underpins the web applications. And learn from more experienced practitioners – I’ve added some links at the end of the article to some of the people I respect who write great acceptance tests, including Antony Marcano and Alan Richardson.
Part 2 of this series helps you create more specialized tests (for example, to emulate mobile web browsers) and gives advice on how to increase the utility and effectiveness of your tests.

Further Information

Intermediate work products
‘The intermediate work products have only one real purpose in life: ‘‘to help the team make their next move’’.’ ‘An intermediate work product might be measured for ‘‘sufficiency” — was it sufficient to remind, inform or inspire? Any amount of effort or detail beyond sufficiency is extraneous to the purpose of the team and the purpose of the work product.’ Cooperative game manifesto for software development (Alistair Cockburn)
Cooperative game manifesto for software development at http://alistair.cockburn.us.

JUnit info

JUnit in Action, available from Manning Publications Co. (2nd edition, early access or 1st edition)

JUnit Recipes, by J. B. Rainsberger with Scott Stirling, available from Manning Publications Co.

Firebug info

Introduction to Firebug on Estelle Weyl’s blog, "CSS, JavaScript and XHTML Explained"

Firebug tutorials in the Firebug Archive at Michael Sync's blog

Fun with Firebug Tutorial on the Google Code site

WebDriver info

webdriver on the Google Code site

AJAX resources
Bulletproof Ajax—An incredibly good book on how to write good AJAX code. It starts with the basics and builds reliably and clearly from good foundations. The DOM manipulation code is relevant for implementing your acceptance tests in tools such as WebDriver.
Building a web site with Ajax —Again, a book that starts simple and builds a simple application step by step.
Acceptance tests are more A+S than T+G (Antony Marcano, in his blog at testingReflections.com)
A+S => Activities + Specific
T+G => Tasks + General
Alan Richardson: any and everything. For example, see:
A generalised model for User Acceptance Testing and
A little abstraction when testing software with Selenium-RC and Java, both at the Evil Tester blog

Beyond Testing: Becoming Part of the Innovation Machine

Tuesday, April 14, 2009

As you may have noticed, we've taken a short hiatus from posting. But all is not lost...we have been building a group of robots that will automatically write witty testing articles for us. We are still tweaking our algorithms for verbosity, humor, and technical detail. Once we are done, the posting will continue. In the mean time, we are sending one of our cyborgs to do a keynote at Star East 2009. Here's the topic of the keynote if you are interested...

Testing, once a marginalized function at Google, is now an integral part of Google’s innovation machine. Patrick Copeland describes how this vital transformation took place. As he was analyzing how to be more efficient and better align his testing team with the needs of the company, Patrick realized they had to move beyond “just testing” and become a force for change. His approach was based on five powerful principles: (1) Building feature factories rather than products, (2) Embedding testing at the grass roots level, (3) Solving problems of scale with technology, (4) Automating at the right level of abstraction, (5) Only doing what the team can do well. Learn how Google test teams used these principles to shift from a “service group” composed predominantly of exploratory testers to an “engineering group” with technical skills. Their focus became “delivering innovation” rather than “testing product.” Learn how Patrick led a cultural shift where product teams saw testing and continuous improvement, not as alien concepts driven by someone else, but as a tool for them to meet their own goals of delivering features quickly and with fewer problems. Discover how you can incorporate the lessons of Google to make your test team a vital force for change.

Fast exploratory tests with IFrames

Friday, February 20, 2009

by Håvard Rast BlokWhile working with a search quality development team, I was asked to collect information from their result pages across all the supported languages. The aim was to quickly get an overview, and then manually look through them for irregularities. One option would have been to grab the pages using tools like Selenium or WebDriver. However, this would have been complex and expensive. Instead, I opted for a much simpler solution: Display each language variation in a separate IFrame within the same HTML page.An example of how this looks, can be seen here, where the Google Search page is shown in 43 different languages. There are also some links for other Google sites, or you can type in your own link, and the query attribute for language, "hl=", will be appended at the end.(Warning: Do not try this with YouTube, as 43 Flash windows on the same pages will crash your browser. Also, Firefox 2 is known to be slow, while Firefox 3 works fine.)The JavaScript CodeCreating the IFrames is easy using JavaScript, as can be seen in the example below. I assume that the array of languages to iterate over is retrieved by the function getLanguages(). Then a simple loop uses document.write(...) to dynamically add the IFrames. It is worth mentioning that this method seemed to be the best way of dynamically creating them; using the document.createElement(...) resulted in some complex race condition issues when adding the IFrames and their content at the same time.var languages = getLanguages();
for (var lang, i = 0; lang = languages[i]; i++) {
  document.write('<hr><a name="' + lang + '"/>' +
                 '<h2>' + lang + '</h2><center>' +
                 '<iframe src="' + url + lang +
                 '" width="' + queryMap['width'] +
                 '" height="' + queryMap['height'] +
                 '"></iframe>' +
                 </center><br/><br/>');
}
The rest of the source code, can be seen in this example. Nothing else is needed to get the overview.ConclusionThe example in this article shows that a very simple and inexpensive solution can be useful for exploratory testing of web pages; especially when quickly looking over the same pages in multiple languages. The small amount of code required, makes it easy to customize for any project or page, and the fact that the requests are done dynamically, gives a view which is always up to date.Of course, this type of overview lends itself best to very simple stateless pages, which do not require complex navigation. It would for example be more difficult to get the same list of IFrames for Gmail, or other complex applications. Also, as the IFrames are loaded dynamically, no history is kept, so tracking when a potential bug was introduced in the page under test might prove more tedious.Furthermore, it should be noted that the overview only simplifies a manual process of looking at the pages. In some situations, this might be very beneficial, and enough for the developer, while in other projects more automated tests might be designed. E.g., it could be difficult to automate tests for aesthetic issues, but easy to spot them manually, while it may prove more beneficial to automate checks for English terms in other languages.Finally, a word on the issue of translations and language skills. The overview in this example, quickly highlights issues like incorrect line wrapping, missing strings, etc. in all variations of the page. Also, it was easy to spot strings not already translated in some of the languages, like Japanese, and in fact I reported a bug against the Search front page for this. However, for other issues, more language specific skills are necessary to spot and file bugs: E.g. should the Arabic page show Eastern or Western Arabic numerals? And have the Danes picked the English term for "Blogs", while the Norwegians and Swedish prefer a localized term? I don't know.

TotT: Partial Mocks using Forwarding Objects

Thursday, February 19, 2009

Partial Mocksome behavior from a real objectsome from a mock objectForwarding Objectforwards calls to a delegateinterface Duck {
Point getLocation();
void quack();
void swimTo(Point p);
}

class ForwardingDuck implements Duck {
private final Duck d;
ForwardingDuck(Duck delegate) {
this.d = delegate;
}
public Point getLocation() {
return d.getLocation();
}
public void quack() {
d.quack();
}
public void swimTo(Point p) {
d.swimTo(p);
}
}
OlympicDuckpublic void testDuckCrossesPoolAndQuacks() {
final Duck mock = EasyMock.createStrictMock(Duck.class);
mock.swimTo(FAR_SIDE);
mock.quack(); // quack after the race
EasyMock.replay(mock);
Duck duck = OlympicDuck.createInstance();
Duck partialDuck = new ForwardingDuck(duck) {
@Override public void quack() {
mock.quack();
}
@Override public void swimTo(Point p) {
mock.swimTo(p);
super.swimTo(p);
}
// no need to @Override “Point getLocation()”
}

OlympicSwimmingEvent.createEventForDucks()
.withDistance(ONE_LENGTH)
.sponsoredBy(QUACKERS_CRACKERS)
.addParticipant(partialDuck)
.doRace();
MatcherAssert.assertThat(duck.getLocation(), is(FAR_SIDE));
EasyMock.verify(mock);

partialDuck

quack() calls the mock object. It verifies that the duck doesn't promote the sponsor (by quacking) until after the race. (We skip the real quack() method so that our continuous build doesn't drive us crazy.)

getLocation() calls the real object. It allows us to use the OlympicDuck's location logic instead of rewriting/simulating the logic from that implementation.

swimTo(point) calls both objects. It allows us to verify the call to the real duck before executing it.

submitToDrugTest()Partial Mock in testslimit, simulate or verify method callsthis episode

Constructor Injection vs. Setter Injection

Thursday, February 19, 2009

Miško HeverySetter-InjectionConstructor-Injectioncircular dependenciesCreditCardProcessor processor = new CreditCardProcessor();OfflineQueue queue = new OfflineQueue();
CreditCardProcessor processor = new CreditCardProcessor();
processor.setOfflineQueue(queue);Database db = new Database();
OfflineQueue queue = new OfflineQueue();
queue.setDatabase(db);
CreditCardProcessor processor = new CreditCardProcessor();
processor.setOfflineQueue(queue);
processor.setDatabase(db);Database db = new Database();
db.setUsername("username");
db.setPassword("password");
db.setUrl("jdbc:....");
OfflineQueue queue = new OfflineQueue();
queue.setDatabase(db);
CreditCardProcessor processor = new CreditCardProcessor();
processor.setOfflineQueue(queue);
processor.setDatabase(db);CreditCardProcessor processor = new CreditCardProcessor(?queue?, ?db?);Database db = new Database("username", "password", "jdbc:....");
OfflineQueue queue = new OfflineQueue(db);
CreditCardProcessor processor = new CreditCardProcessor(queue, db);Model-View-Controller

To Assert or Not To Assert

Monday, February 9, 2009

Miško Heveryclass House {
Door door;
Window window;
Roof roof;
Kitchen kitchen;
LivingRoom livingRoom;
BedRoom bedRoom;

House(Door door, Window window,
Roof roof, Kitchen kitchen,
LivingRoom livingRoom,
BedRoom bedRoom){
this.door = Assert.notNull(door);
this.window = Assert.notNull(window);
this.roof = Assert.notNull(roof);
this.kitchen = Assert.notNull(kitchen);
this.livingRoom = Assert.notNull(livingRoom);
this.bedRoom = Assert.notNull(bedRoom);
}

void secure() {
door.lock();
window.close();
}
}testSecureHouse() {
Door door = new Door();
Window window = new Window();
House house = new House(door, window,
null, null, null, null);

house.secure();

assertTrue(door.isLocked());
assertTrue(window.isClosed());
}testSecureHouse() {
Door door = new Door();
Window window = new Window();
House house = new House(door, window,
new Roof(),
new Kitchen(),
new LivingRoom(),
new BedRoom());

house.secure();

assertTrue(door.isLocked());
assertTrue(window.isClosed());
}testSecureHouse() {
Door door = new Door();
Window window = new Window();
House house = new House(door, window,
new Roof(),
new Kitchen(new Sink(new Pipes()),
new Refrigerator()),
new LivingRoom(new Table(), new TV(), new Sofa()),
new BedRoom(new Bed(), new Closet()));

house.secure();

assertTrue(door.isLocked());
assertTrue(window.isClosed());
}null

TotT: Be an MVP of GUI Testing

Thursday, February 5, 2009

In MVC, the presentation logic is shared by Controller and ViewIn MVP, presentation logic is taken over entirely by a Supervising Controller, also known as a PresenterMVC

MVP

public CongressionalHearingView() {
testimonyWidget.addModifyListener(
new ModifyListener() {
public void modifyText(ModifyEvent e) {
presenter.onModifyTestimony(); // presenter decides action to take
}});
}
public class CongressionalHearingPresenter {
public void onModifyTestimony() {
model.parseTestimony(view.getTestimonyText()); // manipulate model
}
public void setWitness(Witness w) {
view.setTestimonyText(w.getTestimony()); // update view
}
}
more modular codeenables easy unit testing of the Presenter and the Viewpublic void testSetWitness() {
spyView = new SpyCongressionalHearingView();
presenter = new CongressionalHearingPresenter(spyView);
presenter.setWitness(new Witness(“Mark McGwire”, “I didn't do it”));
assertEquals( “I didn't do it”, spyView.getTestimonyText());
}
this episode

Testing Blog

Survival techniques for acceptance tests of web applications (Part 1)

Beyond Testing: Becoming Part of the Innovation Machine

Fast exploratory tests with IFrames

TotT: Partial Mocks using Forwarding Objects

Constructor Injection vs. Setter Injection

To Assert or Not To Assert

TotT: Be an MVP of GUI Testing

Labels

Archive

Feed

Company-wide

Products

Developers