googletesting

Testing on the Toilet: Fake Your Way to Better Tests

Friday, June 28, 2013

By Jonathan Rockway and Andrew Trenk This article was adapted from a Google Testing on the Toilet (TotT) episode. You can download a printer-friendly version of this TotT episode and post it in your office. public void deletePostsWithTag(Tag tag) {
for (Post post : blogService.getAllPosts()) {
if (post.getTags().contains(tag)) { blogService.deletePost(post.getId()); }
}
}
A fake is a lightweight implementation of an API that behaves like the real implementation, but isn't suitable for productionpublic class FakeBlogService implements BlogService {
private final Set<Post> posts = new HashSet<Post>(); // Store posts in memory
public void addPost(Post post) { posts.add(post); }
public void deletePost(int id) {
for (Post post : posts) {
if (post.getId() == id) { posts.remove(post); return; }
}
throw new PostNotFoundException("No post with ID " + id);
}
public Set<Post> getAllPosts() { return posts; }
}
Fakes are useful for when you can't use the real implementation in a testeach fake should be created and maintained by the person or team that owns the real implementationRemember to create the fake at the lowest level possibleFakes should have their own tests

Optimal Logging

Friday, June 14, 2013

by Anthony ValloneChanneling Goldilocks

Important startup configuration

Errors

Warnings

Changes to persistent data

Requests and responses between major system components

Significant state changes

User interactions

Calls with a known risk of failure

Waits on conditions that could take measurable time to satisfy

Periodic progress during long-running tasks

Significant branch points of logic and conditions that led to the branch

Summaries of processing steps or events from high level functions - Avoid logging every step of a complex process in low-level functions.

Function entry - Don’t log a function entry unless it is significant or logged at the debug level.

Data within a loop - Avoid logging from many iterations of a loop. It is OK to log from iterations of small loops or to log periodically from large loops.

Content of large messages or files - Truncate or summarize the data in some way that will be useful to debugging.

Benign errors - Errors that are not really errors can confuse the log reader. This sometimes happens when exception handling is part of successful execution flow.

Repetitive errors - Do not repetitively log the same or similar error. This can quickly fill a log and hide the actual cause. Frequency of error types is best handled by monitoring. Logs only need to capture detail for some of those errors.

There is More Than One Level

Debug - verbose and only useful while developing and/or debugging.

Info - the most popular level.

Warning - strange or unexpected states that are acceptable.

Error - something went wrong, but the process can recover.

Critical - the process cannot recover, and it will shutdown or restart.

Production - Every level is enabled except debug. If something goes wrong in production, the logs should reveal the cause.

Development & Debug - While developing new code or trying to reproduce a production issue, enable all levels.

Test Logs Are Important Too

Test execution environment

Initial state

Setup steps

Test case steps

Interactions with the system

Expected results

Actual results

Teardown steps

Conditional Verbosity With Temporary Log QueuesFailures and Flakiness Are OpportunitiesMight As Well Log Performance Data

Significant system calls

Network requests

CPU intensive operations

Connected device interactions

Transactions

Following the Trail Through Many Threads and ProcessesMonitoring and Logging Complement Each Other

Testing on the Toilet: Don’t Overuse Mocks

Tuesday, May 28, 2013

By Andrew TrenkThis article was adapted from a Google Testing on the Toilet (TotT) episode. You can download a printer-friendly version of this TotT episode and post it in your office.it can seem easy to ignore your code's dependencies by mocking them outpublic void testCreditCardIsCharged() {
paymentProcessor = new PaymentProcessor(mockCreditCardServer);
when(mockCreditCardServer.isServerAvailable()).thenReturn(true);
when(mockCreditCardServer.beginTransaction()).thenReturn(mockTransactionManager);
when(mockTransactionManager.getTransaction()).thenReturn(transaction);
when(mockCreditCardServer.pay(transaction, creditCard, 500)).thenReturn(mockPayment);
when(mockPayment.isOverMaxBalance()).thenReturn(false);
paymentProcessor.processPayment(creditCard, Money.dollars(500));
verify(mockCreditCardServer).pay(transaction, creditCard, 500);
}not using mocks can sometimes result in tests that are simpler and more usefulpublic void testCreditCardIsCharged() {
paymentProcessor = new PaymentProcessor(creditCardServer);
paymentProcessor.processPayment(creditCard, Money.dollars(500));
assertEquals(500, creditCardServer.getMostRecentCharge(creditCard));
}Overusing mocks can cause several problems- Tests can be harder to understand.- Tests can be harder to maintain.- Tests can provide less assurance that your code is working properly.If you're trying to read a test that uses mocks and find yourself mentally stepping through the code being tested in order to understand the test, then you're probably overusing mocksSometimes you can't use a real dependency in a testbut there may better options than using mocksFor more information about using hermetic servers, see http://googletesting.blogspot.com/2012/10/hermetic-servers.html. Stay tuned for a future Testing on the Toilet episode about using fake implementations.

GTAC 2013 Wrap-up

Saturday, May 4, 2013

by The GTAC Committee
GTAC
slides, video recordingsphotosspeakersGoogle Testing Blog
“My first GTAC, and one of the best conferences of any kind I've ever been to. The talks were consistently great and the chance to interact with so many experts from all over the map was priceless.”Netflix“Adding my own thanks as a speaker (and consumer of the material, I learned a lot from the other speakers) -- this was amazingly well run, and had facilities that I've seen many larger conferences not provide. I got everything I wanted from attending and more!”Twitter“This was a wonderful conference. I learned so much in two days and met some great people. Can't wait to get back to Denver and use all this newly acquired knowledge!”Ping Identity“GTAC is hands down the smoothest conference/event I've attended. Well done to Google and all involved.”ThoughtWorks“Thanks and compliments for an amazingly brain activity spurring event. I returned very inspired. First day back at work and the first thing I am doing is looking into improving our build automation and speed (1 min is too long. We are not building that much, groovy is dynamic).”Zynx Health

GTAC is Almost Here!

Saturday, April 13, 2013

by The GTAC Committee

The Schedule

Speaker Profiles

Committee Profiles

Live Stream

Two New Videos About Testing at Google

Friday, April 12, 2013

Anthony ValloneLife at GoogleDo Cool Things That MatterMeet Yvette and Sean from the Google Maps Test TeamGoogle StudentsHangouts On AirSoftware Engineers in Test at Google - Covering your (Code)BasesGoogle test jobs

Testing on the Toilet: Testing State vs. Testing Interactions

Friday, March 22, 2013

By Andrew TrenkThis article was adapted from a Google Testing on the Toilet (TotT) episode. You can download a printer-friendly version of this TotT episode and post it in your office.Testing state means you're verifying that the code under test returns the right results.public void testSortNumbers() {
NumberSorter numberSorter = new NumberSorter(quicksort, bubbleSort);
// Verify that the returned list is sorted. It doesn't matter which sorting
// algorithm is used, as long as the right result is returned.
assertEquals(
new ArrayList(1, 2, 3),
numberSorter.sortNumbers(new ArrayList(3, 1, 2)));
}Testing interactions means you're verifying that the code under test calls certain methods properly.public void testSortNumbers_quicksortIsUsed() {
// Pass in mocks to the class and call the method under test.
NumberSorter numberSorter = new NumberSorter(mockQuicksort, mockBubbleSort);
numberSorter.sortNumbers(new ArrayList(3, 1, 2));
// Verify that numberSorter.sortNumbers() used quicksort. The test should
// fail if mockQuicksort.sort() is never called or if it's called with the
// wrong arguments (e.g. if mockBubbleSort is used to sort the numbers).
verify(mockQuicksort).sort(new ArrayList(3, 1, 2));
}Just because a test that uses interactions is passing doesn't mean the code is working properlyin most cases, you want to test state, not interactions.interactions should be tested when correctness doesn't just depend on what the code's output is, but also how the output is determinedWhat are some other examples of cases where you want to test interactions?The code under test calls a method where differences in the number or order of calls would cause undesired behaviorYou're testing a UI where the rendering details of the UI are abstracted away from the UI logic

Testing Blog

Testing on the Toilet: Fake Your Way to Better Tests

Optimal Logging

Testing on the Toilet: Don’t Overuse Mocks

GTAC 2013 Wrap-up

GTAC is Almost Here!

Two New Videos About Testing at Google

Testing on the Toilet: Testing State vs. Testing Interactions

Labels

Archive

Feed

Company-wide

Products

Developers