googletesting

Google is hiring SETs

Friday, March 12, 2010

Drop me a resume and a brief note if you're interested in joining Google as an SET. email: test.eng.hiring@gmail.com. Thanks -- Patrick Copeland
When we hire people we look for folks with a "testing DNA." These are people who are great computer scientists at their core, but also are very curious, love software, and are passionate about test engineering. People who have those characteristics tend to pursue challenges and continue to learn. Are you one of us? We have positions all over the US and the world.
What is a SET?At Google, Software Engineers in Test (SET) develop test frameworks and build robust, scalable, and effective tests. SETs spend a majority of their time coding in either C++, Java, or scripting in Python. A SET is a software engineer, a core developer, who has a passion for test engineering.
How is testing done differently at Google?

Literally within milliseconds of a code check-in, our build process will automatically select the appropriate tests to run based on dependency analysis, run those tests and report the results.
By reducing the window of opportunity for bad code to go unnoticed, overall debugging and bug isolation time is radically reduced. The net result is that the engineering teams no longer sink hours into debugging build problems and test failures.
Development teams write good tests because they care about the products, but also because they want more time to spend writing features and less on debugging.
Testing teams focus on higher abstractions, like identifying latencies, system or customer focused testing, and enabling the process with tools.
SETs avoid becoming codependents within this system and generally do not write unit tests or other activities that are best done by the developer.

More about SETs

Our SET’s spend time developing code to prevent bugs. Google has a strong cultural emphasis on developers improving quality (i.e. unit tests, code reviews, design reviews, root cause analysis). We want our engineers to spend their time innovating - not fixing bugs.
SETs enable products to launch faster. They have great influence over internal processes and how developers write code.
One of Google's less understood capabilities is our massive distributed computing environment. The testing groups exploit this infrastructure to do huge amounts of work very quickly and elegantly.
For someone who wants to learn and grow as an engineer, the uninhibited access to the entire code base is a unique opportunity.

Google @ ICST 2010

Saturday, March 6, 2010

pointer to the programlink to the abstract
Posted by Patrick Copeland

Still Stuck in the 90s

Monday, March 1, 2010

Flashback. It's 1990. Chances are you do not own a cell phone. And if you do it weighs more than a full sized laptop does now. You certainly have no iPod. The music in your car comes from the one or two local radio stations that play songs you can tolerate and a glove box full of CDs and cassettes. Yes, I said cassettes...you know the ones next to those paper road maps. Music on the go? We carried our boom boxes on our shoulder back then.
If you are a news junkie, you get your fix from the newspaper or you wait until 6 ... or 11. Sports? Same. Oh and I hope you don't like soccer or hockey because you can't watch that stuff in this country more often than every four years. Go find a phone book if you want to call someone and complain.
I could go on, and on, and on, but you get the point. Oh wait, one more: how many of you had an email address in 1990? Be honest. And the people reading this blog are among the most likely to answer that affirmatively.
The world is different. The last 20 years has changed the human condition in ways that no other 20 year period can match. Imagine taking a 16 year old from 1990 and transplanting him or her to a 2010 high school. Culture shock indeed. Imagine transporting a soccer mom, a politician, a university professor... Pick just about any profession and the contrast would be so stark that those 1990 skills would be a debilitating liability.
Except one: that of a software tester. A circa 1990 tester would come from a mainframe/terminal world. Or if they were on the real cutting edge, a locally networked PC. They'd fit into the data center/slim client world with nary a hiccup. They'd know all about testing techniques because input partitioning, boundary cases, load and stress, etc, are still what we do today. Scripting? Yep, he'd be good there too. Syntax may have changed a bit, but that wouldn't take our time traveler long to pick up. That GEICO caveman may look funny at the disco, but he has the goods to get the job done.
Don't get me wrong, software testing has been full of innovation. We've minted patents and PhD theses. We built tools and automated the crud out of certain types of interfaces. But those interfaces change and that automation, we find to our distress, is rarely reuseable. How much real innovation have we had in this discipline that has actually stood the test of time? I argue that we've thrown most of it away. A disposable two decades. It was too tied to the application, the domain, the technology. Each project we start out basically anew, reinventing the testing wheel over and over. Each year's innovation looks much the same as the year before. 1990 quickly turns into 2010 and we remain stuck in the same old rut.
The challenge for the next twenty years will be to make a 2010 tester feel like a complete noob when transported to 2030. Indeed, I think this may be accomplished in far less than 20 years if we all work together. Imagine, for example, testing infrastructure built into the platform. Not enough for you? Imagine writing a single simple script that exercises your app, the browser and the OS at the same time and using the same language. Not enough for you? Imagine building an app and having it automatically download all applicable test suites and execute them on itself. Anyway, what are you working on?
Interested? Progress reports will be given at the following locations this year:
Swiss Testing Day, Zurich, March 17 2010
STAR East, Orlando, May 2010
GTAC, TBD, Fall 2010
Here's to an interesting future.

Interview with Copeland

Tuesday, February 23, 2010

Part 1, Part 2, Part 3
Posted by Patrick Copeland

Testing in the Data Center (Manufacturing No More)

Tuesday, February 9, 2010

was wasbuilt grownshippedmade available

Interviewing Insights and Test Frameworks

Tuesday, January 5, 2010

Google is hiring. We have openings for security testers, test tool developers, automation experts and manual testers. That's right, I said manual testers.

As a result of all this interviewing I've been reading a lot of interview feedback and wanted to pass along some insights about how these applicants approach solving the testing problems we ask in our interviews. I think the patterns I note in this post are interesting insights into the mind of the software tester, at least the ones who want to work for Google.

One of the things our interviewers like to ask is 'how would you test product xyz?' The answers help us judge a tester's instincts, but after reading many hundreds of these interviews I have noticed marked patterns in how testers approach solving such problems. It's as though testers have a default testing framework built into their thinking that guides them in choosing test cases and defines the way they approach test design.

In fact, these built-in frameworks seem to drive a tester's thinking to the extent that when I manage to identify the framework a tester is using, I can predict with a high degree of accuracy how they will answer the interviewers' questions. The framework defines what kind of tester they are. I find this intriguing and wonder if others have similar or counter examples to cite.

Here are the frameworks I have seen just in the last two weeks:

The Input Domain Framework treats software as an input-output mechanism. Subscribers of this framework think in terms of sets of inputs, rules about which inputs are more important and relationships between inputs, input sequences and outputs. This is a common model in random testing, model-based testing and the testing of protocols and APIs. An applicant who uses this framework will talk about which inputs they would use to test a specific application and try to justify why those inputs are important.

The Divide and Conquer Framework treats software as a set of features. Subscribers begin by decomposing an app into its features, prioritizing them and then working through that list in order. Often the decomposition is multi-layered creating a bunch of small testing problems out of one very large one. You don't test the feature so much as you test its constituent parts. An applicant who uses this framework is less concerned with actual test cases and more concerned with reducing the size of the problem to something manageable.

The Fishbowl Framework is a big picture approach to testing in which we manipulate the application while watching and comparing the results. Put the app in a fishbowl, swirl it around in the water and watch what happens. The emphasis is more on the watching and analyzing than it is on exactly how we manipulate the features. An applicant who uses this framework chooses tests that cause visible output and large state changes.

The Storybook Framework consists of developing specific scenarios and making sure the software does what is is supposed to do when presented with those scenarios. Stories start with the expected path and work outward. They don't always get beyond the expected. This framework tests coherence of behavior more than subtle errors. Applicants who employ this framework often take a user's point of view and talk about using the application to get real work done.

The Pessimists Framework starts with edge cases. Subscribers test erroneous input, bad data, misconfigured environments and so on. This is a common strategy on mature products where the main paths are well trodden. Applicants who use this framework like to assume that the main paths will get tested naturally as part of normal dev use and dog-fooding and that the testing challenge is concentrated on lower probability scenarios. They are quick to take credit for prior testing, assume its rationality and pound on problematic scenarios.

There are more and I am taking furious notes to try and make sense of them all. As I get to know the testers who work in my organization, it doesn't take long to see which frameworks they employ and in what order (many are driven by multiple frameworks). Indeed, after studying an applicant's first interview, I can almost always identify the framework they use to answer testing questions and can often predict how they are going to answer the questions other interviewers ask even before I read that far.

Now some interesting questions come out of this that I am still looking into. Which of these frameworks is best? Which is best suited to certain types of functionality? Which is better for getting a job at Google? Already patterns are emerging.

One thing is for sure, we're interviewing at a rate that will provide me with lots of data on this subject. Contact me if you'd like to participate in this little study!

http://twitter.com/googletesting

Monday, December 14, 2009

Testing Blog

Google is hiring SETs

Google @ ICST 2010

Still Stuck in the 90s

Interview with Copeland

Testing in the Data Center (Manufacturing No More)

Interviewing Insights and Test Frameworks

http://twitter.com/googletesting

Labels

Archive

Feed

Company-wide

Products

Developers