Testing Blog
Google Test Automation Conference 2010 - Hyderabad, India
Thursday, April 22, 2010
Thanks for all the enquiries about this year's GTAC event. We have been busy planning and are now ready to share the details. GTAC 2010 will be hosted by
Google office in Hyderabad, India
on
October 28th & 29th, 2010.
As in previous years, the focus of the conference will be on solving software engineering challenges using tools and automation. This year the conference theme is "
Test to Testability
". With this theme
our goal is to look beyond the challenges faced during testing and the complex products that are being developed today. We also want to highlight methodologies and tools that can be used to build testability into our products and to look at how developer tools can be a means towards effective and efficient testing. GTAC 2010 hopes to bring together a group that
shares lessons learned and practical experiences regarding testing web apps, services, and systems.
One of the strengths of the conference is that it's driven by a peer group and vocal participation. As in previous years, GTAC is an
invitation only
conference to share great ideas and to have your thoughts challenged and refined. When you apply, we will be asking you to share with us what ideas and insights you'll be bringing to the conference and how these can further the discussion. The application process for attending will be opening in mid-May 2010.
Today we are also launching the official
site
for this event and will be updating it and this blog with relevant information in the coming weeks. The next announcement will be a call for attendance and proposals, so do watch these spaces.
Please send suggestions, questions and recommendations to:
gtac-2010@google.com
or post your comments here to this blog.
Looking forward to having a great set of participants and presenters to make this a fun and valuable learning event!
Sujay Sahni on behalf of the GTAC 2010 Committees
Google's Innovation Factory (and how testing adapts)
Monday, April 19, 2010
Now that
ICST
2010
has concluded, I'm going ahead and posting the keynote paper and the slides. Enjoy!
Patrick Copeland
Let me know if you have comments or questions.
Here's a
link
to the paper. They are semi-complementary.
Google is hiring SETs
Friday, March 12, 2010
Drop me a resume and a brief note if you're interested in joining Google as an SET. email: test.eng.hiring@gmail.com. Thanks -- Patrick Copeland
When we hire people we look for folks with a "testing DNA." These are people who are great computer scientists at their core, but also are very curious, love software, and are passionate about test engineering. People who have those characteristics tend to pursue challenges and continue to learn. Are you one of us? We have positions all over the US and the world.
What is a SET?
At Google, Software Engineers in Test (SET) develop test frameworks and build robust, scalable, and effective tests. SETs spend a majority of their time coding in either C++, Java, or scripting in Python. A SET is a software engineer, a core developer, who has a passion for test engineering.
How is testing done differently at Google?
Literally within milliseconds of a code check-in, our build process will automatically select the appropriate tests to run based on dependency analysis, run those tests and report the results.
By reducing the window of opportunity for bad code to go unnoticed, overall debugging and bug isolation time is radically reduced. The net result is that the engineering teams no longer sink hours into debugging build problems and test failures.
Development teams write good tests because they care about the products, but also because they want more time to spend writing features and less on debugging.
Testing teams focus on higher abstractions, like identifying latencies, system or customer focused testing, and enabling the process with tools.
SETs avoid becoming codependents within this system and generally do not write unit tests or other activities that are best done by the developer.
More about SETs
Our SET’s spend time developing code to prevent bugs. Google has a strong cultural emphasis on developers improving quality (i.e. unit tests, code reviews, design reviews, root cause analysis). We want our engineers to spend their time innovating - not fixing bugs.
SETs enable products to launch faster. They have great influence over internal processes and how developers write code.
One of Google's less understood capabilities is our massive distributed computing environment. The testing groups exploit this infrastructure to do huge amounts of work very quickly and elegantly.
For someone who wants to learn and grow as an engineer, the uninhibited access to the entire code base is a unique opportunity.
Google @ ICST 2010
Saturday, March 6, 2010
I'll be a presenting a paper at ICST 2010 in Paris April 6-10 about how Google tests and builds software. Here's a
pointer to the program
if you are interested. Also here's a
link to the abstract
of the talk itself. I'll publish the paper after the talk here. Hopefully, I'll see some of you there!
Posted by Patrick Copeland
Still Stuck in the 90s
Monday, March 1, 2010
By James A. Whittaker
Flashback. It's 1990. Chances are you do not own a cell phone. And if you do it weighs more than a full sized laptop does now. You certainly have no iPod. The music in your car comes from the one or two local radio stations that play songs you can tolerate and a glove box full of CDs and cassettes. Yes, I said cassettes...you know the ones next to those paper road maps. Music on the go? We carried our boom boxes on our shoulder back then.
If you are a news junkie, you get your fix from the newspaper or you wait until 6 ... or 11. Sports? Same. Oh and I hope you don't like soccer or hockey because you can't watch that stuff in this country more often than every four years. Go find a phone book if you want to call someone and complain.
I could go on, and on, and on, but you get the point. Oh wait, one more: how many of you had an email address in 1990? Be honest. And the people reading this blog are among the most likely to answer that affirmatively.
The world is different. The last 20 years has changed the human condition in ways that no other 20 year period can match. Imagine taking a 16 year old from 1990 and transplanting him or her to a 2010 high school. Culture shock indeed. Imagine transporting a soccer mom, a politician, a university professor... Pick just about any profession and the contrast would be so stark that those 1990 skills would be a debilitating liability.
Except one: that of a software tester. A circa 1990 tester would come from a mainframe/terminal world. Or if they were on the real cutting edge, a locally networked PC. They'd fit into the data center/slim client world with nary a hiccup. They'd know all about testing techniques because input partitioning, boundary cases, load and stress, etc, are still what we do today. Scripting? Yep, he'd be good there too. Syntax may have changed a bit, but that wouldn't take our time traveler long to pick up. That GEICO caveman may look funny at the disco, but he has the goods to get the job done.
Don't get me wrong, software testing has been full of innovation. We've minted patents and PhD theses. We built tools and automated the crud out of certain types of interfaces. But those interfaces change and that automation, we find to our distress, is rarely reuseable. How much real innovation have we had in this discipline
that has actually stood the test of time
? I argue that we've thrown most of it away. A disposable two decades. It was too tied to the application, the domain, the technology. Each project we start out basically anew, reinventing the testing wheel over and over. Each year's innovation looks much the same as the year before. 1990 quickly turns into 2010 and we remain stuck in the same old rut.
The challenge for the next twenty years will be to make a 2010 tester feel like a complete noob when transported to 2030. Indeed, I think this may be accomplished in far less than 20 years if we all work together. Imagine, for example, testing infrastructure
built into the platform.
Not enough for you? Imagine writing a single simple script that exercises your app, the browser and the OS at the same time and using the same language. Not enough for you? Imagine building an app and having it automatically download all applicable test suites
and execute them on itself
. Anyway, what are you working on?
Interested? Progress reports will be given at the following locations this year:
Swiss Testing Day, Zurich, March 17 2010
STAR East, Orlando, May 2010
GTAC, TBD, Fall 2010
Here's to an interesting future.
Interview with Copeland
Tuesday, February 23, 2010
I recently did an interview with Matt Johnston of uTest (a community based testing company) that talks about our philosophy and approach to testing at Google. Let me know what you think.
Part 1
,
Part 2
,
Part 3
Posted by Patrick Copeland
Testing in the Data Center (Manufacturing No More)
Tuesday, February 9, 2010
By James A. Whittaker
W. Edwards Deming helped to revolutionize the process of manufacturing automobiles in the 1970s and a decade later the software industry ran with the manufacturing analogy and the result was nearly every waterfall, spiral or agile method we have. Some like TQM, Cleanroom and Six Sigma are obvious descendants of Deming while others were just heavily influenced by his thinking. Deming
was
the man.
I repeat,
was
. My time testing in Google's data center makes it clear that this analogy just doesn't fit anymore. I want a new one. And I want one that helps me as a tester. I want one that better guides my behavior.
We just don't write or release software the way we used to. Software isn't so much
built
as it is
grown
. Software isn't
shipped
... it's simply
made available
by, often literally, the flip of a switch. This is not your father's software. 21st century development is a seamless path from innovation to release where every phase of development, including release, is happening all the time. Users are on the inside of the firewall in that respect and feedback is constant. If a product isn't compelling we find out much earlier and it dies in the data center. I fancy these dead products serve to enrich the data center, a digital circle of life where new products are built on the bones of the ones that didn't make it.
In our father's software and Deming's model we talk about quality control and quality assurance while we play the role of inspector. In contrast, my job seems much more like that of an attending physician. In fact, a medical analogy gives us some interesting parallels to think about software testing. A physician's hospital is our data center, there is always activity and many things are happening in parallel. Physicians have patients; we have applications and features. Their medical devices are our infrastructure and tools. I can picture my application's features strewn across the data center in little virtual hospital beds. Over here is the GMail ward, over there is Maps. Search, of course, has a wing of its own and Ads, well, they all have private rooms.
In a hospital records are important. There are too many patients with specific medical conditions and treatment histories for any physician to keep straight. Imagine walking up to the operating table without examination notes and diagnoses? Imagine operating without a constant stream of real time health data?
Yet as software testers we find ourselves in this situation often. That app lying in our data center has been tested before. It has been treated before. Where are our medical notes?
So let's add little clipboards to the virtual data center beds in which our apps lay. Let's add equipment to take vitals and display them for any attending tester to see. Like human patients, apps have a pulse, data runs through code paths like blood through veins. There are important things happening, countable events that lead to statistics, indicators and create a medical history for an attending tester to use in whatever procedure they must now perform. The work of prior testers need not be ignored.
It's an unsettling aspect of the analogy that I have put developers in the role of creator, but so be it. Like other metaphorical creators before them they have spawned intrinsically flawed creatures. Security is their cancer, privacy their aging. Software is born broken and only some things can be fixed. The cancer of security can only be managed. Like actual aging, privacy is a guarantee only young software enjoys. Such is the life of a data center app.
But it is the monitors and clipboards that intrigue me. What do they say of our digital patients? As an app grows from concept into adolescence what part of their growth do we monitor? Where is the best place to place our probes? How do we document treatment and evaluations? Where do we store the notes about surgeries? What maladies have been treated? Are there problematic organs and recurrent illness? The documents and spreadsheets of the last century are inadequate. A patient's records are only useful if they are attached to the patient, up-to-date and in full living color to be read by whatever attending tester happens to be on call.
This is the challenge of the new century of software. It's not a process of get-it-as-reliable-as-possible-before-we-ship. It's health care, cradle to grave health care ... prevention, diagnosis, treatment and cure.
So slip into your scrubs, it's going to be a long night in the ER.
Labels
Aaron Jacobs
1
Adam Porter
1
Alan Faulkner
1
Alan Myrvold
1
Alberto Savoia
4
Alek Icev
2
Alex Eagle
1
Allen Hutchison
6
Andrew Trenk
8
Android
1
Anthony Vallone
25
Antoine Picard
1
APIs
2
App Engine
1
April Fools
2
Arif Sukoco
1
Bruce Leban
1
C++
11
Chaitali Narla
2
Christopher Semturs
1
Chrome
3
Chrome OS
2
Dave Chen
1
Diego Salas
2
Dmitry Vyukov
1
Dori Reuveni
1
Eduardo Bravo Ortiz
1
Ekaterina Kamenskaya
1
Erik Kuefler
3
Espresso
1
George Pirocanac
2
Google+
1
Goranka Bjedov
1
GTAC
54
Hank Duan
1
Harry Robinson
5
Havard Rast Blok
1
Hongfei Ding
1
James Whittaker
42
Jason Arbon
2
Jason Elbaum
1
Jason Huggins
1
Java
5
JavaScript
7
Jay Han
1
Jessica Tomechak
1
Jim Reardon
1
Jobs
14
Joe Allan Muharsky
1
Joel Hynoski
1
John Penix
1
John Thomas
3
Jonathan Rockway
1
Jonathan Velasquez
1
Julian Harty
5
Julie Ralph
1
Karin Lundberg
1
Kaue Silveira
1
Kevin Graney
1
Kirkland
1
Kurt Alfred Kluever
1
Lesley Katzen
1
Marc Kaplan
3
Mark Ivey
1
Mark Striebeck
1
Marko Ivanković
1
Markus Clermont
3
Michael Bachman
1
Michael Klepikov
1
Mike Wacker
1
Misko Hevery
32
Mobile
2
Mona El Mahdy
1
Noel Yap
1
Patricia Legaspi
1
Patrick Copeland
23
Patrik Höglund
5
Peter Arrenbrecht
1
Phil Rollet
1
Philip Zembrod
4
Pooja Gupta
1
Radoslav Vasilev
1
Rajat Dewan
1
Rajat Jain
1
Rich Martin
1
Richard Bustamante
1
Roshan Sembacuttiaratchy
1
Ruslan Khamitov
1
Sean Jordan
1
Sharon Zhou
1
Shyam Seshadri
4
Simon Stewart
2
Stephen Ng
1
Tejas Shah
1
Test Analytics
1
Tony Voellm
2
TotT
54
Vojta Jína
1
WebRTC
2
Yvette Nameth
2
Zhanyong Wan
6
Zuri Kemp
2
Archive
2015
December
GTAC 2015 Wrap Up
November
October
August
June
May
April
March
February
January
2014
December
November
October
September
August
July
June
May
April
March
February
January
2013
December
November
October
August
July
June
May
April
March
January
2012
December
November
October
September
August
2011
November
October
September
August
July
June
May
April
March
February
January
2010
December
November
October
September
August
July
June
May
April
March
February
January
2009
December
November
October
September
August
July
June
May
April
February
January
2008
December
November
October
September
August
July
June
May
April
March
February
January
2007
October
September
August
July
June
May
April
March
February
January
Feed
Follow @googletesting
