Testing Blog
GTAC!
Friday, April 30, 2010
By James A. Whittaker
Yes I know, I've been quiet. Seriously heads down shipping products and developing what I think are some pretty cool new testing ideas and tools. Perhaps GTAC will be the chance for you to judge that for yourselves. Perhaps it will be worth the wait.
I hope I am invited to speak at GTAC. (Is this an appropriate forum for such lobbying? Should I open it up to a vote on whether I should or should not be there? I am happy not going as India is a long trip, but this is GTAC we are talking about!) There are a number of things that will be ready to either debut or, if I am lucky, open source by then.
Would you like to see a Chrome extension to display bug, test case, coverage and other information as an overlay on top of your web app UI? Imagine being able to see bugs at their exact location on the UI, report bugs by simply right-clicking the errant part of the web page, see footprints where test cases (automated and manual and across multiple testers) have been and lots more useful information. Are you tired of querying databases to see these things and just want your test data as a cellophane wrapper around your web app UI? If you were at STAR this week, you got a preview. But that presentation is already out of date.
Would you like to be able to write automated test cases that can control your web app, your browser
and
the operating system they are running on? Well if that stack contains Chrome and Chrome OS, you can do it with a new web test framework we are developing. Would you like to do all of this with Java Script? Sound like magic? Well I think the Web Test Framework is appropriately named: WTF.
Would you like to see a record and playback facility that records directly to Java Script and is actually built-in to your browser? A R/P tool that handles AJAX and won't get confused by self-updating web pages? That stores recordings directly into a test case management system that is accessible to the world?
Would you like to hear about the extensive library of testing tours we have developed and how our manual testing strategy across the web-app/Chrome/Chrome-OS stack is shaping up?
These are some of the things that have kept me from this blog. Forgive me. I will report on them here and, with a little pressure directed toward Sujay The Decider perhaps demo them at GTAC.
Google Test Automation Conference 2010 - Hyderabad, India
Thursday, April 22, 2010
Thanks for all the enquiries about this year's GTAC event. We have been busy planning and are now ready to share the details. GTAC 2010 will be hosted by
Google office in Hyderabad, India
on
October 28th & 29th, 2010.
As in previous years, the focus of the conference will be on solving software engineering challenges using tools and automation. This year the conference theme is "
Test to Testability
". With this theme
our goal is to look beyond the challenges faced during testing and the complex products that are being developed today. We also want to highlight methodologies and tools that can be used to build testability into our products and to look at how developer tools can be a means towards effective and efficient testing. GTAC 2010 hopes to bring together a group that
shares lessons learned and practical experiences regarding testing web apps, services, and systems.
One of the strengths of the conference is that it's driven by a peer group and vocal participation. As in previous years, GTAC is an
invitation only
conference to share great ideas and to have your thoughts challenged and refined. When you apply, we will be asking you to share with us what ideas and insights you'll be bringing to the conference and how these can further the discussion. The application process for attending will be opening in mid-May 2010.
Today we are also launching the official
site
for this event and will be updating it and this blog with relevant information in the coming weeks. The next announcement will be a call for attendance and proposals, so do watch these spaces.
Please send suggestions, questions and recommendations to:
gtac-2010@google.com
or post your comments here to this blog.
Looking forward to having a great set of participants and presenters to make this a fun and valuable learning event!
Sujay Sahni on behalf of the GTAC 2010 Committees
Google's Innovation Factory (and how testing adapts)
Monday, April 19, 2010
Now that
ICST
2010
has concluded, I'm going ahead and posting the keynote paper and the slides. Enjoy!
Patrick Copeland
Let me know if you have comments or questions.
Here's a
link
to the paper. They are semi-complementary.
Google is hiring SETs
Friday, March 12, 2010
Drop me a resume and a brief note if you're interested in joining Google as an SET. email: test.eng.hiring@gmail.com. Thanks -- Patrick Copeland
When we hire people we look for folks with a "testing DNA." These are people who are great computer scientists at their core, but also are very curious, love software, and are passionate about test engineering. People who have those characteristics tend to pursue challenges and continue to learn. Are you one of us? We have positions all over the US and the world.
What is a SET?
At Google, Software Engineers in Test (SET) develop test frameworks and build robust, scalable, and effective tests. SETs spend a majority of their time coding in either C++, Java, or scripting in Python. A SET is a software engineer, a core developer, who has a passion for test engineering.
How is testing done differently at Google?
Literally within milliseconds of a code check-in, our build process will automatically select the appropriate tests to run based on dependency analysis, run those tests and report the results.
By reducing the window of opportunity for bad code to go unnoticed, overall debugging and bug isolation time is radically reduced. The net result is that the engineering teams no longer sink hours into debugging build problems and test failures.
Development teams write good tests because they care about the products, but also because they want more time to spend writing features and less on debugging.
Testing teams focus on higher abstractions, like identifying latencies, system or customer focused testing, and enabling the process with tools.
SETs avoid becoming codependents within this system and generally do not write unit tests or other activities that are best done by the developer.
More about SETs
Our SET’s spend time developing code to prevent bugs. Google has a strong cultural emphasis on developers improving quality (i.e. unit tests, code reviews, design reviews, root cause analysis). We want our engineers to spend their time innovating - not fixing bugs.
SETs enable products to launch faster. They have great influence over internal processes and how developers write code.
One of Google's less understood capabilities is our massive distributed computing environment. The testing groups exploit this infrastructure to do huge amounts of work very quickly and elegantly.
For someone who wants to learn and grow as an engineer, the uninhibited access to the entire code base is a unique opportunity.
Google @ ICST 2010
Saturday, March 6, 2010
I'll be a presenting a paper at ICST 2010 in Paris April 6-10 about how Google tests and builds software. Here's a
pointer to the program
if you are interested. Also here's a
link to the abstract
of the talk itself. I'll publish the paper after the talk here. Hopefully, I'll see some of you there!
Posted by Patrick Copeland
Still Stuck in the 90s
Monday, March 1, 2010
By James A. Whittaker
Flashback. It's 1990. Chances are you do not own a cell phone. And if you do it weighs more than a full sized laptop does now. You certainly have no iPod. The music in your car comes from the one or two local radio stations that play songs you can tolerate and a glove box full of CDs and cassettes. Yes, I said cassettes...you know the ones next to those paper road maps. Music on the go? We carried our boom boxes on our shoulder back then.
If you are a news junkie, you get your fix from the newspaper or you wait until 6 ... or 11. Sports? Same. Oh and I hope you don't like soccer or hockey because you can't watch that stuff in this country more often than every four years. Go find a phone book if you want to call someone and complain.
I could go on, and on, and on, but you get the point. Oh wait, one more: how many of you had an email address in 1990? Be honest. And the people reading this blog are among the most likely to answer that affirmatively.
The world is different. The last 20 years has changed the human condition in ways that no other 20 year period can match. Imagine taking a 16 year old from 1990 and transplanting him or her to a 2010 high school. Culture shock indeed. Imagine transporting a soccer mom, a politician, a university professor... Pick just about any profession and the contrast would be so stark that those 1990 skills would be a debilitating liability.
Except one: that of a software tester. A circa 1990 tester would come from a mainframe/terminal world. Or if they were on the real cutting edge, a locally networked PC. They'd fit into the data center/slim client world with nary a hiccup. They'd know all about testing techniques because input partitioning, boundary cases, load and stress, etc, are still what we do today. Scripting? Yep, he'd be good there too. Syntax may have changed a bit, but that wouldn't take our time traveler long to pick up. That GEICO caveman may look funny at the disco, but he has the goods to get the job done.
Don't get me wrong, software testing has been full of innovation. We've minted patents and PhD theses. We built tools and automated the crud out of certain types of interfaces. But those interfaces change and that automation, we find to our distress, is rarely reuseable. How much real innovation have we had in this discipline
that has actually stood the test of time
? I argue that we've thrown most of it away. A disposable two decades. It was too tied to the application, the domain, the technology. Each project we start out basically anew, reinventing the testing wheel over and over. Each year's innovation looks much the same as the year before. 1990 quickly turns into 2010 and we remain stuck in the same old rut.
The challenge for the next twenty years will be to make a 2010 tester feel like a complete noob when transported to 2030. Indeed, I think this may be accomplished in far less than 20 years if we all work together. Imagine, for example, testing infrastructure
built into the platform.
Not enough for you? Imagine writing a single simple script that exercises your app, the browser and the OS at the same time and using the same language. Not enough for you? Imagine building an app and having it automatically download all applicable test suites
and execute them on itself
. Anyway, what are you working on?
Interested? Progress reports will be given at the following locations this year:
Swiss Testing Day, Zurich, March 17 2010
STAR East, Orlando, May 2010
GTAC, TBD, Fall 2010
Here's to an interesting future.
Interview with Copeland
Tuesday, February 23, 2010
I recently did an interview with Matt Johnston of uTest (a community based testing company) that talks about our philosophy and approach to testing at Google. Let me know what you think.
Part 1
,
Part 2
,
Part 3
Posted by Patrick Copeland
Labels
Aaron Jacobs
1
Adam Porter
1
Alan Faulkner
1
Alan Myrvold
1
Alberto Savoia
4
Alek Icev
2
Alex Eagle
1
Allen Hutchison
6
Andrew Trenk
8
Android
1
Anthony Vallone
25
Antoine Picard
1
APIs
2
App Engine
1
April Fools
2
Arif Sukoco
1
Bruce Leban
1
C++
11
Chaitali Narla
2
Christopher Semturs
1
Chrome
3
Chrome OS
2
Dave Chen
1
Diego Salas
2
Dmitry Vyukov
1
Dori Reuveni
1
Eduardo Bravo Ortiz
1
Ekaterina Kamenskaya
1
Erik Kuefler
3
Espresso
1
George Pirocanac
2
Google+
1
Goranka Bjedov
1
GTAC
54
Hank Duan
1
Harry Robinson
5
Havard Rast Blok
1
Hongfei Ding
1
James Whittaker
42
Jason Arbon
2
Jason Elbaum
1
Jason Huggins
1
Java
5
JavaScript
7
Jay Han
1
Jessica Tomechak
1
Jim Reardon
1
Jobs
14
Joe Allan Muharsky
1
Joel Hynoski
1
John Penix
1
John Thomas
3
Jonathan Rockway
1
Jonathan Velasquez
1
Julian Harty
5
Julie Ralph
1
Karin Lundberg
1
Kaue Silveira
1
Kevin Graney
1
Kirkland
1
Kurt Alfred Kluever
1
Lesley Katzen
1
Marc Kaplan
3
Mark Ivey
1
Mark Striebeck
1
Marko Ivanković
1
Markus Clermont
3
Michael Bachman
1
Michael Klepikov
1
Mike Wacker
1
Misko Hevery
32
Mobile
2
Mona El Mahdy
1
Noel Yap
1
Patricia Legaspi
1
Patrick Copeland
23
Patrik Höglund
5
Peter Arrenbrecht
1
Phil Rollet
1
Philip Zembrod
4
Pooja Gupta
1
Radoslav Vasilev
1
Rajat Dewan
1
Rajat Jain
1
Rich Martin
1
Richard Bustamante
1
Roshan Sembacuttiaratchy
1
Ruslan Khamitov
1
Sean Jordan
1
Sharon Zhou
1
Shyam Seshadri
4
Simon Stewart
2
Stephen Ng
1
Tejas Shah
1
Test Analytics
1
Tony Voellm
2
TotT
54
Vojta Jína
1
WebRTC
2
Yvette Nameth
2
Zhanyong Wan
6
Zuri Kemp
2
Archive
2015
December
GTAC 2015 Wrap Up
November
October
August
June
May
April
March
February
January
2014
December
November
October
September
August
July
June
May
April
March
February
January
2013
December
November
October
August
July
June
May
April
March
January
2012
December
November
October
September
August
2011
November
October
September
August
July
June
May
April
March
February
January
2010
December
November
October
September
August
July
June
May
April
March
February
January
2009
December
November
October
September
August
July
June
May
April
February
January
2008
December
November
October
September
August
July
June
May
April
March
February
January
2007
October
September
August
July
June
May
April
March
February
January
Feed
Follow @googletesting