James A. Whittaker - How Google Tests Software

James Whittaker has long had the pulse of the issues that are shaping testing practice. In the decade of the Cloud Transformation, this book is a must read not just for Googlers, but for all testers who want their practices to remain relevant, competitive, and meaningful.

Sam Guckenheimer, Product Owner,
Visual Studio Strategy, Microsoft

Google has consistently been an innovator in the app testing spacewhether its blending test automation with manual efforts, melding in-house and outsourced resources, or more recently, pioneering the use of in-the-wild testing to complement their in-the-lab efforts. This appetite for innovation has enabled Google to solve new problems and launch better apps.

In this book, James Whittaker provides a blueprint for Googles success in the rapidly evolving world of app testing.

Doron Reuveni, CEO and Cofounder, uTest

This book is a game changer, from daily releases to heads-up displays. James Whittaker takes a computer-science approach to testing that will be the standard for software companies in the future. The process and technical innovations we use at Google are described in a factual and entertaining style. This book is a must read for anyone involved in software development.

Michael Bachman, Senior Engineering Manager
at Google Inc., AdSense/Display

By documenting much of the magic of Googles test engineering practices, the authors have written the equivalent of the Kama Sutra for modern software testing.

Alberto Savoia, Engineering Director, Google

If you ship code in the cloud and want to build a strategy for ensuring a quality product with lots of happy customers, you must study and seriously consider the methods in this book.

Phil Waligora, Salesforce.com

James Whittaker is an inspiration and mentor to many people in the field of testing. We wouldnt have the talent or technology in this field without his contributions. I am consistently in awe of his drive, enthusiasm, and humor. Hes a giant in the industry and his writing should be required reading for anyone in the IT industry.

Stewart Noakes, Chairman TCL Group Ltd.,
United Kingdom

I worked with James Whittaker during his time at Microsoft, and although I miss having him here at Microsoft, I knew he would do great things at Google. James, Jason Arbon, and Jeff Carollo have packed this book with innovative testing ideas, practical examples, and insights into the Google testing machine. Anyone with an ounce of curiosity about Googles approach to testing and quality or with the smallest desire to discover a few new ideas in testing will find value in these pages.

Alan Page, Microsoft Xbox, and Author
of How We Test Software at Microsoft

James Whittaker
Jason Arbon
Jeff Carollo

Upper Saddle River, NJ Boston Indianapolis San Francisco
New York Toronto Montreal London Munich Paris Madrid
Capetown Sydney Tokyo Singapore Mexico City

Many of the designations used by manufacturers and sellers to distinguish their products are claimed as trademarks. Where those designations appear in this book, and the publisher was aware of a trademark claim, the designations have been printed with initial capital letters or in all capitals.

The authors and publisher have taken care in the preparation of this book, but make no expressed or implied warranty of any kind and assume no responsibility for errors or omissions. No liability is assumed for incidental or consequential damages in connection with or arising out of the use of the information or programs contained herein.

The publisher offers excellent discounts on this book when ordered in quantity for bulk purchases or special sales, which may include electronic versions and/or custom covers and content particular to your business, training goals, marketing focus, and branding interests. For more information, please contact:

U.S. Corporate and Government Sales
(800) 382-3419

For sales outside the United States, please contact:

International Sales

Visit us on the Web: informit.com/aw

The Library of Congress cataloging-in-publication data is on file.

Copyright 2012 Pearson Education, Inc.

All rights reserved. Printed in the United States of America. This publication is protected by copyright, and permission must be obtained from the publisher prior to any prohibited reproduction, storage in a retrieval system, or transmission in any form or by any means, electronic, mechanical, photocopying, recording, or likewise. To obtain permission to use material from this work, please submit a written request to Pearson Education, Inc., Permissions Department, One Lake Street, Upper Saddle River, New Jersey 07458, or you may fax your request to (201) 236-3290.

ISBN-13: 978-0-321-80302-3

ISBN-10: 0-321-80302-7

Text printed in the United States on recycled paper at Courier in Westford, Massachusetts.

First printing: March 2012

Publisher
Paul Boger

Executive Editor
Chris Guzikowski

Senior Development Editor
Chris Zahn

Managing Editor
Kristy Hart

Project Editor
Jovana San Nicolas-Shirley

Copy Editor
Ginny Bess Munroe

Indexer
Erika Millen

Proofreader
Mike Henry

Editorial Assistant
Olivia Basegio

Cover Designer
Anne Jones

Compositor
Gloria Schurick

To all testers at Google, Microsoft, and elsewhere whove made me
think differently.

James A. Whittaker

To my wife Heather and my children Luca, Mateo, Dante, and Odessa who
thought I worked at Starbucks all this time.

Jason Arbon

To Mom, Dad, Lauren, and Alex.

Jeff Carollo

Writing a foreword for a book you wish you had written yourself is a dubious honor; its a bit like serving as best man for a friend who is about to spend the rest of his life with the girl you wanted to marry. But James Whittaker is a cunning guy. Before asking me if Id be willing to write this preface, he exploited my weakness for Mexican food by treating me to a very nice dinner and plying me with more than a couple Dos Equis before he popped the question. By the time this happened, I was as malleable and agreeable as the bowl of guacamole I had just finished. Si senor, was pretty much all I could say. His ploy worked and here he stands with his book as his bride and I get to make the wedding speech.

As I said, hes one cunning guy.

So here we go...a preface to the book I wish I had written myself. Cue the mushy wedding music.

Does the world really need yet another software testing book, especially yet another software testing book from the prolific James Whittaker, whom Ive publicly called the Octomom of test book publishing on more than one occasion? Arent there enough books out there describing the same old tired testing methodologies and dishing out dubious and dated advice? Well, there are enough of those books, but this book I am afraid is not one of them. Thats why I wish I had written it myself. The world actually needs this particular testing book.

The Internet has dramatically changed the way most software is designed, developed, and distributed. Many of the testing best practices, embodied in any number of once popular testing books of yesteryear, are at best inefficient, possibly ineffective, and in some cases, downright counterproductive in todays environment. Things have been moving so fast in our industry that many of the software testing books written as recently as a few years ago are the equivalent of surgery books containing advice about leeches and skull drilling to rid the body of evil spirits; it would be best to recycle them into adult diapers to make sure they dont fall into the hands of the gullible.