Adam Barr - The Problem With Software: Why Smart Engineers Write Bad Code

2018 Massachusetts Institute of Technology

Excerpt from Howl by Allen Ginsberg. Copyright 1956, 2010 Allen Ginsberg LLC, used by permission of The Wylie Agency LLC.

All rights reserved. No part of this book may be reproduced in any form by any electronic or mechanical means (including photocopying, recording, or information storage and retrieval) without permission in writing from the publisher.

This book was set in ITC Stone Serif Std by Toppan Best-set Premedia Limited. Printed and bound in the United States of America.

Library of Congress Cataloging-in-Publication Data

Names: Barr, Adam, author.

Title: The problem with software : why smart engineers write bad code / Adam Barr.

Description: Cambridge, MA : The MIT Press, [2018] | Includes bibliographical references and index.

Identifiers: LCCN 2018013460 | ISBN 9780262038515 (hardcover : alk. paper)

eISBN 9780262348195

Subjects: LCSH: Computer software--Development--Anecdotes. | Computer programmers--Anecdotes.

Classification: LCC QA76.76.D47 B373 2018 | DDC 005.3--dc23 LC record available at https://lccn.loc.gov/2018013460

ePub Version 1.0

To my children, Zachary, Madeline, Keenan, and Noah

A sincere thanks to the computer scientists who were kind enough to be interviewed by me, in person or via e-mail: Henry Baird, Victor Basili, Fred Brooks, Robert Harper, Donald Knuth, Adam McKay, David Parnas, Vaughan Pratt, and Ben Shneiderman. Id also like to thank Glenn Dardick and Vincent Erickson, who both responded to my impromptu Facebook messages and filled in some great historical detail.

I would also like to express my deepest thanks to everybody who reviewed the book. My most heartfelt appreciation goes to my sister, Rebecca, who read through the book with her professional editors eyethree times. Im also grateful to my parents, Michael and Marcia, and my brother, Joe, who read the whole thing and gave lots of feedback, and Krishnan Ramaswami, who despite not being related to me, read and commented on every chapter. Id also like to thank my son Zachary, who contributed valuable feedback, and Emily Papel and Carrie Olesen, who gave me early encouragement. Thanks to Bernard Pham, Kate Varni, and Joseph White as well for their comments. Bob Drews once again did an excellent job editing the final manuscript.

A big thank you to everyone at the MIT Press who worked on the book, particularly Maria Lufkin Lee, who expressed initial interest in and eventually acquired the book, and Christine Savage and Stephanie Cohen, who answered many questions, along with the anonymous referees who provided insightful comments. My appreciation also to Virginia Crossman, Cindy Milstein, and Susan Clark.

Thanks to all my colleagues in Engineering Excellence at Microsoft, especially those who worked together with me in Developer Excellence, and specifically Eric Brechner, who hired me into Engineering Excellence and was the guiding light behind much of what we did. Ill also mention Kristen Lane, who was kind enough to explain knob-and-tube wiring to me.

Thanks to the staff members of the King County Library System for allowing me to haunt their many branches while working on the book, and the baristas at various local coffee shops for the sameespecially Yum-E Yogurt in Issaquah. The Living Computer Museum in Seattle let me relive a not-so-small part of my youth, which was much appreciated.

Various anonymous people on Reddit have contributed knowledge that directly or indirectly found its way into the book. I dont know who you are, and you didnt know who I was, but consider yourselves acknowledged. Im also grateful to the maintainers of whatever Wikipedia pages I used in researching the book.

Finally, I would like to thank my wife, Maura, for putting up with the giant pile of dusty software books, and with me, as always.

In November 1988, a computer virus attacked computers connected to the still-nascent Internet. The virus exploited a programmer error: assuming that another computer could be trusted to send the right amount of data. It was a simple mistake, and the fix was trivial, but the programming language used was vulnerable to this type of mistake, and there was not a standard methodology for detecting that sort of problem.

In April 2014, a computer virus attacked computers connected to the now-ubiquitous Internet. The virus exploited a programmer error: assuming that another computer could be trusted to send the right amount of data. It was a simple mistake, and the fix was trivial, but the programming language used was vulnerable to this type of mistake, and there was not a standard methodology for detecting that sort of problem.

Remaining stuck on vulnerable programming language and no way to detect mistakes isnt what we expect after a quarter century of progress. Other new engineering disciplines started out producing unreliable products. In the early days of aviation, people built planes in their garages, with predictable results. Now, a hundred years later, when a world without air travel is unimaginable, we have extremely reliable planes based on well-understood and agreed-on engineering standards.

Not so with writing software. Although labeled an engineering discipline, software has few of the hallmarks of engineering, where a body of knowledge is built up over time based on rigorous experimentation. Questions one would reasonably ask of an engineered productHow strong is it? How long will it last? How it might fail?cannot be reliably answered for software, for either an individual part of a program or an entire suite of software. Professional licensing, a hallmark of most engineering disciplines, is viewed by the software industry as a potential source of lawsuits rather than an opportunity to establish standards.

The effect of this is not just user-visible bugs; its also a lot of wasted effort and reinvention on the part of programmers, leading to frustration and software that is delayed or never ships.

If youve heard about the software industry, it might be because of the unusual way in which programmers are interviewed. Websites, books, and even weeklong training classes are devoted to preparing people for the dreaded coding interview, which is presented as an all-or-nothing chance to impress with your skills and knowledgeespecially through whiteboard coding, in which a candidate has to dash out short programs on the whiteboard. Some candidates complain that this isnt an accurate representation of what their daily job would be, and they want companies to focus on other areas of their background. What they may not realize is that there isnt much else in their background to focus on. Unlike in other engineering disciplines, having a degree in software engineering does not guarantee that you understand a known corpus of programming tools and techniques, because such a thing does not exist. You likely wrote a lot of code when you were in college, but there is no way of knowing if it was any good. So asking people to write code snippets on a whiteboard is the best way we have to evaluate people.

Consider this joke, although its no laughing matter: What do you call the person who graduated last in their medical school class? The answer is doctorbecause graduating from medical school and completing your residency implies that you have learned what is needed to be one. I have asked doctors how they were interviewed when they were hired. They say that they were never asked specific medical questions or to perform simple medical procedures; instead, the talk was about how they speak to patients, how they feel about new medicines, and that sort of thingbecause it is understood that they know the basics of medicine. Computer science graduates can make no such universal claim.