Meredith Broussard - Artificial Unintelligence: How Computers Misunderstand the World

Meredith Broussard

The MIT Press

Cambridge, Massachusetts

London, England

2018 Meredith Broussard

All rights reserved. No part of this book may be reproduced in any form by any electronic or mechanical means (including photocopying, recording, or information storage and retrieval) without permission in writing from the publisher. Some material in this volume is adapted from essays published in The Atlantic: Why Poor Schools Cant Win at Standardized Testing (July 2014) and The Secret Lives of Hackathon Junkies (July 2015). Portions of This Car Wont Drive Itself originally appeared in The Pennsylvania Gazette.

This book was set in ITC Stone Serif Std by Toppan Best-set Premedia Limited. Printed and bound in the United States of America.

Library of Congress Cataloging-in-Publication Data

Names: Broussard, Meredith, author.

Title: Artificial unintelligence : how computers misunderstand the world /

Meredith Broussard.

Description: Cambridge, MA : MIT Press, [2018] | Includes bibliographical

references and index.

Identifiers: LCCN 2017041363 | ISBN 9780262038003 (hardcover : alk. paper)

eISBN 9780262346726

Subjects: LCSH: Electronic data processing--Social aspects. | Computer

programs--Correctness. | Errors.

Classification: LCC QA76.9.C66 B787 2018 | DDC 303.48/34--dc23 LC record available at https://lccn.loc.gov/2017041363

ePub Version 1.0

d_r0

For my family

I love technology. Ive loved it since I was a little girl and my parents bought me an Erector Set that I used to build a giant (to me) robot out of small pierced pieces of metal. The robot was supposed to be powered by a miniature, battery-driven motor. I was an imaginative kid; I convinced myself that once this robot was built, it would move around the house as easily as I did, and I would have a new robot best friend. I would teach the robot to dance. It would follow me around the house, and (unlike my dog) it would play fetch.

I spent hours sitting on the red wool rug in the upstairs hallway of my parents house, daydreaming and assembling the robot. I tightened dozens of nuts and bolts using the sets tiny, child-sized wrenches. The most exciting moment came when I was ready to plug in the motor. My mom and I made a special trip to the store to get the right batteries for the motor. We got home, and I raced upstairs to connect the bare wires to the gears and turn on my robot. I felt like Orville and Wilbur Wright at Kitty Hawk, launching a new machine and hoping it would change the world.

Nothing happened.

I checked the diagrams. I flicked the on/off switch a few times. I flipped the batteries. Still nothing. My robot didnt work. I went to get my mom.

You need to come upstairs. My robot isnt working, I said sadly.

Did you try turning it off and turning it on again? my mom asked.

I did that, I said.

Did you try flipping the batteries? she asked.

Yes, I said. I was getting frustrated.

Ill come look at it, she said. I grabbed her hand and pulled her upstairs. She tinkered with the robot for a little bit, looking at the directions and fiddling with the wiring and turning the switch on and off a few times. Its not working, she said finally.

Why not? I asked. She could have just told me that the motor was broken, but my mother believed in complete explanations. She told me that the motor was broken, and then she also explained global supply chains and assembly lines and reminded me that I knew how factories worked because I liked to watch the videos on Sesame Street featuring huge industrial machines making packages of crayons.

Things can go wrong when you make things, she explained. Something went wrong when they made this motor, and it ended up in your kit anyway, and now were going to get one that works. We called the Erector hotline number printed on the instructions, and the nice people at the toy company sent us a new motor in the mail. It arrived in a week or so, and I plugged it in, and my robot worked. By that point, it was anticlimactic. The robot worked, but not well. It could move slowly across the hardwood floor. It got stuck on the rug. It wasnt going to be my new best friend. After a few days, I took the robot apart to make the next project in the kit, a Ferris wheel.

I learned a few things from making this robot. I learned how to use tools to build technology, and that building things could be fun. I discovered that my imagination was powerful, but that the reality of technology couldnt measure up to what I imagined. I also learned that parts break.

A few years later, when I began writing computer programs, I discovered that these lessons from robot building translated well to the world of computer code. I could imagine vastly complex computer programs, but what the computer could actually do was often a letdown. I ran into many situations where programs didnt work because a part failed somewhere in the computers innards. Nevertheless, I persisted, and I still love building and using technology. I have a vast number of social media accounts. I once hacked a crockpot to build a device for tempering twenty-five pounds of chocolate as part of a cooking project. I even built a computerized system to automatically water my garden.

Recently, however, Ive become skeptical of claims that technology will save the world. For my entire adult life, Ive been hearing promises about what technology can do to change the world for the better. I began studying computer science at Harvard in September 1991, months after Tim Berners-Lee launched the worlds first website at CERN, the particle physics lab run by the European Organization for Nuclear Research. In my sophomore year, my roommate bought a NeXT cube, the same square black computer that Berners-Lee used as a web server at CERN. It was fun. My roommate got a high-speed connection in our dormitory suite, and we used his $5,000 computer to check our email. Another roommate, who had recently come out and was too young for Bostons gay bar scene, used the computer to hang out on online bulletin boards and meet boys. It was easy to believe that in the future, we would do everything online.

For youthful idealists of my generation, it was also easy to believe that the world we were creating online would be better and more just than the world we already had. In the 1960s, our parents thought they could make a better world by dropping out or living in communes. We saw that our parents had gone straight, and communes clearly werent the answerbut there was this entire new, uncharted world of cyberspace that was ours for the making. The connection wasnt just metaphorical. The emerging Internet culture of the time was heavily influenced by the New Communalism movement of the 1960s, as Fred Turner writes in From Counterculture to Cyberculture, a history of digital utopianism. The early Internet was deeply groovy.

By my junior year, I could make a web page or spin up a web server or write code in six different programming languages. For an undergraduate majoring in math, computer science, or engineering at the time, this was completely normal. For a woman, it wasnt. I was one of six undergraduate women majoring in computer science at a university of twenty thousand graduate and undergraduate students. I only knew two of the other women in computer science. The other three felt like a rumor. I felt isolated in all of the textbook ways that cause women to drop out of science, technology, engineering, and mathematics (STEM) careers. I could see what was broken inside the system, for me and for other women, but I didnt have the power to fix it. I switched my major.