Ben Buchanan - The New Fire: War, Peace, and Democracy in the Age of AI

Ben Buchanan and Andrew Imbrie

The MIT Press

Cambridge, Massachusetts

London, England

2022 Benjamin Louis Buchanan and William Andrew Imbrie

All rights reserved. No part of this book may be reproduced in any form by any electronic or mechanical means (including photocopying, recording, or information storage and retrieval) without permission in writing from the publisher.

The MIT Press would like to thank the anonymous peer reviewers who provided comments on drafts of this book. The generous work of academic experts is essential for establishing the authority and quality of our publications. We acknowledge with gratitude the contributions of these otherwise uncredited readers.

This book was set in ITC Stone and Avenir by New Best-set Typesetters Ltd.

Library of Congress Cataloging-in-Publication Data

Names: Buchanan, Ben, author. | Imbrie, Andrew, author.

Title: The new fire : war, peace, and Democracy in the age of AI / Ben Buchanan and Andrew Imbrie.

Other titles: War, peace, and Democracy in the age of artificial intelligence

Description: Cambridge, Massachusetts : The MIT Press, [2022] | Includes bibliographical references and index.

Identifiers: LCCN 2021013280 | ISBN 9780262046541 (Hardcover)

Subjects: LCSH: Artificial intelligence.

Classification: LCC Q335 .B795 2022 | DDC 006.3dc23

LC record available at https://lccn.loc.gov/2021013280

10 9 8 7 6 5 4 3 2 1

d_r0

For Mary

and

For Teresa

Fire begins with a tiny spark. The spark can come from two sticks rubbed together, a smoldering match left astray, an electrical fault, or any number of innocuous yet volatile triggers. The spark ignites and, with the right fuel, grows exponentially. What was once a nearly invisible flame can quickly turn into a small blaze, and then explode into a wildfire that rips across landscapes and destroys everything in its path. Fires power comes not just from its heat and smoke but also from its acceleration in speed and force; unchecked, it will rage and rage until there is nothing left to burn.

And yet, for all the destruction that fire can bring, it is also the basis for civilization. From its earliest days, humanitys capacity to start, control, and stop fires was the prerequisite for warming caves and everything that followed. Many of the most consequential inventionsmetal forging, steam propulsion, glassmaking, and electricityarose from this ability. Some scientists theorize that even the development of the human brain itself depended on cooking meat over flames, which meant that food required less energy to digest. In so many ways, human beings learned to harness fires exponential power, using its might for good and taming its dangers.

Humanity has also wielded fires destructive force. The Byzantine Empire used it to great military success, first during the siege of Constantinople in 672 AD, and then in the centuries that followed. In battle, Byzantine troops shot a specially formulated compound at their enemies, one that would burn even when it came into contact with water. Once the compound hit the target, the power of fire would kick in, torching enemy equipment and causing soldiers to flee. Since then, the flames of war have only become deadlier.

Could there ever be another force so productive and perilous, one so essentially defined by the exponential growth of its core components?

Welcome to the age of artificial intelligence.

In choosing the metaphor of fire to explain this age, we reject the more common and more hopeful assertion that AI is the new electricity. Electricity is everywhere in modern life, so ubiquitous and so utterly mastered by our society that we take its safety for granted. Without a second thought, we dwell and work side by side with the massive buzzing wires that crisscross our neighborhoods and empower our lives. Largely because of the professionalization of electricity generation, transmission, and usage, nearly every modern human interaction with electrical current is a beneficial one. AI will follow the same path, so the electricity metaphor implies, transforming society no less profoundly and no less positively.

While that may be true one day, for now this view is far too rosy. We have not come close to mastering AI as we mastered electricity. Today, we encounter AI as our distant ancestors once encountered fire. If we manage this technology well, it will become a tremendous force for global good, lighting the way to many transformative inventions. If we deploy it too quickly and without adequate foresight, AI will burn in ways we cannot control. If we harness it to destroy, it will enable more powerful weapons for the strongest governments on Earth as they engage one another in a combustible geopolitical competition. That the frequent analogy to electricity denies this wide range of possible outcomes doesnt make us more secure, only less prepared.

Three sparks ignite the new fire: data, algorithms, and computing power. Todays AI systems use computing power to execute algorithms that instruct machines how to learn from data. Due to exponential growth in the size of data sets, the capability of algorithms, and the power of computers, the age of AI has brought advances that have stunned even some skeptics.

AI has demonstrated superb capability in areas ranging from speech and facial recognition to video generation, language translation, storytelling, and beyond. As a result of these advances, AI has come into our homes and our businesses. It empowers Siri and Alexa, provides recommendations as we navigate the web, helps steer our cars on the highway, and silently makes so many of the technological systems we use every day work better than before. Even more impressively, AI systems can now mimic decidedly human qualities, such as imagination and intuition. In rapid succession, these systems have reached milestonesincluding success at games that require fiendishly complex strategic planning and breakthroughs in some of the hardest science problemsthat experts had once thought were more than a decade away. It is overwhelmingly likely that even more surprising and powerful AI advances will come in the years ahead.

AIs new capabilities are both marvels and distractions. Too often, the discussion of AI focuses on what the technology can do, overlooking the people who invent, refine, and deploy it. Commentators make breathless predictions about the arrival of superintelligence capable of exceeding humanitys cognitive capabilities, but far less often do they consider how AI interacts with geopolitics. Any inquiry into AI must focus on ushow we harness the new fire and for what purpose.

This book is about human choices. The decisions people in governments, companies, and universities make about AI will prove to be some of the most consequential of this young century, both technologically and geopolitically. AI compels us to ask anew some fundamental questions about humanity: How do we govern? How do we fight? And, perhaps most importantly, what do we fight for? We present three competing and sometimes overlapping viewsthe evangelists, warriors, and Cassandrasfor what to do about this new technology.

The evangelists believe that humanity should tend the new fire for the benefit of all. Those who hold this view believe that AI can transform human civilization for the better, and worry that using AI as a weapon of war and geopolitics distracts from all the good it can do for civilization. They are often AI researchers and pioneers themselves, motivated by the desire to solve intelligence as a means of cracking some of the thorniest mysteries in science. At a time when scientific discovery and invention seem more difficult than ever before, evangelists want to build machines that can see, create, plan, and aid humanity in reaching its full potential. Just like the discovery of fire, discoveries in AIor by AIwill unlock a new standard of human life and a new level of understanding. If we manage it well, the evangelists believe, the new fires exponential force can propel us all forward and offer positive-sum benefits for everyone.