Jeremy Weinstein - System Error: Where Big Tech Went Wrong and How We Can Reboot

To our amazing children

In times of crisis, ordinary citizens, confused and disoriented, settling into paralysis, can come to believe that, as Plato had argued, they are not up to the job of making difficult decisions. In hard times, democratic citizens may become more willing to hand over the business of politics to experts and to abandon the institutional frameworks, the rights and liberties, that secure their position as participants in the political process. The danger of intellectual paralysis in the face of chaos is finally that it undermines the first premise of democracy: namely, that ordinary citizens will always be ready to think.

Danielle Allen, Aims of Education Address, September 20, 2001

On January 6, 2021, the US Capitol was stormed by insurrectionists who had been whipped into action at a rally earlier that day featuring President Donald Trump. Their goal was to violently overturn the result of the presidential election, which they had been falsely told for weeks had been stolen. That message had been delivered most prominently by Trump himself, despite more than sixty failed lawsuits challenging election results and thorough refutations by election officials across the country.

The big tech platforms had been a key conduit for the accusations of election fraud for months before. On January 6, they finally woke up to the horror they had enabled. Twitter locked Trumps account, which had nearly 90 million followers, denying him permission to post. Two days later, citing a risk of further incitement of violence, Twitter permanently banished Trump from the platform, erasing everything on his account in one fell swoop. Similar suspensions took place on Facebook, Instagram, YouTube, and Snapchat. Trump turned to the still active @POTUS Twitter account and posted that he had been SILENCED! before that tweet was quickly removed by the platform as well.

The platforms rebuke of Trumps election disinformation was also an alarm bell about how much power is concentrated in the hands of a few big tech companies. The president of the United Statesoften touted as the leader of the free worldwas unceremoniously stripped of his favorite means of communicating with his tens of millions of followers. Whether that was a necessary step to reduce the possibility of further violence after the election, a long-overdue decision by the platforms to take away a megaphone from a man whose history of lies far pre-dated the 2020 election, or the tech elites blatant censorship of the highest elected official in the United States, it cast an unmistakable light on the extraordinary power that technology and, more significantly, the people who develop it have over us.

Big techs role in and reaction to the events that led to the storming of the US Capitol only highlight the concerns about technology that have been mounting for years. Seemingly endless reports of privacy breaches and stories of behavior manipulation resulting from vast troves of data mined by large companies have made it commonplace to view big tech through a dark lens. Some argue that the internet, smartphones, and computers have delivered to us a set of devices hell-bent on hijacking our attention and addicting us to the screen, while gathering ever more data of our online behavior. And as borne out at the Capitol, a tidal wave of misinformation and disinformation on social media platforms has served to undermine our trust in science, exacerbate political polarization, and threaten democracy itselfall of this powered by a small number of companies with immense market power and growing political influence.

At this same unprecedented time, we experienced the COVID-19 pandemic, which as of this writing has taken more than 3 million lives worldwide while upending work, education, the economy, and our personal lives. The pandemic caused one of those rare moments of instantaneous behavior change with extraordinary long-term implications. Vladimir Lenin is alleged to have said, There are decades when nothing happens, and then there are weeks where decades happen. Overnight, much of the world shifted to working from home and schools closed as public health authorities imposed social distancing rules and in some areas shelter-in-place orders. Videoconferencing soared as air travel ground to a halt. Technologies for file sharing and workplace collaboration enabled many aspects of the economy to proceed apace. People flocked in record numbers to Netflix as a substitute for movie theaters. The use of Facebook and other social media networks skyrocketed as people sought connections to friends and family. Videoconferencing enabled children to keep attending school and people to retain a connection to their loved ones when it wasnt possible to be together physically. And tech companies across the board stepped up to foreground authoritative scientific information about the pandemic, develop contact-tracing apps to help contain it, and deploy artificial intelligence to hasten the development of medical treatments and potential vaccines and to power robots to handle tasks such as delivering medication to sick hospital patients.

In short, our professional and personal lives, our economy and intimate relationships, and even our health would have been far worse without the internet and our familiar addictive devices.

As we exit the COVID-19 pandemic and enter a new political moment, the window is finally opening for a mature consideration of technology, one that avoids both the technoboosterism that accompanied its early decades and the techlash that has followed.

Sure, there remain plenty of criticisms to be made of Facebook, the privacy policies of Zoom, the acceleration of automation in an age of smart machines without regard for job displacement, and the toxic misinformation and disinformation flowing through social media platforms. But that just underscores the essential work of our new post-pandemic era. We must strive now to find ways to harness the power of technology to deliver its considerable benefits while diminishing its equally apparent harms to individuals and societies. We now possess the wisdom to see technological innovation as something other than an external force that works upon us. The path of technological development and the effects of technology on us are things we can shape. Things we must shape.

When we uncritically celebrate technology or unthinkingly criticize it, the end result is to leave technologists in charge of our future. This book was written to provide an understanding of how we as individuals, and especially together as citizens in a democracy, can exercise our agency, reinvigorate our democracy, and direct the digital revolution to serve our best interests.

* * *

For the past twenty years, we have been teaching at Stanford University, the seedbed of Silicon Valley. It is a research powerhouse with numerous Nobel laureates, MacArthur Foundation geniuses, and Pulitzer Prizewinning writers to rival the best. But behind the faade of this paradisiacal, self-professed nerd nation, we started to observe some concerning patterns.

Innovation and disruption were the buzzwords on campus, and our students broadcasted an almost utopian view that the old ways of doing things were broken and technology was the all-powerful solution: it could end poverty, fix racism, equalize opportunity, strengthen democracy, and even help topple authoritarian regimes. Every year at new-student orientation, one of our students told us enthusiastically, we bring in some tech billionaire who is held up as the paragon of what you can achieve and that thats the life you should want. The former president of the university was heard to say that government was incompetent and the idea of encouraging any student to go into government service in order to make a difference was ridiculous.