Brian Christian - The Alignment Problem

MACHINE LEARNING

AND HUMAN VALUES

BRIAN CHRISTIAN

A Norton Professional Book

This e-book contains some places that ask the reader to fill in questions or comments. Please keep pen and paper handy as you read this e-book so that you can complete the exercises within.

For Peter

who convinced me

And for everyone

doing the work

I remember in 2000 hearing James Martin, the leader of the Viking missions to Mars, saying that his job as a spacecraft engineer was not to land on Mars, but to land on the model of Mars provided by the geologists.

PETER NORVIG

The world is its own best model.

RODNEY BROOKS

All models are wrong.

GEORGE BOX

The

Alignment

Problem

935, Detroit. Walter Pitts is running down the street, chased by bullies.

He ducks into the public library to take shelter, and he hides. He hides so well that the library staff dont even realize hes there, and they close for the night. Walter Pitts is locked inside.

He finds a book on the shelves that looks interesting, and he starts reading it. For three days, he reads the book cover to cover.

The book is a two-thousand-page treatise on formal logic; famously, its proof that 1+1=2 does not appear until Pitts decides to write a letter to one of the authorsBritish philosopher Bertrand Russellbecause he believes hes found several mistakes.

Several weeks go by, and Pitts gets a letter in the mail postmarked from England. Its Bertrand Russell. Russell thanks him for writing, and invites Pitts to become one of his doctoral students at Cambridge.

Unfortunately, Walter Pitts must decline the offerbecause hes only twelve years old, and in the seventh grade.

Three years later, Pitts learns that Russell will be visiting Chicago to give a public lecture. He runs away from home to attend. He never goes back.

A t Russells lecture, Pitts meets another teenager in the audience, named Jerry Lettvin. Pitts only cares about logic. Lettvin only cares about poetry and, a distant second, medicine. They become inseparable best friends.

Pitts begins hanging out around the University of Chicago campus, dropping in on classes; he still lacks a high school diploma and never formally enrolls. One of these classes is by the famed German logician Rudolf Carnap. Pitts walks into his office hours, declaring hes found a few flaws in Carnaps latest book. Skeptically, Carnap consults the book; Pitts, of course, is right. They talk awhile, then Pitts walks out without giving his name. Carnap spends months asking around about the newsboy who knew logic. Eventually Carnap finds him again and, in what will become a motif throughout Pittss academic life, becomes his advocate, persuading the university to give him a menial job so he will at least have some income.

Its now 1941. Lettvinstill a poet first, in his own mindhas, despite himself, gotten into medical school at the University of Illinois, and finds himself working under the brilliant neurologist Warren McCulloch, newly arrived from Yale. One day Lettvin invites Pitts over to meet him. At this point Lettvin is twenty-one and still living with his parents. Pitts is seventeen and homeless. McCulloch and his wife take them both in.

Throughout the year that follows, McCulloch comes home in the evenings and he and Pittswho is barely older than McCullochs own childrenregularly stay up past midnight talking. Intellectually, they are the perfect team: the esteemed midcareer neurologist and the prodigy logician. One lives in practicethe world of nervous systems and neurosesand the other lives in theorythe world of symbols and proofs. They both want nothing more than to understand the nature of truth: what it is, and how we know it. The fulcrum of this questthe thing that sits at the perfect intersection of their two disparate worldsis, of course, the brain.

It was already known by the early 1940s that the brain is built of neurons wired together, and that each neuron has inputs (dendrites) as well as an output (axon). When the impulses coming into a neuron exceed a certain threshold, then that neuron, in turn, emits a pulse. Immediately this begins to feel, to McCulloch and Pitts, like logic: the pulse or its absence signifying on or off, yes or no, true or false.

They realize that a neuron with a low-enough threshold, such that it would fire if any of its inputs did, functioned like a physical embodiment of the logical or. A neuron with a high-enough threshold, such that it would only fire if all of its inputs did, was a physical embodiment of the logical and. There was nothing, then, that could be done with logicthey start to realizethat such a neural network, so long as it was wired appropriately, could not do.

Within months they have written a paper togetherthe middle-aged neurologist and teenage logician. They call it A Logical Calculus of Ideas Immanent in Nervous Activity.

Because of the all-or-none character of nervous activity, they write, neural events and the relations among them can be treated by means of propositional logic. It is found that the behavior of every net can be described in these terms... and that for any logical expression satisfying certain conditions, one can find a net behaving in the fashion it describes.

The paper is published in 1943 in the Bulletin of Mathematical Biophysics. To Lettvins frustration, it makes little impact on the biology community. To Pittss disappointment, the neuroscience work of the 1950s, notably a landmark study of the optic nerve of the frogdone by none other than his best friend, Jerry Lettvinwill show that neurons appear to be much messier than the simple true or false circuits he envisioned. Perhaps propositional logicits ands, ors, and notswas not, ultimately, the language of the brain, or at least not in so straightforward a form. This kind of impurity saddened Pitts.

But the impact of the paperof those long conversations into the night at McCullochs housewould be enormous, if not entirely in the way that McCulloch and Pitts envisioned. It would be the foundation for a completely new field: the project to actually build mechanisms out of these simplified versions of neurons, and see just what such mechanical brains could do.

I n the summer of 2013, an innocuous post appeared on Googles open-source blog titled Learning the Meaning Behind Words.

Today computers arent very good at understanding human language, it began. While state-of-the-art technology is still a ways from this goal, were making significant progress using the latest machine learning and natural language processing techniques.

Google had fed enormous datasets of human language, mined from newspapers and the internetin fact, thousands of times more text than had ever been successfully used beforeinto a biologically inspired neural network, and let the system pore over the sentences for correlations and connections between the terms.

The system, using so-called unsupervised learning, began noticing patterns. It noticed, for instance, that the word Beijing (whatever that meant) had the same relationship to the word China (whatever that was) as the word Moscow did to Russia.