Carlos Montemayor - The Prospect of a Humanitarian Artificial Intelligence: Agency and Value Alignment

The Prospect of a Humanitarian Artificial Intelligence

Also available from Bloomsbury:

Great Philosophical Objections to Artificial Intelligence, by Eric Dietrich, Chris Fields, John P. Sullins, Bram Van Heuveln, and Robin Zebrowski

Philosophy in a Technological World, by James Tartaglia

The Evolution of Consciousness, by Paula Droege

The Human Mind through the Lens of Language, by Nirmalangshu Mukherji

The Philosophy and Science of Predictive Processing, edited by Dina Mendona, Manuel Curado, and Steven S. Gouveia

The Prospect of a Humanitarian Artificial Intelligence

Agency and Value Alignment

Carlos Montemayor

Contents

We are an adventurous and curious species. Our intelligence is our point of pride and the basis of our endless interests and achievements. The various needs and motivations we developed as we evolved have made us both mighty and vulnerable. We inherited our intelligence from an astonishingly diverse lineage, and we share much of our intelligence with other species. Yet our intelligence is unique in its range and power. What is more, our collective intelligence has profoundly shaped the planet, and not always to the good. For instance, without the scientific and industrial revolutions, nuclear weapons would never have been developed. The destructive by-products of the recent expansion and mechanization of our intelligence are part of our vulnerability.

This book centers on the possibility of creating another colossal industrial achievement, just as threatening and awesome as the atomic bomb: the set of computational techniques that may result in artificial intelligence (AI). This possibility does not exist at present, and we do not know how likely is the prospect of its existence. What we know is that the industrial revolution would be dwarfed, a small step in our history, compared to this potential development. Ironically, AI might also decisively demote our species by making us less intelligent than our creation, and this has been identified as the most important risk surrounding AI: the so-called singularity, or the moment when we are left behind (far behind, some fear), outpaced by our newly created super-intelligent systems. Some have posited that we may find a solution in value alignment, in ensuring that the computational techniques we create reflect our ethics and norms, ensuring that it will be beneficial rather than destructive. The value alignment problem is a major theme of this book.

Value alignment appears to be the best answer to AI critics vague predictions of doom and demotion. A key contribution of this book is to note that there is no unique value alignment problem, but a variety of them. That is because there are various forms of value alignment, which must be carefully distinguished from one another. These distinct alignments require meaningful specifications dependent on contextual information and skills for coordinated action in different normative domains (moral, epistemic, and political). Human attunement to what is salient, evidentially important, or morally relevant depends on how we align our attention. These attentive alignments allow us to triangulate with each other with respect to what we value. The book, however, does not merely focus on how to avoid the singularity through strategies for value alignment with AI. A central concern is the control and powerepistemic, social, and politicalthat AI research is already exerting and accumulating.

How probable is it that genuinely intelligent artificial agents will be created? A clear answer to this question remains elusive, though commentators have speculated with tremendous enthusiasm. Echoing William James opinion about the state of psychology at his time, Terry Winograd said in 1977 that developments in artificial intelligence were akin to alchemywe combine, mix, and scale up, in the hopes of finding the holy grail, but without knowing when or how exactly we will succeed. More recently, Melanie Mitchell has returned to the question, whether todays AI research still resembles alchemy in Why AI Is Harder than We Think. Her verdict is that, despite enormous progress, large financial investments, and public media hype, AI research more than forty years after Winograds remark may still be industrial alchemynothing more than a utopian hope.

Mitchell presents several methodological fallacies in AI research that impede its development. One of them, wishful mnemonics, is particularly relevant for this book: AI researchers import terms from philosophy, psychology, and neuroscience (e.g., neural networks, attention, intelligence, knowledge, truth) without any real justification. This is a kind of wishful labeling that would, by alchemical magic, turn massive data mining and layered statistical processing into thoughtful agency. Gratuitous use of psychological terms in descriptions of machine output muddles our evaluations of AI, and confuses everyone with respect to the real nature and potential of AI technology. This book proposes a useful way to categorize different kinds of AI, without simply assuming that psychological terms can be directly applied to all of them equally.

However, if and when AI is developed, the consequences will be dramatic. We need to prepare ourselves for this eventuality by making our technology more beneficial to all humans, starting with the technology we are using now. AI research technology is not benefiting humanity in any clear or measurable way, and on the contrary, it has been misused and can potentially become quite dangerous. An important reference point here is Kate Crawfords book, Atlas of AI: Power, Politics, and the Planetary Costs of Artificial Intelligence. As Crawford forcefully argues, the current investment frenzy in AI technologies is profoundly troubling because of its economic, environmental, social, and political costs. Power and justice are also a central concern of my examination of AI.

As a matter of basic fairness, if intelligence is developed industrially, then it should be used for the benefit of all humanity. It should also be accessible in various ways to all human beings, and not be the exclusive property of a few powerful impresarios or politicians who might keep AI as an industrial or military secret, which is consistent with what is currently happening. This stolen intelligence could potentially be used against all of us. Such a scenario presents a more perverse kind of being left behind, a political maneuver to demote not ultra-rich or ultra-powerful humans by expelling them from the production of knowledge. Instead of a terminator scenario in which machines take over, here we have a more standard form of unfairness or business as usual. As a countermeasure to this possibility, this book argues in favor of a humanitarian approach to the development of AI technology, based on the existent international framework for the protection of human rights. These problems are urgent, whether or not the pursuit of AI turns out to be a form of alchemy, and whether or not AI agents become genuinely intelligent.

Another goal of this book is to demonstrate that we need to distinguish between formidable predictive computing that looks like intelligence and genuinely intelligent agency, a goal that can only be achieved by firmly grounding our arguments in philosophical considerations. With respect to psychology, critical distinctions are drawn from the empirical and philosophical literature on intelligence, attention, consciousness, knowledge, and emotion. Value theory, as well as political and legal philosophy, informs the books proposals concerning the value alignment problem, based on an analysis of human needs. Types of intelligence are examined according to their normative, or value-grounding roles. The importance of control and motivation for intelligent behavior is examined according to basic human needs, which are then shown to be critical in our understanding of human rights and human dignity.