Ethem Alpaydin - Machine Learning, revised and updated edition

The MIT Press Essential Knowledge Series

A complete list of the titles in this series appears at the back of this book.

Ethem Alpaydin

The MIT Press | Cambridge, Massachusetts | London, England

2021 Massachusetts Institute of Technology

All rights reserved. No part of this book may be reproduced in any form by any electronic or mechanical means (including photocopying, recording, or information storage and retrieval) without permission in writing from the publisher.

This book was set in Chaparral Pro by New Best-set Typesetters Ltd.

Library of Congress Cataloging-in-Publication Data

Names: Alpaydn, Ethem, author.

Title: Machine learning / Ethem Alpaydn.

Description: Revised and updated edition. | Cambridge, Massachusetts : The MIT Press, [2021] | Series: The MIT Press essential knowledge series | Includes bibliographical references and index.

Identifiers: LCCN 2020033697 | ISBN 9780262542524 (paperback)

Subjects: LCSH: Machine learning. | Artificial intelligence.

Classification: LCC Q325.5 .A47 2021 | DDC 006.3/1dc23

LC record available at https://lccn.loc.gov/2020033697

10 9 8 7 6 5 4 3 2 1

d_r0

The MIT Press Essential Knowledge series offers accessible, concise, beautifully produced pocket-size books on topics of current interest. Written by leading thinkers, the books in this series deliver expert overviews of subjects that range from the cultural and the historical to the scientific and the technical.

In todays era of instant information gratification, we have ready access to opinions, rationalizations, and superficial descriptions. Much harder to come by is the foundational knowledge that informs a principled understanding of the world. Essential Knowledge books fill that need. Synthesizing specialized subject matter for nonspecialists and engaging critical topics through fundamentals, each of these compact volumes offers readers a point of access to complex ideas.

In November 2019, South Korean Go master Lee Se-dol announced that, after a career of twenty-four years, he was retiring from professional Go competitions. In 2016, he had played a five-game series against a computer program named AlphaGo, which he lost 1 to 4. Since then, later versions of AlphaGo had gotten even better, so much so that when announcing his retirement, Se-dol said that, With the debut of AI in Go games, Ive realized that Im not at the top even if I become the number one through frantic efforts. Even if I become the number one, there is an entity that cannot be defeated.

The ancient strategy game Go had long been believed to be beyond the capability of AI. In 1997, when the chess-playing program Deep Blue defeated the reigning world champion Garry Kasparov, researchers believed that it would take another generation for the same to happen with Go. While the chess board is 8 8, the Go board is 19 19a larger board meaning so many more possible positionings of the pieces on the board exist, hence so many more different ways of playing the game that a game-playing computer program should be able to handle.

The crucial difference between Deep Blue and AlphaGo was the shift from programming to learning. Whereas Deep Blue was programmed by human experts to play as well as possible, AlphaGo learned to play well by playing many games and updating itself using this experience, favoring moves and strategies that led to winning the game and penalizing those that led to losses.

How this is done is the topic of this book, and as we will see, game playing is only one of the many domains where we have witnessed this unforeseen sudden jump in ability through learning. In the last two decades, using systems that learn, we have seen drastic improvements in accuracy in various applications that have since been successfully commercialized. We now have programs that can recognize people from their faces, understand spoken speech, recommend a movie, translate text from one language to another, and drive a carall of which have been made possible by machine learning.

Once, it used to be the programmer who had to come up with a way to solve the problem; the sequence of operations that needs to be carried out is named an algorithm. The algorithm is then coded as a program using a programming language, and the program is executed on a computer. In a learning program, on the other hand, the programmer specifies how the data (collected through experience) is used to update the program so as to improve performance; it is the data that determines the final form of the program.

In a programmed system, the programmer knows at the time of writing the program how the system is going to behave in any situation. The program has no intelligence by itself; it is just a machine that is hardwired to duplicate the intelligence of the programmer. It just does what the programmer would do; its only advantage may be its speed; it is no more than a calculator.

With learning, however, how a system will act in a situation is the result of the interaction between the learning program and the data, and as we will see, the final system very much depends on the quantity and quality of the data (i.e., how well the data covers all possible scenarios). In such a case, how a trained program will act cannot be foreseen by the programmer at the time of writing the program, and as such it can be said that a program that has learned from data has acquired intelligence beyond that of the programmer.

In retrospect, it is not surprising that the learning program AlphaGo defeated Lee Se-dol. AlphaGo played (and learned from) many more games than any human being can play in a lifetime. Likewise, a doctor gains experience from their own patients only; a learning medical diagnosis system can be trained with the collection of patients of thousands of doctors. Similarly, a car that learns to drive itself can be trained with many more and much more varied scenarios than even the most experienced human driver can encounter in a lifetime. That is the advantage of collecting big data and analyzing it to infer knowledge.

Of course, learning from data is not new; it is at the heart of science. In the past, scientists like Galileo and Kepler designed experiments to make observations and collected data; they then came up with laws that explain those data. In medicine, cures for many diseases were found by collecting information from patients and analyzing them for commonalities and differences. But we are now at a point where we want to automate this process of going from data to knowledge, because now we have much more data and many more application domains.

Since the advent of computers in the middle of the last century, our lives have become increasingly computerized and digital. Computers are no longer just the numeric calculators they once were. Databases and digital media have taken the place of printing on paper as the main medium of information storage, and digital communication over computer networks supplanted the post as the main mode of information transfer. First with the personal computer with its easy-to-use graphical interface, and then with the phone and other smart devices, the computer has become a ubiquitous device, a household appliance just like the TV or the microwave. Currently, all sorts of information, not only numbers and text but also image, video, audio, and so on, are stored, processed, andthanks to online connectivitytransferred digitally. All this digital processing results in a lot of data, and it is this surge of data that is mainly responsible for triggering the widespread interest in data analysis and machine learning.