Leonida Gianfagna - Explainable AI Using Python

This Springer imprint is published by the registered company Springer Nature Switzerland AG

The registered company address is: Gewerbestrasse 11, 6330 Cham, Switzerland

For our purposes we place the birth of AI with the seminal work of Alan Turing () in which the author posed the question Can machines think? and the later famous mental experiment proposed by Searle called the Chinese Room.

The point is simple: suppose to have a black-box-based AI system that pretends to speak Chinese in the sense that it can receive questions in Chinese and provide answers in Chinese. Assume also that this agent may pass a Turing test that means it is indistinguishable from a real person that speaks Chinese. Would we be fine on saying that this AI system is capable of speaking Chinese as well? Or do we want more? Do we want the black box to explain itself clarifying some Chinese language grammar?

So, the root of Explainable AI was at the very beginning of Artificial Intelligence, albeit not in the current form as a specific discipline. The key to trust the system as a real Chinese speaker would be to make the system less opaque and explainable as a further requirement besides getting proper answers.

Jumping to our days, it is worth to mention the statement of GO champion Fan Hui commenting the famous 37th move of AlphaGo , the software developed by Google to play GO, that defeated in March 2016 the Korean champion Lee Sedol with a historical result: Its not a human move, Ive never seen a man playing such a move (Metz ). GO is known as a computationally complex game, more complex than chess, and before this result, the common understanding was that it was not a game suitable for a machine to play successfully. But for our purposes and to start this journey, we need to focus on Fan Huis quoted statement. The GO champion could not make sense of the move even after having looked at all the match; he recognized it as brilliant, but he had no way to provide an explanation. So, we have an AI system (AlphaGo) that performed very well (defeating the GO champion), but no explanation of how it worked to win the game; that is where Explainable AI inside the wider Machine Learning and Artificial Intelligence starts to play a critical role.

Before presenting the full landscape, we will give some examples that are less sensationalistic but more practical in terms of understanding what we mean by the fact that most of the current Machine Learning models are opaque and not explainable. And the fil rouge of the book will be to learn in practice leveraging different methods and how to make ML models explainable, that is, to answer the questions What, How, and Why on the results.

Explainable AI (aka XAI) is more than just a buzz word, but it is not easy to provide a definition that includes the different angles to look at the term. Basically speaking, XAI is a set of methods and tools that can be adopted to make ML models understandable to human beings in terms of providing explanations on the results provided by the ML models elaboration.

One of the most brilliant successes of modern Deep Learning techniques against the traditional approach comes from computer vision. We can train a convolutional neural network (CNN) to understand the difference between different classes of labelled pictures. The applications are probably infinite: we can train a model to discriminate between different kinds of pneumonia RX pictures or teach it to translate sign language into speech. But are the results truly reliable?

Lets follow a famous toy task in which we have to classify pictures of wolves and dogs (Fig. ).

Fig. 1.1

ML classification of wolves and dogs (Singh )

After the training, the algorithm learned to distinguish the classes with remarkable accuracy: only a misclassification over 100 images! But if we use an Explainable AI method asking the model Why have you predicted wolf?, the answer will be with a little of surprise because there is snow! (Ribeiro et al. ).