Jean Walrand - Probability in Electrical Engineering and Computer Science: An Application-Driven Course

This book is an open access publication.

Open Access This book is licensed under the terms of the Creative Commons Attribution 4.0 International License ( http://creativecommons.org/licenses/by/4.0/ ), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

The images or other third party material in this book are included in the book's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the book's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

This Springer imprint is published by the registered company Springer Nature Switzerland AG

The registered company address is: Gewerbestrasse 11, 6330 Cham, Switzerland

This book is about extracting information from noisy data, making decisions that have uncertain consequences, and mitigating the potentially detrimental effects of uncertainty.

Applications of those ideas are prevalent in computer science and electrical engineering: digital communication, GPS, self-driving cars, voice recognition, natural language processing, face recognition, computational biology, medical tests, radar systems, games of chance, investments, data science, machine learning, artificial intelligence, and countless (in a colloquial sense) others.

This material is truly exciting and fun. I hope you will share my enthusiasm for the ideas.

This book is about applications of probability in electrical engineering and computer science. It is not a survey of all the important applications. That would be too ambitious. Rather, the course describes real, important, and representative applications that make use of a fairly wide range of probability concepts and techniques.

Probabilistic modeling and analysis are essential skills for computer scientists and electrical engineers. These skills are as important as calculus and discrete mathematics. The systems that these scientists and engineers use and/or design are complex and operate in an uncertain environment. Understanding and quantifying the impact of this uncertainty is critical to the design of systems.

The book was written for the upper-division course EECS126 Probability in EECS in the Department of Electrical Engineering and Computer Sciences of the University of California, Berkeley. The students have taken an elementary course on probability. They know the concepts of event, probability, conditional probability, Bayes rule, discrete random variables and their expectation. They also have some basic familiarity with matrix operations. The students in this class are smart, hard-working, and interested in clever and sophisticated ideas. After taking this course, the students are familiar with Markov chains, stochastic dynamic programming, detection, and estimation. They have both an intuitive understanding and a working knowledge of these concepts and their methods. Subsequently, many students go on to study artificial intelligence and machine learning. This course provides them with a background that enables them to go beyond blindly using toolboxes.

In contrast to most introductory books on probability, the material is organized by applications. Instead of the usual sequenceprobability space, random variables, expectation, detection, estimation, Markov chainswe start each topic with a concrete, real, and important EECS application. We introduce the theory as it is needed to study the applications. We believe that this approach makes the theory more relevant by demonstrating its usefulness as it is introduced. Moreover, an emphasis is on hands-on projects where the students use Python notebooks available from the book website to simulate and calculate. Our colleagues at Berkeley designed these projects carefully to reinforce the intuitive understanding of the concepts and to prepare the students for their own investigations.

The chapters, except for the last one and the appendices, are divided into two parts: A and B. Parts A contain the key ideas that should be accessible to junior-level students. Parts B contain more difficult aspects of the material. It is possible to teach only the appendices and parts A. This would constitute a good junior-level course. One possible approach is to teach parts A in a first course and parts B in a second course. For a more ambitious course, one may teach parts A, then parts B. It is also possible to teach the chapters in order. The last chapter is a collection of more advanced topics that the reader and instructor can choose from.

The appendices should be useful for most readers. Appendix A discusses the elementary notions of probability on simple examples. Students might benefit from a quick read of this chapter.

Appendix B reviews the basic concepts of probability. Depending on the background of the students, it may be recommended to start the course with a review of that appendix.