Osvaldo Martin - Bayesian Analysis with Python

Probabilistic programming is a framework that allows you to flexibly build Bayesian statistical models in computer code. Once built, powerful inference algorithms that work independently of the model you formulated can be used to fit your model to data. This combination of flexible model specification and automatic inference provides a powerful tool for the researcher to quickly build, analyze, and iteratively improve novel statistical models. This iterative approach is in stark contrast to the way Bayesian models were fitted to data before: previous inference algorithms usually only worked for one specific model. Not only did this require strong mathematical skills to formulate the model and devise an inference scheme, it also considerably slowed down the iterative cycle: change the model, re-derive your inference. Probabilistic programming thus democratizes statistical modeling by considerably lowering the mathematical understanding and time required to successfully build novel models and gain unique insights into your data.

The idea behind probabilistic programming is not new: BUGS, the first of its kind, was first released in 1989. The kinds of model that could be fitted successfully were extremely limited and inference was slow, rendering these first-generation languages not very practical. Today, there are a multitude of probabilistic programming languages that are widely used in academia and at companies such as Google, Microsoft, Amazon, Facebook, and Uber to solve large and complex problems. What has changed? The key factor in lifting probabilistic programming from being a cute toy to the powerful engine that can solve complex large-scale problems is the advent of Hamiltonian Monte Carlo samplers, which are several orders of magnitude more powerful than previous sampling algorithms. While originally devised in 1987, only the more recent probabilistic programming systems named Stan and PyMC3 made these samplers widely available and usable.

This book will give you a practical introduction to this extremely powerful and flexible tool. It will have a big impact on how you think about and solve complex analytical problems. There are few people better suited to have written it than PyMC3 core developer Osvaldo Martin. Osvaldo has the rare talent of breaking complex topics down to make them easily digestible. His deep practical understanding, gained through hard-won experience, allows him to take you, the reader, on the most efficient route through this terrain, which could otherwise easily seem impenetrable. The visualizations and code examples make this book an eminently practicable resource through which you will gain an intuitive understanding of their theoretical underpinnings.

I also would like to commend you, dear reader, for having picked up this book. It is not the fast and easy route. In a time where headlines advertise deep learning as the technique to solve all current and future analytical problems, the more careful and deliberate approach of building a custom model for a specific purpose might not seem quite as attractive. However, you will be able to solve problems that can hardly be solved any other way.

This is not to say that deep learning is not an extremely exciting technique. In fact, probabilistic programming itself is not constrained to classic statistical models. Reading the current machine learning literature, you will find that Bayesian statistics is emerging as a powerful framework to express and understand next-generation deep neural networks. This book will thus equip you not only with the skills to solve hard analytical problems, but also to have a front-row seat in humanity's perhaps greatest endeavor: the development of artificial intelligence. Enjoy!

Thomas Wiecki, PhD

Head of Research at Quantopian.