Howard Wainer - Picturing the Uncertain World: How to Understand, Communicate, and Control Uncertainty Through Graphical Display

Picturing the Uncertain World

Picturing the Uncertain World

How to Understand, Communicate,
and Control Uncertainty through
Graphical Display

Howard Wainer

PRINCETON UNIVERSITY PRESS

Princeton & Oxford

Copyright 2009 by Princeton University Press

Published by Princeton University Press, 41 William Street,

Princeton, New Jersey 08540

In the United Kingdom: Princeton University Press, 6 Oxford Street, Woodstock, Oxfordshire OX20 1TW

All Rights Reserved

Library of Congress Cataloging-in-Publication Data

Wainer, Howard.

Picturing the uncertain world : how to understand, communicate, and control uncertainty through graphical display / Howard Wainer. p. cm.

Includes bibliographical references and index.

ISBN 978-0-691-13759-9 (cloth : alk. paper)

1. Uncertainty (Information theory)Graphic methods. 2. Communication in scienceGraphic methods. I. Title.

Q375.W35 2009

0039.54dc22 2008053529

press.princeton.edu

eISBN: 978-1-400-83289-7

R0

To my colleagues, past and present,

who contributed their ideas

and their kind thoughts.

This book is the result of

the beauty of their minds

and the labor of mine.

Sine quibus non

Preface and Acknowledgments

The French have an expression Jtais marri aprs lge de raison, mais avant lge de la connaissance. This translates as I was married after the age of reason, but before the age of knowing. Lge de la connaissance is a deep idea, and hubris sometimes traps us into thinking that, while last year we hadnt quite reached lge de la connaissance, this year we have. This has happened to me many times in the past, but each time when a year had passed I realized that the age of knowing still lurked, just out of reach, in the future.

As I sit here, in my sixty-fifth year, I am writing this book on understanding uncertainty, because I realize that many other people share with me the fact that a search for understanding, for knowing, is a constant force in their lives. This search manifests itself in many ways, for the world we live in is filled with subtle uncertainty that can be interpreted in different ways. Among optimists, it is merely an intriguing mystery to be unraveled. Others, more cynical, see evil forces combining to trick you and thence lure you into trouble. Einsteins famous observation that God is subtle, but not malevolent clearly places him among the optimists. Among its deepest and most basic tenets, statistics, the science of uncertainty, follows Einsteins lead.

You dont have to be a statistician to be concerned with uncertainty; although training in statistical thinking is an invaluable aid in navigating this uncertain world. Not too long ago Sid Franks, an old college buddy and my financial planner, told me that he believes that if he did his job to perfection, the check to the undertaker would bounce. When I asked how he engineered the planning necessary to achieve this remarkably precise conclusion, he said that he plans for a life span of ninety-five years and hopes that I make it.

Why ninety-five? I thought. Since the likelihood of making it to ninety-five is so small, wouldnt there be money to spare after the undertaker was paid? And wouldnt I get more enjoyment out of my money by spending it myself?

Sid agreed, explaining that it was a tough prediction and said that he could do a much better job if I would just tell him well in advance exactly when it was that I was going to expire.

He understood that it is rare that such preknowledge is possible and so he had to make his plans under uncertainty. He emphasized that it is a more serious error to live beyond your money than it is to have some money left over after you die. Hence he used an upper bound for the life span.

I asked if knowing my death date was all he needed to be able to manage my account satisfactorily. He replied that, no, it would also help to know what the rate of return on my portfolio would be over the next two decades, as well as the path that inflation will take. He also pointed out that all these things change, and that the amount and direction of those changes can have a profound effect. He then emphasized the importance of having a plan as well as the flexibility to modify it as conditions change and we gain more information.

This one task contains two of the most vexing elements found in most serious decisions: uncertainty about future conditions and an asymmetric cost function (it is worse to outlive your money than to leave some over). And it is but one of thousands in all our lives that illustrate how important it is to understand and manage the uncertainty that we will inevitably encounter. But to manage and understand uncertainty, so that we can plan appropriately, we must always be gathering more data so that we test and perhaps modify our plans. This is as true of buying a house and raising a family as it is for planning our retirements. And if we do this poorly the results can be very serious indeed.

This book is about the science of uncertainty and is filled with stories that provide real life lessons on recognizing and managing uncertainty. I have accumulated these stories over twenty or so years, and in those years many debts have accumulated. It is my pleasure now to acknowledge those debts and to express my gratitude to the colleagues whose thoughts and labor are contained here.

First, my gratitude to my employer, the National Board of Medical Examiners, and its management, especially Don Melnick, Ron Nungester, and Brian Clauser, who have enthusiastically allowed me to pursue the issues described here. My special thanks to Editha Chase, who has been my right hand for the past seven years in most of my scholarly endeavors. Also, my gratitude to Caroline Derbyshire, whose sharp eyes are much appreciated.

Next to my coauthors and advisors who have generously allowed me to use our joint work here: Jacques Bertin, Henry Braun, Brian Clauser, Michael Friendly, Andrew Gelman, Marc Gessaroli, Dan Koretz, William Skorupski, Stephen Stigler, Edward Tufte, John Tukey, Danielle Vasilescu, Monica Verdi, and Harris Zwerling.

Much of the material contained here appeared initially in Chance magazine under the editorship serially of Hal Stern, Dalene Stangl, and Michael Lavine. I owe all three of them my thanks for the help they provided in shaping the material to be both compact and comprehensible as well as for their enthusiasm for the topics. appeared in an earlier form, in the American Statistician, and I would like to thank the editor, Kinley Larntz, for his help.

My appreciation to the American Statistical Association for giving me permission to rework and then reuse all of this material. A complete list of the source material follows the references.

The first draft of this book owes much to the uncertain weather of the Adirondack Mountains and to the realization that activities there are limited when it rains. During one such rainy period in the summer of 2007 I sequestered myself from the rest of my family and began putting pixel to screen. During dinner each night I would regale my wife and son with the product of my activities. As a tribute to either their love or the dearth of more interesting topics, they listened intently and offered their suggestions. They added clarity, completeness, quintessence, quiet, spirituality, serenity, transcendence, detachment, intelligence, sensitivity, consonance, and grammar, while at the same time subtracting incoherence, noise, schmutz, schmerz, anxiety, catharsis, accident, grotesquerie, palpitations, exploitation, and humbuggery. I thank them.