Darrin Speegle - Probability, Statistics, and Data: A Fresh Approach Using R

First edition published 2022

by CRC Press

6000 Broken Sound Parkway NW, Suite 300, Boca Raton, FL 33487-2742

and by CRC Press

2 Park Square, Milton Park, Abingdon, Oxon, OX14 4RN

2022 Taylor & Francis Group, LLC

CRC Press is an imprint of Taylor & Francis Group, LLC

Reasonable efforts have been made to publish reliable data and information, but theauthor and publisher cannot assume responsibility for the validity of all materialsor the consequences of their use. The authors and publishers have attempted to tracethe copyright holders of all material reproduced in this publication and apologizeto copyright holders if permission to publish in this form has not been obtained.If any copyright material has not been acknowledged please write and let us know sowe may rectify in any future reprint.

Except as permitted under U.S. Copyright Law, no part of this book may be reprinted,reproduced, transmitted, or utilized in any form by any electronic, mechanical, orother means, now known or hereafter invented, including photocopying, microfilming,and recording, or in any information storage or retrieval system, without writtenpermission from the publishers.

For permission to photocopy or use material electronically from this work, access

Trademark notice Product or corporate names may be trademarks or registered trademarks and are usedonly for identification and explanation without intent to infringe.

ISBN: 978-0-367-43667-4 (hbk)

ISBN: 978-1-032-15441-1 (pbk)

ISBN: 978-1-003-00489-9 (ebk)

DOI: 10.1201/9781003004899

Publisher's note: This book has been prepared from camera-ready copy provided by the authors.

This book represents a fundamental rethinking of a calculus based first course in probability and statistics. We offer a breadth first approach, where the essentials of probability and statistics can be taught in one semester. The statistical programming language R plays a central role throughout the text through simulations, data wrangling, visualizations, and statistical procedures. Data sets from a variety of sources, including many from recent, open source scientific articles, are used in examples and exercises. Demonstrations of important facts are given through simulations, with some formal mathematical proofs as well.

This book is an excellent choice for students studying data science, statistics, engineering, computer science, mathematics, science, business, or for any student wanting a practical course grounded in simulations.

The book assumes a mathematical background of one semester of calculus along with some infinite series in , but in other chapters the use of calculus is minimal. Since an emphasis is placed on understanding results (and robustness to departures from assumptions) via simulation, most if not all parts of the book can be understood without calculus. Proofs of many results are provided, and justifications via simulations for many more, but this text is not intended to support a proof based course. Readers are encouraged to follow the proofs, but often one wants to understand a proof only after first understanding the result and why it is important.

Our philosophy in this book is to not shy away from messy data sets. The book contains extensive sections and many exercises that require data cleaning and manipulation. This is an essential part of the text.

A one-semester course using this book could reasonably cover most material in (Rank Based Tests) is particularly important because it uses simulation techniques developed throughout the text to help students understand power and the effects of assumptions on testing.

There is enough material for a more leisurely and thorough two-semester sequence that would delve deeper into probability theory, spend more time on data wrangling, and cover all of the inference chapters.

Most chapters in the book contain at least one vignette. These short sections are not part of the development of the base material. We imagine these vignettes as starting points for further study for some students, or as interesting additions to the main material. Examples include chloropleth maps, data and gender, Stein's paradox, and a treatment of Covid-19 data.

FIGURE Chapter dependencies.

Base R and tidyverse tools are interspersed, depending on which is better for a particular job, though we don't introduce any tidyverse tools until . Other tidyverse tools introduced in this text, in order of emphasis, are stringr for string manipulation, tidyr for pivot_longer and pivot_wider , lubridate and janitor::clean_names .

All data sets for this book are found in freely available R packages. The bulk of the data sets are in the package associated to this book, fosdata , and are mostly sourced from open access publications that are linked to in the help pages of the data. We encourage readers to spend time reading the publications that were written using the data in the book. We have taken two approaches to the data from original papers. In some instances, fosdata provides essentially all of the data from the published paper. This allows you to explore the data further and think about other visualizations and analyses that would be useful. It also typically requires some wrangling to get the data in a format for the analysis. In other instances, we have simplified the data from the paper quite a bit. In particular, in a few instances we have modified the data by filtering out observations or averaging in order to make it reasonable to assume independence. Please see the links provided in the help pages of fosdata for details.

No book like this would be complete without resources for the student who wishes to learn more. Here are some suggestions for further study that the authors have enjoyed:

• ggplot2, by Hadley Wickham, gives a nice overview of the capabilities of the ggplot2 package. Students interested in data visualization would find this book interesting.

• Advanced R, by Hadley Wickham, provides much more information on R than what we cover in this book. Computer Science students might enjoy reading this book.

• The Statistical Sleuth, by Ramsey and Schafer, will help the student think more like a statistician when dealing with data sets. This book is on a lower level mathematically.

• Modern Applied Statistics with S, by Venables and Ripley, is a book that covers more advanced statistical topics without much mathematics.

• Introductory Statistics with R, by Peter Dalgaard, is a concise introduction to using R for many types of statistical procedures.

• Mathematical Statistics with Applications, by Wackerly, Mendenhall, and Scheaffer, is a more mathematical (but still only requiring multivariate calculus and perhaps basic linear algebra) look at the topics of this book. Students interested in learning how to do the material in this book by hand without access to a computer may enjoy this book.

• Data Feminism, by D'Ignazio and Klein, offers a way of thinking about data science and data ethics informed by the ideas of intersectional feminism. About more than just gender, this book investigates the use and abuse of the power of data science.

This book is written in R Markdown using the bookdown package, by Yihui Xie. The original idea for a course of this type is due to Michael Lamar. The authors wish to thank Matt Schuelke, Kerith Conron, and Christophe Dervieux for helpful discussions. Thanks to Haijun Gong, Kimberly Druschel, Luis Miguel Anguas, Mustafa Attallah, Xue Li, and Caden Beddingfield for working through early editions. The anonymous reviewers provided useful comments, for which we are also grateful. We had two editors at CRC: John Kimmel, who believed in us from the start, and Lara Spieker, who got us to the finish line. Many thanks to both of you and a happy retirement to John!