Simon Rogers - A First Course in Machine Learning

AIMS AND SCOPE

This series reflects the latest advances and applications in machine learning and pattern recognition through the publication of a broad range of reference works, textbooks, and handbooks. The inclusion of concrete examples, applications, and methods is highly encouraged. The scope of the series includes, but is not limited to, titles in the areas of machine learning, pattern recognition, computational intelligence, robotics, computational/statistical learning theory, natural language processing, computer vision, game Al, game theory, neural networks, computational neuroscience, and other relevant topics, such as machine learning applied to bioinformatics or cognitive science, which might be proposed by potential contributors.

PUBLISHED TITLES

MACHINE LEARNING: An Algorithmic Perspective

Stephen Marsland

HANDBOOK OF NATURAL LANGUAGE PROCESSING,

Second Edition

Nitin Indurkhya and Fred J Damerau

UTILITY-BASED LEARNING FROM DATA

Craig Friedman and Sven Sandow

A FIRST COURSE IN MACHINE LEARNING

Simon Rogers and Mark Girolami

Chapman & Hall/CRC

Machine Learning & Pattern Recognition Series

Simon Rogers

Mark Girolami

xi

xiii

xix

1 1

............ 12

........... 15

...... 25

......... 28

.......... 33

............... 35

.............. 37

............. 39

........ 41

......... 53

.... 55

....... 58

........ 65

.............. 67

.......... 75

....... 76

......... 78

...... 81

.. 82

...... 85

...... 86

............. 90

.............. 93

.............. 95

........... 97

.......... 98

............ 103

............ 104

.......... 104

......... 111

............ 114

...... 116

.......... 116

............ 117

.. 118

............ 119

............ 120

............. 121

... 122

............ 122

........... 124

............. 124

............ 124

......... 126

.......... 129

. 131

............ 133

............... 133

.............. 137

........... 139

............. 140

....... 140

....... 143

.......... 149

........... 150

............ 154

............ 154

...... 156

.......... 164

............... 165

............... 165

.............. 167

............ 169

........... 170

.......... 170

. 171

...... 171

.. 172

....... 173

..... 175

..... 175

.......... 177

.......... 179

.......... 180

..... 181

.......... 183

......... 183

......... 184

. 186

.......... 186

...... 187

....... 190

........ 191

......... 192

........... 193

............. 197

....... 198

.......... 198

........ 198

....... 199

.......... 201

...... 203

............... 203

............... 203

.............. 205

............ 207

............ 208

...... 210

.......... 212

.......... 212

............. 214

............. 215

.......... 216

........ 217

.......... 219

......... 220

......... 221

.......... 222

......... 223

......... 224

............. 225

......... 226

..... 228

..... 230

........ 232

........ 233

............... 234

............... 234

.............. 237

............ 239

....... 239

......... 242

............ 247

.......... 247

........... 248

.... 248

............. 249

............ 249

........... 251

......... 252

......... 252

.............. 254

............. 256

............. 257

........ 258

........... 258

............ 260

............. 260

...... 262

........ 264

........... 264

............ 264

....... 268

... 272

............... 273

............... 273

.............. 275

1.1 Synthetic dataset for linear regression example......... 9

1.2 Olympics men's 100 in data .................... 11

1.3 Olympics women's 100 in data.................. 13

1.4 Some useful identities when differentiating with respect to a vector................................ 21

2.1 Events we might want to model with random variables.... 42

5.1 Likelihood and priors for xnew = [2, 0]T for the Gaussian classconditional Bayesian classification example........... 174

5.2 A binary confusion matrix.................... 201

5.3 Confusion matrix for the 20 class newsgroup data....... 202

1.1 Winning men's 100 in times at the Summer Olympics since 1896 ................................. 2

1.2 Effect of varying wo and wl in the linear model defined by Equation 1.1............................ 4

1.3 Example loss function of one parameter (w) ........... 5

1.4 Data and function for the worked example of Section 1.1.5.. 10

1.5 The least squares fit (f (x; wo, wi) = 36.416 - 0.013x) to the men's 100 in Olympics dataset.................. 12

1.6 Zoomed-in plot of the winning time in the Olympics men's 100 in sprint from 1980 showing predictions for both the 2012 and 2016 Olympics........................ 13

1.7 Women's Olympics 100 in data with a linear model that minimises the squared loss...................... 14

1.8 Male and female functions extrapolated into the future.... 14

1.9 Example of linear and quadratic models fitted to a dataset generated from a quadratic function................. 26

1.10 8th order polynomial fitted to the Olympics 100 in men's sprint data ................................. 27

1.11 Least squares fit of 100 in sprint data (a = 2660, b = 4.3).............. 28

1.12 Training and validation loss for Olympics men's 100 in data.. 29

1.13 Generalisation ability of 1st, 4th and 8th order polynomials on Olympics men's 100 in data .................... 30

1.14 Cross-validation.......................... 30

1.15 Mean LOOCV loss as polynomials of increasing order are fitted to the Olympics men's 100 in data................ 31