Machine Learning for Email
Drew Conway
John Myles White
Copyright 2011 Drew Conway and John Myles White
OReilly books may be purchased for educational, business, or sales promotional use. Online editions are also available for most titles (.
Nutshell Handbook, the Nutshell Handbook logo, and the OReilly logo are registered trademarks of OReilly Media, Inc. Machine Learning for Email , the image of an axolotl, and related trade dress are trademarks of OReilly Media, Inc.
Many of the designations used by manufacturers and sellers to distinguish their products are claimed as trademarks. Where those designations appear in this book, and OReilly Media, Inc., was aware of a trademark claim, the designations have been printed in caps or initial caps.
While every precaution has been taken in the preparation of this book, the publisher and authors assume no responsibility for errors or omissions, or for damages resulting from the use of the information contained herein.
O'Reilly Media
Preface
Machine Learning for Hackers: Email
To explain the perspective from which this book was written, it will be helpful to define the terms machine learning and hackers .
What is machine learning? At the highest level of abstraction, we can think of machine learning as a set of tools and methods that attempt to infer patterns and extract insight from a record of the observable world. For example, if were trying to teach a computer to recognize the zip codes written on the fronts of envelopes, our data may consist of photographs of the envelopes along with a record of the zip code that each envelope was addressed to. That is, within some context we can take a record of the actions of our subjects, learn from this record, and then create a model of these activities that will inform our understanding of this context going forward. In practice, this requires data, and in contemporary applications this often means a lot of data (several terabytes). Most machine learning techniques take the availability of such a data set as givenwhich, in light of the quantities of data that are produced in the course of running modern companies, means new opportunities.
What is a hacker? Far from the stylized depictions of nefarious teenagers or Gibsonian cyber-punks portrayed in pop culture, we believe a hacker is someone who likes to solve problems and experiment with new technologies. If youve ever sat down with the latest OReilly book on a new computer language and knuckled out code until you were well past Hello, World, then youre a hacker. Or, if youve dismantled a new gadget until you understood the entire machinerys architecture, then we probably mean you, too. These pursuits are often undertaken for no other reason than to have gone through the process and gained some knowledge about the how and the why of an unknown technology.
Along with an innate curiosity for how things work and a desire to build, a computer hacker (as opposed to a car hacker, life hacker, food hacker, etc.) has experience with software design and development. This is someone who has written programs before, likely in many different languages. To a hacker, UNIX is not a four-letter word, and command-line navigation and bash operations may come as naturally as working with windowing operating systems. Using regular expressions and tools such as sed
, awk
and grep
are a hackers first line of defense when dealing with text. In the chapters of this book, we will assume a relatively high level of this sort of knowledge.
How This Book is Organized
Machine learning exists at the intersection of traditional mathematics and statistics with software engineering and computer science. As such, there are many ways to learn the discipline. Considering its theoretical foundations in mathematics and statistics, newcomers would do well to attain some degree of mastery of the formal specifications of basic machine learning techniques. There are many excellent books that focus on the fundamentals, the seminal work being Hastie, Tibshirani, and Friedmans The Elements of Statistical Learning ] But another important part of the hacker mantra is to learn by doing. Many hackers may be more comfortable thinking of problems in terms of the process by which a solution is attained, rather than the theoretical foundation from which the solution is derived.
From this perspective, an alternative approach to teaching machine learning would be to use cookbook style examples. To understand how a recommendation system works, for example, we might provide sample training data and a version of the model, and show how the latter uses the former. There are many useful texts of this kind as wellToby Segarans . Such a discussion would certainly address the how of a hackers method of learning, but perhaps less of the why . Along with understanding the mechanics of a method, we may also want to learn why it is used in a certain context or to address a specific problem .
To provide a more complete reference on machine learning for hackers, therefore, we need to compromise between providing a deep review of the theoretical foundations of the discipline and a broad exploration of its applications. To accomplish this, we have decided to teach machine learning through selected case studies.
For that reason, each chapter of this book is a self-contained case study focusing on a specific problem in machine learning. The case studies in this book will focus on a single corpus of text data from email. This corpus will be used to explore techniques for classification and ranking of these messages.
The primary tool we will use to explore these case studies is the R statistical programming language (] This will allow us to focus on the how and the why of these problems, rather than reviewing and rewriting the foundational code for each case.
Conventions Used in This Book
The following typographical conventions are used in this book:
Italic
Indicates new terms, URLs, email addresses, filenames, and file extensions.
Constant width
Used for program listings, as well as within paragraphs to refer to program elements such as variable or function names, databases, data types, environment variables, statements, and keywords.
Constant width bold
Shows commands or other text that should be typed literally by the user.
Constant width italic
Shows text that should be replaced with user-supplied values or by values determined by context.
Tip
This icon signifies a tip, suggestion, or general note.
Caution
This icon indicates a warning or caution.
Using Code Examples
This book is here to help you get your job done. In general, you may use the code in this book in your programs and documentation. You do not need to contact us for permission unless youre reproducing a significant portion of the code. For example, writing a program that uses several chunks of code from this book does not require permission. Selling or distributing a CD-ROM of examples from OReilly books does require permission. Answering a question by citing this book and quoting example code does not require permission. Incorporating a significant amount of example code from this book into your products documentation does require permission.