Steven Struhl - Practical Text Analytics: Interpreting Text and Unstructured Data for Business Intelligence

STEVEN STRUHL

PRACTICAL
TEXT ANALYTICS

interpreting text and unstructured data for business intelligence

MARKETING SCIENCE SERIES

This book is dedicated to my wife Debra and my mother Lydia.

Test banks, slides and useful web links are available online at: www.koganpage.com/PracticalTextAnalytics

A lthough and the table of contents will tell you all about what you will find in the rest of the book, this preface still has its uses. It can give you a first inkling of the authors writing style, as historian Frank Muir aptly noted. You can get some sense of whether the person doing the writing is going to drag you on heedlessly until you are smothered in tedium, or if this is somebody who will at least occasionally think about what you as a reader are likely to be experiencing.

You certainly can get some sense about the likely pace of the book whether it will go quickly, or perhaps not, or if this is one to be read late at night when nothing else gets you to sleep.

In fact, this is a place where you can meet the author on neutral ground, without having to wonder if you are misunderstanding him because you are somehow lacking in the subject area. It is your chance to start forming some good opinions about who this person is anyhow, who is asking you to journey through territories that could be fraught with complexity, obscurity and obfuscation.

If you are flipping through a preview online, or (rarity of rarities) looking at this in a bookstore, here could be your spot to decide if you want to continue to the next peek inside. If you opened this book by mistake, then this could be a good spot to realize that this was a serendipitous stroke after all.

One intention of this book is to cut through as much of the recondite language, the murky formulations, the jargon, and even utter nonsense surrounding this field. We will go over vocabulary, but only so you will be prepared when you encounter such terms as named entity and know that they are nothing to fear.

We will steer around equations, subscripted notation and Greek letters whenever possible. If you were hoping to see all of those, then you definitely have sat down in the wrong theatre.

We hear that unstructured data is a really hot topic. Also, that in the arena of unstructured data, text is one of the hottest of areas. (This holds just in certain circles, though dont try it as an ice-breaker at your next party.) Text has received a lot of attention from friendly vendors as a result. While this book will focus on the types of analyses that you can do within your organization, we also want to arm you to deal with this strong promotional interest that text analysis has accrued.

This book is not a guide to the types of services that vendors typically supply, including document retrieval, document search, and organizing a document library. Reading this book, though, should help you deal with sellers as they bombard you with their newest, latest things. The information you get here should enable you to evaluate their claims in an informed and suitably inquiring way.

Looking at other prefaces, it seems this is the place where the book needs to take a turn into the first person, and I tell you a little about myself. You may be questioning what my bona fides are to write a book like this in the first place. Heres what I can come up with. I have been working in applying data and data analysis to practical problems for over 25 years. My clients have included many Fortune 100 companies, but also a host of mid-size and smaller entities, along with charitable, educational and non-profit organizations. I have written over 25 articles and another book, which has been in print for over 20 years and which you can see (and even buy, not that I am hinting at anything) on Amazon. I have taught advanced statistics to bored doctoral students who had to take it to get their degrees, given numerous other courses and seminars, and continue to teach certification courses online.

In education, I have an MBA (University of Chicago), doctorate in psychology (Chicago School of Professional Psychology) and an MA in language and linguistics (Boston University). This combination does seem to fall into place with the topic of the book, and also gives rise to the question of why, when I was younger, I didnt just get a job.

Concerning text analytics, I have been working with this for over 10 years. It started while I was at a major market research company, when a somewhat shifty colleague came to me, asking if we couldnt do something better than was possible with the latest text analytics software we had licensed. This massive program had automatically categorized thousands of comments from a large online community, after somebody had manually trained it over the course of a couple of weeks. (Things were comparatively primitive as recently as a decade ago.) The programs big gimmick was printing out a list of words that occurred most often near a word that you entered. This co-worker was not satisfied, and should not have been.

After a few false starts, I came up with a predictive model using the categories (or codes) this program generated. The method was classification trees, which are discussed in Chapter 7. Neither I nor the other fellow had ever seen anything like this, but it worked. He said something like, Gee this is great, and then did the equivalent of packing his bags and climbing out of a window at midnight. I didnt hear from him for a couple of weeks. It was a very busy time, and when I finally went over to his side of the building to talk about the next steps, I found he had cleared out his office and vanished. Soon afterwards, he turned up as a Wandering-Genius-Scholar type at a major software company. Apparently, this was based on his expertise in analysing text. They must have really liked that predictive model.

However, this story does turn out well for all involved. I learned that analysing text could be useful in a number of ways, and of course I now have the great opportunity to write this book.

Many thanks are in order here. We should start with Melody Dawes at Kogan Page, and agent Richard Burton, for making this possible. I owe particular thanks to my editor Jasmin Naim for being a constant delight to work with and the most courteous and professional possible of colleagues. Also, many thanks to Anna Moss at Kogan Page for being unfailingly helpful and patient in dealing with all the official materials related to getting this book ready.

I would like also to give particular thanks to my wife Debra for tolerating months of very late nights and working weekends and in particular for her acting as a kind of royal food taster for you the readers, trying out various sections of the book to find whether they were particularly indigestible.

Lastly, more thanks to several colleagues who talked about points in this book, offered expert advice and were early readers of various chapters. These include Professor Josh Eliashberg at the Wharton School of Business, Larry Durkin, Well Howell, John Jeter and Edward Zyvith.

This book always aims always to follow the advice set forth by former US President Gerald Ford:

When a man is asked to make a speech, the first thing he has to decide is what to say.

Now lets go on to what we decided, and see where this journey takes us.

IMAGE 0.1

Source: Victor Garaix, French aviator in cockpit of plane with dog: 1914 March. Reproduction Number: LC-USZ62-48802, LOT 10818, Library of Congress Prints and Photographs Division Washington, DC 20540 USA.