Matthew A. Russell - Mining the Social Web: Finding Needles in the Social Haystack

The Web is more a social creation than a technical one. I designed it for a social effectto help people work together and not as a technical toy. The ultimate goal of the Web is to support and improve our weblike existence in the world. We clump into families, associations, and companies. We develop trust across the miles and distrust around the corner.

If you have a basic programming background and are interested in insight surrounding the opportunities that arise from mining and analyzing data from the social web, youve come to the right place. Well begin getting our hands dirty after just a few more pages of frontmatter. Ill be forthright, however, and say upfront that one of the chief complaints youre likely to have about this book is that all of the chapters are far too short. Unfortunately, thats always the case when trying to capture a space thats evolving daily and is so rich and abundant with opportunities. That said, Im a fan of the 80-20 rule, and I sincerely believe that this book is a reasonable attempt at presenting the most interesting 20 percent of the space that youd want to explore with 80 percent of your available time.

This book is short, but it does cover a lot of ground. Generally speaking, theres a little more breadth than depth, although where the situation lends itself and the subject matter is complex enough to warrant a more detailed discussion, there are a few deep dives into interesting mining and analysis techniques. The book was written so that you could have the option of either reading it from cover to cover to get a broad primer on working with social web data, or pick and choose chapters that are of particular interest to you. In other words, each chapter is designed to be bite-sized and fairly standalone, but special care was taken to introduce material in a particular order so that the book as a whole is an enjoyable read.

Social networking websites such as Facebook, Twitter, and LinkedIn have transitioned from fad to mainstream to global phenomena over the last few years. In the first quarter of 2010, the popular social networking site Facebook surpassed Google for the most page visits,[] and they are enabling technology to bring out the best (and sometimes the worst) in us. The explosion of social networks is just one of the ways that the gap between the real world and cyberspace is continuing to narrow.

Generally speaking, each chapter of this book interlaces slivers of the social web along with data mining, analysis, and visualization techniques to answer the following kinds of questions:

The answers to these types of questions generally connect two or more people together and point back to a context indicating why the connection exists. The work involved in answering these kinds of questions is only the beginning of more complex analytic processes, but you have to start somewhere, and the low-hanging fruit is surprisingly easy to grasp, thanks to well-engineered social networking APIs and open source toolkits .

Loosely speaking, this book treats the social web [ (GGG) instead of World Wide Web (WWW), because the terms web and graph can be so freely interchanged in the context of defining a topology for the Internet. Whether the fullness of Tim Berners-Lees original vision will ever be realized remains to be seen, but the Web as we know it is getting richer and richer with social data all the time. When we look back years from now, it may well seem obvious that the second- and third-level effects created by an inherently social web were necessary enablers for the realization of a truly semantic web. The gap between the two seems to be closing.

Activities such as building your own natural language processor from scratch, venturing far beyond the typical usage of visualization libraries, and constructing just about anything state-of-the-art are not within the scope of this book. Youll be really disappointed if you purchase this book because you want to do one of those things. However, just because its not realistic or our goal to capture the holy grail of text analytics or record matching in a mere few hundred pages doesnt mean that this book wont enable you to attain reasonable solutions to hard problems, apply those solutions to the social web as a domain, and have a lot of fun in the process. It also doesnt mean that taking a very active interest in these fascinating research areas wouldnt potentially be a great idea for you to consider. A short book like this one cant do much beyond whetting your appetite and giving you enough insight to go out and start making a difference somewhere with your newly found passion for data hacking.

Maybe its obvious in this day and age, but another important item of note is that this book generally assumes that youre connected to the Internet. This wouldnt be a great book to take on vacation with you to a remote location, because it contains many references that have been hyperlinked, and all of the code examples are hyperlinked directly to GitHub, a very social Git repository that will always reflect the most up-to-date example code available. The hope is that social coding will enhance collaboration between like-minded folks such as ourselves who want to work together to extend the examples and hack away at interesting problems. Hopefully, youll fork, extend, and improve the sourceand maybe even make some new friends along the way. Readily accessible sources of online information such as API docs are also liberally hyperlinked, and it is assumed that youd rather look them up online than rely on inevitably stale copies in this printed book.