Holden Karau - Learning Spark

by Holden Karau , Andy Konwinski , Patrick Wendell , and Matei Zaharia

Copyright 2015 Databricks. All rights reserved.

Printed in the United States of America.

Published by OReilly Media, Inc. , 1005 Gravenstein Highway North, Sebastopol, CA 95472.

OReilly books may be purchased for educational, business, or sales promotional use. Online editions are also available for most titles (http://safaribooksonline.com). For more information, contact our corporate/institutional sales department: 800-998-9938 or corporate@oreilly.com .

• Editors: Ann Spencer and Marie Beaugureau
• Production Editor: Kara Ebrahim
• Copyeditor: Rachel Monaghan
• Proofreader: Charles Roumeliotis
• Indexer: Ellen Troutman
• Interior Designer: David Futato
• Cover Designer: Ellie Volckhausen
• Illustrator: Rebecca Demarest

• February 2015: First Edition

See http://oreilly.com/catalog/errata.csp?isbn=9781449358624 for release details.

The OReilly logo is a registered trademark of OReilly Media, Inc. Learning Spark, the cover image of a small-spotted catshark, and related trade dress are trademarks of OReilly Media, Inc.

While the publisher and the authors have used good faith efforts to ensure that the information and instructions contained in this work are accurate, the publisher and the authors disclaim all responsibility for errors or omissions, including without limitation responsibility for damages resulting from the use of or reliance on this work. Use of the information and instructions contained in this work is at your own risk. If any code samples or other technology this work contains or describes is subject to open source licenses or the intellectual property rights of others, it is your responsibility to ensure that your use thereof complies with such licenses and/or rights.

978-1-449-35862-4

[LSI]

In a very short time, Apache Spark has emerged as the next generation big data processing engine, and is being applied throughout the industry faster than ever. Spark improves over Hadoop MapReduce, which helped ignite the big data revolution, in several key dimensions: it is much faster, much easier to use due to its rich APIs, and it goes far beyond batch applications to support a variety of workloads, including interactive queries, streaming, machine learning, and graph processing.

I have been privileged to be closely involved with the development of Spark all the way from the drawing board to what has become the most active big data open source project today, and one of the most active Apache projects! As such, Im particularly delighted to see Matei Zaharia, the creator of Spark, teaming up with other longtime Spark developers Patrick Wendell, Andy Konwinski, and Holden Karau to write this book.

With Sparks rapid rise in popularity, a major concern has been lack of good reference material. This book goes a long way to address this concern, with 11 chapters and dozens of detailed examples designed for data scientists, students, and developers looking to learn Spark. It is written to be approachable by readers with no background in big data, making it a great place to start learning about the field in general. I hope that many years from now, you and other readers will fondly remember this as the book that introduced you to this exciting new field.

Ion Stoica, CEO of Databricks and Co-director, AMPlab, UC Berkeley