Boris Lublinsky - Professional Hadoop Solutions

Contents

Professional Hadoop Solutions

Published by

John Wiley & Sons, Inc.

10475 Crosspoint Boulevard

Indianapolis, IN 46256

www.wiley.com

Copyright 2013 by John Wiley & Sons, Inc., Indianapolis, Indiana

Published simultaneously in Canada

ISBN: 978-1-118-61193-7

ISBN: 978-1-118-61254-5 (ebk)

ISBN: 978-1-118-82418-4 (ebk)

No part of this publication may be reproduced, stored in a retrieval system or transmitted in any form or by any means, electronic, mechanical, photocopying, recording, scanning or otherwise, except as permitted under Sections 107 or 108 of the 1976 United States Copyright Act, without either the prior written permission of the Publisher, or authorization through payment of the appropriate per-copy fee to the Copyright Clearance Center, 222 Rosewood Drive, Danvers, MA 01923, (978) 750-8400, fax (978) 646-8600. Requests to the Publisher for permission should be addressed to the Permissions Department, John Wiley & Sons, Inc., 111 River Street, Hoboken, NJ 07030, (201) 748-6011, fax (201) 748-6008, or online at http://www.wiley.com/go/permissions .

Limit of Liability/Disclaimer of Warranty: The publisher and the author make no representations or warranties with respect to the accuracy or completeness of the contents of this work and specifically disclaim all warranties, including without limitation warranties of fitness for a particular purpose. No warranty may be created or extended by sales or promotional materials. The advice and strategies contained herein may not be suitable for every situation. This work is sold with the understanding that the publisher is not engaged in rendering legal, accounting, or other professional services. If professional assistance is required, the services of a competent professional person should be sought. Neither the publisher nor the author shall be liable for damages arising herefrom. The fact that an organization or Web site is referred to in this work as a citation and/or a potential source of further information does not mean that the author or the publisher endorses the information the organization or Web site may provide or recommendations it may make. Further, readers should be aware that Internet Web sites listed in this work may have changed or disappeared between when this work was written and when it is read.

For general information on our other products and services please contact our Customer Care Department within the United States at (877) 762-2974, outside the United States at (317) 572-3993 or fax (317) 572-4002.

Wiley publishes in a variety of print and electronic formats and by print-on-demand. Some material included with standard print versions of this book may not be included in e-books or in print-on-demand. If this book refers to media such as a CD or DVD that is not included in the version you purchased, you may download this material at http://booksupport.wiley.com . For more information about Wiley products, visit www.wiley.com .

Library of Congress Control Number: 2013946768

Trademarks: Wiley, Wrox, the Wrox logo, Programmer to Programmer, and related trade dress are trademarks or registered trademarks of John Wiley & Sons, Inc. and/or its affiliates, in the United States and other countries, and may not be used without written permission. Hadoop is a registered trademark of The Apache Software Foundation. All other trademarks are the property of their respective owners. John Wiley & Sons, Inc., is not associated with any product or vendor mentioned in this book.

To my late parents, who always encouraged me to try

something new and different.

Boris Lublinsky

To Gwen, Isabella, Emma, and Henry.

Kevin T. Smith

To my family, where I always have support and

understanding.

Alexey Yakubovich

CREDITS

EXECUTIVE EDITOR

Robert Elliott

PROJECT EDITOR

Kevin Shafer

TECHNICAL EDITORS

Michael C. Daconta

Ralph Perko

Michael Segel

PRODUCTION EDITOR

Christine Mugnolo

COPY EDITOR

Kimberly A. Cofer

EDITORIAL MANAGER

Mary Beth Wakefield

FREELANCER EDITORIAL MANAGER

Rosemarie Graham

ASSOCIATE DIRECTOR OF MARKETING

David Mayhew

MARKETING MANAGER

Ashley Zurcher

BUSINESS MANAGER

Amy Knies

PRODUCTION MANAGER

Tim Tate

VICE PRESIDENT AND EXECUTIVE GROUP PUBLISHER

Richard Swadley

VICE PRESIDENT AND EXECUTIVE PUBLISHER

Neil Edde

ASSOCIATE PUBLISHER

Jim Minatel

PROJECT COORDINATOR, COVER

Katie Crocker

PROOFREADER

Daniel Aull, Word One New York

INDEXER

John Sleeva

COVER DESIGNER

Ryan Sneed

COVER IMAGE

iStockphoto.com/Tetiana Vitsenko

ABOUT THE AUTHORS

BORIS LUBLINSKY is a principal architect at Nokia, where he actively participates in all phases of the design of numerous enterprise applications focusing on technical architecture, Service-Oriented Architecture (SOA), and integration initiatives. He is also an active member of the Nokia Architectural council. Boris is the author of more than 80 publications in industry magazines, and has co-authored the book Service-Oriented Architecture and Design Strategies (Indianapolis: Wiley, 2008). Additionally, he is an InfoQ editor on SOA and Big Data, and a frequent speaker at industry conferences. For the past two years, he has participated in the design and implementation of several Hadoop and Amazon Web Services (AWS) based implementations. He is currently an active member, co-organizer, and contributor to the Chicago area Hadoop User Group.

KEVIN T. SMITH is the Director of Technology Solutions and Outreach in the Applied Mission Solutions division of Novetta Solutions, where he provides strategic technology leadership and develops innovative, data-focused, and highly secure solutions for customers. A frequent speaker at technology conferences, he is the author of numerous technology articles related to web services, Cloud computing, Big Data, and cybersecurity. He has written a number of technology books, including Applied SOA: Service-Oriented Architecture and Design Strategies (Indianapolis: Wiley, 2008); The Semantic Web: A Guide to the Future of XML, Web Services, and Knowledge Management (Indianapolis: Wiley, 2003); Professional Portal Development with Open Source Tools (Indianapolis: Wiley, 2004); More Java Pitfalls (Indianapolis: Wiley, 2003); and others.

ALEXEY YAKUBOVICH is a system architect with Hortonworks. He worked in the Hadoop/Big Data environment for five years for different companies and projects: petabyte stores, process automation, natural language processing (NLP), data science with data streams from mobile devices, and social media. Earlier, he worked in technology domains of SOA, Java 2 Enterprise Edition (J2EE), distributed applications, and code generation. He earned his Ph.D. in mathematics for solving the final part of the First Hilberts Problem. He worked as a member of the MDA OMG group, and has participated and presented at the Chicago area Hadoop User Group.

ABOUT THE TECHNICAL EDITORS

MICHAEL C. DACONTA is the Vice President of Advanced Technology for InCadence Strategic Solutions ( http://www.incadencecorp.com ), where he currently guides multiple advanced technology projects for government and commercial customers. He is a well-known author, lecturer, and columnist who has authored or co-authored 11 technical books (on such subjects as Semantic Web, XML, XUL, Java, C++, and C), numerous magazine articles, and online columns. He also writes the monthly Reality Check column for Government Computer News . He earned his Masters degree in Computer Science from Nova Southeastern University, and his bachelors degree in Computer Science from New York University.