Y. Lakshmi Prasad - Big Data Analytics Made Easy

Notion Press

Old No. 38, New No. 6
McNichols Road, Chetpet
Chennai - 600 031

First Published by Notion Press 2016
Copyright Y. Lakshmi Prasad 2016
All Rights Reserved.

ISBN 978-1-946390-72-1

This book has been published with all efforts taken to make the material error-free after the consent of the authors. However, the authors and the publisher do not assume and hereby disclaim any liability to any party for any loss, damage, or disruption caused by errors or omissions, whether such errors or omissions result from negligence, accident, or any other cause.

No part of this book may be used, reproduced in any manner whatsoever without written permission from the authors, except in the case of brief quotations embodied in critical articles and reviews.

This book is dedicated to

A.P.J. Abdul Kalam

( Thinking should become your capital asset, no matter whatever ups and downs you come across in your life. )

To download the data files used in this book,
please use the below link:

www.praanalytix.com/Bigdata-Analytics-MadeEasy-Datafiles.rar

Preface

This book is an indispensable guide focuses on Machine Learning and R Programming, in an instructive and conversational tone which helps them who want to make their career in Big Data Analytics/ Data Science and entry level Data Scientist for their day to day tasks with practical examples, detailed description, Issues, Resolutions, key techniques and many more.

This book is like your personal trainer, explains the art of Big data Analytics/ Data Science with R Programming in 18 steps which covers from Statistics, Unsupervised Learning, Supervised Learning as well as Ensemble Learning. Many Machine Learning Concepts are explained in an easy way so that you feel confident while using them in Programming. If you are already working as a Data Analyst, still you need this book to sharpen your skills. This book will be an asset to you and your career by making you a better Data Scientist.

Authors Note

One interesting thing in Big Data Analytics, it is the career Option for people with various study backgrounds. I have seen Data Analyst/Business Analyst/Data Scientists with different qualifications like M.B.A, Statistics, M.C.A, M. Tech, M.sc Mathematics and many more. It is wonderful to see people with different backgrounds working on the same project, but how can we expect Machine Learning and Domain knowledge from a person with technical qualification.

Every person might be strong in their own subject but Data Scientist needs to know more than one subject (Programming, Machine Learning, Mathematics, Business Acumen and Statistics). This might be the reason I thought it would be beneficial to have a resource that brings together all these aspects in one volume so that it would help everybody who wants to make Big Data Analytics/ Data Science as their career Option.

This book was written to assist learners in getting started, while at the same time providing techniques that I have found to be useful to Entry level Data Analyst and R programmers. This book is aimed more at the R programmer who is responsible for providing insights on both structured and unstructured data.

This book assumes that the reader has no prior knowledge of Machine Learning and R programming. Each one of us has our own style of approach to an issue; it is likely that others will find alternate solutions for many of the issues discussed in this book. The sample data that appears in a number of examples throughout this book was just an imaginary, any resemblance was simply accidental.

This book was organized in 18 Steps from introduction to Ensemble Learning, which offers the different thinking patterns in Data Scientist work environment. The solutions to some of the questions are not written fully but only some steps of hints are mentioned. It is just for the sake of recalling the memory involving important facts in common practice.

Y. Lakshmi Prasad

Acknoweldgements

A great deal of information was received from the numerous people who offered their time. I would like to thank each and every person who helped me in creating this book.

I heartily express my gratitude to all of my peers, ISB colleagues, friends and students whose sincere response geared up to meet the exigent way of expressing the contents. I am very much grateful to our Press, editors and designers whose scrupulous assistance completed this work to reach your hands.

Finally, I am personally indebted to my wonderful partner Prajwala , and my kid Prakhyath , for their support, enthusiasm, and tolerance without which this book would have never been completed.

Y. Lakshmi Prasad

STEP 1

Introduction to Big Data Analytics

1.1 What Big Data?

Big Data is any voluminous amount of Structured, Semi-structured and Unstructured data that has the potential to be mined for information where the Individual records stop mattering and only aggregates matter. Data becomes Big data when it is difficult to process using traditional techniques.

1.2 Characteristics of Big data:

There are many characteristics of Big data. Let me discuss a few here.

1. Volume: Big data implies enormous volumes of data generated by Sensors, Machines combined with internet explosion, social media, e-commerce, GPS devices etc.

2. Velocity: It implies to the rate at which the data is pouring in like Facebook users generate 3 million likes per day and around 450 million of tweets are created per day by users.

3. Variety: It implies to the type of formats and they can be classified into 3 types:

Structured RDBMS like Oracle, MySQL, Legacy systems like Excel, Access

Semi- Structured Emails, Tweets, Log files, User reviews

Un-Structured Photos, Video, Audio files.

4. Veracity: It refers to the biases, noise, and abnormality in data. If we want meaningful insight from this data we need to cleanse it initially.

5. Validity: It refers to appropriateness and precision of the data since the validity of the data is very important to make decisions.

6. Volatility: It refers to how long the data is valid since the data which is valid right now might not be valid just a few minutes or fewer days later.

1.3 Why Big data Important?

The success of the organization not just lies in how good there are in doing their business but also on how well they can analyze their data and derive insights about their company, their competitors etc. Big data can help you in taking the right decision at right time.

Why not RDBMS? Scalability is the major problem in RDBMS, it is very difficult to manage RDBMS when the requirements or the number of users change. One more problem with RDBMS is that we need to decide the structure of the database at the start and making any changes later might be a huge task. While dealing with Big data we need flexibility and unfortunately, RDBMS cannot provide that.

1.4 Analytics Terminology

Analytics is one of the few fields where a lot of different terms thrown around by everyone and lot of these terms sound similar to each other yet they are used in different contexts. There are some terms which sound very different to each other yet they are similar and can be used interchangeably. Someone who is new to Analytics expected to confuse with this abundance of terminology which is there in this field.