FRU Kingsly - Effective Data Wrangling and Exploration with R

FRU Kingsly

2021-01-21

by FRU Kingsly

Copyright 2021 FRU Kingsly. All rights reserved.
This book or any portion thereof may not be reproduced or used in any manner whatsoever without the express written permission of the publisher except for the use of brief quotations in a book review.

The goal of this book is to provide beginners and intermediate users of R, an opportunity to learn practically all there is to know about data wrangling, cleansing and exploration. Before data can be visualised or used in analysis and reporting, it needs to be in the right shape and be of the right type, this book helps you achieve all these without having to search the internet every now and then. This book will teach you programming with R, plotting with base graphics and ggplot2, importing data from diverse sources, manipulating dates, texts and data frames, and dealing with outliers, missing and duplicate data.

FRU Kingsly is a seasoned Business Intelligence developer, with more than ten years in the field of data management and analysis. He has worked with various applications and software including R, Python, Power BI, Tableau, MS SQL Server, MS Excel, MySQL etc. He is passionate about data and analytics and always encourages others around him to take the data analysis route. You can contact him at frukingsly@live.com.

This book is for anybody interested in understanding and applying data manipulation and transformation techniques with R.

This book is divided into seven parts which include:

• Part1: Programming with R (chapter 1 to 15)
• Part2: Import and export data (chapter 16 to 18)
• Part3: String and categorical data manipulation (chapter 19 to 21)
• Part4: Date manipulation (chapter 22 to 24)
• Part5: Data manipulation (chapter 25 to 28)
• Part6: Data cleaning (chapter 29 to 30)
• Part7: Data exploration (chapter 31 to 31)

Introduction, introduces you to the world of data wrangling and exploration.
Variables and Data types, teaches you all there is to know about variables and data types in R.
Operators, teaches you the various operators found in R.
Data Structures I - Atomic Vectors, introduces you to data structures and covers vectors in depth.
Data Structures II - Matrices and Arrays, teaches you matrices.
Data Structures III - factors, describes factors and their manipulation.
Data Structures IV - Recursive Vectors (lists), teaches you all there is to know about lists.
Data Structures V - data frames, takes you to data frames which is the main data structure for holding tabular data.
Control flows, teaches you how to write code which either runs or does not under certain conditions.
Functions I - Built-in functions, takes a closer look at functions and gives you a summary of the most important built-in functions.
Functions II - User-defined functions, continuous from the previous chapter but teaches you how to write your own functions.
Importing and exporting data, teaches you how to import data into and export data from R using base R.
Packages, introduces you to packages in R.
Introduction to plotting with base graphics, introduces you to plotting with base graphics.
Statistical plots with base graphics, walks you through various statistical plots.
Import and export data from a delimited text file, teaches you how to make use of different packages to import tabular formatted data.
Import and export data from excel, takes a closer look at importing data from and exporting data to excel files.
Import and export data from statistical software files and others, looks at importing data from statistical applications and others.
String manipulation with base R, teaches you how to manipulate text using base R.
String manipulation with stringr, teaches you how to manipulate text using the popular text manipulation package stringr.
Manipulating categorical data with forcats, looks into manipulating factors using forcats.
Date manipulation with base R, takes a closer look at the various date and date-time classes that come with base R.
Date Manipulation with chron, teaches you how to manipulate dates and date-times using the chron package.
Date Manipulation with lubridate, teaches you how to manipulate dates and date-times using the lubridate package.
Data Manipulation with Base R, we take a closer look at manipulating data frames using base R.
Data Manipulation with dplyr and tidyr, builds on the previous chapter and teaches you how to manipulate data using dplyr and tidyr.
Data Manipulation with data.table, teaches you a special package built for manipulating large data sets.
Data Manipulation with SQL in R, teaches you how to manipulate data within R using the grandfather of data manipulation SQL.
Detecting and dealing with missing values and outliers, opens the section of data cleaning and deals with missing data and outliers.
Dealing with duplicate values, deals with duplicate and unique values.
Intro to plotting with ggplot2, introduces you to the most popular data visualization package ggplot2.
Statistical plots with ggplot2, walks you through the production of different statistical plots.

I dedicate this book to my late father Mr. Joseph Anguh Gariwa.

Data wrangling is one of the most important steps in data science and analytics, for it is claimed that it takes between 80% to 90% of an analysts time. Data wrangling goes by many names including data munging, data manipulation, data preparation and data transformations. Just as there are many names to data wrangling, there are also many definitions to it. Below we look at two of the most important ones:

TRIFACTA which is a leading provider of data wrangling software by the same name defines data wrangling as:

Data wrangling is the process of cleaning, structuring and enriching raw data into a desired format for better decision making in less time.

Gartner defines data wrangling as:

Data preparation is an iterative-agile process for exploring, combining, cleaning, and transforming raw data into curated datasets for self-service data integration, data science, data discovery, and BI/analytics.

Clearly from the above, we can deduce that data wrangling is the process of converting raw data from one form to another that is appropriate for a specific task at hand. It is rare in analytics to receive data in the form and shape that we want to perform our analysis. Most often, we will be required to transform, clean, enrich and explore that data before we move to our analysis.

Data wrangling involves:

• Importing and exporting data: to and from csv, excel, databases etc.
• Cleaning data: identifying and dealing with missing data, outliers, and duplicates
• Manipulating text and categorical data
• Manipulating dates
• Encoding and enriching data
• Manipulating columns and rows
• Split-apply-combine data
• Merging data
• Reshaping data
• Grouping and Aggregating data
• Exploring data

Data exploration is and should be the initial step of any data analysis project. It is a mini form of data analysis in which we make use of both descriptive statistics and data visualization techniques to better understand our dataset. With traditional analysis and research, we know with exactitude what we are after (that is the hypothesis is known) before collecting data. With exploratory analysis, the process is reversed; we assume little or no information about the outcome of the analysis but instead explore the data to come up with some meaningful insight or hypothesis. Data exploration involves: