Robert Ilijason - Beginning Apache Spark Using Azure Databricks: Unleashing Large Cluster Analytics in the Cloud

Any source code or other supplementary material referenced by the author in this book is available to readers on GitHub via the books product page, located at www.apress.com/9781484257807 . For more detailed information, please visit http://www.apress.com/source-code .

So you wanna be a data analyst, data scientist, or data engineer? Good choice. The world needs more of us. Also, it is not only fun and rewarding work but also easy! At least if youre willing to put in the effort.

The bar for getting into heavy-duty data analytics has never been lower. You dont need servers, advanced Linux skills, or a ton of money to get started anymore. Graphical tools like Tableau and Power BI made small-scale data work available for the masses. Now Databricks does the same for huge datasets. Millions, billions, and trillions of rows are not a problem.

This book will ease you into both the tool and the field. Well start by looking at the data analytics field in general why its hot, what has changed, and where Apache Spark and Databricks fit into the overall picture.

Then its time for you to learn about how Databricks works. Well spend a few chapters on learning how it works internally and how you actually use it everything from getting around the user interface to spinning up clusters and importing data.

Once you know how to use the tool, its time to start coding. Youll familiarize yourself with (Structured Query Language) SQL and Python, the two main languages for data analysis work. It doesnt stop there, well follow it up by digging deeper into advanced data wrangling techniques where well see a lot of the ifs and buts youll come across when you work with data in reality.

Finally, Ill drag you through a few short chapters with more advanced topics. Coming out of them, youll have an understanding of how to run machine learning algorithms, manage delta loads, and run Databricks through the application programming interface (API).

And with that, youll be ready to get started for real, solving small and large problems in the real world. This is an introductory book, but once youre through it, youll have the tools you need to start exploring huge datasets around you or in your business.

Looking forward to seeing you around in the world of data experts.

Lets start at the very top. This book is about large-scale data analytics. Itll teach you to take a dataset, load it into a database, scrub the data if necessary, analyze it, run algorithms on it, and finally present the discoveries you make.

Well use a fairly new tool on the market called Databricks for all of this, simply because its the most ambitious tool on the market right now. While there are other tools that can provide you with the same capabilities, more or less, no one does it like Databricks.

Databricks hands you the massive analysis power of Apache Spark in a very easy-to-use way. Ease of use is important. You should be spending time looking at data, not figuring out the nitty-gritty of configuration files, virtual machines, and network setups. Its far too easy to get stuck in tech. With Databricks, you wont.

But before we start, lets spend a few pages talking about what this data analysis thing is all about, whats happened the last few years, and what makes large-scale analytics different from the stuff you might have run in Excel or SQL Server Analysis Services.

If you havent lived under a rock the last few years, youve seen advanced analytics being referenced pretty much everywhere. Smart algorithms are winning elections, driving cars, and about to get us to Mars. Theyll soon solve all our problems, possibly including singularity. The future is here! At least if you are to believe the press.