Singh - LEARN PYSPARK: build python-based machine learning and deep learning models

Any source code or other supplementary material referenced by the author in this book is available to readers on GitHub via the books product page, located at www.apress.com/978-1-4842-4960-4 . For more detailed information, please visit www.apress.com/source-code .

The idea of writing this book had already been seeded while I was working on my first book, and there was a strong reason for that. The earlier book was more focused on machine learning using big data and essentially did not deep-dive sufficiently into supporting aspects, but this book goes a little deeper into the internals of Sparks machine learning library, as well as analyzing of streaming data. It is a good reference point for someone who wants to learn more about how to automate different workflows and build pipelines to handle real-time data.

This book is divided into three main sections. The first provides an introduction to Spark and data analysis on big data; the middle section discusses using Airflow for executing different jobs, in addition to data analysis on streaming data, using the structured streaming features of Spark. The final section covers translation of a business problem into machine learning and solving it, using Sparks machine learning library, with a deep dive into deep learning as well.

This book might also be useful to data analysts and data engineers, as it covers the steps of big data processing using PySpark. Readers who want to make a transition to the data science and machine learning fields will also find this book a good starting point and can gradually tackle more complicated areas later. The case studies and examples given in the book make it really easy to follow and understand the related fundamental concepts. Moreover, there are very few books available on PySpark, and this book certainly adds value to readers knowledge. The strength of this book lies in its simplicity and on its application of machine learning to meaningful datasets.

I have tried my best to put all my experience and knowledge into this book, and I feel it is particularly relevant to what businesses are seeking in order to solve real challenges. I hope that it will provide you with some useful takeaways.

This is my second book on Spark, and along the way, I have come to realize my love for handling big data and performing machine learning as well. Going forward, I intend to write many more books, but first, let me thank a few people who have helped me along this journey. First, I must thank the most important person in my life, my beloved wife, Neha, who selflessly supported me throughout and sacrificed so much to ensure that I completed this book.

I must thank Celestin Suresh John, who believed in me and extended the opportunity to write this book. Aditee Mirashi is one of the best editors in India. This is my second book with her, and it was even more exciting to work with her this time. As usual, she was extremely supportive and always there to accommodate my requests. I especially would like to thank Jim Markham, who dedicated his time to reading every single chapter and offered so many useful suggestions. Thanks, Jim, I really appreciate your input. I also want to thank Manoj Patil, who had the patience to review every line of code and check the appropriateness of each example. Thank you for your feedback and encouragement. It really made a difference to me and the book.

I also want to thank the many mentors who have constantly forced me to pursue my dreams. Thank you Sebastian Keupers, Dr. Vijay Agneeswaran, Sreenivas Venkatraman, Shoaib Ahmed, and Abhishek Kumar, for your time. Finally, I am infinitely grateful to my son, Ziaan, and my parents, for their endless love and support, irrespective of circumstances. You all make my world beautiful.