Fabian Hueske - Stream Processing with Apache Flink: Fundamentals, Implementation, and Operation of Streaming Applications

• 2019-04-03: First Release

This book will teach you everything you need to know about stream processing with Apache Flink. It consists of 11 chapters that hopefully tell a coherent story. While some chapters are descriptive and aim to introduce high-level design concepts, others are more hands-on and contain many code examples.

While we intended for the book to be read in chapter order when we were writing it, readers familiar with a chapters content might want to skip it. Others more interested in writing Flink code right away might want to read the practical chapters first. In the following, we briefly describe the contents of each chapter, so you can directly jump to those chapters that interest you most.

• gives an overview of stateful stream processing, data processing application architectures, application designs, and the benefits of stream processing over traditional approaches. It also gives you a brief look at what it is like to run your first streaming application on a local Flink instance.

• discusses the fundamental concepts and challenges of stream processing, independent of Flink.

• describes Flinks system architecture and internals. It discusses distributed architecture, time and state handling in streaming applications, and Flinks fault-tolerance mechanisms.

• explains how to set up an environment to develop and debug Flink applications.

• introduces you to the basics of the Flinks DataStream API. You will learn how to implement a DataStream application and which stream transformations, functions, and data types are supported.

• discusses the time-based operators of the DataStream API. This includes window operators and time-based joins as well as process functions that provide the most flexibility when dealing with time in streaming applications.

• explains how to implement stateful functions and discusses everything around this topic, such as the performance, robustness, and evolution of stateful functions. It also shows how to use Flinks queryable state.

• presents Flinks most commonly used source and sink connectors. It discusses Flinks approach to end-to-end application consistency and how to implement custom connectors to ingest data from and emit data to external systems .

• discusses how to set up and configure Flink clusters in various environments .

• covers operation, monitoring, and maintenance of streaming applications that run 24/7.

• Finally, contains resources you can use to ask questions, attend Flink-related events, and learn how Flink is currently being used.

The following typographical conventions are used in this book:

Indicates new terms, URLs, email addresses, filenames, and file extensions.

Used for program listings, as well as within paragraphs to refer to program elements such as variable or function names, databases, data types, environment variables, statements, and keywords. Also used for module and package names, and to show commands or other text that should be typed literally by the user and the output of commands.

Shows text that should be replaced with user-supplied values or by values determined by context.