Hubert Dulay and Stephen Mooney - Streaming Data Mesh (8th Early Release)

by Hubert Dulay and Stephen Mooney

Copyright 2023 Hubert Dulay and Stephen Mooney. All rights reserved.

Printed in the United States of America.

Published by OReilly Media, Inc., 1005 Gravenstein Highway North, Sebastopol, CA 95472.

OReilly books may be purchased for educational, business, or sales promotional use. Online editions are also available for most titles (http://oreilly.com). For more information, contact our corporate/institutional sales department: 800-998-9938 or corporate@oreilly.com.

• Acquisitions Editor: Andy Kwan
• Development Editor: Jeff Bleiel
• Production Editor: Beth Kelly
• Copyeditor: Sonia Saruba
• Proofreader:
• Indexer:
• Interior Designer: David Futato
• Cover Designer: Karen Montgomery
• Illustrator: Kate Dullea

• June 2023: First Edition

See http://oreilly.com/catalog/errata.csp?isbn=9781098130725 for release details.

The OReilly logo is a registered trademark of OReilly Media, Inc. Streaming Data Mesh, the cover image, and related trade dress are trademarks of OReilly Media, Inc.

The views expressed in this work are those of the authors and do not represent the publishers views. While the publisher and the authors have used good faith efforts to ensure that the information and instructions contained in this work are accurate, the publisher and the authors disclaim all responsibility for errors or omissions, including without limitation responsibility for damages resulting from the use of or reliance on this work. Use of the information and instructions contained in this work is at your own risk. If any code samples or other technology this work contains or describes is subject to open source licenses or the intellectual property rights of others, it is your responsibility to ensure that your use thereof complies with such licenses and/or rights.

This work is part of a collaboration between OReilly and Confluent. See our statement of editorial independence.

978-1-098-13072-5

Welcome to this first edition of Streaming Data Mesh! Your guide to understanding and building a streaming data mesh that meets all of the pillars of a data mesh.

Data mesh is one of the most popular architecture for data platforms that many are exploring today. This book will help you get a full understanding of this self-servicing data platform in a streaming context. Today, batch processing dominates all ETL processes most businesses. This book will help show a different perspective to data pipelines and apply the same concepts you already understand in batch ETL but in a streaming ETL in the context of a data mesh.

This book is designed to help you understand the essential concepts around streaming data meshthe concepts, architectures, and technologies at its core. The book covers all the essential topics related to streaming mesh, from the basics of data architecture, to the use of big data tools for data warehousing, to business-oriented approaches for streaming data mesh architectures. Additionally, we will look at a stack of services involved in a successful streaming data mesh project.

This book does not require you to have pre-knowledge of the pillars that make up a data mesh. We will briefly introduce the pillars at a very high level but specifically define them with streaming in mind. If you feel you need to understand data mesh in more detail, please refer to Zhamak Dehghanis book Data Mesh: Delivering Data-Driven Value at Scale 1st Edition.