Dave Fowler - The Informed Company: How to Build a Cloud-Based Data Stack to Explore and Understand Data

1. Chapter 5
2. Chapter 10
3. Chapter 13
4. Chapter 19
5. Chapter 21

1. About This Book
2. Introduction
3. Chapter 1
4. Chapter 2
5. Chapter 3
6. Chapter 4
7. Chapter 5
8. Chapter 6
9. Chapter 7
10. Chapter 8
11. Chapter 9
12. Chapter 10
13. Chapter 11
14. Chapter 12
15. Chapter 13
16. Chapter 14
17. Chapter 15
18. Chapter 16
19. Chapter 17
20. Chapter 21
21. Part 3
22. Part 4

Dave Fowler
Matt David

Copyright 2022 by Dave Fowler and Matt David. All rights reserved.

Published by John Wiley & Sons, Inc., Hoboken, New Jersey.

Published simultaneously in Canada.

No part of this publication may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, electronic, mechanical, photocopying, recording, scanning, or otherwise, except as permitted under Section 107 or 108 of the 1976 United States Copyright Act, without either the prior written permission of the Publisher, or authorization through payment of the appropriate percopy fee to the Copyright Clearance Center, Inc., 222 Rosewood Drive, Danvers, MA 01923, (978) 7508400, fax (978) 6468600, or on the Web at www.copyright.com. Requests to the Publisher for permission should be addressed to the Permissions Department, John Wiley & Sons, Inc., 111 River Street, Hoboken, NJ 07030, (201) 7486011, fax (201) 7486008, or online at www.wiley.com/go/permissions.

Limit of Liability/Disclaimer of Warranty: While the publisher and author have used their best efforts in preparing this book, they make no representations or warranties with respect to the accuracy or completeness of the contents of this book and specifically disclaim any implied warranties of merchantability or fitness for a particular purpose. No warranty may be created or extended by sales representatives or written sales materials. The advice and strategies contained herein may not be suitable for your situation. You should consult with a professional where appropriate. Neither the publisher nor author shall be liable for any loss of profit or any other commercial damages, including but not limited to special, incidental, consequential, or other damages.

For general information on our other products and services or for technical support, please contact our Customer Care Department within the United States at (800) 7622974, outside the United States at (317) 5723993, or fax (317) 5724002.

Wiley publishes in a variety of print and electronic formats and by printondemand. Some material included with standard print versions of this book may not be included in ebooks or in printondemand. If this book refers to media such as a CD or DVD that is not included in the version you purchased, you may download this material at http://booksupport.wiley.com. For more information about Wiley products, visit www.wiley.com.

Library of Congress CataloginginPublication Data

Names: Fowler, Dave (Computer scientist), author. | Matt David, author.

Title: The informed company : how to build modern agile data stacks that drive winning insights / Dave Fowler, Matt David.

Description: Hoboken, New Jersey : Wiley, [2022] | Includes index.

Identifiers: LCCN 2021028324 (print) | LCCN 2021028325 (ebook) | ISBN 9781119748007 (paperback) | ISBN 9781119748021 (adobe pdf) | ISBN 9781119748014 (epub)

Subjects: LCSH: Data structures (Computer science) | Big data. | Cloud computing.

Classification: LCC QA76.9.D35 F69 2022 (print) | LCC QA76.9.D35 (ebook) | DDC 005.7/3dc23

LC record available at https://lccn.loc.gov/2021028324

LC ebook record available at https://lccn.loc.gov/2021028325

Cover image: Neo Geometric/Shutterstock

Cover design: Wiley

To my mother who continues to be my most supportive and patient teacher. As a software engineer you taught me to code for my sixthgrade science project. Today as a Data Analyst you helped a 38yearold me in discussions and edits of this book. Thank you for always supporting and encouraging my curiosities and for all your love.

Dave Fowler

I dedicate this book to my Mom, an educator who is fueled by helping others learn. Thank you for always believing in me and being an example of how much you can affect other peoples lives.

Matt David

Most comprehensive books on analytics architecture that we've found are over a decade old, most of them precloud. Because there really isn't a modern equivalent to Kimball's seminal The Data Warehouse Toolkit, today's data teams have to reinvent the principles of building a data stack. Too often, they do this without guidance. To solve this problem, we have created a bestpractices guide for bootstrapping and nurturing a technologically current data warehouse.

We wrote this book for whoever values data and believes that informed companies are competitive. It's a book for the working professional who is creating a practical, modern data stack. It's for the lone analyst or the professional embedded in a team. It's for anyone interested in what design practices underlie robust data architecture, the kind that equips entire companies with business intelligence insights. At its heart, this book is written with collaboration in mind ().

Data management is a collaborative process.

This book is not written for big data professionals. To be clear, even large corporations like Doordash, Discord, and the owners of The Financial Times and The New York Times (all previous customers of ours) do not qualify as big data companies. As a rule of thumb, the big data label applies to data architectures with raw input that exceeds 100 GB per day.

No doubt, many elements of this text map onto the big data workflow, especially since warehouses support all sorts of tables, not just, say, event streams. However, our aim is to focus on the central pillars of a modern data stack, so that the widest set of readers can readily benefit from the information herein. In this spirit, we forgo recommendations for megascale architectures.

This book is not for AIenabled teams and does not cover AI workflows, machine learning models, or realtime operational use cases. Instead, its goal is to provide best practices for building and maintaining a robust data analytics stack (i.e. the analytics foundation on which an AI workflow can be built).

If you are a small business that can run everything with Quickbooks and Excel, that ability is great. Data is important for all companies, but if these tools are already serving you well, the book may not offer helpful guidance. If you start exceeding the data capacity of Excel or bring in a data source that needs to be in a database to be analyzed, then keep reading.

This book was written by Dave Fowler and Matt David.