Alex DeBrie - The DynamoDB Book

My DynamoDB story begins with an abundance of unfounded confidence, slowly beaten out of me by the club of experience.

I first used DynamoDB in 2015 while I was building an internal application for the engineering team I was on. Whenever there was a friction point in the development process, an engineer could type a quick command in Slack. That message would be sent to my webhook and stored for posterity. Occasionally, we would pull all the data out of my table, look it over, nod approvingly, and move along.

I was proud of my application. But its a good thing it never had to scale past single-digit requests per day.

At my next job, I helped implement a few different data models. I was all-in on the serverless ecosystem at this point, and DynamoDB was the en vogue database for serverless applications. I read what I could and thought I was pretty good at DynamoDB as I implemented a RDBMS data model on top of DynamoDB.

In December of 2017, I listened to podcasts of breakout sessions from AWS re:Invent on my commute to and from work. One morning, I stumbled upon a talk on Advanced Design Patterns with DynamoDB by some guy named Rick Houlihan. I expected a tidy review of the impressive DynamoDB knowledge I had already gathered and stored. Wrong.

That talk changed my career.

I couldnt believe the witchcraft I was hearing. Modeling with NoSQL databases is nothing like modeling with relational databases! Storage is cheap; its compute thats sacred! Put all your data into a single table!

When I got home that night, I watched the video recording of the same session. Sure enough, my ears werent deceiving me. Rick Houlihan was an assassin, and DynamoDB was his weapon of choice.

Over the next month, I spent my free time trying to decipher the ins and outs of Ricks talk. I compared my notes against DynamoDB documentation and tried replicating his models in my own examples. The more I read, the more excited I became. My Christmas holiday was focused on sharing this knowledge with others, and in January 2018, I published DynamoDBGuide.com, a website aimed at sharing my newfound love for DynamoDB.

In the time since hitting publish on that site, Ive learned much more about DynamoDB. DynamoDBGuide.com will get you started, but its not going to take you to the top of the mountain.

This is the book I wish I had when I started with DynamoDB. The first few chapters will warm you up with the basics on DynamoDB features and characteristics. But to paraphrase James Carville, "Its the data model, stupid!" The hard part about DynamoDB is making the shift from an RDBMS mindset to a NoSQL mindset. Well go deep on DynamoDB data modeling in this book, from discussion of the DynamoDB API, to various strategies to use when using DynamoDB, to five full-length walkthroughs. There are some things you can only learn by doing, but it doesnt hurt to have a guide along the way.

And that Rick Houlihan guy? He now has the most popular re:Invent session year after year, enjoys a cult following on Twitter, and somehow agreed to write the foreword to this book.

This book would not have been possible without help from so many people. Im sure to leave out many of them.

Thanks to Rick Houlihan for introducing me to DynamoDB, for answering my questions over the years, for taking it easy on me in a live debate in early 2020, and for writing the foreword to this book.

Thanks to many other folks at AWS for building a great service, helping increase my engagement with DynamoDB, and teaching me. There are too many to name here, but Seayoung Rhee, Lizzy Nguyen, Pete Naylor, Edin Zulich, and Colin Lazier have all been great helps.

This book has been significantly improved by reviews, discussions, and encouragement from a number of people in the community. Special thanks to Paul Swail, Jeremy Daly, Jared Short, Corey Quinn, and Shawn Wang (aka Swyx) for encouragement and help with the draft, and thanks to Chris Biscardi for consistent support and encouragement. Also, thanks to so many of you that sent me messages or responded to emails and gave feedback on preview chapters.

Thanks to Ryan Hinojosa for great design help (a perennial weakness for me) including detailed feedback on fonts and Powerpoint slides. Hes the reason your eyes arent bleeding as you read this. Thanks, also, to Daniel Vassallo for sharing not only his experience self-publishing a technical book but also his AsciiDoc config without any expectation of return. I spent way too much time trying to configure AsciiDoc myself. Thanks to David Wells for sharing his marketing knowledge with me and for the breakdown of my landing page. Thanks to Andrea Passwater for assistance with copy on the DynamoDB Book website and for teaching me a ton about writing when we worked together.

Thanks to my parents, siblings, in-laws, and extended family members that stood by me as I moved from lawyer to developer to "self-employed" (unemployed?) author. Im grateful for the support from all of you.

Finally, thanks to my wonderful wife, Elsie. For unrelenting support. For reading every single word (sometimes multiple times!) in a 450-page book about freaking DynamoDB. For wrangling our kids as I worked early mornings, late evenings, and way too many weekends. DynamoDB changed my career, but you changed my life. I love you!

For some, data modeling is a passion. Identifying relationships, structuring the data, and designing optimal queries is like solving a complex puzzle. Like a builder when a job is done and a structure is standing where an empty lot used to be, the feelings are the same: satisfaction and achievement, validation of skill, and pride in the product of hard work.

Data has been my life for a long time. Throughout a career that has spanned three decades, data modeling has been a constant. I cannot remember working on a project where I did not have a hand in the data layer implementation, and if there is one place in the stack I am most comfortable, it is there. The data layer is not usually considered to be the most exciting component, which meant I owned it. Determining the best way to model the data, crafting the most efficient queries, and generally making things run faster and cheaper has always been a big part of my developer life.

Data layer implementation for the masses has been about abstracting developers from the internals of the database. SQL is generally considered painful by developers, and as a result many tools have been invented over the years to facilitate this abstraction. ORM frameworks such as Hibernate, JDO, Django and nHydrate exist to insulate developers from the language of data which for them was convoluted and difficult to understand. These tools have succeeded to some extent by insulating developers from the inner workings of the database. However, this has impacted the overall cost and/or performance of the system in most cases. Until recently this has been a non-issue since the majority of workloads did not scale to the point where a pain threshold was crossed.

There is a general belief that ORM tooling increases developer productivity by allowing them to focus on the code instead of the data. Create business objects in code and let the underlying ORM tooling manage the database for you. This type of abstraction can produce reasonable results when relational databases are queried to produce views of data required by the application. A view is by definition an abstraction of the data model since it is a representation of data stored across many tables and is not persisted in the same form that the data is presented. The developer specifies a set of queries to populate business objects, and then they can leverage the ORM framework to handle the more complex process of managing data stored in multi-table relational models.