Pushing for a better developer experience for DynamoDB

Last week, I enjoyed listening to Rick Houlihan talk with Corey Quinn on the Screaming in the Cloud podcast (episode link), and I found this quote from Rick particularly interesting (at timestamp 08:50):

“This is the problem that I’ve always had with DynamoDB. I just wish that we’d spent more time on improving the developer experience, right, enhancing the API, implementing some of these features that, you know, help. Let’s make single table design a first-class citizen of the DynamoDB API… It’s treated like a stepchild. You know, it’s like, come on, we’re fully funding the solutions within our own umbrella that are competing with ourselves, and at the same time, we’re letting the DynamoDB API languish while our competitors are moving ahead.”

If you’re not familiar with Rick, he worked for several years in the DynamoDB team at AWS where he became “the godfather of single-table design”, an approach for scalable data modelling within DynamoDB (and indeed most NoSQL databases). At the end of last year, Rick left AWS to work with MongoDB Atlas, and while he clearly still has a lot of love for DynamoDB, it seems he is now able to speak more freely about its shortcomings (which is a good thing for those of us who are DynamoDB customers).

I used MongoDB as my main application database of choice for several years before moving to DynamoDB around 3–4 years ago. And while I don’t at all miss most of the operational pains I encountered with Mongo, I do miss its nice API with richer querying and service-side indexing capabilities.

Alex DeBrie has an excellent article comparing MongoDB and DynamoDB, where he explains “DynamoDB’s philosophy is that it won’t let you write a bad query”, whereas “MongoDB has a different philosophy… designed for developer productivity and easier on-boarding.” I love DynamoDB’s philosophy here but while I appreciate that I can’t have my cake and eat it, I do still think AWS could be doing more on the DX side.

DynamoDB, along with Lambda and either API Gateway or AppSync, are the most fundamental services that application developers coming into the AWS serverless ecosystem will encounter. Yet reading through the release history for DynamoDB over the past few years, the vast majority of the features listed are either an ops concern, are only relevant to workloads running at very high scale or are for a niche use case. The addition of transaction capability is the only big productivity-improving general purpose feature that I could find post-2018 which undoubtedly benefits application developers (I guess PartiQL could also fall into this category, but the jury is still out on this as it seems to blatantly violate the “no bad query” default philosophy).

I personally would love to see features that allow some data integrity concerns to be moved out of application code into some service-side configuration. Maybe even the “cloud access layer” that Rick mentions in the episode. 🙂

Anyway, I definitely recommend you check out Rick on the Screaming in the Cloud podcast.