Pushing for a better developer experience for DynamoDB

Last week, I enjoyed listening to Rick Houlihan talk with Corey Quinn on the Screaming in the Cloud podcast (episode link), and I found this quote from Rick particularly interesting (at timestamp 08:50):

“This is the problem that I’ve always had with DynamoDB. I just wish that we’d spent more time on improving the developer experience, right, enhancing the API, implementing some of these features that, you know, help. Let’s make single table design a first-class citizen of the DynamoDB API… It’s treated like a stepchild. You know, it’s like, come on, we’re fully funding the solutions within our own umbrella that are competing with ourselves, and at the same time, we’re letting the DynamoDB API languish while our competitors are moving ahead.”

If you’re not familiar with Rick, he worked for several years in the DynamoDB team at AWS where he became “the godfather of single-table design”, an approach for scalable data modelling within DynamoDB (and indeed most NoSQL databases). At the end of last year, Rick left AWS to work with MongoDB Atlas, and while he clearly still has a lot of love for DynamoDB, it seems he is now able to speak more freely about its shortcomings (which is a good thing for those of us who are DynamoDB customers).

I used MongoDB as my main application database of choice for several years before moving to DynamoDB around 3–4 years ago. And while I don’t at all miss most of the operational pains I encountered with Mongo, I do miss its nice API with richer querying and service-side indexing capabilities.

Alex DeBrie has an excellent article comparing MongoDB and DynamoDB, where he explains “DynamoDB’s philosophy is that it won’t let you write a bad query”, whereas “MongoDB has a different philosophy… designed for developer productivity and easier on-boarding.” I love DynamoDB’s philosophy here but while I appreciate that I can’t have my cake and eat it, I do still think AWS could be doing more on the DX side.

DynamoDB, along with Lambda and either API Gateway or AppSync, are the most fundamental services that application developers coming into the AWS serverless ecosystem will encounter. Yet reading through the release history for DynamoDB over the past few years, the vast majority of the features listed are either an ops concern, are only relevant to workloads running at very high scale or are for a niche use case. The addition of transaction capability is the only big productivity-improving general purpose feature that I could find post-2018 which undoubtedly benefits application developers (I guess PartiQL could also fall into this category, but the jury is still out on this as it seems to blatantly violate the “no bad query” default philosophy).

I personally would love to see features that allow some data integrity concerns to be moved out of application code into some service-side configuration. Maybe even the “cloud access layer” that Rick mentions in the episode. 🙂

Anyway, I definitely recommend you check out Rick on the Screaming in the Cloud podcast.

Join daily email list

I publish short emails like this on building software with serverless on a daily-ish basis. They’re casual, easy to digest, and sometimes thought-provoking. If daily is too much, you can also join my less frequent newsletter to get updates on new longer-form articles.

    View Emails Archive

    Architecture & Process Review

    Built a serverless app on AWS, but struggling with performance, maintainability, scalability or DevOps practices?

    I can help by reviewing your codebase, architecture and delivery processes to identify risk areas and their causes. I will then recommend solutions and help you with their implementation.

    Learn more >>

    🪲 Testing Audit

    Are bugs in production slowing you down and killing confidence in your product?

    Get a tailored plan of action for overhauling your AWS serverless app’s tests and empower your team to ship faster with confidence.

    Learn more >>