[Oldletter #25] Turning the database inside out, Inside the AWS datacenters, and Opinionated thoughts on SQL databases

Every Friday, I’m sharing a short list of evergreen articles/resources broadly related to software engineering that wouldn’t typically get shared in most tech newsletters or social media feeds.

This week’s links and commentary were kindly provided by guest curator Alex DeBrie. Alex is an independent consultant, AWS Data Hero and author of the superb DynamoDB Book (a book I could not do without!).

---

This week’s links

• Turning the database inside out (2015) by Martin Kleppmann. Before I read this, databases were an impenetrable black box. After, they were understandable, at least at a practical level. More than anything else, this got me excited to learn more about database concepts. Martin’s book, Designing Data-Intensive Applications, is a must read.

  We’re trying to get away from shared-memory concurrency, but with databases we’re still stuck with this big, shared, mutable state. So it’s worth thinking about this: if we’re trying to get rid of shared memory in our single-process application architecture, what would happen if we tried to get rid of this shared mutable state on a whole-system level?

• Tuesday Night Live with James Hamilton (AWS re:Invent) (video) (2016) by James Hamilton. A wild, fun ride from Distinguished Engineer James Hamilton as he talks about the infrastructure underlying AWS datacenters. From transatlantic cables to custom routers, there’s a ton here. This one really lit my fire about the size and scale of AWS, even if it’s talking at a level with which I have no experience. Still a classic 6 years later.

  “In 2015 AWS deployed almost enough server capacity every day to support Amazon in 2005 (when it was a $8.49B enterprise).”

• Some opinionated thoughts on SQL databases (2021) by Nelson Elhage. Measured thoughts, both pro and con, around SQL databases. I like this because Nelson is generally in favor of SQL databases but offers construtive notes on the downsides. Well-earned knowledge from someone that has seen a lot of systems and the various ways they fail.

  The query planner is the antithesis of predictability; most of the time it will choose right, but it’s hard to know when it won’t or what the impact will be if it doesn’t. … SQL engines just make it really hard to guarantee predictable performance in query excecution.

---

Submitting your recommendations

If you’d like to share an evergreen article/book which has significantly influenced your thinking or practice around software delivery, please email it through to me and I’ll add it to my backlog for sharing in future editions.

Have a great weekend!

— Paul