Is your serverless app too DRY?

DRY (Don’t Repeat Yourself) is a principle that most software engineers learn early in their career. In this email, I’ll explore how it applies in a serverless context.

What artifacts can be duplicated?

I find the term DRY is almost always used in the context of imperative run-time code and involves refactoring duplicated logic out into functions or modules of their own. But that’s only a small part of where duplication can happen:

• CloudFormation/infrastructure-as-code templates
• Infrastructure configuration parameters
• Build and deployment scripts

• Imperative run-time code:

  • within a single Lambda function
  • in multiple Lambda functions within a single service
  • in multiple Lambda functions within different services in the same overall project/system
  • in multiple Lambda functions within different projects/systems inside the same organisation
• Data in a database

What’s the big problem with repetition?

First of all, it’s important to state that duplication is not inherently bad and can often be desirable, e.g. in a NoSQL database to optimise for performant reads. In these cases we want to add measures to ensure the duplication is controlled rather than remove it altogether. Where I’ve particularly felt the pain of duplication is the boilerplate involved when bootstrapping new applications. I’ve created several utility modules that I’ve added to over time that exist as files inside project repos. I would then go back to an older project and want to use a newer feature and find that the same module had diverged across different projects. So I would then need to cherry-pick the relevant functions of the module from each project that I then need to copy across. Messy.

Methods for DRYing out serverless applications

Serverless applications (being inherently distributed) are more prone to duplication than their monolithic counterparts. There’s often more friction involved in making a component reusable. Here are some DRYing techniques that you can use:

• Replace your custom code with an equivalent publicly available library in a package registry (e.g. NPM)
• Create your own publicly available library in a package registry (e.g. NPM)
• Create your own private library in a package registry (restricted to your organisation)
• Use a mono-repo to store all code for the different services of your project in a single Git repo instead of separate repos for each service
• Create a “common” package within your project mono-repo that defines shared utility modules that can be referenced by different services
• Lambda Layers
• Serverless Application Repository (SAR)
• CDK Components
• Reference external files (static YAML or dynamic JS) from your Serverless Framework serverless.yml files.
• Code generators (not strictly avoiding duplication, but avoiding the manual effort of creating a well-known structure)
• API Gateway service integrations

Be wary of the costs of DRYing

It’s very easy to go overboard trying to combat duplication in your application. There are 2 types of costs you need to consider when implementing DRY:

1. the upfront refactoring cost (usually measured in your time and risk of regressions)
2. the future cost of undoing it or living with the pain caused by a bad abstraction

For some DRYing methods, the cost will be trivial, for some it will be considerable. I read a great Twitter comment recently (that I can’t now find to cite) along the lines of:

“A bad abstraction is much harder to undo than duplicated code”

I have definitely been responsible for building bad abstractions in the past, and the worst bit was the embarrassment of having to explain this complex mess to the poor junior developers on my team. Today I still find myself needing to curb an innate urge to build a component more generically.

“Write Everything Twice”

I hadn’t heard this before but the DRY wikipedia article describes “WET” solutions as the opposite of DRY, with one expansion of WET being “Write Everything Twice”. I’m not sure if this is the context in which it was meant, but I actually think this is good advice to avoid premature abstractions. Only consider spending time on DRYing out a piece of your application if you’ve found the need to write it at least twice. For tasks requiring a bigger DRYing effort, increase this number. And if you find yourself starting into a non-trivial refactoring exercise, first stop and ask yourself: What future pain will this refactoring save me and my team from, and what is the likelihood of such a pain occurring?” And finally, remember:

KISS 💋 > DRY ☂️

Until next time,

— Paul.

P.S. I want to let you know about the upcoming ServerlessDays conference in my home city of Belfast, N.Ireland on Friday 24th January. I will be giving a lightning talk titled “Top 8 mistakes developer teams make in their first serverless project”. There are a lot of really great speakers from the serverless community around the world flying in, so if you’re interested you can still get tickets here. If you’re going to be there, I’d love to hear from you so drop me a reply and we can arrange to catch up on the day. 😀