The delayed pain of dependency churn

Yesterday I discussed the forces at play affecting software maintainabilty and Max Arnold added a great point to the Twitter discussion:

I think one missing piece is the increasing churn of dependencies. We rely on many libraries, abstractions, and opensource components, and they evolve with accelerating pace. This pace lowers expectations about stability/compatibility and introduces more regressions and breakages. Back in time we had to implement more stuff ourselves, but because our efforts were limited, the churn was lower. These days the API surface we depend on is much bigger, and so is the amount of collective efforts causing churn.

The highest priority for a developer in almost any organisation will be their immediate goal of getting a feature shipped in a timely fashion. Any third-party dependency which helps them achieve this is often evaluated purely in terms of its functionality and expedience and not any ongoing operational concerns.

The problem is that the trade-offs of such a choice are delayed and sometimes not even felt by the developer who made the decision in the first place.

Code-based dependencies are the worse culprit. Experienced developers in the JavaScript/Node.js ecosystem will know the pain of updating packages on applications already in production. I dread running npm-check-updates on my personal projects because I know I could be kissing goodbye to the next few hours of my life.

I ran the popular aws-amplify library through an NPM dependency visualisation tool and the result is literally off the charts! I’m particularly wary of multi-purpose developer-side tools such as the Amplify SDK and CLI, not just because of their huge nested dependency graph, but also because it generates code that gets deployed as part of the application stack (e.g. VTL resolvers for AppSync). Front-end frameworks which intermingle runtime application libraries with a suite of dev-time tools (such as Gatsby and Create-React-App) are also a huge time-suck to update.

As these packages are either code that I have to deploy or generate code I’m going to deploy, I own all the updates to them— ensuring all CVEs are patched and manually resolving any breaking API changes. Or alternatively I do nothing, and keep my dependencies pinned to old versions and only update when I really really have to.

So what can you do to minimise the effect of dependency churn? Here are a few pointers:

  • Prefer low-feature libraries with a minimal API surface area over full-featured packages
  • Evaluate the downstream dependency chains of any NPM library/tool you’re adding to your project (for NPM, do this by checking the dependencies array inside the project’s package.json). If this is long, this is a red flag that you’re going to be updating this a lot
  • Ensure additions to package.json (or your language’s equivalent) are reviewed in detail as part of peer code reviews
  • Automate dependency updates with a tool like Dependabot. Dependabot isn’t perfect (can be overly noisy and need custom tuning) but it keeps dependency churn front of mind and can be put on autopilot for patch and minor bumps (if you have a good set of automated regression tests that can run in GitHub Actions). It also has a nice side effect of highlighting any existing dependencies which have too much downstream churn so you can look into alternatives.
  • Consider instead using a new managed cloud service or a Functionless /low-code feature of a managed service you’re already using. Now your cloud provider owns the updates
  • If you’re only saving a few lines of code, just copy and paste in the code yourself!

Join daily email list

I publish short emails like this on building software with serverless on a daily-ish basis. They’re casual, easy to digest, and sometimes thought-provoking. If daily is too much, you can also join my less frequent newsletter to get updates on new longer-form articles.

    View Emails Archive

    ☎️ Serverless Clarity Call

    Need quick guidance on a specific issue on your AWS serverless project? Or just wondering where to start with serverless?

    Book a call and ask me anything.

    Learn more >>

    🛫 Serverless Launchpad

    Ready to start building your new AWS serverless project but need help with getting everything setup?

    The Serverless Launchpad is a done-for-you DevOps service installed in under a week. You get a leading-practice multi-account AWS environment, a scaffolded codebase and architecture including the common AWS serverless services, isolated cloud environments for individual developers, automated delivery pipelines right through to production and much more. Everything is IaC, extensively documented and handed over to your developers.

    Learn more >>