The delayed pain of dependency churn

Yesterday I discussed the forces at play affecting software maintainabilty and Max Arnold added a great point to the Twitter discussion:

I think one missing piece is the increasing churn of dependencies. We rely on many libraries, abstractions, and opensource components, and they evolve with accelerating pace. This pace lowers expectations about stability/compatibility and introduces more regressions and breakages. Back in time we had to implement more stuff ourselves, but because our efforts were limited, the churn was lower. These days the API surface we depend on is much bigger, and so is the amount of collective efforts causing churn.

The highest priority for a developer in almost any organisation will be their immediate goal of getting a feature shipped in a timely fashion. Any third-party dependency which helps them achieve this is often evaluated purely in terms of its functionality and expedience and not any ongoing operational concerns.

The problem is that the trade-offs of such a choice are delayed and sometimes not even felt by the developer who made the decision in the first place.

Code-based dependencies are the worse culprit. Experienced developers in the JavaScript/Node.js ecosystem will know the pain of updating packages on applications already in production. I dread running npm-check-updates on my personal projects because I know I could be kissing goodbye to the next few hours of my life.

I ran the popular aws-amplify library through an NPM dependency visualisation tool and the result is literally off the charts! I’m particularly wary of multi-purpose developer-side tools such as the Amplify SDK and CLI, not just because of their huge nested dependency graph, but also because it generates code that gets deployed as part of the application stack (e.g. VTL resolvers for AppSync). Front-end frameworks which intermingle runtime application libraries with a suite of dev-time tools (such as Gatsby and Create-React-App) are also a huge time-suck to update.

As these packages are either code that I have to deploy or generate code I’m going to deploy, I own all the updates to them— ensuring all CVEs are patched and manually resolving any breaking API changes. Or alternatively I do nothing, and keep my dependencies pinned to old versions and only update when I really really have to.

So what can you do to minimise the effect of dependency churn? Here are a few pointers:

• Prefer low-feature libraries with a minimal API surface area over full-featured packages
• Evaluate the downstream dependency chains of any NPM library/tool you’re adding to your project (for NPM, do this by checking the dependencies array inside the project’s package.json). If this is long, this is a red flag that you’re going to be updating this a lot
• Ensure additions to package.json (or your language’s equivalent) are reviewed in detail as part of peer code reviews
• Automate dependency updates with a tool like Dependabot. Dependabot isn’t perfect (can be overly noisy and need custom tuning) but it keeps dependency churn front of mind and can be put on autopilot for patch and minor bumps (if you have a good set of automated regression tests that can run in GitHub Actions). It also has a nice side effect of highlighting any existing dependencies which have too much downstream churn so you can look into alternatives.
• Consider instead using a new managed cloud service or a Functionless /low-code feature of a managed service you’re already using. Now your cloud provider owns the updates
• If you’re only saving a few lines of code, just copy and paste in the code yourself!