Infrastructure-as-Code is the linchpin practice for successful serverless teams

A key benefit of adopting serverless is that the engineering teams who build and operate a software system can be smaller. This significantly lowers the barrier for solo developers, startups and SMBs to build and scale great products.

Certain tasks that before were sufficiently complex and time-consuming to warrant a dedicated full-time role such as a DevOps Engineer can now be subsumed into the role of an application developer.

For example:

• Provisioning resources for each environment
• Managing environment configuration
• Building a CI/CD pipeline to automatically deploy and test changes across each environment

For folks coming to serverless application development from a fully backend developer or DevOps engineer background, these tasks will be familiar territory. But if you’re coming from the pure application development side — which I think will become more and more the norm as serverless grows in popularity — this may cause some disruption as you now have something new to learn.

Understanding and embracing Infrastructure-as-Code (IaC) is key to performing all of these tasks effectively, and to maximising the benefits that serverless gives you.

Three ways to deploy your serverless application

Let’s look at the different methods available to deploy and configure the code and resources that make up a serverless application:

1. Pointing and clicking in your cloud provider’s web console
2. Writing scripts that uses your cloud provider’s CLI or SDK
3. Using an IaC tool

Many application developers with limited exposure to implementing DevOps practices will tend towards option 1 initially. But this will get old very fast once they start having to reproduce changes manually across environments.

At this stage, they may be tempted to code their way out of this problem by choosing option 2. While this solves the problem of repeatability, it also introduces a new problem of hundreds of lines of custom code with a high bug surface area that need to be maintained. We want to write less code, not more.

Which leads to option 3: using IaC…

The benefits of Infrastructure-as-Code

There are several benefits of using IaC over a CLI/SDK approach:

• Standardised components mean there are fewer (or only one) ways of specifying a particular higher-level concept (e.g. an API Gateway route that’s handled by a Lambda function). This can typically be achieved with minimal (usually declarative) code.
• IaC templates are parameterisable to allow for differences between environments.
• IaC tools allow for dependency chains between cloud resources to be built into them and many can determine these dependencies automatically for you.
• Most IaC tools have pre-deployment checks built into them to prevent misconfigurations.
• Many IaC tools provide transactional deployments of groups of resources (stacks), allowing deployments to be more atomic.
• IaC deployments are typically incremental and idempotent — they calculate the changes between the deployed resource set and the IaC configuration file and only deploy the necessary changes.
• Stack-based deployments enable quicker teardown of environments.
• Many IaC tools execute the deployment steps cloudside as a hosted service and not on a developer’s workstation or centralised build machine. So you don’t need to worry about a build machine dying mid-deploy.
• IaC templates for serverless apps are valuable from an earlier stage of the development lifecycle (as soon as a developer is setting up their personal dev environment) and are not just for when the app is being deployed to test, staging or production environments.

The drawbacks of Infrastructure-as-Code

So far this all sounds great, but beware that IaC is not a panacea. There are several limitations to be aware of that can be jarring to folks coming to it for the first time:

• Authoring an IaC template is usually slower than using the browser console to create a resource (at least the first time you’re using a particular service).
• The deployment of resources via IaC rather than the browser/CLI/SDK will also be slower as there is an overhead involved in analysing the stack dependency chain and performing other pre-deployment checks.
• Some cloud providers or services don’t offer any or have limited IaC support, so you need to use their browser console/CLI/SDK/API to provision/configure them separately from the rest of your stack.
• There are some technical limitations on IaC stacks that may cause you to rethink your deployment approach (such as CloudFormation’s 200 resource limit).
• Occasionally stack deployments can get “stuck” and require manual troubleshooting and resolution.

How do you get started with Infrastructure-as-Code?

I’ve written before about how I believe all developers coming to serverless should start with an IaC-based deployment framework from day 1 and avoid the temptation of using the browser. Yan Cui wrote up a good comparison of the different serverless-oriented deployment frameworks available in the AWS ecosystem.

I personally use the Serverless Framework for almost all my serverless applications. It acts as an abstraction over the lower-level AWS CloudFormation. This combination gives me the agility of using a serverless/FaaS-optimised framework with the flexibility to dig lower if I need to.

I also think that giving each developer their own sandboxed cloud account can act as a forcing factor for developers with low familiarity with DevOps (or cloud in general) to safely experiment and gain the knowledge they need.

Serverless will take Infrastructure-as-Code mainstream

Infrastructure-as-Code is an extremely powerful practice but right now it’s far from being a widespread practice in software development teams. Powerful server-based IaC tools such as Chef, Puppet and Ansible were out-of-reach for smaller teams of application developers due to higher learning curves and the inherent complexity of server-based architectures that these tools aim to define. An unfortunate follow-on from this is that many application developers today still see defining infrastructure resources as “not my job”.

But I believe this attitude is changing within serverless application teams. The learning curve for FaaS-optimised IaC tools is much shallower and I expect these tools to continue to improve the DX over the next few years.

Application developers will have greater power, but greater responsibility. And maybe this will go some way towards ending the “Is Devops a job title or shared responsibility?” debate! 😉

— Paul