The smallest way to introduce serverless into a brownfield application

Migrating an existing server-based product or workload entirely to serverless will rarely make business sense.

As I’m sure you know, I’m a strong advocate for always taking a serverless-first approach to building new applications — it enables your app to get to market faster with a smaller team, allowing your engineers to focus much more on the business needs.

Unfortunately for existing workloads, there is no lift-and-shift migration path to serverless like there is with VMs to containers or with on-premise to cloud. Most of your existing code will need to be rewritten, there’s simply no avoiding this. The time spent doing this will usually cancel out any longer term benefits of adopting serverless. And then there’s the opportunity cost of doing the migration, which could take several months.

I myself started down this path early in 2019 with my own SaaS product that was primarily container-based and had been in production for 5 years. I migrated several components over to serverless but have still not moved everything over. The opportunity cost to me as a one-person company in rewriting container-based API endpoints as Lambda functions and replacing MongoDB with DynamoDB has just been too great.

If you’re keen to start using serverless in production but cannot find a way to introduce it into the existing systems in your organisation, you may be slightly disheartened to hear this.

But fear not! There is a single low-risk way to introduce serverless components into an existing architecture…

The humble cron job — executing a recurring task at some regular interval.

This will be obvious to folks well versed in serverless, but it continues to amaze me both how easy and effective it is to quickly build a new or replace an existing cron job within a brownfield server-based system using a CloudWatch schedule triggered Lambda function.

Cron jobs are a pain to manage in EC2 instance-based systems.

Often they just get lumped in on an existing web/API instance for convenience. But cron jobs are often CPU-intensive so you now need to be careful about bringing down your user-facing APIs. If their workload grows, it’s going to be very difficult to scale them. You could move them to a dedicated server but now you’re getting into provisioning a new instance, paying for it to be always on, and maintaining all the necessary network, security group config and patching that this involves.

Here are a few reasons why serverless is a big improvement here:

• Most small jobs will only require a single new resource, 1 Lambda function, to be deployed.
• If you need to process heavier workloads, your cron Lambda can very easily fan out to multiple other Lambdas concurrently (e.g. using SNS).
• You can use Lambda function concurrency to throttle throughput on less scalable downstream systems such as RDBMS databases.
• The existing app doesn’t need to be microservice-based. Cron jobs within monolithic apps are kind of microservices in their own right, even if they weren’t intended as such. So swapping in a serverless implementation is pretty easy — just turn off the old cron rule and activate the new Lambda trigger rule.
• Obvious one last — you are only paying for the time that your cron job is running. If it only runs once a day for 1 minute, you aren’t paying for it to remain idle the remaining 1,339.

My own favourite use case for a serverless cron job is a daily or weekly email report for a SaaS product.

Say you want to send an email to all users of your app who match certain criteria. This email will contain a report of their activity in the app over a certain time period. I’ve implemented several variations of this for my own apps and for clients’ over the past few years.

I’ll try to write up a more detailed tutorial of how to do this in a future article, but if you’re working on an existing non-serverless AWS-hosted product and you think this would be a good use case for you to implement, please get in touch. I’d be happy to pass on more advice on the best services to use and how to fit them together.

— Paul