Async all the things

Event-driven architecture (EDA) is often heralded as one of the best reasons for adopting serverless. But this paradigm has been around a lot longer than serverless, so why’s it only now getting so much attention?

Let’s first look at what EDA enables:

• Lower latency user-facing API requests
• Increased system maintainability through easier division of a monolithic system into loosely coupled services

Asynchronous message processing is the key to achieving these core benefits. System event notifications are asynchronously relayed in the form of messages to different services for processing.

For example, if a user-facing API handler needs to perform a lot of time-consuming processing, it could instead just write a job to a queue and return immediately to the user and have the job processed as a background task instead.

But this asynchronous message processing, architecturally elegant as it may be, has typically come with a significant cost to development velocity and operational complexity in server-based systems.

How were async workflows achieved pre-serverless?

Consider the case where you, the application developer, are implementing a feature where an end user in a web app can add a new post to a group. You then need to record this post in your application’s database and also reflect the update in a legacy system.

To implement this asynchronously in a server-based system you would need to do the following (in addition to setting up your web API and database servers):

1. Provision “middleware” server(s) and install messaging software (e.g. Kafka, RabbitMQ).
2. Ensure middleware servers are reliable and durable (i.e. won’t lose messages).
3. Provision “job” server(s) where the background job code will run.
4. Write code to poll the queue for new messages.
5. Write code to handle retrying failures when updating legacy system.
6. Setup monitoring alerts on your middleware and job servers to make sure they don’t go down unexpectedly

This is a huge overhead for a small team of application developers. The synchronous implementation (where a REST API call updates the database and legacy system in series before sending response back to user), while less scalable and performant, is much simpler to implement and maintain.

How serverless architectures simplify this

Let’s see how we would implement the same async flow using AWS serverless services. There are a few ways we could do it, with the simplest probably being Lambda and SNS:

1. Create SNS topic with 2 lines of CloudFormation YAML:

   GroupEventsTopic:
       Type: AWS::SNS::Topic

2. Configure a Lambda function that will perform the background tasks (writing to database and updating legacy system) and wire it up to your SNS topic (using Serverless Framework):

   functions:
       processGroupEvent:
         handler: src/sns/process-group-event.handler
         events:
           - sns:
               arn: !Ref GroupEventsTopic

3. Implement your business logic inside the processGroupEvent handler function

AWS guarantees message durability and delivery, and will also automatically retry failures during the Lambda function invocation. With a few further lines of YAML, you can configure persistently failing messages to be sent to a Dead Letter Queue and tell CloudWatch to notify you if failure levels hit a certain threshold.

As you can see, asynchronous message flows are no longer the intimidating beasts they used to be but are now well within the capability of small teams of application developers with no dedicated ops experts.

Encouraging async thinking

Yet despite this, I find that they’re still not used as much as they should be. Old synchronous habits die hard for many application developers.

To counter this, a good rule of thumb I often employ when reviewing the code of a user-facing Lambda function (e.g. triggered by API Gateway) is that it should make a call to at most 1 downstream service/API before returning to the user. If it’s making 2 or more calls then that’s often a sign of a flow that can be made asynchronous.

— Paul.