A checklist for testing your serverless API

I’ve had a few discussions recently around the “what” and “how” of testing serverless applications. What should I write integration tests for? When should I write unit tests? What tools should I use? Should I run them locally or in the cloud? These are all great questions that I have some thoughts on and hope to write more about soon.

But before you can properly plan these aspects of your testing strategy, you first need to understand the different risks involved that your tests will help to mitigate. What can go wrong? What is the probability of this failure occurring? How big an impact could it have?

To help with this, I’ve started to put together a checklist of risks that your automated tests might want to address when building your serverless API in AWS.

I’ve broke each risk down into the following categories:

• Business Rules — failures related to domain-specific rules and data models
• Deployment Configuration — failures related to bad configuration in a deployed environment.
• Integration — failures related to integrating with other services (within AWS or third party)

Business Rules

• Have you even run your code? Is it syntactically correct?
• Does your API endpoint reject all types of invalid request data? (headers, path and querystring variables, body payloads)
• Does the database query return the expected results for each potential combination of filters? Are all the required fields projected in each result item?
• Does the API response status code and body match those expected by clients?
• Is data persisted correctly to database or downstream services? Are the correct defaults set on newly created items?
• Do update operations (e.g. PUT/PATCH requests) prevent modification of any immutable fields?

Deployment Configuration

• Are unauthenticated users prevented from accessing each endpoint?
• Is your Lambda function talking to the correct DynamoDB table/SQS topic, etc?
• Are authenticated users with insufficient privileges forbidden access to this endpoint?
• Does your Lambda function have sufficient IAM permissions to execute and call downstream AWS services?
• Does your API return the correct CORS headers?
• Are non user-facing Lambda functions that use async or streaming event sources (such as SNS or DynamoDB streams) being correctly triggered?
• Does your async Lambda function retry on failure (if desirable)? And do multiple failures get routed to a DLQ?

Integration

• Does your Lambda function have the correct credentials to talk to third party systems (e.g. API key, client secret)?
• Does your Lambda function reliably handle all the different types of errors that could be returned from downstream AWS or third-party services?
• Does your Lambda function reliably handle the lack of availability of downstream AWS or third-party services?

I will extend this list over time.

Do you have questions on any of the above items? Or is there a specific area with regards to testing serverless applications that you would like to understand better? If so, please don’t hesitate to get in touch.

— Paul.