How to wait for an async task to complete inside an E2E test

When writing an integration or E2E test, you sometimes need to wait for an asynchronous cloudside action to complete before continuing to the next step. This can be a difficult area to write reliable tests for.

Examples of such actions could be:

  • Any Lambda function that is triggered asynchronously (e.g. from an EventBridge event, S3 upload, DynamoDB stream, SQS message)
  • A Step Functions execution that is triggered from an async event
  • Waiting on an eventually consistent data store to be queryable (e.g. Algolia)

The amount of time to wait for such actions can be hard to gauge as factors such as cold starts often come into play as E2E tests are often run immediately after a deployment.

Consider a test case for verifying that an S3 file upload event correctly triggers a Step Functions execution.

A naive solution (using Jest framework here) is to hardcode a wait time into your tests like so:

// ...
describe('Uploading a valid file to S3 bucket folder', () => {
  let s3SourceKey: string;
  const fileId = uuid();

  beforeAll(async () => {
      s3SourceKey = await uploadSampleFileToS3(fileId);
  });

  it('starts StepFunction execution with fileId as the execution name prefix', async () => {
    // Wait 15 seconds to allow execution time to trigger
    await new Promise((r) => setTimeout(r, 15000));

    // Now query StepFunctions API to find the execution
    const executionsResponse = await stepfunctions.listExecutions({
      stateMachineArn: STATE_MACHINE_ARN,
    }).promise();
    const foundExecution = executionsResponse.executions.find(
        e => e.name.startsWith(fileId),
    )!;
    expect(foundExecution).toBeTruthy();
    expect(foundExecution.status).not.toEqual('FAILED');
  });

  // ...
});

However, while this will probably work for most test runs, it can result in non-deterministic flakiness. Or a test that is always slow because the wait time is set so high to account for the slowest of runs.

A better solution is to use a wait-and-retry polling strategy that attempts to fetch the data using a configurable number of attempts before giving up.

The async-retry NPM module makes this easy in Node.js. Here’s how it could be used to test the same S3 to StepFunctions async use case:

import retry from 'async-retry';
// ...

describe('Uploading a valid file to S3 bucket folder', () => {
  let s3SourceKey: string;
  const fileId = uuid();

  beforeAll(async () => {
      s3SourceKey = await uploadSampleFileToS3(fileId);
  });

  it('starts StepFunction execution with fileId as the execution name prefix', async () => {
      await retry(async () => {
          const executionsResponse = await stepfunctions.listExecutions({
              stateMachineArn: STATE_MACHINE_ARN,
          }).promise();
          const foundExecution = executionsResponse.executions.find(
              e => e.name.startsWith(fileId),
          )!;
          expect(foundExecution).toBeTruthy();
          expect(foundExecution.status).not.toEqual('FAILED');
      }, {
          retries: 15, // Make at most 15 attempts
          factor: 1,
          minTimeout: 1000, // Wait 1 second between each attempt
      });
  });

  // ...
});

There are a few AWS-specific Node.js modules which have this polling pattern built into them:

  • aws-testing-library — Jest and Chai extension functions for checking presence of data in several AWS services
  • sls-test-tools — new Jest extension library which currently verifies delivery of events to EventBridge (but I expect will support more AWS services soon)

Hope you find this useful.

—Paul.

View Emails Archive


Free Email Course

How to transition your team to a serverless-first mindset

In this 5-day email course, you’ll learn:

  • Lesson 1: Why serverless is inevitable
  • Lesson 2: How to identify a candidate project for your first serverless application
  • Lesson 3: How to compose the building blocks that AWS provides
  • Lesson 4: Common mistakes to avoid when building your first serverless application
  • Lesson 5: How to break ground on your first serverless project