How to access cloud resource configuration from your local tests

When I’m developing Lambda functions in Node.js, writing integration tests that can invoke the handler function locally is an important part of my workflow.

Often these handler functions talk to downstream cloud services such as DynamoDB tables or SNS topics. So my code needs the ARN (the unique Amazon Resource Name) of these resources in order to execute.

When running as a Lambda function in the cloud, I can easily provide these ARNs via environment variables. Using the Serverless Framework, this looks like so:

#serverless.yml
...
provider:
  environment:
    DYNAMODB_TABLE_MAIN: !Ref MainTable
    SNS_TOPIC_GROUP_EVENTS: !Ref GroupEventsTopic

But how can I do this when I want to run my handler function locally? I don’t want to have to deploy the function to the cloud and invoke it remotely for every change I make.

I could use the invoke-local command which will evaluate and populate these environment variables for me and invoke the handler using a test event payload. However, this is just a CLI command and doesn’t allow me to write richer test scenarios using frameworks like Jest.

Another option is to hardcode the resource ARNs into a .env file that the Jest test runner loads at startup time. But then I have to manage this file for every potential environment where I want to run tests. When you’re working in a team where developers have their own personal AWS accounts and the CI/CD pipeline needs to run these tests against different environments, this strategy becomes unmanageable.

So my preferred solution is to use the serverless-export-env plugin. It works by taking the environment variables you have defined in your serverless.yml file, evaluating them and exporting them to a .env file in the root of your service that your test runner can then reference.

Setting it up

Here’s how you can put it all together (assuming you are already using Serverless Framework and Jest):

1. Install a few NPM packages:

   npm install --save-dev arabold/serverless-export-env dotenv

2. Add plugin to your serverless.yml file:

   plugins:
   - serverless-export-env

3. Add this Javascript file to your folder structure. This will be run by Jest before it starts running test suites.

   // ./tests/config/test-env.js
   const { config } = require('dotenv');
   const { resolve } = require('path');
   // Load environment variables generated from serverless.yml
   config({
     path: resolve(__dirname, '../../.env'),
   });

You may need to adjust the path depending on where this file is located relative to the .env file generated by the plugin (which will be in the same folder as your serverless.yml file).

4. Hook up the test-env.js file to your jest.config.js file:

   // jest.config.js
   module.exports = {
     // ...other config
     setupFilesAfterEnv: ['./tests/config/test-env.ts'],
   };

5. Create NPM scripts:

   // package.json
   {
   "scripts": {
     "dotenv": "STAGE=${STAGE:=personal} && sls export-env -s $STAGE",
     "test": "npm run dotenv && jest"
   }
   }

The dotenv command generates the .env file containing the environment variables and then when jest is executed it can reference it.

That’s it! Now whenever you need to run your tests locally with all the necessary cloud configuration settings, you just call npm run test. And what’s more, you can have your CI/CD pipeline run the exact same command for any stage it’s deploying to 😀.

Have a great weekend!

— Paul.