Wim Arys on Unsplash

How to run serverless step functions locally

Richard Miles
Sep 6 · 5 min read

In the following article we will be looking at how to set up serverless step functions locally. As well as look at some state machine flow examples.

TLDR; code examples can be found here— https://github.com/richlloydmiles/serverless-step-functions-local-example

I’ll assume that you went ahead and clicked on the TLDR link — however, if this still doesn’t help give you all of the information about how step functions work locally, continue reading.

Getting Setup

I’m going to assume that you have cloned the repo above and run the commands to get yourself up and running, namely: and

You should see something like this in your terminal:

By default the serverless framework will look at our file to give us some instructions about how the serverless app should run.

The default file has some basic serverless configuration entries including plugin and function definitions, but where it really gets interesting is the section.

Here you will see a couple of things:

  • — this is the key where we define a list of state machines (step functions are run in the context of a state machine). In our example we have defined a single stateMachine called . Which in itself will have its own set of definitions and will specify step functions to run.
  • The Comment entry is pretty self explanatory.
  • The is similarly self-explanatory with the exception that the value of this needs to reference a state found in the definition, (if not blindingly obvious, this is the first state (or step function) that will be executed in our state machine by default).
  • Are a list of states that have key / value pairs based on their functions (these are what could be considered our “step functions”).

There are various types of states.

Task State

  • When referencing a Task type, the state needs to contain an ARN for that resource in the definition (usually a lambda function task). By default, serverless version and up supports the shorthand references for resources within your serverless configuration.

Side note: when running your state machine in the real world (e.g. not locally), this is all you should need, however, the plugin requires that your specify these Task resource arn’s in the custom configuration, so that it knows what to reference when executing your state machine locally.

You will see this in the definition in the custom configuration section. e.g.

TaskResourceMapping:  FirstState: arn:aws:lambda:us-east-1:101010101010:function:hello  FinalState: arn:aws:lambda:us-east-1:101010101010:function:world

The region (us-east-1) and AWS account ID (101010101010) are local values that are irrelevant when using this in the real world.

As we continue to go through our definition in the section of the you will notice a or . This tells our state machine whether or not to continue to the next step (usually based on some condition), or to end the execution of the state machine.

Wait State

The next thing you’ll notice is that the state does not specify a resource arn, this is because it is a baked in state whose purpose is to wait for a specified amount of time (in this case 10 seconds), before moving onto the next state (FinalState) or ending. When you execute this particular example locally you should see ‘hello’ being logged to the console, followed by ‘world’ logged 10 seconds later.

Parallel State

If you load up (or replace the serverless.yml file with) the file you will notice the addition of the state (FirstState). This state also does not specify a resource arn(much like the wait state), but does have a definition for . This allows us to run multiple independant states (or step functions) at the same time, that can have completely independent flows that branch off at this point. In our case they are just running two parallel task states and .

Choice State

Finally, if you look at the file you will see a new type of state called , this allows us to conditionally run step functions based on a set of conditions, which in our case is an object key returned by the response in the lambda function called,which conditionally runs the next step based off its value (whether it is equal to or ) This can be found in the function contained in .

module.exports.hello = async event => {  console.log('hello')  return { foo: 1 }}

There are a few more Types of state functions that can be used found in the offical aws documentation— https://docs.aws.amazon.com/step-functions/latest/dg/concepts-states.html

with an infite amount of combinitions in which they can be used.

Executing our state machine

The final piece in the puzzle would be how we go about actually triggering the execution of our state machine.

In our case the function in is where this happens:

const AWS = require('aws-sdk')module.exports.startSF = (event, context, callback) => {  const stepFunctions = new AWS.StepFunctions({ endpoint: 'http://localhost:8083' })  const stateMachineArn =     process.env.OFFLINE_STEP_FUNCTIONS_ARN_WaitMachine  const params = {    stateMachineArn  }  return stepFunctions.startExecution(params).promise().then(() => {    callback(null, `Your state machine ${stateMachineArn} executed   successfully`)  }).catch(error => {    callback(error.message);  })}

Let’s go through the main parts of this.

const stepFunctions = new AWS.StepFunctions({ endpoint: 'http://localhost:8083' })

is where the underlying state machine will be run locally. If you were running this in the real world you would obviously run this in a different way (no using your local port)

Next, the environment variable

OFFLINE_STEP_FUNCTIONS_ARN_WaitMachine

is automatically added to the context of our local serverless execution. With a base value of:

OFFLINE_STEP_FUNCTIONS_ARN_

and the suffix specifying the name of the state machine that we want to start executing:

WaitMachine

we then execute our state machine, calling it with:

stepFunctions.startExecution

In order to actually call this lambda function

startSF()

we need to invoke it in some way, in our case we have chosen to invoke it via an http endpoint within our existing serverless context:

startSF:  handler: handler.startSF  events:  - http:    path: hello    method: GET

You should be able to fire this by hitting the following in your web browser or postman:

http://localhost:3000/local-dev/hello

Error handling

If you get the following error when trying to start your offline server (which I suspect you will be after making any changes to the serverless state machine definitions):

StateMachineAlreadyExists: State Machine Already Exists: 'arn:aws:states:us-east-1:101010101010:stateMachine:WaitMachine'

You can delete your state machine manually by running the following after stopping the serverless offline server.:

npm run delete

You should be able to run successfully afterwards.

Conclusion

In my opinion step functions are the future of AWS serverless infrustucture, especially with the rising popularity of the low-code no code approaches - (https://aws.amazon.com/blogs/aws/new-aws-step-functions-workflow-studio-a-low-code-visual-tool-for-building-state-machines/. Having a low overhead way to test this is almost a nessesity, making the use of local step functions an integral part of the serverless journey.

That’s all folks!

Atheneum Partners digitalization

Connecting the people who need to know with the people who know.

Atheneum Partners digitalization

Developing at a fast and furious pace, Atheneum is a global knowledge sharing platform, which connects leading investment funds, consulting firms, and corporations with the world’s most renowned experts to empower business decisions.

Richard Miles

Written by

Full-stack Developer

Atheneum Partners digitalization

Developing at a fast and furious pace, Atheneum is a global knowledge sharing platform, which connects leading investment funds, consulting firms, and corporations with the world’s most renowned experts to empower business decisions.