Nowadays, the Serverless computing topic becomes extremely popular. It is so tasty to implement the business processes using several AWS Lambda functions and pay a low price for its execution. But everybody who tried to implement several Lambda functions interactions knows that it is not an easy task.

On the 1st of December of 2016 AWS announced, that they brought us a Step Functions – a visual workflow way to coordinate microservices. The main idea of the service is to give us the ability to describe the whole microservice architecture as a finite state machine and visualize it of cause.

One of the most common questions of using Step Functions is how to walk through and process arrays. In this post, I’ll show you how to do it.

The problem

Let’s imagine that we have a Lambda Function that returns an array of elements. We need to process these elements by another Lambda Function. How could we build such kind of microservice interaction using Step Functions?

Lambda Functions

Let’s create a Lambda Function that produces an array (fruits in our example):

exports.handler = (event, context, callback) => {
  var fruits = ['apple', 'orange', 'pinaple'];

  event.fruits = fruits;

  if (event.fruits.length > 0) {
    event.has_elements = true;
  } else {
    event.has_elements = false;
  }

  callback(null, event);
};

Pay attention that I’m adding has_elements flag to the function output. We’ll use this flag to check if we need to process more elements.

Let’s create another Lambda Function that will process array elements:

exports.handler = (event, context, callback) => {
  let fruit = event.fruits.shift();
  // Process array element
  console.log('Processing fruit: ' + fruit);

  // Array still has elements:
  if (event.fruits.length > 0) {
    event.has_elements = true;
  } else {
    event.has_elements = false;
  }

  // Log array elements (for demonstration purpose)
  console.log('Elements in array: ' + event.fruits);
  console.log('Array has more elements: ' + event.has_elements);

  callback(null, event);
};

The main idea is to remove processed elements from the array. We’re using .shift() method for that purpose here, which will remove the first element from the array.

And again, we’re setting up has_elements flag at the end of element processing.

Now we’re ready to create a Step Function algorithm.

Step Function

Here’s our algorithm visualization:

And our Step Function algorithm:

{
  "Comment": "Array processing example",
  "StartAt": "GetArray",
  "States": {
    "GetArray": {
      "Type": "Task",
      "Resource": "arn:aws:lambda:us-east-1:012850762433:function:array_example",
      "ResultPath": "$",
      "Next": "CheckArrayForMoreElements"
    },
    "CheckArrayForMoreElements": {
      "Type": "Choice",
      "Choices": [
        {
          "Variable": "$.has_elements",
          "BooleanEquals": true,
          "Next": "ProcessArrayElement"
        }
      ],
      "Default": "Done"
    },
    "ProcessArrayElement": {
      "Type": "Task",
      "Resource": "arn:aws:lambda:us-east-1:012850762433:function:array_item_process_example",
      "ResultPath": "$",
      "Next": "CheckArrayForMoreElements"
    },

    "Done": {
      "Type": "Pass",
      "End": true
    }
  }
}

This algorithm consists of several steps:

• GetArray – Step Function Task which will execute array_example Lambda functions, which will return fruits array:

{
  "fruits": ["apple", "orange", "pinaple"],
  "has_elements": true
}

• CheckArrayForMoreElements – Conditional Step to check if we have more array elements to process. If yes, go to ProcessArrayElement Step, otherwise finish going to Step Done
• ProcessArrayElement – Step Function Task that will process an array item. For example, it’s Input for last element:

{
  "fruits": ["pinaple"],
  "has_elements": true
}

And the final Output:

{
  "fruits": [],
  "has_elements": false
}

Summary

We’ve just implemented a simple Step Function algorithm to orchestrate the processing of an array of elements. I hope you’ll find this article useful. If you have any questions, feel free to contact me in the comments!

Related articles

• How to start using AWS Step Functions
• Bash For Loop – The Most Practical Guide
• AWS Step Functions – How to manage long-running tasks
• Working with Tuples in Python