How to start using AWS Step Functions

Andrei Maksimov

Andrei Maksimov

0
(0)

Automation is an essential part of any small company or large-scale enterprise to survive in the modern world. With cloud computing services such as AWS Lambda and Step Functions, you can automate any business or technical process easily and at almost no cost. This beginner’s guide describes how to start using AWS Step Functions to automate your routine tasks using modern Serverless cloud technologies.

What is AWS Step Functions

AWS Step Functions is an orchestration service that allows you to use AWS Lambda functions and other AWS services to build and automate business-critical workflows. It allows you to describe your processes as a series of event-driven steps each of which is powered by AWS Lambda or other supported AWS services and responsible for its own part of the process. Every workflow is described in JSON-like syntax.

Here’s a list of currently supported services, which you can directly call from Step Functions:

  • AWS Lambda
  • DynamoDB
  • AWS Batch
  • Amazon SNS
  • Amazon SQS
  • Amazon ECS
  • Amazon EMR (using Boto3 library to manage EMR clusters)
  • Amazon Sagemaker
  • AWS Step Functions (you can call one workflow from another)
  • AWS Codebuild
  • Amazon Athena
  • Amazon EKS
  • API Gateway

For any unsupported services, for example, when you need to call a third-party API or another cloud provider services, you need to use AWS Lambda.

AWS Step Functions ties all steps in a particular execution order to form a single automated workflow (state machine) and allows you to control execution flow based on the outputs of every executed step. The simplest workflow may contain only one step.

Here’s an example.

1. Beginner's Guide to AWS Step functions - Simple example

More complex workflow might include many different steps and have complex execution logic, as an example if CICD automation workflow:

2. Beginner's Guide to AWS Step functions - CICD Workflow

As soon as the workflow is described, you can launch it manually or in an automated way.

All Step Functions workflows can scale on-demand and handle a virtually unlimited amount of executions. If you’re using AWS Lambda as a step at Step Function workflow, remember, that its execution time is limited to 15 minutes. If the step might take longer, use the asynchronous execution model or split your task into smaller subtasks.

The workflows for Step Functions can be easily created using an interactive AWS console. In addition to that, every workflow execution is easily traceable that allows you easily see which step is executing right now or which step has been failed.

3. Beginner's Guide to AWS Step functions - Failed CICD Workflow Example

Concepts of AWS Step Functions

Before we dive into implementing a new Step Function, it is important to understand certain terminologies that we would be using throughout the article.

A Step function defines an automated workflow or a state machine. Each state machine can receive some input data (JSON format) and produce an output data after the execution.

Every state machine consists of Tasks and States. Each Task can receive an input data and produce an output data. That’s how you can control data flow in your state machine.

Let’s take a look at simple Step Function workflow syntax and its graph.

{
  "Comment": "A Hello World example of the Amazon States Language using an AWS Lambda function",
  "StartAt": "HelloWorld",
  "States": {
    "HelloWorld": {
      "Type": "Task",
      "Resource": "arn:aws:lambda:us-east-1:123456789012:function:HelloFunction",
      "End": true
    }
  }
}

This declaration leads to the following state machine graph.

4. Beginner's Guide to AWS Step functions - HelloWorld Example

State

Each step (State) in a state machine executes a service or makes a decision. A state can be a task to execute, a conditional choice operator or can perform another action. Finally, it can lead to the next state, the end of a state machine.

Each state described as a JSON block in the state machine as shown below.

For out HelloWord state machine, here is the State definition:

"HelloWorld": {
      "Type": "Task",
      "Resource": "arn:aws:lambda:us-east-1:123456789012:function:HelloFunction",
      "End": true
}

Tasks

Tasks are the atomic unit of a state machine that does the actual job inside a State. A task is responsible to conditionally choose the execution, perform the desired action, produce output and return it back to the state. A task can call supported AWS resource or service or take necessary input from humans too. A task is contained inside a state definition as shown below:

{
      "Type": "Task",
      "Resource": "arn:aws:lambda:us-east-1:123456789012:function:HelloFunction",
      "End": true
}

State Machine

A state machine is the collection of multiple states tied together to define an automated flow. The example state machine contains a single state – HelloWorld. This single state is used to define this state machine. The state machine is the “brain” that manages the orchestration of all the states defined inside of it.

Getting started

To get started with AWS Step Functions, make sure you have the following prerequisites:

Once ready, you can go ahead and create your first step machine.

AWS Console

The easiest way to build a Step Functions state machine is to use an AWS Step Functions service and its wizard at AWS console.

5. Beginner's Guide to AWS Step functions - Welcome to Step Functions

Click “Get Started” button to move to the next screen.

It will display a sample function as shown below:

6. Beginner's Guide to AWS Step functions - AWS HelloWorld example

There are two main sections in the screen:

  • Code – here you’re defining your state machine. Browse through the code and try to understand what it is supposed to do. Currently, the sample code would be read-only mode, but we’ll change it as soon as we walk through the wizard.
  • Graph: This is an area where your code is shown in the form of an image or directional graph. This graph gets dynamically updated as you change the code.

Once you have read the code and matched the graph with it, you can move to the next step to create your own Step Function.

Click the Next button.

7. Beginner's Guide to AWS Step functions - AWS HelloWorld example - Step 1

In the next step, it will ask you to provide some basic details like – Step Function workflow name, IAM role, logging, tracing, and tagging configuration.

8. Beginner's Guide to AWS Step functions - AWS HelloWorld example - Step 2

For this first function let’s continue with the defaults.

Click Create state machine button.

Workflow execution

After successful creation, you will see a dialog for executing the state machine as shown below:

9. Beginner's Guide to AWS Step functions - AWS HelloWorld example - Execute workflow

Click Start execution button.

Congratulations! You just successfully created and ran your first AWS Step function.

Look through the output graph and click each green State block in the image to get additional details on the execution.

10. Beginner's Guide to AWS Step functions - AWS HelloWorld example - Execution results

Each state will have a name, resource type, and execution state information.

Click Step input and Step output tab after Details to see inputs and outputs for every step.

Integration with AWS Lambda

In the real world, this state machine is useless, and now it is a good time to go ahead and create a real workflow powered by AWS Lambda. I recommend you to AWS Step Functions – How to manage long-running tasks guide as the next step.

Benefits of using Step Functions

The benefits of a flexible and traceable workflow are endless.

Below are a few of the most important benefits:

  • Virtually not limited workflow execution – you can easily build long-running workflows. This service allows executing each workflow in a fully serverless way making it very cost-effective. Currently, a single workflow has a hard execution limit set to 1 year.
  • Manual approvals – Require manual approvals in your workflows? I recommend you Implementing Serverless Manual Approval Steps in AWS Step Functions and Amazon API Gateway article as a solution.
  • Easy workflow traceability – without this service, you would go to every single Lambda function log to gather the required information. This could get cumbersome with more Lambda functions getting involved. Step functions make it easy for you. In case of failure, you’ll get the Lambda function stack trace right in the failed State error message. Tracing workflow execution is also very easy.

Summary

In this article, we described the basics of the AWS Step Functions built an example workflow. We also provided the most important benefits of using it. This service solves a large real-world problem of orchestrating automated workflows. With extensive traceability, serverless infrastructure, and ease of execution – AWS Step Functions is the future of Serverless applications and automation.

We hope, that article was useful for you. If so, please, help us to spread it to the world!

How useful was this post?

Click on a star to rate it!

As you found this post useful...

Follow us on social media!

We are sorry that this post was not useful for you!

Let us improve this post!

Tell us how we can improve this post?

Top rated Udemy Courses to improve you career

Subscribe to our updates

Like this article?

Share on facebook
Share on Facebook
Share on twitter
Share on Twitter
Share on linkedin
Share on Linkdin
Share on pinterest
Share on Pinterest

Want to be an author of another post?

We’re looking for skilled technical authors for our blog!

Leave a comment

If you’d like to ask a question about the code or piece of configuration, feel free to use https://codeshare.io/ or a similar tool as Facebook comments are breaking code formatting.