How to manage AWS Lambda using Terraform

Andrei Maksimov
Andrei Maksimov

AWS Lambda is a compute service that allows you to run code without provisioning or managing servers. One of the most common use cases of AWS Lambda is to use it for service integration in the AWS cloud. Terraform is a tool for building, changing, and versioning your AWS infrastructure using the infrastructure as code (IaC) approach. Terraform simplifies AWS infrastructure management by making the deployment process easy and repeatable. In this blog post, we are going to look at how we can use Terraform to manage our AWS Lambda functions.

The source code of all examples for this article is available at our GitHub repository: managing-aws-lambda-terraform.

What is AWS Lambda, and what are its benefits?

AWS Lambda is one of AWS’s most powerful and innovative services. AWS Lambda allows developers to simplify and streamline their applications by enabling functions-based development. This approach provides a number of significant benefits, including higher efficiency, faster development times, lower costs, and more customization options.

First and foremost, AWS Lambda helps developers break down complex tasks into simpler functions that can be run independently and efficiently. Instead of building large monolithic applications that have to perform many different tasks all at once, AWS Lambda enables you to better focus on each individual action that needs to be taken in order for your application to be successful. This simplifies your application architecture and speeds up development time since you only need to focus on writing specific functions rather than building entire components from scratch.

Another major benefit of AWS Lambda is its low-cost structure. Since AWS Lambda runs only when certain events occur or when users request specific service actions, it consumes only a fraction of the resources required by traditional server-based applications. In addition, AWS offers generous free tier plans which allow you to get started with AWS Lambda at no cost and give you ample opportunity to experiment with different use cases.

Why use Terraform to deploy Lambda functions?

There are many benefits to using Terraform for deploying AWS Lambda. For starters, Terraform is a highly flexible tool that makes it easy to define and manage AWS resources of all kinds, including custom resources, networking components, application stacks, and more. With Terraform, you can easily create new AWS Lambda functions and configure them as needed to meet the specific needs of your project.

Additionally, Terraform allows you to define reusable components that can be reused across your AWS environment. This not only saves you time when creating new AWS Lambda functions but also ensures consistency and reliability across your entire system.

Another major benefit of using Terraform to deploy AWS Lambda is that it gives you full control over your AWS architecture. Using simple declarative syntax, Terraform lets you specify exactly how AWS resources should be configured – down to the level of individual parameters – without requiring any manual code or complicated CLI commands. This gives you greater flexibility in meeting the unique needs of your projects while reducing the risk of configuration errors that could compromise performance or security.

Overall, there are many reasons why Terraform is a great choice for deploying AWS Lambda functions.

How build Lambda using Terraform?

This section will review several scenarios of building the Lambda function. We’ll start from the simplest scenario possible and then move towards more complex examples, which should cover almost all your Lambda deployment requirements.

Terraform module for simple Lambda function

Let’s build the simplest but functional example of the AWS Lambda function which can be deployed to your AWS account. Check out this Terraform module at our GitHub repository: managing-aws-lambda-terraform/1_simple_lambda.

How to manage AWS Lambda using Terraform - Simple Lambda

Our Lambda function will have only basic AWSLambdaBasicExecutionRole role permissions.

AWSLambdaBasicExecutionRole role grants AWS Lambda permissions to upload logs to Amazon CloudWatch.

AWS Lambda execution role AWS documentation

Here’s what our folder structure would look like:

├── backend.tf
├── lambdas
│   └── simple_lambda
│       ├── demo_event.json
│       └── index.py
├── main.tf
├── outputs.tf
├── providers.tf
├── simple_lambda.tf
└── variables.tf

2 directories, 9 files

Such a project or module structure allows you to deploy multiple AWS Lambda functions at the same time. Just create a separate folder for every new Lambda function, replicate simple_lambda.tf file content and adjust it to new Lambda function deployment requirements.

Note: If your Lambda functions are similar to each other, it makes sense to c

If you’re using Terraform S3 backend for storing state files and DynamoDB for Terraform execution locks, you need to create backend.tf file, which will look something like this:

terraform {
  backend "s3" {
    bucket  = "hands-on-cloud-terraform-remote-state-s3"
    key     = "managing-aws-lambda-terraform-simple-lambda.tfstate"
    region  = "us-west-2"
    encrypt = "true"
    dynamodb_table = "hands-on-cloud-terraform-remote-state-dynamodb"
  }
}

You can easily deploy the S3 bucket and DynamoDB table for your own Terraform-managed AWS infrastructure using our already existing module: managing-aws-lambda-terraform/0_remote_state.

Here you’re specifying the following parameters for Terraform S3 backend:

  • bucket – the name of the S3 bucket to store Terraform state files
  • key – Terraform state file name (for the current Terraform module)
  • region – the region of the S3 bucket for storing Terraform state
  • encrypt – optional feature, allowing you to encrypt Terraform state file using S3 server-side encryption
  • dynamodb_table – optional DynamoDB table which is used to lock Terraform module executions from different machines at the same time

Next, we need to define some local variables, which we’ll use in this module. I’d prefer storing the common naming prefix for all module Terraform resources and common tags in the main.tf file:

locals {
  prefix   = "managing-alb-using-terraform"

  common_tags = {
    Environment = "dev"
    Project     = "hands-on.cloud"
  }
}

It is a good practice to restrict Terraform provider version and define AWS Region for the default Terraform provider:

# Set up Terraform provider version (if required)
terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 4.9"
    }
  }
}

# Defining AWS provider
provider "aws" {
  region = var.aws_region
}

The AWS provider definition contains a reference to Terraform variable aws_region. Let’s define this variable in the variables.tf file:

variable "aws_region" {
  default     = "us-east-1"
  description = "AWS Region to deploy VPC"
}

Now, we can jump straight to the meat of the AWS Lambda function definition in the Terraform code:

locals {
  resource_name_prefix          = "${local.prefix}-simple-lambda"
  lambda_code_path              = "${path.module}/lambdas/simple_lambda"
  lambda_archive_path           = "${path.module}/lambdas/simple_lambda.zip"
  lambda_handler                = "index.lambda_handler"
  lambda_description            = "This is simple Lambda function"
  lambda_runtime                = "python3.9"
  lambda_timeout                = 1
  lambda_concurrent_executions  = -1
  lambda_cw_log_group_name      = "/aws/lambda/${aws_lambda_function.simple_lambda.function_name}"
  lambda_log_retention_in_days  = 1
}

data "archive_file" "simple_lambda_zip" {
  source_dir = local.lambda_code_path
  output_path = local.lambda_archive_path
  type = "zip"
}

data "aws_iam_policy_document" "simple_lambda_assume_role_policy" {
  statement {
    actions = ["sts:AssumeRole"]

    principals {
      identifiers = ["lambda.amazonaws.com"]
      type        = "Service"
    }
  }
}

resource "aws_iam_role" "simple_lambda" {
  name = "${local.resource_name_prefix}-role"
  assume_role_policy = data.aws_iam_policy_document.simple_lambda_assume_role_policy.json
  managed_policy_arns = [
    "arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole"
  ]
  tags = merge(
    {
      Name = "${local.resource_name_prefix}-role"
    },
    local.common_tags
  )
}

resource "aws_lambda_function" "simple_lambda" {
  function_name = "${local.resource_name_prefix}-lambda"
  source_code_hash = data.archive_file.simple_lambda_zip.output_base64sha256
  filename = data.archive_file.simple_lambda_zip.output_path
  description = local.lambda_description
  role          = aws_iam_role.simple_lambda.arn
  handler = local.lambda_handler
  runtime = local.lambda_runtime
  timeout = local.lambda_timeout

  tags = merge(
    {
      Name = "${local.resource_name_prefix}-lambda"
    },
    local.common_tags
  )

  reserved_concurrent_executions = local.lambda_concurrent_executions
}

resource "aws_cloudwatch_log_group" "simple_lambda" {
  name = local.lambda_cw_log_group_name
  retention_in_days = local.lambda_log_retention_in_days
}

Here we’re defining several local variables:

  • resource_name_prefix – This variable will be used as a common naming prefix for all AWS resources that support names; such an approach allows deploying several versions of the same Terraform module having different prefixes.
  • lambda_code_path – AWS Lambda source code location (Python in our case); the ${path.module} construct allows specifying a path to the code based on the current module folder location on the file system.
  • lambda_archive_path – AWS Lambda zip file location (the zip file of the Lambda function will be created by Terraform automatically)
  • lambda_handlerthe Lambda function handler (Lambda function configuration)
  • lambda_description – the Lambda function text description (Lambda function configuration)
  • lambda_runtimeLambda function runtime (we’re using Python 3.9)
  • lambda_timeout – Lambda function execution timeout in seconds, max 900 (15 mins)
  • lambda_concurrent_executionsLambda reserved concurrency (-1 removes concurrency limitations)
  • lambda_cw_log_group_name – The name of CloudWatch Log Group for Lambda function logs
  • lambda_log_retention_in_days – The maximum number of days for the CloudWatch to store Lambda function logs in the Log Group

The Terraform archive_file simple_lambda_zip data resource creates an AWS Lambda deployment package that contains Lambda function source code and maybe several required dependencies (keep in mind AWS Lambda quotas – 50 MB max).

The aws_iam_policy_document constructs the IAM policy document allowing the Lambda function to assume the simple_lambda IAM role (defined below in the code).

The aws_iam_role sets of exclusive IAM AWSLambdaBasicExecutionRole managed policy ARNs to attach to the IAM role consumed by the Lambda function.

The aws_lambda_function resource defines AWS Lambda function configuration parameters such as timeout, IAM role, runtime environment, etc (all of them are defined using Terraform local variables.

The aws_cloudwatch_log_group resource defines the CloudWatch Log Group for Lambda function logs. This resource is required if you want Terraform to wipe out all Lambda function logs when destroying the module.

Simple Lambda function

As soon as we defined all required Terraform resources for deploying AWS Lambda to your account, we need to develop the function itself. I’ll use the following code to define the demo Lambda function that returns the Lambda event in response to its invocation:

def lambda_handler(event, context):
    response = {
        'event': event
    }
    return {
        'statusCode': 200,
        'response': response
    }

Now, as soon as we defined the AWS Lambda function code and all required Terraform resources to deploy the simplest Lambda function, we can jump to the deployment process.

How do I use Terraform to deploy AWS Lambda?

To deploy AWS Lambda using Terraform, you need to go to Terraform module source code folder and run the following commands: terraform init and terraform apply:

The first command will download all required Terraform modules, providers and initialize your project (make it ready) for the deployment:

terraform init

Next, if you’d like to see what changes Terraform will make in your infrastructure or test that you’ve defined everything without any syntax errors, use the following command:

terraform plan

Finally, you can apply module changes. Use the -auto-approve flag to prevent Terraform from asking you a question if you really want to deploy module changes:

terraform apply -auto-approve

Testing deployed Lambda function

As soon as you’ve deployed AWS Lambda to your AWS account, you can execute it by using the AWS CLI command (provided in the module outputs):

aws lambda invoke \
  --function-name managing-alb-using-terraform-simple-lambda-lambda \
  --cli-binary-format raw-in-base64-out \
  /tmp/managing-alb-using-terraform-simple-lambda-lambda-response.json

Another method of invoking AWS Lambda is by generating a test event from the AWS console:

How to manage AWS Lambda using Terraform - AWS Lambda Test button

For more information about AWS Lambda testing, check out our articles:

Deploying AWS Lambda inside VPC

To deploy AWS Lambda inside VPC using Terraform you have to attach it to the VPC private subnets ONLY. Several use cases to keep in mind:

  • If AWS Lambda has to interact with internet resources, VPC private subnet has to have a route to 0.0.0.0/0 CIDR block through NAT Gateway deployed in the same VPC’s public subnet.
  • If AWS Lambda has to interact with AWS services without accessing the internet, you have to deploy the required VPC Endpoints to the same VPC private subnet where you’re attaching AWS Lambda.

As soon as most of the Terraform configuration for deploying AWS Lambda inside a VPC will be the same, we’ll show only the most important piece of the code. As usual, check out the complete Terraform module code example in our GitHub repository: managing-aws-lambda-terraform/3_vpc_lambda.

How to manage AWS Lambda using Terraform - AWS Lambda inside VPC

Here’s the AWS Lambda function attached to the VPC Terraform definition:

locals {
  resource_name_prefix          = "${local.prefix}-vpc-lambda"
  lambda_code_path              = "${path.module}/lambdas/vpc_lambda"
  lambda_archive_path           = "${path.module}/lambdas/vpc_lambda.zip"
  lambda_handler                = "index.lambda_handler"
  lambda_description            = "This is VPC Lambda function"
  lambda_runtime                = "python3.9"
  lambda_timeout                = 5
  lambda_concurrent_executions  = -1
  lambda_cw_log_group_name      = "/aws/lambda/${aws_lambda_function.vpc_lambda.function_name}"
  lambda_log_retention_in_days  = 1
}

data "archive_file" "vpc_lambda_zip" {
  source_dir = local.lambda_code_path
  output_path = local.lambda_archive_path
  type = "zip"
}

data "aws_iam_policy_document" "vpc_lambda_assume_role_policy" {
  statement {
    actions = ["sts:AssumeRole"]

    principals {
      identifiers = ["lambda.amazonaws.com"]
      type        = "Service"
    }
  }
}

data "aws_iam_policy_document" "vpc_lambda_list_s3_buckets" {
  statement {
    actions = [
      "s3:ListAllMyBuckets",
      "s3:ListBucket"
    ]

    resources = [
      "*"
    ]
  }
}

resource "aws_iam_policy" "vpc_lambda_list_s3_buckets" {
  policy = data.aws_iam_policy_document.vpc_lambda_list_s3_buckets.json
}

resource "aws_iam_role" "vpc_lambda" {
  name = "${local.resource_name_prefix}-role"
  assume_role_policy = data.aws_iam_policy_document.vpc_lambda_assume_role_policy.json
  managed_policy_arns = [
    aws_iam_policy.vpc_lambda_list_s3_buckets.arn,
    "arn:aws:iam::aws:policy/service-role/AWSLambdaVPCAccessExecutionRole",
    "arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole"
  ]
  tags = merge(
    {
      Name = "${local.resource_name_prefix}-role"
    },
    local.common_tags
  )
}

resource "aws_security_group" "vpc_lambda" {
  name        = "${local.resource_name_prefix}-sg"
  description = "Allow outbound traffic for ${local.resource_name_prefix}-lambda"
  vpc_id      = local.vpc_id

  egress {
    from_port        = 0
    to_port          = 0
    protocol         = "-1"
    cidr_blocks      = ["0.0.0.0/0"]
    ipv6_cidr_blocks = ["::/0"]
  }

  tags = merge(
    {
      Name = local.resource_name_prefix
    },
    local.common_tags
  )
}

resource "aws_lambda_function" "vpc_lambda" {
  function_name = local.resource_name_prefix
  source_code_hash = data.archive_file.vpc_lambda_zip.output_base64sha256
  filename = data.archive_file.vpc_lambda_zip.output_path
  description = local.lambda_description
  role          = aws_iam_role.vpc_lambda.arn
  handler = local.lambda_handler
  runtime = local.lambda_runtime
  timeout = local.lambda_timeout

  vpc_config {
    security_group_ids = [aws_security_group.vpc_lambda.id]
    subnet_ids         = local.private_subnets
  }

  tags = merge(
    {
      Name = local.resource_name_prefix
    },
    local.common_tags
  )

  reserved_concurrent_executions = local.lambda_concurrent_executions
}

# CloudWatch Log Group for the Lambda function
resource "aws_cloudwatch_log_group" "vpc_lambda" {
  name = local.lambda_cw_log_group_name
  retention_in_days = local.lambda_log_retention_in_days
}

The most important part of AWS Lambda configuration, in this case, is the vpc_config attribute of the aws_lambda_function Terraform resource. Which requires AWS VPC private subnets’ IDs for the AWS Lambda ENIs and Security Group IDs for controlling egress traffic initiated by the AWS Lambda.

FAQ

Does Terraform support Lambda?

Yes, Terraform supports AWS Lambda. You can find more information about Lambda support here: https://www.terraform.io/docs/providers/aws/r/lambda_function.html

What is AWSLambdaBasicExecutionRole?

An execution role is a special type of AWS IAM policy that defines the permissions and resources required for an AWS Lambda function to run. AWSLambdaBasicExecutionRole is a predefined managed policy with basic permissions that are necessary for executing Lambda functions, such as reading logs or accessing other AWS services.

What is AWSLambdaVPCAccessExecutionRole?

AWSLambdaVPCAccessExecutionRole is one of the predefined managed IAM policies that provide additional permissions for AWS Lambda functions running in a VPC. This policy grants access to resources such as EC2, IAM, and Elastic Load Balancing, allowing Lambda functions to interact with other services running in the same VPC.

Can I run Terraform in Lambda?

There is currently no support for running Terraform in Lambda. However, you can use AWS Lambda custom runtime to put Terraform binaries inside of AWS Lambda functions. At the same time, keep in mind, that AWS Lambda execution time is limited to 15 mins, which makes it impossible to deploy several AWS services like Amazon RDS, Amazon OpenSearch Service, and Terraform modules with a long execution time. Ultimately, the best way to determine whether or not it is possible to run Terraform in Lambda will depend on your specific use case and requirements.

Summary

Terraform is a powerful tool that can be used to manage your AWS Lambda functions. In this blog post, we’ve covered how to use Terraform to create and deploy a simple AWS Lambda function and how to deploy the AWS Lambda function inside of a VPC. If you’re looking for a way to easily manage your AWS Lambda functions, Terraform is the tool for you.

Like this article?

Share on Facebook
Share on Twitter
Share on Linkdin
Share on Pinterest

Want to be an author of another post?

We’re looking for skilled technical authors for our blog!

Leave a comment

If you’d like to ask a question about the code or piece of configuration, feel free to use https://codeshare.io/ or a similar tool as Facebook comments are breaking code formatting.