Terraform Lambda Tutorial – Easy Function Deployment

AWS Lambda is a compute service that allows you to run code without provisioning or managing servers. One of the most common use cases of AWS Lambda is to use it for service integration in the AWS cloud. Terraform is a tool for building, changing, and versioning your AWS infrastructure using the infrastructure as code (IaC) approach. Terraform simplifies AWS infrastructure management by making the deployment process easy and repeatable. This blog post describes how we can use Terraform Lambda resources to manage our AWS Lambda functions deployment.

The source code for this article’s Terraform and AWS Lambda examples is available at our GitHub repository: managing-aws-lambda-terraform.

What is AWS Lambda, and what are its benefits?

AWS Lambda is one of AWS’s most powerful and innovative services. AWS Lambda allows developers to simplify and streamline their applications by enabling functions-based development. This approach provides significant benefits, including higher efficiency, faster development times, lower costs, and more customization options.

First and foremost, AWS Lambda helps developers break down complex tasks into simpler functions that can be run independently and efficiently. Instead of building large monolithic applications that have to perform many different tasks simultaneously, AWS Lambda enables you to focus better on each action needed for your application to succeed. This simplifies your application architecture and speeds up development time since you only need to focus on writing specific functions rather than building entire components from scratch.

Another major benefit of AWS Lambda is its low-cost structure. Since AWS Lambda runs only when certain events occur or when users request specific service actions, it consumes only a fraction of the resources required by traditional server-based applications. In addition, AWS offers generous free tier plans which allow you to get started with AWS Lambda at no cost and give you ample opportunity to experiment with different use cases.

Why use Terraform to deploy Lambda functions?

There are many benefits to using Terraform for deploying AWS Lambda. For starters, Terraform is a highly flexible tool that makes it easy to define and manage AWS resources of all kinds, including custom resources, networking components, application stacks, and more. With Terraform, you can easily create new AWS Lambda functions and configure them to meet your project’s specific needs.

Additionally, Terraform allows you to define reusable components that can be reused across your AWS environment. This saves you time creating new AWS Lambda functions and ensures consistency and reliability across your entire system.

Another significant benefit of using Terraform to deploy AWS Lambda is that it gives you complete control over your AWS architecture. Using simple declarative syntax, Terraform lets you specify precisely how AWS resources should be configured – down to individual parameters – without requiring any manual code or complicated CLI commands. This gives you greater flexibility in meeting the unique needs of your projects while reducing the risk of configuration errors that could compromise performance or security.

Overall, there are many reasons why Terraform is an excellent choice for deploying AWS Lambda functions.

How build Lambda using Terraform?

This section will review several scenarios of building the Lambda function. We’ll start from the most straightforward scenario possible and then move towards more complex examples, which should cover almost all your Lambda deployment requirements.

Terraform module for simple Lambda function

Let’s build the most straightforward but functional example of the AWS Lambda function, which can be deployed to your AWS account. Check out this Terraform Lambda module at our GitHub repository: managing-aws-lambda-terraform/1_simple_lambda.

Our Terraform Lambda function will have only basic AWSLambdaBasicExecutionRole role permissions. AWSLambdaBasicExecutionRole role grants AWS Lambda permissions to upload logs to Amazon CloudWatch.

Here’s what our folder structure would look like:

├── backend.tf
├── lambdas
│   └── simple_lambda
│       ├── demo_event.json
│       └── index.py
├── main.tf
├── outputs.tf
├── providers.tf
├── simple_lambda.tf
└── variables.tf
2 directories, 9 files

Such a project or module structure allows you to deploy multiple AWS Lambda functions at the same time. Just create a separate folder for every new Lambda function, replicate simple_lambda.tf file content and adjust it to new Lambda function deployment requirements.

If you’re using Terraform S3 backend for storing state files and DynamoDB for Terraform execution locks, you need to create a backend.tf file, which will look something like this:

terraform {
  backend "s3" {
    bucket  = "hands-on-cloud-terraform-remote-state-s3"
    key     = "managing-aws-lambda-terraform-simple-lambda.tfstate"
    region  = "us-west-2"
    encrypt = "true"
    dynamodb_table = "hands-on-cloud-terraform-remote-state-dynamodb"
  }
}

You can quickly deploy the S3 bucket and DynamoDB table for your Terraform-managed AWS infrastructure using our existing module: managing-aws-lambda-terraform/0_remote_state.

Here you’re specifying the following parameters for Terraform S3 backend:

• bucket – the name of the S3 bucket to store Terraform state files
• key – Terraform state file name (for the current Terraform module)
• region – the region of the S3 bucket for storing Terraform state
• encrypt – optional feature, allowing you to encrypt Terraform state file using S3 server-side encryption
• dynamodb_table – optional DynamoDB table, which is used to lock Terraform module executions from different machines at the same time

Next, we need to define some local variables, which we’ll use in this module. I’d prefer storing the common naming prefix for all module Terraform resources and common tags in the main.tf file:

locals {
  prefix   = "managing-alb-using-terraform"
  common_tags = {
    Environment = "dev"
    Project     = "hands-on.cloud"
  }
}

It is a good practice to restrict Terraform provider version and define AWS Region for the default Terraform provider:

# Set up Terraform provider version (if required)
terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 4.9"
    }
  }
}
# Defining AWS provider
provider "aws" {
  region = var.aws_region
}

The AWS provider definition contains a reference to Terraform variable aws_region. Let’s define this variable in the variables.tf file:

variable "aws_region" {
  default     = "us-east-1"
  description = "AWS Region to deploy VPC"
}

Now, we can jump straight to the meat of the AWS Lambda function definition in the Terraform code:

locals {
  resource_name_prefix          = "${local.prefix}-simple-lambda"
  lambda_code_path              = "${path.module}/lambdas/simple_lambda"
  lambda_archive_path           = "${path.module}/lambdas/simple_lambda.zip"
  lambda_handler                = "index.lambda_handler"
  lambda_description            = "This is simple Lambda function"
  lambda_runtime                = "python3.9"
  lambda_timeout                = 1
  lambda_concurrent_executions  = -1
  lambda_cw_log_group_name      = "/aws/lambda/${aws_lambda_function.simple_lambda.function_name}"
  lambda_log_retention_in_days  = 1
}
data "archive_file" "simple_lambda_zip" {
  source_dir = local.lambda_code_path
  output_path = local.lambda_archive_path
  type = "zip"
}
data "aws_iam_policy_document" "simple_lambda_assume_role_policy" {
  statement {
    actions = ["sts:AssumeRole"]
    principals {
      identifiers = ["lambda.amazonaws.com"]
      type        = "Service"
    }
  }
}
resource "aws_iam_role" "simple_lambda" {
  name = "${local.resource_name_prefix}-role"
  assume_role_policy = data.aws_iam_policy_document.simple_lambda_assume_role_policy.json
  managed_policy_arns = [
    "arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole"
  ]
  tags = merge(
    {
      Name = "${local.resource_name_prefix}-role"
    },
    local.common_tags
  )
}
resource "aws_lambda_function" "simple_lambda" {
  function_name = "${local.resource_name_prefix}-lambda"
  source_code_hash = data.archive_file.simple_lambda_zip.output_base64sha256
  filename = data.archive_file.simple_lambda_zip.output_path
  description = local.lambda_description
  role          = aws_iam_role.simple_lambda.arn
  handler = local.lambda_handler
  runtime = local.lambda_runtime
  timeout = local.lambda_timeout
  tags = merge(
    {
      Name = "${local.resource_name_prefix}-lambda"
    },
    local.common_tags
  )
  reserved_concurrent_executions = local.lambda_concurrent_executions
}
resource "aws_cloudwatch_log_group" "simple_lambda" {
  name = local.lambda_cw_log_group_name
  retention_in_days = local.lambda_log_retention_in_days
}

Here we’re defining several local variables:

• resource_name_prefix – This variable will be used as a common naming prefix for all AWS resources that support names; such an approach allows deploying several versions of the same Terraform module having different prefixes.
• lambda_code_path – AWS Lambda source code location (Python in our case); the ${path.module} construct allows specifying a path to the code based on the current module folder location on the file system.
• lambda_archive_path – AWS Lambda zip file location (the zip file of the Lambda function will be created by Terraform automatically)
• lambda_handler – the Lambda function handler (Lambda function configuration)
• lambda_description – the Lambda function text description (Lambda function configuration)
• lambda_runtime – Lambda function runtime (we’re using Python 3.9)
• lambda_timeout – Lambda function execution timeout in seconds, max 900 (15 mins)
• lambda_concurrent_executions – Lambda reserved concurrency (-1 removes concurrency limitations)
• lambda_cw_log_group_name – The name of CloudWatch Log Group for Lambda function logs
• lambda_log_retention_in_days – The maximum number of days for the CloudWatch to store Lambda function logs in the Log Group

The Terraform archive_file data resource creates an AWS Lambda deployment package that contains Lambda function source code and maybe several required dependencies (keep in mind AWS Lambda quotas – 50 MB max).

The aws_iam_policy_document constructs the IAM policy document allowing the Lambda function to assume the simple_lambda IAM role (defined below in the code).

The aws_iam_role sets of exclusive IAM AWSLambdaBasicExecutionRole managed policy ARNs to attach to the IAM role consumed by the Lambda function.

The Terraform Lambda resources (aws_lambda_function resource) define AWS Lambda function configuration parameters such as timeout, IAM role, runtime environment, etc. All of them are defined using Terraform local variables.

The aws_cloudwatch_log_group resource defines the CloudWatch Log Group for Lambda function logs. This resource is required if you want Terraform to wipe out all Lambda function logs when destroying the module.

Simple Lambda function

We must develop the function as soon as we define all required Terraform resources for deploying AWS Lambda to your account. I’ll use the following code to define the demo Lambda function that returns the Lambda event in response to its invocation:

def lambda_handler(event, context):
    response = {
        'event': event
    }
    return {
        'statusCode': 200,
        'response': response
    }

As soon as we define the AWS Lambda function code and all required Terraform resources to deploy the simplest Lambda function, we can jump to the deployment process.

How do I use Terraform to deploy AWS Lambda?

To deploy AWS Lambda using Terraform, you need to go to Terraform module source code folder and run the following commands: terraform init and terraform apply:

The first command will download all required Terraform modules and providers and initialize your project (make it ready) for the deployment:

terraform init

Next, if you’d like to see what changes Terraform will make in your infrastructure or test that you’ve defined everything without any syntax errors, use the following command:

terraform plan

Finally, you can apply module changes. Use the -auto-approve flag to prevent Terraform from asking you a question if you want to deploy module changes:

terraform apply -auto-approve

Testing deployed Lambda function

As soon as you’ve deployed AWS Lambda to your AWS account, you can execute it by using the AWS CLI command (provided in the module outputs):

aws lambda invoke \
  --function-name managing-alb-using-terraform-simple-lambda-lambda \
  --cli-binary-format raw-in-base64-out \
  /tmp/managing-alb-using-terraform-simple-lambda-lambda-response.json

Another method of invoking AWS Lambda is by generating a test event from the AWS console:

For more information about AWS Lambda testing, check out our articles:

• Unit Test AWS Lambda Python – Complete Tutorial
• Testing Python AWS applications using LocalStack

Deploying AWS Lambda inside VPC

To deploy AWS Lambda inside VPC using Terraform, you must only attach it to the VPC private subnets. Several use cases to keep in mind:

• If AWS Lambda has to interact with internet resources, VPC private subnet has to have a route to 0.0.0.0/0 CIDR block through NAT Gateway deployed in the same VPC’s public subnet.
• If AWS Lambda has to interact with AWS services without accessing the internet, you must deploy the required VPC Endpoints to the same VPC private subnet where you’re attaching AWS Lambda.

As soon as most of the Terraform configuration for deploying AWS Lambda inside a VPC is the same, we’ll show only the most critical piece of the code. As usual, check out the complete Terraform module code example in our GitHub repository: managing-aws-lambda-terraform/3_vpc_lambda.

Here’s the AWS Lambda function attached to the VPC Terraform definition:

locals {
  resource_name_prefix          = "${local.prefix}-vpc-lambda"
  lambda_code_path              = "${path.module}/lambdas/vpc_lambda"
  lambda_archive_path           = "${path.module}/lambdas/vpc_lambda.zip"
  lambda_handler                = "index.lambda_handler"
  lambda_description            = "This is VPC Lambda function"
  lambda_runtime                = "python3.9"
  lambda_timeout                = 5
  lambda_concurrent_executions  = -1
  lambda_cw_log_group_name      = "/aws/lambda/${aws_lambda_function.vpc_lambda.function_name}"
  lambda_log_retention_in_days  = 1
}
data "archive_file" "vpc_lambda_zip" {
  source_dir = local.lambda_code_path
  output_path = local.lambda_archive_path
  type = "zip"
}
data "aws_iam_policy_document" "vpc_lambda_assume_role_policy" {
  statement {
    actions = ["sts:AssumeRole"]
    principals {
      identifiers = ["lambda.amazonaws.com"]
      type        = "Service"
    }
  }
}
data "aws_iam_policy_document" "vpc_lambda_list_s3_buckets" {
  statement {
    actions = [
      "s3:ListAllMyBuckets",
      "s3:ListBucket"
    ]
    resources = [
      "*"
    ]
  }
}
resource "aws_iam_policy" "vpc_lambda_list_s3_buckets" {
  policy = data.aws_iam_policy_document.vpc_lambda_list_s3_buckets.json
}
resource "aws_iam_role" "vpc_lambda" {
  name = "${local.resource_name_prefix}-role"
  assume_role_policy = data.aws_iam_policy_document.vpc_lambda_assume_role_policy.json
  managed_policy_arns = [
    aws_iam_policy.vpc_lambda_list_s3_buckets.arn,
    "arn:aws:iam::aws:policy/service-role/AWSLambdaVPCAccessExecutionRole",
    "arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole"
  ]
  tags = merge(
    {
      Name = "${local.resource_name_prefix}-role"
    },
    local.common_tags
  )
}
resource "aws_security_group" "vpc_lambda" {
  name        = "${local.resource_name_prefix}-sg"
  description = "Allow outbound traffic for ${local.resource_name_prefix}-lambda"
  vpc_id      = local.vpc_id
  egress {
    from_port        = 0
    to_port          = 0
    protocol         = "-1"
    cidr_blocks      = ["0.0.0.0/0"]
    ipv6_cidr_blocks = ["::/0"]
  }
  tags = merge(
    {
      Name = local.resource_name_prefix
    },
    local.common_tags
  )
}
resource "aws_lambda_function" "vpc_lambda" {
  function_name = local.resource_name_prefix
  source_code_hash = data.archive_file.vpc_lambda_zip.output_base64sha256
  filename = data.archive_file.vpc_lambda_zip.output_path
  description = local.lambda_description
  role          = aws_iam_role.vpc_lambda.arn
  handler = local.lambda_handler
  runtime = local.lambda_runtime
  timeout = local.lambda_timeout
  vpc_config {
    security_group_ids = [aws_security_group.vpc_lambda.id]
    subnet_ids         = local.private_subnets
  }
  tags = merge(
    {
      Name = local.resource_name_prefix
    },
    local.common_tags
  )
  reserved_concurrent_executions = local.lambda_concurrent_executions
}
# CloudWatch Log Group for the Lambda function
resource "aws_cloudwatch_log_group" "vpc_lambda" {
  name = local.lambda_cw_log_group_name
  retention_in_days = local.lambda_log_retention_in_days
}

The most important part of AWS Lambda configuration, in this case, is the vpc_config attribute of the aws_lambda_function Terraform resource, which requires AWS VPC private subnets’ IDs for the AWS Lambda ENIs and Security Group IDs for controlling egress traffic initiated by the AWS Lambda.

FAQ

Summary

Terraform is a powerful tool that can be used to manage your AWS Lambda functions. In this blog post, we’ve covered how to use Terraform to create and deploy a simple AWS Lambda function and how to deploy the AWS Lambda function inside of a VPC.