AWS Lambda is a serverless compute service that allows you to run code without provisioning or managing servers. Docker is a containerization platform that allows you to package and deploy applications as containers. Terraform is an open-source infrastructure as a code software tool that enables you to safely and predictably create, change, and improve infrastructure.
This article will show you how to use Terraform to deploy Docker Lambda functions. We will cover the following topics:
- What is Terraform?
- What is Docker?
- What is AWS Lambda?
- How to use Terraform to deploy Docker Lambda functions
Check the Terraform Lambda Tutorial – Easy Function Deployment to see how to deploy simple AWS Lambda functions using Terraform.
Table of contents
What is Terraform?
Terraform is an open-source infrastructure as a code software tool that enables you to safely and predictably create, change, and improve infrastructure. Terraform can manage infrastructure on various platforms, including AWS, Azure, and Google Cloud.
What is Docker?
Docker is a containerization platform that allows you to package and deploy applications as containers. Containers are isolated from each other and the host environment, making them portable and easy to deploy.
What is AWS Lambda?
AWS Lambda is a serverless compute service that allows you to run code without provisioning or managing servers. Events, such as HTTP requests, database changes, or file uploads, trigger Lambda functions.
How to use Terraform to deploy Docker Lambda functions
One of the common tasks in the cloud world is to replicate source code repositories from on-premises to the cloud or between cloud environments. So, to illustrate the approach, we decided to add Git and GitPython support to the Lambda function.
Let’s use Terraform to deploy Python Lambda functions backed by the container image.
Project structure
Here’s a project structure that we’ll be using during this demo:
$ tree lambda_container
lambda_container
├── README.md
├── lambdas
│ └── git_client
│ ├── Dockerfile
│ └── index.py
└── main.tf
2 directories, 4 files
lambdas
– the folder where we put Lambda functions source codemain.tf
– Terraform demo code, which will build a Docker container for the git_client Lambda function and deploy the function afterward
Dockerfile
Let’s describe a Docker container that will host all dependencies for our lambda functions. Here’s the Dockerfile
content:
FROM public.ecr.aws/lambda/python:3.8
RUN yum update -y && \
yum install -y git && \
rm -Rf /var/cache/yum && \
pip install git-remote-codecommit boto3 GitPython awscli
COPY index.py ${LAMBDA_TASK_ROOT}
CMD [ "index.handler" ]
We’re taking Python 3.8 public Docker image from Amazon as a base. Then we’re installing Git, cleaning the yum caches to make the container smaller, and installing required dependencies, which allow us to use Git with CodeCommit using IAM for authentication.
Next, we’re copying the index.py
file to the folder where the Lambda function code should reside. Check Using AWS Lambda environment variables for additional information.
Finally, we’re specifying to execute the handler method from the index.py file at the container launch.
Lambda code
Once the Lambda container declaration is finalized, we can write a Lambda function to use it. Here’s a code example showing how to clone the Git repository. I’m sure you’ll be able to adjust this example for your personal needs:
import logging
import os
import git
TMP_DIR = "/tmp"
REPO_DIR = 'aws-config-rules'
REPO_URL = f'https://github.com/andreivmaksimov/{REPO_DIR}'
CLONE_PATH = os.path.join(TMP_DIR, REPO_DIR)
LOGGER = logging.getLogger(__name__)
LOGGER.setLevel(logging.INFO)
def clone(branch='master'):
repo = git.Repo.clone_from(REPO_URL, CLONE_PATH, branch=branch)
with repo.config_writer() as git_config:
git_config.set_value('user', 'email', 'no-reply@hands-on.cloud')
git_config.set_value('user', 'name', 'Git Lambda')
def handler(event, context):
LOGGER.info('Event: %s', event)
LOGGER.info('Cloning repo: %s', REPO_URL)
clone()
In this code, we’re declaring required Python libraries, some constants, configuring logger, and a couple of functions:
def clone(branch='master')
– this function shows how to clone Git repositorydef handler(event, context)
– this function is the main entry point to the Lambda function, it logs the incoming event and callingclone
function
Terraform Docker Lambda code
Once we have Lambda code and declare its container, we can write Terraform code to automate the deployment. Here it is:
variable region {
default = "us-east-1"
}
provider aws {
region = var.region
}
data aws_caller_identity current {}
locals {
prefix = "git"
account_id = data.aws_caller_identity.current.account_id
ecr_repository_name = "${local.prefix}-demo-lambda-container"
ecr_image_tag = "latest"
}
resource aws_ecr_repository repo {
name = local.ecr_repository_name
}
resource null_resource ecr_image {
triggers = {
python_file = md5(file("${path.module}/lambdas/git_client/index.py"))
docker_file = md5(file("${path.module}/lambdas/git_client/Dockerfile"))
}
provisioner "local-exec" {
command = <<EOF
aws ecr get-login-password --region ${var.region} | docker login --username AWS --password-stdin ${local.account_id}.dkr.ecr.${var.region}.amazonaws.com
cd ${path.module}/lambdas/git_client
docker build -t ${aws_ecr_repository.repo.repository_url}:${local.ecr_image_tag} .
docker push ${aws_ecr_repository.repo.repository_url}:${local.ecr_image_tag}
EOF
}
}
data aws_ecr_image lambda_image {
depends_on = [
null_resource.ecr_image
]
repository_name = local.ecr_repository_name
image_tag = local.ecr_image_tag
}
resource aws_iam_role lambda {
name = "${local.prefix}-lambda-role"
assume_role_policy = <<EOF
{
"Version": "2012-10-17",
"Statement": [
{
"Action": "sts:AssumeRole",
"Principal": {
"Service": "lambda.amazonaws.com"
},
"Effect": "Allow"
}
]
}
EOF
}
data aws_iam_policy_document lambda {
statement {
actions = [
"logs:CreateLogGroup",
"logs:CreateLogStream",
"logs:PutLogEvents"
]
effect = "Allow"
resources = [ "*" ]
sid = "CreateCloudWatchLogs"
}
statement {
actions = [
"codecommit:GitPull",
"codecommit:GitPush",
"codecommit:GitBranch",
"codecommit:ListBranches",
"codecommit:CreateCommit",
"codecommit:GetCommit",
"codecommit:GetCommitHistory",
"codecommit:GetDifferences",
"codecommit:GetReferences",
"codecommit:BatchGetCommits",
"codecommit:GetTree",
"codecommit:GetObjectIdentifier",
"codecommit:GetMergeCommit"
]
effect = "Allow"
resources = [ "*" ]
sid = "CodeCommit"
}
}
resource aws_iam_policy lambda {
name = "${local.prefix}-lambda-policy"
path = "/"
policy = data.aws_iam_policy_document.lambda.json
}
resource aws_lambda_function git {
depends_on = [
null_resource.ecr_image
]
function_name = "${local.prefix}-lambda"
role = aws_iam_role.lambda.arn
timeout = 300
image_uri = "${aws_ecr_repository.repo.repository_url}@${data.aws_ecr_image.lambda_image.id}"
package_type = "Image"
}
output "lambda_name" {
value = aws_lambda_function.git.id
}
This Terraform code was tested using Terraform version 1.5.0.
In this example, we’re using the following terraform resources:
aws_ecr_repository
– creates an ECR registry where Terraform will save Docker container image, which our Lambda function will later usenull_resource
– is used to build a Docker container and push it to the ECR registry, triggers checks changes in the Lambda function code and Dockerfile, and allows Terraform to understand when to rebuild the image and update the Lambda functionaws_ecr_image
– allows us to query information about published Docker imageaws_iam_role
,aws_iam_policy_document
andaws_iam_policy
– declares permissions (send logs to CloudWatch, CodeCommit access) for the Lambda functionaws_lambda_function
– Lambda function declaration itself
Deployment
To test the solution, you need to deploy Terraform code:
terraform init
terraform apply -auto-approve
After that, you can execute the Lambda function:
aws lambda invoke --function-name git-lambda out --log-type Tail --query 'LogResult' --output text | base64 -d
Here’s an expected output:
START RequestId: b8b742d6-5bd6-4098-90e3-5e30f5c6e816 Version: $LATEST
[INFO] 2021-03-16T02:10:28.064Z b8b742d6-5bd6-4098-90e3-5e30f5c6e816 Event: {}
[INFO] 2021-03-16T02:10:28.064Z b8b742d6-5bd6-4098-90e3-5e30f5c6e816 Cloning repo: https://github.com/andreivmaksimov/aws-config-rules
END RequestId: b8b742d6-5bd6-4098-90e3-5e30f5c6e816
REPORT RequestId: b8b742d6-5bd6-4098-90e3-5e30f5c6e816 Duration: 4069.15 ms Billed Duration: 6131 ms Memory Size: 128 MB Max Memory Used: 83 MB Init Duration: 2061.73 ms
Cleaning up
To clean up everything, execute the following command:
terraform destroy
Summary
This article built a Docker container for the AWS Lambda function and deployed the entire solution using Terraform. We hope you found this article useful. If so, please, help us to spread it to the world. If you have any questions, please, feel free to ask them in the chat section below.
Nice!