Terraform Docker Lambda – Deployment Example

ByAndrei Maksimov

AWS Lambda is a serverless compute service that allows you to run code without provisioning or managing servers. Docker is a containerization platform that allows you to package and deploy applications as containers. Terraform is an open-source infrastructure as a code software tool that enables you to safely and predictably create, change, and improve infrastructure.

This article will show you how to use Terraform to deploy Docker Lambda functions. We will cover the following topics:

  • What is Terraform?
  • What is Docker?
  • What is AWS Lambda?
  • How to use Terraform to deploy Docker Lambda functions

Check the Terraform Lambda Tutorial – Easy Function Deployment to see how to deploy simple AWS Lambda functions using Terraform.

What is Terraform?

Terraform is an open-source infrastructure as a code software tool that enables you to safely and predictably create, change, and improve infrastructure. Terraform can manage infrastructure on various platforms, including AWS, Azure, and Google Cloud.

What is Docker?

Docker is a containerization platform that allows you to package and deploy applications as containers. Containers are isolated from each other and the host environment, making them portable and easy to deploy.

What is AWS Lambda?

AWS Lambda is a serverless compute service that allows you to run code without provisioning or managing servers. Events, such as HTTP requests, database changes, or file uploads, trigger Lambda functions.

How to use Terraform to deploy Docker Lambda functions

One of the common tasks in the cloud world is to replicate source code repositories from on-premises to the cloud or between cloud environments. So, to illustrate the approach, we decided to add Git and GitPython support to the Lambda function.

Let’s use Terraform to deploy Python Lambda functions backed by the container image.

Project structure

Here’s a project structure that we’ll be using during this demo:

$ tree lambda_container
lambda_container
├── README.md
├── lambdas
│   └── git_client
│       ├── Dockerfile
│       └── index.py
└── main.tf
2 directories, 4 files
  • lambdas – the folder where we put Lambda functions source code
  • main.tf – Terraform demo code, which will build a Docker container for the git_client Lambda function and deploy the function afterward

Dockerfile

Let’s describe a Docker container that will host all dependencies for our lambda functions. Here’s the Dockerfile content:

FROM public.ecr.aws/lambda/python:3.8
RUN yum update -y && \
  yum install -y git && \
  rm -Rf /var/cache/yum && \
  pip install git-remote-codecommit boto3 GitPython awscli
COPY index.py ${LAMBDA_TASK_ROOT}
CMD [ "index.handler" ]

We’re taking Python 3.8 public Docker image from Amazon as a base. Then we’re installing Git, cleaning the yum caches to make the container smaller, and installing required dependencies, which allow us to use Git with CodeCommit using IAM for authentication.

Next, we’re copying the index.py file to the folder where the Lambda function code should reside. Check Using AWS Lambda environment variables for additional information.

Finally, we’re specifying to execute the handler method from the index.py file at the container launch.

Lambda code

Once the Lambda container declaration is finalized, we can write a Lambda function to use it. Here’s a code example showing how to clone the Git repository. I’m sure you’ll be able to adjust this example for your personal needs:

import logging
import os
import git
 
TMP_DIR = "/tmp"
REPO_DIR = 'aws-config-rules'
REPO_URL = f'https://github.com/andreivmaksimov/{REPO_DIR}'
CLONE_PATH = os.path.join(TMP_DIR, REPO_DIR)
 
LOGGER = logging.getLogger(__name__)
LOGGER.setLevel(logging.INFO)
 
def clone(branch='master'):
   repo = git.Repo.clone_from(REPO_URL, CLONE_PATH, branch=branch)
 
   with repo.config_writer() as git_config:
       git_config.set_value('user', 'email', 'no-reply@hands-on.cloud')
       git_config.set_value('user', 'name', 'Git Lambda')
  
def handler(event, context):
   LOGGER.info('Event: %s', event)
 
   LOGGER.info('Cloning repo: %s', REPO_URL)
   clone()

In this code, we’re declaring required Python libraries, some constants, configuring logger, and a couple of functions:

  • def clone(branch='master') – this function shows how to clone Git repository
  • def handler(event, context) – this function is the main entry point to the Lambda function, it logs the incoming event and calling clone function

Terraform Docker Lambda code

Once we have Lambda code and declare its container, we can write Terraform code to automate the deployment. Here it is:

variable region {
 default = "us-east-1"
}
 
provider aws {
 region = var.region
}
 
data aws_caller_identity current {}
 
locals {
 prefix = "git"
 account_id          = data.aws_caller_identity.current.account_id
 ecr_repository_name = "${local.prefix}-demo-lambda-container"
 ecr_image_tag       = "latest"
}
 
resource aws_ecr_repository repo {
 name = local.ecr_repository_name
}
 
resource null_resource ecr_image {
 triggers = {
   python_file = md5(file("${path.module}/lambdas/git_client/index.py"))
   docker_file = md5(file("${path.module}/lambdas/git_client/Dockerfile"))
 }
 
 provisioner "local-exec" {
   command = <<EOF
           aws ecr get-login-password --region ${var.region} | docker login --username AWS --password-stdin ${local.account_id}.dkr.ecr.${var.region}.amazonaws.com
           cd ${path.module}/lambdas/git_client
           docker build -t ${aws_ecr_repository.repo.repository_url}:${local.ecr_image_tag} .
           docker push ${aws_ecr_repository.repo.repository_url}:${local.ecr_image_tag}
       EOF
 }
}
 
data aws_ecr_image lambda_image {
 depends_on = [
   null_resource.ecr_image
 ]
 repository_name = local.ecr_repository_name
 image_tag       = local.ecr_image_tag
}
 
resource aws_iam_role lambda {
 name = "${local.prefix}-lambda-role"
 assume_role_policy = <<EOF
{
   "Version": "2012-10-17",
   "Statement": [
       {
           "Action": "sts:AssumeRole",
           "Principal": {
               "Service": "lambda.amazonaws.com"
           },
           "Effect": "Allow"
       }
   ]
}
 EOF
}
 
data aws_iam_policy_document lambda {
   statement {
     actions = [
         "logs:CreateLogGroup",
         "logs:CreateLogStream",
         "logs:PutLogEvents"
     ]
     effect = "Allow"
     resources = [ "*" ]
     sid = "CreateCloudWatchLogs"
   }
 
   statement {
     actions = [
         "codecommit:GitPull",
         "codecommit:GitPush",
         "codecommit:GitBranch",
         "codecommit:ListBranches",
         "codecommit:CreateCommit",
         "codecommit:GetCommit",
         "codecommit:GetCommitHistory",
         "codecommit:GetDifferences",
         "codecommit:GetReferences",
         "codecommit:BatchGetCommits",
         "codecommit:GetTree",
         "codecommit:GetObjectIdentifier",
         "codecommit:GetMergeCommit"
     ]
     effect = "Allow"
     resources = [ "*" ]
     sid = "CodeCommit"
   }
}
 
resource aws_iam_policy lambda {
   name = "${local.prefix}-lambda-policy"
   path = "/"
   policy = data.aws_iam_policy_document.lambda.json
}
 
resource aws_lambda_function git {
 depends_on = [
   null_resource.ecr_image
 ]
 function_name = "${local.prefix}-lambda"
 role = aws_iam_role.lambda.arn
 timeout = 300
 image_uri = "${aws_ecr_repository.repo.repository_url}@${data.aws_ecr_image.lambda_image.id}"
 package_type = "Image"
}
 
output "lambda_name" {
 value = aws_lambda_function.git.id
}

This Terraform code was tested using Terraform version 1.5.0.

In this example, we’re using the following terraform resources:

  • aws_ecr_repository – creates an ECR registry where Terraform will save Docker container image, which our Lambda function will later use
  • null_resource – is used to build a Docker container and push it to the ECR registry, triggers checks changes in the Lambda function code and Dockerfile, and allows Terraform to understand when to rebuild the image and update the Lambda function
  • aws_ecr_image – allows us to query information about published Docker image
  • aws_iam_role, aws_iam_policy_document and aws_iam_policy – declares permissions (send logs to CloudWatch, CodeCommit access) for the Lambda function
  • aws_lambda_function – Lambda function declaration itself

Deployment

To test the solution, you need to deploy Terraform code:

terraform init
terraform apply -auto-approve

After that, you can execute the Lambda function:

aws lambda invoke --function-name git-lambda out --log-type Tail --query 'LogResult' --output text |  base64 -d

Here’s an expected output:

START RequestId: b8b742d6-5bd6-4098-90e3-5e30f5c6e816 Version: $LATEST
[INFO]  2021-03-16T02:10:28.064Z        b8b742d6-5bd6-4098-90e3-5e30f5c6e816    Event: {}
[INFO]  2021-03-16T02:10:28.064Z        b8b742d6-5bd6-4098-90e3-5e30f5c6e816    Cloning repo: https://github.com/andreivmaksimov/aws-config-rules
END RequestId: b8b742d6-5bd6-4098-90e3-5e30f5c6e816
REPORT RequestId: b8b742d6-5bd6-4098-90e3-5e30f5c6e816  Duration: 4069.15 ms    Billed Duration: 6131 ms        Memory Size: 128 MB     Max Memory Used: 83 MB  Init Duration: 2061.73 ms

Cleaning up

To clean up everything, execute the following command:

terraform destroy

Summary

This article built a Docker container for the AWS Lambda function and deployed the entire solution using Terraform. We hope you found this article useful. If so, please, help us to spread it to the world. If you have any questions, please, feel free to ask them in the chat section below.

Andrei Maksimov

I’m a passionate Cloud Infrastructure Architect with more than 20 years of experience in IT. In addition to the tech, I'm covering Personal Finance topics at https://amaksimov.com.

Any of my posts represent my personal experience and opinion about the topic.