Terraform Docker Lambda – Deployment Example

At the end of 2020, AWS announced support of container images for Lambda. This feature allows you to package and deploy Lambda functions as container images of up to 10 GB in size. This Terraform Docker Lambda example covers how you can use Terraform to deploy Python Lambda functions backed by the container image.

One of the common tasks in the cloud world is to replicate source code repositories from on-premises to the cloud or between cloud environments. So, to illustrate the approach, we decided to add Git and GitPython support to the Lambda function.

Project structure

Here’s a project structure that we’ll be using during this demo:

$ tree lambda_container
lambda_container
├── README.md
├── lambdas
│   └── git_client
│       ├── Dockerfile
│       └── index.py
└── main.tf
2 directories, 4 files

• lambdas – the folder where we put Lambda functions source code
• main.tf – Terraform demo code, which will build a Docker container for the git_client Lambda function and deploy the function afterward

Dockerfile

Let’s describe a Docker container that will host all dependencies for our lambda functions. Here’s the Dockerfile content:

FROM public.ecr.aws/lambda/python:3.8
RUN yum update -y && \
  yum install -y git && \
  rm -Rf /var/cache/yum && \
  pip install git-remote-codecommit boto3 GitPython awscli
COPY index.py ${LAMBDA_TASK_ROOT}
CMD [ "index.handler" ]

We’re taking Python 3.8 public Docker image from Amazon as a base. Then we’re installing Git, cleaning the yum caches to make the container smaller, and installing required dependencies, which allow us to use Git with CodeCommit using IAM for authentication.

Next, we’re copying the index.py file to the folder where the Lambda function code should reside. Check Using AWS Lambda environment variables for additional information.

Finally, we’re specifying to execute the handler method from the index.py file at the container launch.

Lambda code

Once the Lambda container declaration is finalized, we can write a Lambda function to use it. Here’s a code example showing how to clone the Git repository. I’m sure you’ll be able to adjust this example for your personal needs:

import logging
import os
import git
 
TMP_DIR = "/tmp"
REPO_DIR = 'aws-config-rules'
REPO_URL = f'https://github.com/andreivmaksimov/{REPO_DIR}'
CLONE_PATH = os.path.join(TMP_DIR, REPO_DIR)
 
LOGGER = logging.getLogger(__name__)
LOGGER.setLevel(logging.INFO)
 
def clone(branch='master'):
   repo = git.Repo.clone_from(REPO_URL, CLONE_PATH, branch=branch)
 
   with repo.config_writer() as git_config:
       git_config.set_value('user', 'email', 'no-reply@hands-on.cloud')
       git_config.set_value('user', 'name', 'Git Lambda')
  
def handler(event, context):
   LOGGER.info('Event: %s', event)
 
   LOGGER.info('Cloning repo: %s', REPO_URL)
   clone()

In this code, we’re declaring required Python libraries, some constants, configuring logger, and a couple of functions:

• def clone(branch='master') – this function shows how to clone Git repository
• def handler(event, context) – this function is the main entry point to the Lambda function, it logs the incoming event and calling clone function

Terraform Docker Lambda code

Once we have Lambda code and declare its container, we can write Terraform code to automate the deployment. Here it is:

variable region {
 default = "us-east-1"
}
 
provider aws {
 region = var.region
}
 
data aws_caller_identity current {}
 
locals {
 prefix = "git"
 account_id          = data.aws_caller_identity.current.account_id
 ecr_repository_name = "${local.prefix}-demo-lambda-container"
 ecr_image_tag       = "latest"
}
 
resource aws_ecr_repository repo {
 name = local.ecr_repository_name
}
 
resource null_resource ecr_image {
 triggers = {
   python_file = md5(file("${path.module}/lambdas/git_client/index.py"))
   docker_file = md5(file("${path.module}/lambdas/git_client/Dockerfile"))
 }
 
 provisioner "local-exec" {
   command = <<EOF
           aws ecr get-login-password --region ${var.region} | docker login --username AWS --password-stdin ${local.account_id}.dkr.ecr.${var.region}.amazonaws.com
           cd ${path.module}/lambdas/git_client
           docker build -t ${aws_ecr_repository.repo.repository_url}:${local.ecr_image_tag} .
           docker push ${aws_ecr_repository.repo.repository_url}:${local.ecr_image_tag}
       EOF
 }
}
 
data aws_ecr_image lambda_image {
 depends_on = [
   null_resource.ecr_image
 ]
 repository_name = local.ecr_repository_name
 image_tag       = local.ecr_image_tag
}
 
resource aws_iam_role lambda {
 name = "${local.prefix}-lambda-role"
 assume_role_policy = <<EOF
{
   "Version": "2012-10-17",
   "Statement": [
       {
           "Action": "sts:AssumeRole",
           "Principal": {
               "Service": "lambda.amazonaws.com"
           },
           "Effect": "Allow"
       }
   ]
}
 EOF
}
 
data aws_iam_policy_document lambda {
   statement {
     actions = [
         "logs:CreateLogGroup",
         "logs:CreateLogStream",
         "logs:PutLogEvents"
     ]
     effect = "Allow"
     resources = [ "*" ]
     sid = "CreateCloudWatchLogs"
   }
 
   statement {
     actions = [
         "codecommit:GitPull",
         "codecommit:GitPush",
         "codecommit:GitBranch",
         "codecommit:ListBranches",
         "codecommit:CreateCommit",
         "codecommit:GetCommit",
         "codecommit:GetCommitHistory",
         "codecommit:GetDifferences",
         "codecommit:GetReferences",
         "codecommit:BatchGetCommits",
         "codecommit:GetTree",
         "codecommit:GetObjectIdentifier",
         "codecommit:GetMergeCommit"
     ]
     effect = "Allow"
     resources = [ "*" ]
     sid = "CodeCommit"
   }
}
 
resource aws_iam_policy lambda {
   name = "${local.prefix}-lambda-policy"
   path = "/"
   policy = data.aws_iam_policy_document.lambda.json
}
 
resource aws_lambda_function git {
 depends_on = [
   null_resource.ecr_image
 ]
 function_name = "${local.prefix}-lambda"
 role = aws_iam_role.lambda.arn
 timeout = 300
 image_uri = "${aws_ecr_repository.repo.repository_url}@${data.aws_ecr_image.lambda_image.id}"
 package_type = "Image"
}
 
output "lambda_name" {
 value = aws_lambda_function.git.id
}

This Terraform code was tested using Terraform version 0.14.8.

In this example, we’re using the following terraform resources:

• aws_ecr_repository – creates an ECR registry where Terraform will save Docker container image, which will be later used by our Lambda function
• null_resource – is used to build a Docker container and push it to the ECR registry, triggers checks changes in the Lambda function code and Dockerfile, and allows Terraform to understand when to rebuild the image and update the Lambda function
• aws_ecr_image – allows us to query information about published Docker image
• aws_iam_role, aws_iam_policy_document and aws_iam_policy – declares permissions (send logs to CloudWatch, CodeCommit access) for the Lambda function
• aws_lambda_function – Lambda function declaration itself

Deployment

To test solution you need to deploy Terraform code first:

terraform init
terraform apply -auto-approve

Then you need to execute the Lambda function:

aws lambda invoke --function-name git-lambda out --log-type Tail --query 'LogResult' --output text |  base64 -d

Here’s an expected output:

START RequestId: b8b742d6-5bd6-4098-90e3-5e30f5c6e816 Version: $LATEST
[INFO]  2021-03-16T02:10:28.064Z        b8b742d6-5bd6-4098-90e3-5e30f5c6e816    Event: {}
[INFO]  2021-03-16T02:10:28.064Z        b8b742d6-5bd6-4098-90e3-5e30f5c6e816    Cloning repo: https://github.com/andreivmaksimov/aws-config-rules
END RequestId: b8b742d6-5bd6-4098-90e3-5e30f5c6e816
REPORT RequestId: b8b742d6-5bd6-4098-90e3-5e30f5c6e816  Duration: 4069.15 ms    Billed Duration: 6131 ms        Memory Size: 128 MB     Max Memory Used: 83 MB  Init Duration: 2061.73 ms

Cleaning up

To clean up everything, execute the following command:

terraform destroy

Summary

This article built a Docker container for the AWS Lambda function and deployed the entire solution using Terraform. We hope you found this article useful. If so, please, help us to spread it to the world. If you have any questions, please, feel free to ask them in the chat section below.