AWS CloudFormation Lambda – Simple Tutorial

ByAndrei Maksimov

Deploying AWS Lambda using CloudFormation offers several advantages. It allows you to manage your Lambda functions and their associated resources as code, which makes it easier to version control and maintain consistency across multiple environments. Additionally, it allows you to define your infrastructure and the runtime environment for your functions in a single place, helping reduce the complexity and manual effort needed for managing these services. Finally, it provides a way to automatically roll out updates and changes quickly across your entire deployment without manual intervention.

This AWS CloudFormation Lambda tutorial demonstrates how to define and deploy simple and complex Lambda functions using CloudFormation.

Table of contents

What is AWS Lambda Function

AWS Lambda

AWS Lambda is a compute service offered by Amazon Web Services (AWS) that allows users to execute specific code functions without needing server provisioning or management. It’s an event-driven and serverless platform that runs backend code in response to events, such as inputs from other AWS services like S3 or DynamoDB or HTTP requests made through API Gateway. The Lambda code executes quickly and efficiently, as it is written to fit the user’s needs with no overhead or wasted resources. With AWS Lambda functions, users can build reliable applications, including backends for web and mobile applications, fast, with minimal effort and cost. In addition to computing power, AWS Lambda provides monitoring and logging capabilities so users can easily track their usage and cost while still affording themselves time to focus on developing new features instead of managing servers.

Lambda function use cases

For example, here are some perfect use cases for AWS Lambda:

  • Trigger AWS services in response to S3 file upload.
  • Trigger AWS services in response to API Gateway incoming requests.
  • Do third-party API service integrations.
  • Design and execute complex workflows with the help of Step Functions.
  • Log analysis on the fly.
  • Automated backups and everyday tasks.
  • Filtering and transforming data on the fly.
  • Many-many others.

Look at our [The Ultimate Guide] – AWS Lambda Real-world use-cases article for additional use cases.

At the same time, AWS Lambda has its limitations. The most important of them are:

  • Execution timeout – 15 mins.
  • Maximum memory – 3008 MB.
  • Deployment package size – up to 50 MB (zipped) and 250 MB (unzipped).

But as soon as you decide to jump to the serverless world and start using AWS Lambda, you need to know how to write and deploy it.

Of course, there’re some already existing interesting frameworks available for you at the moment, which can simplify that and many other tasks:

But in this article, I’ll show how to do it using CloudFormation stacks.

AWS CloudFormation Lambda – Simple example

AWS CloudFormation Lambda Tutorial - Simple example

I like it very much when I need to write simple and short Lamda Functions because that allows me to embed them directly into CloudFormation stack templates like that:

AWSTemplateFormatVersion: 2010-09-09
Description: >
  This CloudFormation template creates simple Lambda function,
  which prints CloudFormation resource Arn from the stack.  
Resources:
  LambdaFunctionRole:
    Type: AWS::IAM::Role
    Properties:
      AssumeRolePolicyDocument:
        Version: '2012-10-17'
        Statement:
        - Effect: Allow
          Principal:
            Service:
              - lambda.amazonaws.com
          Action:
            - sts:AssumeRole
      Path: "/"
      Policies:
      - PolicyName: LambdaFunctionPolicy
        PolicyDocument:
          Version: '2012-10-17'
          Statement:
          - Effect: Allow
            Action:
              - logs:CreateLogGroup
              - logs:CreateLogStream
              - logs:PutLogEvents
            Resource: '*'
  LambdaFunctionLogGroup:
    Type: AWS::Logs::LogGroup
    DependsOn: LambdaFunction
    Properties:
      RetentionInDays: 1
      LogGroupName: !Join
        - ""
        - -  "/aws/lambda/"
          - !Ref LambdaFunction
  LambdaFunction:
    Type: AWS::Lambda::Function
    Properties:
      Runtime: python3.9
      Timeout: 5
      Handler: index.handler
      Role: !GetAtt LambdaFunctionRole.Arn
      Code:
        ZipFile:
          !Sub
            - |-
              #!/usr/bin/env python3
              import logging
              import traceback
              LOGGER = logging.getLogger()
              LOGGER.setLevel(logging.INFO)
              def handler(event, context):
                try:
                  LOGGER.info('Event structure: %s', event)
                  response = {
                    'event': event
                  }
                  return {
                    'statusCode': 200,
                    'response': response
                  }
                except Exception as e:
                  LOGGER.error(e)
                  traceback.print_exc()            
            -
              lambda_function_role_arn: !Ref LambdaFunctionRole
Outputs:
  LambdaFunctionName:
    Value: !Ref LambdaFunction

The above CloudFormation stack deploys several AWS resources:

  • The AWS::Lambda::Function resource contains function properties such as the function runtime, execution timeout, function entry point (handler), function execution role, code location (inline code in our example), etc. Check out the official AWS::Lambda::Function resource documentation for more information about available function properties.
  • The AWS::Logs::LogGroup resource deploys a CloudWatch log group with a retention period of 1 day. AWS Lambda sends its logs to that Log Group.
  • The LambdaFunctionRole is a basic customer-managed IAM Role for Lambda Function execution, which grants our Lambda Function permission to send Lambda function logs to CloudWatch.

For more information about interacting with AWS services using the Boto3 library, check out our Using Python and Boto3 for AWS Automation – Free course.

Note: the Handler specifies the method that AWS Lambda calls every time it executes your function.

Note: the Code property contains the inline Python code of our function.

Note: such an approach allows you to deploy simple Lambda functions. If your Lambda function is complex, we recommend deploying the Lambda function using Terraform and unit testing its implementation beforehand.

Deploy stack

To deploy the CloudFormation stack from such a simple template, you may use AWS CLI tools:

aws cloudformation create-stack \
    --stack-name 'my-simple-lambda-function-stack' \
    --capabilities CAPABILITY_IAM \
    --template-body file://$(pwd)/simple-lambda.yaml
aws cloudformation wait \
    stack-create-complete \
    --stack-name 'my-simple-lambda-function-stack'

The aws cloudformation create-stack command will deploy a new Lambda function using the CloudFormation stack template file.

Invoke Lambda

Let’s test our function by invoking the function manually in your AWS account:

FUNCTION_NAME=$(aws cloudformation describe-stacks \
  --stack-name 'my-simple-lambda-function-stack' \
  --query "Stacks[0].Outputs[?OutputKey=='LambdaFunctionName'].OutputValue" \
  --output text)
aws lambda invoke \
  --function-name $FUNCTION_NAME \
  $(tty) >/dev/null

Also, you can use the AWS management console to invoke your function manually.

Update stack

To update the CloudFormation template deployment or function code, execute the following AWS CLI commands:

aws cloudformation update-stack \
    --stack-name 'my-simple-lambda-function-stack' \
    --capabilities CAPABILITY_IAM \
    --template-body file://$(pwd)/simple-lambda.yaml
aws cloudformation wait \
    stack-update-complete \
    --stack-name 'my-simple-lambda-function-stack'

Delete stack

To delete the CloudFormation stack, execute the following AWS CLI command:

aws cloudformation delete-stack \
    --stack-name 'my-simple-lambda-function-stack' \
    --capabilities CAPABILITY_IAM

AWS CloudFormation Lambda – Complex Example

AWS CloudFormation Lambda Tutorial - Complex example

AWS Lambda and CloudFormation AWS::Lambda::Function have limitations not allowing you to build Lambda Function with external libraries and binaries and deploy them directly using the CloudFormation template only.

So, the development workflow for complex Lambda Functions may include the following phases:

  • Development and testing
  • Archiving AWS Lambda code
  • Uploading function code .zip archive to S3 bucket
  • Deployment of Lambda function from S3 bucket

Development

There’s currently no easy-to-develop Lambda function locally without using additional frameworks. I recommend using AWS SAM or LocalStack to build and test your functions locally.

Archiving AWS Lambda code

All steps for archiving the AWS Lambda function for Python programming language are covered by AWS Lambda Deployment Package in Python official documentation. I suggest using the make utility to automate all steps end-to-end for Linux and OS X operating systems. Here’s a great make file tutorial for getting started.

Note: sometimes, you may use platform-dependent Python libraries and spend many hours figuring out why everything is working locally but not as soon as you deploy your Lambda Function. To avoid such problems, I suggest using Docker to build your Lambda Function and its dependencies and put them in the zip file.

For example, here’s the Makefile for building AWS Lambda Function depending on the platform-dependent cx_Oracle library:

SHELL=/bin/bash
.PHONY: init package clean
# instantclient 19.5
INSTANT_CLIENT_ZIP_DIR="./oracle_libs"
# extracts into this dir
INSTANT_CLIENT_DIR=instantclient_19_5
LAMBDA_ZIP=reset-schema-password.zip
# taking specific url of .whl for linux from https://pypi.org/project/cx-Oracle/#files &&
init:
  docker run --rm -v `pwd`:/src -w /src python /bin/bash -c "mkdir -p ./lib && \
    apt-get update && \
    apt-get install -y libaio1 && \
    cp /usr/lib/x86_64-linux-gnu/libaio.so.1 ./lib/ && \
    unzip $(INSTANT_CLIENT_ZIP_DIR)/instantclient-basiclite-linux.x64-19.5.0.0.0dbru.zip -d /tmp && \
    mv /tmp/$(INSTANT_CLIENT_DIR)/* ./lib && \
    pip install https://files.pythonhosted.org/packages/d5/15/d38862a4bd0e18d8ef2a3c98f39e743b8951ec5efd8bc63e75db04b9bc31/cx_Oracle-7.3.0-cp36-cp36m-manylinux1_x86_64.whl -t ."
lambda.zip:
  docker run --rm -v `pwd`:/src -w /src python /bin/bash -c "apt-get update && \
    apt-get install -y zip && \
    zip --symlinks -r $(LAMBDA_ZIP) index.py lib cx*"
package: lambda.zip
install: lambda.zip
  aws s3 cp $(LAMBDA_ZIP) s3://$(SERVICE_S3_BUCKET)/lambdas/$(LAMBDA_ZIP)
  echo "Use CFN stack to deploy Lambda from s3://$(SERVICE_S3_BUCKET)/lambdas/$(LAMBDA_ZIP)"
clean:
  rm -rf ./lib;
  rm -rf ./$(LAMBDA_ZIP);
  rm -rf ./cx_Oracle*;

For more information about managing Amazon S3 using AWS CLI, check out our How to use AWS CLI to manage Amazon S3 article.

This will give you an idea of using Docker to script, build and package processes to avoid platform dependency problems. Linux-based Docker container with Python and Bash allows you to automate the building of the AWS Lambda Function.

You’ll use the following command to build and package your AWS Lambda function:

make && make package

Uploading .zip file to the S3 bucket

As shown above, you may use old-school Makefile and awscli to automate your function .zip file upload with its source code to the deployment S3 bucket.

To deploy your function, you’ll use the following command:

make install

Deployment from the S3 bucket

As soon as your Lambda Function .zip file is uploaded to the S3 bucket, you may change your AWS CloudFormation template file:

LambdaFunction:
  Type: AWS::Lambda::Function
  Properties:
    Runtime: python3.6
    Timeout: 5
    Handler: index.handler
    Role: !GetAtt LambdaFunctionRole.Arn
    Code:
        # Deploy Lambda zip file from S3 bucket
        S3Bucket: 'my-service-s3-bucket-with-lambda-sources'
        S3Key: 'lambdas/reset-schema-password.zip'

Summary

In this article, we covered what the Lambda function is and what its features and limits are. Additionally, we developed a simple Lambda function and deployed it using CloudFormation. And finally, we discussed the Lambda developer workflow and platform-independent build of Lambda functions and Layers using Docker.

We hope you found this article useful! If so, please, help us to spread it to the world.

Stay tuned!

Andrei Maksimov

I’m a passionate Cloud Infrastructure Architect with more than 20 years of experience in IT. In addition to the tech, I'm covering Personal Finance topics at https://amaksimov.com.

Any of my posts represent my personal experience and opinion about the topic.