This article will describe what the Lambda function is, what its features and limits are. We’ll develop a simple Lambda function and deploy it using CloudFormation. Finally, we’ll discuss the Lambda developer workflow and platform-independent build of Lambda functions and Layers using Docker.

What is Lambda Function

AWS Lambda Functions is a service, which provides a short execution time compute service for running your code in response to any kind of event.

Lambda function use cases

For example, here are some perfect use-cases for Lambda Functions:

• Trigger other AWS service in response on S3 file upload.
• Trigger other AWS service in response on API Gateway incomming request.
• Do third-party service integrations.
• Design and execute complex workflows with help of Step Functions.
• Log analysis on the fly.
• Automated backups and everyday tasks.
• Filtering and transforming data on the fly.
• Many-many others.

For additional use cases take a look at our [The Ultimate Guide] – AWS Lambda Real-world use-cases article.

At the same time, Lambda functions have their own limitations. The most important of them are:

• Execution timeout – 15 mins.
• Maximum memory – 3008 MB.
• Deployment package size – up to 50 MB (zipped) and 250 MB (unzipped).

But as soon as you decide to jump to the serverless world and start using Lambda Functions, the first thing you need to know is how to write and deploy it.

Of cause, there’re some already existing interesting frameworks available for you at the moment, which can simplify that and many other tasks:

• AWS Serverless Application Model (SAM)
• AWS Amplify Framework
• Serverless

But in this article, I’ll show how to do it using basic CloudFormation stacks.

For more information about AWS Lambda consider checking out the Python for AWS Lambda Functions: A Beginner’s Guide article.

Simple Lambda Function

I like it very much when I need to write simple and short Lamda Functions because that allows to embed them directly to CloudFormation stack templates like that:

AWSTemplateFormatVersion: 2010-09-09
Description: >
  This CloudFormation template creates simple Lambda functions,
  which prints CloudFormation resource Arn from the stack.  

Resources:
  LambdaFunctionRole:
    Type: AWS::IAM::Role
    Properties:
      AssumeRolePolicyDocument:
        Version: '2012-10-17'
        Statement:
        - Effect: Allow
          Principal:
            Service:
              - lambda.amazonaws.com
          Action:
            - sts:AssumeRole
      Path: "/"
      Policies:
      - PolicyName: LambdaFunctionPolicy
        PolicyDocument:
          Version: '2012-10-17'
          Statement:
          - Effect: Allow
            Action:
              - logs:CreateLogGroup
              - logs:CreateLogStream
              - logs:PutLogEvents
            Resource: '*'

  LambdaFunction:
    Type: AWS::Lambda::Function
    Properties:
      Runtime: python3.6
      Timeout: 5
      Handler: index.handler
      Role: !GetAtt LambdaFunctionRole.Arn
      Code:
        ZipFile:
          !Sub
            - |-
              #!/usr/bin/env python3

              import cfnresponse
              import logging
              import traceback

              LOGGER = logging.getLogger()
              LOGGER.setLevel(logging.INFO)

              def handler(event, context):
                try:
                  LOGGER.info('Event structure: %s', event)

                  LOGGER.info('Our Lambda function role Arn ${lambda_function_role_arn}')

                except Exception as e:
                  LOGGER.error(e)
                  traceback.print_exc()
                finally:
                  cfnresponse.send(event, context, cfnresponse.SUCCESS, {})              
            -
              lambda_function_role_arn: !Ref LambdaFunctionRole

Here LambdaFunctionRole is a very basic boilerplate IAM Role for Lambda Function execution, which grants our Lambda Function permissions to sent its stdout and stderr logs to CloudWatch.

LambdaFunction is a basic boilerplate Lambda Function written in Python, which is logging an incoming event structure and CloudFormation stack resource ARN. It is very useful to set up the Lambda function in this way when you do not understand what data structure is to expect from its input.

To deploy CloudFormation stack from such a simple template you may use awscli:

aws cloudformation create-stack \
    --stack-name 'my-simple-lambda-function-stack' \
    --template-body file://$(pwd)/my_cloudformation_template.yaml

aws cloudformation wait \
    stack-create-complete \
    --stack-name 'my-simple-lambda-function-stack'

Complex Lambda Functions

Because of Lambda Functions and CloudFormation AWS::Lambda::Function resource limitations you can not use the same approach to build Lambda Function with external libraries, binaries and deploy them directly using only the CloudFormation template.

In general, the whole process of working with such Lambda Function may be split into several phases:

• Development.
• Building Lambda Function .zip archive.
• Uploading Lambda Function .zip archive to S3 bucket.
• Deployment of Lambda function from S3 bucket.

Development

There’s currently no easy to develop Lambda function locally without using additional frameworks. I can recommend using AWS SAM if you have a need to build and test your functions locally.

Building Lambda Function zip archive

Most of the cases for Python are covered by AWS Lambda Deployment Package in Python official documentation.

What I can extend here, for now, is that sometimes you may use platform-dependent Python libraries and spend many-many hours figuring out, why everything is working locally, but not working as soon as you deploy your Lambda Function. To avoid such problems I’d suggest using Docker to build and pack your Lambda Functions and their dependencies.

For example, here’s the Makefile for building Lambda Functions depending on cx_Oracle library.

SHELL=/bin/bash
.PHONY: init package clean

# instantclient 19.5
INSTANT_CLIENT_ZIP_DIR="./oracle_libs"
# extracts into this dir
INSTANT_CLIENT_DIR=instantclient_19_5

LAMBDA_ZIP=reset-schema-password.zip

# taking specific url of .whl for linux from https://pypi.org/project/cx-Oracle/#files &&
init:
  docker run --rm -v `pwd`:/src -w /src python /bin/bash -c "mkdir -p ./lib && \
    apt-get update && \
    apt-get install -y libaio1 && \
    cp /usr/lib/x86_64-linux-gnu/libaio.so.1 ./lib/ && \
    unzip $(INSTANT_CLIENT_ZIP_DIR)/instantclient-basiclite-linux.x64-19.5.0.0.0dbru.zip -d /tmp && \
    mv /tmp/$(INSTANT_CLIENT_DIR)/* ./lib && \
    pip install https://files.pythonhosted.org/packages/d5/15/d38862a4bd0e18d8ef2a3c98f39e743b8951ec5efd8bc63e75db04b9bc31/cx_Oracle-7.3.0-cp36-cp36m-manylinux1_x86_64.whl -t ."

lambda.zip:
  docker run --rm -v `pwd`:/src -w /src python /bin/bash -c "apt-get update && \
    apt-get install -y zip && \
    zip --symlinks -r $(LAMBDA_ZIP) index.py lib cx*"

package: lambda.zip

install: lambda.zip
  aws s3 cp $(LAMBDA_ZIP) s3://$(SERVICE_S3_BUCKET)/lambdas/$(LAMBDA_ZIP)
  echo "Use CFN stack to deploy Lambda from s3://$(SERVICE_S3_BUCKET)/lambdas/$(LAMBDA_ZIP)"

clean:
  rm -rf ./lib;
  rm -rf ./$(LAMBDA_ZIP);
  rm -rf ./cx_Oracle*;

For more information about managing Amazon S3 using AWS CLI, check out our How to use AWS CLI to manage Amazon S3 article.

This will give you an idea, how to use Docker to script build and package processes to avoid platform dependency problems. Linux-based Docker container with Python and Bash allows you to automate the building of the Lambda Function.

To build and package your Lambda function you’ll use the following command.

make && make package

Uploading Lambda Function .zip archive to S3 bucket

As shown above, you may use old-school Makefile and awscli to automate the upload of your Lambda Function .zip archive with its source code to some kind of “service” S3 bucket.

To deploy your Lambda function you’ll use the following command:

make install

Deployment of Lambda function from S3 bucket

As soon as your Lambda Function .zip archive is uploaded to the S3 bucket, you may declare it in your CloudFormation stack template like that:

LambdaFunction:
  Type: AWS::Lambda::Function
  Properties:
    Runtime: python3.6
    Timeout: 5
    Handler: index.handler
    Role: !GetAtt LambdaFunctionRole.Arn
    Code:
        S3Bucket: 'my-service-s3-bucket-with-lambda-sources'
        S3Key: 'lambdas/reset-schema-password.zip'

Summary

In this article, we covered what the Lambda function is, what its features and limits are. Additionally, we developed a simple Lambda function and deploy it using CloudFormation. And finally, we discussed the Lambda developer workflow and platform-independent build of Lambda functions and Layers using Docker.

We hope, you found this article useful! If so, please, help us to spread it to the world.

Stay tuned!

Related articles

• Quick Intro to Python for AWS Automation Engineers
• How to run Jupiter, Keras, Tensorflow, and other ML libs in Docker
• Working with S3 in Python using Boto3
• Using Terraform to deploy S3->SQS->Lambda integration
• Terraform – Deploy Python Lambda (container image)
• AWS Step Functions – How to manage long-running tasks