How to create and deploy your first Python 3 AWS Lambda Function

This article will describe what Lambda function is, what its features and limits are. We’ll develop a simple Lambda function and deploy it using CloudFormation. Finally, we’ll discuss the Lambda developer workflow and platform-independent build of Lambda functions and Layers using Docker.

What is Lambda Function

AWS Lambda Functions is a service, which provides short execution time compute service for running your code in response to any kind of events.

Lambda function use cases

For example, here’s some perfect use-cases for Lambda Functions:

• Trigger other AWS service in response on S3 file upload.
• Trigger other AWS service in response on API Gateway incomming request.
• Do third-party service integrations.
• Design and execute complex workflows with help of Step Functions.
• Log analysis on the fly.
• Automated backups and everyday tasks.
• Filtering and transforming data on the fly.
• Many-many others.

For additional use cases take a look at our [The Ultimate Guide] – AWS Lambda Real-world use-cases article.

At the same time Lambda functions has their own limitations. Most important of them are:

• Execution timeout – 15 mins.
• Maximum memory – 3008 MB.
• Deployment package size – up to 50 MB (zipped) and 250 MB (unzipped).

But as soon as you decide to jump to serverless world and start using Lambda Functions, the first thing you need to know, is how to write and deploy it.

Of cause, there’re some already existing interesting frameworks available for you at the moment, which can simplify that and many other tasks:

• AWS Serverless Application Model (SAM)
• AWS Amplify Framework
• Serverless

But at this article I’ll show how to do it using basic CloudFormation stacks.

Simple Lambda Function

I like it very much, when I need to write a simple and short Lamda Functions, because that allows to embed them directly to CloudFormation stack templates like that:

AWSTemplateFormatVersion: 2010-09-09
Description: >
  This CloudFormation template creates simple Lambda functions,
  which prints CloudFormation resource Arn from the stack.  

Resources:
  LambdaFunctionRole:
    Type: AWS::IAM::Role
    Properties:
      AssumeRolePolicyDocument:
        Version: '2012-10-17'
        Statement:
        - Effect: Allow
          Principal:
            Service:
              - lambda.amazonaws.com
          Action:
            - sts:AssumeRole
      Path: "/"
      Policies:
      - PolicyName: LambdaFunctionPolicy
        PolicyDocument:
          Version: '2012-10-17'
          Statement:
          - Effect: Allow
            Action:
              - logs:CreateLogGroup
              - logs:CreateLogStream
              - logs:PutLogEvents
            Resource: '*'

  LambdaFunction:
    Type: AWS::Lambda::Function
    Properties:
      Runtime: python3.6
      Timeout: 5
      Handler: index.handler
      Role: !GetAtt LambdaFunctionRole.Arn
      Code:
        ZipFile:
          !Sub
            - |-
              #!/usr/bin/env python3

              import cfnresponse
              import logging
              import traceback

              LOGGER = logging.getLogger()
              LOGGER.setLevel(logging.INFO)

              def handler(event, context):
                try:
                  LOGGER.info('Event structure: %s', event)

                  LOGGER.info('Our Lambda function role Arn ${lambda_function_role_arn}')

                except Exception as e:
                  LOGGER.error(e)
                  traceback.print_exc()
                finally:
                  cfnresponse.send(event, context, cfnresponse.SUCCESS, {})              
            -
              lambda_function_role_arn: !Ref LambdaFunctionRole

Here LambdaFunctionRole is very basic boilerplate IAM Role for Lambda Function execution, which grants our Lambda Function permissions to sent its stdout and stderr logs to CloudWatch.

LambdaFunction is a basic boilerplate Lambda Function written in Python, which is logging an incoming event structure and CloudFormation stack resource arn. It is very useful to setup Lambda function in this way when you do not understand what datastructure is to expect to its input.

To deploy CloudFormation stack from such simple template you may use awscli:

aws cloudformation create-stack \
    --stack-name 'my-simple-lambda-function-stack' \
    --template-body file://$(pwd)/my_cloudformation_template.yaml

aws cloudformation wait \
    stack-create-complete \
    --stack-name 'my-simple-lambda-function-stack'

Complex Lambda Functions

Because of Lambda Functions and CloudFormation AWS::Lambda::Function resource limitations you can not use the same approach to build Lambda Function with external libraries, binaries and deploy them directly using only CloudFormation template.

In general whole process of working with such Lambda Function may be splitted to several phases:

• Development.
• Building Lambda Function .zip archive.
• Uploading Lambda Function .zip archive to S3 bucket.
• Deployment of Lambda function from S3 bucket.

Development

There’s currently no easy to develop Lambda function locally without using additional frameworks. I can recommend to use AWS SAM, if you have a need to build and test your functions locally.

Building Lambda Function zip archive

Most of the cases for Python are covered by AWS Lambda Deployment Package in Python official documentation.

What I can extend here for now, is that sometimes you may use platform dependent Python libraries and spend many-many hours figuring out, why everything is working locally, but not working as soon as you deploy your Lambda Function. To avoid such problems I’d suggest to use Docker to build and pack you Lambda Functions and their dependencies.

For example, here’s the Makefile for building Lambda Functions depending on cx_Oracle library.

SHELL=/bin/bash
.PHONY: init package clean

# instantclient 19.5
INSTANT_CLIENT_ZIP_DIR="./oracle_libs"
# extracts into this dir
INSTANT_CLIENT_DIR=instantclient_19_5

LAMBDA_ZIP=reset-schema-password.zip

# taking specific url of .whl for linux from https://pypi.org/project/cx-Oracle/#files &&
init:
  docker run --rm -v `pwd`:/src -w /src python /bin/bash -c "mkdir -p ./lib && \
    apt-get update && \
    apt-get install -y libaio1 && \
    cp /usr/lib/x86_64-linux-gnu/libaio.so.1 ./lib/ && \
    unzip $(INSTANT_CLIENT_ZIP_DIR)/instantclient-basiclite-linux.x64-19.5.0.0.0dbru.zip -d /tmp && \
    mv /tmp/$(INSTANT_CLIENT_DIR)/* ./lib && \
    pip install https://files.pythonhosted.org/packages/d5/15/d38862a4bd0e18d8ef2a3c98f39e743b8951ec5efd8bc63e75db04b9bc31/cx_Oracle-7.3.0-cp36-cp36m-manylinux1_x86_64.whl -t ."

lambda.zip:
  docker run --rm -v `pwd`:/src -w /src python /bin/bash -c "apt-get update && \
    apt-get install -y zip && \
    zip --symlinks -r $(LAMBDA_ZIP) index.py lib cx*"

package: lambda.zip

install: lambda.zip
  aws s3 cp $(LAMBDA_ZIP) s3://$(SERVICE_S3_BUCKET)/lambdas/$(LAMBDA_ZIP)
  echo "Use CFN stack to deploy Lambda from s3://$(SERVICE_S3_BUCKET)/lambdas/$(LAMBDA_ZIP)"

clean:
  rm -rf ./lib;
  rm -rf ./$(LAMBDA_ZIP);
  rm -rf ./cx_Oracle*;

This will give you an idea, how to use Docker to script build and package process to avoid platform dependency problem. Linux based Docker container with Python and Bash allows you to automate the building of the Lambda Function.

To build and package your Lambda function you’ll use the following command.

make && make package

Uploading Lambda Function .zip archive to S3 bucket

As it shown above, you may use old-school Makefile and awscli to automate upload of your Lambda Function .zip archive with its source code to some kind of “service” S3 bucket.

To deploy your Lambda function you’ll use the following command:

make install

Deployment of Lambda function from S3 bucket

As soon as your Lambda Function .zip archive is uploaded to S3 bucket, you may declare it in your CloudFormation stack temlate like that:

LambdaFunction:
  Type: AWS::Lambda::Function
  Properties:
    Runtime: python3.6
    Timeout: 5
    Handler: index.handler
    Role: !GetAtt LambdaFunctionRole.Arn
    Code:
        S3Bucket: 'my-service-s3-bucket-with-lambda-sources'
        S3Key: 'lambdas/reset-schema-password.zip'

Summary

In this article, we covered what Lambda function is, what its features and limits are. Additionally, we developed a simple Lambda function and deploy it using CloudFormation. And finally, we discussed the Lambda developer workflow and platform-independent build of Lambda functions and Layers using Docker.

We hope, you found this article useful! If so, please, help us to spread it to the world.

Stay tuned!

Related articles

• What is Serverless computing
• Serverless Framework – Run your Kubernetes Workloads on Amazon EC2 Spot Instances with Amazon EKS – Part 2
• [The Ultimate Guide] – AWS Lambda Real-world use-cases
• Building Thumbnails And GIFs Generator Using Lambda And FFmpeg
• Terraform – Deploy Lambda To Copy Files Between S3 Buckets