AWS EventBridge – Building loosely coupled event-driven Serverless architectures
From this article, you’ll learn, what is Amazon EventBridge, what problems it helps to solve, and how you can use it to simplify your event-driven workflows.
Table of contents
What’s Amazon EventBridge
The very first thing we need to do is to describe EventBridge. From the official service page:
Amazon EventBridge is a serverless event bus that makes it easy to connect applications together using data from your own applications, integrated Software-as-a-Service (SaaS) applications, and AWS services.
The main purpose of Amazon EventBridge is to accept events from many different sources and route them to one or more targets. That helps you simplify event routing before doing meaningful work in response to received events.
EventBridge consists of the following entities:
- Events – represents a state changes at remote system; you’re receiving events as a JSON structure.
- Rules – matches incoming events and routes them to targets for futher processing.
- Targets – something, that should react on the events; it is usually something like a Lambda Function, SQS, SNS, EC2 instance, Fargate Task, etc.
- Event buses – those receives and routes events.
defaultone is receiving your AWS events. You may add more Event Buses to receive events from your partners.
Using Amazon EventBridge in your Serverless architecture has several very important benefits:
- It simplifies your entire architecture, as you don’t have to manage complex events routing rules.
- It simplifies your development process – most of heavy-lifting is done for you by AWS; all you need to do, is to develop you lambda function logic.
- It unifies the event processing logic – all events are delivered to your targets, no more need to do SQS pull.
Creating your own EventBridge bus
If you already built a system, which produces events, you may want to start publishing them to EventBidge. Here’s an example of doing that.
Let’s create a simple EventBus, which receives Events and logs them to CloudWatch (for debugging purposes, for example).
CloudFormation stack is to create EventBus, which uses CloudWatch LogGroup as a target, may look something like that:
AWSTemplateFormatVersion: 2010-09-09
Description: >
This stack shows how to build your own EventBridge bus
Resources:
MyCustomEventBus:
Type: AWS::Events::EventBus
Properties:
Name: !Sub '${AWS::StackName}-MyCustomEventBus'
MyCustomEventBusLogGroup:
Type: AWS::Logs::LogGroup
Properties:
LogGroupName: !Sub '/aws/events/${AWS::StackName}-mycustomevents'
RetentionInDays: 7
MyCustomEventBusEventRule:
Type: AWS::Events::Rule
Properties:
Description: 'Catch All EventRule'
State: 'ENABLED'
EventBusName: !Ref 'MyCustomEventBus'
EventPattern:
account:
- !Sub '${AWS::AccountId}'
Targets:
-
Arn: !Sub 'arn:aws:logs:${AWS::Region}:${AWS::AccountId}:log-group:${MyCustomEventBusLogGroup}'
Id: !Sub 'MyCustomEventBusLogGroup'Sending events to your own EventBridge bus
As soon as you have your custom EventBus created, we may use the boto3 library to send up to 10 events at a time from to it:
import json
from datetime import datetime
import boto3
# EventBridge client
eventbridge_client = boto3.client('events')
# Example event from your application
my_application_event_example = {
'service': 'myapp service',
'status': 'restored'
}
# Structure of EventBridge Event
eventbridge_event = {
'Time': datetime.utcnow(),
'Source': 'com.mycompany.myapp',
'Detail': json.dumps(my_application_event_example),
'DetailType': 'service_status',
'EventBusName': 'MyCustomEventBus'
}
# Send event to EventBridge
response = eventbridge_client.put_events(
Entries=[
eventbridge_event
]
)As a result, you should see something similar in your CloudWatch logs:
Subscribing to EventBridge S3 events
Now let’s take a look at a more complex example of using EventBridge. Let’s subscribe to S3 events, which are coming from CloudTrail service, and use Lambda Function as a target.
In the stack template below we’re creating two buckets:
S3bucket– bucket to demo upload operation.CloudTrailS3bucket– bucket to store CloudTrail Logs.
CloudTrail may be configured for each of you, so we’ll create a new trail to guarantee the result. We need to enable Object Level Logging (S3ObjectLevelCloudTrail) for S3 bucket first. Now we can receive EventBridge events and process them in Lambda function.
Lambda function does only logging operation of the incoming event for simplicity of an example. And we also use CloudWatch logging as a second target (which helped me to debug the stack).
AWSTemplateFormatVersion: 2010-09-09
Description: >
This stack shows how to trigger Lambda function in response
on S3 event
Resources:
CloudTrailS3bucket:
Type: AWS::S3::Bucket
S3bucket:
Type: AWS::S3::Bucket
LambdaIamRole:
Type: 'AWS::IAM::Role'
Properties:
AssumeRolePolicyDocument:
Version: 2012-10-17
Statement:
-
Effect: Allow
Principal:
Service:
- lambda.amazonaws.com
Action:
- 'sts:AssumeRole'
Path: /
Policies:
-
PolicyName: root
PolicyDocument:
Version: 2012-10-17
Statement:
-
Effect: Allow
Action:
- 's3:GetObject'
Resource:
- !Sub '${S3bucket.Arn}'
- !Sub '${S3bucket.Arn}/*'
-
Effect: Allow
Action:
- 'logs:CreateLogGroup'
- 'logs:CreateLogStream'
- 'logs:PutLogEvents'
Resource: 'arn:aws:logs:*:*:*'
LambdaPermissions:
Type: AWS::Lambda::Permission
Properties:
FunctionName: !GetAtt LambdaFunction.Arn
Action: lambda:InvokeFunction
Principal: events.amazonaws.com
SourceArn: !GetAtt EventBridgeRule.Arn
LambdaFunction:
Type: AWS::Lambda::Function
Properties:
Handler: index.handler
Role: !GetAtt LambdaIamRole.Arn
Runtime: python3.6
Timeout: 30
Code:
ZipFile: |
import logging
LOGGER = logging.getLogger(__name__)
LOGGER.setLevel(logging.INFO)
def handler(event, context):
LOGGER.info('Event: %s', event)
EventBridgeLogGroup:
Type: AWS::Logs::LogGroup
Properties:
LogGroupName: !Sub '/aws/events/${AWS::StackName}-s3-events'
RetentionInDays: 7
EventBridgeRule:
Type: AWS::Events::Rule
Properties:
Description: 'Rule, that send S3 events to Lambda function'
State: 'ENABLED'
EventPattern:
source:
- 'aws.s3'
detail-type:
- 'AWS API Call via CloudTrail'
detail:
eventSource:
- 's3.amazonaws.com'
eventName:
- 'PutObject'
requestParameters:
bucketName:
- !Sub '${S3bucket}'
Targets:
-
Arn: !Sub 'arn:aws:logs:${AWS::Region}:${AWS::AccountId}:log-group:${EventBridgeLogGroup}'
Id: !Sub 'EventBridgeLogGroup'
-
Arn: !Sub '${LambdaFunction.Arn}'
Id: 'TargetFunctionV1'
CloudTrailS3Policy:
Type: AWS::S3::BucketPolicy
Properties:
Bucket: !Sub '${CloudTrailS3bucket}'
PolicyDocument:
Version: '2012-10-17'
Statement:
-
Effect: Allow
Principal:
Service: 'cloudtrail.amazonaws.com'
Action: 's3:GetBucketAcl'
Resource: !Sub '${CloudTrailS3bucket.Arn}'
-
Effect: Allow
Principal:
Service: 'cloudtrail.amazonaws.com'
Action: 's3:PutObject'
Resource: !Sub '${CloudTrailS3bucket.Arn}/*AWSLogs/${AWS::AccountId}/*'
Condition:
StringEquals:
's3:x-amz-acl': 'bucket-owner-full-control'
S3ObjectLevelCloudTrail:
Type: 'AWS::CloudTrail::Trail'
DependsOn:
- CloudTrailS3Policy
Properties:
IsLogging: true
IsMultiRegionTrail: true
IncludeGlobalServiceEvents: true
S3BucketName: !Sub '${CloudTrailS3bucket}'
EventSelectors:
-
DataResources:
-
Type: 'AWS::S3::Object'
Values:
- !Sub '${S3bucket.Arn}/'
IncludeManagementEvents: true
ReadWriteType: AllSimplifying Step Functions long-running tasks
Not too far ago we published an article, which is showing How to manage long-running tasks using AWS Step Functions. There we created a state machine to wait for the DB snapshot operation to finish in a loop. Using EventBridge we may get rid of the long-running loop and just subscribe only one lambda function to RDS DB Snapshot Events:
{
"source": [
"aws.rds"
],
"detail-type": [
"RDS DB Snapshot Event"
]
}Here’s the CloudFormation template to do that:
---
AWSTemplateFormatVersion: 2010-09-09
Description: >
This stack shows how to react on RDS DB Snapshot events using
EventBridge
Resources:
MyCustomEventBusLogGroup:
Type: AWS::Logs::LogGroup
Properties:
LogGroupName: !Sub '/aws/events/${AWS::StackName}-mycustomevents'
RetentionInDays: 7
MyCustomEventBusEventRule:
Type: AWS::Events::Rule
Properties:
Description: 'Catch All EventRule'
State: 'ENABLED'
EventPattern:
source:
- 'aws.rds'
account:
- !Sub '${AWS::AccountId}'
detail-type:
- 'RDS DB Snapshot Event'
Targets:
-
Arn: !Sub 'arn:aws:logs:${AWS::Region}:${AWS::AccountId}:log-group:${MyCustomEventBusLogGroup}'
Id: !Sub 'MyCustomEventBusLogGroup'Of cause, you can use Lambda Function as a target to do something meaningful with this information. For example, you may copy the DB snapshot to another region for DR purposes.
AWS Serverless Workshops
If you’d like to get more hands-on experience using EventBridge and serverless technologies, I’d recommend you to take a look at those AWS Workshops:
- CDK Workshop.
- Building Event-Driven Architectures on AWS.
- Serverless Web Application Workshop.
- CI/CD For Serverless Applications.
- Serverless Image Processing on AWS.
- Serverless Security Workshop.
You may find a complete list of serverless workshops at the official AWS Samples GitHub repository.
Summary
In this article, we’ve covered how to use EventBridge in several ways to react on AWS and non-AWS events. I hope, this article will save you some time and help to incorporate EventBridge into your own architecture.
If you found this article useful, please, help to spread it to the world.
