Amazon DynamoDB is a flexible, scalable, managed NoSQL database service offered by Amazon Web Services. It supports document and key-value store models and built-in support for popular data types such as JSON, XML, and HTML. DynamoDB’s key features include automatic scalability, high performance, and low latency. DynamoDB is fully managed by Amazon, making it a good choice for mission-critical applications that require high availability and durability while still being cost-effective. DynamoDB is a good choice for applications with high throughput and low latency, such as mobile gaming, real-time bidding, live chat, and social media applications. DynamoDB scales automatically to accommodate changing throughput requirements.
One of the main benefits of using DynamoDB is that it automatically replicates data across multiple Availability Zones (AZs) in an AWS Region, providing enhanced protection against outages and data loss. DynamoDB offers comprehensive security features, including fine-grained access control, encryption at rest, and user activity logging, and integrates with other AWS services, such as Amazon S3 and Amazon EMR, to give you even more flexibility and power.
This Boto3 DynamoDB tutorial covers how to create tables, load all the data, perform CRUD operations, and query tables in AWS DynamoDB using Python and Boto3 library.
Table of contents
Prerequisites
To start working with DynamoDB using Python, you must set up your Python environment.
In summary, this is what you will need:
- Python 3
- Boto3
- AWS CLI tools
Alternatively, you can quickly set up and use Cloud9 IDE.
Introduction to Amazon DynamoDB
Amazon DynamoDB enables the building of flexible microservices, serverless web applications, and mobile backends, and it acts as a highly scalable and fully managed database for applications.
The most interesting topic of DynamoDB is how to design your tables to get maximum performance and efficiency from the DynamoDB service. Regardless of your current experience, I highly recommend you a couple of videos on that topic before getting started:
Now, as soon as the concepts of designing DynamoDB tables are covered, you may look at how to start working with this service in Python using the Boto3 library.
Pro tip: Whenever you are doing a search query for something related to Amazon DynamoDB in Google, you can use “ddb” keyword instead of “dynamodb” in a search query, for example: “boto3 ddb”. Google is smart enough to understand you. Check it out!
Connecting to DynamoDB APIs using Boto3
The first thing you need to define in your Python script or Lambda function is to tell Boto3 how to connect to DynamoDB. To connect to Amazon DynamoDB API, you need to use the client and resource boto3 library methods.
Here’s an example of using the Boto DynamoDB client() method:
import boto3
dynamodb_client = boto3.client('dynamodb')Here’s an example of using the Boto DynamoDB resource() method:
import boto3
dynamodb_resource = boto3.resource('dynamodb')Right after that, you can use the dynamodb_client variable to get access to the Boto3 DynamoDB client methods, and dynamodb_resource variable to get access to the Boto3 DynamoDB resource methods and collections.
The boto3.resource('dynamodb') resource allows developers to create, update, and delete DynamoDB tables and all the items. This resource supports table-level security via ACLs and item-level security via Condition Expressions. The boto3.resource('dynamodb') resource can be used in conjunction with AWS Lambda functions to provide serverless access to DynamoDB data.
Create Tables in DynamoDB using Boto3
First, import the boto3 module and then create a Boto3 DynamoDB resource.
Next, create a table named Employees with a primary key that has the following attributes;
Name a partition key (also known as the “hash key”) with AttributeType set to S for string.
Email a sort key (also known as the “range key”) with AttributeType set to S for string.
ProvisionedThroughput means the maximum number of consistent reads and writes per second on your table.
import boto3
dynamodb = boto3.resource('dynamodb')
table = dynamodb.create_table (
TableName = 'Employees',
KeySchema = [
{
'AttributeName': 'Name',
'KeyType': 'HASH'
},
{
'AttributeName': 'Email',
'KeyType': 'RANGE'
}
],
AttributeDefinitions = [
{
'AttributeName': 'Name',
'AttributeType': 'S'
},
{
'AttributeName':'Email',
'AttributeType': 'S'
}
],
ProvisionedThroughput={
'ReadCapacityUnits':1,
'WriteCapacityUnits':1
}
)
print(table)Here is the execution output:
Alternatively, you can create the DynamoDB table using AWS CLI: How to use AWS CLI to manage Amazon DynamoDB.
List Tables using Boto3 DynamoDB
To list all tables in DynamoDB, use the dynamodb.tables.all() method.
import boto3
dynamodb = boto3.resource('dynamodb')
print(list(dynamodb.tables.all()))Here is the execution output.
CRUD Operations in DynamoDB using Boto3
Tables, items, and attributes are the core components that makeup DynamoDB. A table is a collection of items, and each item is a collection of attributes.
The following section will look at the four basic ways to perform CRUD operations on Items in DynamoDB Table.
Create Items
To add a new item data to a table, use the put_item() method and add an item with a primary Name and Email, as shown in the example.
import boto3
dynamodb = boto3.resource('dynamodb')
table = dynamodb.Table('Employees')
response = table.put_item(
Item = {
'Name': 'Kelvin Galabuzi',
'Email': 'kelvingalabuzi@handson.cloud'
}
)
print(response)Here is the Execution output:
Batch Write Items
The batch_writer() method in Boto3 implements the BatchWriteItem AWS API call, which allows you to write multiple items to an Amazon DynamoDB table in a single request. This can be useful when you want to perform many write operations in a single request or to write items spread across multiple partitions. The BatchWriteItem API call supports both put and delete operations. Each request can write up to 100 items, and each item can be up to 400 KB in size. If you need to write more than 100 DynamoDB items, or if your items are larger than 400 KB, you can use the boto3 BatchWriteItem API call instead. BatchWriteItem API call has the following limitations:
- You can only write items to a single table in each request.
- You cannot use filters when writing items (e.g., you cannot specify a condition expression).
- All items in the request must belong to the same partition primary key value.
- You cannot update an existing item; you can only insert or delete new items. If you need to update an item, you must delete it and then insert a new item with the updated data.
For more information about the BatchWriteItem API call, see the Amazon DynamoDB Developer Guide and AWS Resources.
NB: You can also add multiple attributes to items as required.
import boto3
dynamodb = boto3.resource('dynamodb')
table = dynamodb.Table('Employees')
with table.batch_writer() as batch:
batch.put_item(Item={"Name": "Luzze John", "Email": "john@handson.cloud",
"Department": "IT", "Section": { "QA": "QA-1", "Reporting Line": "L1" } })
batch.put_item(Item={"Name": "Lugugo Joshua", "Email": "joshua@handson.cloud",
"Department": "IT", "Section": { "Development": "SD-1", "Reporting Line": "L1" } })
batch.put_item(Item={"Name": "Robert Nsamba", "Email": "robert@handson.cloud",
"Department": "IT", "Section": { "PM": "PM-1", "Reporting Line": "L1" } })
print(batch)Here is the execution output:
Read Items
To read a single item from the DynamoDB table, use the get_item() method with the key-value pair of items in your table.
import boto3
dynamodb = boto3.resource('dynamodb')
table = dynamodb.Table('Employees')
# boto3 dynamodb getitem
response = table.get_item(
Key={
'Name': 'Kelvin Galabuzi',
'Email': 'kelvingalabuzi@handson.cloud'
}
)
print(response['Item'])Here is the Execution output:
To all read Items in the table, you can use the scan() method to return all items in the table.
import boto3
dynamodb = boto3.resource('dynamodb')
table = dynamodb.Table('Employees')
response = table.scan()
response['Items']
print(response)Here is the Execution output:
Update Items
To update items, you can use the update_item() method to edit an existing item’s attributes. You can also use the same method to add or remove new data attributes.
import boto3
dynamodb = boto3.resource('dynamodb')
table = dynamodb.Table('Employees')
response = table.update_item(
Key={'Name': 'Luzze John', 'Email': 'john@handson.cloud'},
ExpressionAttributeNames={
"#section": "Section",
"#qa": "QA",
},
ExpressionAttributeValues={
':id': 'QA-2'
},
UpdateExpression="SET #section.#qa = :id",
)
print(response)
Delete Items
To delete an item, use the delete_item() method and specify the item’s primary key.
import boto3
dynamodb = boto3.resource('dynamodb')
table = dynamodb.Table('Employees')
response = table.delete_item(Key = {'Name': 'Peter Matovu', 'Email': 'petermatovu@handson.cloud'})
print(response)Here is the Execution output.
Query Tables in DynamoDB using Boto3
To query dynamondb items, you can use the query() method to fetch items based on primary key values. In addition, you can use the KeyConditionExpression to specify the value of the partition key and return all items from the table with that partition key.
Ensure to use the DynamoDB conditions where for example, the eq() method will fetch all the attributes that match the specified key.
import boto3
from boto3.dynamodb.conditions import Key
dynamodb = boto3.resource('dynamodb')
table = dynamodb.Table('Employees')
# boto3 ddb query
response = table.query(KeyConditionExpression=Key('Name').eq('Luzze John'))
print("The query returned the following items:")
for item in response['Items']:
print(item)Here is the Execution output.
Scan Tables in DynamoDB using Boto3
Scanning can be used to access all the data in tables. The scan() method will read every item in the table and return the data. When you provide additional options such as the FilterExpression, you can return fewer and desired results from the table scan operation.
You can also add additional DynamoDB conditions and retrieve items with specific attributes from the table.
import boto3
from boto3.dynamodb.conditions import Key, Attr
dynamodb = boto3.resource('dynamodb')
table = dynamodb.Table('Employees')
response = table.scan(FilterExpression=Attr('Department').eq('IT'))
print("The query returned the following items:")
for item in response['Items']:
print(item)Here is the Execution output.
Global Secondary Index
A Global Secondary Index allows you to query attributes not part of the main table’s primary key. This will help you avoid the slowness and inefficiencies associated with a full table scan operation.
In addition, the global secondary index will contain attributes from the main table but will be organized by a primary key of its own, enabling faster queries.
To learn more about Global Secondary Indexes, consider looking at the information shared in the AWS documentation on this link.
Creating a Global Secondary Index using Boto3 Dynamodb
In the following example, I will create a global secondary index to the CapacityBuildingLevel attribute with all employees who have achieved specific capacity-building levels.
import boto3
AWS_REGION='us-east-1'
client = boto3.client('dynamodb', region_name=AWS_REGION)
try:
resp = client.update_table(
TableName="Employees",
AttributeDefinitions=[
{
"AttributeName": "CapacityBuildingLevel",
"AttributeType": "S"
},
],
GlobalSecondaryIndexUpdates=[
{
"Create": {
"IndexName": "CapacityBuildingIndex",
"KeySchema": [
{
"AttributeName": "CapacityBuildingLevel",
"KeyType": "HASH"
}
],
"Projection": {
"ProjectionType": "ALL"
},
"ProvisionedThroughput": {
"ReadCapacityUnits": 1,
"WriteCapacityUnits": 1,
}
}
}
],
)
print("Secondary index added!")
except Exception as e:
print("Error updating table:")
print(e)Here is the Execution output.
Querying a Global Secondary Index
Next, you can use the created secondary index to retrieve all Employees with the specific CapacityBuildingLevel set to 4.
import time
import boto3
from boto3.dynamodb.conditions import Key
AWS_REGION='us-east-1'
dynamodb = boto3.resource('dynamodb', region_name=AWS_REGION)
table = dynamodb.Table('Employees')
while True:
if not table.global_secondary_indexes or \
table.global_secondary_indexes[0]['IndexStatus'] != 'ACTIVE':
print('Waiting for index to backfill...')
time.sleep(5)
table.reload()
else:
break
# boto3 dynamo query
resp = table.query(
IndexName="CapacityBuildingIndex",
KeyConditionExpression=Key('CapacityBuildingLevel').eq('Level 4'),
)
print("The query returned the following items:")
for item in resp['Items']:
print(item)Here is the Execution output.
Backup a DynamoDB Table using Boto3
To create on-demand backups for dynamod table using Boto3, use the create_backup() method and pass the Table name and the destination backup Table name.
import boto3
client = boto3.client('dynamodb')
response = client.create_backup(
TableName='Employees',
BackupName='Employees-Backup-01'
)
print(response)Here is the execution output.
Delete a DynamoDB Table using Boto3
To delete a DynamoDB table using Boto3, use the delete() table method and insert the name of the Dynamodb table you want to delete.
import boto3
client = boto3.client('dynamodb')
try:
resp = client.delete_table(
TableName="Test_Delete",
)
print("Table deleted successfully!")
except Exception as e:
print("Error deleting table:")
print(e)Here is the execution output.
FAQ
How to get DynamoDB table status?
To get DynamoDB table status, you must use the DynamoDB Boto3 resource to instantiate the Table class and access the table_status property:
import boto3
dynamodb = boto3.resource('dynamodb')
table = dynamodb.Table('name')
print(f'Table status: {table.table_status}')Additional Learning Resources
Free hands-on AWS workshops
We recommend the following free AWS Lambda Workshops to get more hands-on experience on the subject:
- AWS Lambda Workshop
- Migrate Spring Boot applications to AWS Lambda
- Building CI/CD pipelines for Lambda canary deployments using AWS CDK
- Content Generation Lambda@Edge
Also, we recommend the following free AWS Step Functions Workshops:
Paid courses
From our experience, these are the best hands-on paid learning materials today related to Serverless, AWS Lambda, and Step Functions:
Summary
This article covered how to use Python to interact with Amazon DynamoDB to create, list, and perform CRUD operations, query DynamoDB tables, and additional management operations in Amazon DynamoDB.
