Working with DynamoDB in Python using Boto3

Kelvin Galabuzi

Kelvin Galabuzi

0
(0)

Amazon DynamoDB is a key-value and document database that is provided as a fully managed service in AWS which provides the possibility to build applications handling millions of requests and can scale to meet the demands of the market. Amazon DynamoDB has been proven efficient for use in various industries ranging from consumer apps to enterprise apps. In this article, we are going to look at how to create tables, load data, perform CRUD operations, and query tables in AWS DynamoDB by using Python and Boto3 library.

Prerequisites

To start working with DynamoDB using Python, you need to first set up your Python environment.

In summary, this is what you will need:

  • Python 3
  • Boto3
  • AWS CLI tools

Alternatively, you can quickly set up and use Cloud9 IDE.

Introduction to Amazon DynamoDB

Amazon DynamoDB enables the building of flexible microservices, serverless web applications, mobile backends, and it acts as the highly scalable and fully managed database for applications.

The most interesting topic of DynamoDB is how to design your tables to get maximum performance and efficiency from DynamoDB service. Regardless of your current experience I highly recommend you a couple of videos on that topic before getting started:

Now, as soon as the concepts of designing DynamoDB tables are covered, you may take a look at how to start working with this service in Python by using the Boto3 library.

Create Tables in DynamoDB using Boto3

First, import the boto3 module and then create a Boto3 DynamoDB resource.

Next, create a table named Employees with a primary key that has the following attributes;

Name a partition key with AttributeType set to S for string.

Email a sort key with AttributeType set to S for string.

ProvisionedThroughput means the maximum number of consistent reads and writes per second on your table.

import boto3

dynamodb = boto3.resource('dynamodb')

table = dynamodb.create_table (
    TableName = 'Employees',
       KeySchema = [
           {
               'AttributeName': 'Name',
               'KeyType': 'HASH'
           },
           {
               'AttributeName': 'Email',
               'KeyType': 'RANGE'
           }
           ],
           AttributeDefinitions = [
               {
                   'AttributeName': 'Name',
                   'AttributeType': 'S'
               },
               {
                   'AttributeName':'Email',
                   'AttributeType': 'S'
               }
            ],
            ProvisionedThroughput={
                'ReadCapacityUnits':1,
                'WriteCapacityUnits':1
            }
          
    )
print(table)

Here is the execution output:

List Tables in DynamoDB using Boto3

To list all tables in DynamoDB, use the dynamodb.tables.all() method.

import boto3

dynamodb = boto3.resource('dynamodb')

print(list(dynamodb.tables.all()))

Here is the execution output.

CRUD Operations in DynamoDB using Boto3

Tables, items, and attributes are the core components that make up DynamoDB. A table is a collection of items and each item is a collection of attributes.

In the following section, we are going to look at the four basic ways to perform CRUD operations on Items in DynamoDB Tables.

Create Items

To add a new item to a table, use the put_item() method and add an item with a primary of Name and Email as shown in the example.

import boto3

dynamodb = boto3.resource('dynamodb')

table = dynamodb.Table('Employees')

response = table.put_item(
Item = { 
     'Name': 'Kelvin Galabuzi',
     'Email': '[email protected]'
       }
)
print(response)

Here is the Execution output:

Batch Write Items

In addition to the put item, DynamoDb also has the batch_writer() method that can be used to create multiple items in a table using a single request.

NB: You can also add multiple attributes to items as required.

import boto3

dynamodb = boto3.resource('dynamodb')

table = dynamodb.Table('Employees')

with table.batch_writer() as batch:
    batch.put_item(Item={"Name": "Luzze John", "Email": "[email protected]",
        "Department": "IT", "Section": { "QA": "QA-1", "Reporting Line": "L1" } })
    batch.put_item(Item={"Name": "Lugugo Joshua", "Email": "[email protected]",
        "Department": "IT", "Section": { "Development": "SD-1", "Reporting Line": "L1" } })
    batch.put_item(Item={"Name": "Robert Nsamba", "Email": "[email protected]",
        "Department": "IT", "Section": { "PM": "PM-1", "Reporting Line": "L1" } })
print(batch)

Here is the execution output:

Read Items

To read a single item from the table, use the get_item() method with the key value pair of items in your table.

import boto3

dynamodb = boto3.resource('dynamodb')

table = dynamodb.Table('Employees')

response = table.get_item(
    Key={
        'Name': 'Kelvin Galabuzi',
        'Email': '[email protected]'
    }
)
print(response['Item'])

Here is the Execution output:

To all read Items in the table, you can use the scan() method to return all items in the table.

import boto3

dynamodb = boto3.resource('dynamodb')

table = dynamodb.Table('Employees')

response = table.scan()
response['Items']

print(response)

Here is the Execution output:

Update Items

To update items, you can use the update_item() method to edit an existing item’s attributes. You can also use the same method to add new attributes or remove attributes.

import boto3

dynamodb = boto3.resource('dynamodb')

table = dynamodb.Table('Employees')

response = table.update_item(
    Key={'Name': 'Luzze John', 'Email': '[email protected]'},
    ExpressionAttributeNames={
        "#section": "Section",
        "#qa": "QA",
        },
        ExpressionAttributeValues={
            ':id': 'QA-2'
        },
        UpdateExpression="SET #section.#qa = :id",
    )
print(response)

Delete Items

To delete an item, use the delete_item() method and specify the item’s primary key.

import boto3

dynamodb = boto3.resource('dynamodb')

table = dynamodb.Table('Employees')

response = table.delete_item(Key = {'Name': 'Peter Matovu', 'Email': '[email protected]'})

print(response)

Here is the Execution output.

Query Tables in DynamoDB using Boto3

To query items in DynamoDB, you can use the query() method to fetch items based on primary key values. In addition, you can use the KeyConditionExpression to specify the value of the partition key and return all items from the table with that partition key.

Ensure to make use of the DynamoDB conditions where for example the eq() method will be used to fetch all the attributes that match the specified key.

import boto3
from boto3.dynamodb.conditions import Key

dynamodb = boto3.resource('dynamodb')
table = dynamodb.Table('Employees')

response = table.query(KeyConditionExpression=Key('Name').eq('Luzze John'))

print("The query returned the following items:")
for item in response['Items']:
    print(item)

Here is the Execution output.

Scan Tables in DynamoDB using Boto3

Scanning can be used to access all the data in tables. The scan() method will read every item in the table and return the data. When you provide additional options such as the FilterExpression, you will be able to return fewer and desired results from the table scan operation.

You can also add additional DynamoDB conditions and retrieve items with specific attributes from the table.

import boto3
from boto3.dynamodb.conditions import Key, Attr

dynamodb = boto3.resource('dynamodb')
table = dynamodb.Table('Employees')

response = table.scan(FilterExpression=Attr('Department').eq('IT'))

print("The query returned the following items:")
for item in response['Items']:
    print(item)

Here is the Execution output.

Global Secondary Index

A Global Secondary Index allows you to query attributes that are not part of the main table’s primary key and this will help you to avoid the slowness and inefficiencies that are associated when performing a full table scan operation.

In addition, the global secondary index will contain attributes from the main table but will be organized by a primary key of its own hence enabling faster queries.

To learn more about Global Secondary Indexes, consider looking at the information shared in the AWS documentation found on this link.

Creating a Global Secondary Index using Boto3

In the following example, I will create a global secondary index to the CapacityBuildingLevel attribute that has all Employees that have achieved specific levels of Capacity building.

import boto3

client = dynamodb = boto3.client('dynamodb')

try:
    resp = client.update_table(
        TableName="Employees",
        AttributeDefinitions=[
            {
                "AttributeName": "CapacityBuildingLevel",
                "AttributeType": "S"
            },
        ],
        GlobalSecondaryIndexUpdates=[
            {
                "Create": {
                    "IndexName": "CapacityBuildingIndex",
                    "KeySchema": [
                        {
                            "AttributeName": "CapacityBuildingLevel",
                            "KeyType": "HASH"
                        }
                    ],
                    "Projection": {
                        "ProjectionType": "ALL"
                    },
                    "ProvisionedThroughput": {
                        "ReadCapacityUnits": 1,
                        "WriteCapacityUnits": 1,
                    }
                }
            }
        ],
    )
    print("Secondary index added!")
except Exception as e:
    print("Error updating table:")
    print(e)

Here is the Execution output.

Querying a Global Secondary Index

Next, you can use the created secondary index to retrieve all Employees with the specific CapacityBuildingLevel set to 4.

import time

import boto3
from boto3.dynamodb.conditions import Key

dynamodb = boto3.resource('dynamodb')
table = dynamodb.Table('Employees')

while True:
    if not table.global_secondary_indexes or table.global_secondary_indexes[0]['IndexStatus'] != 'ACTIVE':
        print('Waiting for index to backfill...')
        time.sleep(5)
        table.reload()
    else:
        break

resp = table.query(
    IndexName="CapacityBuildingIndex",
    KeyConditionExpression=Key('CapacityBuildingLevel').eq('Level 4'),
)

print("The query returned the following items:")
for item in resp['Items']:
    print(item)

Here is the Execution output.

Backup a DynamoDB Table using Boto3

To create on-demand backups for tables using Boto3, use the create_backup() method and pass the Table name and the destination back up Table name.

import boto3

client = boto3.client('dynamodb')

response = client.create_backup(
    TableName='Employees',
    BackupName='Employees-Backup-01'
)
print(response)

Here is the execution output.

Delete a DynamoDB Table using Boto3

To delete a DynamoDB table using Boto3, use the delete() table method and insert the name of the Dynamodb table that you want to delete.

import boto3

client = boto3.client('dynamodb')

try:
    resp = client.delete_table(
        TableName="Test_Delete",
    )
    print("Table deleted successfully!")
except Exception as e:
    print("Error deleting table:")
    print(e)

Here is the execution output.

Summary

This article covered how to use Python to interact with Amazon DynamoDB to create, list, perform CRUD operations, query tables, and concluded with additional management operations in Amazon DynamoDB.

How useful was this post?

Click on a star to rate it!

As you found this post useful...

Follow us on social media!

We are sorry that this post was not useful for you!

Let us improve this post!

Tell us how we can improve this post?

Top rated Udemy Courses to improve you career

Subscribe to our updates

Like this article?

Share on facebook
Share on Facebook
Share on twitter
Share on Twitter
Share on linkedin
Share on Linkdin
Share on pinterest
Share on Pinterest

Want to be an author of another post?

We’re looking for skilled technical authors for our blog!

Leave a comment

If you’d like to ask a question about the code or piece of configuration, feel free to use https://codeshare.io/ or a similar tool as Facebook comments are breaking code formatting.