Mastering AWS DynamoDB: An Easy And Complete Guide

Amazon DynamoDB is a fully managed NoSQL database service that provides quick and predictable performance with seamless scalability. Whether running microservices, mobile backends, or real-time bidding systems, DynamoDB is designed to handle various data models, like key-value and document, offering developers a flexible platform for web-scale applications.

Introduction to Amazon DynamoDB

Amazon DynamoDB acts as a highly scalable and fully managed database for:

  • Flexible microservices
  • Serverless web applications
  • Mobile backends

The most important topic of DynamoDB is how to design your tables to get maximum performance and efficiency. Regardless of your current experience, I highly recommend a couple of videos on that topic before getting started:

After covering DynamoDB design concepts, you can start using it.

Pro tip: Use “ddb” keyword instead of “dynamodb” when searching DynamoDB related information. For example: “boto3 ddb”. Google is smart enough to understand you. Check it out!

Setting Up DynamoDB with Terraform

Terraform provides an excellent approach to creating and managing AWS resources, including DynamoDB. It uses infrastructure as code, which allows you to describe and provision all the infrastructure resources in your cloud environment using declarative language.

Basic Terraform Configuration for DynamoDB

Firstly, let’s start by setting up a basic DynamoDB table using Terraform. Here’s an example Terraform configuration for a DynamoDB table:

provider "aws" {
  region = "us-west-2"
resource "aws_dynamodb_table" "example" {
  name           = "exampleTable"
  read_capacity  = 20 
  write_capacity = 20
  hash_key       = "id"
  attribute {
    name = "id"
    type = "S"

In this configuration:

  • provider "aws" specifies that we are using AWS as the cloud provider, and the region argument is set to “us-west-2”.
  • resource "aws_dynamodb_table" "example" creates a DynamoDB table named “exampleTable”.
  • read_capacity and write_capacity are set to 20, which determines this table’s read and write capacity units.
  • hash_key is set to “id”, meaning DynamoDB uses “id” as the partition key for the table.
  • The attribute block defines an attribute with the name “id” and type “S” (for string).

This basic setup creates a DynamoDB table ready for your applications.

Advanced Configuration: Provisioning Throughput and Indexes

Sometimes, you may need a more advanced configuration, such as setting up provisioned throughput and global secondary indexes (GSIs). Here is a sample Terraform configuration that includes these features:

resource "aws_dynamodb_table" "advanced_example" {
  name           = "advancedExampleTable"
  read_capacity  = 100
  write_capacity = 200
  hash_key       = "id"
  attribute {
    name = "id"
    type = "S"
  attribute {
    name = "email"
    type = "S"
  global_secondary_index {
    name               = "emailIndex"
    hash_key           = "email"
    write_capacity     = 100
    read_capacity      = 200
    projection_type    = "ALL"

In this advanced configuration:

  • We’ve introduced another attribute, “email” of type string.
  • We’ve added a global_secondary_index block, which creates a global secondary index named “emailIndex”. The hash_key for this index is “email”, allowing us to perform high-performance queries using email as the search key.
  • write_capacity and read_capacity within the global_secondary_index block are set to 100 and 200, determining this index’s read and write capacity.
  • projection_type is set to “ALL” meaning that all of the attributes in the table are replicated in the index.

Please note it’s important to manage your read and write capacity wisely, as it directly impacts the cost and performance of your DynamoDB table.

The next section delves into DynamoDB data modeling and how to manipulate your DynamoDB data using Python.

Understanding DynamoDB Limits

When designing applications with DynamoDB, understanding the service’s limitations is crucial to ensure a smooth, scalable, and efficient architecture. Let’s look at some of the most important DynamoDB limits.

Item Size Limit

The maximum item size in DynamoDB is 400 KB. This limit includes attribute name binary length (UTF-8 encoded) and attribute value lengths (again binary length). Large items might also use up your provisioned throughput more quickly, so you must factor this into your capacity planning if your workload has large items.

Read and Write Capacity Unit Limits

When using the provisioned capacity mode, specify your table’s capacity in read capacity units (RCUs) and write capacity units (WCUs). A single table in DynamoDB can have up to 40,000 RCUs or WCUs when using on-demand mode. If you need more, you can request a limit increase from AWS.

Remember that in provisioned mode, each RCU gives you one strongly consistent read per second for items up to 4 KB, while each WCU gives you one write per second for items up to 1 KB.

Secondary Indexes

You can create one or more secondary indexes on a DynamoDB table. You can create up to 20 global secondary indexes (GSIs) and 5 local secondary indexes (LSIs) per table. Each index has its read and write capacity, separate from the base table.

Partition Limit

DynamoDB data is stored across multiple partitions. Each partition can store up to 10 GB of data and handle up to 3,000 RCUs or 1,000 WCUs. If your data volume or throughput needs exceed these limits, DynamoDB will automatically split your data into additional partitions.

DynamoDB Streams Shard Limit

Each table in DynamoDB can have a DynamoDB Stream associated with it. Each stream is composed of shards, which can support up to 1,000 write transactions per second, up to a maximum of 2 MB of data per second.

It’s important to design your applications with these limits in mind. If you hit any of these limits, you may need to refactor your data model, request a limit increase, or find a workaround.

For more information on DynamoDB limits and how to work with them, refer to the AWS DynamoDB Limits Documentation. Reviewing this page regularly’s always a good idea, as AWS occasionally adjusts these limits.

DynamoDB Accelerator (DAX)

DynamoDB Accelerator (DAX) is a fully managed, highly available, in-memory cache that can reduce Amazon DynamoDB response times from milliseconds to microseconds, even at millions of requests per second. If your applications require microsecond latency, DAX can be a game-changer.

DAX is API-compatible with DynamoDB, meaning you don’t need to modify your application logic to take advantage of it. Simply change your DynamoDB client in your application to the DAX client.

Remember that DAX does add additional cost to your DynamoDB deployment, so it’s best suited for read-intensive, latency-sensitive workloads.

DynamoDB Global Tables

If your application needs to serve users globally and requires fast local performance in multiple regions, DynamoDB Global Tables can be a perfect solution. Global tables replicate your DynamoDB tables across your choice of AWS regions.

With global tables, your applications can read and write data with low latency, regardless of the users’ location. It provides a fully managed multi-master, multi-region replication solution, adding support for automatic conflict resolution.

When creating a global table, specify the AWS regions where you want the table to be available. DynamoDB takes care of the data replication, allowing you to focus on your application logic.

DynamoDB Pricing and Free Tier

DynamoDB pricing has several components:

  1. Provisioned Throughput: If you’re using the provisioned capacity mode, you pay for the read and write capacity you provision.
  2. On-Demand Capacity: If you’re using the on-demand capacity mode, you pay for the read and write capacity that your application consumes.
  3. Data Storage: You pay for the total amount of stored data in your DynamoDB tables, measured in GB.
  4. DAX: If you’re using DAX, there’s an additional cost based on the node type and number of nodes.
  5. Global Tables: Replicating your tables across multiple regions incurs additional costs.
  6. Data Transfer: There is a cost for transferring data out of DynamoDB (data transfer in is free), but this cost is waived if the data is transferred to another AWS service within the same region.

DynamoDB offers a generous free tier. The DynamoDB free tier includes 25 GB of monthly data storage, 25 provisioned read capacity units (RCUs), and 25 provisioned write capacity units (WCUs). This is enough to handle up to 200 million requests per month.

It’s important to note that the free tier does not cover DAX, global tables, or data transfer costs. For full details and the most current pricing information, check the DynamoDB Pricing Page.

DynamoDB Data Modeling

Understanding how to model your data correctly in DynamoDB is essential for building scalable, efficient applications. It’s not only about storing data but also about how you retrieve it. DynamoDB can easily handle large-scale, high-throughput read and write workloads when modeled correctly.

Keys and Indexes: Primary, Sort, and Secondary Indexes

Every DynamoDB table starts with a primary key, which uniquely identifies each item in the table. The primary key can be simple (partition key) or composite (partition key and sort key).

  • Partition Key: It’s a simple primary key composed of one attribute known as the partition key. DynamoDB uses the partition key’s value to distribute data across multiple partitions for scalability and performance.
  • Sort Key: When using a composite primary key, the second attribute is the sort key. DynamoDB stores record with the same partition key on the same partition, sorted by sort key value.

Here’s a sample Terraform code illustrating how to set both keys:

resource "aws_dynamodb_table" "key_example" {
  name           = "keyExampleTable"
  read_capacity  = 20
  write_capacity = 20
  hash_key       = "id"  // Partition key
  range_key      = "date" // Sort key
  attribute {
    name = "id"
    type = "S"
  attribute {
    name = "date"
    type = "S"

In addition to primary keys, DynamoDB supports secondary indexes for more flexible querying. There are two types:

  • Global Secondary Indexes (GSIs): These are indexes with a partition key and sort key that can differ from the primary key. GSIs span all partitions and are stored separately from the table.
  • Local Secondary Indexes (LSIs): These indexes have the same partition key as the table but a different sort key. LSIs provide more querying flexibility but must be created when the table is created and cannot be modified or removed afterward.

Data Types in DynamoDB

DynamoDB supports multiple data types that can be used for any attribute in a table or index. Below are the primary data types:

  • Scalar Types: These are single-valued types, including string (S), number (N), binary (B), boolean (BOOL), and null (NULL).
  • Document Types: These include list (L) and map (M). Lists are ordered collections of values, while maps are unordered collections of name-value pairs.
  • Set Types: These include string set (SS), number set (NS), and binary set (BS). A set can only contain unique values.

Here’s an example of a DynamoDB table with different data types using Terraform:

resource "aws_dynamodb_table" "types_example" {
  name           = "typesExampleTable"
  read_capacity  = 20
  write_capacity = 20
  hash_key       = "id"
  attribute {
    name = "id"
    type = "N"  // Number
  attribute {
    name = "tags"
    type = "SS"  // String Set
  attribute {
    name = "info"
    type = "M"  // Map

In the above example, the “id” is a number, “tags” is a set of strings, and “info” is a map.

In the next section, we’ll demonstrate how to interact with your DynamoDB tables using Python and the Boto3 library, which allows for more dynamic data operations beyond the static setup of Terraform.

Working with DynamoDB using Python

Check our Boto3 DynamoDB Tutorial for more information on creating and managing tables, handling various data operations, and using advanced features like Global Secondary Indexes and PartiQL to performance optimization, security considerations, and best practices in DynamoDB usage.

DynamoDB Performance Optimization

Amazon DynamoDB is designed to deliver consistent, fast performance at any scale. However, to get the most out of DynamoDB, it’s crucial to understand how to optimize for your specific use case correctly.

Understanding and Implementing Auto-Scaling

DynamoDB’s auto-scaling functionality is one of its key features. It allows the table’s capacity to increase or decrease automatically in response to traffic changes, which can be cost-efficient and ensure consistent performance.

Here is a sample Terraform configuration that sets up auto-scaling for a DynamoDB table:

resource "aws_dynamodb_table" "autoscale_example" {
  name           = "autoscaleExampleTable"
  hash_key       = "id"
  attribute {
    name = "id"
    type = "S"
resource "aws_appautoscaling_target" "read_target" {
  max_capacity       = 200
  min_capacity       = 5
  resource_id        = "table/${}"
  scalable_dimension = "dynamodb:table:ReadCapacityUnits"
  service_namespace  = "dynamodb"
resource "aws_appautoscaling_policy" "read_policy" {
  name               = "ReadCapacityScalingPolicy"
  policy_type        = "TargetTrackingScaling"
  resource_id        = aws_appautoscaling_target.read_target.resource_id
  scalable_dimension = aws_appautoscaling_target.read_target.scalable_dimension
  service_namespace  = aws_appautoscaling_target.read_target.service_namespace
  target_tracking_scaling_policy_configuration {
    predefined_metric_specification {
      predefined_metric_type = "DynamoDBReadCapacityUtilization"
    target_value       = 70
    scale_in_cooldown  = 60
    scale_out_cooldown = 60

In this example, the aws_appautoscaling_target resource sets the table’s maximum and minimum read capacity. The aws_appautoscaling_policy resource sets up a target tracking policy that adjusts the read capacity so that the DynamoDBReadCapacityUtilization metric stays around the target value of 70 percent. The scale_in_cooldown and scale_out_cooldown parameters ensure that there’s a 60-second pause between capacity changes.

Best Practices for Read/Write Capacity Modes

When setting up a DynamoDB table, you need to choose between two capacity modes: provisioned and on-demand.

  • Provisioned: You specify the number of the read and write capacity units you expect your application to require. You can use auto-scaling to adjust these limits in response to traffic patterns. This is the best option for predictable workloads and cost optimization.
  • On-Demand: DynamoDB manages capacity planning, providing ample read and write capacity as required by your workload. This is the best option for unpredictable workloads where traffic can spike suddenly.

Here’s how you can specify the capacity mode in your Terraform configuration:

resource "aws_dynamodb_table" "capacity_example" {
  name           = "capacityExampleTable"
  hash_key       = "id"
  billing_mode   = "PAY_PER_REQUEST"  // for on-demand mode
  attribute {
    name = "id"
    type = "S"

The billing_mode parameter sets the capacity mode. Set it to PROVISIONED (default) or PAY_PER_REQUEST (for on-demand mode).

Optimizing DynamoDB’s performance requires a deep understanding of your application’s needs and how these match DynamoDB’s features. With its range of capacity modes and auto-scaling options, DynamoDB can meet the needs of almost any application, from small projects to large, high-throughput systems.

In the next section, we will dive into DynamoDB’s security features and how to ensure your data is secure while still being readily accessible to your applications.

Security in DynamoDB

Data security is paramount in today’s digital world. Fortunately, DynamoDB offers robust built-in security features to ensure your data is safe. Let’s explore these capabilities further.

Managing IAM Policies for DynamoDB

IAM (Identity and Access Management) is the AWS service that allows you to manage access to your AWS resources. With IAM, you can create users, groups, and roles to which you can attach policies that dictate what actions they can perform on your DynamoDB tables.

Here is a sample IAM policy in JSON format that allows a user to perform select actions on a specific DynamoDB table:

    "Version": "2012-10-17",
    "Statement": [
            "Effect": "Allow",
            "Action": [
            "Resource": "arn:aws:dynamodb:us-east-1:123456789012:table/exampleTable"

In this example, the policy allows a user to put items, get items, scan, and delete items from the table named exampleTable. Using the AWS Management Console, AWS CLI, or AWS SDKs, you can attach this policy to an IAM user, group, or role.

Encryption at Rest and In Transit

Encryption is a critical aspect of data security. DynamoDB automatically encrypts all data at rest using AWS-owned keys by default. Still, you can also choose to use AWS-managed keys or customer-managed keys via AWS Key Management Service (KMS).

Here’s an example of a Terraform configuration that creates a DynamoDB table with server-side encryption using an AWS-managed KMS key:

resource "aws_dynamodb_table" "encryption_example" {
  name           = "encryptionExampleTable"
  hash_key       = "id"
  attribute {
    name = "id"
    type = "S"
  server_side_encryption {
    enabled     = true
    kms_key_arn = "arn:aws:kms:us-east-1:123456789012:key/abcd1234-a123-456a-a12b-a123b4cd56ef"

All data in transit between your application and DynamoDB is encrypted using HTTPS.

In addition to encryption, DynamoDB also supports VPC endpoints, which allow secure, private connectivity between your VPCs and DynamoDB, eliminating the need to expose your traffic to the public internet.

Remember, while AWS provides the tools to secure your data, it’s your responsibility to use them effectively. Stay aware of the AWS shared responsibility model, and always follow best practices for data security.

In the next section, we’ll delve into more advanced features of DynamoDB and how to use them effectively for your use case.

Real-World Scenarios: Use Cases for DynamoDB

Due to its flexibility and performance, DynamoDB fits a variety of real-world scenarios. Let’s explore some specific use cases where DynamoDB shines, from the gaming industry to finance.

Gaming: Player Data and State Management

Modern online games often need to manage data for millions of players concurrently. This includes player profiles, game state data, and leaderboard information. DynamoDB’s ability to handle high-throughput, low-latency workloads makes it an excellent choice for this scenario.

For instance, we can use DynamoDB’s atomic counters to update player scores incrementally to create a global leaderboard. With the Boto3 Python SDK, updating a score could look something like this:

def update_score(player_id, score):
    dynamodb = boto3.resource('dynamodb')
    table = dynamodb.Table('Leaderboard')
            'player_id': player_id
        UpdateExpression="set score = score + :val",
            ':val': score

Microservices: Session Storage

Microservices often require a fast, scalable session storage solution. With its automatic multi-AZ data replication, DynamoDB provides high availability and durability.

Here’s a Python example of storing session data in DynamoDB:

def store_session(session_id, data):
    dynamodb = boto3.resource('dynamodb')
    table = dynamodb.Table('Sessions')
            'session_id': session_id,
            'data': data

Finance: Real-Time Bidding

In the financial industry, DynamoDB is an excellent choice for storing bid data due to its ability to write and read rapidly. Bids can be recorded in DynamoDB as soon as they’re made, and when the auction ends, a quick scan can find the highest bid.

We must ensure our table is set up for high throughput for real-time bidding. In this case, auto-scaling or on-demand capacity mode would be useful.

Each use case for DynamoDB has its unique requirements and considerations. Understanding the specific needs of your application will help you make the most of DynamoDB’s powerful capabilities.

In the next section, we will provide additional resources for diving deeper into DynamoDB and its wide range of features.

Troubleshooting Common DynamoDB Issues

Working with DynamoDB can be straightforward, but you may encounter issues or errors like any technology. Let’s discuss some common DynamoDB issues and how to troubleshoot them.

Exceeded Provisioned Throughput

One common issue when working with DynamoDB is exceeding your provisioned throughput. If you’re seeing errors like ProvisionedThroughputExceededException, it means your application is trying to read or write data more quickly than the provisioned capacity allows.

To solve this issue, you can:

  1. Increase your table’s provisioned read or write capacity: This can be done through the AWS Management Console, AWS CLI, or with Terraform: resource "aws_dynamodb_table" "example" { // ... read_capacity = 20 // Increase this value write_capacity = 20 // Increase this value }
  2. Enable auto-scaling: As discussed earlier in this post, auto-scaling automatically adjusts your table’s read and write capacity based on traffic patterns.
  3. Switch to on-demand capacity mode: DynamoDB can automatically manage your table’s capacity.

High Latency

If you’re experiencing high latency with DynamoDB operations, it could be due to several reasons, such as:

  1. You’re using eventually consistent reads when strongly consistent reads are required: In this case, changing your read consistency settings could solve the issue.
  2. The distance between your application and the AWS region: For applications that require low latency, AWS recommends running your application in the same region as your DynamoDB table.
  3. Your application is not correctly handling DynamoDB’s pagination: If your application performs scan or query operations that return large amounts of data, you must ensure your application is correctly handling the paginated responses from DynamoDB.

Problems with Conditional Writes

Conditional writes in DynamoDB are a powerful feature that allows you to maintain data integrity. However, they can sometimes lead to unexpected ConditionalCheckFailedException errors. If you’re seeing these errors, check the following:

  1. Ensure the condition expression is correct: Conditional expressions must be written in the correct format and refer to the correct attribute names and values.
  2. Check if the item you’re trying to update exists: If the item doesn’t exist and your conditional expression tests for attributes on that item, the conditional check will fail.

Troubleshooting issues in DynamoDB involves understanding the service deeply and interpreting error messages correctly. AWS provides comprehensive documentation that can help identify and solve these problems. Remember, it’s also important to monitor your DynamoDB tables regularly using CloudWatch metrics to identify potential issues proactively.

In the next section, we will wrap up this complete guide to DynamoDB and summarize key takeaways.

Conclusion: Mastering DynamoDB

Mastering Amazon DynamoDB requires a strong understanding of its core concepts, capabilities, and potential pitfalls. Through this guide, we’ve explored what DynamoDB is, how to set it up using Terraform, key aspects of data modeling, and how to interact with it using Python. We’ve also touched on important features like performance optimization, security considerations, and troubleshooting common issues.

DynamoDB’s flexibility, scalability, and speed make it a reliable choice for many applications. It can efficiently handle a broad range of real-world scenarios, from gaming to finance. Still, aligning DynamoDB’s features with your specific needs is crucial to get the most out of this powerful service.

While we’ve covered a lot, there’s still much more to DynamoDB. Things like global tables for multi-region replication, streams for real-time data processing, or integrations with other AWS services like Lambda and Redshift are worth exploring.

Remember that mastering any technology is a journey. Continue exploring, practicing, and building with DynamoDB. Use the abundant resources and tools AWS provides and the vibrant AWS community.

As you continue your journey with DynamoDB, consider the following:

  • Refer back to this guide as needed, and don’t hesitate to explore AWS’s extensive documentation.
  • Monitor your DynamoDB tables using CloudWatch and respond to changes in usage patterns.
  • Stay informed about updates and new features. AWS continually evolves its services based on user feedback and technological advancements.

To truly master DynamoDB, it’s not enough to know it in theory; you need to get your hands dirty. Build something, make mistakes, learn, and grow. The most important skill in using any technology is the ability to adapt and learn, and DynamoDB is no exception.

Thank you for reading this guide on Mastering Amazon DynamoDB. We hope you found it useful and wish you success in your future DynamoDB adventures!