EBS volumes are virtual hard drives that might be securely attached to your EC2 instances, providing persistent storage with durable and high-performance levels of throughput. They provide block-level storage for Amazon EC2 instances in the cloud, allowing users and applications to store their data.

Using this Boto3 EBS volumes tutorial and AWS SDK for Python to interact with the Amazon EC2 service and programmatically create, list, filter, describe, and delete Amazon Elastic Block Store (EBS) volumes using the Boto3 library. Additionally, you can attach EBS volumes to EC2 instances, detach them from EC2 instances, and increase EBS volume size when required.

This article provides code examples of each operation, which can be easily adjusted to your needs.

Prerequisites

To start automating Amazon EC2 and making API calls to manage EBS volumes, snapshots, and AMIs, you must first configure your Python environment.

In general, here’s what you need to have installed:

  • Python 3
  • Boto3
  • AWS CLI tools

We also suggest you use aws-vault for managing access to multiple AWS environments.

What is an EBS volume?

EC2 instances use block-level storage called Amazon Elastic Block Store (EBS). Amazon EBS offers durable, high-performance, and highly scalable volumes ranging from one GiB up to 16 TiB of storage size for EC2 instances. The data stored on EBS volume is automatically replicated across the Availability Zone (AZ) within AWS Region.

Working with EBS volumes, snapshots, and AMIs using Boto3 in Python - Workflow diagram

EBS volumes are easy to use, whether you need to add or remove them from the EC2 instance, modify their size, or change their type. When choosing your EBS volume types, you’ll find multiple options based on your business application requirements.

Did you know? You can configure AWS Backup to backup your EC2 instances and EBS volumes.

How to connect to Amazon EC2 using Boto3?

The Boto3 library provides you with two ways to access APIs for managing AWS services:

  • The client that allows you to access the low-level API data. For example, you can access API response data in JSON format.
  • The resource that allows you to use AWS services in a higher-level object-oriented way. For more information on the topic, look at AWS CLI vs. botocore vs. Boto3.

Here’s how you can instantiate the Boto3 EC2 client to start working with Amazon EC2 APIs:

import boto3
AWS_REGION = "us-east-1"
ec2_client = boto3.client("ec2", region_name=AWS_REGION)

You can similarly instantiate the Boto3 EC2 resource:

import boto3
AWS_REGION = "us-east-1"
ec2_resource = boto3.resource("ec2", region_name=AWS_REGION)

How to create EBS volume using Boto3?

One of the daily operations for AWS EBS volumes is creating new volumes and attaching them to the running EC2 instances. To create a new EBS volume, you can use the create_volume() method of the Boto3 EC2 client or resource.

Creating an EBS volume using Boto3 client

To create a new EBS volume, you can use the create_volume() method of the Boto3 EC2 client:

import boto3
AWS_REGION = "us-east-2"
ec2_client = boto3.client('ec2', region_name=AWS_REGION)
new_volume = ec2_client.create_volume(
    AvailabilityZone=f'{AWS_REGION}c',
    Size=10,
    VolumeType='gp2',
    TagSpecifications=[
        {
            'ResourceType': 'volume',
            'Tags': [
                {
                    'Key': 'Name',
                    'Value': 'hands-on-cloud-ebs-boto3'
                }
            ]
        }
    ]
)
print(f'Created volume ID: {new_volume["VolumeId"]}')

The required arguments are:

  • AvailabilityZone – specifies the Availability Zone where to create the Volume
  • Size – specifies the size of the Volume in GiB

The rest of the arguments are optional, but we set them up for demo purposes:

  • VolumeType – defines an EBS volume type. This parameter can be one of the following values:
    • General Purpose SSD: gp2 | gp3
    • Provisioned IOPS SSD: io1 | io2
    • Throughput Optimized HDD: st1
    • Cold HDD: sc1
    • Magnetic: standard
  • TagSpecifications allow you to define the Tags for the volume.

The new_volume variable contains an AWS EC2 service response serialized in the form of the Python dictionary, so, to get access to the volume ID, we have to use the VolumeId key.

Here’s an execution output:

2. Working with EBS volumes using Boto3 - Creating an EBS volume using Boto3 client

Creating an EBS volume using Boto3 resource

To create a new EBS volume, you can use the create_volume() method of the Boto3 EC2 resource:

import boto3
AWS_REGION = "us-east-2"
ec2_resource = boto3.resource('ec2', region_name=AWS_REGION)
new_volume = ec2_resource.create_volume(
    AvailabilityZone=f'{AWS_REGION}c',
    Size=10,
    VolumeType='gp2',
    TagSpecifications=[
        {
            'ResourceType': 'volume',
            'Tags': [
                {
                    'Key': 'Name',
                    'Value': 'hands-on-cloud-ebs-boto3'
                }
            ]
        }
    ]
)
print(f'Created volume ID: {new_volume.id}')

Here’s an execution output:

1. Working with EBS volumes using Boto3 - Creating an EBS volume using Boto3 resource

How to list all EC2 volumes using Boto3?

The easiest way to list all EC2 volumes is to use the all() method of the volumes‘ collection of the EC2 resource:

import boto3
AWS_REGION = "us-east-2"
ec2_resource = boto3.resource('ec2', region_name=AWS_REGION)
for volume in ec2.volumes.all():
    print(volume)

Here’s an execution output:

3. Working with EBS volumes using Boto3 - Listing all EC2 volumes using Boto3

How to search EC2 volumes Boto3?

The easiest way to search for EC2 volumes by the specified condition is to use the filter() method of the volumes resource.

Searching EC2 volumes by Tag

In the following example, we’ll find all volumes which have the Tag Name and Tag value hands-on-cloud-ebs-boto3:

import boto3
AWS_REGION = "us-east-2"
ec2_resource = boto3.resource('ec2', region_name=AWS_REGION)
for volume in ec2_resource.volumes.filter(
    Filters=[
        {
            'Name': 'tag:Name',
            'Values': [
                'hands-on-cloud-ebs-boto3',
            ]
        }
    ]
):
    print(f'Volume {volume.id} ({volume.size} GiB) -> {volume.state}')

Here’s an execution output:

4. Working with EBS volumes using Boto3 - Searching EC2 volumes by Tag using Boto3

Searching EC2 volumes by ID

In the following example, we’ll find all volumes that belong to a list of IDs:

import boto3
AWS_REGION = "us-east-2"
ec2_resource = boto3.resource('ec2', region_name=AWS_REGION)
for volume in ec2_resource.volumes.filter(
    VolumeIds=[
        'vol-07f77d0a13239ef7e',
        'vol-01e5646dd54edd848',
    ],
):
    print(f'Volume {volume.id} ({volume.size} GiB) -> {volume.state}')

Here’s an execution output:

5. Working with EBS volumes using Boto3 - Searching EC2 volumes by ID using Boto3

How to delete EC2 volumes using Boto3?

The easiest way to delete EC2 volume is to use the delete() method of the Volume Boto3 class.

Deleting a single EC2 volume

To delete a single EC2 volume, you can use the following code snippet:

import boto3
AWS_REGION = "us-east-2"
ec2_resource = boto3.resource('ec2', region_name=AWS_REGION)
volume = ec2_resource.Volume('vol-01e5646dd54edd848')
if volume.state == "available":
    volume.delete()
    print(f'Volume successfully deleted')
else:
    print(f"Can't delete volume attached to EC2 instance")

Note: you can delete only volumes in the state available.

Here’s an execution output:

6. Working with EBS volumes using Boto3 - Deleting single EC2 volume

Deleting multiple EC2 volumes

To delete multiple EC2 volumes, you can filter for required modules and then use for-loop to delete them one by one. Let’s delete all volumes in the state available. This is the common code that allows cleaning up unused EC2 volume resources:

import boto3
AWS_REGION = "us-east-2"
ec2_resource = boto3.resource('ec2', region_name=AWS_REGION)
for volume in ec2_resource.volumes.filter(
    Filters=[
        {
            'Name': 'status',
            'Values': [
                'available',
            ]
        }
    ]
):
    if volume.state == "available":
        volume_id = volume.id
        volume.delete()
        print(f'Volume {volume_id} successfully deleted')
    else:
        print(f"Can't delete volume attached to EC2 instance")

Here’s an execution output:

7. Working with EBS volumes using Boto3 - Deleting unused EC2 volumes

How to describe EBS volumes using Boto3?

The most straightforward way to describe the volume is to use the describe_volumes() method of the EC2 client provides complete information about the volume in JSON format.

The describe_volumes() method accepts the same arguments as the filter() method, so you can specify various conditions to find the required volumes. We’ll use VolumeID to simplify the example:

import json
from datetime import date, datetime
import boto3
AWS_REGION = "us-east-2"
EC2_CLIENT = boto3.client('ec2', region_name=AWS_REGION)

# Helper method to serialize datetime fields
def json_datetime_serializer(obj):
    if isinstance(obj, (datetime, date)):
        return obj.isoformat()
    raise TypeError ("Type %s not serializable" % type(obj))

describe_response = EC2_CLIENT.describe_volumes(
    VolumeIds=[
        'vol-0c0ce77e0b27ed800'
    ]
)
print('Volumes information:')
print(json.dumps(
        describe_response,
        indent=4,
        default=json_datetime_serializer
    )
)

In the example above, we’re using additional json_datetime_serializer() method to serialize (convert to string) datetime.datetime fields returned by the describe_volumes() method.

Here’s an example output:

7. Working with EBS volumes using Boto3 - Describing EBS volumes using Boto3

How to attach EBS volume to EC2 instance using Boto3?

To attach the EBS volume to the EC2 instance, you need to use attach_to_instance() method of the EC2 resource.

Note: you can only attach EBS volumes to the EC2 instances within the same Availability Zone. If you try to attach EBS volume to the instance from another Availability Zone, you’ll receive the botocore.exceptions.ClientError: An error occurred (InvalidVolume.ZoneMismatch) when calling the AttachVolume operation exception.

import boto3
AWS_REGION = "us-east-2"
ec2_resource = boto3.resource('ec2', region_name=AWS_REGION)
volume = ec2_resource.Volume('vol-0d4abbb9e7da7ed9f')
print(f'Volume {volume.id} status -> {volume.state}')
volume.attach_to_instance(
    Device='/dev/sdh',
    InstanceId='i-07125d93aed65ea84'
)
print(f'Volume {volume.id} status -> {volume.state}')

Here’s an execution output:

8. Working with EBS volumes using Boto3 - Attaching EBS volume to EC2 Instance

How to detach EBS volume from EC2 instance using Boto3?

To detach the EBS volume from the EC2 instance, you need to use the detach_from_instance() method of the Volume class. In addition to that, you have to use the waiter.VolumeAvailable class checks every 15 seconds that the EBS volume has been successfully disconnected from the EC2 instance by running the describe_volumes() method in the background. That’s the correct way to wait for the successful completion of detaching operation.

import boto3
AWS_REGION = "us-east-2"
EC2_RESOURCE = boto3.resource('ec2', region_name=AWS_REGION)
EC2_CLIENT = boto3.client('ec2', region_name=AWS_REGION)
volume = EC2_RESOURCE.Volume('vol-0d4abbb9e7da7ed9f')
print(f'Volume {volume.id} status -> {volume.state}')
volume.detach_from_instance(
    Device='/dev/sdh',
    InstanceId='i-07125d93aed65ea84'
)
# Vaiting for volume to become available
waiter = EC2_CLIENT.get_waiter('volume_available')
waiter.wait(
    VolumeIds=[
        volume.id,
    ]
)
print(f'Volume {volume.id} status -> {volume.state}')

Here’s an execution output:

9. Working with EBS volumes using Boto3 - Detaching EBS volume from EC2 instance

How to increase the size of the EBS volume using Boto3?

If you need to increase this size based on application requirements, you can use the modify_volume() method of the EC2 client.

You can only increase the size of EBS volume. There is no way to decrease the size of volume from AWS console, CLI, OR API.

Request modifications to your EBS volumes
import time
import boto3
AWS_REGION = "us-east-2"
EC2_CLIENT = boto3.client('ec2', region_name=AWS_REGION)
VOLUME_ID = 'vol-029786bbbf29faa8d'
def get_modification_state(volume_id):
    response = EC2_CLIENT.describe_volumes_modifications(
        VolumeIds=[
            VOLUME_ID
        ]
    )
    return response['VolumesModifications'][0]['ModificationState']
modify_volume_response = EC2_CLIENT.modify_volume(
    VolumeId=VOLUME_ID,
    Size=30
)
while True:
    state = get_modification_state(VOLUME_ID)
    if state == 'completed' or state == None:
        break
    elif state == 'failed':
        raise Exception('Failed to modify volume size')
    else:
        time.sleep(60)
print(f'Volume {VOLUME_ID} successfully resized')

In the example above, we have to use the get_modification_state() helper method, which returns the EBS volume modification status.

The while loop helps to wait till the volume modification operation finishes.

Note: pay attention that for large volumes, the volume modification operation might take many hours.

Here’s an execution output:

10. Working with EBS volumes using Boto3 - Increasing the size of the EBS volume

Summary

This article covered how to use the Python Boto3 library to implement the most common EBS volume operations, such as creating, listing, filtering (searching), deleting, and modifying.