Boto3 Secrets Manager Tutorial

Managing sensitive information securely is a crucial aspect of modern application development. AWS Secrets Manager, an efficient tool for storing and retrieving secrets, can be seamlessly integrated with your Python applications using the Boto3 library. This comprehensive guide covers everything from basic interactions to advanced usage, ensuring that your secrets are handled safely and efficiently.

Prerequisites

To access AWS Secrets Manager, you must install Boto3, an AWS SDK for Python. Also, you need to have AWS CLI configured to use the Boto3 library. Boto3 uses your AWS Access Key Id and Secret Access Key, or Boto3 session to programmatically manage AWS resources.

Second, install the boto3 library using the following command:

pip install boto3

For more in-depth information, we recommend you check out the Introduction to Boto3 library and How to use aws-vault to securely access multiple AWS accounts articles.

Creating A Secret

Secrets Manager stores the encrypted secret data in a collection of “versions” associated with the secret. Each version contains a copy of the encrypted secret data. Each version is associated with one or more “staging labels” that identify the version in the rotation cycle.

The Boto3 Secrets Manager client is a low-level class that provides methods to connect to AWS Secrets Manager, similar to the AWS API service. All service APIs in the Boto3 client map 1:1 to the AWS service API.

You can provide the secret data to be encrypted by putting text in either the SecretString parameter or binary data in the SecretBinary parameter, but not both. If you include SecretString or SecretBinary then Secrets Manager also creates an initial secret version and automatically attaches the staging label AWSCURRENT to the new version.

#!/usr/bin/env python3
import boto3
client = boto3.client('secretsmanager')
response = client.create_secret(
    Name='DatabaseProdSecrets',
    SecretString='{"username": "prod", "password": "hello-world-prod"}'
)

Output:

If you need to provide a custom KMS key, you can use the KmsKeyId parameter in create_secret() method that Specifies the ARN, Key ID, or alias of the Amazon Web Services KMS customer master key (CMK) to be used to encrypt the SecretString or SecretBinary values in the versions stored in this secret.

If you don’t provide theKmsKeyId, then Secrets Manager uses the account’s default CMK (the one named aws/secretsmanager ). If an Amazon Web Services KMS CMK with that name doesn’t exist, then Secrets Manager will create it for you automatically the first time it needs to encrypt a version’s SecretString or SecretBinary fields.

Listing secrets

You can use the list_secrets() method to list all secrets stored in AWS Secrets Manager. When listing secrets, you can also filter and limit the number of results to a specific number.

Note: The encrypted fields SecretString and SecretBinary are not included in the output. To get that information, you need to call the GetSecretValue operation.

#!/usr/bin/env python3
import boto3
client = boto3.client('secretsmanager')
response = client.list_secrets()
print(response['SecretList'])

Here’s an execution output:

Retrieving secret value

To retrieve a secret value from AWS Secrets Manager using Boto3, you need to use theget_secret_value() method.

The following code example will get the secret with SecretId (or Name when creating) of DatabaseProdSecrets. For more information, please read the Boto3 Secrets Manager documentation.

You also need to parse the SecretString value using the json.loads which converts JSON string into the Python dictionary so that you can access the items of a dictionary.

#!/usr/bin/env python3
import boto3 
import json 
client = boto3.client('secretsmanager')
response = client.get_secret_value(
    SecretId='DatabaseProdSecrets'
)
database_secrets = json.loads(response['SecretString'])
print(database_secrets['password'])

Here’s an execution output:

Permissions required to retrieve a secret

To retrieve the secret, you need to allow the secretsmanager:GetSecretValue API call in your IAM policy.

If you’re using a customer-managed Amazon Web Services KMS key to encrypt the secret, you also need to have kms:Decrypt permission.

Alternatively, you can attach the SecretsManagerReadWrite policy to the user who needs permission to manage AWS Secrets Manager.

Retrieve secret values from the Python code

To retrieve a secret value from AWS Secrets Manager using Boto3, you need to use theget_secret_value() method.

#!/usr/bin/env python3
import boto3
import json 
client = boto3.client('secretsmanager')
response = client.get_secret_value(
    SecretId='DatabaseProdSecrets'
)
database_secrets = json.loads(response['SecretString'])
print(database_secrets['password'])

Retrieve secret values in Bash

Using the AWS CLI, you can retrieve secret values in the Bash shell

aws secretsmanager get-secret-value --secret-id <SecretId>

Retrieve secret values in Powershell

The cmdlets in the AWS Tools for PowerShell for each service are based on the methods provided by the AWS SDK for the service.

You can use Get-SECSecretValue cmdlets to retrieve secrets. Read more about this cmdlet from here.

Get-SECSecretValue -SecretId <SecretId>

Retrieve secret values in AWS console

In the AWS Console, search for the Secret Manager; you will see all your Secrets there.

Click on one of the secrets and then click on the Retrieve secret value button to see the secret value.

Here’s what you should get:

Updating existing secret

There are two methods for updating secrets in Boto3.

The first one is put_secret_value(). This method creates a new version and attaches it to the secret

#!/usr/bin/env python3
import boto3 
import json 
client = boto3.client('secretsmanager')
response = client.put_secret_value(
    SecretId='DatabaseProdSecrets',
    SecretString='{"username": "prod", "password": "hello-world-updated2"}'
)
print(response)

Output:

The second one is the update_secret() method. This method modifies many of the details of the specified secret. If you include a ClientRequestToken and either SecretString or SecretBinary then it also creates a new version attached to the secret.

#!/usr/bin/env python3
import boto3 
import json
client = boto3.client('secretsmanager')
response = client.update_secret(
    SecretId='DatabaseProdSecrets',
    Description='Description updated'
)
print(response)

Here’s an execution output:

Creating a new version of the secret

You can use either put_secret_value or update_secret to create a new version of the secret.

The put_secret_value creates a new version and attaches it to the secret.

The update_secret method creates a new version attached to the secret when a ClientRequestToken and either SecretString or SecretBinary parameters are used.

Deleting a secret

You can use the delete_secret() function from Boto3 to delete secret and all of its versions. You can optionally include a recovery window during which you can restore the secret. If you don’t specify a recovery window value, the secret will be deleted within 30 days.

At any time before the recovery window ends, you can use RestoreSecret to remove the DeletionDate and cancel the deletion of the secret.

#!/usr/bin/env python3
import boto3
client = boto3.client('secretsmanager')
response = client.delete_secret(
    SecretId='DatabaseProdSecrets',
    RecoveryWindowInDays=10,
    ForceDeleteWithoutRecovery=False
)
print(response)

Here’s an execution output:

To restore deleted secrets before the recovery window end, you can use the restore_secret() method.

#!/usr/bin/env python3
import boto3
client = boto3.client('secretsmanager')
response = client.restore_secret(
    SecretId='DatabaseProdSecrets'
)
print(response)

Here’s an execution output:

Summary

This article covered Python to interact with AWS Secret Manager to create, update, and delete secrets using the Boto3 Python SDK.