Boto3 Parameter Store Tutorial

The Boto3 Parameter Store, a key feature within AWS Systems Manager (AWS SSM), offers a versatile platform for managing and configuring AWS resources as well as on-premises systems across different scales. It serves as a powerful tool for automating operational processes for resources like Amazon EC2 and RDS instances, as well as servers situated on-premises. Additionally, it allows for the grouping of resources based on their applications, facilitates the auditing of resource modifications, and aids in the secure storage of configuration details and sensitive data secrets. This article delves into practical examples demonstrating the effective utilization of the boto3 ssm client within the Boto3 Parameter Store.

What is AWS Systems Manager Parameter Store?

The Parameter Store is a capability of AWS Systems Manager which helps you store your application configuration data in a systematic hierarchical format for better reference and management. Data can be of any type, like API keys, passwords, URLs, Amazon Machine Image (AMI) IDs, licenses, database connections, or other strings. You can store data in encrypted or plain text format. Parameter Store stores all parameters in key-value format. In addition, it is integrated with AWS Key Management Service (AWS KMS), which provides the ability to encrypt the stored parameters securely.

The Parameter Store is an excellent service to store your application’s secrets and configurations across various environments without managing application configuration files. The Parameter Store has some advantages over other methods of managing variables:

For more information about the Parameter Store, we recommend checking out the Introduction to AWS Systems Manager article.

Parameter Store Types

A parameter is a key-value pair stored within the AWS Parameter Store. There are three primary types of Parameter Store parameters with an additional fourth sub-type parameter that can reference a valid Amazon Machine Image ID (AMI):

• String – unencrypted text value, for example, Hello World, hands-on.cloud, 10.0.0.1/24, etc.
• StringList – a collection of comma-separated strings, for example, EC2, S3, EBS Volumes, RDS Database or Python, Go, C++.
• SecureString – text value encrypted by the AWS KMS key.
• aws:ec2:image – When creating a String parameter, specify this data type to ensure that the parameter value you enter is a valid Amazon Machine Image (AMI) ID.

Connecting Parameter Store using Boto3

To connect to the AWS Parameter Store using Python SDK, you need to import the boto3 library and use its SSM client. The client that allows you to access the low-level SSM API data. For example, you can get access to API response data in JSON format:

import boto3
AWS_REGION = "us-east-1"
ssm_client = boto3.client("ssm", region_name=AWS_REGION)

Writing Parameter Store values using Boto3

To create a new parameter programmatically using Python, you need to use the put_parameter() method of the SSM client.

Creating Parameter Store String Parameter

import boto3
AWS_REGION = "us-east-1"
ssm_client = boto3.client("ssm", region_name=AWS_REGION)
new_string_parameter = ssm_client.put_parameter(
    Name='/dev/ec2Instance/type',
    Description='EC2 Instance type for Dev environment',
    Value='t3.medium',
    Type='String',
    Overwrite=True,
    Tier='Standard',
    DataType='text')
print(
    f"String Parameter created with version {new_string_parameter['Version']} and Tier {new_string_parameter['Tier']}"
)

The required arguments are:

• Name – specifies the fully qualified name of the parameter. The fully qualified name usually includes the complete hierarchy of the parameter path and name. For example. /dev/ec2Instance/type, which means EC2 instance type in a development environment

• Value – specifies the parameter value. Standard parameters have a value limit of 4 KB, whereas Advanced parameters have a value limit of 8 KB.

The rest of the arguments are optional, but we set them up for demo purposes:

• Description – specifies the additional information about the parameter
• Type – specifies the Parameter type. As explained previously, the parameter can be of the following three types
  • String
  • StringList
  • SecureString
• Overwrite – used to overwrite a current parameter value. The default value is False.
• Tier – the parameter tier to assign to a parameter. For more information, visit here.
• DataType – the data type for a String parameter. The following data type values are supported.
  • text
  • aws:ec2:image

Here’s an execution output:

Creating Parameter Store StringList Parameter

To create a parameter with the StringList type, we need to change the Type argument from the previous block of code for the String type parameter.

Type – StringList, gives the ability to create parameters with comma-separated values.

import boto3
AWS_REGION = "us-east-1"
ssm_client = boto3.client("ssm", region_name=AWS_REGION)
new_list_parameter = ssm_client.put_parameter(
    Name='/dev/ec2Instance/tags',
    Description='EC2 Instance tags for Dev environment',
    Value='dev,staging,pre-production',
    Type='StringList',
    Overwrite=True,
    Tier='Standard',
    DataType='text'
)
print(
    f"StringList Parameter created with version {new_list_parameter['Version']} and Tier {new_list_parameter['Tier']}"
)

Here’s an execution output:

Creating a SecureString Parameter

To store parameters such as Passwords, API Keys, Database connections, etc., securely and in an encrypted format, SSM Parameter Store integrates with AWS KMS. To store encrypted parameters, we can use the SecureString type.

Type – SecureString encrypts the parameter value using the AWS Managed Key from AWS KMS.

import boto3
AWS_REGION = "us-east-1"
ssm_client = boto3.client("ssm", region_name=AWS_REGION)
new_secure_parameter = ssm_client.put_parameter(
    Name='/dev/ec2Instance/sshPassword',
    Description='EC2 Instance SSH Password for Dev environment',
    Value='HighlySecurePassword',
    Type='SecureString',
    # KeyId='<customer-managed-kms-key-id>',
    Overwrite=True,
    Tier='Standard',
    DataType='text')
print(
    f"SecureString Parameter created with version {new_secure_parameter['Version']} and Tier {new_secure_parameter['Tier']}"
)

To use the Customer Managed Key from AWS KMS, we can use the KeyId argument and specify the KMS Key ID. The above example uses the AWS Managed Key, set default when using SecureString parameter type.

Here’s an execution output:

Reading Parameter Store values using Boto3

To get information about one or more parameters, we can use the get_parameter() and get_parameters() methods of the SSM client. The response of these methods will be a JSON object. We will use the JSON module and json_datetime_serializer() helper method to read it.

Reading Parameter Store StringList Parameter single parameter

We will use the simple get_parameter() method to fetch the information of a single parameter.

import json
from datetime import date, datetime
import boto3

# Helper method to serialize datetime fields
def json_datetime_serializer(obj):
    if isinstance(obj, (datetime, date)):
        return obj.isoformat()
    raise TypeError("Type %s not serializable" % type(obj))

AWS_REGION = "us-east-1"
ssm_client = boto3.client("ssm", region_name=AWS_REGION)
get_response = ssm_client.get_parameter(Name='/dev/ec2Instance/tags',
                                        WithDecryption=True)
print('Parameter information:')
print(
    json.dumps(get_response['Parameter'],
               indent=4,
               default=json_datetime_serializer))

The required argument is:

• Name – specifies the fully qualified name of the parameter. For example, /dev/ec2Instance/type

Optional Argument:

• WithDecryption – Returns decrypted values for SecureString parameters. This flag is ignored for the String and StringList parameters types.

Execution output:

Reading Parameter Store multiple parameters

To read multiple parameters, we will use the get_parameters() method.

import json
from datetime import date, datetime
import boto3

# Helper method to serialize datetime fields
def json_datetime_serializer(obj):
    if isinstance(obj, (datetime, date)):
        return obj.isoformat()
    raise TypeError("Type %s not serializable" % type(obj))

AWS_REGION = "us-east-1"
ssm_client = boto3.client("ssm", region_name=AWS_REGION)
get_response = ssm_client.get_parameters(Names=[
    '/dev/ec2Instance/tags', '/dev/ec2Instance/type',
    '/dev/ec2Instance/sshPassword'
],
    WithDecryption=True)
print('Parameters information:')
print(
    json.dumps(get_response['Parameters'],
               indent=4,
               default=json_datetime_serializer))

The required argument is:

• Names – specifies a list of the fully qualified names of the parameters.

Optional Argument:

• WithDecryption – Returns decrypted values for SecureString parameters. This flag is ignored for the String and StringList parameters types.

Describing Parameter Store parameter values using Boto3

The most accessible way to describe the parameters is to use the describe_parameters() method of the SSM client because it provides complete information about the parameter in JSON format.

The describe_parameters() method allows you to specify multiple conditions to find the required parameters. Let’s use Type to find all string parameters, for example:

import json
from datetime import date, datetime
import boto3

# Helper method to serialize datetime fields
def json_datetime_serializer(obj):
    if isinstance(obj, (datetime, date)):
        return obj.isoformat()
    raise TypeError("Type %s not serializable" % type(obj))

AWS_REGION = "us-east-1"
ssm_client = boto3.client("ssm", region_name=AWS_REGION)
describe_response = ssm_client.describe_parameters(Filters=[
    {
        'Key': 'Type',
        'Values': [
            'String',
        ]
    },
])
print('Parameters information:')
print(
    json.dumps(describe_response['Parameters'],
               indent=4,
               default=json_datetime_serializer))

In the above example, we’re using additional json_datetime_serializer() method to serialize (convert to string) datetime.datetime fields returned by the describe_parameters() method.

Here’s an example output:

Listing Parameter Store values using Boto3

To get a complete list of parameters and values, we will use the get_parameter() and describe_parameters() methods explained in the above sections.

Also, we will use the Boto3 paginator feature to get the complete output from the describe_parameters() method.

Some AWS requests return incomplete output, therefore, requires subsequent requests to get the complete result. The process of sending subsequent requests to continue where a previous request left off is called pagination.

import boto3
AWS_REGION = "us-east-1"
ssm_client = boto3.client("ssm", region_name=AWS_REGION)
# creating paginator object for describe_parameters() method
paginator = ssm_client.get_paginator('describe_parameters')
# creating a PageIterator from the paginator
page_iterator = paginator.paginate().build_full_result()
# loop through each page from page_iterator
for page in page_iterator['Parameters']:
    response = ssm_client.get_parameter(
        Name=page['Name'], 
        WithDecryption=True
    )
    
    value = response['Parameter']['Value']
    
    print("Parameter Name is: " + page['Name'] + " and Value is: " + value)

In the above example:

get_paginator() – specifies the paginator object for the describe_parameters() method to get all the available parameters and details without sending subsequent requests.

paginate().build_full_result() – iterates over the pages of output from the describe_parameters() method and creates page iterator object,

The for-each loop will iterate through each page from the page_iterator object and get the parameter name.

We will use the get_parameter() method and pass the parameter name as input to get the parameter value.

Here’s an execution output:

Labeling Parameter version

A parameter label allows users to create different alias for the different versions of the parameter. When a parameter value is modified, a new version of the parameter is created by AWS and set as default. A parameter label allows us to remember the purpose of the modification easily.

However, there are a few requirements and restrictions while creating parameter labels.

• A version of a parameter can have a maximum of 10 labels.
• Same label can’t be attached to different versions of the same parameter.
• You can’t create a label when you create a new parameter. You must attach a label to a specific version of a parameter.
• Labels can’t begin with a number, “aws ” or “ssm ” (not case sensitive).

import boto3
AWS_REGION = "us-east-1"
ssm_client = boto3.client("ssm", region_name=AWS_REGION)
label_response = ssm_client.label_parameter_version(
    Name='/dev/ec2Instance/tags',
    ParameterVersion=1,
    Labels=['aws-dev-Instance-Tags'])
if not label_response['InvalidLabels']:
    print('Labels created on the parameter.')
else:
    print('Error while creating lables. Enter valid labels.')
    print('Invalid Labels - ', label_response['InvalidLabels'])

label_parameter_version() – method takes the parameter name, version, and a list of labels as input and creates the specified label on the given version of the parameter.

In the above example, we are parsing that response and verifying if Invalid Labels are produced. If no invalid label is returned as a response, the label_parameter_version() method will apply the specified labels to the specific parameter version.

The required arguments are:

• Name – specifies the name of the parameter
• Labels – specifies a list of valid labels to apply of parameter version.

Optional argument:

• ParameterVersion – if no specific version is given, labels will be applied to the latest version of the parameter.

Execution output:

Unlabeling Parameter version

To remove a label or labels from a parameter, we will use the unlabel_parameter_version() method from the SSM client.

import boto3
AWS_REGION = "us-east-1"
ssm_client = boto3.client("ssm", region_name=AWS_REGION)
unlabel_response = ssm_client.unlabel_parameter_version(
    Name='/dev/ec2Instance/tags',
    ParameterVersion=1,
    Labels=['dev-Instance-Tags'])
if unlabel_response['RemovedLabels']:
    print('Labels removed - ', unlabel_response['RemovedLabels'])
if unlabel_response['InvalidLabels']:
    print('Error while removing lables. Enter valid labels.')
    print('Invalid Labels - ', unlabel_response['InvalidLabels'])

The required arguments are:

• Name – specifies the name of the parameter
• Labels – specifies a list of valid labels or labels to be removed from the parameter version.
• ParameterVersion – labels will be removed to the specified version of the parameter. If not set, the request will fail.

Here is an execution output:

Deleting Parameter Store values using Boto3

Delete single parameter

To delete a parameter from the AWS SSM Parameter store, we will use the delete_parameter() method from the SSM client.

import boto3
AWS_REGION = "us-east-1"
ssm_client = boto3.client("ssm", region_name=AWS_REGION)
delete_response = ssm_client.delete_parameter(Name='/dev/ec2Instance/tags')
print(delete_response)

The required argument is:

• Name – specifies the name of the parameter to delete.

This method does not return any output on successful parameter deletion.

The output shown in the below code execution is the response metadata for the Boto3 request.

Delete multiple parameters

For deletion of multiple parameters from the SSM Parameter store, we will be using the delete_parameters() method.

import boto3
AWS_REGION = "us-east-1"
ssm_client = boto3.client("ssm", region_name=AWS_REGION)
delete_response = ssm_client.delete_parameters(
    Names=['/dev/ec2Instance/type', '/dev/ec2Instance/sshPassword'])
if delete_response['DeletedParameters']:
    print('Parameters deleted - ', delete_response['DeletedParameters'])
if delete_response['InvalidParameters']:
    print('Error while deleting parameters. Enter valid parameters.')
    print('Invalid Parameters - ', delete_response['InvalidParameters'])

The required argument is:

• Names – specifies a list of names of the parameters to delete.

Code Execution output:

Summary

In conclusion, we have explained how to make basic AWS Systems Manager Parameter Store API calls using the Boto3 library in this article.