Working with Parameter Store in Python

Abhinav Dumbre

Abhinav Dumbre

0
(0)

AWS Systems Manager (AWS SSM) is a collection of services for configuring and managing your AWS and on-premises resources at any scale. You can use AWS SSM to automate operational tasks for Amazon EC2 instances, Amazon RDS instances, on-premises servers, grouping resources by applications, auditing changes in the resources, storing configuration and data secrets, and various other tasks. Though AWS Systems Manager is an umbrella for many different services, in this article, we will cover the AWS SSM Parameter Store and how to interact with it programmatically using Python and Boto3 library.

What is AWS Systems Manager Parameter Store?

The Parameter Store is a capability of AWS Systems Manager which helps you store your application configuration data in a systematic hierarchical format for better reference and management. Data can be of any type like API keys, passwords, URLs, Amazon Machine Image (AMI) IDs, and licenses, database connections, or any other kind of strings. You can store data in encrypted or plain text format. Parameter Store stores all parameters in key-value format. In addition to this, it is integrated with AWS Key Management Service (AWS KMS), which provides the ability to encrypt the stored parameters securely.

The Parameter Store is an excellent service to store your application’s secrets and configurations across various environments without managing application configuration files. The Parameter Store has some advantages over other methods of managing variables:

Introduction to AWS Systems Manager - Features

For more information about the Parameter Store, we recommend checking out the Introduction to AWS Systems Manager article.

Parameter Store Types

A parameter is a key-value pair stored within the AWS SSM Parameter Store. There are three primary types of Parameter Store parameters with an additional fourth sub-type parameter that can reference a valid Amazon Machine Image ID (AMI):

  • String – unencrypted text value, for example, Hello World, hands-on.cloud, 10.0.0.1/24, etc.
  • StringList – a collection of comma-separated strings, for example, EC2, S3, EBS Volumes, RDS Database or Python, Golang, C++.
  • SecureString – text value encrypted by the AWS KMS key.
  • aws:ec2:image – when you’re creating a String parameter, you can specify this data type to ensure that the parameter value you enter is a valid Amazon Machine Image (AMI) ID.

Connect to AWS SSM Parameter Store using Boto3

To connect to AWS SSM Parameter Store using Boto3, you can use only SSM client. The client that allows you to access the low-level SSM API data. For example, you can get access to API response data in JSON format:

import boto3

AWS_REGION = "us-east-1"

ssm_client = boto3.client("ssm", region_name=AWS_REGION)

Write AWS SSM Parameter Store values using Boto3

AWS SSM Parameters provides a secure and centralized place to manage various applications’ configuration states and secrets. These configurations and secrets are stored as parameters in AWS SSM Parameter Store. To create a new parameter programmatically using Python, you need to use the put_parameter() method of the SSM client.

Create an AWS SSM Parameter Store String Parameter

import boto3

AWS_REGION = "us-east-1"

ssm_client = boto3.client("ssm", region_name=AWS_REGION)

new_string_parameter = ssm_client.put_parameter(
    Name='/dev/ec2Instance/type',
    Description='EC2 Instance type for Dev environment',
    Value='t3.medium',
    Type='String',
    Overwrite=True,
    Tier='Standard',
    DataType='text')

print(
    f"String Parameter created with version {new_string_parameter['Version']} and Tier {new_string_parameter['Tier']}"
)

The required arguments are:

  • Name – specifies the fully qualified name of the parameter. The fully qualified name usually includes the complete hierarchy of the parameter path and name. For example. /dev/ec2Instance/type, which means EC2 instance type in development environment
  • Value – specifies the parameter value. Standard parameters have a value limit of 4 KB, whereas Advanced parameters have a value limit of 8 KB.

The rest of the arguments are optional, but we set them up for demo purposes:

  • Description – specifies the additional information about the parameter
  • Type – specifies the Parameter type. As explained in previously, the parameter can be of the following three types
    • String
    • StringList
    • SecureString
  • Overwrite – used to overwrite a current parameter value. The default value is False.
  • Tier – the parameter tier to assign to a parameter. For more information, visit here.
  • DataType – the data type for a String parameter. The following data type values are supported.
    • text
    • aws:ec2:image

Here’s an execution output:

Creating a AWS SSM Parameter Store String Parameter

Create an AWS SSM Parameter Store StringList Parameter

To create a parameter with the StringList type, we need to change the Type argument from the previous block of code for the String type parameter.

TypeStringList, gives the ability to create parameters with comma-separated values.

import boto3

AWS_REGION = "us-east-1"

ssm_client = boto3.client("ssm", region_name=AWS_REGION)

new_list_parameter = ssm_client.put_parameter(
    Name='/dev/ec2Instance/tags',
    Description='EC2 Instance tags for Dev environment',
    Value='dev,staging,pre-production',
    Type='StringList',
    Overwrite=True,
    Tier='Standard',
    DataType='text'
)

print(
    f"StringList Parameter created with version {new_list_parameter['Version']} and Tier {new_list_parameter['Tier']}"
)

Here’s an execution output:

Create a SecureString Parameter

To store parameters such as Passwords, API Keys, Database connections, etc., securely and in an encrypted format, AWS SSM Parameter Store integrates with AWS KMS. To store encrypted parameters, we can use the SecureString type.

TypeSecureString encrypts the parameter value using the AWS Managed Key from AWS KMS.

import boto3

AWS_REGION = "us-east-1"

ssm_client = boto3.client("ssm", region_name=AWS_REGION)

new_secure_parameter = ssm_client.put_parameter(
    Name='/dev/ec2Instance/sshPassword',
    Description='EC2 Instance SSH Password for Dev environment',
    Value='HighlySecurePassword',
    Type='SecureString',
    # KeyId='<customer-managed-kms-key-id>',
    Overwrite=True,
    Tier='Standard',
    DataType='text')

print(
    f"SecureString Parameter created with version {new_secure_parameter['Version']} and Tier {new_secure_parameter['Tier']}"
)

To use the Customer Managed Key from AWS KMS, we can use the KeyId argument and specify the KMS Key ID. The above example uses the AWS Managed Key, set default when using SecureString parameter type.

Here’s an execution output:

Creating a SecureString Parameter

Read AWS SSM Parameter Store values using Boto3

To get information about one or more parameters, we can use get_parameter() and get_parameters() methods of the SSM client. The response of these methods will be a JSON object. We will use the JSON module and json_datetime_serializer() helper method to read it.

Read an AWS SSM Parameter Store StringList Parameter single parameter

To fetch the information of a single parameter, we will use the simple get_parameter() method.

import json
from datetime import date, datetime
import boto3


# Helper method to serialize datetime fields
def json_datetime_serializer(obj):
    if isinstance(obj, (datetime, date)):
        return obj.isoformat()
    raise TypeError("Type %s not serializable" % type(obj))


AWS_REGION = "us-east-1"

ssm_client = boto3.client("ssm", region_name=AWS_REGION)

get_response = ssm_client.get_parameter(Name='/dev/ec2Instance/tags',
                                        WithDecryption=True)

print('Parameter information:')
print(
    json.dumps(get_response['Parameter'],
               indent=4,
               default=json_datetime_serializer))

The required argument is:

  • Name – specifies the fully qualified name of the parameter. For example, /dev/ec2Instance/type

Optional Argument:

  • WithDecryption – Returns decrypted values for SecureString parameters. This flag is ignored for the String and StringList parameters types.

Execution output:

Reading an AWS SSM Parameter Store StringList Parameter single parameter

Read an AWS SSM Parameter Store multiple parameters

To read multiple parameters, we will use the get_parameters() method.

import json
from datetime import date, datetime
import boto3


# Helper method to serialize datetime fields
def json_datetime_serializer(obj):
    if isinstance(obj, (datetime, date)):
        return obj.isoformat()
    raise TypeError("Type %s not serializable" % type(obj))


AWS_REGION = "us-east-1"

ssm_client = boto3.client("ssm", region_name=AWS_REGION)

get_response = ssm_client.get_parameters(Names=[
    '/dev/ec2Instance/tags', '/dev/ec2Instance/type',
    '/dev/ec2Instance/sshPassword'
],
    WithDecryption=True)

print('Parameters information:')
print(
    json.dumps(get_response['Parameters'],
               indent=4,
               default=json_datetime_serializer))

The required argument is:

  • Names – specifies a list of the fully qualified names of the parameters.

Optional Argument:

  • WithDecryption – Returns decrypted values for SecureString parameters. This flag is ignored for the String and StringList parameters types.
Reading an AWS SSM Parameter Store multiple parameters

Describe AWS SSM Parameter Store parameter values using Boto3

The most accessible way to describe the parameters is to use the describe_parameters() method of the SSM client because it provides complete information about the parameter in JSON format.

The describe_parameters() method allows you to specify multiple conditions to find the required parameters. Let’s use Type to find all string parameters, for example:

import json
from datetime import date, datetime
import boto3


# Helper method to serialize datetime fields
def json_datetime_serializer(obj):
    if isinstance(obj, (datetime, date)):
        return obj.isoformat()
    raise TypeError("Type %s not serializable" % type(obj))


AWS_REGION = "us-east-1"

ssm_client = boto3.client("ssm", region_name=AWS_REGION)

describe_response = ssm_client.describe_parameters(Filters=[
    {
        'Key': 'Type',
        'Values': [
            'String',
        ]
    },
])

print('Parameters information:')
print(
    json.dumps(describe_response['Parameters'],
               indent=4,
               default=json_datetime_serializer))

In the above example, we’re using additional json_datetime_serializer() method to serialize (convert to string) datetime.datetime fields returned by the describe_parameters() method.

Here’s an example output:

Describing AWS SSM Parameter Store parameter values

List AWS SSM Parameter Store values using Boto3

To get a complete list of parameters and values, we will use the get_parameter() and describe_parameters() methods explained in the above sections.

Also, we will use the Boto3 library paginator feature to get the complete output from the describe_parameters() method.

Some AWS requests return incomplete output, therefore, requires subsequent requests to get the complete result. The process of sending subsequent requests to continue where a previous request left off is called pagination.

import boto3

AWS_REGION = "us-east-1"

ssm_client = boto3.client("ssm", region_name=AWS_REGION)

# creating paginator object for describe_parameters() method
paginator = ssm_client.get_paginator('describe_parameters')

# creating a PageIterator from the paginator
page_iterator = paginator.paginate().build_full_result()

# loop through each page from page_iterator
for page in page_iterator['Parameters']:
    response = ssm_client.get_parameter(
        Name=page['Name'], 
        WithDecryption=True
    )
    
    value = response['Parameter']['Value']
    
    print("Parameter Name is: " + page['Name'] + " and Value is: " + value)

In the above example:

get_paginator() – specifies the paginator object for the describe_parameters() method to get all the available parameters and details without sending subsequent requests.

paginate().build_full_result() – iterates over the pages of output from the describe_parameters() method and creates page iterator object,

The for-each loop will iterate through each page from the page_iterator object and get the parameter name.

We will use the get_parameter() method and pass the parameter name as input to get the parameter value.

Here’s an execution output:

Listing AWS SSM Parameter Store values

Label AWS SSM Parameter version

A parameter label allows users to create different alias for the different versions of the parameter. When a parameter value is modified, a new version of the parameter is created by AWS and set as default. A parameter label allows us to remember the purpose of the modification easily.

However, there are a few requirements and restrictions while creating parameter labels.

  • A version of a parameter can have a maximum of 10 labels.
  • Same label can’t be attached to different versions of the same parameter.
  • You can’t create a label when you create a new parameter. You must attach a label to a specific version of a parameter.
  • Labels can’t begin with a number, “aws ” or “ssm ” (not case sensitive).
import boto3

AWS_REGION = "us-east-1"

ssm_client = boto3.client("ssm", region_name=AWS_REGION)

label_response = ssm_client.label_parameter_version(
    Name='/dev/ec2Instance/tags',
    ParameterVersion=1,
    Labels=['aws-dev-Instance-Tags'])

if not label_response['InvalidLabels']:
    print('Labels created on the parameter.')
else:
    print('Error while creating lables. Enter valid labels.')
    print('Invalid Labels - ', label_response['InvalidLabels'])

label_parameter_version() – method takes the parameter name, version, and a list of labels as input and creates the specified label on the given version of the parameter.

In the above example, we are parsing that response and verifying if Invalid Labels are produced. If no invalid label is returned as a response, the label_parameter_version() method will apply the specified labels to the specific parameter version.

The required arguments are:

  • Name – specifies the name of the parameter
  • Labels – specifies a list of valid labels to apply of parameter version.

Optional argument:

  • ParameterVersion – if no specific version is given, labels will be applied to the latest version of the parameter.

Execution output:

Labeling AWS SSM Parameter version

Unlabel AWS SSM Parameter version

To remove a label or labels from a parameter, we will use the unlabel_parameter_version() method from the SSM client.

import boto3

AWS_REGION = "us-east-1"

ssm_client = boto3.client("ssm", region_name=AWS_REGION)

unlabel_response = ssm_client.unlabel_parameter_version(
    Name='/dev/ec2Instance/tags',
    ParameterVersion=1,
    Labels=['dev-Instance-Tags'])

if unlabel_response['RemovedLabels']:
    print('Labels removed - ', unlabel_response['RemovedLabels'])

if unlabel_response['InvalidLabels']:
    print('Error while removing lables. Enter valid labels.')
    print('Invalid Labels - ', unlabel_response['InvalidLabels'])

The required arguments are:

  • Name – specifies the name of the parameter
  • Labels – specifies a list of valid labels or labels to be removed from the parameter version.
  • ParameterVersion – labels will be removed to the specified version of the parameter. If not set, the request will fail.

Here is an execution output:

Unlabeling AWS SSM Parameter version

Delete AWS SSM Parameter Store values using Boto3

Delete single parameter

To delete a parameter from the AWS SSM Parameter store, we will use the delete_parameter() method from the SSM client.

import boto3

AWS_REGION = "us-east-1"

ssm_client = boto3.client("ssm", region_name=AWS_REGION)

delete_response = ssm_client.delete_parameter(Name='/dev/ec2Instance/tags')

print(delete_response)

The required argument is:

  • Name – specifies the name of the parameter to delete.

This method does not return any output on successful parameter deletion.

The output shown in the below code execution is the response metadata for the Boto3 request.

Deleting single parameter

Delete multiple parameters

For deletion of multiple parameters from the SSM Parameter store, we will be using the delete_parameters() method.

import boto3

AWS_REGION = "us-east-1"

ssm_client = boto3.client("ssm", region_name=AWS_REGION)

delete_response = ssm_client.delete_parameters(
    Names=['/dev/ec2Instance/type', '/dev/ec2Instance/sshPassword'])

if delete_response['DeletedParameters']:
    print('Parameters deleted - ', delete_response['DeletedParameters'])

if delete_response['InvalidParameters']:
    print('Error while deleting parameters. Enter valid parameters.')
    print('Invalid Parameters - ', delete_response['InvalidParameters'])

The required argument is:

  • Names – specifies a list of names of the parameters to delete.

Code Execution output:

Deleting multiple parameters

Summary

In conclusion, we have explained how to make basic AWS Systems Manager Parameter Store API calls using the Boto3 library in this article.

How useful was this post?

Click on a star to rate it!

As you found this post useful...

Follow us on social media!

We are sorry that this post was not useful for you!

Let us improve this post!

Tell us how we can improve this post?

Top rated Udemy Courses to improve you career

Subscribe to our updates

Like this article?

Share on facebook
Share on Facebook
Share on twitter
Share on Twitter
Share on linkedin
Share on Linkdin
Share on pinterest
Share on Pinterest

Want to be an author of another post?

We’re looking for skilled technical authors for our blog!

Leave a comment

If you’d like to ask a question about the code or piece of configuration, feel free to use https://codeshare.io/ or a similar tool as Facebook comments are breaking code formatting.