How automated deployment works with DynamoDB global tables
Automating DynamoDB deployment in AWS platforms
In May of 2021 AWS released CloudFormation support for DynamoDB global tables. This much-anticipated feature is a boon for those looking to automate DynamoDB deployments.
DynamoDB has many great features that make it one of the fastest growing datastores on the AWS platform. One of the premier features of DynamoDB are global tables, which allow you to replicate your data across regions seamlessly. I’ve written about the many benefits of DynamoDB and global tables on this blog before. However a key DynamoDB feature was missing until now: the automated deployment of global tables.
The use of DynamoDB with CloudFormation
If you are not familiar with DynamoDB global tables, it is a feature that allows you to replicate your DynamoDB database to other regions in real time. Global tables provides several benefits:
- Data can be accessed in the local region, increasing performance.
- The active-active nature of global tables allows updates in each region which are automatically replicated to the others in real time.
- Cross-region fault tolerance is gained as each region has a copy of the data increasing the availability and durability of the data store.
CloudFormation allows you to provision AWS resources through the use of template files, providing infrastructure-as-code capabilities. While this article focuses specifically on DynamoDB, CloudFormation allows you to automate the creation of most AWS resources.
While you can read the CloudFormation documentation yourself, there are some important concepts to keep in mind. Global tables are treated as top level components equivalent to, but separate from, a standard DynamoDB table. The global table configuration shares many of the same elements as DynamoDB tables, but some of the elements are configured inside the replica, on a per-region basis, such as tags. In the CloudFormation Template (CFT) each replica is configured as a peer with each table defined in a list. This is conceptually different from setting up DynamoDB tables in the AWS console where you typically create an initial table and then add the global tables in other regions from the initial table.
What to consider when creating the CloudFormation template
There are a few restrictions and idiosyncrasies to be aware of when creating the CloudFormation template for DynamoDB global tables.
- Only Version 2019.11.21 of global tables is supported in CloudFormation, version 2017.11.29 is not currently supported.
- If you try to create a global table with the same name as an existing table in the same region, the existing table may be deleted.
- At least one replica must exist in the same region in which you deploy the CFT stack. For instance, if you are deploying your CFT stack from us-east-1, at least one of the replicas must be in us-east-1 as well.
What to consider when configuring global tables
When configuring your global tables, some properties have been split between the main configuration and the individual replicas.
- When using server-side encryption, the properties SSEEnabled and SSEType are specified in the main portion of the template, but the KMSMasterKeyId is specified under each replica. This makes sense when you consider that KMS keys are region specific and may have different names in each region. Though for clarity's sake it makes sense to stick with a common naming conversion between regions.
- When you initially create your CFT, you can only specify two replicas. If you need more than two, you need to update your CFT but only one replica can be added or removed with each update.
Example DynamoDB global table
The following template creates a DynamoDB global table named mytable in both us-east-1 and us-west-1.
AWSTemplateFormatVersion: 2010-09-09
Description: AWS CloudFormation Template to create global tables
Parameters: {}
Resources:
globalTableExample:
Type: 'AWS::DynamoDB::GlobalTable'
Properties:
TableName: mytable
AttributeDefinitions:
- AttributeName: PK
AttributeType: S
KeySchema:
- AttributeName: PK
KeyType: HASH
BillingMode: PAY_PER_REQUEST
StreamSpecification:
StreamViewType: NEW_AND_OLD_IMAGES
SSESpecification:
SSEEnabled: true
SSEType: "KMS"
Replicas:
- Region: us-east-1
PointInTimeRecoverySpecification:
PointInTimeRecoveryEnabled: true
SSESpecification:
KMSMasterKeyId: alias/dynamodb-key-east
Tags:
- Key: Name
Value: mytable
- Key: Region
Value: east
- Region: us-west-1
PointInTimeRecoverySpecification:
PointInTimeRecoveryEnabled: true
SSESpecification:
KMSMasterKeyId: alias/dynamodb-key-west
Tags:
- Key: Name
Value: mytable
- Key: Region
Value: west
As I mentioned previously, some properties are common to global tables, including that some are region specific, and a few where the full definition is spread between the common properties and regional specific properties.
- The table name is common across all regions, this is the case whether it is created through CloudFormation, the AWS console, or some other method.
- Tags are region specific and declared under each replica's definition.
- The SSESpecifiction is partially in the main portion of the template and partially in each replica.
It’s important to read and understand the template definition because placing an element in the wrong section will cause CloudFormation to fail. I made this mistake myself when converting my test templates from regular DynamoDB tables to global tables.
Start your table creation process with DynamoDB and CloudFormation
If you are new to DynamoDB and want to automate your table creation process, CloudFormation is a great tool to consider using. If you’ve been using DynamoDB for a while like I have, this is a welcome addition and finally allows automation of a process that previously required manual steps or custom scripting. More automation is always better in my personal opinion!
