core-concepts.md

Browse the Repo

.circleci

.github

_docs

examples

modules

aurora

backup-plan

backup-vault

efs

lambda-cleanup-snapshots

lambda-copy-shared-snapshot

lambda-create-snapshot

create-rds-snapshot

CHANGELOG.md

README.adoc

core-concepts.md

main.tf

outputs.tf

variables.tf

lambda-share-snapshot

rds-proxy

rds-replicas

rds

redshift

test

.gitignore

.pre-commit-config.yaml

CODEOWNERS

LICENSE.txt

README.adoc

terraform-cloud-enterprise-private-module-...

Browse the Repo

.circleci

.github

_docs

examples

modules

aurora

backup-plan

backup-vault

efs

lambda-cleanup-snapshots

lambda-copy-shared-snapshot

lambda-create-snapshot

create-rds-snapshot

CHANGELOG.md

README.adoc

core-concepts.md

main.tf

outputs.tf

variables.tf

lambda-share-snapshot

rds-proxy

rds-replicas

rds

redshift

test

.gitignore

.pre-commit-config.yaml

CODEOWNERS

LICENSE.txt

README.adoc

terraform-cloud-enterprise-private-module-...

Database backup

Snapshot your RDS databases and copy the snapshots to other AWS accounts on a scheduled basis for disaster recovery.

Data backup core concepts

How does this differ from RDS automatic snapshots?

Note that RDS comes with nightly snapshots by default. The main reason to use this function is:

You want to take snapshots of your database more often than once per night.
You want to store all of your snapshots in a separate AWS account for security and redundancy purposes.
You want to retain backups for longer than the 35-day limit for automatic snapshots.

How do you backup your RDS snapshots to a separate AWS account?

One of the main use cases for this module is to be able to store your RDS snapshots in a completely separate AWS account. That reduces the chances that you, or perhaps an intruder who breaks into your AWS account, can accidentally or intentionally delete all your snapshots.

Let's say you have an RDS database in account A and you want to store snapshots in account B. To set that up, you need to do the following:

Create a KMS key and grant permission access to account B . Your key configuration should look something like this:

Note: it is not possible to use the default AWS-managed KMS key because you cannot change the key policy to grant permission to account B to access this key. The KMS key is required to copy the snapshot over to account B.
Note: we also have a KMS module to define a new CMK for this purpose. Please refer to this module for more information

"Statement": [
		...
    {
        "Sid": "Allow use of the key",
        "Effect": "Allow",
        "Principal": {
            "AWS": "arn:aws:iam::<account_B>:root"
        },
        "Action": [
            "kms:Encrypt",
            "kms:Decrypt",
            "kms:ReEncrypt*",
            "kms:DescribeKey"
        ],
        "Resource": "*"
    }
		...
]

Deploy the following lambda functions in account A to i) create RDS snapshot and ii) share it with account B
- lambda-create-snapshot : lambda function to periodically create RDS snapshot with your desired schedule and invoke lambda-share-snapshot lambda function to share the snapshot with account B
- lambda-share-snapshot : lambda function to share a RDS snapshot with another account (e.g., account B). Note that you will not see the RDS snapshot in account B because the snapshot only exists in account A. This lambda function simply changes the attribute of the snapshot so that the account B can see the snapshot.
Note: look at examples/lambda-rds-snapshot as reference to use the lambda modules along with your RDS cluster.

Create a cross-account IAM policy in account A to allow account B to access the KMS key. Your cross-account IAM policy should look something like this:

Trust Entity:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::<account_B>:root"
            },
            "Action": "sts:AssumeRole",
            "Condition": {}
        }
    ]
}

Permission Policy:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "VisualEditor0",
            "Effect": "Allow",
		        "Action": [
		            "kms:Encrypt",
		            "kms:Decrypt",
		            "kms:ReEncrypt*",
		            "kms:DescribeKey"
		        ],
            "Resource": "arn:aws:kms:<region>:<account_A>:key/<kms_key>"
        }
    ]
}

Deploy the lambda-copy-shared-snapshot lambda function in account B to copy the shared snapshot. You need the following informations to use this lambda function:
- rds_db_identifier - RDS identifier of the shared snapshot.
- external_account_id - original account that shared the snapshot.
- kms_key_id: KMS key used to encrypt the RDS snapshot.

Note: before running this lambda function, you need to grant permission explicitly for the lambda function to access the KMS key in account A by attaching a policy something like this:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "VisualEditor0",
            "Effect": "Allow",
		        "Action": [
		            "kms:Encrypt",
		            "kms:Decrypt",
		            "kms:ReEncrypt*",
		            "kms:DescribeKey"
		        ],
            "Resource": "arn:aws:kms:<region>:<account A>:key/<KMS key>"
        }
    ]
}

Why use lambda functions?

The reason we use lambda functions for handling snapshots is:

It's easy to use scheduled events and schedule expressions to run a lambda function on a periodic basis that is more reliable than just using cron.
You can give your lambda function access to RDS via IAM roles instead of using API keys with an external app.
The main use case for these lambda snapshot modules is to copy RDS snapshots to an external AWS account. That means you need to run code in multiple accounts. It's easier to deploy the necessary lambda functions in each account and give those functions access to RDS via IAM roles than it is to create a CI job that can securely access both accounts.

How do you configure this module?

This module allows you to configure a number of parameters, such as which database to backup, how often to run the backups, what account to share the backups with, and more. For a list of all available variables and their descriptions, see variables.tf.

How do you configure multiple backup schedules?

You can use this module multiple times by configuring different namespaces for the snapshots, which allows you to have multiple backup schedules with different retention periods. For example you could keep hourly backups for three days, and weekly backups for one year by configuring two instances of this modules.

module "create_daily_snapshot" {
    source = "git::git@github.com:gruntwork-io/terraform-aws-data-storage.git//modules/lambda-create-snapshot?ref=v1.0.8"

    # ... (other params omitted) ...

    lambda_namespace    = "${var.rds_db_identifier}-create-weekly-snapshot"
    snapshot_namespace  = "daily"
    schedule_expression = "rate(1 day)"
}

module "create_weekly_snapshot" {
    source = "git::git@github.com:gruntwork-io/terraform-aws-data-storage.git//modules/lambda-create-snapshot?ref=v1.0.8"

    # ... (other params omitted) ...
    lambda_namespace    = "${var.rds_db_identifier}-create-weekly-snapshot"
    snapshot_namespace  = "weekly"
    schedule_expression = "rate(1 week)"
}

Configure sharing in the same way as described earlier. Only the snapshots from the module with sharing enabled will be copied.

It's important to use both snapshot and lambda namespaces in all instances to avoid ambiguity for the lambda-cleanup-snapshots module. The lambda-cleanup-snapshots module can be configured with a snapshot_namespace too so different retention periods can be configured for each set of snapshots. See the lambda-rds-snapshot-multiple-schedules example.

Questions? Ask away.

We're here to talk about our services, answer any questions, give advice, or just to chat.

Ready to hand off the Gruntwork?

Buy Now

"https://cdn.gruntwork.io/gruntwork-website/"

{"index":{"js":"https://cdn.gruntwork.io/gruntwork-website/index.bundle.c7884255553b53fbca3a.js","map":"https://cdn.gruntwork.io/gruntwork-website/index.bundle.1b14c1b7d19f1f5eb35d6e118e838255.map"},"styles":{"css":"https://cdn.gruntwork.io/gruntwork-website/styles.bundle.f22938926651ddec7c49.css","js":"https://cdn.gruntwork.io/gruntwork-website/styles.bundle.e782420e74a20dcb8691.js","map":"https://cdn.gruntwork.io/gruntwork-website/styles.bundle.d5e2af49807c6ca33f8367d621ece507.map"},"vendors":{"css":"https://cdn.gruntwork.io/gruntwork-website/vendors.bundle.29f7d0366a0978763f96.css","js":"https://cdn.gruntwork.io/gruntwork-website/vendors.bundle.fa8174a130797d75d12c.js","map":"https://cdn.gruntwork.io/gruntwork-website/vendors.bundle.57243d94deeeb29d5061288a338b4eb6.map"}}

Database backup

Code Preview

Preview the Code

Data backup core concepts

How does this differ from RDS automatic snapshots?

How do you backup your RDS snapshots to a separate AWS account?

Why use lambda functions?

How do you configure this module?

How do you configure multiple backup schedules?

Questions? Ask away.

Ready to hand off the Gruntwork?