How does this differ from RDS automatic snapshots?
Note that RDS comes with nightly snapshots by default. The main reason to use this function is:
You want to take snapshots of your database more often than once per night.
You want to store all of your snapshots in a separate AWS account for security and redundancy purposes.
How do you backup your RDS snapshots to a separate AWS account?
One of the main use cases for this module is to be able to store your RDS snapshots in a completely separate AWS account.
That reduces the chances that you, or perhaps an intruder who breaks into your AWS account, can accidentally or
intentionally delete all your snapshots.
Let's say you have an RDS database in account A and you want to store snapshots in account B. To set that up, you need
to do the following:
Deploy this lambda function (lambda-create-snapshot) and the lambda-share-snapshot
lambda function in account A. Configure this lambda function to trigger the
lambda-share-snapshot function by setting the following variables:
module"create_snapshot" {
source = "git::git@github.com:gruntwork-io/module-data-storage.git//modules/data-storage/lambda-create-snapshot?ref=v1.0.8"# ... (other params ommitted) ...
share_snapshot_with_another_account = true
share_snapshot_lambda_arn = "(ARN of the lambda-share-snapshot function)"
share_snapshot_with_account_id = "(The ID of account B)"
}
This will make the snapshots from account A visible in account B, but it won't actually copy them into the
account. To copy them into account B, deploy the lambda-copy-shared-snapshot
module in account B and configure it with the account ID of account A:
module"copy_shared_snapshot" {
source = "git::git@github.com:gruntwork-io/module-data-storage.git//modules/data-storage/lambda-copy-shared-snapshot?ref=v1.0.8"# ... (other params ommitted) ...
rds_db_identifier = "(The identifier of the RDS DB in account A)"
rds_db_account_id = "(The ID of account A)"
}
Why use lambda functions?
The reason we use lambda functions for handling snapshots is:
It's easy to use scheduled events and
schedule expressions
to run a lambda function on a periodic basis that is more reliable than just using cron.
You can give your lambda function access to RDS via IAM roles instead of using API keys with an external app.
The main use case for these lambda snapshot modules is to copy RDS snapshots to an external AWS account. That means
you need to run code in multiple accounts. It's easier to deploy the necessary lambda functions in each account
and give those functions access to RDS via IAM roles than it is to create a CI job that can securely access both
accounts.
How do you configure this module?
This module allows you to configure a number of parameters, such as which database to backup, how often to run the
backups, what account to share the backups with, and more. For a list of all available variables and their
descriptions, see vars.tf.
Questions? Ask away.
We're here to talk about our services, answer any questions, give advice, or just to chat.
{"treedata":{"name":"root","toggled":true,"children":[{"name":".circleci","children":[{"name":"config.yml","path":".circleci/config.yml","sha":"c3cbb3eb78791fbd6c701afb0f06125367ef162c"}]},{"name":".gitignore","path":".gitignore","sha":"13b2bbfb93f58d19fe66d274069b96f32ca5901e"},{"name":".pre-commit-config.yaml","path":".pre-commit-config.yaml","sha":"dd17d910687309b3ad64bae6e967df72e5417357"},{"name":"CODEOWNERS","path":"CODEOWNERS","sha":"d31d238cf5a50d3b077eedad6e7bc1f7404813be"},{"name":"LICENSE.txt","path":"LICENSE.txt","sha":"f4e3d9bd4717a044ed31ad847a300eee74371a78"},{"name":"README.adoc","path":"README.adoc","sha":"3febc0568334062c8b19707fc4dbcb3d6ab5450b"},{"name":"_docs","children":[{"name":"aurora-serverless.png","path":"_docs/aurora-serverless.png","sha":"5a53145be56705c76f7f7aa6a25aa0ddee78e4a3"},{"name":"aurora.png","path":"_docs/aurora.png","sha":"fc218831bfa34097a56f1b0e47fe05521bdb4a8a"},{"name":"data-backup-architecture.png","path":"_docs/data-backup-architecture.png","sha":"fcc7ce8753e28c19af87ea5cea96e6ded648d429"},{"name":"data-backup.png","path":"_docs/data-backup.png","sha":"116b10f231073f8c52255ca98e48cc228c48a2c3"},{"name":"mariadb.png","path":"_docs/mariadb.png","sha":"d540d3d3ff8797c4a4c3a62c65e7d3f63621568f"},{"name":"mysql.png","path":"_docs/mysql.png","sha":"73b55bd0d517dcba53c878712544abf96be3a66e"},{"name":"oracle.png","path":"_docs/oracle.png","sha":"b5f1ca801f5af4a30f1b812eea17cec516c1fe6c"},{"name":"postgresql.png","path":"_docs/postgresql.png","sha":"fd9c7ec282aef38a5813e8542d92227b96bd5be8"},{"name":"rds-architecture.png","path":"_docs/rds-architecture.png","sha":"8f2b1b5b4015a5777032c6aa64627ceee24330fc"},{"name":"sqlserver.png","path":"_docs/sqlserver.png","sha":"a800d188398262593f4f89f27c8f3ce2ce1e76a4"}]},{"name":"examples","children":[{"name":"aurora-global-cluster","children":[{"name":"README.md","path":"examples/aurora-global-cluster/README.md","sha":"897210c9a52ec96f902948e5a0bb10fb460747d0"},{"name":"main.tf","path":"examples/aurora-global-cluster/main.tf","sha":"86f00cb270ff79546ce995fb6c2e204016745d28"},{"name":"outputs.tf","path":"examples/aurora-global-cluster/outputs.tf","sha":"105d706d41684b4c7be092ebd43bf6ba8ebf6f80"},{"name":"vars.tf","path":"examples/aurora-global-cluster/vars.tf","sha":"29b33911054fde667cd778658a7bc63f3a61bedd"}]},{"name":"aurora","children":[{"name":"README.md","path":"examples/aurora/README.md","sha":"b71c1265123bd1bb680589d7436008d259f5e6a1"},{"name":"main.tf","path":"examples/aurora/main.tf","sha":"787286c5e4167d802e5c856a51ba47490f8c1f35"},{"name":"outputs.tf","path":"examples/aurora/outputs.tf","sha":"142569b7d9772c741ee28846de617f5a4b7f0d84"},{"name":"vars.tf","path":"examples/aurora/vars.tf","sha":"f3aa6ab272a9609d375debf73af3a5f49af22bf4"}]},{"name":"lambda-rds-snapshot","children":[{"name":"README.md","path":"examples/lambda-rds-snapshot/README.md","sha":"d7422ca3a039c7e1056c17fc73fbbddfddce6741"},{"name":"main.tf","path":"examples/lambda-rds-snapshot/main.tf","sha":"78e0c6e310230a8f817f29f514f8d2868a7c268e"},{"name":"outputs.tf","path":"examples/lambda-rds-snapshot/outputs.tf","sha":"375c8bb979dd4133c3675d9e6263e1138c448973"},{"name":"vars.tf","path":"examples/lambda-rds-snapshot/vars.tf","sha":"7719435a118a83273f733398c1b579d916a76c9c"}]},{"name":"rds-mariadb","children":[{"name":"README.md","path":"examples/rds-mariadb/README.md","sha":"6540229dddcb38071d37b6697762d27a54e28cc8"},{"name":"main.tf","path":"examples/rds-mariadb/main.tf","sha":"6e057cc1d3dee3f67c075d7094132c4b341bcbe8"},{"name":"outputs.tf","path":"examples/rds-mariadb/outputs.tf","sha":"bd45b0035943021763d0365da3fe0d1e9f9b16b9"},{"name":"vars.tf","path":"examples/rds-mariadb/vars.tf","sha":"6a8212a3c4e8d4bbd1fe6f000bb699b9f5ecf1e0"}]},{"name":"rds-mysql","children":[{"name":"README.md","path":"examples/rds-mysql/README.md","sha":"6a7fb5d5560cc42ec9a6ac830358aa7ab7ca502d"},{"name":"main.tf","path":"examples/rds-mysql/main.tf","sha":"0d75918ebe89cdb708fa459db9c1938a9dc598eb"},{"name":"outputs.tf","path":"examples/rds-mysql/outputs.tf","sha":"e2ae2afdbcc0d27baa5d50be333f9d0a717e1a33"},{"name":"vars.tf","path":"examples/rds-mysql/vars.tf","sha":"a54ce8449d94444f4d0d122e267fe1b3039ac6f8"}]},{"name":"rds-postgres","children":[{"name":"README.md","path":"examples/rds-postgres/README.md","sha":"ab4e18c200fc749a54ee700223d27261c731e436"},{"name":"main.tf","path":"examples/rds-postgres/main.tf","sha":"c0290c5ed183e848900bff33c75816799755019e"},{"name":"outputs.tf","path":"examples/rds-postgres/outputs.tf","sha":"5371c2284c0ea3d4de5790077ba45d9d445c2965"},{"name":"vars.tf","path":"examples/rds-postgres/vars.tf","sha":"ad285f33a53435e2039d5c0c88ba49a214335093"}]},{"name":"rds-sqlserver","children":[{"name":"README.md","path":"examples/rds-sqlserver/README.md","sha":"26bf52d347c247e5f63bc113476e87d75375ba84"},{"name":"main.tf","path":"examples/rds-sqlserver/main.tf","sha":"376fd7e96798604d81c1a8526f204114500ff835"},{"name":"outputs.tf","path":"examples/rds-sqlserver/outputs.tf","sha":"a1726fd73528af38b436c053128f24a17da740c3"},{"name":"vars.tf","path":"examples/rds-sqlserver/vars.tf","sha":"dabccb3da2548b8a9e302ed5e9be01b18c275ca9"}]},{"name":"rds-with-replicas","children":[{"name":"README.md","path":"examples/rds-with-replicas/README.md","sha":"327357a98e9b2bbf650d91a207bd96fc4a6f452f"},{"name":"main.tf","path":"examples/rds-with-replicas/main.tf","sha":"0f1c6e8363260849c031b38e9f696d8b931ea077"},{"name":"outputs.tf","path":"examples/rds-with-replicas/outputs.tf","sha":"991d5436a635194fec1ad1476eb7be6616032c7a"},{"name":"vars.tf","path":"examples/rds-with-replicas/vars.tf","sha":"a1ab0eaadeaee910d4b81265ba28524917926e6c"}]}]},{"name":"modules","children":[{"name":"aurora","children":[{"name":"README-Aurora-Serverless.adoc","path":"modules/aurora/README-Aurora-Serverless.adoc","sha":"7a28bba029bd95affa541548ac19cb5c49a07e27"},{"name":"README-Aurora.adoc","path":"modules/aurora/README-Aurora.adoc","sha":"89ebb7a7c5aa8632a930c34993fc6c4cebe7efb4"},{"name":"README.adoc","path":"modules/aurora/README.adoc","sha":"f1717f1c088760357ec7b87edce843360aef6924"},{"name":"core-concepts.md","path":"modules/aurora/core-concepts.md","sha":"a3bdce9f1bb2a1d8b68c61856073cc01944d0b81"},{"name":"main.tf","path":"modules/aurora/main.tf","sha":"dcf43c8493d5572001687fedca8e73c21dbcdeaa"},{"name":"outputs.tf","path":"modules/aurora/outputs.tf","sha":"60b68848bb307c9b2cb8ed5138c5e4703fbfcffb"},{"name":"vars.tf","path":"modules/aurora/vars.tf","sha":"37596f6a11fab47fe8a288de278597bac0a271a9"}]},{"name":"lambda-cleanup-snapshots","children":[{"name":"README.md","path":"modules/lambda-cleanup-snapshots/README.md","sha":"2c491aeca99fac0536e6c86ef850e00146ac7259"},{"name":"cleanup-rds-snapshots","children":[{"name":"index.py","path":"modules/lambda-cleanup-snapshots/cleanup-rds-snapshots/index.py","sha":"69a65b5022465ddf7e95579ead7a68ab64483b95"}]},{"name":"main.tf","path":"modules/lambda-cleanup-snapshots/main.tf","sha":"953df7fe7e9a29f3090ab293e59357c1a8010e9c"},{"name":"outputs.tf","path":"modules/lambda-cleanup-snapshots/outputs.tf","sha":"a99c0265d859dd0c87a6eba62aaf2b013e224873"},{"name":"vars.tf","path":"modules/lambda-cleanup-snapshots/vars.tf","sha":"5f6b12e00a66aa7a0d5d6717cd73496cbde7a25a"}]},{"name":"lambda-copy-shared-snapshot","children":[{"name":"README.md","path":"modules/lambda-copy-shared-snapshot/README.md","sha":"ec722eef9494e55b2945e0ac194561b4d0e1850c"},{"name":"copy-shared-rds-snapshot","children":[{"name":"index.py","path":"modules/lambda-copy-shared-snapshot/copy-shared-rds-snapshot/index.py","sha":"63194eb6ad0325e8b130d83858aec34d861fbab3"}]},{"name":"main.tf","path":"modules/lambda-copy-shared-snapshot/main.tf","sha":"63f8c01c58edf40b604038e926222be354280f68"},{"name":"outputs.tf","path":"modules/lambda-copy-shared-snapshot/outputs.tf","sha":"f4833d96fa6d47190b9d2c3af243142aefc59d59"},{"name":"vars.tf","path":"modules/lambda-copy-shared-snapshot/vars.tf","sha":"dd4a28dc1ca42d9fcc649f49999156559839c808"}]},{"name":"lambda-create-snapshot","children":[{"name":"README.adoc","path":"modules/lambda-create-snapshot/README.adoc","sha":"4c6a9c7fa49b66aeeeae492b895bb98d97ab3062"},{"name":"core-concepts.md","path":"modules/lambda-create-snapshot/core-concepts.md","sha":"a5cf8e7f3850c1e7b780a7cc0789202e88add463","toggled":true},{"name":"create-rds-snapshot","children":[{"name":"index.py","path":"modules/lambda-create-snapshot/create-rds-snapshot/index.py","sha":"454ef5211ea590dba7e8a891f95ce2bcf4e1d6dd"}]},{"name":"main.tf","path":"modules/lambda-create-snapshot/main.tf","sha":"4435c68aa884342554543bc9a62a9ef549a40c79"},{"name":"outputs.tf","path":"modules/lambda-create-snapshot/outputs.tf","sha":"a0f5ffafa8ef11d00b72f1858b81e182ab2471dd"},{"name":"vars.tf","path":"modules/lambda-create-snapshot/vars.tf","sha":"81d9f4ed54a0361e25ea25318240a5ee5c56f51a"}],"toggled":true},{"name":"lambda-share-snapshot","children":[{"name":"README.md","path":"modules/lambda-share-snapshot/README.md","sha":"f00a0ab9745632e85d5f4c8e7a9389e1a8608b6b"},{"name":"main.tf","path":"modules/lambda-share-snapshot/main.tf","sha":"4431dc6165139d4c2eb9d3d148f44e10a1c69bd2"},{"name":"outputs.tf","path":"modules/lambda-share-snapshot/outputs.tf","sha":"c0d2854f967a6c963662c660d6ae96d8cabe471a"},{"name":"share-rds-snapshot","children":[{"name":"index.py","path":"modules/lambda-share-snapshot/share-rds-snapshot/index.py","sha":"a72d6191cf23e451ff2b8cd3789100827fd782df"}]},{"name":"vars.tf","path":"modules/lambda-share-snapshot/vars.tf","sha":"e90d02ff7f2915dda7d1b1cb6401efb53de4cea8"}]},{"name":"rds","children":[{"name":"README-MariaDb.adoc","path":"modules/rds/README-MariaDb.adoc","sha":"52c88eedb3410b14c6ccc4db8ea3eaa484b7c13a"},{"name":"README-MySQL.adoc","path":"modules/rds/README-MySQL.adoc","sha":"75a2e92b04368988ffe7fb405a99155881f2c4f7"},{"name":"README-Oracle.adoc","path":"modules/rds/README-Oracle.adoc","sha":"a9f084cfbd084413bbcc818fd9f438a4faee367b"},{"name":"README-PostgreSQL.adoc","path":"modules/rds/README-PostgreSQL.adoc","sha":"2486401acaa724eba2f0a8814ef9dfa19c510ae0"},{"name":"README-SqlServer.adoc","path":"modules/rds/README-SqlServer.adoc","sha":"76d7220a727d84567e819102617bd01a3bda0cb2"},{"name":"README.adoc","path":"modules/rds/README.adoc","sha":"fa2c8270106116804fa8ee61a7b5885ab8a4833f"},{"name":"core-concepts.md","path":"modules/rds/core-concepts.md","sha":"11bfb303642008e506c4e822089b284e10626275"},{"name":"main.tf","path":"modules/rds/main.tf","sha":"8e79ee8984a73b80bcb5de656e460d95b5b0275c"},{"name":"outputs.tf","path":"modules/rds/outputs.tf","sha":"f86cf8efcb26ce2338b04502fb192879e2722b5b"},{"name":"vars.tf","path":"modules/rds/vars.tf","sha":"421b9d85f2c9956a309495af9440dd2dfb612f6e"}]}],"toggled":true},{"name":"test","children":[{"name":"Gopkg.lock","path":"test/Gopkg.lock","sha":"c8664629f9a0cbabca39e5ab84530f4764d1d763"},{"name":"Gopkg.toml","path":"test/Gopkg.toml","sha":"b41f04dc6b8d3fd5e03771dd2648a74cb1119f83"},{"name":"README.md","path":"test/README.md","sha":"ebcf2313b7664113168ca7e73d01acbd232d8f39"},{"name":"example_aurora_global_test.go","path":"test/example_aurora_global_test.go","sha":"f45e0eeaf9a7f6646a523cb4d1006bd41509b284"},{"name":"example_aurora_test.go","path":"test/example_aurora_test.go","sha":"43e1e37be5917d20c67f3c407508da33377f56de"},{"name":"example_lambda_rds_snapshot_test.go","path":"test/example_lambda_rds_snapshot_test.go","sha":"3aa5cc64122f7ce0d8edb42976b203de86a14b27"},{"name":"example_rds_mariadb_test.go","path":"test/example_rds_mariadb_test.go","sha":"bb26730828a0ccb71f524647be35e9f93876638b"},{"name":"example_rds_mysql_test.go","path":"test/example_rds_mysql_test.go","sha":"7a08ee04fb8b57790d4dd791ff46b62c452142da"},{"name":"example_rds_postgres_test.go","path":"test/example_rds_postgres_test.go","sha":"4758166580a255344ee07a332e31e6f63e60a880"},{"name":"example_rds_sqlserver_test.go","path":"test/example_rds_sqlserver_test.go","sha":"29a4f9ef76683c024623e44f2f54c2560d2690cf"},{"name":"example_rds_with_replicas_test.go","path":"test/example_rds_with_replicas_test.go","sha":"5744e4b8454f538720e472069f12ef7e1ee0195d"},{"name":"util.go","path":"test/util.go","sha":"12abab86511af62a50eebe68c0a329848e9543c2"}]}]},"detailsContent":"<h1 class=\"preview__body--title\" id=\"data-backup-core-concepts\">Data backup core concepts</h1><div class=\"preview__body--border\"></div><h2 class=\"preview__body--subtitle\" id=\"how-does-this-differ-from-rds-automatic-snapshots\">How does this differ from RDS automatic snapshots?</h2>\n<p>Note that RDS comes with nightly snapshots by default. The main reason to use this function is:</p>\n<ol>\n<li>You want to take snapshots of your database more often than once per night.</li>\n<li>You want to store all of your snapshots in a separate AWS account for security and redundancy purposes.</li>\n</ol>\n<h2 class=\"preview__body--subtitle\" id=\"how-do-you-backup-your-rds-snapshots-to-a-separate-aws-account\">How do you backup your RDS snapshots to a separate AWS account?</h2>\n<p>One of the main use cases for this module is to be able to store your RDS snapshots in a completely separate AWS account.\nThat reduces the chances that you, or perhaps an intruder who breaks into your AWS account, can accidentally or\nintentionally delete all your snapshots.</p>\n<p>Let's say you have an RDS database in account A and you want to store snapshots in account B. To set that up, you need\nto do the following:</p>\n<ol>\n<li>\n<p>Deploy this lambda function (<code>lambda-create-snapshot</code>) and the <a href=\"/repos/v0.11.2/module-data-storage/modules/lambda-share-snapshot\" class=\"preview__body--description--blue\">lambda-share-snapshot\nlambda function</a> in account A. Configure this lambda function to trigger the\n<code>lambda-share-snapshot</code> function by setting the following variables:</p>\n<pre><span class=\"hljs-keyword\">module</span> <span class=\"hljs-string\">\"create_snapshot\"</span> {\n source = <span class=\"hljs-string\">\"git::git@github.com:gruntwork-io/module-data-storage.git//modules/data-storage/lambda-create-snapshot?ref=v1.0.8\"</span>\n\n <span class=\"hljs-comment\"># ... (other params ommitted) ...</span>\n\n share_snapshot_with_another_account = true\n share_snapshot_lambda_arn = <span class=\"hljs-string\">\"(ARN of the lambda-share-snapshot function)\"</span>\n share_snapshot_with_account_id = <span class=\"hljs-string\">\"(The ID of account B)\"</span>\n}\n</pre>\n</li>\n<li>\n<p>This will make the snapshots from account A <em>visible</em> in account B, but it won't actually copy them into the\naccount. To copy them into account B, deploy the <a href=\"/repos/v0.11.2/module-data-storage/modules/lambda-copy-shared-snapshot\" class=\"preview__body--description--blue\">lambda-copy-shared-snapshot\nmodule</a> in account B and configure it with the account ID of account A:</p>\n<pre><span class=\"hljs-keyword\">module</span> <span class=\"hljs-string\">\"copy_shared_snapshot\"</span> {\n source = <span class=\"hljs-string\">\"git::git@github.com:gruntwork-io/module-data-storage.git//modules/data-storage/lambda-copy-shared-snapshot?ref=v1.0.8\"</span>\n\n <span class=\"hljs-comment\"># ... (other params ommitted) ...</span>\n\n rds_db_identifier = <span class=\"hljs-string\">\"(The identifier of the RDS DB in account A)\"</span>\n rds_db_account_id = <span class=\"hljs-string\">\"(The ID of account A)\"</span>\n}\n</pre>\n</li>\n</ol>\n<h2 class=\"preview__body--subtitle\" id=\"why-use-lambda-functions\">Why use lambda functions?</h2>\n<p>The reason we use lambda functions for handling snapshots is:</p>\n<ol>\n<li>\n<p>It's easy to use <a href=\"http://docs.aws.amazon.com/lambda/latest/dg/with-scheduled-events.html\" class=\"preview__body--description--blue\" target=\"_blank\">scheduled events</a> and\n<a href=\"http://docs.aws.amazon.com/lambda/latest/dg/tutorial-scheduled-events-schedule-expressions.html\" class=\"preview__body--description--blue\" target=\"_blank\">schedule expressions</a>\nto run a lambda function on a periodic basis that is more reliable than just using cron.</p>\n</li>\n<li>\n<p>You can give your lambda function access to RDS via IAM roles instead of using API keys with an external app.</p>\n</li>\n<li>\n<p>The main use case for these lambda snapshot modules is to copy RDS snapshots to an external AWS account. That means\nyou need to run code in multiple accounts. It's easier to deploy the necessary lambda functions in each account\nand give those functions access to RDS via IAM roles than it is to create a CI job that can securely access both\naccounts.</p>\n</li>\n</ol>\n<h2 class=\"preview__body--subtitle\" id=\"how-do-you-configure-this-module\">How do you configure this module?</h2>\n<p>This module allows you to configure a number of parameters, such as which database to backup, how often to run the\nbackups, what account to share the backups with, and more. For a list of all available variables and their\ndescriptions, see <a href=\"/repos/v0.11.2/module-data-storage/modules/lambda-create-snapshot/vars.tf\" class=\"preview__body--description--blue\">vars.tf</a>.</p>\n","repoName":"module-data-storage","repoRef":"v0.11.2","serviceDescriptor":{"serviceName":"Database backup","serviceRepoName":"module-data-storage","serviceRepoOrg":"gruntwork-io","serviceMainReadmePath":"/modules/lambda-create-snapshot/README.adoc","cloudProviders":["aws"],"description":"Snapshot your RDS databases and copy the snapshots to other AWS accounts on a scheduled basis for disaster recovery.","imageUrl":"grunt.png","licenseType":"subscriber","technologies":["Terraform","JavaScript","Lambda"],"compliance":[],"tags":[""]},"serviceCategoryName":"Backup & recovery","fileName":"core-concepts.md","filePath":"/modules/lambda-create-snapshot/core-concepts.md","title":"Repo Browser: Database backup","description":"Browse the repos in the Gruntwork Infrastructure as Code Library."}
.circleci
main.tf
