Browse the Repo

file-type-icon.circleci
file-type-icon_ci
file-type-icon_docs
file-type-iconexamples
file-type-iconmodules
file-type-iconaws-helpers
file-type-iconbuild-helpers
file-type-iconcheck-url
file-type-iconcircleci-helpers
file-type-icondocs-generator
file-type-iconec2-backup
file-type-iconecs-deploy-runner-invoke-iam-policy
file-type-iconecs-deploy-runner
file-type-icongit-helpers
file-type-icongruntwork-module-circleci-helpers
file-type-iconiam-policies
file-type-iconinfrastructure-deploy-script
file-type-iconinfrastructure-deployer
file-type-iconinstall-jenkins
file-type-iconjenkins-server
file-type-iconkubernetes-circleci-helpers
file-type-iconterraform-helpers
file-type-iconbin
file-type-iconREADME.md
file-type-iconinstall.sh
file-type-icontest
file-type-icontestdep
file-type-icon.gitignore
file-type-icon.pre-commit-config.yaml
file-type-iconCODEOWNERS
file-type-iconLICENSE.txt
file-type-iconREADME-CircleCI.adoc
file-type-iconREADME-Jenkins.adoc
file-type-iconREADME-Terraform-Terragrunt-Pipeline.adoc
file-type-iconREADME-TravisCI.adoc
file-type-iconREADME.adoc
file-type-iconsetup.cfg

Browse the Repo

file-type-icon.circleci
file-type-icon_ci
file-type-icon_docs
file-type-iconexamples
file-type-iconmodules
file-type-iconaws-helpers
file-type-iconbuild-helpers
file-type-iconcheck-url
file-type-iconcircleci-helpers
file-type-icondocs-generator
file-type-iconec2-backup
file-type-iconecs-deploy-runner-invoke-iam-policy
file-type-iconecs-deploy-runner
file-type-icongit-helpers
file-type-icongruntwork-module-circleci-helpers
file-type-iconiam-policies
file-type-iconinfrastructure-deploy-script
file-type-iconinfrastructure-deployer
file-type-iconinstall-jenkins
file-type-iconjenkins-server
file-type-iconkubernetes-circleci-helpers
file-type-iconterraform-helpers
file-type-iconbin
file-type-iconREADME.md
file-type-iconinstall.sh
file-type-icontest
file-type-icontestdep
file-type-icon.gitignore
file-type-icon.pre-commit-config.yaml
file-type-iconCODEOWNERS
file-type-iconLICENSE.txt
file-type-iconREADME-CircleCI.adoc
file-type-iconREADME-Jenkins.adoc
file-type-iconREADME-Terraform-Terragrunt-Pipeline.adoc
file-type-iconREADME-TravisCI.adoc
file-type-iconREADME.adoc
file-type-iconsetup.cfg
EC2 backup

EC2 backup

Snapshot your EC2 instances on a scheduled basis.

Code Preview

Preview the Code

mobile file icon

README.md

down

Terraform Helpers

This folder contains several helper scripts for working with Terraform. The scripts are:

  1. terraform-update-variable: This script can automatically update a variable in a terraform.tfvars or terragrunt.hcl file and commit and push the changes to Git. It's common practice to store the version number of the currently deployed app in a variable and this script comes in handy for updating those variables automatically, such as in a CI job after building a new AMI or Docker image.
  2. git-updated-folders: This script can be used to detect all the folders that have have been updated between two git refs (branch, tag, sha, etc). This can be used in a CI pipeline to find the terraform and terragrunt modules that have changed, and only run plan or apply on the modules that has been affected.

Using the helper scripts in your code

You can install these scripts using the Gruntwork Installer:

gruntwork-install --module-name "terraform-helpers" --repo "https://github.com/gruntwork-io/module-ci" --tag "0.0.1"

Note that terraform-update-variable depends on the git-helpers module being installed!

See the examples in the next section for how to use them.

Examples

The examples below should give you an idea of how to use the scripts. Run the scripts above with the --help flag to see full documentation.

terraform-update-variable

Terragrunt

Imagine you have a set of Terragrunt config in the live folder to deploy an app called rails-app. You create Amazon Machine Images (AMIs) for each new version of the app using Packer and the currently deployed version of the app is defined in a variable called rails_app_version, set to ami-12345 in live/rails-app/terragrunt.hcl.

If you just build a new version of the AMI, ami-67890, you could deploy it automatically, perhaps in a CI job, as follows:

cd live/rails-app
# Due to the existence of types in terragrunt.hcl, we need to be explicit about how the variable values are quoted.
terraform-update-variable --name rails_app_version --value "\"ami-67890\""
terragrunt apply

Running the commands above would:

  1. Set the rails_app_version variable in the inputs map to the ID of the new AMI, ami-67890 in terragrunt.hcl.
  2. commit and push the changes to terragrunt.hcl to Git.
  3. Deploy the changes using Terragrunt.

Terraform

Imagine you have a set of Terraform templates in the templates folder to deploy an app called rails-app. You create Amazon Machine Images (AMIs) for each new version of the app using Packer and the currently deployed version of the app is defined in a variable called rails_app_version, set to ami-12345 in templates/terraform.tfvars.

If you just build a new version of the AMI, ami-67890, you could deploy it automatically, perhaps in a CI job, as follows:

cd templates
terraform-update-variable --vars-path terraform.tfvars --name rails_app_version --value "\"ami-67890\""
terraform apply

Running the commands above would:

  1. Set the rails_app_version variable to the ID of the new AMI, ami-67890 in terraform.tfvars.
  2. commit and push the changes to terraform.tfvars to Git.
  3. Deploy the changes using Terraform.

git-updated-folders

Terragrunt

Consider the following typical terragrunt folder structure:

.
├── non-prod
│   ├── terragrunt.hcl
│   └── us-east-1
│       ├── qa
│       │   ├── env.yaml
│       │   ├── mysql
│       │   │   └── terragrunt.hcl*
│       │   └── webserver-cluster
│       │       └── terragrunt.hcl
│       ├── region.yaml
│       └── stage
│           ├── env.yaml
│           ├── mysql
│           │   └── terragrunt.hcl*
│           └── webserver-cluster
│               └── terragrunt.hcl
└── prod
    ├── terragrunt.hcl
    └── us-east-1
        ├── prod
        │   ├── env.yaml
        │   ├── mysql
        │   │   └── terragrunt.hcl*
        │   └── webserver-cluster
        │       └── terragrunt.hcl
        └── region.yaml

Here we use * to indicate files that have been modified between the git refs development and master. To get the terragrunt modules that have changed:

$ terraform-git-updated-modules --source-ref development --target-ref master --terragrunt
non-prod/us-east-1/qa/mysql
non-prod/us-east-1/stage/mysql
prod/us-east-1/prod/mysql

Each of these folders were detected as terragrunt modules (because they contain a terragrunt.hcl file), and were marked as changed because the underlying terragrunt config was updated.

Note that this will only detect terragrunt.hcl files that have changed. For example, in the following scenario where the root terragrunt.hcl file and a few yaml files were changed (the files denoted with * are those that changed), only the root folder is returned:

.
├── non-prod
│   ├── terragrunt.hcl*
│   └── us-east-1
│       ├── qa
│       │   ├── env.yaml
│       │   ├── mysql
│       │   │   └── terragrunt.hcl
│       │   └── webserver-cluster
│       │       └── terragrunt.hcl
│       ├── region.yaml*
│       └── stage
│           ├── env.yaml
│           ├── mysql
│           │   └── terragrunt.hcl
│           └── webserver-cluster
│               └── terragrunt.hcl
└── prod
    ├── terragrunt.hcl
    └── us-east-1
        ├── prod
        │   ├── env.yaml
        │   ├── mysql
        │   │   └── terragrunt.hcl
        │   └── webserver-cluster
        │       └── terragrunt.hcl
        └── region.yaml*
$ terraform-git-updated-modules --source-ref development --target-ref master --terragrunt
non-prod

If you want to additionally pull in the folders with the yaml files that changed, you can pass in --ext .yaml:

$ terraform-git-updated-modules --source-ref development --target-ref master --terragrunt --ext .yaml
non-prod
non-prod/us-east-1
prod/us-east-1

Terraform

Consider the following project structure for a monorepo terraform project:

.
├── modules
│   ├── mysql
│   │   ├── main.tf
│   │   ├── outputs.tf*
│   │   └── variables.tf
│   └── webserver-cluster
│       ├── main.tf
│       ├── outputs.tf
│       └── variables.tf
└── live
    └── us-east-1
        ├── prod
        │   ├── terraform.tfvars
        │   ├── main.tf*
        │   ├── outputs.tf*
        │   └── variables.tf
        └── non-prod
            ├── terraform.tfvars*
            ├── main.tf
            ├── outputs.tf
            └── variables.tf

Here we use * to indicate files that have been modified between the git refs development and master. To get the terraform modules that have changed:

$ terraform-git-updated-modules --source-ref development --target-ref master --terraform
modules/mysql
live/us-east-1/prod
live/us-east-1/non-prod

Each of these were detected as terraform modules (because they contain .tf files), and were marked as changed because one of the underlying .tf or .tfvars files have changed. Specifically:

  • modules/mysql <= modules/mysql/outputs.tf
  • live/us-east-1/prod <= live/us-east-1/prod/main.tf and live/us-east-1/prod/outputs.tf
  • live/us-east-1/non-prod <= live/us-east-1/non-prod/terraform.tfvars

Questions? Ask away.

We're here to talk about our services, answer any questions, give advice, or just to chat.

Ready to hand off the Gruntwork?