Browse the Repo

file-type-icon.circleci
file-type-icon_ci
file-type-icon_docs
file-type-iconexamples
file-type-iconmodules
file-type-iconaws-helpers
file-type-iconbuild-helpers
file-type-iconbin
file-type-iconREADME.md
file-type-iconinstall.sh
file-type-iconcheck-url
file-type-iconcircleci-helpers
file-type-iconec2-backup
file-type-iconecs-deploy-runner-invoke-iam-policy
file-type-iconecs-deploy-runner-standard-configuration
file-type-iconecs-deploy-runner
file-type-icongit-helpers
file-type-icongruntwork-module-circleci-helpers
file-type-iconiam-policies
file-type-iconinfrastructure-deploy-script
file-type-iconinfrastructure-deployer
file-type-iconinstall-jenkins
file-type-iconjenkins-server
file-type-iconkubernetes-circleci-helpers
file-type-iconterraform-helpers
file-type-icontest
file-type-icontestdep
file-type-icon.gitignore
file-type-icon.pre-commit-config.yaml
file-type-iconCODEOWNERS
file-type-iconLICENSE.txt
file-type-iconREADME-CircleCI.adoc
file-type-iconREADME-Jenkins.adoc
file-type-iconREADME-Terraform-Terragrunt-Pipeline.adoc
file-type-iconREADME-TravisCI.adoc
file-type-iconREADME.adoc
file-type-iconsetup.cfg

Browse the Repo

file-type-icon.circleci
file-type-icon_ci
file-type-icon_docs
file-type-iconexamples
file-type-iconmodules
file-type-iconaws-helpers
file-type-iconbuild-helpers
file-type-iconbin
file-type-iconREADME.md
file-type-iconinstall.sh
file-type-iconcheck-url
file-type-iconcircleci-helpers
file-type-iconec2-backup
file-type-iconecs-deploy-runner-invoke-iam-policy
file-type-iconecs-deploy-runner-standard-configuration
file-type-iconecs-deploy-runner
file-type-icongit-helpers
file-type-icongruntwork-module-circleci-helpers
file-type-iconiam-policies
file-type-iconinfrastructure-deploy-script
file-type-iconinfrastructure-deployer
file-type-iconinstall-jenkins
file-type-iconjenkins-server
file-type-iconkubernetes-circleci-helpers
file-type-iconterraform-helpers
file-type-icontest
file-type-icontestdep
file-type-icon.gitignore
file-type-icon.pre-commit-config.yaml
file-type-iconCODEOWNERS
file-type-iconLICENSE.txt
file-type-iconREADME-CircleCI.adoc
file-type-iconREADME-Jenkins.adoc
file-type-iconREADME-Terraform-Terragrunt-Pipeline.adoc
file-type-iconREADME-TravisCI.adoc
file-type-iconREADME.adoc
file-type-iconsetup.cfg
EC2 backup

EC2 backup

Snapshot your EC2 instances on a scheduled basis.

Code Preview

Preview the Code

mobile file icon

README.md

down

Build Helpers

This folder contains several helper scripts for automatically building deployable, versioned artifacts of your apps:

  • build-docker-image: This script is meant to be run from a CI job to automatically build a Docker image. It builds the Docker image, tags the image with the sha1 of the most recent Git commit (or a custom tag, if specified), and then pushes the new image to a Docker registry (unless the same image/tag already exists in the registry!).

  • build-packer-artifact: This script can be used to automatically build an artifact, such as an AMI, defined in a Packer template. It runs a Packer build, runs an optional test command to verify the new artifact, extracts the artifact ID from the build, and writes the ID to a properties file. This script is meant to be run from a CI job and the properties file is a convenient way to pass information about the new artifact to another CI job.

Using the helper scripts in your code

You can install these scripts using the Gruntwork Installer:

gruntwork-install --module-name "build-helpers" --repo "https://github.com/gruntwork-io/module-ci" --tag "v0.29.1"

Properties files

After a successful build, the build helper scripts write the versions of the artifacts they produce to a properties file. The idea is that this properties file is easy to use in other scripts or other CI jobs to deploy those artifacts.

The build-docker-image script adds an entry to the properties file of the form:

DOCKER_IMAGE_TAG=<IMAGE_TAG>

The build-packer-artifact script adds an entry to the properties file of the form:

ARTIFACT_ID=<ARTIFACT_ID>

Examples

The examples below should give you an idea of how to use the scripts. Run the scripts above with the --help flag to see full documentation.

Imagine you have a Packer template templates/build.json that specifies how to build an AMI for your app. You could set up automatic deployment for this app using two Jenkins CI jobs: build-app and deploy-app.

The build-app CI job would first use the build-packer-artifact script to automatically build your AMI:

build-packer-artifact --packer-template-path templates/build.json --output-properties-file artifacts.properties

Next, the build-app CI job would then pass the contents of artifacts.properties directly to the deploy-app CI job using the Jenkins Parameterized Trigger Plugin. The deploy-app CI job, in turn, would be a parameterized build that takes as input a parameter called ARTIFACT_ID (the same parameter name that's in the artifacts.properties file) and use it, along with the scripts in the terraform-helpers module to automatically deploy the new AMI:

cd templates
terraform-update-variable --name rails_app_version --value $ARTIFACT_ID
terragrunt apply

Remote packer templates

build-packer-artifact also supports building Packer templates that are stored in a git repository. When a path of the format git::GIT_REPO_URL//RELATIVE_PATH_TO_PACKER_TEMPLATE?ref=GIT_REF is passed in as the Packer template path, build-packer-artifact will clone the repository GIT_REPO_URL at the ref GIT_REF to a temporary directory and build the template referenced at RELATIVE_PATH_TO_PACKER_TEMPLATE in the git repository.

For example, to build the example jenkins Packer template in this repo on the release tag v0.19.0:

build-packer-artifact \
  --packer-template-path git::git@github.com:gruntwork-io/module-ci.git//examples/jenkins/packer/jenkins.json?ref=v0.19.0

Idempotent packer templates

For AWS AMI builds that use tags, build-packer-artifact supports idempotent builds by AMI tags. When you pass in --idempotent true to the script, build-packer-artifact will only build a new AMI if an existing AMI with the same tags and similar name does not already exist in your account. A name is similar if everything matches except for unique build time identifiers (e.g., that returned by isotime, uuid, or timestamp).

Note that we include the name in the idempotency check to ensure that you can preserve AMIs in a multiaccount scenario, where AMI tags are not visible in the target account.

For example, consider the following packer template:

{
  "variables": {
    "aws_region": "us-east-1",
    "tag": ""
  },
  "builders": [{
    "ami_name": "gruntwork-test-{{user `tag`}}-packer-build-{{uuid | clean_resource_name}}",
    "ami_description": "An AMI created as part of testing the build-packer-artifact script.",
    "instance_type": "t2.micro",
    "region": "{{user `aws_region`}}",
    "type": "amazon-ebs",
    "source_ami": "ami-fce3c696",
    "ssh_username": "ubuntu",
    "tags": {
      "tag": "{{user `tag`}}"
    }
  }],
  "provisioners": [{
    "type": "shell",
    "inline": [
      "echo 'Hello, World'"
    ]
  }]
}

If you pass in the --idempotent true and --var tag=v1, then this will only build the AMI if it does not find any AMI with the tag tag=v1 and name that begins with gruntwork-test-v1-packer-build- in the us-east-1 region.

Under the hood the tags, names, and regions are computed at runtime using packer console with the provided vars.

Note that the following conditions must be true in order to use this feature:

  • Packer version is at least v1.4.2 (version that introduced packer console).
  • Templates are in json format (you can not use HCL based templates at this time).
  • Build is for an AMI (builder type amazon-ebs).
  • Builder is configured to tag the AMIs (tags is set).

Questions? Ask away.

We're here to talk about our services, answer any questions, give advice, or just to chat.

Ready to hand off the Gruntwork?