Browse the Repo
.circleci.githubaccountscommandsdnsformatoptionspromptretrysecretswizard
.gitignoreCODEOWNERSREADME.mdgo.modgo.summain.goBrowse the Repo
.circleci.githubaccountscommandsdnsformatoptionspromptretrysecretswizard.gitignoreCODEOWNERSREADME.mdgo.modgo.summain.gogruntwork
Code Preview
View on GitHub:
https://github.com/gruntwork-io/gruntwork/tree/v0.2.5Preview the Code
README.md
Gruntwork
This repo contains a CLI tool to perform Gruntwork tasks for the Gruntwork Reference Architecture. The currently supported functionality includes:
- Creating the AWS accounts and permissions.
- Registering domain names.
- Setting up VCS tokens as secrets in AWS Secrets Manager.
Note that at this time, the Reference Architecture does not configure or manage the root/management account of an AWS Organization set up. That is,
it does not include the account-baseline-root module.
Gruntwork is planning to add this in a future enhancement.
Install
- Download the latest binary for your OS on the releases page.
- Move the binary to a folder on your
PATH. E.g.:mv gruntwork_darwin_amd64 /usr/local/bin/gruntwork. - Add execute permissions to the binary. E.g.:
chmod u+x /usr/local/bin/gruntwork. - Test it installed correctly:
gruntwork --help.
Credentials
To make any changes in your AWS account(s), such as creating new accounts, giving Gruntwork access to those accounts, or registering domain names, you will need to provide your AWS credentials. There are two ways to do this:
-
Use a standard AWS CLI credential mechanism, such as setting environment variables:
export AWS_ACCESS_KEY_ID="your access key" export AWS_SECRET_ACCESS_KEY="your secret key"-
NOTE: If you're using SSO or another source of temporary credentials, then you must use this option and you also need to set
AWS_SESSION_TOKEN, the session token given to you by the SSO provider. See the AWS GetSessionToken API docs for more info.export AWS_ACCESS_KEY_ID="your access key" export AWS_SECRET_ACCESS_KEY="your secret key" export AWS_SESSION_TOKEN="your session token from SSO"
-
-
Allow the
gruntworkCLI to prompt you interactively for your credentials. E.g.,:> gruntwork aws grant --account prod --managed-policy ReadOnlyAccess INFO[2017-12-17T22:56:56Z] Checking for AWS credentials... INFO[2017-12-17T22:57:02Z] Did not find any AWS credentials in the environment. Will prompt for credentials interactively. Enter your AWS Access Key ID: xxx Enter your AWS Secret Access Key: yyy
Note that if are using the gruntwork CLI to create new AWS accounts, then:
-
The CLI uses AWS Organizations to create the new AWS accounts as "children" of a "master" AWS account. So the credentials you provide must be of an admin user in what will become your "root" (a.k.a. "management") account.
-
If you haven't enabled AWS Organizations already for your AWS account, follow these instructions to do so. Make sure to enable "All features" (and not just consolidated billing) for your AWS organization!
Bootstrapping the Reference Architecture
Once you've configured your credentials, there are two main ways to bootstrap the Git repos and AWS accounts for the Gruntwork Reference Architecture:
Bootstrap using the wizard (recommended)
The easiest way to bootstrap your GitHub repos and AWS accounts for the Gruntwork Reference Architecture is to run the
wizard command:
gruntwork wizard
This is an interactive wizard that will take you through the process step by step, including:
- Bootstrapping your AWS accounts
- Bootstrapping your domain names
- Setting up machine user VCS tokens
Bootstrap manually
If you want more control over the bootstrap process, you can run a separate gruntwork command for each step:
Bootstrap the AWS accounts
We deploy the Reference Architecture into new AWS accounts to ensure we cannot have any effect on any infrastructure you already have running. If you bought the Standard Reference Architecture, everything will be deployed into a single new AWS account. If you bought the Multi-Account add-on for the Reference Architecture, each environment will be deployed into a separate AWS account.
To bootstrap your AWS accounts, you need to:
Create the AWS accounts
You can use the create command to create these new AWS accounts as child accounts of your AWS organization:
gruntwork aws create \
--account "dev=<EMAIL_ADDRESS_FOR_DEV_ROOT_USER>" \
--account "stage=<EMAIL_ADDRESS_FOR_STAGE_ROOT_USER>" \
--account "prod=<EMAIL_ADDRESS_FOR_PROD_ROOT_USER>" \
--account "security=<EMAIL_ADDRESS_FOR_SECURITY_ROOT_USER>" \
--account "shared=<EMAIL_ADDRESS_FOR_SHARED_ROOT_USER>" \
--account "logs=<EMAIL_ADDRESS_FOR_LOGS_ROOT_USER>"
Note that you must specify a unique email address for each of these accounts that is not already associated with any
other AWS account! If you use GMail or Google Apps, a handy trick is that you can create aliases of your email address
using a plus sign (e.g., if your email
is jon@doe.com, you can use jon+any-text@doe.com as an alias)!
To see all available options, run gruntwork aws create --help.
Grant Gruntwork employees access to the AWS accounts
To Grant Gruntwork employees access to your AWS account(s), you need to create an IAM role in each account that can be assumed from a Gruntwork AWS account.
To create this IAM role, use the following command:
gruntwork aws grant \
--managed-policy "AdministratorAccess" \
--iam-user-account-name "security" \
--account "dev" \
--account "stage" \
--account "prod" \
--account "security" \
--account "shared" \
--account "logs"
If you don't have access to the root account, or you want to grant access to a single child account, you can
authenticate directly to the child account and run the grant command with --account set to __current__. Note that
in this case, you must also specify the ID of the AWS account where your IAM users are defined: that is, the ID of
the security account:
gruntwork aws grant --managed-policy "AdministratorAccess" \
--account "__current__" \
--iam-user-account-id "<SECURITY_ACCOUNT_ID>"
The grant commands above give Gruntwork employees administrator access to your AWS accounts, which is necessary to
deploy the Reference Architecture. If you just want to give Gruntwork employees access for help with debugging or
troubleshooting, then you can grant read-only access by setting the --managed-policy flag to ReadOnlyAccess.
To see all available options, run gruntwork aws grant --help.
Ensure sufficient quota capacity
New AWS Accounts typically have low quota limits for resources to avoid fraud. The initial service quotas for EC2 are unfortunately too low to deploy the Gruntwork Reference Architecture. To ensure there is enough quota capacity to deploy the Reference Architecture, we need to increase the service quota limits.
To check and request service quota increases, use the following command:
gruntwork aws ensure-quota \
--managed-policy "AdministratorAccess" \
--iam-user-account-name "security" \
--account "dev" \
--account "stage" \
--account "prod" \
--account "shared"
Note that we only need to check the quota limits in the dev, stage, prod, and shared accounts. These accounts
are the only accounts that have EC2 resources in the Gruntwork Reference Architecture.
If you don't have access to the root account, or you want to only check the quota on a single child account, you can
authenticate directly to the child account and run the ensure-quota command with --account set to __current__. Note that
in this case, you must also specify the ID of the AWS account where your IAM users are defined: that is, the ID of
the security account:
gruntwork aws ensure-quota --managed-policy "AdministratorAccess" \
--account "__current__" \
--iam-user-account-id "<SECURITY_ACCOUNT_ID>"
To see all available options, run gruntwork aws ensure-quota --help.
Bootstrap the domain names
The Reference Architecture uses Route 53 to manage DNS, configuring domain names for
the public load balancers, Jenkins, OpenVPN server, Bastion Host, and CloudFront. We strongly recommend buying
placeholder domain names from Route 53 so we can get everything working with the Reference Architecture without
affecting any of your production domain names. Most .com domain names are just $12! You can find the latest pricing
on this page.
To check if a domain name is available, you can run:
gruntwork dns check --domain "<DOMAIN_NAME>"
If the domain name is available and the pricing listed on this page
for that domain looks reasonable, you can register it using the register command.
You should register a different domain name in each of the dev, stage, prod, and shared accounts:
gruntwork dns register \
--account "dev=<DEV_DOMAIN_NAME>" \
--account "stage=<STAGE_DOMAIN_NAME>" \
--account "prod=<PROD_DOMAIN_NAME>" \
--account "shared=<SHARED_DOMAIN_NAME>" \
--first "<FIRST_NAME>" \
--last "<LAST_NAME>" \
--org "<COMPANY_NAME>" \
--address "<ADDRESS>" \
--city "<CITY>" \
--state "<STATE_CODE>" \
--zip "<ZIP_CODE>" \
--country "<COUNTRY_CODE>" \
--email "<EMAIL>" \
--phone "<PHONE>"
Here's a concrete example so you can see the types of values you have to specify:
gruntwork dns register \
--account "dev=acme-corp-dev.com" \
--account "stage=acme-corp-stage.com" \
--account "prod=acme-corp-prod.com" \
--account "shared=acme-corp-shared.com" \
--first "Jane" \
--last "Smith" \
--org "Acme Corp" \
--address "123 Main St" \
--city "San Francisco" \
--state "CA" \
--zip "94016" \
--country "US" \
--email "jane.smith@acme-corp.com" \
--phone "+1.5551234567"
To register a domain name in the "current" account—the one you are authenticated to in the terminal—use the name
__current__ (i.e., --account "__current__=acme-corp.com").
To see all available options, run gruntwork dns register --help.
WARNING: Using the register command will buy one or more domain names and charge your account. Make sure to check
the pricing listed on this page to make sure the price is
reasonable!
Setting up machine user VCS tokens
The reference architecture includes an end-to-end CI/CD pipeline for infrastructure. For this pipeline to work, you'll need to set up a machine user (also known as a service account) that will perform the following duties:
- Automatically checkout your code.
- Push artifacts (such as Docker images and AMIs).
- Access the Gruntwork IaC Library.
You'll need one machine user in GitHub to access the repos in the Gruntwork IaC Library. If you'll be using a different VCS provider for your infrastructure code (e.g., in BitBucket or GitLab), you'll also need to create a machine user for that version control system. Unfortunately, none of these systems allow the machine user to be created automatically, so you'll have to do it manually.
Setting up the machine user
Steps to set up the GitHub machine user (do this even if you aren't using GitHub internally!):
- In GitHub, create a new user account, then create a Personal Access Token (PAT). Keep track of the username as you will need it later.
- The PAT should be granted
repo,user:email, andadmin:public_keypermissions.
If you are using GitHub as your VCS, you're done! If you're using a different VCS for your own code, you must also create a Personal Access Token in that system.
- If you are using GitLab, use these instructions, and grant
write_repositoryaccess. - If you are using Bitbucket, use these instructions, and grant
_Repository Write_access.
By now you should have:
- A PAT for GitHub
- If you are not using GitHub for your infrastructure code, you should also have a PAT for your other VCS.
Be sure to keep track of which PAT is associated with each VCS!
Setting up the tokens in AWS Secrets Manager
Now you can use the gruntwork CLI to create the secrets in AWS Secrets Manager. Note that the secrets are
created in the shared account. You can pass --account __current__ to any of the commands below to use the currently authenticated account if you need
to, but note that the reference architecture depends on having the secrets in the shared account. No other accounts are currently supported. When Gruntwork
deploys the reference architecture, these secrets will be automatically shared via code with all of the other accounts in the architecture. If the secret
needs to change for some reason, you'll only need to update it in the shared account, thus making secrets management more DRY.
You must pass the AWS region in which you want to create the token. This should be the same region as you're using for the reference architecture. Each
command accepts a --region flag, or you can alternatively set the AWS_DEFAULT_REGION environment variable. If you do not specify a region, the tool
will default to us-east-1.
If you use the gruntwork wizard to create the tokens, as discussed above, you will be prompted for each
token and the region, avoiding the need to pass the tokens via flags as described here. If you choose to use this method, we recommend either:
- Reading the tokens from an encrypted source, such as by using the
pass, the standard unix password manager. - Reading the tokens from an environment variable (
GITHUB_TOKENandVCS_TOKEN).
If you're using an alternative to GitHub for your own code repository, use the following command:
gruntwork secrets create --github-token "<YOUR GITHUB TOKEN>" --vcs-token "<YOUR VCS TOKEN>" --region "<YOUR AWS REGION>"
Or the equivalent if you're using pass, assuming that the GitHub token item is named GitHub/machine-user-token:
gruntwork secrets create --github-token "$(pass GitHub/machine-user-token)" --vcs-token "$(pass YourVCS/machine-user-token)" --region "<YOUR AWS REGION>"
Or the equivalent if you're using an environment variable:
export VCS_TOKEN=<YOUR VCS TOKEN>
export GITHUB_TOKEN=<YOUR GITHUB TOKEN>
export AWS_DEFAULT_REGION=<YOUR AWS REGION>
gruntwork secrets create
This command creates a new AWS Secrets Manager secret containing the GitHub token. The command will output the name and ARN of the secret. Be sure to save this as you'll need it when you complete the reference architecture YAML form.
If you're using GitHub for your code repository, you only need to set the GitHub token:
gruntwork secrets create --github-token "<GITHUB TOKEN>" --region "<YOUR AWS REGION>"
This command creates two new secrets: one containing the GitHub token, and another containing the token for your other VCS system. The secrets will be tagged as having been created by this tool for future discovery. The tool will output the ARN of both secrets. Be sure to save them as you'll need them when you complete the reference architecture YAML form.
You may wish to identify which secrets have been created by the gruntwork CLI tool. You can list them with the following command:
gruntwork secrets list --region "<YOUR AWS REGION>"
You can update the secret(s) with the following command:
gruntwork secrets update --github-token "<NEW GITHUB TOKEN>" --vcs-token "<NEW VCS TOKEN>" --region "<YOUR AWS REGION>"
You may omit either --github-token or --vcs-token, but at least one of them (or both) must be provided.
You can also delete the secret(s) with the following command:
gruntwork secrets delete --delete-github-token ---delete-vcs-token --region "<YOUR AWS REGION>"
You can also pass the --force-delete-immediately flag to delete the flag immediately. If you use this option, the secret will be irrecoverable.
If you do not use this option, the secret will be recoverable through the AWS Secrets Manager API/UI for 30 days. Note that even when using this flag, it may take a few minutes for AWS to delete the secret before you can create a new secret.
Revoking access to AWS
Once the Reference Architecture is deployed and you are fully up and running on it, it can be a good idea to revoke Gruntwork's access to your accounts (this reduces your surface area to potential attackers). If you ever need Gruntwork to access your accounts in the future (e.g., to help with troubleshooting), you can re-run the commands from the Grant Gruntwork access to the AWS accounts section.
To remove the IAM roles for the Standard Reference Architecture with the Multi-Account Upgrade (multiple AWS accounts):
gruntwork aws revoke \
--account "dev" \
--account "stage" \
--account "prod" \
--account "security" \
--account "shared" \
--account "logs"
To revoke Gruntwork's access from the "current" account—the one you are authenticated to in the terminal—use the name
__current__ (i.e., --account "__current__").
To see all available options, run gruntwork aws revoke --help.
Resetting IAM user passwords
Since you're already authenticated while running the gruntwork CLI tool, we thought it would be handy
to allow resetting an IAM user's password from the tool as well.
If running non-interactively, you can provide the username and password on the command line directly:
gruntwork aws reset-password --iam-user-name <IAM_username> --iam-user-password <password>
Or if you prefer interactive mode, you can run the base command:
gruntwork aws reset-password
and the tool will prompt you for both the username and the password.
⚠️ Avoid storing passwords in plain text in your shell history!
Use pass to store your passwords so that you can use them anywhere in shell commands:
gruntwork aws reset-password --iam-user-name <IAM_username> --iam-user-password $(pass secret-password)
Or use the command interactively, passing in only the username on the command line:
gruntwork aws reset-password --iam-user-name <IAM_username>
and the tool will prompt you for the new password.
Questions? Ask away.
We're here to talk about our services, answer any questions, give advice, or just to chat.
