View on GitHub:
https://github.com/gruntwork-io/gruntwork/tree/v0.4.11README.md
This repo contains a CLI tool to perform Gruntwork tasks for the Gruntwork Reference Architecture. The currently supported functionality includes:
Note that at this time, the Reference Architecture does not configure or manage the root/management account of an AWS Organization set up. That is,
it does not include the account-baseline-root
module.
Gruntwork is planning to add this in a future enhancement.
PATH
. E.g.: mv gruntwork_darwin_amd64 /usr/local/bin/gruntwork
.chmod u+x /usr/local/bin/gruntwork
.gruntwork --help
.To make any changes in your AWS account(s), such as creating new accounts, giving Gruntwork access to those accounts, or registering domain names, you will need to provide your AWS credentials. There are two ways to do this:
Use a standard AWS CLI credential mechanism, such as setting environment variables:
export AWS_ACCESS_KEY_ID="your access key"
export AWS_SECRET_ACCESS_KEY="your secret key"
NOTE: If you're using SSO or another source of temporary credentials, then you must use this option and
you also need to set AWS_SESSION_TOKEN
, the session token given to you by the SSO provider. See the AWS
GetSessionToken API docs for
more info.
export AWS_ACCESS_KEY_ID="your access key"
export AWS_SECRET_ACCESS_KEY="your secret key"
export AWS_SESSION_TOKEN="your session token from SSO"
Allow the gruntwork
CLI to prompt you interactively for your credentials. E.g.,:
> gruntwork aws grant --account prod --managed-policy ReadOnlyAccess
INFO[2017-12-17T22:56:56Z] Checking for AWS credentials...
INFO[2017-12-17T22:57:02Z] Did not find any AWS credentials in the environment. Will prompt for credentials interactively.
Enter your AWS Access Key ID: xxx
Enter your AWS Secret Access Key: yyy
Note that if are using the gruntwork
CLI to create new AWS accounts, then:
The CLI uses AWS Organizations to create the new AWS accounts as "children" of a "root" AWS account. So the credentials you provide must be of an admin user in what will become your "root" (a.k.a. "management") account.
If you haven't enabled AWS Organizations already for your AWS account, follow these instructions to do so. Make sure to enable "All features" (and not just consolidated billing) for your AWS organization!
Once you've configured your credentials, there are two main ways to bootstrap the Git repos and AWS accounts for the Gruntwork Reference Architecture:
The easiest way to bootstrap your GitHub repos and AWS accounts for the Gruntwork Reference Architecture is to run the
wizard
command:
gruntwork wizard
This is an interactive wizard that will take you through the process step by step, including:
If you want more control over the bootstrap process, you can run a separate gruntwork
command for each step:
We deploy the Reference Architecture into new AWS accounts to ensure we cannot have any effect on any infrastructure you already have running. If you bought the Standard Reference Architecture, everything will be deployed into a single new AWS account. If you bought the Multi-Account add-on for the Reference Architecture, each environment will be deployed into a separate AWS account.
To bootstrap your AWS accounts, you need to:
You can use the create
command to create these new AWS accounts as child accounts of your AWS organization:
gruntwork aws create \
--account "dev=<EMAIL_ADDRESS_FOR_DEV_ROOT_USER>" \
--account "stage=<EMAIL_ADDRESS_FOR_STAGE_ROOT_USER>" \
--account "prod=<EMAIL_ADDRESS_FOR_PROD_ROOT_USER>" \
--account "security=<EMAIL_ADDRESS_FOR_SECURITY_ROOT_USER>" \
--account "shared=<EMAIL_ADDRESS_FOR_SHARED_ROOT_USER>" \
--account "logs=<EMAIL_ADDRESS_FOR_LOGS_ROOT_USER>"
Note that you must specify a unique email address for each of these accounts that is not already associated with any
other AWS account! If you use GMail or Google Apps, a handy trick is that you can create aliases of your email address
using a plus sign (e.g., if your email
is jon@doe.com
, you can use jon+any-text@doe.com
as an alias)!
To see all available options, run gruntwork aws create --help
.
To Grant Gruntwork employees access to your AWS account(s), you need to create an IAM role in each account that can be assumed from a Gruntwork AWS account.
To create this IAM role, use the following command:
gruntwork aws grant \
--managed-policy "AdministratorAccess" \
--account "dev" \
--account "stage" \
--account "prod" \
--account "security" \
--account "shared" \
--account "logs"
If you don't have access to the root account, or you want to grant access to a single child account, you can
authenticate directly to the child account and run the grant
command with --account
set to __current__
.
The grant
commands above give Gruntwork employees administrator access to your AWS accounts, which is necessary to
deploy the Reference Architecture. If you just want to give Gruntwork employees access for help with debugging or
troubleshooting, then you can grant read-only access by setting the --managed-policy
flag to ReadOnlyAccess
.
To see all available options, run gruntwork aws grant --help
.
New AWS Accounts typically have low quota limits for resources to avoid fraud. The initial service quotas for EC2 are unfortunately too low to deploy the Gruntwork Reference Architecture. To ensure there is enough quota capacity to deploy the Reference Architecture, we need to increase the service quota limits.
To check and request service quota increases, use the following command:
# NOTE: The region passed to --region should be the default region for your reference architecture.
gruntwork aws ensure-quota \
--region "us-east-1" \
--account "dev" \
--account "stage" \
--account "prod" \
--account "shared"
Note that we only need to check the quota limits in the dev
, stage
, prod
, and shared
accounts. These accounts
are the only accounts that have EC2 resources in the Gruntwork Reference Architecture.
If you don't have access to the root account, or you want to only check the quota on a single child account, you can
authenticate directly to the child account and run the ensure-quota
command with --account
set to __current__
. Note that
in this case, you must also specify the ID of the AWS account where your IAM users are defined: that is, the ID of
the security account:
# NOTE: The region passed to --region should be the default region for your reference architecture.
gruntwork aws ensure-quota \
--account "__current__" \
--region "us-east-1"
To see all available options, run gruntwork aws ensure-quota --help
.
The Reference Architecture uses Route 53 to manage DNS, configuring domain names for
the public load balancers, OpenVPN server, Bastion Host, and CloudFront. We strongly recommend buying
placeholder domain names from Route 53 so we can get everything working with the Reference Architecture without
affecting any of your production domain names. Most .com
domain names are just $12! You can find the latest pricing
on this page.
To check if a domain name is available, you can run:
gruntwork dns check --domain "<DOMAIN_NAME>"
If the domain name is available and the pricing listed on this page
for that domain looks reasonable, you can register it using the register
command.
You should register a different domain name in each of the dev, stage, prod, and shared accounts:
gruntwork dns register \
--account "dev=<DEV_DOMAIN_NAME>" \
--account "stage=<STAGE_DOMAIN_NAME>" \
--account "prod=<PROD_DOMAIN_NAME>" \
--account "shared=<SHARED_DOMAIN_NAME>" \
--first "<FIRST_NAME>" \
--last "<LAST_NAME>" \
--org "<COMPANY_NAME>" \
--address "<ADDRESS>" \
--city "<CITY>" \
--state "<STATE_CODE>" \
--zip "<ZIP_CODE>" \
--country "<COUNTRY_CODE>" \
--email "<EMAIL>" \
--phone "<PHONE>"
Here's a concrete example so you can see the types of values you have to specify:
gruntwork dns register \
--account "dev=acme-corp-dev.com" \
--account "stage=acme-corp-stage.com" \
--account "prod=acme-corp-prod.com" \
--account "shared=acme-corp-shared.com" \
--first "Jane" \
--last "Smith" \
--org "Acme Corp" \
--address "123 Main St" \
--city "San Francisco" \
--state "CA" \
--zip "94016" \
--country "US" \
--email "jane.smith@acme-corp.com" \
--phone "+1.5551234567"
To register a domain name in the "current" account—the one you are authenticated to in the terminal—use the name
__current__
(i.e., --account "__current__=acme-corp.com"
).
To see all available options, run gruntwork dns register --help
.
WARNING: Using the register
command will buy one or more domain names and charge your account. Make sure to check
the pricing listed on this page to make sure the price is
reasonable!
The reference architecture includes an end-to-end CI/CD pipeline for infrastructure. For this pipeline to work, you'll need to set up a machine user (also known as a service account) that will perform the following duties:
You'll need one machine user in GitHub to access the repos in the Gruntwork IaC Library. If you'll be using a different VCS provider for your infrastructure code (e.g., in BitBucket or GitLab), you'll also need to create a machine user for that version control system. Unfortunately, none of these systems allow the machine user to be created automatically, so you'll have to do it manually.
Steps to set up the GitHub machine user (do this even if you aren't using GitHub internally!):
repo
, user:email
, and admin:public_key
permissions.If you are using GitHub as your VCS, you're done! If you're using a different VCS for your own code, you must also create a Personal Access Token in that system.
write_repository
access._Repository Write_
access.By now you should have:
Be sure to keep track of which PAT is associated with each VCS!
Now you can use the gruntwork
CLI to create the secrets in AWS Secrets Manager. Note that the secrets are
created in the shared
account. You can pass --account __current__
to any of the commands below to use the currently authenticated account if you need
to, but note that the reference architecture depends on having the secrets in the shared account. No other accounts are currently supported. When Gruntwork
deploys the reference architecture, these secrets will be automatically shared via code with all of the other accounts in the architecture. If the secret
needs to change for some reason, you'll only need to update it in the shared account, thus making secrets management more DRY.
You must pass the AWS region in which you want to create the token. This should be the same region as you're using for the reference architecture. Each
command accepts a --region
flag, or you can alternatively set the AWS_DEFAULT_REGION
environment variable. If you do not specify a region, the tool
will default to us-east-1
.
If you use the gruntwork wizard
to create the tokens, as discussed above, you will be prompted for each
token and the region, avoiding the need to pass the tokens via flags as described here. If you choose to use this method, we recommend either:
pass
, the standard unix password manager.GITHUB_TOKEN
and VCS_TOKEN
).If you're using an alternative to GitHub for your own code repository, use the following command:
gruntwork secrets create --github-token "<YOUR GITHUB TOKEN>" --vcs-token "<YOUR VCS TOKEN>" --region "<YOUR AWS REGION>"
Or the equivalent if you're using pass
, assuming that the GitHub token item is named GitHub/machine-user-token
:
gruntwork secrets create --github-token "$(pass GitHub/machine-user-token)" --vcs-token "$(pass YourVCS/machine-user-token)" --region "<YOUR AWS REGION>"
Or the equivalent if you're using an environment variable:
export VCS_TOKEN=<YOUR VCS TOKEN>
export GITHUB_TOKEN=<YOUR GITHUB TOKEN>
export AWS_DEFAULT_REGION=<YOUR AWS REGION>
gruntwork secrets create
This command creates a new AWS Secrets Manager secret containing the GitHub token. The command will output the name and ARN of the secret. Be sure to save this as you'll need it when you complete the reference architecture YAML form.
If you're using GitHub for your code repository, you only need to set the GitHub token:
gruntwork secrets create --github-token "<GITHUB TOKEN>" --region "<YOUR AWS REGION>"
This command creates two new secrets: one containing the GitHub token, and another containing the token for your other VCS system. The secrets will be tagged as having been created by this tool for future discovery. The tool will output the ARN of both secrets. Be sure to save them as you'll need them when you complete the reference architecture YAML form.
You may wish to identify which secrets have been created by the gruntwork
CLI tool. You can list them with the following command:
gruntwork secrets list --region "<YOUR AWS REGION>"
You can update the secret(s) with the following command:
gruntwork secrets update --github-token "<NEW GITHUB TOKEN>" --vcs-token "<NEW VCS TOKEN>" --region "<YOUR AWS REGION>"
You may omit either --github-token
or --vcs-token
, but at least one of them (or both) must be provided.
You can also delete the secret(s) with the following command:
gruntwork secrets delete --delete-github-token ---delete-vcs-token --region "<YOUR AWS REGION>"
You can also pass the --force-delete-immediately
flag to delete the flag immediately. If you use this option, the secret will be irrecoverable.
If you do not use this option, the secret will be recoverable through the AWS Secrets Manager API/UI for 30 days. Note that even when using this flag, it may take a few minutes for AWS to delete the secret before you can create a new secret.
Once the Reference Architecture is deployed and you are fully up and running on it, it can be a good idea to revoke Gruntwork's access to your accounts (this reduces your surface area to potential attackers). If you ever need Gruntwork to access your accounts in the future (e.g., to help with troubleshooting), you can re-run the commands from the Grant Gruntwork access to the AWS accounts section.
To remove the IAM roles for the Standard Reference Architecture with the Multi-Account Upgrade (multiple AWS accounts):
gruntwork aws revoke \
--account "dev" \
--account "stage" \
--account "prod" \
--account "security" \
--account "shared" \
--account "logs"
To revoke Gruntwork's access from the "current" account—the one you are authenticated to in the terminal—use the name
__current__
(i.e., --account "__current__"
).
To see all available options, run gruntwork aws revoke --help
.
Since you're already authenticated while running the gruntwork
CLI tool, we thought it would be handy
to allow resetting an IAM user's password from the tool as well.
If running non-interactively, you can provide the username and password on the command line directly:
gruntwork aws reset-password --iam-user-name <IAM_username> --iam-user-password <password>
Or if you prefer interactive mode, you can run the base command:
gruntwork aws reset-password
and the tool will prompt you for both the username and the password.
gruntwork supports using aws-vault to generate profiles for your Reference Architecture's AWS accounts, and exposes login
and exec
commands that leverage these generated profiles to simplify accessing and working with your accounts.
You should first run the gruntwork vault generate
command as described below, and subsequently use the
login
and exec
commands.
Gruntwork can generate valid aws-vault
account profiles for each of the AWS accounts that comprise your Reference Architecture.
Run the command, passing the exact name of your personal IAM user in your Security account, and the full path on your system to your infrastructure-live repository:
gruntwork vault generate --iam-user-name <IAM_username> --infra-live-path <full-path-to-your-infra-live-repo>
You should now see your new account profiles appended to the end of your ~/.aws/config
file.
After your account profiles are generated, you can use gruntwork vault login
to choose an account and either log into it directly, by opening a new tab in your browser to the AWS web console, or by copying the login string to your clipboard that you can paste into any browser you wish.
This command has two modes: interactive and non-interactive.
Interactive mode uses a fuzzy-matching list that helps you narrow down and pick your target account quickly. Use this mode if you have many accounts, if you can't remember the exact account name, or if you're just in a hurry to log in. To run the command in interactive mode:
gruntwork vault login
You can use arrrow keys or vim-like navigation hotkeys (hjkl) to navigate the list.
You can press /
to enter filter mode, which will allow you to filter through the list.
When you have found the account you wish to log into, you have two options:
[enter]
, which will open the AWS web console to that account in a new tab in your default browser[ctrl+g]
(for "grab") to copy the login string to that account to your system clipboardNon-interactive mode allows you to specify the exact name of the account you want to log into, so it's convenient when writing out runbooks or instructions that others are intended to follow later. To run the command in Non-interactive mode, supply the name of the account you want to log into:
gruntwork vault login security
When running in Non-interactive mode, the AWS console to your target account will be opened in your default browser.
After your account profiles are generated, you can use gruntwork vault exec
to run arbitrary commands with your target account's credentials exposed. This is useful, for example, when you need to run the AWS command line tool against an account.
This command has two modes: Interactive and Non-interactive.
Interactive mode uses a fuzzy-matching list that helps you narrow down and pick your target account quickly. Use this mode if you have many accounts, if you can't remember the exact account name, or if you're just in a hurry to execute a command. To run the command in interactive mode:
gruntwork vault exec
You can use arrrow keys or vim-like navigation hotkeys (hjkl) to navigate the list.
You can press /
to enter filter mode, which will allow you to filter through the list.
When you have found the account you wish to execute your command against, press [enter]
. You'll next be prompted to supply the command you wish to run. Enter your command and press [enter]
.Your command will be run against your target account and the result will be written to STDOUT
.
Non-interactive mode allows you to specify the exact name of the account you want to execute your command against, so it's convenient when writing out runbooks or instructions that others are intended to follow later.
Non-interactive mode is also ideal if you're looking to use pipes and redirection. Interactive mode is better suited for quick, simple commands.
To run the command in Non-interactive mode, supply the name of the account you want to execute against, plus the command, like so:
gruntwork vault exec logs -- aws s3 ls
or
gruntwork vault exec security --no-session -- aws iam list-roles | jq > output.json
Use pass
to store your passwords so that you can use them anywhere in shell commands:
gruntwork aws reset-password --iam-user-name <IAM_username> --iam-user-password $(pass secret-password)
Or use the command interactively, passing in only the username on the command line:
gruntwork aws reset-password --iam-user-name <IAM_username>
and the tool will prompt you for the new password.
We're here to talk about our services, answer any questions, give advice, or just to chat.