aws

Authenticating to AWS with Environment Variables

This is Part 2 of the Comprehensive Guide to Authenticating to AWS on the Command Line. In Part 1, we went over how to use the Credentials…
Authenticating to AWS with Environment Variables
YB
Yevgeniy Brikman
Co-Founder
Published August 4, 2018

This is Part 2 of the Comprehensive Guide to Authenticating to AWS on the Command Line. In Part 1, we went over how to use the Credentials File, but found that while easy to use, it was not particularly secure. In this post, we’ll introduce a second option for authenticating to AWS on the Command Line: Environment Variables.

Basic Usage

Another way to authenticate to AWS on the CLI is to set your Access Keys as the environment variables AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY:

export AWS_ACCESS_KEY_ID=AKIAIOSFODNN7EXAMPLE
export AWS_SECRET_ACCESS_KEY=wJalrXUtnFEMI/K7MDENG/bPxRfiCEXAMPLEKEY

Or if you’re on Windows:

set AWS_ACCESS_KEY_ID=AKIAIOSFODNN7EXAMPLE
set AWS_SECRET_ACCESS_KEY=wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY

Now you can run any CLI tool and it will use these credentials automatically:

export AWS_ACCESS_KEY_ID=AKIAIOSFODNN7EXAMPLE
export AWS_SECRET_ACCESS_KEY=wJalrXUtnFEMI/K7MDENG/bPxRfiCEXAMPLEKEY
terraform apply

Important note: Setting environment variables as shown above, at least on Linux/Unix/Mac, will store your credentials in your bash history. That means your credentials would be stored in plaintext, on disk! There are a few ways to prevent this.

The first is simply to add a space before the export commands:

export AWS_ACCESS_KEY_ID=AKIAIOSFODNN7EXAMPLE
export AWS_SECRET_ACCESS_KEY=wJalrXUtnFEMI/K7MDENG/bPxfiCEXAMPLEKEY
terraform apply

By default, commands with a leading space are not stored in bash history (note: you may need to tweak your HISTCONTROL setting to be sure, as described here).

The second, and more secure option, is to store your credentials in a CLI-friendly password store, such as pass. For example, you can store the credentials as follows:

$ pass insert aws-access-key-id
Enter aws-access-key-id: AKIAIOSFODNN7EXAMPLE
$ pass insert aws-secret-access-key
Enter aws-secret-access-key: wJalrXUtnFEMI/K7MDENG/bPxRfiCEXAMPLEKEY

When you run the commands above, pass will save the credentials in two files, aws-access-key-id and aws-secret-access-key, encrypting each file with gpg. Now, you can set your environment variables as follows, with no risk of any secrets being stored in bash history:

export AWS_ACCESS_KEY_ID=$(pass aws-access-key-id)
export AWS_SECRET_ACCESS_KEY=$(pass aws-secret-access-key)

You can even take the two lines above, put them into a script called auth.sh, and set your environment variables with a single command:

. auth.sh
terraform apply

In the rest of this blog post, whenever you see credentials being exported as environment variables, make sure to use one of the methods above to ensure those secrets aren’t stored in your bash history!

Working with multiple sets of Access Keys

If you have multiple sets of Access Keys (e.g., for multiple IAM Users in different AWS accounts), you just set the same environment variable names for each set of Access Keys. For example, in one terminal tab, you could authenticate using the Access Keys from your stage account:

export AWS_ACCESS_KEY_ID=AKIAIOSFODNN7EXAMPLE1
export AWS_SECRET_ACCESS_KEY=wJalrXUtnFEMI/K7MDENG/bxRfiCEXAMPLEKEY1

In another terminal, you can use the Access Keys from your prod account:

export AWS_ACCESS_KEY_ID=AKIAIOSFODNN7EXAMPLE2
export AWS_SECRET_ACCESS_KEY=wJalrXUtnFEMI/K7MDENG/bxRfiCEXAMPLEKEY2

Working with IAM Roles

If you want to assume IAM Roles — for example, you have an IAM User in the security account and want to assume an IAM Role in your dev account — you have two options. The first option, as discussed in the Credentials File blog post, is to do it in the CLI tool you’re using (e.g., Terraform), if it supports it.

The second option, with environment variables, is quite a bit trickier. First, you need to authenticate with your normal (permanent) Access Keys:

export AWS_ACCESS_KEY_ID=AKIAIOSFODNN7EXAMPLE
export AWS_SECRET_ACCESS_KEY=wJalrXUtnFEMI/K7MDENG/bPxRfiCEXAMPLEKEY

Next, you call aws sts assume-role, passing it the ARN of the IAM Role you want to assume, plus a “role session name” that can be used to tell who is assuming the IAM Role and why (as the same IAM Role may be assumed by may different users):

aws sts assume-role \
--role-arn arn:aws:iam::123456789012:role/dev-full-access \
--role-session-name username@company.com

This will return a blob of JSON that contains Temporary Access Keys:

{
"Credentials": {
"SecretAccessKey": "secret-access-key",
"SessionToken": "temporary-session-token",
"Expiration": "expiration-date-time",
"AccessKeyId": "access-key-id"
}
}

You must now set these Temporary Access Keys as environment variables, overriding the old environment variables:

export AWS_ACCESS_KEY_ID=<Access-key-from-output>
export AWS_SECRET_ACCESS_KEY=<Secret-access-key-from-output>
export AWS_SESSION_TOKEN=<Session-Token-from-output>

Note that with Temporary Access Keys, you must not only set the AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY environment variables, but also AWS_SESSION_TOKEN. Going through this aws sts assume-role process manually each time you want to assume an IAM Role is tedious, so most teams use scripts to automate this process (more on this below).

Also, take note that, by default, the Temporary Access Keys you get from aws sts assume-role expire after just 1 hour. If you plan to use those Temporary Access Keys as your credentials all day long, and don’t want to have to re-authenticate every hour, you should:

  1. Update your IAM Roles to increase the maximum expiration time. You can set it as high as 12 hours.
  2. When calling aws sts assume-role, use the --duration-seconds argument to request a longer expiration duration (e.g., 43,200 seconds = 12 hours):
aws sts assume-role \
--role-arn arn:aws:iam::123456789012:role/dev-full-access \
--role-session-name username@company.com \
--duration-seconds 43200

Working with MFA

Using MFA with environment variables is also a tricky, multi-step process. First, you need to authenticate with your normal (permanent) Access Keys:

export AWS_ACCESS_KEY_ID=AKIAIOSFODNN7EXAMPLE
export AWS_SECRET_ACCESS_KEY=wJalrXUtnFEMI/K7MDENG/bPxRfiCEXAMPLEKEY

Next, you run the aws sts get-session-token command, passing it the ARN of your MFA device and an MFA token from the Google Authenticator App or your key fob:

aws sts get-session-token \
--serial-number arn:aws:iam::123456789012:mfa/jon-doe \
--token-code 123456 \
--duration-seconds 43200

This will return a blob of JSON that contains Temporary Access Keys** (note the --duration-seconds argument in the earlier command, which specifies when these Temporary Access Keys will expire):

{
"Credentials": {
"SecretAccessKey": "secret-access-key",
"SessionToken": "temporary-session-token",
"Expiration": "expiration-date-time",
"AccessKeyId": "access-key-id"
}
}

You must now set these Temporary Access Keys as environment variables, overriding the old environment variables:

export AWS_ACCESS_KEY_ID=<Access-key-from-output>
export AWS_SECRET_ACCESS_KEY=<Secret-access-key-from-output>
export AWS_SESSION_TOKEN=<Session-Token-from-output>

Note that if you want to both use MFA and assume an IAM Role, then you should call aws sts assume-role as in the previous section, but include --serial-number and --token-code parameters as in this section:

aws sts assume-role \
--role-arn arn:aws:iam::123456789012:role/dev-full-access \
--role-session-name username@company.com \
--serial-number arn:aws:iam::123456789012:mfa/jon-doe \
--token-code 123456 \
--duration-seconds 43200

Working with assume-role and get-session-token is a tedious and error-prone process, so most team use scripts and tools to automate this process, as described in the next section.

Scripts and tools for using environment variables

There are a several tools that make it easier to use environment variables to authenticate to AWS:

aws-vault

The team at 99designs has created an open source, cross-platform CLI tool called aws-vault that can:

  1. Securely store your AWS credentials in your operating system’s keystore (e.g., Keychain, KWallet)
  2. Automatically set those credentials as environment variables when executing a command.
  3. Handle all the aws sts commands for you when using IAM Roles or MFA.

The basic usage is as follows. First, you tell aws-vault to store your AWS Access Keys under a Named Profile (note: aws-vault uses Named Profiles, just like the Credentials File, but it does not store the credentials in plaintext in the Credentials File itself):

$ aws-vault add dev-full-access
Enter Access Key Id: AKIAIOSFODNN7EXAMPLE
Enter Secret Key: wJalrXUtnFEMI/K7MDENG/bPxRfiCEXAMPLEKEY

Now you can use the aws-vault exec command to set those Access Keys as environment variables when executing a CLI command:

aws-vault exec home -- terraform apply

To assume an IAM role, you can specify the role_arn parameter in your Config File in ~/.aws/config:

[profile dev-full-access]
role_arn = arn:aws:iam::123456789012:role/dev-full-access

Now, next time you run aws-vault exec dev-full-access, aws-vault will automatically assume an IAM Role for you. Similarly, you can specify an mfa_serial in ~/.aws/config:

[profile dev-full-access]
mfa_serial = arn:aws:iam::123456789012:mfa/jon-doe

Each time you run aws-vault exec dev-full-access, it’ll prompt you for an MFA token.

aws-auth

aws-auth is a bash script available in the Gruntwork Infrastructure as Code Library that automates all the aws sts steps for using environment variables. For example, to assume an IAM Role, you first set your normal (permanent) AWS Access Keys as environment variables:

export AWS_ACCESS_KEY_ID=AKIAIOSFODNN7EXAMPLE
export AWS_SECRET_ACCESS_KEY=wJalrXUtnFEMI/K7MDENG/bPxRfiCEXAMPLEKEY

And then you can run:

eval "$(aws-auth --role-arn arn:aws:iam::123456789011:role/my-role)"

This will automatically set new AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, and AWS_SESSION_TOKEN environment variables for you, so you can immediately begin running commands that need AWS authentication:

eval "$(aws-auth --role-arn arn:aws:iam::123456789011:role/my-role)"
terraform apply

Similarly, to use MFA, you can run:

eval "$(aws-auth \
--serial-number arn:aws:iam::123456789011:mfa/jondoe \
--token-code 123456)"
terraform apply

Again, this will set the AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, and AWS_SESSION_TOKEN environment variables for you. And, of course, you can assume an IAM Role and use MFA at the same time:

eval "$(aws-auth \
--serial-number arn:aws:iam::123456789011:mfa/jondoe \
--token-code 123456 \
--role-arn arn:aws:iam::123456789011:role/my-role)
terraform apply

aws-auth works especially well when combined with a CLI-friendly password manager, such as pass. Instead of having to manually set your permanent AWS Access Keys as environment variables, and to manually pass the IAM Role ARN and MFA serial as a command-line args, you can store them securely on disk (encrypted with gpg) using pass:

$ pass insert aws-access-key-id
Enter aws-access-key-id: AKIAIOSFODNN7EXAMPLE
$ pass insert aws-secret-access-key
Enter aws-secret-access-key: wJalrXUtnFEMI/K7MDENG/bPxRfiCEXAMPLEKEY
$ pass insert aws-mfa-arn
Enter aws-mfa-arn: arn:aws:iam::123456789011:mfa/jondoe
$ pass insert aws-iam-role-arn
Enter aws-iam-role-arn: arn:aws:iam::123456789011:role/foo

Now you can create a script called, for example, auth.sh, with the following contents:

read -p "Enter your MFA token: " token
eval $(AWS_ACCESS_KEY_ID=$(pass aws-access-key-id) \
AWS_SECRET_ACCESS_KEY=$(pass aws-secret-access-key) \
aws-auth \
--serial-number $(pass aws-mfa-arn) \
--token-code "$token" \
--role-arn $(pass aws-iam-role-arn))

When you run this script, it will set permanent AWS Access Keys as environment variables, prompt you for an MFA Token, do all the aws sts calls to use the MFA Token and assume an IAM Role, and set environment variables for you with your new Temporary Access Keys, all in a single command:

eval "$(auth.sh)"
terraform apply

If you’re a Gruntwork Subscriber, you can download aws-auth here (note: if you’re not a subscriber, you’ll get a 404!).

Pros and cons

Pros

  • Your Access Keys are never stored in plaintext on disk, so this is more secure than the Credentials File.
  • Environment variables are only set in the current shell (i.e., the current terminal tab), so you’re less likely to accidentally use a default profile unintentionally.

Cons

  • You’re always using permanent Access Keys for auth rather than Temporary Access Keys that are rotated.
  • Using MFA is complicated and error prone (however, aws-vault and aws-auth make it easier).
  • Using IAM Roles is complicated and error prone (however, aws-vault and aws-auth make it easier).

Conclusion

Environment Variables offer more security than the Credentials File, but at the cost of usability, especially if you’re using IAM Roles or MFA. Fortunately, there are third party tools such as aws-vault and aws-auth to make your life a little easier.

In the next part of the series, we’ll talk about an alternative to Environment Variables that you can use for apps running in EC2: Authenticating to AWS with Instance Metadata.

Your entire infrastructure. Defined as code. In about a day. Gruntwork.io.