Authenticating to AWS with Instance Metadata

This is Part 3 of the Comprehensive Guide to Authenticating to AWS on the Command Line. In Part 2, we went over how to use Environment Variables, which were secure, but complicated to use. In this post, we’ll introduce a third option for authenticating to AWS on the Command Line: Instance Metadata.

Basic usage

What do you do if you want to authenticate to AWS from an EC2 Instance? For example, you have an app that needs to make API calls to AWS to download data from S3. Normally, you’d authenticate to AWS using Access Keys, but how do you get those Access Keys onto the EC2 Instance? Putting them directly in your application code or a config file is a bad idea, as that means your credentials will be in plain text, on disk, accessible to any attacker that manages to get access to the EC2 Instance or your code.

A better idea is to use Instance Metadata. AWS exposes an Instance Metadata endpoint on every EC2 Instance at http://169.254.169.254. If you SSH to an EC2 Instance, you can use any HTTP client, such as curl, to get information about that Instance from its Instance Metadata endpoint:

Copy
$ curl http://169.254.169.254/latest/meta-data/
ami-id
ami-launch-index
ami-manifest-path
block-device-mapping/
hostname
iam/
instance-action
instance-id
instance-type
local-hostname
local-ipv4
mac
metrics/
network/
placement/
profile
public-hostname
public-ipv4
public-keys/
reservation-id
security-groups
services/

Copy
$ curl http://169.254.169.254/latest/meta-data/public-hostname
ec2-203-0-113-25.compute-1.amazonaws.com

Part of the information available in EC2 Metadata are Temporary Access Keys that are automatically provided and rotated by AWS! The permissions that you get from these Temporary Access Keys are determined by the IAM Role (if any) attached to that EC2 Instance. You’ve seen earlier in this blog post that IAM Roles can be assumed by IAM Users to give them certain permissions; well, you can also create IAM Roles that can be assumed by various AWS Services, including EC2 Instances.

For example, you can create an IAM Role with a Trust Policy that allows an EC2 Instance to assume that IAM Role:

Copy
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": { "Service": "ec2.amazonaws.com"},
"Action": "sts:AssumeRole"
}
]
}

You can then give this IAM Role various IAM permissions, such as being able to read data from S3:

Copy
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": ["s3:GetObject"],
"Resource": ["arn:aws:s3:::examplebucket/*"]
}
]
}

Next, you can attach this IAM Role to your EC2 Instance, and the Instance Metadata endpoint will give any AWS CLI or SDK tool running on that Instance permission to read from that S3 bucket without you having to manually deal with Access Keys!

Working with multiple sets of Access Keys

With Instance Metadata and IAM Roles, you don’t manage the Access Keys yourself, so there’s nothing to do here!

Working with IAM Roles

The permissions for Instance Metadata come from the IAM Role attached to your EC2 Instance, so this is handled automatically for you.

Working with MFA

Since Instance Metadata is meant to be used by applications and automated tools, where there is no person to type in an MFA token, MFA is not supported.

ECS Task Metadata

If you are running Docker containers using Amazon’s Elastic Container Service (ECS), then you don’t want your Docker container relying on the Instance Metadata endpoint of the underlying EC2 Instance. Instead, you can use the Task Metadata endpoint, which serves the same purpose, but is exposed by ECS directly within your Docker container at http://169.254.170.2 (all AWS CLI and SDK tools know to check this endpoint).

Likewise, instead of attaching an IAM Role to your EC2 Instance, you’ll want to attach an IAM Role directly to the ECS Task using ECS Task IAM Roles. This way, you can give your Docker containers specific IAM permissions (e.g., read access to an S3 bucket) without having to manually fuss with Access Keys.

Protecting the Instance Metadata endpoint

A major drawback of the Instance Metadata endpoint is that it can be accessed by any OS user on your EC2 Instance. So if a hacker manages to break into a low-level user account, they can immediately use all of the permissions that EC2 Instance has from its IAM Role.

Therefore, we strongly recommend locking down the Instance Metadata endpoint so it is only accessible to specific OS users. You can lock down IPs on Linux Instances using iptables or via the ip-lockdown script in the Infrastructure as Code Library:

Copy
ip-lockdown 169.254.169.254 root

The command above will only allow the Instance Metadata endpoint to be accessed by the root user, which makes life much harder for attackers. Moreover, if your Instance only needs IAM permissions during boot (e.g., to download a config file from S3), you could even block access to the Instance Metadata endpoint entirely after boot has completed:

Copy
# Fetch data during boot
aws s3 cp s3:// ...

Copy
# Block the Instance Metadata endpoint entirely
ip-lockdown 169.254.169.254

Pros and cons

Pros

• You don’t have to manually manage Access Keys.
• Access Keys are never stored in plaintext on disk.
• Uses Temporary Access Keys that expire frequently and are automatically rotated by AWS.

Cons

• Only works with AWS Services (e.g., EC2 Instances).
• The Instance Metadata endpoint is vulnerable to attackers (unless you use something like ip-lockdown to protect it).

Conclusion

Thanks to Instance Metadata, you should never have to manually copy Access Keys over to your EC2 Instances. Instead, attach an IAM Role to those Instances, give that IAM Role the permissions your apps need, and everything will “just work.” However, make sure to use a tool such as ip-lockdown to protect the Instance Metadata endpoint from attackers!

We hope you’ve found this guide to AWS authentication helpful. If you’re stuck or need help, feel free to ask questions in the comments!

Get your DevOps superpowers at Gruntwork.io.