Locating AWS Marketplace AMI Owner Id and Image Name for Packer Builds

When creating immutable build AMIs using packer, we often want start with the latest version of a particular AMI from the AWS Marketplace as a base image. Unfortunately, the process of locating all the information we need on the AWS marketplace isn’t at all straightforward. In this post, I’ll walk you through all the steps you need to build an AMI from any image found on the AWS Marketplace.

For our walkthrough, let’s say we want to build a new packer image based on the latest available Ubuntu 16.04 image.

To start, let’s head over to https://aws.amazon.com/marketplace and do a quick search for our base Ubuntu image:

Once we’ve located the image we want, click on its heading to view the details page. Then, click on its Continue button.

You will be redirected to the Launch on EC2 page. From this page, we actually just want to extract the productid for the AMI from the page’s URL.

Now that we have the productid, we can use the aws cli to help us find the rest info we need to build a new packer image.

Let’s go ahead and query for images that have our productid in them:

Copy
$ aws ec2 describe-images  —-owners aws-marketplace --filters "Name=name,Values=*d83d0782-cb94–46d7–8993-f4ce15d1a484*" | jq -r '.Images[] | "\(.OwnerId)\t\(.Name)"'

This will return a list of images belonging to that productid. The reason this returns a list rather than a single AMI is that many versions of the same AMI have been published to various different regions.

Copy
679593333241 ubuntu/images/hvm-ssd/ubuntu-xenial-16.04-amd64-server-20171121.1-d83d0782-cb94-46d7-8993-f4ce15d1a484-ami-aa2ea6d0.4

Copy
679593333241 ubuntu/images/hvm-ssd/ubuntu-xenial-16.04-amd64-server-20170307-d83d0782-cb94-46d7-8993-f4ce15d1a484-ami-2757f631.4

Copy
679593333241 ubuntu/images/hvm-ssd/ubuntu-xenial-16.04-amd64-server-20160721-d83d0782-cb94-46d7-8993-f4ce15d1a484-ami-cf68e0d8.3

Copy
679593333241 ubuntu/images/hvm-ssd/ubuntu-xenial-16.04-amd64-server-20161214-d83d0782-cb94-46d7-8993-f4ce15d1a484-ami-e13739f6.3

Copy
679593333241 ubuntu/images/hvm-ssd/ubuntu-xenial-16.04-amd64-server-20170224-d83d0782-cb94-46d7-8993-f4ce15d1a484-ami-1ac0120c.4

Copy
679593333241 ubuntu/images/hvm-ssd/ubuntu-xenial-16.04-amd64-server-20161115-d83d0782-cb94-46d7-8993-f4ce15d1a484-ami-45b69e52.3

Copy
679593333241 ubuntu/images/hvm-ssd/ubuntu-xenial-16.04-amd64-server-20170619.1-d83d0782-cb94-46d7-8993-f4ce15d1a484-ami-d15a75c7.4

Copy
679593333241 ubuntu/images/hvm-ssd/ubuntu-xenial-16.04-amd64-server-20170803-d83d0782-cb94-46d7-8993-f4ce15d1a484-ami-d14062aa.4

Copy
679593333241 ubuntu/images/hvm-ssd/ubuntu-xenial-16.04-amd64-server-20170330-d83d0782-cb94-46d7-8993-f4ce15d1a484-ami-4dd2575b.4

Now, we want to grab both the owner id (first column) and the common part of the image name (without version information). So in our example, these would be:

Copy
Owner ID: 679593333241
Name: ubuntu/images/hvm-ssd/ubuntu-xenial-16.04-amd64-server-

Now we can create our packer template and use the [source_ami_filter](https://www.packer.io/docs/builders/amazon-ebs.html#source_ami_filter) functionality to find our image at build time.

Copy
{
"variables": {
"aws_access_key": "",
"aws_secret_key": ""
},
"builders": [
{
"type": "amazon-ebs",
"access_key": "{{user `aws_access_key`}}",
"secret_key": "{{user `aws_secret_key`}}",
"region": "us-east-1",
"source_ami_filter": {
"filters": {
"virtualization-type": "hvm",
"name": "ubuntu/images/hvm-ssd/ubuntu-xenial-16.04-amd64-server-*",
"root-device-type": "ebs"
},
"owners": [
"679593333241"
],
"most_recent": true
},
"instance_type": "t2.micro",
"ssh_username": "ubuntu",
"ami_name": "gruntwork-packer-example {{isotime | clean_ami_name}}"
}
]
}

And let’s go ahead and build it to make sure it works:

Copy
$ packer build gruntwork-demo.json

Copy
amazon-ebs output will be in this color.

Copy
==> amazon-ebs: Prevalidating AMI Name...
amazon-ebs: Found Image ID: ami-08c35d72
==> amazon-ebs: Creating temporary keypair: packer_5a2b6814-be01-1b45-06b3-53dde35d277c
==> amazon-ebs: Creating temporary security group for this instance: packer_5a2b6850-9445-131b-16f2-921f377fb288
==> amazon-ebs: Authorizing access to port 22 on the temporary security group...
==> amazon-ebs: Launching a source AWS instance...
==> amazon-ebs: Adding tags to source instance
amazon-ebs: Adding tag: "Name": "Packer Builder"
amazon-ebs: Instance ID: i-03bbde5691aa4f455
==> amazon-ebs: Waiting for instance (i-03bbde5691aa4f455) to become ready...
==> amazon-ebs: Waiting for SSH to become available...
==> amazon-ebs: Connected to SSH!
==> amazon-ebs: Stopping the source instance...
amazon-ebs: Stopping instance, attempt 1
==> amazon-ebs: Waiting for the instance to stop...
==> amazon-ebs: Creating the AMI: gruntwork-packer-example 2017-12-09T04-35-32Z
amazon-ebs: AMI: ami-b2dbb7c8
==> amazon-ebs: Waiting for AMI to become ready...
==> amazon-ebs: Terminating the source AWS instance...
==> amazon-ebs: Cleaning up any extra volumes...
==> amazon-ebs: No volumes to clean up, skipping
==> amazon-ebs: Deleting temporary security group...
==> amazon-ebs: Deleting temporary keypair...
Build 'amazon-ebs' finished.

Copy
==> Builds finished. The artifacts of successful builds are:
--> amazon-ebs: AMIs were created:
us-east-1: ami-b2dbb7c8

Voilà, everything looks good! But wait, there’s more! You can use this same method with the Terraform aws_ami data source too.

I’m not sure why Amazon hasn’t exposed this information on the AWS Marketplace directly, but in the meantime, I hope this helps you find what you need!

Get your DevOps superpowers at Gruntwork.io.