This folder contains scripts you can use to work with EBS Volumes that persist between redeploys of an EC2 instance:
mount-ebs-volume: This script can be run on an EC2 instance to attach and mount a persistent EBS volume. It uses
the AWS API to attach the volume, creates a file system on it (if it doesn't have one already), and mounts it at a
specified mount point. If you're mounting volumes dynamically, such as with instances in an Auto Scaling Group, you
can pass in multiple volume ids and this script will try to attach each one until it finds one in the same
availability zone that is unattached.
unmount-ebs-volume: This script can be run on an EC2 instance to unmount and detach a persistent EBS volume. It
unmounts the volume at the specified mount point and then uses the AWS API to detach the volume.
Why not use the aws_ebs_volume and aws_volume_attachment resources in Terraform to do this for us? Because:
Due to a bug, the
aws_volume_attachment resource does not work correctly and cannot currently be used.
The aws_volume_attachment resource does not work for dynamically attaching volumes, such as with an Auto Scaling
Group.
A persistent EBS volume is useful for servers that persist data to the local hard disk and need that data to still be
available after a redeploy, such as a database like MySQL, CI server like Jenkins, or CMS like WordPress. Note, an EBS
volume can only be associated with a single EC2 Instance, so if you need the data on the disk to be shared amongst
multiple servers, check out the Amazon Elastic File System, which provides a service
built on top of NFS.
The EC2 instance must have an IAM role with permissions to list, attach, and detach volumes (see the
persistent-ebs-volume example)
The EC2 instance must have the AWS CLI and jq installed
Run the mount-ebs-volume script in the User Data of your EC2 instances so it mounts the volume at boot. Run the
unmount-ebs-volume script just before your instance shuts down (make sure nothing is writing to the volume when
running this script).
This will attach volume vol-123456 as device /dev/xvdf and then mount it at the path /data (see Device Naming on
Linux Instances for an explanation of why we
chose /dev/xvdf). For dynamic setups where you don't have a static mapping of instances to volumes, such as in an
Auto Scaling Group, you can specify the --volume-id parameter multiple times:
This tells the script to try each volume until it finds one that it can attach. Run mount-ebs-volume --help to see
all the options.
How do you use this on Nitro based instances with NVMe block devices?
If you are using a Nitro based EC2
instance, the EBS volumes
will be exposed as NVMe block devices. These devices behave differently when compared to non-NVMe based EBS block
devices. The scripts in this module support NVMe block devices, but have a few caveats around device naming.
Prerequisites
For NVMe block devices, the scripts in this module depend on the nvme command line utility to function. You can
install this using the following commands:
Debian / Ubuntu:
apt-get install nvme-cli
Amazon Linux 2 / Amazon Linux / CentOS / RHEL:
yum install nvme-cli
Background: How NVMe devices are named
When attaching an EBS volume to an EC2 instance, you need to provide AWS with a device name for the volume. This mapping
between EBS volumes and device names are used by EC2 to help ensure a consistent name for the device that can be used to
identify the EBS volume on the host OS. AWS restricts these block device names for the mapping when attaching the volume
to the instance. You can refer to the official
documentation for the available volume device
names.
On non-NVMe block devices, the device name on the OS will be the same as the device name you select when attaching the
volume. However, for NVMe block devices, the OS drivers will automatically rename the device name to be of the form
/dev/nvme[0-26]n1. Note how this form of the name is not a valid name when selecting the device name during attachment
of the EBS volume. Because this is managed by the OS driver and not AWS, the remapped name is not consistent. E.g
between instance reboots, a volume that was named as /dev/xvdh in the block device mapping, may be named
/dev/nvme1n1 in one boot, while it might be named to /dev/nvme2n1 in another boot. To ensure the correct device is
discoverable for mounting, AWS will tag the volume with the EBS volume ID and block device mapping name. You can read
more about how to identify the right volume in the official
documentation.
What name to use for the persistent-ebs-volume scripts
Because the remapped name (/dev/nvme[0-26]n1) is inconsistent, you can not rely on that name when mounting and
unmounting the block device, as you may end up mounting the wrong volume at the specified location. To handle this,
both mount-ebs-volume and unmount-ebs-volume expect the name used in the block device mapping (/dev/sd[f-p])
instead of the NVMe name, which will be consistent across reboots. The scripts will then search for the correct block
device that is tagged with this device name, and correctly mount the device using the remapped name.
So for example, suppose you had a volume in region us-east-1 with id vol-123456 that you want to attach to the
instance using the block device mapping name /dev/sdf and mount to the directory /data. Here is the call to
mount-ebs-volume to do this:
Like non-NVMe devices, this will attach volume vol-123456 to the EC2 instance using the name /dev/sdf. However, when
mounting, instead of using the device name directly, the script will first identify which block device on the file
system maps to the block device mapping name /dev/sdf. Then, it will use the actual device name when mounting to
/data.
When you run this, you might see output similar to below in the logs:
<pre>
2019-02-20 04:35:23 [INFO] [mount-ebs-volume] Searching for EBS volumes with ids: vol-123456
2019-02-20 04:35:27 [INFO] [mount-ebs-volume] Filtering EBS volumes for those that are in us-east-1f and are unattached
2019-02-20 04:35:27 [INFO] [mount-ebs-volume] Attempting to attach volume vol-123456 with device name /dev/sdf
{
"AttachTime": "2019-02-20T04:35:27.974Z",
"InstanceId": "i-123456",
"VolumeId": "vol-123456",
"State": "attaching",
"Device": "/dev/sdf"
}
2019-02-20 04:35:28 [INFO] [mount-ebs-volume] Volume vol-123456 is now mounted.
2019-02-20 04:35:28 [INFO] [mount-ebs-volume] Waiting for volume sdf to be attached...
2019-02-20 04:35:28 [INFO] [mount-ebs-volume] Volume sdf is still not attached. Will retry after 10 seconds...
2019-02-20 04:35:38 [INFO] [mount-ebs-volume] Volume <b>nvme1n1</b> is now attached.
2019-02-20 04:35:38 [INFO] [mount-ebs-volume] Creating ext4 file system on <b>/dev/nvme1n1</b>...
mke2fs 1.42.13 (17-May-2015)
Creating filesystem with 1310720 4k blocks and 327680 inodes
Filesystem UUID: f920e310-b7d4-4e47-98db-14246acf63aa
Superblock backups stored on blocks:
32768, 98304, 163840, 229376, 294912, 819200, 884736
Allocating group tables: done
Writing inode tables: done
Creating journal (32768 blocks): done
Writing superblocks and filesystem accounting information: done
2019-02-20 04:35:48 [INFO] [mount-ebs-volume] Creating mount point /data...
2019-02-20 04:35:48 [INFO] [mount-ebs-volume] <b>Adding device /dev/nvme1n1 to /etc/fstab, using the UUID: UUID=f920e310-b7d4-4e47-98db-14246acf63aa with mount point /data...</b>
2019-02-20 04:35:48 [INFO] [mount-ebs-volume] Mounting volumes...
2019-02-20 04:35:48 [INFO] [mount-ebs-volume] Changing ownership of /data to ubuntu...
2019-02-20 04:35:48 [INFO] [mount-ebs-volume] Success! Volume mounted with the following details:
2019-02-20 04:35:48 [INFO] [mount-ebs-volume] <b>device_name=/dev/nvme1n1</b>
2019-02-20 04:35:48 [INFO] [mount-ebs-volume] mount_point=/data
</pre>
Note how the volume name changes from sdf to nvme1n1 part way through the logs (bolded for your convenience). This
is because after the volume is attached, the script identifies the name used by the OS and continues its steps to mount
the device using that name.
Like the mount-ebs-volume script, unmount-ebs-volume expects the device name used in the mapping when mounting the
device and not the remapped name.
When you run this, it will produce output similar to below:
<pre>
2019-02-20 04:36:52 [INFO] [unmount-ebs-volume] Unmounting <b>/dev/nvme1n1</b> at /data...
2019-02-20 04:36:52 [INFO] [unmount-ebs-volume] Using AWS CLI to detach volume vol-123456 with device name /dev/sdf from instance i-123456 in us-east-1...
{
"AttachTime": "2019-02-20T04:35:27.000Z",
"InstanceId": "i-123456",
"VolumeId": "vol-123456",
"State": "detaching",
"Device": "/dev/sdf"
}
2019-02-20 04:36:53 [INFO] [unmount-ebs-volume] Success! Volume vol-0a47d0c5bcb074299 has been unmounted from /data and detached from device /dev/sdf.
</pre>
Note how the script uses the device name /dev/nvme1n1 when unmounting the device, but switches to /dev/sdf when
detaching the device.
Questions? Ask away.
We're here to talk about our services, answer any questions, give advice, or just to chat.
{"treedata":{"name":"root","toggled":true,"children":[{"name":".circleci","children":[{"name":"config.yml","path":".circleci/config.yml","sha":"b210516f03c95b2a526a67150d18f6219f544aec"}]},{"name":".gitignore","path":".gitignore","sha":"6c4ebe4426586b7febbaba178294ef59b8272c05"},{"name":"CODEOWNERS","path":"CODEOWNERS","sha":"7769a004705df41853017db41c273a1ae668fe49"},{"name":"LICENSE.txt","path":"LICENSE.txt","sha":"f4e3d9bd4717a044ed31ad847a300eee74371a78"},{"name":"README.md","path":"README.md","sha":"bb749b57c528b40496c43682f05a977340840644"},{"name":"examples","children":[{"name":"attach-eni","children":[{"name":"README.md","path":"examples/attach-eni/README.md","sha":"86ce148cc900ce48f3debc490ce771ff5cd9fded"},{"name":"main.tf","path":"examples/attach-eni/main.tf","sha":"624da18516f8eb9556025ab278d40e44554704a7"},{"name":"outputs.tf","path":"examples/attach-eni/outputs.tf","sha":"cf9472109a00b477ec4eb950e5b4e5ce9d21f4d1"},{"name":"packer","children":[{"name":"build.json","path":"examples/attach-eni/packer/build.json","sha":"289179e20edb0b228f886123ec890137f193e4a6"}]},{"name":"user-data","children":[{"name":"user-data-1.sh","path":"examples/attach-eni/user-data/user-data-1.sh","sha":"978ca952d47f3dcd84881deb3723614339edbd6f"},{"name":"user-data-2.sh","path":"examples/attach-eni/user-data/user-data-2.sh","sha":"898fd7a3c7179cc0c6d595b6ccbb4b92b1b2cedb"}]},{"name":"vars.tf","path":"examples/attach-eni/vars.tf","sha":"c8bd5aa7b155cf011bd03e3c6bf76a79fd1be4d5"}]},{"name":"bastion-host","children":[{"name":"README.md","path":"examples/bastion-host/README.md","sha":"29868dbc60695e2418906c568734eec22a411454"},{"name":"main.tf","path":"examples/bastion-host/main.tf","sha":"0705506a64c05e1def52aa4436c66004c1e68d72"},{"name":"outputs.tf","path":"examples/bastion-host/outputs.tf","sha":"5f0c2d739b9646c39e19bfd05cacb852c6080c0c"},{"name":"user-data","children":[{"name":"user-data.sh","path":"examples/bastion-host/user-data/user-data.sh","sha":"1b42f0578f9db505bca7e5ae649a116818cd5592"}]},{"name":"vars.tf","path":"examples/bastion-host/vars.tf","sha":"3864d03b7a4035b0c6e41d35f8829b886790f282"}]},{"name":"persistent-ebs-volume","children":[{"name":"README.md","path":"examples/persistent-ebs-volume/README.md","sha":"7acf321a8a3183bfe4d7187be94208ed281bcd66"},{"name":"main.tf","path":"examples/persistent-ebs-volume/main.tf","sha":"4f63a52e643559cef221b6b0cee31f334da6f7bd"},{"name":"outputs.tf","path":"examples/persistent-ebs-volume/outputs.tf","sha":"54f33c96b796373b12b9702f46d30a1c85043f65"},{"name":"packer","children":[{"name":"build.json","path":"examples/persistent-ebs-volume/packer/build.json","sha":"2af311a5e22546182a61d3506c93648ddc9e2299"},{"name":"nvme_build.json","path":"examples/persistent-ebs-volume/packer/nvme_build.json","sha":"f67e4760622729ab762788408ef8a750dfead67c"}]},{"name":"user-data","children":[{"name":"user-data.sh","path":"examples/persistent-ebs-volume/user-data/user-data.sh","sha":"6793c2066296ff24917f6a86c43ad9674dd53624"}]},{"name":"vars.tf","path":"examples/persistent-ebs-volume/vars.tf","sha":"6f26667744846a28777af25bb3b311b1bf34a1bf"}]},{"name":"route53-helpers","children":[{"name":"README.md","path":"examples/route53-helpers/README.md","sha":"e302ef0f18f85a3a191105dea785005685b9522f"},{"name":"main.tf","path":"examples/route53-helpers/main.tf","sha":"63f89d02af52ada2ae60dd5696f83d0ce2c9df3a"},{"name":"outputs.tf","path":"examples/route53-helpers/outputs.tf","sha":"4c3c3ccccf964155ffbce7546fcbc9ef6e886de6"},{"name":"packer","children":[{"name":"build.json","path":"examples/route53-helpers/packer/build.json","sha":"b419e0ef86077ce8149908febbf7049c5e7eb659"}]},{"name":"user-data","children":[{"name":"user-data.sh","path":"examples/route53-helpers/user-data/user-data.sh","sha":"7f4de426ead32d966fe46d7091a1d293cb3ca099"}]},{"name":"vars.tf","path":"examples/route53-helpers/vars.tf","sha":"3c990b1e4ae4da54aac558cf24124e29cd362931"}]}]},{"name":"modules","children":[{"name":"attach-eni","children":[{"name":"README.md","path":"modules/attach-eni/README.md","sha":"baa975b1c3bf21ec9ed61792f983ae2cd889570e"},{"name":"bin","children":[{"name":"attach-eni","path":"modules/attach-eni/bin/attach-eni","sha":"451ce2e389f74b988a0a3bdc4a1ac28fd150a8e3"}]},{"name":"install.sh","path":"modules/attach-eni/install.sh","sha":"542cbd93ca08fd9cbe6f121f4d8fea2b1d76ca8b"}]},{"name":"persistent-ebs-volume","children":[{"name":"README.md","path":"modules/persistent-ebs-volume/README.md","sha":"d1c5af199f667269db707427f7aa4df745242a98","toggled":true},{"name":"bin","children":[{"name":"mount-ebs-volume","path":"modules/persistent-ebs-volume/bin/mount-ebs-volume","sha":"46cc65171354b261affdccde92f846d43b39268a"},{"name":"unmount-ebs-volume","path":"modules/persistent-ebs-volume/bin/unmount-ebs-volume","sha":"2eddbccd97812577f68964e3903a46953611a270"}]},{"name":"install.sh","path":"modules/persistent-ebs-volume/install.sh","sha":"e0ce5862cd6975992dc011a6dfe94fdf14a9b607"}],"toggled":true},{"name":"route53-helpers","children":[{"name":"README.md","path":"modules/route53-helpers/README.md","sha":"5db9fd7c46f8ea3abdaf7b58da18a22a09762e9e"},{"name":"bin","children":[{"name":"add-dns-a-record","path":"modules/route53-helpers/bin/add-dns-a-record","sha":"7195a02479864ec79071cfae429bd545bf7d8d32"}]},{"name":"install.sh","path":"modules/route53-helpers/install.sh","sha":"535ed35d64611c5f12e9924b3cb8df0a77180ab8"}]},{"name":"single-server","children":[{"name":"README.md","path":"modules/single-server/README.md","sha":"e13edaea6a6642708e8225a236c5a5cd24f9210f"},{"name":"main.tf","path":"modules/single-server/main.tf","sha":"fa5cfba31e449d322027940353c890dd61dafb17"},{"name":"outputs.tf","path":"modules/single-server/outputs.tf","sha":"d741f20d4df1b70b29f3261ef14b8636248c2738"},{"name":"vars.tf","path":"modules/single-server/vars.tf","sha":"15f8bf8f812eff41231df0be6a62930b17a59d39"}]}],"toggled":true},{"name":"test","children":[{"name":"Gopkg.lock","path":"test/Gopkg.lock","sha":"0fa4c35d5eadc7dfc6d357761de6573e1b22c5c5"},{"name":"Gopkg.toml","path":"test/Gopkg.toml","sha":"2d99de7b5a81ebd0fab05a6f77de4aa01d80b7a7"},{"name":"README.md","path":"test/README.md","sha":"ef26d3851db2fff0b36dfa61379724c0db9ff281"},{"name":"attach_eni_test.go","path":"test/attach_eni_test.go","sha":"9f09501b7854ae6853a96810042b1b0be794a997"},{"name":"bastion_host_test.go","path":"test/bastion_host_test.go","sha":"f3ab20153e5d5ce657e054a3fec6a647c914e645"},{"name":"persistent_ebs_volume_test.go","path":"test/persistent_ebs_volume_test.go","sha":"d03eab0c1940e15396b047e24d12b4ac8b0f54b1"},{"name":"route53_helpers_test.go","path":"test/route53_helpers_test.go","sha":"ec0f7a355075ff7d4362203b94abcc5fffaf0c4b"},{"name":"test_helpers.go","path":"test/test_helpers.go","sha":"8709fe4e7bd2d6112fe31a6ce58b62d76c8267fc"}]}]},"detailsContent":"<h1 class=\"preview__body--title\" id=\"persistent-ebs-volume-scripts\">Persistent EBS Volume Scripts</h1><div class=\"preview__body--border\"></div><p>This folder contains scripts you can use to work with EBS Volumes that persist between redeploys of an EC2 instance:</p>\n<ol>\n<li><code>mount-ebs-volume</code>: This script can be run on an EC2 instance to attach and mount a persistent EBS volume. It uses\nthe AWS API to attach the volume, creates a file system on it (if it doesn't have one already), and mounts it at a\nspecified mount point. If you're mounting volumes dynamically, such as with instances in an Auto Scaling Group, you\ncan pass in multiple volume ids and this script will try to attach each one until it finds one in the same\navailability zone that is unattached.</li>\n<li><code>unmount-ebs-volume</code>: This script can be run on an EC2 instance to unmount and detach a persistent EBS volume. It\nunmounts the volume at the specified mount point and then uses the AWS API to detach the volume.</li>\n</ol>\n<p>Why not use the <code>aws_ebs_volume</code> and <code>aws_volume_attachment</code> resources in Terraform to do this for us? Because:</p>\n<ol>\n<li><a href=\"https://github.com/hashicorp/terraform/issues/2957#issuecomment-150613677\" class=\"preview__body--description--blue\" target=\"_blank\">Due to a bug</a>, the\n<code>aws_volume_attachment</code> resource does not work correctly and cannot currently be used.</li>\n<li>The <code>aws_volume_attachment</code> resource does not work for dynamically attaching volumes, such as with an Auto Scaling\nGroup.</li>\n</ol>\n<p>A persistent EBS volume is useful for servers that persist data to the local hard disk and need that data to still be\navailable after a redeploy, such as a database like MySQL, CI server like Jenkins, or CMS like WordPress. Note, an EBS\nvolume can only be associated with a single EC2 Instance, so if you need the data on the disk to be shared amongst\nmultiple servers, check out the <a href=\"https://aws.amazon.com/efs/\" class=\"preview__body--description--blue\" target=\"_blank\">Amazon Elastic File System</a>, which provides a service\nbuilt on top of NFS.</p>\n<p>Check out the <a href=\"/repos/v0.8.2/module-server/examples/persistent-ebs-volume\" class=\"preview__body--description--blue\">persistent-ebs-volume example</a> for how to use these scripts with\nTerraform.</p>\n<h2 class=\"preview__body--subtitle\" id=\"installing-the-scripts\">Installing the scripts</h2>\n<p>You can install these scripts using the <a href=\"/repos/gruntwork-installer\" class=\"preview__body--description--blue\">Gruntwork Installer</a>:</p>\n<pre><span class=\"hljs-string\">gruntwork-install </span><span class=\"hljs-built_in\">--module-name</span> <span class=\"hljs-string\">\"persistent-ebs-volume\"</span> <span class=\"hljs-built_in\">--repo</span> <span class=\"hljs-string\">\"https://github.com/gruntwork-io/module-server\"</span> <span class=\"hljs-built_in\">--tag</span> <span class=\"hljs-string\">\"0.0.3\"</span>\n</pre>\n<h2 class=\"preview__body--subtitle\" id=\"using-the-scripts\">Using the scripts</h2>\n<p>The scripts have the following prerequisites:</p>\n<ol>\n<li>They must be run as root</li>\n<li>They must be run on an EC2 instance</li>\n<li>The EC2 instance must have an IAM role with permissions to list, attach, and detach volumes (see the\n<a href=\"/repos/v0.8.2/module-server/examples/persistent-ebs-volume\" class=\"preview__body--description--blue\">persistent-ebs-volume example</a>)</li>\n<li>The EC2 instance must have the AWS CLI and jq installed</li>\n</ol>\n<p>Run the <code>mount-ebs-volume</code> script in the User Data of your EC2 instances so it mounts the volume at boot. Run the\n<code>unmount-ebs-volume</code> script just before your instance shuts down (make sure nothing is writing to the volume when\nrunning this script).</p>\n<p>Here is an example usage:</p>\n<pre>mount-ebs-volume <span class=\"hljs-params\">--aws-region</span> us-east-1 <span class=\"hljs-params\">--volume-id</span> vol-123456 <span class=\"hljs-params\">--device-name</span> <span class=\"hljs-string\">/dev/xvdf</span> <span class=\"hljs-params\">--mount-point</span> <span class=\"hljs-string\">/data</span> <span class=\"hljs-params\">--owner</span> ubuntu\n</pre>\n<p>This will attach volume <code>vol-123456</code> as device <code>/dev/xvdf</code> and then mount it at the path <code>/data</code> (see <a href=\"http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/device_naming.html\" class=\"preview__body--description--blue\" target=\"_blank\">Device Naming on\nLinux Instances</a> for an explanation of why we\nchose <code>/dev/xvdf</code>). For dynamic setups where you don't have a static mapping of instances to volumes, such as in an\nAuto Scaling Group, you can specify the <code>--volume-id</code> parameter multiple times:</p>\n<pre>mount-ebs-volume <span class=\"hljs-params\">--aws-region</span> us-east-1 <span class=\"hljs-params\">--volume-id</span> vol-123456 <span class=\"hljs-params\">--volume-id</span> vol-7891011 <span class=\"hljs-params\">--volume-id</span> vol-12131415 <span class=\"hljs-params\">--device-name</span> <span class=\"hljs-string\">/dev/xvdf</span> <span class=\"hljs-params\">--mount-point</span> <span class=\"hljs-string\">/data</span> <span class=\"hljs-params\">--owner</span> ubuntu\n</pre>\n<p>This tells the script to try each volume until it finds one that it can attach. Run <code>mount-ebs-volume --help</code> to see\nall the options.</p>\n<h2 class=\"preview__body--subtitle\" id=\"how-do-you-use-this-on-nitro-based-instances-with-nv-me-block-devices\">How do you use this on Nitro based instances with NVMe block devices?</h2>\n<p>If you are using <a href=\"https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/instance-types.html#ec2-nitro-instances\" class=\"preview__body--description--blue\" target=\"_blank\">a Nitro based EC2\ninstance</a>, the EBS volumes\nwill be exposed as NVMe block devices. These devices behave differently when compared to non-NVMe based EBS block\ndevices. The scripts in this module support NVMe block devices, but have a few caveats around device naming.</p>\n<h3 class=\"preview__body--subtitle\" id=\"prerequisites\">Prerequisites</h3>\n<p>For NVMe block devices, the scripts in this module depend on the <code>nvme</code> command line utility to function. You can\ninstall this using the following commands:</p>\n<p><strong>Debian / Ubuntu</strong>:</p>\n<pre>apt-<span class=\"hljs-builtin-name\">get</span> install nvme-cli\n</pre>\n<p><strong>Amazon Linux 2 / Amazon Linux / CentOS / RHEL</strong>:</p>\n<pre>yum install nvme-<span class=\"hljs-keyword\">cli</span>\n</pre>\n<h3 class=\"preview__body--subtitle\" id=\"background-how-nv-me-devices-are-named\">Background: How NVMe devices are named</h3>\n<p>When attaching an EBS volume to an EC2 instance, you need to provide AWS with a device name for the volume. This mapping\nbetween EBS volumes and device names are used by EC2 to help ensure a consistent name for the device that can be used to\nidentify the EBS volume on the host OS. AWS restricts these block device names for the mapping when attaching the volume\nto the instance. You can refer to <a href=\"https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/device_naming.html\" class=\"preview__body--description--blue\" target=\"_blank\">the official\ndocumentation</a> for the available volume device\nnames.</p>\n<p>On non-NVMe block devices, the device name on the OS will be the same as the device name you select when attaching the\nvolume. However, for NVMe block devices, the OS drivers will automatically rename the device name to be of the form\n<code>/dev/nvme[0-26]n1</code>. Note how this form of the name is not a valid name when selecting the device name during attachment\nof the EBS volume. Because this is managed by the OS driver and not AWS, the remapped name is not consistent. E.g\nbetween instance reboots, a volume that was named as <code>/dev/xvdh</code> in the block device mapping, may be named\n<code>/dev/nvme1n1</code> in one boot, while it might be named to <code>/dev/nvme2n1</code> in another boot. To ensure the correct device is\ndiscoverable for mounting, AWS will tag the volume with the EBS volume ID and block device mapping name. You can read\nmore about how to identify the right volume in <a href=\"https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/nvme-ebs-volumes.html#identify-nvme-ebs-device\" class=\"preview__body--description--blue\" target=\"_blank\">the official\ndocumentation</a>.</p>\n<h3 class=\"preview__body--subtitle\" id=\"what-name-to-use-for-the-persistent-ebs-volume-scripts\">What name to use for the persistent-ebs-volume scripts</h3>\n<p>Because the remapped name (<code>/dev/nvme[0-26]n1</code>) is inconsistent, you can not rely on that name when mounting and\nunmounting the block device, as you may end up mounting the wrong volume at the specified location. To handle this,\nboth <code>mount-ebs-volume</code> and <code>unmount-ebs-volume</code> expect the name used in the block device mapping (<code>/dev/sd[f-p]</code>)\ninstead of the NVMe name, which will be consistent across reboots. The scripts will then search for the correct block\ndevice that is tagged with this device name, and correctly mount the device using the remapped name.</p>\n<p>So for example, suppose you had a volume in region <code>us-east-1</code> with id <code>vol-123456</code> that you want to attach to the\ninstance using the block device mapping name <code>/dev/sdf</code> and mount to the directory <code>/data</code>. Here is the call to\n<code>mount-ebs-volume</code> to do this:</p>\n<pre>mount-ebs-volume <span class=\"hljs-params\">--aws-region</span> us-east-1 <span class=\"hljs-params\">--volume-id</span> vol-123456 <span class=\"hljs-params\">--device-name</span> <span class=\"hljs-string\">/dev/sdf</span> <span class=\"hljs-params\">--mount-point</span> <span class=\"hljs-string\">/data</span> <span class=\"hljs-params\">--owner</span> ubuntu\n</pre>\n<p>Like non-NVMe devices, this will attach volume <code>vol-123456</code> to the EC2 instance using the name <code>/dev/sdf</code>. However, when\nmounting, instead of using the device name directly, the script will first identify which block device on the file\nsystem maps to the block device mapping name <code>/dev/sdf</code>. Then, it will use the actual device name when mounting to\n<code>/data</code>.</p>\n<p>When you run this, you might see output similar to below in the logs:</p>\n<p><pre>\n2019-02-20 04:35:23 [INFO] [mount-ebs-volume] Searching for EBS volumes with ids: vol-123456\n2019-02-20 04:35:27 [INFO] [mount-ebs-volume] Filtering EBS volumes for those that are in us-east-1f and are unattached\n2019-02-20 04:35:27 [INFO] [mount-ebs-volume] Attempting to attach volume vol-123456 with device name /dev/sdf\n{\n"AttachTime": "2019-02-20T04:35:27.974Z",\n"InstanceId": "i-123456",\n"VolumeId": "vol-123456",\n"State": "attaching",\n"Device": "/dev/sdf"\n}\n2019-02-20 04:35:28 [INFO] [mount-ebs-volume] Volume vol-123456 is now mounted.\n2019-02-20 04:35:28 [INFO] [mount-ebs-volume] Waiting for volume sdf to be attached...\n2019-02-20 04:35:28 [INFO] [mount-ebs-volume] Volume sdf is still not attached. Will retry after 10 seconds...\n2019-02-20 04:35:38 [INFO] [mount-ebs-volume] Volume <b>nvme1n1</b> is now attached.\n2019-02-20 04:35:38 [INFO] [mount-ebs-volume] Creating ext4 file system on <b>/dev/nvme1n1</b>...\nmke2fs 1.42.13 (17-May-2015)\nCreating filesystem with 1310720 4k blocks and 327680 inodes\nFilesystem UUID: f920e310-b7d4-4e47-98db-14246acf63aa\nSuperblock backups stored on blocks:\n32768, 98304, 163840, 229376, 294912, 819200, 884736</p>\n<p>Allocating group tables: done\nWriting inode tables: done\nCreating journal (32768 blocks): done\nWriting superblocks and filesystem accounting information: done</p>\n<p>2019-02-20 04:35:48 [INFO] [mount-ebs-volume] Creating mount point /data...\n2019-02-20 04:35:48 [INFO] [mount-ebs-volume] <b>Adding device /dev/nvme1n1 to /etc/fstab, using the UUID: UUID=f920e310-b7d4-4e47-98db-14246acf63aa with mount point /data...</b>\n2019-02-20 04:35:48 [INFO] [mount-ebs-volume] Mounting volumes...\n2019-02-20 04:35:48 [INFO] [mount-ebs-volume] Changing ownership of /data to ubuntu...\n2019-02-20 04:35:48 [INFO] [mount-ebs-volume] Success! Volume mounted with the following details:\n2019-02-20 04:35:48 [INFO] [mount-ebs-volume] <b>device_name=/dev/nvme1n1</b>\n2019-02-20 04:35:48 [INFO] [mount-ebs-volume] mount_point=/data\n</pre></p>\n<p>Note how the volume name changes from <code>sdf</code> to <code>nvme1n1</code> part way through the logs (bolded for your convenience). This\nis because after the volume is attached, the script identifies the name used by the OS and continues its steps to mount\nthe device using that name.</p>\n<p>Similarly, to unmount the device, you can use:</p>\n<pre>unmount-ebs-volume <span class=\"hljs-params\">--aws-region</span> us-east-1 <span class=\"hljs-params\">--volume-id</span> vol-123456 <span class=\"hljs-params\">--device-name</span> <span class=\"hljs-string\">/dev/sdf</span> <span class=\"hljs-params\">--mount-point</span> <span class=\"hljs-string\">/data</span>\n</pre>\n<p>Like the <code>mount-ebs-volume</code> script, <code>unmount-ebs-volume</code> expects the device name used in the mapping when mounting the\ndevice and not the remapped name.</p>\n<p>When you run this, it will produce output similar to below:</p>\n<p><pre>\n2019-02-20 04:36:52 [INFO] [unmount-ebs-volume] Unmounting <b>/dev/nvme1n1</b> at /data...\n2019-02-20 04:36:52 [INFO] [unmount-ebs-volume] Using AWS CLI to detach volume vol-123456 with device name /dev/sdf from instance i-123456 in us-east-1...\n{\n"AttachTime": "2019-02-20T04:35:27.000Z",\n"InstanceId": "i-123456",\n"VolumeId": "vol-123456",\n"State": "detaching",\n"Device": "/dev/sdf"\n}\n2019-02-20 04:36:53 [INFO] [unmount-ebs-volume] Success! Volume vol-0a47d0c5bcb074299 has been unmounted from /data and detached from device /dev/sdf.\n</pre></p>\n<p>Note how the script uses the device name <code>/dev/nvme1n1</code> when unmounting the device, but switches to <code>/dev/sdf</code> when\ndetaching the device.</p>\n","repoName":"module-server","repoRef":"v0.7.7","serviceDescriptor":{"serviceName":"Single EC2 Instance","serviceRepoName":"module-server","serviceRepoOrg":"gruntwork-io","cloudProviders":["aws"],"description":"Run a single EC2 instance for stateless or stateful apps. Supports IAM roles, EBS volumes, ENIs, and EIPs.","imageUrl":"single-service.png","licenseType":"subscriber","technologies":["Terraform","Bash"],"compliance":[],"tags":[""]},"serviceCategoryName":"Server orchestration","fileName":"README.md","filePath":"/modules/persistent-ebs-volume","title":"Repo Browser: Single EC2 Instance","description":"Browse the repos in the Gruntwork Infrastructure as Code Library."}
.circleci
install.sh
