Gruntwork Newsletter, July 2018

Once a month, we send out a newsletter to all Gruntwork customers that describes all the updates we’ve made in the last month, news in the DevOps industry, and important security updates. Note that many of the links below go to private repos in the Gruntwork Infrastructure as Code Library and Reference Architecture that are only accessible to customers.

Hello Grunts,

In the last month, we launched the beta of Gruntwork Houston, which we believe offers a fundamentally better DevOps experience, added support for NVMe volumes, fixed some important bugs in package-openvpn, and made lots of improvements to Terragrunt and Terratest. In other news, Terraform 0.12 preview is available, and Consul 1.2 adds service mesh support.

As always, if you have any questions or need help, email us at support@gruntwork.io!

• Gruntwork Updates
• DevOps News
• Security Updates

Gruntwork Updates

Gruntwork Houston, a fundamentally better DevOps experience, is in beta!

Motivation: Modern DevOps is the story of death by a thousand cuts. You have countless tools to manage — including AWS and all of its separate services (EC2, ECS, RDS, CloudWatch, etc), GitHub, Jenkins, OpenVPN, Terraform, Docker, Packer, DataDog, Loggly, PagerDuty, and so on — and there’s no coherent user experience that ties them all together. Before, your choices were to either use a Platform as a Service (PaaS) that gave you a nice UI, but no ability to control or customize anything, or to use Infrastructure as Code (IaC), which gave you full control and power over everything, but no nice UI.

Solution: Introducing, Gruntwork Houston!

Gruntwork Houston gives you DevOps super powers. On the surface, it’s a simple web interface that your Dev team can use to deploy and manage infrastructure. Under the hood, the web interface and how it manages infrastructure are completely defined and controlled by your Ops team using infrastructure as code.

It’s the best of both worlds: your Dev team gets an easy-to-use, self-service experience, while your Ops team still has all the power and control they need to ensure reliability, security, and compliance. Best of all, Houston runs in your own AWS account (so it can securely access your infrastructure) and is included in the Gruntwork Subscription for no extra fee!

We’re currently testing out Gruntwork Houston with a few customers in a private beta. Some of the first features we’ve released include single sign-on (SSO) that allows you to login to any of your AWS accounts using any SAML provider, including Google, ADFS, and Okta. SSO with Houston works for the AWS web console, AWS CLI tools, VPN access, and SSH access. Here’s a screen capture that shows you just how much easier it is to authenticate to AWS from the CLI using Houston:

What to do about it: Check out the Gruntwork Houston announcement blog post for more details, including more screenshots and videos. If you’re interested in joining the waiting list, email us at info@gruntwork.io.

ssh-iam has been refactored into ssh-grunt

Motivation: Our customers wanted to use external Identity Providers (IdP’s), such as Google and ADFS, to access their AWS accounts, including accessing their servers over SSH. However, ssh-iam only supported IAM as an IdP.

Solution: We have refactored ssh-iam and renamed it to ssh-grunt so that we can use it with all the IdP’s supported by Gruntwork Houston! That means you can manage SSH access to your EC2 Instances using roles in your SAML IdP, such as Google, ADFS, or Okta. Team members with the appropriate SSH Roles will be able to upload their publish SSH keys to Houston and then use their own username and that SSH key to SSH to EC2 Instances.

What to do: Check out the release notes in module-security, v0.13.0 for how to upgrade to ssh-grunt and see module-security, v0.14.0 for the latest version. If you’d like to start using Gruntwork Houston, email us at info@gruntwork.io.

You can now mount NVMe volumes

Motivation: AWS has launched a number of new instance types (C5, C5d, i3.metal, M5, and M5d) that use NVMe block devices, which require extra logic to mount properly in Linux.

Solution: We’ve updated the mount-ebs-volume script with support for NVMe block devices!

What to do: Update to module-server, v0.5.0 and use the mount-ebs-volume with all your NVMe block devices.

package-openvpn fixes

Motivation: We added several new features and fixed several important bugs in package-openvpn.

Solution: Here are the new releases from the last month:

• package-openvpn, v0.6.0: The supervisor install has been moved from the run-process-requests and run-process-revokes scripts to the install-openvpn script where it belongs. You'll need to build a new OpenVPN AMI to take advantage of this change.
• package-openvpn, v0.6.1: You can now configure DNS search domains for your VPN server using the --search-domainoption. That way, internal domain names (e.g., foo.acme.internal) will resolve properly while connected over VPN.
• package-openvpn, v0.7.0: Fix a bug with how the init-openvpn script configures the PKI backup cron job. This is an important fix, so we recommend upgrading. Default iops to 0 in the openvpn-server module. This is to work around a Terraform bug.
• package-openvpn, v0.7.1: Fix a bug with in init-openvpn with a kernel-level parameter not being persisted across reboots. This caused DNS resolution across the VPN from working after a reboot.

What to do about it: The init-openvpn fixes are important, so we strongly recommend updating to package-openvpn, v0.7.1.

Terragrunt updates

Motivation: Terragrunt usage is growing quickly (it has over 1,400 stars on GitHub!), so we are beginning to invest in improving its user experience.

Solution: We’ve added a number of new features and fixed a bunch of bugs in the last month:

• Terragrunt, v0.14.11: If you are using the S3 backend with DynamoDB for locking, and the S3 bucket or DynamoDB table doesn’t exist, Terragrunt will now not only automatically create them for you, but can also optionally apply tags to them via the s3_bucket_tags and dynamodb_table_tagsparameters.
• Terragrunt, v0.15.0: Reverse the “yes/no” behavior for how dependencies are handled in xxx-all commands to make it more intuitive.
• Terragrunt, v0.15.1: Fix the NeedsInit method so if you specify s3_bucket_tags or dynamodb_table_tags, Terragrunt doesn't try to re-run init every time.
• Terragrunt, v0.15.2: Fix potential nil pointer dereference introduced in v0.15.1.
• Terragrunt, v0.15.3: Fix the Terragrunt cache dir on Windows to be in the $HOME dir rather than temp.
• Terragrunt, v0.16.0: A number of improvements to how Terragrunt downloads remote configurations that should make things faster, clearer, and easier to debug.
• Terragrunt, v0.16.1: Make sure the xxx-all commands (e.g., apply-all) don't accidentally try to run Terragrunt in a .terragrunt-cache directory.

What to do about it: Upgrade to the latest release of Terragrunt and keep your eyes open for many more improvements in the next few months.

Terratest improvements

Terratest is also growing in popularity on GitHub (over 1,100 stars!), and the community has contributed some great new features this month:

• Terratest, v0.9.7: You can now use the ScpFileTo and ScpFileToE functions to SCP files to servers.
• Terratest, v0.9.8: Terratest will now log the SSH command it is executing before executing it to help debugging.
• Terratest, v0.9.9: You can now use the DeleteEbsSnapshot/DeleteEbsSnapshotE methods to delete EBS snapshots of your AMIs, in addition to deregistering the AMIs with the DeleteAmi/DeleteAmiEmethods that were there previously. You can also use the new DeleteAmiAndAllSnapshots/DeleteAmiAndAllSnapshotsE method to both deregister the AMIs and delete the snapshots.
• Terratest, v0.9.10: Fix bugs with how Terratest coerced boolean, list, and map arguments passed to Terraform via the -var option.
• Terratest, v0.9.11: You can now use the AddTagsToResource/AddTagsToResourceE methods to tag AWS resources at test time.
• Terratest, v0.9.12: You can now set the NoColor option in terraform.Options to tell Terraform to not use color escape codes in stdout/stderr.
• Terratest, v0.9.13: You can now configure extra -backend-config arguments to pass when calling terraform init by specifying BackendConfig parameters in terraform.Options.
• Terratest, v0.9.14: Use the -auto-approve flag instead of the deprecated -force flag for Terraform commands.

Give the latest release of Terratest a shot and let us know what else we can do to make it easier to test your infrastructure!

Other updates

• bash-commons, v0.0.4: Add an os_is_redhat method to os.sh.
• module-asg, v0.6.13: Fix a bug where the server-group module would hit an error trying to create an IAM Policy for EBS volumes when the size param was set to 0.
• module-ci, v0.11.0: The git-add-commit-push script will now detect "Updates were rejected because the remote contains work that you do not have locally" errors and automatically git pull --rebase and git push in a retry loop (up to a max number of retries). This allows the script to work properly even if someone else happened to push some code to the same branch at the exact same time.
• module-ci, v0.12.0: The terraform-update-variable script used to require setting --skip-git "true", which is a non-idiomatic way to do flags in bash, and the parsing for it could fail silently. The script has now been updated so you just specify --skip-git to disable Git, without any need to say "true".
• module-security, v0.11.0: Setting should_require_mfa to false in the iam-policies module should now work correctly, allowing you to disable the MFA requirement. This module is used under the hood in the iam-groups, cross-account-iam-roles, and saml-iam-roles modules, so upgrade those modules if you need this fix.
• module-security, v0.12.0: The saml-iam-roles module now sets a default max expiration of 12 hours for IAM Roles intended for human users (e.g., allow-read-only-access-from-saml) and a default max expiration of 1 hour for IAM Roles intended for machine users (e.g., allow-auto-deploy-access-from-saml). Both of these expiration values are configurable via the new input variables max_session_duration_human_usersand max_session_duration_machine_users.
• terraform-aws-vault, v0.8.0: The Vault user is now marked as a system user.

DevOps News

Terraform 0.12 is coming

What happened: HashiCorp has announced a preview release of Terraform 0.12.

Why it matters: Terraform 0.12 brings a number of major changes to HCL, the language used in Terraform. Here are just a few of the highlights:

• First-class expressions. Prior to 0.12, expressions had to be wrapped in interpolation sequences with double quotes, such as "${var.foo}". With 0.12, expressions are a native part of the language and can be used directly. Example: ami = var.ami[1]
• For expressions. A for expression is available for iterating and filtering lists and map values. This expression always can be used anywhere a list or map is expected.
• Dynamic blocks. Child blocks such as rule in aws_security_group can now be dynamically generated based on lists/maps and support iteration.
• Conditional improvements. The conditional operator ... ? ... : ... now supports any value type and lazily evaluates results, as those familiar with this operator in other languages would expect.
• Nullable arguments. The special value null can now be assigned to any field to represent the absence of a value. This causes Terraform to omit the field from upstream API calls, which is important in some cases for triggering certain default behaviors.

What to do about it: For now, do nothing. Over the next few months, we will start updating all of our modules, as well as Terratest and Terragrunt, to work with Terraform 0.12. This should greatly simplify a lot of our code, but will also require a large number of backwards incompatible changes, so be prepared for code changes.

Consul 1.2 is out, adding Service Mesh functionality

What happened: HashiCorp has released Consul 1.2, which adds a major new feature called Consul Connect, which turns your Consul cluster into a service mesh.

Why it matters: A service mesh is useful in any microservices and cloud architecture where you need:

• Service Discovery: Services must be able to find each other.
• Configuration: Services need a way to find their configuration.
• Security: Service communication must be authorized and encrypted.

Up until now, Consul has offered the first two features; Consul Connect adds the third feature, giving you an easy way to enable secure service-to-service communication with automatic TLS encryption and identity-based authorization.

What to do about it: Check out the announcement blog post for all the details.

Amazon Linux 2 is now generally available

What happened: After several release candidates, Amazon Linux 2 is now generally available and comes with 5 years of Long Term Support (LTS).

Why it matters: Amazon Linux 2 is the new generation of AWS-supported Linux distribution. It includes an updated Linux Kernel (4.14), systemd support, a newer compiler (GCC 7.3), an updated C runtime (Glibc 2.26), modern tooling (Binutils 2.29.1), and more.

What to do about it: The Amazon Linux 2 AMI and Docker images are available for your use now. We have already updated a few of our modules with support for Amazon Linux 2, but we have many more to update, which we’ll be doing over the next couple months. If there is a specific module you’d like updated urgently, let us know!

Security Updates

Below is a list of critical security updates that may impact your services. We notify Gruntwork customers of these vulnerabilities as soon as we know of them via the Gruntwork Security Alerts mailing list. It is up to you to scan this list and decide which of these apply and what to do about them, but most of these are severe vulnerabilities, and we recommend patching them ASAP.

NTP

• USN-3707–1: Several vulnerabilities have been discovered in NTP that range from allowing remote attackers to crash NTP and possibly causing denial of service attack all the way to allowing a remote attacker to execute arbitrary code. If you use NTP, it’s worth following up and making sure that you are not vulnerable.