Gruntwork Newsletter, April 2018

Once a month, we send out a newsletter to all Gruntwork customers that describes all the updates we’ve made in the last month, news in the DevOps industry, and important security updates. Note that many of the links below go to private repos in the Gruntwork Infrastructure as Code Library and Reference Architecture that are only accessible to customers.

Hello Grunts,

In the last month, we created a new Network Load Balancer Module and added support for sharding and auth in our Redis module, before and after “hooks” in Terragrunt, spot pricing in the ECS cluster module, encryption and CORS in the s3 module, and a number of other fixes and updates. In the meantime, AWS has added native service discovery for ECS, the ability to assume IAM roles for up to 12 hours, continuous backup and point-in-time-restore for DynamoDB, and more. Read on for all the details.

As always, if you have any questions or need help, email us at support@gruntwork.io!

• Gruntwork Updates
• DevOps News
• Security Updates

Gruntwork Updates

New Network Load Balancer Module

Motivation: AWS recently added a new type of Load Balancer called the Network Load Balancer (or NLB for short), which is a TCP (Layer 4) load balancer that supports very high throughput, rapid scaling, and static IP addresses.

Solution: We added a new nlb module to make it easier to deploy the NLB.

What to do: If you’d like to take advantage of the scalability and throughput benefits that an NLB brings, use module-load-balancer, v0.8.0 (example usage).

Sharding and auth for Redis

Motivation: One of our customers wanted to run a Redis cluster in ElastiCache that supported (a) multiple shards and (b) user authentication, but our redis module did not yet have support for either feature.

Solution: To support sharding, we added a new cluster_mode parameter and exposed new variables to allow you to specify the number of shards, and read replicas per shard, in the cluster. We’ve also added support for authentication via an auth_token parameter (note, this can only be used when transit_encryption_enabled is set to true).

What to do: If you’d like to take advantage of the benefits of multi-sharding, use module-cache, v0.3.4 (example usage). If you’d like to enable authentication, update to module-cache, v0.4.0 (note, this is a backwards incompatible upgrade, see the release notes for details).

Terragrunt before and after hooks

Motivation: Many of our customers wanted a way to be able to run custom logic before or after executing Terraform/Terragrunt. For example, you might want to generate some code, or check there are no uncommitted changes, or run some sort of static analysis tool to check for errors. Unfortunately, the only way to do this was to use custom wrapper scripts or Makefiles with Terraform/Terragrunt.

Solution: We’ve added support for before and after “hooks” to Terragrunt that let you run arbitrary shell commands before or after executing Terraform! All you do is specify the command to execute in your terraform.tfvars file:

Copy
terragrunt = {
terraform {
before_hook "before_hook_1" {
commands = ["apply", "plan"]
execute = ["echo", "Foo"]
}

after_hook "after_hook_1" {
commands = ["apply", "plan"]
execute = ["echo", "Bar"]
}
}
}

With the configuration above, when you run terragrunt apply, you’ll see “Foo” before terraform apply executes and “Bar” after it executes.

What to do: Update to Terragrunt v0.14.5 and check out the Before And After Hook docs for instructions.

Spot pricing for ECS cluster instances

Motivation: A couple of our customers wanted to be able to use Spot Pricing on their non-prod ECS clusters to save money.

Solution: The ecs-cluster module now supports spot pricing! All you need to do is set the new cluster_instance_spot_price to the maximum bid price in your non-prod environments, and ECS will automatically try to find cheaper instances for you to use!

What to do: Give it a shot with module-ecs, v0.6.4.

Server-side encryption and CORS for websites on S3

Motivation: One of our customers wanted to enable server-side encryption and CORS for the static websites they were hosting in S3.

Solution: The s3-static-website module now enables server-side encryption by default and you can now configure CORS for the S3 bucket using the new cors_rule parameter.

What to do: Update to package-static-assets, v0.3.0 for server-side encryption support; update to package-static-assets, v0.3.1 for CORS support.

More CentOS support

Motivation: One of our customers is standardized on CentOS 7 and asked that all our modules support CentOS 7 as well.

Solution: Last month, we added CentOS support to a number of our modules. We’ve continued that work this month, adding CentOS support to:

• attach-eni (module-server, v0.4.2)
• All the modules in package-zookeeper (package-zookeeper, v0.3.2)
• All the modules in package-kafka (new release coming shortly!)

What to do: If you’d like to use CentOS 7, start using the latest version of the above modules in your infrastructure.

Open source module updates

• terraform-aws-nomad, v0.3.0: The nomad-cluster module no longer supports the target_group_arns and load_balancersparameters. Instead, to associate a load balancer with your Nomad cluster, you should use a separate aws_autoscaling_attachment resource.
• terraform-aws-consul, v0.3.0: The consul-cluster module no longer supports the target_group_arns and load_balancersparameters. Instead, to associate a load balancer with your Consul cluster, you should use a separate aws_autoscaling_attachment resource.
• terraform-aws-consul, v0.3.1: The consul-cluster module will now apply a Name tag, set to var.name, to the Consul security group.
• terraform-aws-vault, v0.4.0: The Vault ASG is now namespaced using the cluster_name variable, just like all the other resources.
• terraform-aws-vault, v0.5.0: The vault-cluster module no longer supports the target_group_arns and load_balancersparameters. Instead, to associate a load balancer with your Vault cluster, you should use a separate aws_autoscaling_attachment resource. If you're using the vault-elb module, it will create the aws_autoscaling_attachment resource for you automatically.
• terragrunt, v0.14.3: Terragrunt will now automatically run init if providers aren't downloaded.

Other updates

• module-data-storage, v0.5.2: If you wish to make your Aurora database accessible from the public Internet, you can now set the publicly_accessible flag to true (WARNING: NOT RECOMMENDED FOR PRODUCTION USAGE!!). The default is false, which means the database is only accessible from within the VPC, which is much more secure.
• module-load-balancer, v0.7.2: You can now configure the SSL policy for the ALB using the ssl_policy variable.
• module-aws-monitoring, v0.9.1: Fix run-cloudwatch-logs-agent.sh so for Amazon Linux and CentOS so instead of sending /var/log/auth.log, which doesn't exist, it sends /var/log/secure to CloudWatch Logs.

DevOps News

At the recent AWS Summit in San Francisco on April 4, 2018, Amazon announced a number of exciting releases. Here’s a list of the most important ones, plus other announcements from the past month.

Service Discovery for ECS

What happened: AWS has added native support for Service Discovery to ECS (Fargate supports it too).

Why it matters: Before, if you wanted to run microservices in ECS and have the communicate with each other, you’d have to run load balancers or your own service discovery mechanism (e.g., Consul). Now, you can just enable Service Discovery as a setting on your Docker containers, and they will automatically register/unregister their IPs with Route 53.

What to do about it: We hope to add Service Discovery support to our ECS modules in the future, but if you need it soon, email us at support@gruntwork.io to let us know! For now, your best bet is to enable it manually. Note that as of today, Service Discovery is only available in a handful of regions.

Amazon ECS Now Supports Docker-Level Health Checks

What happened: AWS announced Container Health Checks and Task Health Management.

Why it matters: Previously, you were forced to launch a Load Balancer just to run a health check on a container. Now, you can declare a Docker container’s health check using the HEALTHCHECK directive in a Dockerfile. Amazon ECS will use this to determine the health status of a single container or group of containers.

What to do about it: If your ECS Service needs a health check but not a load balancer (e.g. background job services), or if you want a more sophisticated health check than an HTTP or TCP request, consider removing the load balancer and adding a HEALTHCHECK to your Docker image.

AWS Fargate is now ISO, PCI, HIPAA, and SOC compliant

What happened: AWS announced that Docker containers running on AWS Fargate now meet the criteria for ISO, PCI, SOC 1, SOC 2, and SOC 3 compliance along with the criteria for HIPAA eligibility.

Why it matters: If your company has to meet the criteria for one of those compliance programs, you can now use Fargate to run your Docker containers, even if those containers deal with personally identifiable information (PII) or protected health information (PHI)!

What to do about it: We do not yet have Terraform modules for working with Fargate, so your best bet for now is to use Fargate manually. We hope to add Fargate support in the future, but if you need it soon, email us at support@gruntwork.io to let us know!

You can now assume IAM roles for up to 12 hours!

What happened: AWS now allows you to configure the maximum session duration for IAM roles.

Why it matters: Before, if you assumed an IAM role, the maximum session duration was 1 hour. Since assuming IAM roles is the recommended mechanism for accessing multiple AWS accounts and using federated identities (e.g., logging into your AWS account using your Google or AD credentials), this meant that you would have to authenticate many times throughout the day, which was very tedious. We created all sorts of workarounds (e.g., the --terragrunt-iam-role param), but now these are no longer necessary!

What to do about it: Terraform does not yet support the new IAM session duration parameter (follow this issue for progress), so for now, you can modify the session duration in your IAM roles manually as explained in this blog post.

Amazon DynamoDB Continuous Backups and Point-In-Time Recovery

What happened: DynamoDB now supports continuous backups and point-in-time recovery.

Why it matters: The data in your DynamoDB tables can now be backed up at a per-second granularity, and you can restore to any previous point in the last 35 days with just a click. This protects you from data loss, accidental writes, unintentional deletes, etc.

What to do about it: Check out the blog post for how to enable this powerful new feature.

Use AWS Certificate Manager as a Private CA

What happened: AWS Certificate Manager (ACM) now supports creating TLS/SSL certificates for private use with a private CA certificate.

Why it matters: If you want to provision TLS certs on ELBs, CloudFront Distributions, API Gateway or other Amazon resources for private use only, you can now use ACM to do it.

What to do about it: Start using ACM if you need private TLS certs!

You can now render “metrics calculations” in CloudWatch graphs

What happened: Amazon announced support for CloudWatch Metric Math, which allows you to perform calculations across multiple metrics for real-time analysis.

Why it matters: You can now compute metrics from existing metrics using arithmetic (+,-, *, /) and statistical functions (sum, avg, min, max) and render these computed metrics directly on a graph. For example, you could show %IOPS used. No external scripts or tooling needed!

What to do about it: If this is useful for you, start adding “CloudWatch Metrics Math” to some of your CloudWatch Metrics. When Terraform support lands, we’ll consider adding some of these to our modules.

AWS Secrets Manager

What happened: A new AWS service, AWS Secrets Manager, is now available.

Why it matters: This is AWS’s attempt to mimic the ability of Vault to issue ephemeral secrets such as temporary database credentials. Otherwise, it’s virtually identical in functionality to SSM Parameter Store, but with the added ability to auto-rotate credentials. Its main downside is that the service is surprisingly expensive, at $0.40/secret/month.

What to do about it: We continue to recommend using GruntKMS to encrypt secrets that you store in git for secrets management. While not as powerful as Vault or AWS Secrets Manager, it’s simple, secure, and convenient for teams whose secrets change relatively infrequently.

Docker Cloud is shutting down

What happened: Docker has announced that Cluster and application management services in Docker Cloud are shutting down on May 21.

Why it matters: If you’re still running Docker containers in Docker Cloud, you need to migrate your applications to another cloud ASAP!

What to do about it: If you’d like to migrate to Amazon ECS, we have modules available for that already. If you want to use Fargate or EKS, email us at support@gruntwork.io to let us know!

Security Updates

Below is a list of critical security updates that may impact your services. We notify Gruntwork customers of these vulnerabilities as soon as we know of them via the Gruntwork Security Alerts mailing list. It is up to you to scan this list and decide which of these apply and what to do about them, but most of these are severe vulnerabilities, and we recommend patching them ASAP.

Drupal

• SA-CORE-2018–002: A remote code execution vulnerability has been found within multiple subsystems of Drupal 7.x and 8.x. This potentially allows attackers to exploit multiple attack vectors on a Drupal site, which could result in the site being completely compromised. If you use Drupal, we recommend updating immediately. We sent an email about this vulnerability to the security alerts mailing list on March 30, 2018.