At Gruntwork, we strive to accelerate the deployment of production grade infrastructure by prodiving a library of
stable, reusable, and battle tested infrastructure as code organized into a series of modules with
submodules. Each module represents a particular set of infrastructure that is componentized into
smaller pieces represented by the submodules within the module. By doing so, we have built a composable library that can
be combined into building out everything from simple single service deployments to complicated microservice setups so
that your infrastructure can grow with your business needs. Every module we provide is built with the production grade
infrastruture checklist in mind, ensuring that the services you deploy are
resilient, fault tolerant, and scalable.
What is a Module?
A Module is a reusable, tested, documented, configurable, best-practices definition of a single piece of Infrastructure
(e.g., Docker cluster, VPC, Jenkins, Consul), written using a combination of Terraform, Go,
and Bash. A module contains a set of automated tests, documentation, and examples that have been proven in production,
providing the underlying infrastructure for Gruntwork's customers.
Instead of figuring out the details of how to run a piece of infrastructure from scratch, you can reuse existing code
that has been proven in production. And instead of maintaining all that infrastructure code yourself, you can leverage
the work of the community to pick up infrastructure improvements through a version number bump.
What is a Submodule?
Each Infrastructure Module consists of one or more orthogonal Submodules that handle some specific aspect of that
Infrastructure Module's functionality. Breaking the code up into multiple submodules makes it easier to reuse and
compose to handle many different use cases. Although Modules are designed to provide an end to end solution to manage
the relevant infrastructure by combining the Submodules defined in the Module, Submodules can be used independently for
specific functionality that you need in your infrastructure code.
Production Grade Infrastructure Checklist
At Gruntwork, we have learned over the years that it is not enough to just get the services up and running in a publicly
accessible space to call your application "production-ready." There are many more things to consider, and oftentimes
many of these considerations are missing in the deployment plan of applications. These topics come up as afterthoughts,
and are learned the hard way after the fact. That is why we codified all of them into a checklist that can be used as a
reference to help ensure that they are considered before your application goes to production, and conscious decisions
are made to neglect particular components if needed, as opposed to accidentally omitting them from consideration.
Task
Description
Example tools
Install
Install the software binaries and all dependencies.
Bash, Chef, Ansible, Puppet
Configure
Configure the software at runtime. Includes port settings, TLS certs, service discovery, leaders, followers, replication, etc.
Bash, Chef, Ansible, Puppet
Provision
Provision the infrastructure. Includes EC2 instances, load balancers, network topology, security gr oups, IAM permissions, etc.
Terraform, CloudFormation
Deploy
Deploy the service on top of the infrastructure. Roll out updates with no downtime. Includes blue-green, rolling, and canary deployments.
Scripts, Orchestration tools (ECS, k8s, Nomad)
High availability
Withstand outages of individual processes, EC2 instances, services, Availability Zones, and regions.
Multi AZ, multi-region, replication, ASGs, ELBs
Scalability
Scale up and down in response to load. Scale horizontally (more servers) and/or vertically (bigger servers).
ASGs, replication, sharding, caching, divide and conquer
Performance
Optimize CPU, memory, disk, network, GPU, and usage. Includes query tuning, benchmarking, load testing, and profiling.
Dynatrace, valgrind, VisualVM, ab, Jmeter
Networking
Configure static and dynamic IPs, ports, service discovery, firewalls, DNS, SSH access, and VPN access.
EIPs, ENIs, VPCs, NACLs, SGs, Route 53, OpenVPN
Security
Encryption in transit (TLS) and on disk, authentication, authorization, secrets management, server hardening.
ACM, EBS Volumes, Cognito, Vault, CIS
Metrics
Availability metrics, business metrics, app metrics, server metrics, events, observability, tracing, and alerting.
CloudWatch, DataDog, New Relic, Honeycomb
Logs
Rotate logs on disk. Aggregate log data to a central location.
CloudWatch logs, ELK, Sumo Logic, Papertrail
Backup and Restore
Make backups of DBs, caches, and other data on a scheduled basis. Replicate to separate region/account.
RDS, ElastiCache, ec2-snapper, Lambda
Cost optimization
Pick proper instance types, use spot and reserved instances, use auto scaling, and nuke unused resources.
ASGs, spot instances, reserved instances
Documentation
Document your code, architecture, and practices. Create playbooks to respond to incidents.
READMEs, wikis, Slack
Tests
Write automated tests for your infrastructure code. Run tests after every commit and nightly.
Terratest
Questions? Ask away.
We're here to talk about our services, answer any questions, give advice, or just to chat.
{"treedata":{"name":"root","toggled":true,"children":[{"name":".circleci","children":[{"name":"config.yml","path":".circleci/config.yml","sha":"cef3f13f1128f203609bd70491c5789049d97396"}]},{"name":".gitignore","path":".gitignore","sha":"ca31ff35c5b25c686571a0430a14f86a38f15e77"},{"name":".pre-commit-config.yaml","path":".pre-commit-config.yaml","sha":"2dd1a8d4e16b65a1991537d51b5b59b6df1866f8"},{"name":"CODEOWNERS","path":"CODEOWNERS","sha":"89db2c0afb6268a0fa92d8e841018cef4bc653cb"},{"name":"CONTRIBUTING.md","path":"CONTRIBUTING.md","sha":"b12849077d576a7fad88da42db1131bb3369e194"},{"name":"GRUNTWORK_PHILOSOPHY.md","path":"GRUNTWORK_PHILOSOPHY.md","sha":"02d9873a74c99fe6d9b6b26bd9f8eb4a7a699c32","toggled":true},{"name":"LICENSE","path":"LICENSE","sha":"276620ad6ffbc9954fd6633d167b0501155441d4"},{"name":"README.md","path":"README.md","sha":"444e80d608763aabb990f37a8badbe3bdf24b872"},{"name":"examples","children":[{"name":"k8s-namespace-with-service-account","children":[{"name":"README.md","path":"examples/k8s-namespace-with-service-account/README.md","sha":"cdff492defdaa77f736adc1c1ce9c1db1b3a9e1e"},{"name":"main.tf","path":"examples/k8s-namespace-with-service-account/main.tf","sha":"071b39770653ee2d6d12c80119bb290fff5c35d8"},{"name":"outputs.tf","path":"examples/k8s-namespace-with-service-account/outputs.tf","sha":"71e367b5fa5fc4d940c68b1689d340ab6fa2e17c"},{"name":"variables.tf","path":"examples/k8s-namespace-with-service-account/variables.tf","sha":"f38169a042290747cd6cda6375d99af81b1df97e"}]},{"name":"k8s-tiller-kubergrunt-minikube","children":[{"name":"README.md","path":"examples/k8s-tiller-kubergrunt-minikube/README.md","sha":"f4ee8247319417b5d433017005af1284c2feda71"},{"name":"main.tf","path":"examples/k8s-tiller-kubergrunt-minikube/main.tf","sha":"a2d71286a91c7cf02371ff10336ad6c8c65c5a99"},{"name":"outputs.tf","path":"examples/k8s-tiller-kubergrunt-minikube/outputs.tf","sha":"f839014a1879cfef47954641a01edf7820ac3bdc"},{"name":"variables.tf","path":"examples/k8s-tiller-kubergrunt-minikube/variables.tf","sha":"3da8530a497b5102a1bd8d88c473e15794e7f9d5"}]},{"name":"k8s-tiller-minikube","children":[{"name":"README.md","path":"examples/k8s-tiller-minikube/README.md","sha":"d579c9f2952bb63704def4d2c799a267e2ca3b2a"}]}]},{"name":"main.tf","path":"main.tf","sha":"506f789e6c7b0a9c767a683a4bc101498daef4d2"},{"name":"modules","children":[{"name":"k8s-helm-client-tls-certs","children":[{"name":"README.md","path":"modules/k8s-helm-client-tls-certs/README.md","sha":"4806c8188e5da43941b60b091133ad58f2537532"},{"name":"main.tf","path":"modules/k8s-helm-client-tls-certs/main.tf","sha":"f9727619e0606e5710d81934c541d94d528ee633"},{"name":"outputs.tf","path":"modules/k8s-helm-client-tls-certs/outputs.tf","sha":"59994511786075fe1a930e3ec6b46dd6134fff29"},{"name":"variables.tf","path":"modules/k8s-helm-client-tls-certs/variables.tf","sha":"445e4da760004173140af1176fa80b0cc8722a81"}]},{"name":"k8s-namespace-roles","children":[{"name":"README.md","path":"modules/k8s-namespace-roles/README.md","sha":"9aaca3f9e32408e23c02622d33942e0ce4586e34"},{"name":"main.tf","path":"modules/k8s-namespace-roles/main.tf","sha":"f82819379a8864bd6b9e71a8a18d815dbbc02559"},{"name":"outputs.tf","path":"modules/k8s-namespace-roles/outputs.tf","sha":"ab91d72a436cc5e450795182b4b43b78be207293"},{"name":"variables.tf","path":"modules/k8s-namespace-roles/variables.tf","sha":"0e5be83826073b0f786ff8f9b04826c555208758"}]},{"name":"k8s-namespace","children":[{"name":"README.md","path":"modules/k8s-namespace/README.md","sha":"4fa9469fbbd22faae11ac3f461487b2bfbe167e6"},{"name":"main.tf","path":"modules/k8s-namespace/main.tf","sha":"eba9f2b3dbc0191d2d4bfe2931caac9b58122541"},{"name":"outputs.tf","path":"modules/k8s-namespace/outputs.tf","sha":"d8e2f96f44b67f9d0d59431c789b39609a745996"},{"name":"variables.tf","path":"modules/k8s-namespace/variables.tf","sha":"572a62e2ca963233931c527bab99fd9f0a3a048b"}]},{"name":"k8s-service-account","children":[{"name":"README.md","path":"modules/k8s-service-account/README.md","sha":"a53dfad1ff1d991dfed08fb5da77f9c15e3b6d50"},{"name":"main.tf","path":"modules/k8s-service-account/main.tf","sha":"3c5efa6c04722d679f1707ab49c118d39ef6a806"},{"name":"outputs.tf","path":"modules/k8s-service-account/outputs.tf","sha":"c5c2389e4646bb2a16b87bec129330e0c3c4dcf6"},{"name":"variables.tf","path":"modules/k8s-service-account/variables.tf","sha":"4f0cfad5f8a5869ff201fad385ecaa0de7454674"}]},{"name":"k8s-tiller-tls-certs","children":[{"name":"README.md","path":"modules/k8s-tiller-tls-certs/README.md","sha":"bf4e7de237e87459f8f617b6fa60f0ed1d94cd86"},{"name":"main.tf","path":"modules/k8s-tiller-tls-certs/main.tf","sha":"69d3adb8717f12381f0683c0ab37581975b5a8dc"},{"name":"outputs.tf","path":"modules/k8s-tiller-tls-certs/outputs.tf","sha":"1a4038f3a59478a9f2d4b76eef51817dd4200dc7"},{"name":"variables.tf","path":"modules/k8s-tiller-tls-certs/variables.tf","sha":"259ebea616195628a68b8eb8a1da83465cd0c782"}]},{"name":"k8s-tiller","children":[{"name":"README.md","path":"modules/k8s-tiller/README.md","sha":"7dec15a673134ebc488a070dc614d04c2bf538d3"},{"name":"main.tf","path":"modules/k8s-tiller/main.tf","sha":"c278eeebee9bc47d059f59a22916e5cc6dac1335"},{"name":"outputs.tf","path":"modules/k8s-tiller/outputs.tf","sha":"e23f872780e656e4f12578d439eb910692f59a60"},{"name":"variables.tf","path":"modules/k8s-tiller/variables.tf","sha":"5d03e38bfec082950d20c0f48865d7c5107f259f"}]}]},{"name":"outputs.tf","path":"outputs.tf","sha":"e81c16d641a0e30b706292ad37ce8b8ef343becb"},{"name":"test","children":[{"name":"Gopkg.lock","path":"test/Gopkg.lock","sha":"cdae09784de4638a1b0eea0525c3be70cebcf2d7"},{"name":"Gopkg.toml","path":"test/Gopkg.toml","sha":"a8644e81d7acf83db32419833ca1ca318d2559c1"},{"name":"README.md","path":"test/README.md","sha":"c4361f3756f62c10366b7302401e8a53552061bb"},{"name":"k8s_namespace_with_service_account_test.go","path":"test/k8s_namespace_with_service_account_test.go","sha":"8fac6e063e5c03371b681cdd48a98a8d27137712"},{"name":"k8s_tiller_kubergrunt_test.go","path":"test/k8s_tiller_kubergrunt_test.go","sha":"67f31a218feaa2840cdad23308b577f442bd3f4d"},{"name":"k8s_tiller_test.go","path":"test/k8s_tiller_test.go","sha":"7501db1bfc755b2709c688a5bbd1867b4238b44a"},{"name":"kubefixtures","children":[{"name":"curl-kubeapi-as-service-account.yml.tpl","path":"test/kubefixtures/curl-kubeapi-as-service-account.yml.tpl","sha":"12fa119c7e183bb8d35cda86322c92e6a36a5307"},{"name":"namespace-check-create-pod.json.tpl","path":"test/kubefixtures/namespace-check-create-pod.json.tpl","sha":"ba88dfa440d815c221febd455550dd6fdbe7cbac"},{"name":"namespace-check-list-pod.json.tpl","path":"test/kubefixtures/namespace-check-list-pod.json.tpl","sha":"047bb650ac081c5f63d9491cde3ca80b92603489"}]},{"name":"terratest_options.go","path":"test/terratest_options.go","sha":"675f98b4c34be584c9f7eea240f290949f08f460"}]},{"name":"variables.tf","path":"variables.tf","sha":"7efc75a2f7d990de3cdf52c7589e9468da6f4133"}]},"detailsContent":"<h1 class=\"preview__body--title\" id=\"gruntwork-philosophy\">Gruntwork Philosophy</h1><div class=\"preview__body--border\"></div><p>At Gruntwork, we strive to accelerate the deployment of production grade infrastructure by prodiving a library of\nstable, reusable, and battle tested infrastructure as code organized into a series of <a href=\"#what-is-a-module\" class=\"preview__body--description--blue\">modules</a> with\n<a href=\"#what-is-a-submodule\" class=\"preview__body--description--blue\">submodules</a>. Each module represents a particular set of infrastructure that is componentized into\nsmaller pieces represented by the submodules within the module. By doing so, we have built a composable library that can\nbe combined into building out everything from simple single service deployments to complicated microservice setups so\nthat your infrastructure can grow with your business needs. Every module we provide is built with the <a href=\"#production-grade-infrastructure-checklist\" class=\"preview__body--description--blue\">production grade\ninfrastruture checklist</a> in mind, ensuring that the services you deploy are\nresilient, fault tolerant, and scalable.</p>\n<h2 class=\"preview__body--subtitle\" id=\"what-is-a-module\">What is a Module?</h2>\n<p>A Module is a reusable, tested, documented, configurable, best-practices definition of a single piece of Infrastructure\n(e.g., Docker cluster, VPC, Jenkins, Consul), written using a combination of <a href=\"https://www.terraform.io/\" class=\"preview__body--description--blue\" target=\"_blank\">Terraform</a>, Go,\nand Bash. A module contains a set of automated tests, documentation, and examples that have been proven in production,\nproviding the underlying infrastructure for <a href=\"https://www.gruntwork.io/customers\" class=\"preview__body--description--blue\" target=\"_blank\">Gruntwork's customers</a>.</p>\n<p>Instead of figuring out the details of how to run a piece of infrastructure from scratch, you can reuse existing code\nthat has been proven in production. And instead of maintaining all that infrastructure code yourself, you can leverage\nthe work of the community to pick up infrastructure improvements through a version number bump.</p>\n<h2 class=\"preview__body--subtitle\" id=\"what-is-a-submodule\">What is a Submodule?</h2>\n<p>Each Infrastructure Module consists of one or more orthogonal Submodules that handle some specific aspect of that\nInfrastructure Module's functionality. Breaking the code up into multiple submodules makes it easier to reuse and\ncompose to handle many different use cases. Although Modules are designed to provide an end to end solution to manage\nthe relevant infrastructure by combining the Submodules defined in the Module, Submodules can be used independently for\nspecific functionality that you need in your infrastructure code.</p>\n<h2 class=\"preview__body--subtitle\" id=\"production-grade-infrastructure-checklist\">Production Grade Infrastructure Checklist</h2>\n<p>At Gruntwork, we have learned over the years that it is not enough to just get the services up and running in a publicly\naccessible space to call your application "production-ready." There are many more things to consider, and oftentimes\nmany of these considerations are missing in the deployment plan of applications. These topics come up as afterthoughts,\nand are learned the hard way after the fact. That is why we codified all of them into a checklist that can be used as a\nreference to help ensure that they are considered before your application goes to production, and conscious decisions\nare made to neglect particular components if needed, as opposed to accidentally omitting them from consideration.</p>\n<p></p>\n<table>\n<thead>\n<tr>\n<th>Task</th>\n<th>Description</th>\n<th>Example tools</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>Install</td>\n<td>Install the software binaries and all dependencies.</td>\n<td>Bash, Chef, Ansible, Puppet</td>\n</tr>\n<tr>\n<td>Configure</td>\n<td>Configure the software at runtime. Includes port settings, TLS certs, service discovery, leaders, followers, replication, etc.</td>\n<td>Bash, Chef, Ansible, Puppet</td>\n</tr>\n<tr>\n<td>Provision</td>\n<td>Provision the infrastructure. Includes EC2 instances, load balancers, network topology, security gr oups, IAM permissions, etc.</td>\n<td>Terraform, CloudFormation</td>\n</tr>\n<tr>\n<td>Deploy</td>\n<td>Deploy the service on top of the infrastructure. Roll out updates with no downtime. Includes blue-green, rolling, and canary deployments.</td>\n<td>Scripts, Orchestration tools (ECS, k8s, Nomad)</td>\n</tr>\n<tr>\n<td>High availability</td>\n<td>Withstand outages of individual processes, EC2 instances, services, Availability Zones, and regions.</td>\n<td>Multi AZ, multi-region, replication, ASGs, ELBs</td>\n</tr>\n<tr>\n<td>Scalability</td>\n<td>Scale up and down in response to load. Scale horizontally (more servers) and/or vertically (bigger servers).</td>\n<td>ASGs, replication, sharding, caching, divide and conquer</td>\n</tr>\n<tr>\n<td>Performance</td>\n<td>Optimize CPU, memory, disk, network, GPU, and usage. Includes query tuning, benchmarking, load testing, and profiling.</td>\n<td>Dynatrace, valgrind, VisualVM, ab, Jmeter</td>\n</tr>\n<tr>\n<td>Networking</td>\n<td>Configure static and dynamic IPs, ports, service discovery, firewalls, DNS, SSH access, and VPN access.</td>\n<td>EIPs, ENIs, VPCs, NACLs, SGs, Route 53, OpenVPN</td>\n</tr>\n<tr>\n<td>Security</td>\n<td>Encryption in transit (TLS) and on disk, authentication, authorization, secrets management, server hardening.</td>\n<td>ACM, EBS Volumes, Cognito, Vault, CIS</td>\n</tr>\n<tr>\n<td>Metrics</td>\n<td>Availability metrics, business metrics, app metrics, server metrics, events, observability, tracing, and alerting.</td>\n<td>CloudWatch, DataDog, New Relic, Honeycomb</td>\n</tr>\n<tr>\n<td>Logs</td>\n<td>Rotate logs on disk. Aggregate log data to a central location.</td>\n<td>CloudWatch logs, ELK, Sumo Logic, Papertrail</td>\n</tr>\n<tr>\n<td>Backup and Restore</td>\n<td>Make backups of DBs, caches, and other data on a scheduled basis. Replicate to separate region/account.</td>\n<td>RDS, ElastiCache, ec2-snapper, Lambda</td>\n</tr>\n<tr>\n<td>Cost optimization</td>\n<td>Pick proper instance types, use spot and reserved instances, use auto scaling, and nuke unused resources.</td>\n<td>ASGs, spot instances, reserved instances</td>\n</tr>\n<tr>\n<td>Documentation</td>\n<td>Document your code, architecture, and practices. Create playbooks to respond to incidents.</td>\n<td>READMEs, wikis, Slack</td>\n</tr>\n<tr>\n<td>Tests</td>\n<td>Write automated tests for your infrastructure code. Run tests after every commit and nightly.</td>\n<td>Terratest</td>\n</tr>\n</tbody>\n</table>\n","repoName":"terraform-kubernetes-helm","repoRef":"v0.6.1","serviceDescriptor":{"serviceName":"Tiller / Helm","serviceRepoName":"terraform-kubernetes-helm","serviceRepoOrg":"gruntwork-io","cloudProviders":["aws"],"description":"Deploy Tiller (Helm Server) to your Kubernetes cluster as a service/package manager. Supports namespaces, service accounts, RBAC roles, and TLS.","imageUrl":"kubernetes.png","licenseType":"subscriber","technologies":["Terraform","Bash","Helm"],"compliance":[],"tags":[""]},"serviceCategoryName":"Docker services","fileName":"GRUNTWORK_PHILOSOPHY.md","filePath":"/GRUNTWORK_PHILOSOPHY.md","title":"Repo Browser: Tiller / Helm","description":"Browse the repos in the Gruntwork Infrastructure as Code Library."}