Has two dedicated worker node groups managed by separate ASGs for different functions:
core nodes: Worker nodes intended to run core services, such as kiam. These services require additional
permissions to function, and therefore require a more locked down EC2 instance configuration.
application nodes: Worker nodes intended to run application services.
Nodes are tagged with labels inherited from the EC2 instance tags, using the map-ec2-tags-to-node-labels script in
the eks-scripts module.
Deploys core admin services that provide various features:
FluentD for shipping Kubernetes logs (including container logs) to CloudWatch.
ALB Ingress Controller for exposing services using ALBs.
external-dns for tying Route 53 domains to exposed services.
Prerequisites
This example depends on Terraform, Packer, and kubergrunt. You can also optionally install kubectl if
you would like explore the newly provisioned cluster. You can find instructions on how to install each tool below:
Finally, before you begin, be sure to set up your AWS credentials as environment variables so that all the commands
below can authenticate to the AWS account where you wish to deploy this example. You can refer to our blog post series
on AWS authentication (A Comprehensive Guide to Authenticating to AWS on the Command
Line) for
more information.
Once the cluster is deployed, take a look at Where to go from here for ideas on what to do
next.
Create a New AMI with the Helper Scripts Installed
This example depends on the map-ec2-tags-to-node-labels script to assist with mapping the EC2 instance
tags into Node
Labels. We will use packer to build a
customized AMI on based on the EKS optimized
AMI that includes the script.
To build the AMI, you need to provide packer the build template and required variables. Since we will be installing a
Gruntwork module, we will need to setup Github access. This can be done by defining the GITHUB_OAUTH_TOKEN environment
variable with a personal access token. See https://github.com/gruntwork-io/gruntwork-installer#authentication for more
information on how to set this up.
Once the environment variable is set, you can run packer build to build the AMI:
packerbuild packer/build.json
This will spin up an EC2 instance, run the shell scripts to provision the machine, burn a new AMI, spin down the
instance, and then output the newly built AMI.
Note: By default, the provided packer template will build a new AMI in the us-east-1 region. If you would like to
change the region to build in, you can pass in -var "region=us-east-2" to override the default region.
Apply Terraform Templates
Once the AMI is built, we are ready to use it to deploy our EKS cluster. Unlike the other examples in this repo, this
example breaks up the code into multiple submodules. Refer to Why are there multiple Terraform submodules in this
example? for more information on why the example is
structured this way.
To deploy our cluster, we will apply the templates in the following order:
The code for deploying an EKS cluster with its worker groups is defined in the eks-cluster submodule.
This Terraform example, when applied, will deploy a VPC, launch an EKS control plane in there, and then provision two
worker groups to run workloads. The two groups provided by the example are:
A core worker group that is dedicated for running supporting services like kiam that may require more lockdown.
The nodes in this group are tainted with NoSchedule so that Pods are not scheduled there by default.
An application worker group that is dedicated for running application services.
To deploy the example, we need to first define the required variables. To define variables to use, create a new file in
the example directory called terraform.tfvars:
touch ./eks-cluster/terraform.tfvars
Then, create a new entry for each required variable (and any optional variables you would like to override). See the
variables.tf file for a list of available variables. Below is a sample terraform.tfvars file:
NOTE: If you attempt to deploy into the us-east-1 region, note that the availability zone us-east-1e does not
support EKS. To work around this, use the allowed_availability_zones to control which zones are used to deploy EKS by
adding the following to the tfvars file:
allowed_availability_zones = ["us-east-1a", "us-east-1b", "us-east-1c", "us-east-1d", "us-east-1f"].
Once the variables are filled out, we are ready to apply the templates to provision our cluster. To do this, we need to
run terraform init followed by terraform apply:
cd eks-cluster
terraform init
terraform apply
cd .. # go back to eks-cluster-with-supporting-services example folder
At the end of this, you will have an EKS cluster with 2 ASG node worker pools. We will use kubectl to verify this.
In order to use kubectl, we need to first set it up so that it can authenticate with our new EKS cluster. You can
learn more about how authentication works with EKS in our guide How do I authenticate kubectl to the EKS
cluster?. For now, you can run the kubergrunt eks configure command:
At the end of this command, your default kubeconfig file (located at ~/.kube/config) will have a new context that
authenticates with EKS. This context will be set as the default so that subsequent kubectl calls will target your
deployed eks cluster.
You can now use kubectl to verify the two worker groups. Run kubectl get nodes and kubectl describe nodes to see
the associated labels of the nodes and verify there are two distinct labels.
This will output information about the deployed cluster. Record the entries for vpc_id,
eks_openid_connect_provider_arn, and eks_openid_connect_provider_url, as we will be using those in the next step.
Deploy Core Services
Once our EKS cluster is deployed, we can deploy core services on to it. The code for core services is defined in the
core-services submodule. This Terraform example, when applied, deploys administrative services:
fluentd-cloudwatch: Used to ship Kubernetes logs (including container logs) to CloudWatch.
aws-alb-ingress-controller: Used to map Ingress resources into AWS ALBs.
external-dns: Used for tieing Route 53 domains to exposed services.
Like the previous step, create a terraform.tfvars file and fill it in. Below is a sample entry:
The eks_vpc_id, openid_connect_provider_arn, and openid_connect_provider_url input variables should be
the outputs that you recorded in the previous step.
Once the tfvars file is created, you can init and apply the templates:
touch ./core-services/terraform.tfvars
# Fill in the tfvars file with the variable inputs
cd core-services
terraform init
terraform apply
cd .. # go back to eks-cluster-with-supporting-services example folder
Additionally, the cluster should be shipping logs to CloudWatch. You can load the CloudWatch logs in the UI by
navigating to CloudWatch in the AWS console and looking for the log group with the same name as the EKS cluster.
(Optional) Deploy Nginx Service
The nginx-service submodule shows an example of how to use Helm to deploy an application on to your
EKS cluster. This example will:
Once the tfvars file is created, you can init and apply the templates:
touch ./nginx-service/terraform.tfvars
# Fill in the tfvars file with the variable inputs
cd nginx-service
terraform init
terraform apply
cd .. # go back to eks-cluster-with-supporting-services example folder
Once Terraform finishes, the nginx service should be available. To get the endpoint, you can query Kubernetes using
kubectl for the Service information:
# Prerequisite: Setup environment variables to auth to AWS# Use kubectl to get Service endpoint
kubectl \
get ingresses \
--namespace kube-system \
--selector app.kubernetes.io/instance=nginx,app.kubernetes.io/name=nginx \
-o jsonpath \
--template '{.items[0].status.loadBalancer.ingress[0].hostname}'
This will output the ALB endpoint to the console. When you hit the endpoint, you should be able to see the welcome page
for nginx. If the service isn't available or you don't get the endpoint, wait a few minutes to give Kubernetes a chance
provision the ALB.
Where to go from here
Now that you have your cluster, you can do a few things to explore the cluster:
If you setup an SSH key with the eks_worker_keypair_name variable, try SSH-ing to the nodes to see the running
processes.
Why are there multiple Terraform submodules in this example?
Breaking up your code not only improves readability, but it also helps with maintainability by keeping the surface area
small. Typically you would want to structure your code so that resources that are deployed more frequently are isolated
from critical resources that might bring down the cluster. For example, you might upgrade the supporting services
regularly, but you might not touch your VPC once it is set up. Therefore, you do not want to manage your Helm resources
with the VPC, as everytime you update the services, you would be putting the cluster at risk.
Additionally, breaking up the code into modules helps introduce dependency ordering. Terraform is notorious for having
bugs and subtle issues that make it difficult to reliably introduce a dependency chain, especially when you have
modules. For example, you can't easily define dependencies such that the resources in a module depend on other resources
or modules external to the module (see https://github.com/hashicorp/terraform/issues/1178).
The dependency logic between launching the EKS cluster, setting up Helm, and deploying services using Helm is tricky to
encode in terraform such that it works reliably. For example, while it is fairly easy to get the resources to deploy in
order such that they are available, it is tricky to destroy the resources in the right order. For example, it is ok to
deploy services in parallel with the worker nodes, such that the worker nodes are coming up while the deployment script
is executing. The same thing can't be done for destruction, because the script needs to undeploy the services before
undeploying the worker nodes.
In summary, this example breaks up the Terraform code as an example of how one might modularize their EKS cluster code,
in addition to making the dependency management more explicit.
Troubleshooting
When destroying eks-cluster, I get an error with destroying VPC related resources.
EKS relies on the amazon-vpc-cni-k8s plugin to allocate IP addresses to
the pods in the Kubernetes cluster. This plugin works by allocating secondary ENI devices to the underlying worker
instances. Depending on timing, this plugin could interfere with destroying the cluster in this example. Specifically,
terraform could shutdown the instances before the VPC CNI pod had a chance to cull the ENI devices. These devices are
managed outside of terraform, so if they linger, it could interfere with destroying the VPC.
To workaround this limitation, you have to go into the console and delete the ENI associated with the VPC. Then,
retry the destroy call.
Questions? Ask away.
We're here to talk about our services, answer any questions, give advice, or just to chat.
{"treedata":{"name":"root","toggled":true,"children":[{"name":".circleci","children":[{"name":"config.yml","path":".circleci/config.yml","sha":"39ad5e2989f63bdc610ade49e1934d480c3aae11"}]},{"name":".gitignore","path":".gitignore","sha":"c95113b7b14c0533152b21a0491d6fd76aacb253"},{"name":".pre-commit-config.yaml","path":".pre-commit-config.yaml","sha":"2a146b2385ade859614e293dc083f39a9972b5fe"},{"name":"CODEOWNERS","path":"CODEOWNERS","sha":"ecbeaab263c59e955b621268f161059633041e3d"},{"name":"CONTRIBUTING.md","path":"CONTRIBUTING.md","sha":"a7cc7bd94443c252390564fa988755dbbe80d87d"},{"name":"GRUNTWORK_PHILOSOPHY.md","path":"GRUNTWORK_PHILOSOPHY.md","sha":"02d9873a74c99fe6d9b6b26bd9f8eb4a7a699c32"},{"name":"LICENSE.md","path":"LICENSE.md","sha":"a2cf01ecdd725fddd718ab91c80c115882c94f3c"},{"name":"README.adoc","path":"README.adoc","sha":"38728b9372ee535edd8dffdac56ef361a2c7680a"},{"name":"_docs","children":[{"name":"eks-architecture.png","path":"_docs/eks-architecture.png","sha":"b4c9c46f88ed465c5575e915af54ad9920b56941"},{"name":"eks-icon.png","path":"_docs/eks-icon.png","sha":"83a29dc46e7bc6234ba5bb825e8ae283c56229a0"},{"name":"iam-role-icon.png","path":"_docs/iam-role-icon.png","sha":"c05bb05e6caae9b9db46505ce505a386f21fa2e4"}]},{"name":"core-concepts.md","path":"core-concepts.md","sha":"05a677fa120572a099fe546f16ec55c21207c10d"},{"name":"examples","children":[{"name":"README.md","path":"examples/README.md","sha":"668e34198ac21f150743f330ab1208f5f67e6b3c"},{"name":"eks-cluster-managed-workers","children":[{"name":"README.md","path":"examples/eks-cluster-managed-workers/README.md","sha":"b8f9642b8db4b69f3e35faee0801f0421f92a83d"},{"name":"dependencies.tf","path":"examples/eks-cluster-managed-workers/dependencies.tf","sha":"c51d22849120296cb44e2637625fbe0ef4405a53"},{"name":"main.tf","path":"examples/eks-cluster-managed-workers/main.tf","sha":"8e23d204681041bcc67ede8cce989cab30b04402"},{"name":"outputs.tf","path":"examples/eks-cluster-managed-workers/outputs.tf","sha":"bbf83d99c547cf6ccff2b6f0782d083de971cd32"},{"name":"user-data","children":[{"name":"user_data.sh","path":"examples/eks-cluster-managed-workers/user-data/user_data.sh","sha":"c5fdd13d5bb04f765f1c90e9f12d23c48e94a252"}]},{"name":"variables.tf","path":"examples/eks-cluster-managed-workers/variables.tf","sha":"cebb93007096724b08453a8d6c2fe01cb45b29e5"}]},{"name":"eks-cluster-with-iam-role-mappings","children":[{"name":"README.md","path":"examples/eks-cluster-with-iam-role-mappings/README.md","sha":"67c3a556cedc312b67bd50f8f439dff7bbad0034"},{"name":"dependencies.tf","path":"examples/eks-cluster-with-iam-role-mappings/dependencies.tf","sha":"9652dab961175e0f2273b109b5f1724a38e3970f"},{"name":"main.tf","path":"examples/eks-cluster-with-iam-role-mappings/main.tf","sha":"bffbdd5acfc3a5d0b87d3747988986314dfbc370"},{"name":"outputs.tf","path":"examples/eks-cluster-with-iam-role-mappings/outputs.tf","sha":"e641213ad585fffe0f165b9a543d54c9175a1a7d"},{"name":"user-data","children":[{"name":"user-data.sh","path":"examples/eks-cluster-with-iam-role-mappings/user-data/user-data.sh","sha":"b10c34bfe4c9d10101472b47edbc3b7dff42a88e"}]},{"name":"variables.tf","path":"examples/eks-cluster-with-iam-role-mappings/variables.tf","sha":"3b47aa612be8a86f738d8f432f88c82b34c05510"}]},{"name":"eks-cluster-with-supporting-services","children":[{"name":"README.md","path":"examples/eks-cluster-with-supporting-services/README.md","sha":"cd0e673a332314bdf5dfd0bae662e5eaea48e613","toggled":true},{"name":"core-services","children":[{"name":"README.md","path":"examples/eks-cluster-with-supporting-services/core-services/README.md","sha":"c1eb41e7cc60a67d29ef846daf3b2e974ca59e6e"},{"name":"dependencies.tf","path":"examples/eks-cluster-with-supporting-services/core-services/dependencies.tf","sha":"977c72682567c034c4effe391757cab2f342086f"},{"name":"main.tf","path":"examples/eks-cluster-with-supporting-services/core-services/main.tf","sha":"9506374dd947d986889926045d4877aca1359c3d"},{"name":"outputs.tf","path":"examples/eks-cluster-with-supporting-services/core-services/outputs.tf","sha":"e69de29bb2d1d6434b8b29ae775ad8c2e48c5391"},{"name":"variables.tf","path":"examples/eks-cluster-with-supporting-services/core-services/variables.tf","sha":"9554163542e532a0ceb5f78600ae1aae46f589f5"}]},{"name":"eks-cluster","children":[{"name":"README.md","path":"examples/eks-cluster-with-supporting-services/eks-cluster/README.md","sha":"8a60a01004a93bbbf2091b730f0207f6dd2cc07e"},{"name":"dependencies.tf","path":"examples/eks-cluster-with-supporting-services/eks-cluster/dependencies.tf","sha":"58c85fb4cb629a91afe41602e56072c19905e79b"},{"name":"main.tf","path":"examples/eks-cluster-with-supporting-services/eks-cluster/main.tf","sha":"a083c6b34038d6d42f77b3574bf2f811d3666fd0"},{"name":"outputs.tf","path":"examples/eks-cluster-with-supporting-services/eks-cluster/outputs.tf","sha":"be23a13dd6f4063be394b8ca7358b631d50fab8a"},{"name":"user-data","children":[{"name":"app_worker_user_data.sh","path":"examples/eks-cluster-with-supporting-services/eks-cluster/user-data/app_worker_user_data.sh","sha":"c5fdd13d5bb04f765f1c90e9f12d23c48e94a252"},{"name":"core_worker_user_data.sh","path":"examples/eks-cluster-with-supporting-services/eks-cluster/user-data/core_worker_user_data.sh","sha":"0fa26153108b3d030ceeaae777aeb0a7e115404e"}]},{"name":"variables.tf","path":"examples/eks-cluster-with-supporting-services/eks-cluster/variables.tf","sha":"a24c08bd4409bb1a0a8bf1e86a1a2dce48db82fa"}]},{"name":"nginx-service","children":[{"name":"README.md","path":"examples/eks-cluster-with-supporting-services/nginx-service/README.md","sha":"0f6649ddb0cbb5aa80a5bc1f3318ea1fd5d0dc35"},{"name":"dependencies.tf","path":"examples/eks-cluster-with-supporting-services/nginx-service/dependencies.tf","sha":"0176248910eed450c12b54d10e3d74c8702c17ca"},{"name":"main.tf","path":"examples/eks-cluster-with-supporting-services/nginx-service/main.tf","sha":"92d904a2682749d2b67c1dfcca2f9d63af1102fb"},{"name":"templates","children":[{"name":"values.yaml","path":"examples/eks-cluster-with-supporting-services/nginx-service/templates/values.yaml","sha":"298435e01df9fa495b15d512073c62662d292cd3"}]},{"name":"variables.tf","path":"examples/eks-cluster-with-supporting-services/nginx-service/variables.tf","sha":"36ea6f8a36b19e34dbeeb25ae7e5fcf30c956b0f"}]},{"name":"packer","children":[{"name":"README.md","path":"examples/eks-cluster-with-supporting-services/packer/README.md","sha":"6a974a7fd5da7ac13309d9e0c4aaba7bd8cb46c7"},{"name":"build.json","path":"examples/eks-cluster-with-supporting-services/packer/build.json","sha":"5e2459d21b8705c3e3882473b28fd1c1cb890d14"}]}],"toggled":true},{"name":"eks-fargate-cluster-with-irsa","children":[{"name":"README.md","path":"examples/eks-fargate-cluster-with-irsa/README.md","sha":"27e8ad7773480a5431283c43609e3c7a47632f23"},{"name":"dependencies.tf","path":"examples/eks-fargate-cluster-with-irsa/dependencies.tf","sha":"88e84376868ae8dfc7b90483aa0fffe1c9d1a9ae"},{"name":"main.tf","path":"examples/eks-fargate-cluster-with-irsa/main.tf","sha":"e46b41d66a94ec894debff43fb6101b045081cfc"},{"name":"outputs.tf","path":"examples/eks-fargate-cluster-with-irsa/outputs.tf","sha":"f059d7b74ffbfb06a0868d6d0a5d1831c8f45f10"},{"name":"variables.tf","path":"examples/eks-fargate-cluster-with-irsa/variables.tf","sha":"ba960ca7cca1e1abd0598cdd53df530b85895735"}]},{"name":"eks-fargate-cluster-with-supporting-services","children":[{"name":"README.md","path":"examples/eks-fargate-cluster-with-supporting-services/README.md","sha":"03531a9ce95edd0babef101bb2eeafde47a84505"},{"name":"core-services","children":[{"name":"README.md","path":"examples/eks-fargate-cluster-with-supporting-services/core-services/README.md","sha":"cde0ae405e4d73e9e39c67045fb82de8187a673d"},{"name":"dependencies.tf","path":"examples/eks-fargate-cluster-with-supporting-services/core-services/dependencies.tf","sha":"977c72682567c034c4effe391757cab2f342086f"},{"name":"main.tf","path":"examples/eks-fargate-cluster-with-supporting-services/core-services/main.tf","sha":"cdca435bb46af34116f5d2ac801aedc53adc43e0"},{"name":"variables.tf","path":"examples/eks-fargate-cluster-with-supporting-services/core-services/variables.tf","sha":"fbe18fbde325d89c51737f66fc86762fbee090c5"}]},{"name":"eks-cluster","children":[{"name":"dependencies.tf","path":"examples/eks-fargate-cluster-with-supporting-services/eks-cluster/dependencies.tf","sha":"c7d533db5e590f72eddbe987d0b5353c11b570e1"},{"name":"main.tf","path":"examples/eks-fargate-cluster-with-supporting-services/eks-cluster/main.tf","sha":"8acaf2e75eaef315823c50b018a91e340753f567"},{"name":"outputs.tf","path":"examples/eks-fargate-cluster-with-supporting-services/eks-cluster/outputs.tf","sha":"edddf9a6ab6f5927db366689db79e1b91db9d8c8"},{"name":"variables.tf","path":"examples/eks-fargate-cluster-with-supporting-services/eks-cluster/variables.tf","sha":"53d1d38268ee5fa3534b4f75d3fa68b47d5a9516"}]},{"name":"nginx-service","children":[{"name":"dependencies.tf","path":"examples/eks-fargate-cluster-with-supporting-services/nginx-service/dependencies.tf","sha":"0176248910eed450c12b54d10e3d74c8702c17ca"},{"name":"main.tf","path":"examples/eks-fargate-cluster-with-supporting-services/nginx-service/main.tf","sha":"42de5f43776db66963b29ce4797377afa05a2a16"},{"name":"templates","children":[{"name":"values.yaml","path":"examples/eks-fargate-cluster-with-supporting-services/nginx-service/templates/values.yaml","sha":"655914f91177135cb7c5f15b62166cfc82a62a91"}]},{"name":"variables.tf","path":"examples/eks-fargate-cluster-with-supporting-services/nginx-service/variables.tf","sha":"d3c166441cdc556b0839930fbc281b7e8a1bd57f"}]}]},{"name":"eks-fargate-cluster","children":[{"name":"README.md","path":"examples/eks-fargate-cluster/README.md","sha":"a1013138cb6f63a7109f287e76030b7cc71f447e"},{"name":"dependencies.tf","path":"examples/eks-fargate-cluster/dependencies.tf","sha":"88e84376868ae8dfc7b90483aa0fffe1c9d1a9ae"},{"name":"main.tf","path":"examples/eks-fargate-cluster/main.tf","sha":"2eff37c02f2e14c4314ad43ce46b27eb7fc1e9f8"},{"name":"outputs.tf","path":"examples/eks-fargate-cluster/outputs.tf","sha":"9fb0eacd494d51072898a36f4d110a6c6ad77f6b"},{"name":"user-data","children":[{"name":"user-data.sh","path":"examples/eks-fargate-cluster/user-data/user-data.sh","sha":"b10c34bfe4c9d10101472b47edbc3b7dff42a88e"}]},{"name":"variables.tf","path":"examples/eks-fargate-cluster/variables.tf","sha":"ea24bdd726a83dcd4c64976dc7322081304837cb"}]}],"toggled":true},{"name":"modules","children":[{"name":"eks-alb-ingress-controller-iam-policy","children":[{"name":"README.md","path":"modules/eks-alb-ingress-controller-iam-policy/README.md","sha":"c87be2ee00f8f59403f827303915b5a70c602002"},{"name":"iampolicy.json","path":"modules/eks-alb-ingress-controller-iam-policy/iampolicy.json","sha":"11b345cb86734392f4efca98ccbf76761bb5e313"},{"name":"main.tf","path":"modules/eks-alb-ingress-controller-iam-policy/main.tf","sha":"b8fb9fcca04467e294b4f25a7966bbaec1e6338e"},{"name":"outputs.tf","path":"modules/eks-alb-ingress-controller-iam-policy/outputs.tf","sha":"b551b0bcc6eb1b43bfff1606696566658564cfb4"},{"name":"variables.tf","path":"modules/eks-alb-ingress-controller-iam-policy/variables.tf","sha":"250152e6bfeb02a16bed4151ffc7156636db1bd9"}]},{"name":"eks-alb-ingress-controller","children":[{"name":"README.md","path":"modules/eks-alb-ingress-controller/README.md","sha":"39684ae899557d5b509bcf75cffab6a196fd389e"},{"name":"main.tf","path":"modules/eks-alb-ingress-controller/main.tf","sha":"a520d5cfc3738919b1d80570f7983a6474b7c7f6"},{"name":"variables.tf","path":"modules/eks-alb-ingress-controller/variables.tf","sha":"94494df0f0feab70c0bc8f0d76c50a253959b84a"}]},{"name":"eks-aws-auth-merger","children":[{"name":"Dockerfile","path":"modules/eks-aws-auth-merger/Dockerfile","sha":"5ac1f9c32ce10ae515a41a9e339fab08bce6faea"},{"name":"README.adoc","path":"modules/eks-aws-auth-merger/README.adoc","sha":"07d666370cec8ff194314ce0ebd587e5104ace0d"},{"name":"aws-auth-merger","children":[{"name":"aws_auth_merger.go","path":"modules/eks-aws-auth-merger/aws-auth-merger/aws_auth_merger.go","sha":"63cf5b70211f5e2dc60ddb25b08e111787a3939c"},{"name":"aws_auth_merger_test.go","path":"modules/eks-aws-auth-merger/aws-auth-merger/aws_auth_merger_test.go","sha":"218a7dbb20c3e5ba80e6540156a81241360c6930"},{"name":"cli.go","path":"modules/eks-aws-auth-merger/aws-auth-merger/cli.go","sha":"8c335463605f65db25d773271da67a19d4caea7b"},{"name":"configmap_watch_controller.go","path":"modules/eks-aws-auth-merger/aws-auth-merger/configmap_watch_controller.go","sha":"15852b7b2437ef1a46c4d66176ffaf48db294d20"},{"name":"debouncer.go","path":"modules/eks-aws-auth-merger/aws-auth-merger/debouncer.go","sha":"2376c66675e380b636421c17ea573fa12a141dd0"},{"name":"debouncer_test.go","path":"modules/eks-aws-auth-merger/aws-auth-merger/debouncer_test.go","sha":"e6cd8e44503b4dea3e81b26ff20db39c35c8c72e"},{"name":"go.mod","path":"modules/eks-aws-auth-merger/aws-auth-merger/go.mod","sha":"ad9c8a634f05d1a9da60b58f9926c5daf25d8a87"},{"name":"go.sum","path":"modules/eks-aws-auth-merger/aws-auth-merger/go.sum","sha":"2a774f51d177f093270357e0ece6ec160ad9ddc3"},{"name":"main.go","path":"modules/eks-aws-auth-merger/aws-auth-merger/main.go","sha":"9061cabd38668a323387cf8b7252ed2b45cbf2b9"},{"name":"mapping.go","path":"modules/eks-aws-auth-merger/aws-auth-merger/mapping.go","sha":"1d0a36862e4b4a24ddd6112b016acce54e643775"},{"name":"mapping_test.go","path":"modules/eks-aws-auth-merger/aws-auth-merger/mapping_test.go","sha":"02c91e45a128d6c306ec42a548ac9023df350e78"}]},{"name":"core-concepts.md","path":"modules/eks-aws-auth-merger/core-concepts.md","sha":"2da0061c35747e9f280f8a440adfc4534da40fa4"},{"name":"main.tf","path":"modules/eks-aws-auth-merger/main.tf","sha":"b3c04ea9580103dd085d998a828d1727b10f4b11"},{"name":"outputs.tf","path":"modules/eks-aws-auth-merger/outputs.tf","sha":"d733fb246403f97ac011cbedf3f1d2761badef82"},{"name":"variables.tf","path":"modules/eks-aws-auth-merger/variables.tf","sha":"429e5990df785c4c01c4a07668d41ce648e4e68b"}]},{"name":"eks-cluster-control-plane","children":[{"name":"README.md","path":"modules/eks-cluster-control-plane/README.md","sha":"ad90d83c6eea189d50c22069865c3f8cda1acd6a"},{"name":"dependencies.tf","path":"modules/eks-cluster-control-plane/dependencies.tf","sha":"5c71c526fdc6e550e5cf51dcb9a89dae10e8fca8"},{"name":"main.tf","path":"modules/eks-cluster-control-plane/main.tf","sha":"ee4cc58084adb3853b365931df2c2056cce2843c"},{"name":"outputs.tf","path":"modules/eks-cluster-control-plane/outputs.tf","sha":"321bbacc1b948640032fa099b9b268f2e8ccbb08"},{"name":"scripts","children":[{"name":"find_and_run_kubergrunt.py","path":"modules/eks-cluster-control-plane/scripts/find_and_run_kubergrunt.py","sha":"9007eeb70b6d693de19862fb6833a1acba756519"}]},{"name":"templates","children":[{"name":"kubectl_config.tpl","path":"modules/eks-cluster-control-plane/templates/kubectl_config.tpl","sha":"083a5e914505363541190db3ee412d8d9e15b4ec"}]},{"name":"variables.tf","path":"modules/eks-cluster-control-plane/variables.tf","sha":"f335dc11246e0c5b69f4798b25cf7adfe5e13228"}]},{"name":"eks-cluster-managed-workers","children":[{"name":"README.md","path":"modules/eks-cluster-managed-workers/README.md","sha":"54382d2f993b4b5e9cac30e9b44c36138d0e8ea2"},{"name":"main.tf","path":"modules/eks-cluster-managed-workers/main.tf","sha":"a34f1009fac7b210f34234d7bf0eb282c9acdbd1"},{"name":"outputs.tf","path":"modules/eks-cluster-managed-workers/outputs.tf","sha":"391b5aff36a080568d94aae450d00b78488fb2e4"},{"name":"variables.tf","path":"modules/eks-cluster-managed-workers/variables.tf","sha":"8f9df289653714407fb63c5dac0a9ec4047feec1"}]},{"name":"eks-cluster-workers-cross-access","children":[{"name":"README.md","path":"modules/eks-cluster-workers-cross-access/README.md","sha":"6c4e50bda62acc6c06d836488ef54f7119f27aee"},{"name":"main.tf","path":"modules/eks-cluster-workers-cross-access/main.tf","sha":"b71c29192fb85d5258060504068ec73daa223e1c"},{"name":"outputs.tf","path":"modules/eks-cluster-workers-cross-access/outputs.tf","sha":"c6c7f7a89007c55be5470ffd639c05c3fb052ad7"},{"name":"variables.tf","path":"modules/eks-cluster-workers-cross-access/variables.tf","sha":"d64aab893b6e909416189e985f072dd8809dfa2f"}]},{"name":"eks-cluster-workers","children":[{"name":"README.md","path":"modules/eks-cluster-workers/README.md","sha":"e2ff0e1c7054539c1802087972ce6f2954c57ab0"},{"name":"dependencies.tf","path":"modules/eks-cluster-workers/dependencies.tf","sha":"d177e89ddc5cb6b4ab5b36ec96fd1ec22a008a49"},{"name":"main.tf","path":"modules/eks-cluster-workers/main.tf","sha":"2d02f215af96e34bc8e96fd5afbff759f9d19b9c"},{"name":"outputs.tf","path":"modules/eks-cluster-workers/outputs.tf","sha":"d2dddf7803dbd3712c8f953c811c926eb141ace3"},{"name":"variables.tf","path":"modules/eks-cluster-workers/variables.tf","sha":"8c9cfc6df64a257898381eeda46df2a42e053ed5"}]},{"name":"eks-container-logs","children":[{"name":"README.md","path":"modules/eks-container-logs/README.md","sha":"3af3ffd83b5c0205f122ecba654dec888c6efd72"},{"name":"main.tf","path":"modules/eks-container-logs/main.tf","sha":"7eeee493d2178976655d57cf8ccccda63aebd38f"},{"name":"outputs.tf","path":"modules/eks-container-logs/outputs.tf","sha":"7061ed458fec528c8b8b587291f0eccb4324fb72"},{"name":"variables.tf","path":"modules/eks-container-logs/variables.tf","sha":"054e7ec18beb7df99e6bd58a205a07ef20a782c0"}]},{"name":"eks-iam-role-assume-role-policy-for-service-account","children":[{"name":"README.md","path":"modules/eks-iam-role-assume-role-policy-for-service-account/README.md","sha":"efbbbd70fea3661c662750768facb7950239ffa3"},{"name":"main.tf","path":"modules/eks-iam-role-assume-role-policy-for-service-account/main.tf","sha":"fb3a46eb067b6cc7d27eebec673e90faea148640"},{"name":"outputs.tf","path":"modules/eks-iam-role-assume-role-policy-for-service-account/outputs.tf","sha":"c2910cec89910bb06a157311ac8c4bf72835dfe5"},{"name":"variables.tf","path":"modules/eks-iam-role-assume-role-policy-for-service-account/variables.tf","sha":"dc660ddf84158851145289f6036a0fc19fbf7ce4"}]},{"name":"eks-k8s-cluster-autoscaler-iam-policy","children":[{"name":"README.md","path":"modules/eks-k8s-cluster-autoscaler-iam-policy/README.md","sha":"a22e2264a296fe1bf00f2c8b2f72ae728d0277c3"},{"name":"main.tf","path":"modules/eks-k8s-cluster-autoscaler-iam-policy/main.tf","sha":"7cbdf6fd5bf472c64e6ec4afe215a63e05de9352"},{"name":"outputs.tf","path":"modules/eks-k8s-cluster-autoscaler-iam-policy/outputs.tf","sha":"8b6c4e1747b3fa6a88c6233ec87aa2f450dfd334"},{"name":"variables.tf","path":"modules/eks-k8s-cluster-autoscaler-iam-policy/variables.tf","sha":"7534ac11f3cd71e1a1bf9521065a474f5f56ec3c"}]},{"name":"eks-k8s-cluster-autoscaler","children":[{"name":"README.md","path":"modules/eks-k8s-cluster-autoscaler/README.md","sha":"a74848607c42fcef696f121c2506ace0b83ced87"},{"name":"main.tf","path":"modules/eks-k8s-cluster-autoscaler/main.tf","sha":"48d7ce0d970628e6d790d5fedd9404619c386f56"},{"name":"templates","children":[{"name":"node_affinity.yaml","path":"modules/eks-k8s-cluster-autoscaler/templates/node_affinity.yaml","sha":"c6eaf8e94fa7c893857cc009df954443239a8fe0"},{"name":"values.yaml","path":"modules/eks-k8s-cluster-autoscaler/templates/values.yaml","sha":"081ab7f76dc4e753975b427d7cee51589965715d"}]},{"name":"variables.tf","path":"modules/eks-k8s-cluster-autoscaler/variables.tf","sha":"5f6eb57a008bc54ca3cb57c5e9d81ea41982251f"}]},{"name":"eks-k8s-external-dns-iam-policy","children":[{"name":"README.md","path":"modules/eks-k8s-external-dns-iam-policy/README.md","sha":"a33d41f9824e6270ef4573d6b7e22b394224689c"},{"name":"main.tf","path":"modules/eks-k8s-external-dns-iam-policy/main.tf","sha":"ec6e9579c2dfe1a7bcd6cc18518e6e8ed69ee58d"},{"name":"outputs.tf","path":"modules/eks-k8s-external-dns-iam-policy/outputs.tf","sha":"21604a63b741b94ea9ebffd20b18772131020fcf"},{"name":"variables.tf","path":"modules/eks-k8s-external-dns-iam-policy/variables.tf","sha":"250152e6bfeb02a16bed4151ffc7156636db1bd9"}]},{"name":"eks-k8s-external-dns","children":[{"name":"README.md","path":"modules/eks-k8s-external-dns/README.md","sha":"0feaa9793a59843f8999449ffcb2a53e9d33a120"},{"name":"main.tf","path":"modules/eks-k8s-external-dns/main.tf","sha":"834d11a8e507418afd068cdcd95da58ad52df27c"},{"name":"variables.tf","path":"modules/eks-k8s-external-dns/variables.tf","sha":"3b5a3a6ec58f4ac03f9139d88311c4980d75c8fc"}]},{"name":"eks-k8s-role-mapping","children":[{"name":"README.md","path":"modules/eks-k8s-role-mapping/README.md","sha":"2962e93307761b2356c62f0ac8068dc01f98d9f4"},{"name":"main.tf","path":"modules/eks-k8s-role-mapping/main.tf","sha":"b32ed731deb7aee40a5cbf8e5935b7109dbf703f"},{"name":"outputs.tf","path":"modules/eks-k8s-role-mapping/outputs.tf","sha":"95d4d4ec652bb541b91a2844e00f68064b423e60"},{"name":"variables.tf","path":"modules/eks-k8s-role-mapping/variables.tf","sha":"5c696a015d9e8b44e5687b758c61ac49f3a4b7ef"}]},{"name":"eks-scripts","children":[{"name":"README.md","path":"modules/eks-scripts/README.md","sha":"96baaf535647b9f4c364d6a19057bcccb42df2be"},{"name":"bin","children":[{"name":"map-ec2-tags-to-node-labels","path":"modules/eks-scripts/bin/map-ec2-tags-to-node-labels","sha":"8fd3a233d7ac8244394bd6e167271ca012da191b"},{"name":"map_ec2_tags_to_node_labels.py","path":"modules/eks-scripts/bin/map_ec2_tags_to_node_labels.py","sha":"f75ad19587e95b2bd8924125ea2a1a697154909f"}]},{"name":"dev_requirements.txt","path":"modules/eks-scripts/dev_requirements.txt","sha":"f56f9d1629a85734fe16ed70f00f36b830cd97c9"},{"name":"install.sh","path":"modules/eks-scripts/install.sh","sha":"7f192fca97b098482a8a398019d4d53f45dba478"}]},{"name":"eks-vpc-tags","children":[{"name":"README.md","path":"modules/eks-vpc-tags/README.md","sha":"b53e923baaa79718b55a272158ff9b710871a6ce"},{"name":"main.tf","path":"modules/eks-vpc-tags/main.tf","sha":"1464722445fde7b595de4cfeff0703bc3049f7d9"},{"name":"outputs.tf","path":"modules/eks-vpc-tags/outputs.tf","sha":"0ef2787cfd02ea8668c687302b1929618079a0b2"},{"name":"variables.tf","path":"modules/eks-vpc-tags/variables.tf","sha":"a6e332e9da4e473e1e42b1ca6c7b0ba139a77cfb"}]}]},{"name":"rfc","children":[{"name":"shipping-logs-to-cloudwatch.md","path":"rfc/shipping-logs-to-cloudwatch.md","sha":"606410240f3c5b5fe85175c05429c52c7e46ad42"}]},{"name":"setup.cfg","path":"setup.cfg","sha":"981bc2bfd0b35029438d56c6d862a7f1519b8fe6"},{"name":"terraform-cloud-enterprise-private-module-registry-placeholder.tf","path":"terraform-cloud-enterprise-private-module-registry-placeholder.tf","sha":"ae586c0fe830819580e1009d41a9074f16e65bed"},{"name":"test","children":[{"name":"README.md","path":"test/README.md","sha":"9bf8180d731bdc892279fcdbcbb03d245f31f83a"},{"name":"eks_cluster_integration_test.go","path":"test/eks_cluster_integration_test.go","sha":"471a3100ec1db2b42e2f80f2184621b3553020f3"},{"name":"eks_cluster_managed_workers_test.go","path":"test/eks_cluster_managed_workers_test.go","sha":"0e67a65ab14f81d0872895e117b8ecb31acaca03"},{"name":"eks_cluster_test_helpers.go","path":"test/eks_cluster_test_helpers.go","sha":"e6b2979ffa4b9e933e18dcb4461ec319cf9a5932"},{"name":"eks_cluster_upgrade_test.go","path":"test/eks_cluster_upgrade_test.go","sha":"acd5c95a8024ed2d0cd4b3a76ddf7b9314b88fbc"},{"name":"eks_cluster_with_auth_merger_test.go","path":"test/eks_cluster_with_auth_merger_test.go","sha":"8be3ac2f37f7a939a7d94737188bd63c4e714863"},{"name":"eks_cluster_with_iam_role_test.go","path":"test/eks_cluster_with_iam_role_test.go","sha":"962deb50187b8daed1df05102a30562248b8137e"},{"name":"eks_cluster_with_supporting_services_test.go","path":"test/eks_cluster_with_supporting_services_test.go","sha":"d1e60137ff053fa816c11fd4cb10e73c61b41126"},{"name":"eks_cluster_workers_optional_test.go","path":"test/eks_cluster_workers_optional_test.go","sha":"0f6179eb2f1d0d6809aebf7a16c7afa3f4e49fc3"},{"name":"eks_envelope_encryption_test.go","path":"test/eks_envelope_encryption_test.go","sha":"3d8b92c4d3d4244c6431ccae95f0faeb0328bdce"},{"name":"eks_fargate_cluster_disable_public_endpoint_test.go","path":"test/eks_fargate_cluster_disable_public_endpoint_test.go","sha":"25ba0984ef5979ca146d16b63654559939d822db"},{"name":"eks_fargate_cluster_irsa_test.go","path":"test/eks_fargate_cluster_irsa_test.go","sha":"8a1e4f4599f63b357af1ca36b135898e7663f1fe"},{"name":"eks_fargate_cluster_public_access_cidr_test.go","path":"test/eks_fargate_cluster_public_access_cidr_test.go","sha":"2a82ad5a0bbb9311bb9c91a2c0be3f3dbe1b4d5e"},{"name":"eks_fargate_cluster_test.go","path":"test/eks_fargate_cluster_test.go","sha":"a50d3691cbdec0ba41e2212015105254d7a516c7"},{"name":"eks_fargate_cluster_with_supporting_services_test.go","path":"test/eks_fargate_cluster_with_supporting_services_test.go","sha":"a236dc2c1647da144a3fa973492b18ad80d64103"},{"name":"eks_mixed_cluster_dns_test.go","path":"test/eks_mixed_cluster_dns_test.go","sha":"069332615ab046026f91262ebfb3715786132895"},{"name":"errors.go","path":"test/errors.go","sha":"be062fe0205ff82db8183d0fde639aa1883013ad"},{"name":"go.mod","path":"test/go.mod","sha":"33ee239df2ce37e72081b40ac7309dea8ec12a42"},{"name":"go.sum","path":"test/go.sum","sha":"82630e6bf6b9e8f15f3caf3a751e17077135cd50"},{"name":"kubefixtures","children":[{"name":"autoscaler-test-pods-deployment.yml","path":"test/kubefixtures/autoscaler-test-pods-deployment.yml","sha":"b2d94c4bfa729b639290ee21629c19ca6ea694ee"},{"name":"eks-irsa-test.yml","path":"test/kubefixtures/eks-irsa-test.yml","sha":"db5439cf6d38873dbae71daa4197d6947990a94a"},{"name":"eks-k8s-role-mapping-test-role.yml","path":"test/kubefixtures/eks-k8s-role-mapping-test-role.yml","sha":"ede7587308d2a4ecf55042b05800099c43f3af7d"},{"name":"kube-system-sa-admin-binding.yml","path":"test/kubefixtures/kube-system-sa-admin-binding.yml","sha":"282d406512102cbe54e952575f26e7e0fbb2aa9a"},{"name":"nginx-deployment.yml","path":"test/kubefixtures/nginx-deployment.yml","sha":"a58866e59c113635af24982cfb0b530f0c416af0"},{"name":"robust-nginx-deployment.yml","path":"test/kubefixtures/robust-nginx-deployment.yml","sha":"87ead0f9733e422099bc430ed281e2054e698f10"}]},{"name":"script_tests","children":[{"name":"executor.sh","path":"test/script_tests/executor.sh","sha":"458c534996fbc045081d1cfae521c090f6787a7f"},{"name":"requirements.txt","path":"test/script_tests/requirements.txt","sha":"7372bb177f500724ac7b3d1730d5018d768e538b"},{"name":"test_map_ec2_tags_to_node_labels.py","path":"test/script_tests/test_map_ec2_tags_to_node_labels.py","sha":"1bb3a5eae3727c0e6caf29c2cf4b7d596bb9a161"},{"name":"tox.ini","path":"test/script_tests/tox.ini","sha":"3a18859a7fbe2f3897fd85f69b081e308f8e47a1"}]},{"name":"terratest_options.go","path":"test/terratest_options.go","sha":"f6fe0b9bc873153f585b1da3c7a2dd4c73fa9eb5"},{"name":"test_debug_helpers.go","path":"test/test_debug_helpers.go","sha":"c71a7a9d5b68f0f59d2518496d9f5893206b5e22"},{"name":"test_helpers.go","path":"test/test_helpers.go","sha":"4633eab64d3382614ac38d1f3840e45c522486e2"}]}]},"detailsContent":"<h1 class=\"preview__body--title\" id=\"eks-cluster-with-supporting-services\">EKS Cluster with Supporting Services</h1><div class=\"preview__body--border\"></div><p>This example provisions an EKS cluster that:</p>\n<ul>\n<li>Has two dedicated worker node groups managed by separate ASGs for different functions:\n<ul>\n<li><code>core</code> nodes: Worker nodes intended to run core services, such as <code>kiam</code>. These services require additional\npermissions to function, and therefore require a more locked down EC2 instance configuration.</li>\n<li><code>application</code> nodes: Worker nodes intended to run application services.</li>\n</ul>\n</li>\n<li>Nodes are tagged with labels inherited from the EC2 instance tags, using the <code>map-ec2-tags-to-node-labels</code> script in\n<a href=\"/repos/v0.44.0/terraform-aws-eks/modules/eks-scripts\" class=\"preview__body--description--blue\">the eks-scripts module</a>.</li>\n<li>Deploys core admin services that provide various features:\n<ul>\n<li>FluentD for shipping Kubernetes logs (including container logs) to CloudWatch.</li>\n<li>ALB Ingress Controller for exposing services using ALBs.</li>\n<li>external-dns for tying Route 53 domains to exposed services.</li>\n</ul>\n</li>\n</ul>\n<h2 class=\"preview__body--subtitle\" id=\"prerequisites\">Prerequisites</h2>\n<p>This example depends on <code>Terraform</code>, <code>Packer</code>, and <code>kubergrunt</code>. You can also optionally install <code>kubectl</code> if\nyou would like explore the newly provisioned cluster. You can find instructions on how to install each tool below:</p>\n<ul>\n<li><a href=\"https://learn.hashicorp.com/terraform/getting-started/install.html\" class=\"preview__body--description--blue\" target=\"_blank\">Terraform</a>, minimum version: <code>0.12.7</code></li>\n<li><a href=\"https://www.packer.io/intro/getting-started/install.html\" class=\"preview__body--description--blue\" target=\"_blank\">Packer</a></li>\n<li><a href=\"/repos/kubergrunt#installation\" class=\"preview__body--description--blue\">kubergrunt</a>, minimum version: <code>0.6.2</code></li>\n<li>(Optional) <a href=\"https://kubernetes.io/docs/tasks/tools/install-kubectl/\" class=\"preview__body--description--blue\" target=\"_blank\">kubectl</a></li>\n</ul>\n<p>Finally, before you begin, be sure to set up your AWS credentials as environment variables so that all the commands\nbelow can authenticate to the AWS account where you wish to deploy this example. You can refer to our blog post series\non AWS authentication (<a href=\"https://blog.gruntwork.io/a-comprehensive-guide-to-authenticating-to-aws-on-the-command-line-63656a686799\" class=\"preview__body--description--blue\" target=\"_blank\">A Comprehensive Guide to Authenticating to AWS on the Command\nLine</a>) for\nmore information.</p>\n<h2 class=\"preview__body--subtitle\" id=\"overview\">Overview</h2>\n<p>Unlike the other examples, this example is spread across multiple terraform submodules. Refer to <a href=\"#why-are-there-multiple-terraform-submodules-in-this-example\" class=\"preview__body--description--blue\">Why are there\nmultiple Terraform submodules in this example?</a> for more\ninformation on why the example is structured this way.</p>\n<p>As such, you will be deploying the example through a multi step process involving the following steps:</p>\n<ol>\n<li><a href=\"#create-a-new-ami-with-the-helper-scripts-installed\" class=\"preview__body--description--blue\">Create a new AMI with the helper scripts installed using <code>packer</code></a></li>\n<li><a href=\"#apply-terraform-templates\" class=\"preview__body--description--blue\">Apply Terraform Templates</a></li>\n<li><a href=\"#deploy-eks-cluster\" class=\"preview__body--description--blue\">Deploy EKS cluster</a></li>\n<li><a href=\"#deploy-core-services\" class=\"preview__body--description--blue\">Deploy Core Services</a></li>\n<li><a href=\"#optional-deploy-nginx\" class=\"preview__body--description--blue\">(Optional) Deploy Nginx</a></li>\n</ol>\n<p>Once the cluster is deployed, take a look at <a href=\"#where-to-go-from-here\" class=\"preview__body--description--blue\">Where to go from here</a> for ideas on what to do\nnext.</p>\n<h2 class=\"preview__body--subtitle\" id=\"create-a-new-ami-with-the-helper-scripts-installed\">Create a New AMI with the Helper Scripts Installed</h2>\n<p>This example depends on the <code>map-ec2-tags-to-node-labels</code> script to assist with mapping <a href=\"https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/Using_Tags.html\" class=\"preview__body--description--blue\" target=\"_blank\">the EC2 instance\ntags</a> into Node\n<a href=\"https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/\" class=\"preview__body--description--blue\" target=\"_blank\">Labels</a>. We will use <code>packer</code> to build a\ncustomized AMI on based on <a href=\"https://docs.aws.amazon.com/eks/latest/userguide/eks-optimized-ami.html\" class=\"preview__body--description--blue\" target=\"_blank\">the EKS optimized\nAMI</a> that includes the script.</p>\n<p>To build the AMI, you need to provide <code>packer</code> the build template and required variables. Since we will be installing a\nGruntwork module, we will need to setup Github access. This can be done by defining the <code>GITHUB_OAUTH_TOKEN</code> environment\nvariable with a personal access token. See https://github.com/gruntwork-io/gruntwork-installer#authentication for more\ninformation on how to set this up.</p>\n<p>Once the environment variable is set, you can run <code>packer build</code> to build the AMI:</p>\n<pre><span class=\"hljs-symbol\">packer</span> <span class=\"hljs-keyword\">build </span>packer/<span class=\"hljs-keyword\">build.json\n</span></pre>\n<p>This will spin up an EC2 instance, run the shell scripts to provision the machine, burn a new AMI, spin down the\ninstance, and then output the newly built AMI.</p>\n<p>Note: By default, the provided <code>packer</code> template will build a new AMI in the <code>us-east-1</code> region. If you would like to\nchange the region to build in, you can pass in <code>-var "region=us-east-2"</code> to override the default region.</p>\n<h2 class=\"preview__body--subtitle\" id=\"apply-terraform-templates\">Apply Terraform Templates</h2>\n<p>Once the AMI is built, we are ready to use it to deploy our EKS cluster. Unlike the other examples in this repo, this\nexample breaks up the code into multiple submodules. Refer to <a href=\"#why-are-there-multiple-terraform-submodules-in-this-example\" class=\"preview__body--description--blue\">Why are there multiple Terraform submodules in this\nexample?</a> for more information on why the example is\nstructured this way.</p>\n<p>To deploy our cluster, we will apply the templates in the following order:</p>\n<ol>\n<li><a href=\"#deploy-eks-cluster\" class=\"preview__body--description--blue\">eks-cluster: Deploy the EKS cluster with workers</a></li>\n<li><a href=\"#deploy-core-services\" class=\"preview__body--description--blue\">core-services: Deploy Core Services (e.g Helm Server) on to EKS cluster</a></li>\n<li><a href=\"#optional-deploy-nginx-service\" class=\"preview__body--description--blue\">(Optional) nginx-service: Deploy nginx on to EKS cluster</a></li>\n</ol>\n<h3 class=\"preview__body--subtitle\" id=\"deploy-eks-cluster\">Deploy EKS cluster</h3>\n<p>The code for deploying an EKS cluster with its worker groups is defined in <a href=\"/repos/v0.44.0/terraform-aws-eks/examples/eks-cluster-with-supporting-services/eks-cluster\" class=\"preview__body--description--blue\">the <code>eks-cluster</code> submodule</a>.\nThis Terraform example, when applied, will deploy a VPC, launch an EKS control plane in there, and then provision two\nworker groups to run workloads. The two groups provided by the example are:</p>\n<ol>\n<li>A <code>core</code> worker group that is dedicated for running supporting services like <code>kiam</code> that may require more lockdown.\nThe nodes in this group are tainted with <code>NoSchedule</code> so that <code>Pods</code> are not scheduled there by default.</li>\n<li>An <code>application</code> worker group that is dedicated for running application services.</li>\n</ol>\n<p>To deploy the example, we need to first define the required variables. To define variables to use, create a new file in\nthe example directory called terraform.tfvars:</p>\n<pre>touch ./eks-cluster/<span class=\"hljs-keyword\">terraform</span>.tfvars\n</pre>\n<p>Then, create a new entry for each required variable (and any optional variables you would like to override). See the\n<code>variables.tf</code> file for a list of available variables. Below is a sample <code>terraform.tfvars</code> file:</p>\n<pre><span class=\"hljs-attr\">aws_region</span> = <span class=\"hljs-string\">\"us-west-2\"</span>\n<span class=\"hljs-attr\">eks_cluster_name</span> = <span class=\"hljs-string\">\"test-eks-cluster-with-supporting-services\"</span>\n<span class=\"hljs-attr\">vpc_name</span> = <span class=\"hljs-string\">\"test-eks-cluster-with-supporting-services-vpc\"</span>\n<span class=\"hljs-attr\">eks_worker_ami</span> = <span class=\"hljs-string\">\"ami-00000000000000000\"</span>\n</pre>\n<p><strong>NOTE</strong>: If you attempt to deploy into the <code>us-east-1</code> region, note that the availability zone <code>us-east-1e</code> does not\nsupport EKS. To work around this, use the <code>allowed_availability_zones</code> to control which zones are used to deploy EKS by\nadding the following to the tfvars file:\n<code>allowed_availability_zones = ["us-east-1a", "us-east-1b", "us-east-1c", "us-east-1d", "us-east-1f"]</code>.</p>\n<p>Once the variables are filled out, we are ready to apply the templates to provision our cluster. To do this, we need to\nrun <code>terraform init</code> followed by <code>terraform apply</code>:</p>\n<pre>cd eks-cluster\n<span class=\"hljs-keyword\">terraform</span> init\n<span class=\"hljs-keyword\">terraform</span> apply\ncd .. <span class=\"hljs-comment\"># go back to eks-cluster-with-supporting-services example folder</span>\n</pre>\n<p>At the end of this, you will have an EKS cluster with 2 ASG node worker pools. We will use <code>kubectl</code> to verify this.</p>\n<p>In order to use <code>kubectl</code>, we need to first set it up so that it can authenticate with our new EKS cluster. You can\nlearn more about how authentication works with EKS in our guide <a href=\"/repos/v0.44.0/terraform-aws-eks/core-concepts.md#how-do-i-authenticate-kubectl-to-the-eks-cluster\" class=\"preview__body--description--blue\">How do I authenticate kubectl to the EKS\ncluster?</a>. For now, you can run the <code>kubergrunt eks configure</code> command:</p>\n<pre>EKS_CLUSTER_ARN=$(cd eks-cluster && <span class=\"hljs-keyword\">terraform</span> <span class=\"hljs-keyword\">output</span> eks_cluster_arn | tr -d \\<span class=\"hljs-string\">\")\nkubergrunt eks configure --eks-cluster-arn $EKS_CLUSTER_ARN\n</span></pre>\n<p>At the end of this command, your default kubeconfig file (located at <code>~/.kube/config</code>) will have a new context that\nauthenticates with EKS. This context will be set as the default so that subsequent <code>kubectl</code> calls will target your\ndeployed eks cluster.</p>\n<p>You can now use <code>kubectl</code> to verify the two worker groups. Run <code>kubectl get nodes</code> and <code>kubectl describe nodes</code> to see\nthe associated labels of the nodes and verify there are two distinct labels.</p>\n<p>This will output information about the deployed cluster. Record the entries for <code>vpc_id</code>,\n<code>eks_openid_connect_provider_arn</code>, and <code>eks_openid_connect_provider_url</code>, as we will be using those in the next step.</p>\n<h3 class=\"preview__body--subtitle\" id=\"deploy-core-services\">Deploy Core Services</h3>\n<p>Once our EKS cluster is deployed, we can deploy core services on to it. The code for core services is defined in <a href=\"/repos/v0.44.0/terraform-aws-eks/examples/eks-cluster-with-supporting-services/core-services\" class=\"preview__body--description--blue\">the\n<code>core-services</code> submodule</a>. This Terraform example, when applied, deploys administrative services:</p>\n<ul>\n<li>fluentd-cloudwatch: Used to ship Kubernetes logs (including container logs) to CloudWatch.</li>\n<li>aws-alb-ingress-controller: Used to map <code>Ingress</code> resources into AWS ALBs.</li>\n<li><code>external-dns</code>: Used for tieing Route 53 domains to exposed services.</li>\n</ul>\n<p>Like the previous step, create a <code>terraform.tfvars</code> file and fill it in. Below is a sample entry:</p>\n<pre><span class=\"hljs-attr\">aws_region</span> = <span class=\"hljs-string\">\"us-west-2\"</span>\n<span class=\"hljs-attr\">eks_cluster_name</span> = <span class=\"hljs-string\">\"test-eks-cluster-with-supporting-services\"</span>\n<span class=\"hljs-attr\">iam_role_for_service_accounts_config</span> = {\n <span class=\"hljs-attr\">openid_connect_provider_arn</span> = <span class=\"hljs-string\">\"arn\"</span>\n <span class=\"hljs-attr\">openid_connect_provider_url</span> = <span class=\"hljs-string\">\"url\"</span>\n}\n<span class=\"hljs-attr\">eks_vpc_id</span> = <span class=\"hljs-string\">\"VPC ID\"</span>\n</pre>\n<p>The <code>eks_vpc_id</code>, <code>openid_connect_provider_arn</code>, and <code>openid_connect_provider_url</code> input variables should be\nthe outputs that you recorded in the previous step.</p>\n<p>Once the tfvars file is created, you can init and apply the templates:</p>\n<pre>touch ./core-services/<span class=\"hljs-keyword\">terraform</span>.tfvars\n<span class=\"hljs-comment\"># Fill in the tfvars file with the variable inputs</span>\ncd core-services\n<span class=\"hljs-keyword\">terraform</span> init\n<span class=\"hljs-keyword\">terraform</span> apply\ncd .. <span class=\"hljs-comment\"># go back to eks-cluster-with-supporting-services example folder</span>\n</pre>\n<p>Additionally, the cluster should be shipping logs to CloudWatch. You can load the CloudWatch logs in the UI by\nnavigating to CloudWatch in the AWS console and looking for the log group with the same name as the EKS cluster.</p>\n<h3 class=\"preview__body--subtitle\" id=\"optional-deploy-nginx-service\">(Optional) Deploy Nginx Service</h3>\n<p>The <a href=\"/repos/v0.44.0/terraform-aws-eks/examples/eks-cluster-with-supporting-services/nginx-service\" class=\"preview__body--description--blue\"><code>nginx-service</code> submodule</a> shows an example of how to use Helm to deploy an application on to your\nEKS cluster. This example will:</p>\n<ul>\n<li>Setup <a href=\"/repos/helmcharts\" class=\"preview__body--description--blue\">the Gruntwork Helm Chart Repository</a></li>\n<li>Install Nginx using the <a href=\"/repos/helm-kubernetes-services/charts/k8s-service\" class=\"preview__body--description--blue\"><code>k8s-service</code> helm chart</a></li>\n<li>As part of the install, an ALB will be provisioned that routes to the nginx Pods.</li>\n</ul>\n<p>Like the previous step, create a <code>terraform.tfvars</code> file and fill it in. Below is a sample entry:</p>\n<pre><span class=\"hljs-attr\">aws_region</span> = <span class=\"hljs-string\">\"us-west-2\"</span>\n<span class=\"hljs-attr\">eks_cluster_name</span> = <span class=\"hljs-string\">\"test-eks-cluster-with-supporting-services\"</span>\n</pre>\n<p>Once the tfvars file is created, you can init and apply the templates:</p>\n<pre>touch ./nginx-service/<span class=\"hljs-keyword\">terraform</span>.tfvars\n<span class=\"hljs-comment\"># Fill in the tfvars file with the variable inputs</span>\ncd nginx-service\n<span class=\"hljs-keyword\">terraform</span> init\n<span class=\"hljs-keyword\">terraform</span> apply\ncd .. <span class=\"hljs-comment\"># go back to eks-cluster-with-supporting-services example folder</span>\n</pre>\n<p>Once Terraform finishes, the nginx service should be available. To get the endpoint, you can query Kubernetes using\n<code>kubectl</code> for the Service information:</p>\n<pre><span class=\"hljs-comment\"># Prerequisite: Setup environment variables to auth to AWS</span>\n<span class=\"hljs-comment\"># Use kubectl to get Service endpoint</span>\nkubectl \\\n get ingresses \\\n --namespace kube-system \\\n --selector app.kubernetes.io/instance=nginx,app.kubernetes.io/name=nginx \\\n -o jsonpath \\\n --template '{.items[<span class=\"hljs-number\">0</span>].status.loadBalancer.ingress[<span class=\"hljs-number\">0</span>].hostname}'\n</pre>\n<p>This will output the ALB endpoint to the console. When you hit the endpoint, you should be able to see the welcome page\nfor nginx. If the service isn't available or you don't get the endpoint, wait a few minutes to give Kubernetes a chance\nprovision the ALB.</p>\n<h2 class=\"preview__body--subtitle\" id=\"where-to-go-from-here\">Where to go from here</h2>\n<p>Now that you have your cluster, you can do a few things to explore the cluster:</p>\n<ul>\n<li>If you setup an SSH key with the <code>eks_worker_keypair_name</code> variable, try SSH-ing to the nodes to see the running\nprocesses.</li>\n</ul>\n<h2 class=\"preview__body--subtitle\" id=\"why-are-there-multiple-terraform-submodules-in-this-example\">Why are there multiple Terraform submodules in this example?</h2>\n<p>Breaking up your code not only improves readability, but it also helps with maintainability by keeping the surface area\nsmall. Typically you would want to structure your code so that resources that are deployed more frequently are isolated\nfrom critical resources that might bring down the cluster. For example, you might upgrade the supporting services\nregularly, but you might not touch your VPC once it is set up. Therefore, you do not want to manage your Helm resources\nwith the VPC, as everytime you update the services, you would be putting the cluster at risk.</p>\n<p>Additionally, breaking up the code into modules helps introduce dependency ordering. Terraform is notorious for having\nbugs and subtle issues that make it difficult to reliably introduce a dependency chain, especially when you have\nmodules. For example, you can't easily define dependencies such that the resources in a module depend on other resources\nor modules external to the module (see https://github.com/hashicorp/terraform/issues/1178).</p>\n<p>The dependency logic between launching the EKS cluster, setting up Helm, and deploying services using Helm is tricky to\nencode in terraform such that it works reliably. For example, while it is fairly easy to get the resources to deploy in\norder such that they are available, it is tricky to destroy the resources in the right order. For example, it is ok to\ndeploy services in parallel with the worker nodes, such that the worker nodes are coming up while the deployment script\nis executing. The same thing can't be done for destruction, because the script needs to undeploy the services before\nundeploying the worker nodes.</p>\n<p>In summary, this example breaks up the Terraform code as an example of how one might modularize their EKS cluster code,\nin addition to making the dependency management more explicit.</p>\n<h2 class=\"preview__body--subtitle\" id=\"troubleshooting\">Troubleshooting</h2>\n<p><strong>When destroying <code>eks-cluster</code>, I get an error with destroying VPC related resources.</strong></p>\n<ul>\n<li>EKS relies on the <a href=\"https://github.com/aws/amazon-vpc-cni-k8s\" class=\"preview__body--description--blue\" target=\"_blank\"><code>amazon-vpc-cni-k8s</code></a> plugin to allocate IP addresses to\nthe pods in the Kubernetes cluster. This plugin works by allocating secondary ENI devices to the underlying worker\ninstances. Depending on timing, this plugin could interfere with destroying the cluster in this example. Specifically,\nterraform could shutdown the instances before the VPC CNI pod had a chance to cull the ENI devices. These devices are\nmanaged outside of terraform, so if they linger, it could interfere with destroying the VPC.\n<ul>\n<li>To workaround this limitation, you have to go into the console and delete the ENI associated with the VPC. Then,\nretry the destroy call.</li>\n</ul>\n</li>\n</ul>\n","repoName":"terraform-aws-eks","repoRef":"v0.36.0","serviceDescriptor":{"serviceName":"EC2 Kubernetes Service (EKS) Cluster","serviceRepoName":"terraform-aws-eks","serviceRepoOrg":"gruntwork-io","cloudProviders":["aws"],"description":"Deploy a Kubernetes cluster on top of Amazon EC2 Kubernetes Service (EKS).","imageUrl":"eks.png","licenseType":"subscriber","technologies":["Terraform","Python","Bash"],"compliance":[],"tags":[""]},"serviceCategoryName":"Docker orchestration","fileName":"README.md","filePath":"/examples/eks-cluster-with-supporting-services","title":"Repo Browser: EC2 Kubernetes Service (EKS) Cluster","description":"Browse the repos in the Gruntwork Infrastructure as Code Library."}
.circleci
.gitignore
