Part 3. How to Manage Your Apps Using Orchestration Tools

The first thing you need for server orchestration is a bunch of servers. You can spin up several EC2 instances by reusing the Ansible playbook and inventory file you created in Part 2.^[11] Head into the fundamentals-of-devops folder you’ve been using to work through the examples in this blog post series, create a new ch3/ansible subfolder, and copy into that subfolder create_ec2_instances_playbook.yml and inventory.aws_ec2.yml from Part 2:

This time, you’ll override the default variables in the playbook to create multiple EC2 instances with different names and ports. To do this, create a file called sample-app-vars.yml, with the contents shown in Example 24:

This variables file will create three servers named sample_app_instances that allow incoming HTTP requests on port 8080. To run the playbook, authenticate to AWS, and run the ansible-playbook command, adding the --extra-vars flag to pass in the variables file:

Next, you’ll need to create a group variables file to configure the SSH user, private key, and host key checking settings. Since sample-app-vars.yml set base_name to sample_app_instances, create a file called group_vars/sample_app_instances.yml, with the contents shown in Example 25:

Now you can configure the servers in this group to run the Node.js sample app, but with improved security and reliability. As explained in Watch out for snakes: these examples have several problems, the code used to deploy apps in the previous blog posts had security and reliability issues: e.g., running the app as a root user, listening on port 80, no automatic app restart in case of crashes, and so on. It’s time to fix these issues and get this code closer to something you could use in production.

Create a new playbook called configure_sample_app_playbook.yml, with the contents shown in Example 26:

Here’s what this playbook does:

For the nodejs-app role, create just one file, roles/nodejs-app/tasks/main.yml:

Put the code shown in Example 27 into tasks/main.yml:

The nodejs-app is fairly generic, usable with almost any Node.js app:

For the sample-app role, create two subfolders, files and tasks:

app.js is the exact same "Hello, World" Node.js sample app you saw in Part 1. Copy it into the files folder:

app.config.js is a new file that is used to configure PM2. So, what is PM2? PM2 is a process supervisor, which is a tool you can use to run your apps, monitor them, restart them after a reboot or a crash, manage their logging, and so on. Process supervisors provide one layer of auto healing for long-running apps. You’ll see other types of auto healing later in this post.

There are many process supervisors out there, including PM2, supervisord, and systemd (full list), with systemd as the one you’re likely to use, as it’s built into most Linux distributions these days. For this example, I picked PM2 because it has features designed specifically for Node.js apps. To use these features, create a configuration file called app.config.js, as shown in Example 28:

This file configures PM2 to do the following:

Finally, create tasks/main.yml with the contents shown in Example 29:

The preceding code does the following:

These changes address the concerns in Watch out for snakes: these examples have several problems: the code is more secure, as you’re no longer running as root, more reliable, as you’re using a process supervisor, and more performant, as you’re using all CPUs and running in production mode.

To try this code out, authenticate to AWS, and run the following command:

Ansible will discover your servers, and on each one, install Node.js, and run your sample app. At the end, you should see the IP addresses of servers, as shown in the following log output (truncated for readability):

Copy the IP of one of the three servers, open http://<IP>:8080 in your web browser, and you should see the familiar "Hello, World!" text once again.

While three servers is great for redundancy, it’s not so great for usability, as your users typically want just a single endpoint to hit. This requires deploying a load balancer, as described in the next section.

A load balancer is a piece of software that can distribute load across multiple servers or apps. You give your users a single endpoint to hit, which is the load balancer, and under the hood, the load balancer forwards the requests it receives to a number of different endpoints, using various algorithms (e.g., round-robin, hash-based, least-response-time, etc.) to process requests as efficiently as possible. There are many load balancer options, including those you run yourself, such as Apache, Nginx, and HAProxy, as well as managed load balancers, such as AWS Elastic Load Balancer and GCP Cloud Load Balancer (full list).

In the cloud, you’d most likely use a cloud load balancer, as you’ll see later in this blog post. However, for the purposes of server orchestration, I decided to show you a simplified example of how to run your own load balancer, as server orchestration techniques should work on-prem as well. Therefore, you’ll be deploying Nginx.

To do that, you need one more server. You can deploy one more EC2 instance using the same create_ec2_instances_playbook.yml, but with a new variables file, nginx-vars.yml, with the contents shown in Example 30:

This will create a single EC2 instance, with the name nginx_instances, and it will allow requests on port 80, which is the default port for HTTP. Authenticate to AWS and run the playbook with this vars file as follows:

This should create one more EC2 instance you can use for nginx. Since the base_name for that instance is nginx_instances, that will also be the group name in the inventory, so configure the variables for this group by creating group_vars/nginx_instances.yml with the contents shown in Example 31:

Now you can configure these servers to run Nginx by using an Ansible role called nginx that I’ve open sourced in the GitHub repo https://github.com/brikis98/devops-book-nginx-role. Using this role will give you a taste of what it’s like to use third-party roles, such as those on GitHub or Ansible Galaxy. The nginx role installs Nginx, has it listen on port 80, and configures it to load balance all traffic to the / URL across the list of servers you pass in via the servers input variable. To use the nginx role, create a file called requirements.yml with the contents shown in Example 32:

Next, run the ansible-galaxy command to install the role:

Now you can create a new playbook that uses this role in a file called configure_nginx_playbook.yml, with the contents shown in Example 33:

This playbook does the following:

Authenticate to AWS and run this playbook as follows:

Wait a few minutes for everything to deploy and in the end, you should see log output that looks like this:

The value on the left, "xxx.us-east-2.compute.amazonaws.com," is a domain name you can use to access the Nginx server. If you open http://xxx.us-east-2.compute.amazonaws.com (this time with no port number, as Nginx is listening on port 80, the default port for HTTP) in your browser, you should see "Hello, World!" yet again. Each time you refresh the page, Nginx will send that request to a different EC2 instance (known as round-robin load balancing). Congrats, you now have a single endpoint you can give your users, and that endpoint will automatically balance the load across multiple servers!

So you’ve now seen how to deploy using a server orchestration tool, but what about doing an update? Some configuration management tools support various deployment strategies (a topic you’ll learn more about in Part 5), such as a rolling deployment, where you update your servers in batches, so some servers are always running and serving traffic, while others are being updated. Example 34 shows how to update configure_sample_app_playbook.yml to do a rolling deployment:

Let’s give the rolling deployment a shot. Update the text that the app responds with in app.js, as shown in Example 35:

And re-run the playbook:

Ansible will roll out the change to one server at a time. When it’s done, if you refresh the Nginx IP in your browser, you should see the text "Fundamentals of DevOps!"

When you’re done experimenting with Ansible, you should manually undeploy the three EC2 instances as shown in Figure 9. This ensures that your account doesn’t start accumulating any unwanted charges.

Now that you’ve seen server orchestration, let’s move on to VM orchestration.

Head into the fundamentals-of-devops folder you created in Part 1 to work through the examples in this blog post series, and create a new subfolder for the Packer code:

Copy the Packer template and install-node.sh script you created in Part 2 into the new ch3/packer folder:

You should also copy app.js (the sample app) and app.config.js (the PM2 configuration file) from the Ansible example into the ch3/packer/sample-app folder:

If you had updated app.js to respond with "Fundamentals of DevOps!" to test out rolling deployments with Ansible, change the response text back to "Hello, World!" in ch3/packer/sample-app/app.js, as shown in Example 36:

Example 37 shows the updates to make to the Packer template:

There is just one change in the Packer template:

Update install-node.sh as shown in Example 38:

These are the same security and reliability improvements you did in the server orchestration section:

To build the AMI, authenticate to AWS, and run the following:

When the build is done, Packer will output the ID of the newly created AMI, which you will deploy in the next section.

The next step is to deploy the AMI. In Part 2, you used OpenTofu to deploy an AMI on a single EC2 instance. The goal now is to see VM orchestration at play, which means deploying a cluster with multiple servers. Most cloud providers offer a native way to run VMs across a cluster: AWS offers Auto Scaling Groups (ASG), GCP offers Managed Instance Groups, and Azure offers Scale Sets (full list). For this example, you’ll be using an AWS ASG.

Let’s use a reusable OpenTofu module called asg from this blog post series’s sample code repo to deploy the ASG. You can find the module in the ch3/tofu/modules/asg folder. This is a simple module that creates three main resources:

To use the asg module, create a live/asg-sample folder to act as a root module:

Inside this folder, create main.tf with the contents shown in Example 39:

The preceding code sets the following parameters:

Create a file called user-data.sh with the contents shown in Example 40:

This user data script switches to app-user, goes into the sample-app folder where Packer copied the sample app code, uses PM2 to run the sample app, and then saves the sample app to the list of apps that should be restarted after a reboot.

If you were to run apply right now, you’d get an ASG with three EC2 instances running your sample app. While this is great for redundancy, as discussed in the server orchestration section, you typically want to give your users just a single endpoint to hit. This requires deploying a load balancer, as described in the next section.

In the server orchestration section, you deployed your own load balancer using Nginx. One of the benefits of the cloud is that you can use managed services to solve common problems such as load balancing: e.g., AWS offers Elastic Load Balancers (ELB), GCP offers Cloud Load Balancers, and so on. Load balancer services such as AWS ELBs have a number of advantages over the simplified Nginx deployment you did earlier:

To be clear, you can do all of this with Nginx, too, but it’s a considerable amount of work. Using a managed service for load balancing can be a huge time saver, so let’s use an AWS ELB. There are actually several types of AWS ELBs; the one that’ll be the best fit for the simple sample app is the Application Load Balancer (ALB).

The blog post series’s sample code repo includes a module called alb in the ch3/tofu/modules/alb folder that you can use to deploy an ALB. It’s a simple module that deploys the ALB into the Default VPC and configures it to forward all requests to your servers, which should suffice for this blog post. Example 41 shows how to update the asg-sample module to use the alb module:

The preceding code sets the following parameters on the alb module:

There is one piece missing: how does the ALB know which EC2 instances to send traffic to? To connect the ALB and ASG, make the changes shown in Example 42:

Setting target_group_arns will change the ASG behavior in the following ways:

The final change to the asg-sample module is to add the load balancer’s domain name as an output variable in outputs.tf, as shown in Example 43:

To deploy the module, authenticate to AWS, and run the following commands:

When apply completes, you should see the ALB domain name as an output:

Open this domain name in your web browser, and you should see "Hello, World!" once again. Congrats, you now have a single endpoint, the load balancer domain name, that you can give your users, and when users hit it, the load balancer will distribute their requests across all the apps in your ASG!

You’ve now seen the initial deployment with VM orchestration, but what about rolling out updates? This is the topic of the next section.

Most VM orchestration tools have support for zero-downtime deployment. For example, AWS ASGs support rolling deployments through a feature called instance refresh. Example 44 shows how to enable instance refresh in the asg module:

The preceding code sets the following parameters:

Run apply one more time to enable the instance refresh setting. Once that’s done, you can try rolling out a change. For example, update app.js in the packer folder to respond with "Fundamentals of DevOps!", as shown in Example 45:

Next, in the ch3/packer folder, build a new AMI:

When the Packer build is complete, go back to the asg-sample module and run apply again. The module will automatically find the newly-built AMI, and you should then see a plan output that looks like this:

This plan output shows that the launch template has changed, due to the new AMI ID, and as a result, the version of the launch template used in the ASG has changed. This will result in an instance refresh. Type in yes, hit Enter, and AWS will kick off the instance refresh process in the background. If you go to the EC2 Console, click Auto Scaling Groups in the left nav, find your ASG, and click the "Instance refresh" tab, you should be able to see the instance refresh in progress, as shown in Figure 23.

During this process, the ASG will launch three new EC2 instances, and the ALB will start performing health checks on them. Once the new instances start to pass health checks, the ASG will undeploy the old instances, leaving you with just the three new instances running the new code. The whole process should take around five minutes.

During this deployment, the load balancer URL should always return a successful response, as this is a zero-downtime deployment. You can even check this by opening a new terminal tab, and running the following Bash one-liner (make sure to replace <LOAD_BALANCER_URL> with your load balancer URL):

This code runs curl, an HTTP client, in a loop, hitting your ALB once per second and allowing you to see the zero-downtime deployment in action. For the first couple minutes, you should see only "Hello, World!" responses from the old instances. Then, as new instances start to pass health checks, the ALB will begin sending traffic to them, and you should see the response from the ALB alternate between "Hello, World!" and "Fundamentals of DevOps!" After another couple minutes, the "Hello, World!" message will disappear, and you’ll see only "Fundamentals of DevOps!", which means all the old instances have been shut down. The output will look something like this:

Congrats, you’ve now seen VM orchestration in action, including rolling out changes following immutable infrastructure practices!

When you’re done experimenting with the ASG, run tofu destroy to undeploy all your infrastructure.

You’ve now seen server and VM orchestration, and how they compare. To give you one more comparison point, let’s move on to container orchestration.

First, install Docker Desktop (minimum version 4.0). Once it’s installed, you should have the docker command available on your command line. You can use the docker run command to run Docker images locally:

where IMAGE is the Docker image to run and COMMAND is an optional command to execute. For example, here’s how you can run a Bash shell in an Ubuntu 24.04 Docker image (the -it flag enables an interactive shell):

And voilà, you’re now in Ubuntu! If you’ve never used Docker before, this can seem fairly magical. Try running some commands. For example, you can look at the contents of /etc/os-release to verify you really are in Ubuntu:

How did this happen? Well, first, Docker searches its local cache for the ubuntu:24.04 image. If you don’t have that image downloaded already, Docker downloads it automatically from Docker Hub, which is a Docker Registry that contains shared Docker images. The ubuntu:24.04 image happens to be a public Docker image—an official one maintained by the Docker team—so you’re able to download it without any authentication. It’s also possible to create private Docker images that only certain authenticated users can use, as you’ll see later in this blog post.

Once the image is downloaded, Docker runs the image, executing the bash command, which starts an interactive Bash prompt, where you can type. Try running the ls command to see the list of files:

You might notice that’s not your filesystem. That’s because Docker images run in containers that are isolated at the userspace level. When you’re in a container, you can only see the filesystem, memory, networking, etc., in that container. Any data in other containers, or on the underlying host operating system, is not accessible to you. This is one of the things that makes Docker useful for running applications: the image format is self-contained, so Docker images run the same way no matter where you run them, and no matter what else is running there. To see this in action, write some text to a test.txt file as follows:

Next, exit the container by hitting Ctrl-D, and you should be back in your original command prompt on your underlying host OS. If you try to look for the test.txt file you just wrote, you’ll see that it doesn’t exist; the container’s filesystem is totally isolated from your host OS.

Now, try running the same Docker image again:

Notice that this time, since the ubuntu:24.04 image is already downloaded, the container starts almost instantly. This is another reason Docker is useful for running applications: unlike virtual machines, containers are lightweight, boot up quickly, and incur little CPU or memory overhead.

You may also notice that the second time you fired up the container, the command prompt looked different. That’s because you’re now in a totally new container, and any data you wrote in the previous one is no longer accessible to you. Run ls -al and you’ll see that the test.txt file does not exist. Containers are isolated not only from the host OS but also from each other.

Hit Ctrl-D again to exit the container, and back on your host OS, run docker ps:

This will show you all the containers on your system, including the stopped ones (the ones you exited). You can start a stopped container again by using the docker start <ID> command, setting ID to an ID from the CONTAINER ID column of the docker ps output. For example, here is how you can start the first container up again (and attach an interactive shell to it via the -ia flags):

You can confirm this is really the first container by outputting the contents of test.txt:

Hit Ctrl-D once more to exit the container and get back to your host OS.

Now that you’ve seen the basics of Docker, let’s look at what it takes to create your own Docker images, and use them to run web apps.

Let’s see how a container can be used to run a web app: in particular, the Node.js sample app you’ve been using throughout this blog post series. Create a new folder called docker:

Copy app.js from the server orchestration section into the docker folder (note: you do not need to copy app.config.js this time):

If you had updated app.js to respond with "Fundamentals of DevOps!" to test out rolling deployments with Ansible, change the response text back to "Hello, World!" in ch3/docker/app.js.

Next, create a file called Dockerfile, with the contents shown in Example 46:

Just as you used a Packer template to define how to build a VM image for your sample app, this Dockerfile is a template that defines how to build a Docker image for your sample app. This Dockerfile does the following:

To build a Docker image from this Dockerfile, use the docker build command:

The -t flag is the tag (name) to use for the Docker image. The preceding code sets the image name to "sample-app" and the version to "v1." Later on, if you make changes to the sample app, you’ll be able to build a new Docker image and give it a new version, such as "v2." The dot (.) at the end specifies the current directory (which should be the folder that contains your Dockerfile) as the build context, which is how you tell Docker what set of files it can access for the build. When the build finishes, you can use the docker run command to run your new image (with the -it and --init flags to ensure CTRL+C works correctly):

Your app is now listening on port 8080! However, if you open a new terminal on your host operating system and try to access the sample app, it won’t work:

What’s the problem? Actually, it’s not a problem but a feature! Docker containers are isolated from the host operating system and other containers, not only at the filesystem level but also in terms of networking. So while the container really is listening on port 8080, that is only on a port inside the container, which isn’t accessible on the host OS. If you want to expose a port from the container on the host OS, you have to do it via the -p flag.

First, hit Ctrl-C to shut down the sample-app container. Note that it’s Ctrl-C this time, not Ctrl-D, as you’re shutting down a process, rather than exiting an interactive prompt. Now rerun the container, but this time with the -p flag as follows:

Adding -p 8080:8080 to the command tells Docker to expose port 8080 inside the container on port 8080 of the host OS. You know to use port 8080 here, as you built this Docker image yourself, but if this was someone else’s image, you could use docker inspect on the image, and that will tell you about any ports that image labeled with EXPOSE. In another terminal on your host OS, you should now be able to see the sample app working:

Congrats, you now know how to run a web app locally using Docker! However, while using docker run directly is fine for local testing and learning, it’s not the way you’d run Dockerized apps in production. For that, you typically want to use a container orchestration tool such as Kubernetes, which is the topic of the next section.

Kubernetes (sometimes referred to as K8S) is a container orchestration tool, which means it’s a platform for running and managing containers on your servers, including scheduling, auto healing, auto scaling, load balancing, and more. Under the hood, Kubernetes consists of two main pieces, as shown in Figure 24:

Kubernetes is open source, and one of its strengths is that you can run it anywhere: in any cloud, on-prem, and even on your personal computer. A little later in this blog post, you’ll run Kubernetes in the cloud (in AWS), but for now, let’s start small and run it locally. This is easy to do if you installed a relatively recent version of Docker Desktop, as it has Kubernetes built-in. Open Docker Desktop’s preferences, and you should see Kubernetes in the nav, as shown in Figure 25.

Check the Enable Kubernetes checkbox, click Apply & Restart, and wait a few minutes for that to complete.

In the meantime, install kubectl (minimum version 1.30), which is the command-line tool for interacting with Kubernetes. To use kubectl, you must first update its configuration file, which lives in $HOME/.kube/config (that is, the .kube folder of your home directory), to tell it what Kubernetes cluster to connect to. Conveniently, when you enable Kubernetes in Docker Desktop, it updates this config file for you, adding a docker-desktop entry to it, so all you need to do is tell kubectl to use this configuration as follows:

Now you can use get nodes to check if your Kubernetes cluster is working:

The get nodes command shows you information about all the nodes in your cluster. Since you’re running Kubernetes locally, your computer is the only node, and it’s running both the control plane and acting as a worker node. You’re now ready to run some Docker containers.

To deploy something in Kubernetes, you create Kubernetes objects, which are persistent entities you write to the Kubernetes cluster (via the API server) that record your intent: e.g., your intent to have specific Docker images running. The cluster runs a reconciliation loop, which continuously checks the objects you stored in it and works to make the state of the cluster match your intent.

There are many different types of Kubernetes objects available. The one we’ll use to deploy your sample app is a Kubernetes Deployment, which is a declarative way to manage an application in Kubernetes. The Deployment allows you to declare what Docker images to run, how many copies of them to run (replicas), a variety of settings for those images (e.g., CPU, memory, port numbers, environment variables), and so on, and the Deployment will then work to ensure that the requirements you declared are always met.

One way to interact with Kubernetes is to create YAML files to define your Kubernetes objects, and to use the kubectl apply command to submit those objects to the cluster. Create a new folder called kubernetes to store these YAML files:

Within the kubernetes folder, create a file called sample-app-deployment.yml with the contents shown in Example 47:

This YAML file gives you a lot of functionality for just ~20 lines of code:

Use the kubectl apply command to apply your Deployment configuration:

This command should complete nearly instantaneously. How do you know if it actually worked? To answer that question, you can use kubectl to explore your cluster. First, run the get deployments command, and you should see your Deployment:

Here, you can see how Kubernetes uses metadata, as the name of the Deployment (sample-app-deployment) comes from your metadata block. You can use that metadata in API calls. For example, to get more details about a specific Deployment, run describe deployment <NAME>, where <NAME> is the name from the metadata:

This Deployment is reporting that all 3 replicas are available. To see those replicas, run the get pods command:

And to get the details about a specific pod, copy its name, and run describe pod:

From this output, you can see the containers that are running for each pod, which in this case, is just one container per pod running the sample-app:v1 Docker image you built earlier. You can also see the logs for a single pod by using the logs command, which is useful for understanding what’s going on and debugging:

Ah, there’s that familiar log output. You now have three replicas of your sample app running. But, just as you saw with server and VM orchestration, users will want just one endpoint, so it’s time to deploy a load balancer with Kubernetes.

Kubernetes has built-in support for load balancing. The typical way to set it up is to make use of another Kubernetes object, called a Kubernetes Service, which is a way to expose an app running in Kubernetes as a service you can talk to over the network. Example 48 shows the YAML code for a Kubernetes service, which you should put in a file called sample-app-service.yml:

Here’s what this code does:

You apply the Service the same way, using kubectl apply:

To see if your service worked, use the get services command:

The first service in the list is Kubernetes itself, which you can ignore. The second is the Service you created, with the name sample-app-loadbalancer (based on its own metadata block). You can get more details about your service by using the describe service command:

You can see that the load balancer is listening on localhost, at port 80, so you can test it out by opening http://localhost:

Congrats, you’re now able to deploy Docker containers and load balancers with Kubernetes! But how do you roll out updates?

Kubernetes Deployments have built-in support for rolling updates. Example 49 shows how to update the spec section of the Deployment to enable rolling updates:

This configures the Deployment to do a rolling update where it can deploy up to 3 extra pods during the deployment, similar to the instance refresh you saw with ASGs. Run apply to update the Deployment with these changes:

Now, make a change to the sample app in docker/app.js, such as returning the text "Fundamentals of DevOps!" instead of "Hello, World!", as shown in Example 50:

To deploy this change, first, build a new Docker image, with v2 as the new version:

The build will likely run in less than a second. This is because Docker has a built-in build cache, which, if used correctly, can dramatically speed up builds. Next, open sample-app-deployment.yml one more time, and in the spec section, update the image from sample-app:v1 to sample-app:v2, as shown in Example 51:

Run apply one more time to deploy this change:

Kubernetes will kick off the rolling update, and if you run get pods during this process, you’ll see up to six pods running at the same time (three old, three new):

After a little while, the three old pods will be undeployed, and you’ll be left with just the new ones. At that point, the load balancer will respond with the new text:

Congrats, you’ve done a zero-downtime rolling deployment with Kubernetes!

When you’re done testing, shut down your app by running kubectl delete:

So far, you’ve been running Kubernetes locally, which is great for learning and testing. However, for production deployments, you’ll need to run a Kubernetes cluster on servers in a data center. Kubernetes is a complicated system that’s more or less a cloud in and of itself, and setting it up and maintaining it is a significant undertaking. Fortunately, if you’re using the cloud, most cloud providers have managed Kubernetes offerings that make this considerably simpler. The one you’ll learn to use in this blog post series is Amazon’s Elastic Kubernetes Service (EKS), which can deploy and manage the control plane and worker nodes for you.

The blog post series’s sample code repo contains a module called eks-cluster in the ch3/tofu/modules/eks-cluster folder that you can use to deploy a simple EKS cluster, which includes the following:

To use the eks-cluster module, create a new folder called live/eks-sample:

Inside that folder, create main.tf with the contents shown in Example 52:

The preceding code configures the following parameters:

To deploy the EKS cluster, authenticate to AWS, and run the following commands:

After 3-5 minutes, the cluster should finish deploying. To explore the cluster with kubectl, you first need to authenticate to your cluster. The aws CLI has a built-in command for doing this:

Where <REGION> is the AWS region you deployed the EKS cluster into and <CLUSTER_NAME> is the name of the EKS cluster. The preceding code used us-east-2 and eks-sample for these, respectively, so you can run the following:

Once this is done, try running get nodes:

This output looks a bit different from when you ran the command with the Kubernetes cluster from Docker Desktop. You should see three nodes, each of which is an EC2 instance in your managed node group.

The next step is to try deploying the sample app into the EKS cluster. However, there’s one problem: you’ve created a Docker image for the sample app, but that image only lives on your own computer. The EKS cluster in AWS won’t be able to fetch the image from your computer, so you need to push the image to a container registry that EKS can read from, as described in the next section.

There are a number of container registries out there, including Docker Hub, Amazon Elastic Container Registry (ECR), and GitHub Container Registry (full list). If you’re using AWS, the easiest one to use is ECR, so let’s set that up.

For each Docker image you want to store in ECR, you have to create an ECR repository (ECR repo for short). The blog post series’s sample code repo includes a module called ecr-repo in the ch3/tofu/modules/ecr-repo folder that you can use to create an ECR repo. To use the ecr-repo module, create a new folder called live/ecr-sample:

In that folder, create main.tf with the contents shown in Example 53:

This code will create an ECR repo called "sample-app." Typically, the repo name should match your Docker image name.

You should also create outputs.tf with an output variable, as shown in Example 54:

The preceding code will output the URL of the ECR repo, which you’ll need to be able to push and pull images. To create the ECR repo, run the following commands:

After a few seconds, you should see the registry_url output:

Copy down that registry_url value, as you’ll need it shortly.

Before you can push your Docker image to this ECR repo, you have to build the image with the right CPU architecture. By default, the docker build command builds your Docker image for whatever CPU architecture you have on your own computer. For example, if you’re on a recent Macbook with an ARM CPU (e.g., the M series), your Docker images will be built for the arm64 architecture, which won’t work in the EKS cluster you just deployed, as the t2.micro worker nodes in that cluster use the amd64 architecture.

Therefore, you need to ensure that you build your Docker images for whatever architecture(s) you plan to deploy onto. Fortunately, Docker now ships with the buildx command, which makes it easy to build Docker images for multiple architectures. The first time you use buildx, you need to create a multi-platform-builder for your target architectures. For example, if you’re on an arm64 Mac, and you’re going to be deploying onto amd64 Linux servers, use the following command:

Now you can run the following command to build a Docker image of the sample app for both architectures (note the use of a new tag, v3, for these images):

Once the Docker image is built, to be able to push it to ECR, you need to tag it using the registry URL of the ECR repo that you got from the registry_url output:

Next, you need to authenticate to your ECR repo, which you can do using the aws and docker CLI tools, making sure to replace the last argument with the registry URL of your own ECR repo that you got from the registry_url output:

Finally, you can push the Docker image to your ECR repo:

The first time you push, it may take a minute or two to upload the image. Subsequent pushes should be faster due to Docker’s layer caching.

At this point, you are ready to deploy the sample app Docker image into the EKS cluster. The only change you need to make to the YAML you used to deploy locally is to switch the image in kubernetes/sample-app-deployment.yml to the v3 ECR repo URL, as shown in Example 55:

You can now apply both YAML files to deploy into your EKS cluster:

After a minute or two, if you run the get pods command, you should see something like this:

And if you run get services, you should see something like this:

If you look at the EXTERNAL-IP for sample-app-loadbalancer, you should see the domain name of an AWS ELB. Open this URL:

If you get "Could not resolve host" errors, it’s probably because the load balancer is still booting up or the health checks haven’t passed yet. Give it a minute or two more, and try again, and you should see the familiar "Fundamentals of DevOps!" text. Congrats, you’re now running a Dockerized application in a Kubernetes cluster in AWS!

When you’re done experimenting with the EKS cluster, run tofu destroy on both the eks-cluster and ecr-repo modules to undeploy all your infrastructure.

You’ve now seen server orchestration, VM orchestration, and container orchestration. That leaves just one orchestration approach to explore: serverless orchestration.

The blog post series’s sample code repo includes a module called lambda in the ch3/tofu/modules/lambda folder that can do the following:

To use the lambda module, create a live/lambda-sample folder to use as a root module:

In that folder, create main.tf with the contents shown in Example 56:

This code sets the following parameters:

Create a folder in lambda-sample/src, and inside that folder, create a file called index.js, which defines the handler, as shown in Example 57:

As you can see, this is a function that takes the event object as input and then uses the callback to return a response which is a 200 OK with the text "Hello, World!" Deploy the lambda-sample module the usual way:

apply should complete in just a few seconds; Lambda is fast! To see if it worked, open the Lambda console in your browser, click on the function called "sample-app-lambda," and you should see your handler code, as shown in Figure 26:

Currently, the function has no triggers, so it doesn’t really do anything. You can manually trigger it by clicking the blue Test button. The console will pop up a box where you can enter test data in JSON format to send to the function as the event object; leave everything at its default value and and click the Invoke button. That should run your function and show you log output that looks similar to Figure 27:

As you can see, your function has run, and responded with the expected 200 OK and "Hello, World!" Triggering Lambda functions manually is great for learning and testing, but in the real world, if you want to build a serverless web app, you need to trigger the function with HTTP requests, as described in the next section.

You can configure AWS to trigger a Lambda function each time you receive an HTTP(S) request by creating a Lambda function URL. Example 58 shows how to update the lambda-sample module to create a Lambda function URL:

Setting create_url to true tells the lambda module to create a Lambda function URL. You should also add the function URL as an output variable in a new file called outputs.tf, as shown in Example 59:

Deploy the updates:

When apply completes, you should see the function_url output:

Open this URL, and you should see "Hello, World!" Congrats, a Lambda function URL is now routing HTTP requests to your Lambda function! AWS will automatically scale your Lambda functions up and down in response to load, including scaling to zero when there is no load.

By default, AWS Lambda natively supports a nearly instantaneous deployment model. That is, if you upload a new deployment package, all new requests will start executing the code in that deployment package more or less immediately. For example, try updating lambda-sample/src/index.js to respond with "Fundamentals of DevOps!" rather than "Hello, World!", as shown in Example 60:

Re-run apply to deploy these changes:

apply should complete in a few seconds, and if you retry the function_url URL, you’ll see "Fundamentals of DevOps!" right away. So again, deployments with Lambda are fast!

When you’re done experimenting with the serverless code, run tofu destroy to undeploy all your infrastructure.