Part 3. How to Deploy Many Apps: Orchestration, VMs, Containers, and Serverless

The first thing you need for server orchestration is a bunch of servers. If you have existing servers you can use—e.g., several physical servers on-prem or several virtual servers in the cloud—and you have SSH access to those servers, you can skip this section, and go to the next one.

If you don’t have servers you can use, this section will show you how to deploy several EC2 instances using Ansible. As mentioned in Part 2, deploying and managing servers (hardware) is not really what configuration management tools were designed to do, but for learning and testing, Ansible is good enough. Note that the way you’ll use Ansible to deploy multiple EC2 instances in this section is meant to showcase server orchestration in its canonical form, with a fixed set of servers, and not the idiomatic approach for running multiple servers in the cloud; you’ll see the more idiomatic approach later in this blog post, in the VM orchestration section.

The blog post series’s sample code repo in GitHub contains an Ansible playbook called create_ec2_instances_playbook.yml (note the "s" in "instances," implying multiple instances, unlike the playbook from Part 2) in the ch3/ansible folder that can do the following:

To use this playbook, git clone the sample code repo, if you haven’t already (if you are new to Git, check out the Git tutorial in Part 4):

This will check out the sample code into the devops-book folder. Next, head into the fundamentals-of-devops folder you created in Part 1 to work through the examples in this blog post series, and create a new ch3/ansible subfolder:

Copy create_ec2_instances_playbook.yml from the devops-book folder into ch3/ansible:

To run this playbook, make sure Ansible is installed, authenticate to AWS as described in Authenticating to AWS on the command line, and run ansible-playbook as before. Ansible will start to interactively prompt you for input variables:

You can enter the values interactively and hit Enter, or, alternatively, you can define the variables in a YAML file, such as the sample-app-vars.yml file shown in Example 26:

You can then use the --extra-vars flag to pass this variables file to the ansible-playbook command:

This will create three empty servers that you can configure and manage as you wish. It’s a great playground to get a sense for server orchestration. As a first step, let’s improve the security and reliability of your app deployments, as discussed next.

As explained in Watch out for snakes: these examples have several problems, the code used to deploy apps in the previous blog posts had a number of concerns related to security and reliability issues: e.g., running the app as a root user, listening on port 80, no automatic app restart in case of crashes, and so on. It’s time to fix these issues and get this code a bit closer to something you could use in production.

First, just as in Section 2.3.2, you need to tell Ansible what servers you want to configure. You do this using either an inventory file or, if you deployed servers in the cloud, such as the EC2 instances in the previous section, you can use an inventory plugin, as shown in Example 27, to discover your servers automatically.

Just as in the previous blog post, this inventory file will create groups based on the Ansible tag. If you used the playbook in the previous section, that tag will be set to the value you entered for the base_name variable. In the preceding section, I used "sample_app_instances" as the base_name, so that’s what the group will be called. You’ll need to configure group variables for this group by creating a YAML file with the name of the group in the group_vars folder. So that will be group_vars/sample_app_instances.yml, as shown in Example 28:

This file configures the user, private key, and host key checking settings for the sample_app_instances group. Now you can use a playbook to configure the servers in this group to run the Node.js sample app. Create a new playbook called configure_sample_app_playbook.yml, with the contents shown in Example 29:

Here’s what this playbook does:

Create just a single file and folder for the nodejs-app role, roles/nodejs-app/tasks/main.yml:

Put the code shown in Example 30 into tasks/main.yml:

Here’s what this role does:

As you can see, the nodejs-app role is fairly generic: it’s designed so you can use it with any Node.js app, which makes this a highly reusable piece of code.

The sample-app role, on the other hand, is specifically designed to run the sample app. Create two subfolders for this role, files and tasks:

app.js is the exact same "Hello, World" Node.js sample app you saw in Part 1. Copy it into the files folder:

app.config.js is a new file that is used to configure PM2. So, what is PM2? PM2 is a process supervisor, which is a tool you can use to run your apps, monitor them, restart them after a reboot or a crash, manage their logging, and so on. Process supervisors provide one layer of auto healing for long-running apps. You’ll see other types of auto healing later in this post.

There are many process supervisors out there, including supervisord, runit, and systemd, with systemd as the one you’re likely to use in most situations, as it’s built into most Linux distributions these days. You’ll see an example of how to use systemd later in this blog post. For this example, I picked PM2 because it has features designed specifically for Node.js apps. To use these features, create a configuration file called app.config.js, as shown in Example 31:

This file configures PM2 to do the following:

Finally, create tasks/main.yml with the contents shown in Example 32:

These changes address most of the concerns in Watch out for snakes: these examples have several problems, improving your security posture (no more root user) and the reliability and performance of your app (process supervisor, cluster mode).

To try this code out, make sure you have Ansible installed, authenticate to AWS as described in Authenticating to AWS on the command line, and run the following command:

Ansible will discover your servers, configure each one with all the dependencies it needs, and run the app on each one. At the end, you should see the IP addresses of servers, as shown in the following log output (truncated for readability):

Copy the IP of one of the three servers, open http://<IP>:8080 in your web browser, and you should see the familiar "Hello, World!" text once again.

While three servers is great for redundancy, it’s not so great for usability, as your users typically want just a single endpoint to hit. This requires deploying a load balancer, as described in the next section.

A load balancer is a piece of software that can distribute load across multiple servers or apps. You give your users a single endpoint to hit, which is the load balancer, and under the hood, the load balancer forwards the requests it receives to a number of different endpoints, using various algorithms (e.g., round-robin, hash-based, least-response-time, etc.) to process requests as efficiently as possible. There are many popular load balancer options out there, such as Apache, Nginx, and HAProxy, as well as cloud-specific load balancing services, such as AWS Elastic Load Balancer, GCP Cloud Load Balancer, and Azure Load Balancer.

In the cloud, you’d most likely use a cloud load balancer, as you’ll see later in this blog post. However, for the purposes of server orchestration, I decided to show you a simplified example of how to run your own load balancer, as server orchestration techniques should work on-prem as well. Therefore, you’ll be deploying Nginx.

To do that, you need one more server. If you have one already with SSH access, you can use it, and skip forward a few paragraphs. If not, you can deploy one more EC2 instance using the same create_ec2_instances_playbook.yml, but with a new variables file, nginx-vars.yml, with the contents shown in Example 33:

This will create a single EC2 instance, with the base_name "nginx_instances," and it will allow requests on port 80, which is the default port for HTTP. Run the playbook with this vars file as follows:

This should create one more EC2 instance you can use for nginx. Since the base_name for that instance is nginx_instances, that will also be the group name in the inventory, so configure the variables for this group by creating group_vars/nginx_instances.yml with the contents shown in Example 34:

Now you can create a new playbook to configure these servers with Nginx. Create a new file called configure_nginx_playbook.yml with the contents shown in Example 35:

This playbook does the following:

Create a new folder for the nginx role with tasks and templates subfolders:

Inside of nginx/templates/nginx.conf.j2, create an Nginx configuration file template, as shown in Example 36:

Most of this file is standard (boilerplate) Nginx configuration—check out the Nginx documentation if you’re curious to understand what it does—so I’ll just point out a few items to focus on:

In short, the preceding configuration file will configure Nginx to load balance traffic across the servers you deployed to run the sample app.

Create nginx/tasks/main.yml with the contents shown in Example 37:

This file defines the tasks for the nginx role, which are the following:

Run this playbook to configure Nginx:

Wait a few minutes for everything to deploy and in the end, you should see log output that looks like this:

The value on the left, "xxx.us-east-2.compute.amazonaws.com," is a domain name you can use to access the Nginx server. If you open http://xxx.us-east-2.compute.amazonaws.com (this time with no port number, as Nginx is listening on port 80, the default port for HTTP) in your browser, you should see "Hello, World!" yet again. Each time you refresh the page, Nginx will send that request to a different EC2 instance. Congrats, you now have a single endpoint you can give your users, and that endpoint will automatically balance the load across multiple servers!

So you’ve now seen how to deploy using a server orchestration tool, but what about doing an update? Some configuration management tools support various deployment strategies (a topic you’ll learn more about in Part 5), such as a rolling deployment, where you update your servers in batches, so some servers are always running and serving traffic, while others are being updated. With Ansible, the easiest way to have it do a rolling update is to add the serial parameter to configure_sample_app_playbook.yml, as shown in Example 38:

Let’s give the rolling deployment a shot. Update the text that the app responds with in app.js, as shown in Example 39:

And re-run the playbook:

You should see Ansible rolling out the change to one server at a time. When it’s done, if you refresh the Nginx IP in your browser, you should see the text "Fundamentals of DevOps!"

When you’re done experimenting with Ansible, you should manually undeploy the EC2 instances by finding each one in the EC2 Console (look for the instance IDs the playbook writes to the log), clicking "Instance state," and choosing "Terminate instance" in the drop down, as shown in Figure 18. This ensures that your account doesn’t start accumulating any unwanted charges.

Head into the fundamentals-of-devops folder you created in Part 1 to work through the examples in this blog post series, and create a new subfolder for the Packer code:

Copy the Packer template you created in Part 2 into the new ch3/packer folder:

You should also copy app.js (the sample app) and app.config.js (the PM2 configuration file) from the server orchestration section of this blog post into the ch3/packer folder:

Example 40 shows the updates to make to the Packer template:

The main changes are to make security and reliability improvements similar to the ones you did in the server orchestration section: that is, use PM2 as a process supervisor and create app-user to run the app (instead of using the root user).

To build the AMI, make sure Packer is installed, authenticate to AWS as described in Authenticating to AWS on the command line, and run the following commands:

When the build is done, Packer will output the ID of the newly created AMI. Make sure to jot this ID down somewhere, as you’ll need it shortly.

The next step is to deploy the AMI. In Part 2, you used OpenTofu to deploy an AMI on a single EC2 instance. The goal now is to see VM orchestration at play, which means deploying multiple servers, or what’s sometimes called a cluster. Most cloud providers offer a native way to run VMs across a cluster: for example, AWS offers Auto Scaling Groups (ASG), GCP offers Managed Instance Groups, and Azure offers Scale Sets. For this example, you’ll be using an AWS ASG, as that offers a number of nice features:

Let’s use a reusable OpenTofu module called asg from this blog post series’s sample code repo to deploy an ASG. You can find the module in the ch3/tofu/modules/asg folder. This is a simple module that creates three main resources:

To use the asg module, create a live/asg-sample folder to act as a root module:

Inside the asg-sample folder, create a main.tf file with the initial contents shown in Example 41:

The preceding code sets the following parameters:

Create a file called user-data.sh with the contents shown in Example 42:

This user data script does the following:

If you were to run apply right now, you’d get an ASG with three EC2 instances running your sample app. While this is great for redundancy, as discussed in the server orchestration section, you typically want to give your users just a single endpoint to hit. This requires deploying a load balancer, as described in the next section.

In the server orchestration section, you deployed your own load balancer using Nginx. This is a simplified deployment that works fine for an example, but has a number of drawbacks if you try to use it for production apps:

To be clear, there’s nothing wrong with Nginx: if you put the work in, there are ways to address all of these issues with Nginx. However, it’s a considerable amount of work. One of the big benefits of the cloud is that most cloud providers offer managed services that can do this work for you. Load balancing is a very common problem, and as I mentioned before, almost every cloud provider offers a managed service for load balancing, such as AWS Elastic Load Balancer, GCP Cloud Load Balancer, and Azure Load Balancer. All of these provide a number of powerful features out-of-the-box. For example, the AWS Elastic Load Balancer (ELB) gives you the following:

Using a managed service for load balancing can be a huge time saver, so let’s use an AWS load balancer. There are actually several types of AWS load balancers to choose from; the one that’ll be the best fit for the simple sample app is the Application Load Balancer (ALB). The ALB consists of several parts, as shown in Figure 24:

The blog post series’s sample code repo includes a module called alb in the ch3/tofu/modules/alb folder that you can use to deploy an ALB. Note that this is a very simple ALB—it deploys into the Default VPC (see the note in A Note on Default Virtual Private Clouds) and only has a single listener rule where it forwards all requests to a single target group—but it should suffice for our purposes in this blog post.

Example 43 shows how to update the asg-sample module to use the alb module:

The preceding code sets the following parameters on the alb module:

The one missing piece is the connection between the ASG and the ALB: that is, how does the ALB know which EC2 instances to send traffic to (which instances to put in its target group)? To tie these pieces together, go back to your usage of the asg module, and update it with one parameter, as shown in Example 44:

The preceding code sets the target_group_arns parameter, which will change the ASG behavior in two ways:

The final change to the asg-sample module is to add the load balancer’s domain name as an output variable in outputs.tf, as shown in Example 45:

To deploy the module, make sure OpenTofu is installed, authenticate to AWS as described in Authenticating to AWS on the command line, and run the following commands:

After a few minutes, everything should be deployed, and you should see the ALB domain name as an output:

Open this domain name up in your web browser, and you should see "Hello, World!" once again. Congrats, you now have a single endpoint, the load balancer domain name, that you can give your users, and when users hit it, the load balancer will distribute their requests across all your apps in your ASG!

You’ve seen the initial deployment with VM orchestration, but what about rolling out updates? Most of the VM orchestration tools have support for zero-downtime deployments and various deployment strategies. For example, the ASGs in AWS have native support for a feature called instance refresh, which can update your instances automatically by doing a rolling deployment. Example 46 shows how to enable instance refresh in the asg module:

The preceding code sets the following parameters:

Run apply one more time to enable the instance refresh setting. Once that’s done, you can try rolling out a change. For example, update app.js in the packer folder to respond with "Fundamentals of DevOps!", as shown in Example 47:

Next, build a new AMI using Packer:

This will give you a new AMI ID. Update the ami_id in the asg-sample module to this new ID and run apply one more time. You should then see a plan output that looks something like this (truncated for readability):

This plan output shows that launch template has changed, due to the new AMI ID, and as a result, the version of the launch template used in the ASG has changed. This will result in an instance refresh. Type in yes, hit Enter, and AWS will kick off the instance refresh process in the background. If you go to the EC2 Console, click Auto Scaling Groups in the left nav, find your ASG, and click the "Instance refresh" tab, you should be able to see the instance refresh in progress, as shown in Figure 25.

During this process, the ASG will launch three new EC2 instances, and the ALB will start performing health checks. Once the new instances start to pass health checks, the ASG will undeploy the old one instances, leaving you with just the three new instances running the new code. The whole process should take around five minutes.

During this deployment, the load balancer URL should always return a successful response, as this is a zero-downtime deployment. You can even check this by opening a new terminal tab, and running the following Bash one-liner:

This code runs curl, an HTTP client, in a loop, hitting your ALB once per second and allowing you to see the zero-downtime deployment in action. For the first couple minutes, you should see only responses from the old instances: "Hello, World!" Then, as new instances start to pass health checks, the ALB will begin sending traffic to them, and you should see the response from the ALB alternate between "Hello, World!" and "Fundamentals of DevOps!" After another couple minutes, the "Hello, World!" message will disappear, and you’ll see only "Fundamentals of DevOps!", which means all the old instances have been shut down. The output will look something like this:

Congrats, you’ve now seen VM orchestration in action, including rolling out changes following immutable infrastructure practices!

When you’re done experimenting with the ASG, run tofu destroy to undeploy all your infrastructure. This ensures that your account doesn’t start accumulating any unwanted charges.

As you may remember from Section 2.4, Docker images are like self-contained "snapshots" of the operating system (OS), the software, the files, and all other relevant details. Let’s now see Docker in action.

First, if you don’t have Docker installed already, follow the instructions on the Docker website to install Docker Desktop for your operating system. Once it’s installed, you should have the docker command available on your command line. You can use the docker run command to run Docker images locally:

where IMAGE is the Docker image to run and COMMAND is an optional command to execute. For example, here’s how you can run a Bash shell in an Ubuntu 24.04 Docker image (note that the following command includes the -it flag so you get an interactive shell where you can type):

And voilà, you’re now in Ubuntu! If you’ve never used Docker before, this can seem fairly magical. Try running some commands. For example, you can look at the contents of /etc/os-release to verify you really are in Ubuntu:

How did this happen? Well, first, Docker searches your local filesystem for the ubuntu:24.04 image. If you don’t have that image downloaded already, Docker downloads it automatically from Docker Hub, which is a Docker Registry that contains shared Docker images. The ubuntu:24.04 image happens to be a public Docker image—an official one maintained by the Docker team—so you’re able to download it without any authentication. It’s also possible to create private Docker images that only certain authenticated users can use, as you’ll see later in this blog post.

Once the image is downloaded, Docker runs the image, executing the bash command, which starts an interactive Bash prompt, where you can type. Try running the ls command to see the list of files:

You might notice that’s not your filesystem. That’s because Docker images run in containers that are isolated at the userspace level: when you’re in a container, you can only see the filesystem, memory, networking, etc., in that container. Any data in other containers, or on the underlying host operating system, is not accessible to you, and any data in your container is not visible to those other containers or the underlying host operating system. This is one of the things that makes Docker useful for running applications: the image format is self-contained, so Docker images run the same way no matter where you run them, and no matter what else is running there.

To see this in action, write some text to a test.txt file as follows:

Next, exit the container by hitting Ctrl-D, and you should be back in your original command prompt on your underlying host OS. If you try to look for the test.txt file you just wrote, you’ll see that it doesn’t exist: the container’s filesystem is totally isolated from your host OS.

Now, try running the same Docker image again:

Notice that this time, since the ubuntu:24.04 image is already downloaded, the container starts almost instantly. This is another reason Docker is useful for running applications: unlike virtual machines, containers are lightweight, boot up quickly, and incur little CPU or memory overhead.

You may also notice that the second time you fired up the container, the command prompt looked different. That’s because you’re now in a totally new container; any data you wrote in the previous one is no longer accessible to you. Run ls -al and you’ll see that the test.txt file does not exist. Containers are isolated not only from the host OS but also from each other.

Hit Ctrl-D again to exit the container, and back on your host OS, run the docker ps -a command:

This will show you all the containers on your system, including the stopped ones (the ones you exited). You can start a stopped container again by using the docker start <ID> command, setting ID to an ID from the CONTAINER ID column of the docker ps output. For example, here is how you can start the first container up again (and attach an interactive prompt to it via the -ia flags):

You can confirm this is really the first container by outputting the contents of test.txt:

Hit Ctrl-D once more to exit the container and get back to your host OS.

Now that you’ve seen the basics of Docker, let’s look at what it takes to create your own Docker images, and use them to run web apps.

Let’s see how a container can be used to run a web app: in particular, the Node.js sample app you’ve been using throughout this blog post series. Create a new folder called docker:

Copy app.js from the server orchestration section into the docker folder (note: you do not need to copy app.config.js this time):

Next, create a file called Dockerfile, with the contents shown in Example 48:

Just as you used a Packer template to define how to build a VM image for your sample app, this Dockerfile is a template that defines how to build a Docker image for your sample app. This Dockerfile does the following:

To build a Docker image from this Dockerfile, use the docker build command:

The -t flag is the tag (name) to use for the Docker image: the preceding code sets the image name to "sample-app" and the version to "v1." Later on, if you make changes to the sample app, you’ll be able to build a new Docker image and give it a new version, such as "v2." The dot (.) at the end tells docker build to run the build in the current directory (which should be the folder that contains your Dockerfile). When the build finishes, you can use the docker run command to run your new image:

Note the use of --init: Node.js doesn’t handle kernel signals (such as Ctrl+C) properly, so this is necessary to ensure that the app will exit correctly if you hit Ctrl+C. See Docker and Node.js best practices for more information, including other practices that you should use with Node.js Docker images.

Your app is now listening on port 8080! However, if you open a new terminal on your host operating system and try to access the sample app, it won’t work:

What’s the problem? Actually, it’s not a problem but a feature! Docker containers are isolated from the host operating system and other containers, not only at the filesystem level but also in terms of networking. So while the container really is listening on port 8080, that is only on a port inside the container, which isn’t accessible on the host OS. If you want to expose a port from the container on the host OS, you have to do it via the -p flag.

First, hit Ctrl-C to shut down the sample-app container: note that it’s Ctrl-C this time, not Ctrl-D, as you’re shutting down a process, rather than exiting an interactive prompt. Now rerun the container but this time with the -p flag as follows:

Adding -p 8080:8080 to the command tells Docker to expose port 8080 inside the container on port 8080 of the host OS. You know to use port 8080 here, as you built this Docker image yourself, but if this was someone else’s image, you could use docker inspect on the image, and that will tell you about any ports that image labeled with EXPOSE. In another terminal on your host OS, you should now be able to see the sample app working:

Congrats, you now know how to run a web app locally using Docker! However, while using docker run directly is fine for local testing and learning, it’s not the way you’d run Dockerized apps in production. For that, you typically want to use a container orchestration tool such as Kubernetes, which is the topic of the next section.

Kubernetes is a container orchestration tool, which means it’s a platform for running and managing containers on your servers, including scheduling, auto healing, auto scaling, load balancing, and much more. Under the hood, Kubernetes consists of two main pieces, as shown in Figure 26:

Kubernetes is open source, and one of its strengths is that you can run it anywhere: in any public cloud (e.g., AWS, Azure, GCP), in your own datacenter, and even on your personal computer. A little later in this blog post, I’ll show you how you can run Kubernetes in the cloud (in AWS), but for now, let’s start small and run it locally. This is easy to do if you installed a relatively recent version of Docker Desktop, as it has the ability to fire up a Kubernetes cluster locally with just a few clicks.

If you open Docker Desktop’s preferences on your computer, you should see Kubernetes in the nav, as shown in Figure 27.

If it’s not enabled already, check the Enable Kubernetes checkbox, click Apply & Restart, and wait a few minutes for that to complete. In the meantime, follow the instructions on the Kubernetes website to install kubectl, which is the command-line tool for interacting with Kubernetes.

To use kubectl, you must first update its configuration file, which lives in $HOME/.kube/config (that is, the .kube folder of your home directory), to tell it what Kubernetes cluster to connect to. Conveniently, when you enable Kubernetes in Docker Desktop, it updates this config file for you, adding a docker-desktop entry to it, so all you need to do is tell kubectl to use this configuration as follows:

Now you can check if your Kubernetes cluster is working with the get nodes command:

The get nodes command shows you information about all the nodes in your cluster. Since you’re running Kubernetes locally, your computer is the only node, and it’s running both the control plane and acting as a worker node. You’re now ready to run some Docker containers!

To deploy something in Kubernetes, you create Kubernetes objects, which are persistent entities you write to the Kubernetes cluster (via the API server) that record your intent: e.g., your intent to have specific Docker images running. The cluster runs a reconciliation loop, which continuously checks the objects you stored in it and works to make the state of the cluster match your intent.

There are many different types of Kubernetes objects available. The one we’ll use to deploy your sample app is a Kubernetes Deployment, which is a declarative way to manage an application in Kubernetes. The Deployment allows you to declare what Docker images to run, how many copies of them to run (replicas), a variety of settings for those images (e.g., CPU, memory, port numbers, environment variables), and so on, and the Deployment will then work to ensure that the requirements you declared are always met.

One way to interact with Kubernetes is to create YAML files to define your Kubernetes objects, and to use the kubectl apply command to submit those objects to the cluster. Create a new folder called kubernetes to store these YAML files:

Within the kubernetes folder, create a file called sample-app-deployment.yml with the contents shown in Example 49:

This YAML file gives you a lot of functionality for just ~20 lines of code:

You can use the kubectl apply command to apply your Deployment configuration:

This command should complete very quickly. How do you know if it actually worked? To answer that question, you can use kubectl to explore your cluster. First, run the get deployments command, and you should see your Deployment:

Here, you can see how Kubernetes uses metadata, as the name of the Deployment (sample-app-deployment) comes from your metadata block. You can use that metadata in API calls yourself. For example, to get more details about a specific Deployment, you can run describe deployment <NAME>, where <NAME> is the name from the metadata:

This Deployment is reporting that all 3 replicas are available. To see those replicas, run the get pods command:

And to get the details about a specific pod, copy its name, and run describe pod:

From this output, you can see the containers that are running for each pod, which in this case, is just one container per pod running the sample-app:v1 Docker image you built earlier.

You can also see the logs for a single pod by using the logs command, which is useful for understanding what’s going on and debugging:

Ah, there’s that familiar log output. You now have three replicas of your sample app running. But, just as you saw with server and VM orchestration, users will want just one endpoint to hit, so now it’s time to figure out how to do load balancing with Kubernetes.

Kubernetes has built-in support for load balancing. The typical way to set it up is to make use of another Kubernetes object, called a Kubernetes Service, which is a way to expose an app running in Kubernetes as a service you can talk to over the network. Example 50 shows the YAML code for a Kubernetes service, which you should put in a file called sample-app-service.yml:

Here’s what this code does:

You apply the Service the same way, using kubectl apply:

To see if your service worked, use the get services command:

The first service in the list is Kubernetes itself, which you can ignore. The second is the Service you created, with the name sample-app-loadbalancer (based on its own metadata block). You can get more details about your service by using the describe service command:

You can see that the load balancer is listening on localhost, at port 80, so you can test it out by opening http://localhost in your browser, or using curl:

Congrats, you’re now able to deploy Docker containers with Kubernetes and distribute traffic across your containers with a load balancer! But what if you want to update your app?

Kubernetes Deployments have built-in support for rolling updates. Open up sample-app-deployment.yml and add the code shown in Example 51 to the bottom of the spec section:

This configures the Deployment to do a rolling update where it can deploy up to 3 extra pods during the deployment, similar to the instance refresh you saw with ASGs. Run apply to update the Deployment with these changes:

Now, make a change to the sample app in docker/app.js, such as returning the text "Fundamentals of DevOps!" instead of "Hello, World!", as shown in Example 52:

To deploy this change, the first step is to build a new Docker image, with v2 as the new version:

The build will likely run in less than a second! This is because Docker has a built-in build cache, which, if used correctly, can dramatically speed up builds.

Next, open sample-app-deployment.yml one more time, and in the spec section, update the image from sample-app:v1 to sample-app:v2, as shown in Example 53:

Run apply one more time to deploy this change:

In the background, Kubernetes will kick off the rolling update. If you run get pods during this process, you’ll see up to six pods running at the same time (three old, three new):

After a little while, the three old pods will be undeployed, and you’ll be left with just the new ones. At that point, the load balancer will be responding with "Fundamentals of DevOps!"

So far, you’ve been running Kubernetes locally, which is great for learning and testing. However, for production deployments, you’ll need to run a Kubernetes cluster on servers in a data center. Kubernetes is a complicated system: it’s more or less a cloud in and of itself, and setting it up and maintaining it is a significant undertaking. Fortunately, if you’re using the cloud, most cloud providers have managed Kubernetes offerings that make this considerably simpler. The one you’ll learn to use in this blog post series is Amazon’s Elastic Kubernetes Service (EKS), which can deploy and manage the control plane and worker nodes for you.

The blog post series’s sample code repo contains a module called eks-cluster in the ch3/tofu/modules/eks-cluster folder that you can use to deploy a simple EKS cluster, which includes the following:

To use the eks-cluster module, create a new folder called live/eks-sample to use as a root module:

Inside of the eks-sample folder, create a file called main.tf, with the contents shown in Example 54:

The preceding code configures the following parameters:

To deploy the EKS cluster, authenticate to AWS as described in Authenticating to AWS on the command line, and run the following commands:

After 3-5 minutes, the cluster should finish deploying. To explore the cluster with kubectl, you first need to authenticate to your cluster. The aws CLI has a built-in command for doing this:

Where <REGION> is the AWS region you deployed the EKS cluster into and <CLUSTER_NAME> is the name of the EKS cluster. The preceding code used us-east-2 and eks-tofu for these, respectively, so you can run the following:

Once this is done, try running get nodes:

This output looks a bit different from when you ran the command with the Kubernetes cluster from Docker Desktop. You should see three nodes, each of which is an EC2 instance in your managed node group.

The next step is to try deploying the sample app into the EKS cluster. However, there’s one problem: you’ve created a Docker image for the sample app, but that image only lives on your own computer. The EKS cluster in AWS won’t be able to fetch the image from your computer, so you need to push the image to a container registry that EKS can read from, as described in the next section.

There are a number of container registries out there, including Docker Hub, Amazon’s Elastic Container Registry (ECR), the Azure Container Registry, the Google Artifact Registry, the JFrog Docker Registry, and the GitHub Container Registry. If you’re using AWS, the easiest one to use is ECR, so let’s set that up.

For each Docker image you want to store in ECR, you have to create an ECR repository (ECR repo for short). The blog post series’s sample code repo includes a module called ecr-repo in the ch3/tofu/modules/ecr-repo folder that you can use to create an ECR repo.

To use the ecr-repo module, create a new folder called live/ecr-sample to use as a root module:

In the ecr-sample folder, create a file called main.tf with the contents shown in Example 55:

This code will create an ECR repo called "sample-app." Typically, the repo name should match your Docker image name.

You should also create outputs.tf with an output variable, as shown in Example 56:

The preceding code will output the URL of the ECR repo, which you’ll need to be able to push and pull images. To create the ECR repo, run the following commands:

After a few seconds, you should see the registry_url output:

Copy down that registry_url value, as you’ll need it shortly.

Before you can push your Docker image to this ECR repo, you have to build the image with the right CPU architecture. Each Docker image you build is built for a specific CPU architecture: the docker build command, by default, builds for whatever CPU architecture you have on your own computer. For example, if you’re on a recent Macbook with an ARM CPU (e.g., the M1 or M2), your Docker images will be built for the arm64 architecture. This is a problem if you try to run those Docker images in the EKS cluster deployed by the eks-cluster module, as the t2.micro worker nodes in that cluster use the amd64 architecture, and won’t be able to run arm64 images.

Therefore, you need to ensure that you build your Docker images for whatever architecture(s) you plan to deploy onto. Fortunately, Docker now ships with the buildx command, which makes it easy to build Docker images for multiple architectures. The very first time you use buildx, you need to create a multi-platform-builder for your target architectures. For example, if you’re on an ARM64 Mac, and you’re going to be deploying onto AMD64 Linux servers, use the following command:

Now you can run the following command to build a Docker image of the sample app for both architectures (note the use of a new tag, v3, for these images):

Once the build is done, to be able to push the Docker image to ECR, you need to tag it using the registry URL of the ECR repo that you got from the registry_url output:

Next, you need to authenticate to your ECR repo, which you can do using a combination of the aws CLI and the docker CLI, making sure to replace the last argument with the registry URL of your own ECR repo that you got from the registry_url output:

Finally, you can push the Docker image to your ECR repo:

The first time you push, it may take a minute or two to upload the image. Subsequent pushes, due to Docker’s layer caching, will be faster.

At this point, you are ready to deploy the sample app Docker image into the EKS cluster. The only change you need to make to the YAML you used to deploy locally is to switch the image in kubernetes/sample-app-deployment.yml to the v3 ECR repo URL, as shown in Example 57:

You can now apply both YAML files to deploy into your EKS cluster:

Deployments to an EKS cluster will take slightly longer than a local Kubernetes cluster, but after a minute or two, if you run the get pods command, you should see something like this:

And if you run get services, you should see something like this:

If you look at the EXTERNAL-IP for sample-app-loadbalancer, you should see the domain name of an AWS ELB. Open this URL up in a web browser or using curl:

If you get "Could not resolve host" errors, this is probably because the load balancer is still booting up or the health checks haven’t passed yet. Give it a minute or two more, and try again, and you should see the familiar "Fundamentals of DevOps!" text.

Congrats, you’re now running a Dockerized application in a Kubernetes cluster in AWS!

When you’re done experimenting with the EKS cluster, run tofu destroy on both the eks-cluster and ecr-repo modules to undeploy all your infrastructure. This ensures that your account doesn’t start accumulating any unwanted charges.

The blog post series’s sample code repo includes a module called lambda in the ch3/tofu/modules/lambda folder that can do the following:

To use the lambda module, create a live/lambda-sample folder to use as a root module:

In the lambda-sample folder, create a file called main.tf with the contents shown in Example 58:

This code sets the following parameters:

Create a folder in lambda-sample/src, and inside that folder, create a file called index.js, which defines the handler, as shown in Example 59:

As you can see, this is a function that takes the event object as input and then uses the callback to return a response which is a 200 OK with the text "Hello, World!"

Deploy the lambda-sample module the usual way:

apply should complete in just a few seconds: Lambda is fast! To see if it worked, open the Lambda console in your browser, click on the function called "sample-app-lambda," and you should see your function and the handler code, as shown in Figure 28:

Currently, the function has no triggers, so it doesn’t really do anything. You can manually trigger it by clicking the blue Test button. The console will pop up a box where you can enter test data in JSON format to send to the function as the event object; leave everything at its default value and and click the Invoke button. That should run your function and show you log output that looks similar to Figure 29:

As you can see, your function has run, and responded with the expected 200 OK and "Hello, World!"

Triggering Lambda functions manually is great for learning and testing, but in the real world, if you want to build a serverless web app, you need to be able to have HTTP requests trigger your function, as described in the next section.

You can configure a variety of events to trigger your Lambda function: e.g., you can have AWS automatically run your Lambda function each time a file is uploaded to Amazon’s Simple Storage Service (S3), or a new message is written to a queue in Amazon’s Simple Queue Service (SQS), or each time you get a new email in Amazon’s Simple Email Service (SES). So Lambda is a great choice for building event-driven systems and background processing jobs.

You can also configure AWS to trigger a Lambda function each time you receive an HTTP request in API Gateway, which is a managed service you can use to expose an entrypoint for your apps, managing routing, authentication, throttling, and so on. You can also use API Gateway to create serverless web apps.

The blog post series’s sample code repo includes a module called api-gateway in the ch3/tofu/modules/api-gateway folder that can deploy an HTTP API Gateway, a version of API Gateway designed for simple HTTP APIs, that knows how to trigger a Lambda function. Example 60 shows how to update the lambda-sample module to use the api-gateway module:

This code sets the following parameters:

You should also add an output variable in outputs.tf, as shown in Example 61:

This code will give you the API Gateway’s domain name as an output.

Deploy the updates:

When apply completes, you should see the api_endpoint output:

Open this output in a web browser, and you should see "Hello, World!" Congrats, API Gateway is now routing requests to your Lambda function! As load goes up and down, AWS will automatically scale your Lambda functions up and down, and API Gateway will automatically distribute traffic across these functions. And when there’s no load, your Lambda function will automatically scale to zero, so it won’t cost you a cent.

By default, AWS Lambda natively supports a nearly instantaneous deployment model: that is, if you upload a new deployment package, all new requests will start executing the code in that deployment package more or less immediately.

For example, try updating lambda-sample/src/index.js to respond with "Fundamentals of DevOps!" rather than "Hello, World!", as shown in Example 62:

Re-run apply to deploy these changes:

apply should complete in a few seconds, and if you retry the api_endpoint URL, you’ll see "Fundamentals of DevOps!" right away. So again, deployments with Lambda are fast! In fact, AWS Lambda does an instantaneous switchover from the old to the new version, so it’s effectively a blue-green deployment (which you’ll learn more about in Part 5).

When you’re done experimenting with the serverless code, run tofu destroy to undeploy all your infrastructure. This ensures that your account doesn’t start accumulating any unwanted charges.