Part 2. How to Manage Your Infrastructure as Code

As an example, let’s create a Bash script that automates all the manual steps you did in Part 1 to deploy a simple Node.js app in AWS. Head into the fundamentals-of-devops folder you created in Part 1 to work through the examples in this blog post series, and create a new subfolder for this part and the Bash script:

Copy the exact same user data script from Part 1 into a file called user-data.sh within the ch2/bash folder:

Next, create a Bash script called deploy-ec2-instance.sh, with the contents shown in Example 3:

If you’re not an expert in Bash syntax, all you have to know about this script is that it uses the AWS Command Line Interface (CLI) to automate the exact steps you did manually in the AWS console in Part 1:

If you want to run the script, you first need to give it execute permissions:

Next, install the AWS CLI (minimum version 2.0), authenticate to AWS, and run the script as follows:

After the script finishes, give the EC2 instance a minute or two to boot up, and then try opening up http://<Public IP> in your web browser, where <Public IP> is the IP address the script outputs at the end. You should see:

Congrats, you are now running your app using an ad hoc script!

When you’re done experimenting with this script, you should manually undeploy the EC2 instance as shown in Figure 9. This ensures that your account doesn’t start accumulating any unwanted charges.

You’ve now seen one way to manage your infrastructure as code. Well, sort of. This script, and most ad hoc scripts, have quite a few drawbacks in terms of using them to manage infrastructure, as discussed in the next section.

Below is a list of criteria, which I’ll refer to as the IaC category criteria, that you can use to compare different categories of IaC tools. In this section, I’ll flush out how ad hoc scripts stack up according to the IaC category criteria; in later sections, you’ll see how the other IaC categories perform along the same criteria, giving you a consistent way to compare the different options.

Ad hoc scripts have always been, and will always be, a big part of software delivery. They are the glue and duct tape of the DevOps world. However, they are not the best choice as a primary tool for managing infrastructure as code.

If you’re going to be managing all of your infrastructure as code, you should use an IaC tool that is purpose-built for the job, such as one of the ones discussed in the next several sections.

To be able to use configuration management, the first thing you need is a server. This section will show you how to deploy an EC2 instance using Ansible. Note that deploying and managing servers (hardware) is not really what configuration management tools were designed to do—later in this blog post, you’ll see how provisioning tools are typically a better fit for this task—but for spinning up a single server for learning and testing, Ansible is good enough.

Create a new folder called ansible:

Inside the Ansible folder, create an Ansible playbook called create_ec2_instances_playbook.yml, with the contents shown in Example 4:

An Ansible playbook specifies the hosts to run on, some variables, and then a list of tasks to execute on those hosts. Each task runs a module, which is a unit of code that can execute various commands. The preceding playbook does the following:

To run this Ansible playbook, install Ansible (minimum version 2.17), authenticate to AWS, and run the following:

When the playbook finishes running, you should have a server running in AWS. Now you can see what configuration management tools are really designed to do: configure servers to run software.

In order for Ansible to be able to configure your servers, you have to provide an inventory, which is a file that specifies which servers you want configured, and how to connect to them. If you have a set of physical servers on-prem, you can put the IP addresses of those servers in an inventory file, as shown in Example 5:

The preceding file organizes your servers into groups: the webservers group has two servers in it and the dbservers group has three servers. You’ll then be able to write Ansible playbooks that target the hosts in specific groups.

If you are running servers in the cloud, where servers come and go often, and IP addresses change frequently, you’re better off using an inventory plugin that can dynamically discover your servers. For example, you can use the aws_ec2 inventory plugin to discover the EC2 instance you deployed in the previous section. Create a file called inventory.aws_ec2.yml with the contents shown in Example 6:

This code does the following:

For each group in your inventory, you can specify group variables to configure how to connect to the servers in that group. You define these variables in YAML files in the group_vars folder, with the name of the file set to the name of the group. For example, for the EC2 instance in the sample_app_ansible group, you should create a file in group_vars/sample_app_ansible.yml with the contents shown in Example 7:

The group variables this file defines are:

Alright, with the inventory stuff out of the way, you can now create a playbook to configure your server to run the Node.js sample app. Create a file called configure_sample_app_playbook.yml with the contents shown in Example 8:

This playbook does two things:

An Ansible role defines a logical profile of an application in a way that promotes modularity and code reuse. Ansible roles also provide a standardized way to organize tasks, templates, files, and other configuration as per the following folder structure:

Each folder has a specific purpose: e.g., the tasks folder defines tasks to run on a server; the files folder has files to copy to the server; the templates folder lets you use Jinja templating to dynamically fill in data in files; and so on. Having this standardized structure makes it easier to navigate and understand an Ansible codebase.

To create the sample-app role for this playbook, create a roles/sample-app folder in the same directory as configure_sample_app_playbook.yml:

Within roles/sample-app, you should create files and tasks subfolders, which are the only parts of the standardized role folder structure you’ll need for this simple example. Copy the Node.js sample app from Part 1 into files/app.js:

Next, create tasks/main.yml with the code shown in Example 9:

This code does the following:

To run this playbook, authenticate to AWS, and run the following command:

You should see log output at the end that looks something like this:

The value on the left, "xxx.us-east-2.compute.amazonaws.com," is a domain name you can use to access the instance. Open http://xxx.us-east-2.compute.amazonaws.com:8080 (note it’s port 8080 this time, not 80) in your web browser, and you should see:

Congrats, you’re now using a configuration management tool to manage your infrastructure as code!

When you’re done experimenting with Ansible, manually undeploy the EC2 instance as shown in Figure 9.

Here is how configuration management tools stack up using the IaC category criteria:

Configuration management tools brought a number of advantages over ad hoc scripts, but they also introduced their own drawbacks. One big drawback is that some configuration management tools have a considerable setup cost: e.g., you may need to set up master servers and agents. A second big drawback is that most configuration management tools were designed for a mutable infrastructure paradigm, where you have long-running servers that the configuration management tools update (mutate) over and over again, over many years. This can be problematic due to configuration drift, where over time, your long-running servers can build up unique histories of changes, so each server is subtly different from the others, which can make it hard to reason about what’s deployed, and even harder to reproduce the issue on another server, all of which makes debugging challenging.

As cloud and virtualization becomes more and more ubiquitous, it’s becoming more common to use an immutable infrastructure paradigm, where instead of long-running physical servers, you use short-lived virtual servers that you replace every time you do an update. This is inspired by functional programming, where variables are immutable, so after you’ve set a variable to a value, you can never change that variable again, and if you need to update something, you create a new variable. Because variables never change, it’s easier to reason about your code.

The idea behind immutable infrastructure is similar. Once you’ve deployed a server, you never make changes to it again. If you need to update something, such as deploying a new version of your code, you deploy a new server. Because servers never change after being deployed, it’s easier to reason about what’s deployed. The typical analogy used here (my apologies to vegetarians and animal lovers), is cattle vs pets: with mutable infrastructure, you treat your servers like pets, giving each one its own unique name, taking care of it, and trying to keep it alive as long as possible; with immutable infrastructure, you treat your servers like cattle, each one more or less indistinguishable from the others, with random or sequential IDs instead of names, and you kill them off and replace them regularly.

While it’s possible to use configuration management tools with immutable infrastructure patterns, it’s not what they were originally designed for, and that led to new approaches, as discussed in the next section.

As an example, let’s take a look at using Packer to create a VM image for AWS called an Amazon Machine Image (AMI). First, create a folder called packer:

Next, copy the Node.js sample app from Part 1 into the packer folder:

Create a Packer template called sample-app.pkr.hcl, with the contents shown in Example 10:

You create Packer templates using the HashiCorp Configuration Language (HCL) in files with a .hcl extension. The preceding template does the following:

Create a file called install-node.sh with the contents shown in Example 11:

This script is identical to the first part of the Bash script, using yum to install Node.js. More generally, the Packer template is nearly identical to the Bash script and Ansible playbook, except the result of executing Packer is not a server running your app, but the image of a server with your app and all its dependencies installed. The idea is to use other IaC tools to launch one or more servers running that image; you’ll see an example of this shortly.

To build the AMI, install Packer (minimum version 1.10), authenticate to AWS, and run the following commands:

The first command, packer init, installs any plugins used in this Packer template. Packer can create images for many cloud providers—e.g., AWS, GCP, Azure, etc.—and the code for each of these providers lives in separate plugins that you install via the init command. The second command, packer build, kicks off the build process. When the build is done, which typically takes 3-5 minutes, you should see some log output that looks like this:

Congrats, you’re now using a server templating tool to manage your server configuration as code! The ami-xxx value is the ID of the AMI that was created from this template. You’ll see how to deploy this AMI shortly.

How do server templating tools stack up using the IaC category criteria?

As I mentioned a few times, server templating tools are powerful, but they don’t work by themselves. You need another tool to actually deploy and manage the images you create, such as provisioning tools, which are the focus of the next section.

As an example of using a provisioning tool, lets create an OpenTofu module that can deploy an EC2 instance. You write OpenTofu modules in HCL (the same language you used with Packer), in configuration files with a .tf extension. OpenTofu will find all files with the .tf extension in a folder, so you can name the files whatever you want, but there are some standard conventions, such as putting the main resources in main.tf, input variables in variables.tf, and output variables in outputs.tf.

First, create a new tofu/ec2-instance folder for the module:

Within the tofu/ec2-instance folder, create a file called main.tf, with the contents shown in Example 12:

The code in main.tf does something similar to the Bash script and Ansible playbook from earlier in the blog post:

Create a file called user-data.sh, with the contents shown in Example 13:

Note how this user data script is a fraction of the size of the one you saw in the Bash code. That’s because all the dependencies (Node.js) and code (app.js) are already installed in the AMI by Packer. So the only thing this user data script does is start the sample app. This is a more idiomatic way to use user data.

Next, create a file called variables.tf to define input variables, which are like the input parameters of a function, as shown in Example 14:

This code defines an input variable called name, which allows you to specify the name to use for the EC2 instance, security group, and other resources. You’ll see how to set variables shortly. Finally, create a file called outputs.tf with the contents shown in Example 15:

The preceding code defines output variables, which are like the return values of a function, which will be printed to the log, and can be shared with other modules. The preceding code defines output variables for the EC2 instance ID, security group ID, and EC2 instance public IP.

Try this code out by installing OpenTofu (minimum version 1.9), authenticating to AWS, and running the following command:

Similar to Packer, OpenTofu works with many providers, and the code for each one lives in separate binaries that you install via the init command. Once init has completed, run the apply command to start the deployment process:

The first thing the apply command will do is prompt you for the name input variable:

You can type in a name like "sample-app-tofu" and hit Enter. Alternatively, if you don’t want to be prompted interactively, you can instead use the -var flag:

You can also set any input variable foo using the environment variable TF_VAR_foo:

One more option is to define a default value, as shown in Example 16:

Once all input variables have been set, the next thing the apply command will do is show you the execution plan (just plan for short), which will look something like this (truncated for readability):

The plan lets you see what OpenTofu will do before actually making any changes, and prompts you for confirmation before continuing. This is a great way to sanity-check your code before unleashing it onto the world. The plan output is similar to the output of the diff command that is part of Unix, Linux, and git: anything with a plus sign (+) will be created, anything with a minus sign (–) will be deleted, anything with both a plus and a minus sign will be replaced, and anything with a tilde sign (~) will be modified in place. Every time you run apply, OpenTofu will show you this execution plan; you can also generate the execution plan without applying any changes by running tofu plan instead of tofu apply.

In the preceding plan output, you can see that OpenTofu is planning on creating an EC2 Instance, security group, and security group rule, which is exactly what you want. Type yes and hit Enter to let OpenTofu proceed. When apply completes, you should see log output that looks like this:

It’s the three output variables, including the public IP address in public_ip. Wait a minute or two for the EC2 instance to boot up, open http://<public_ip>:8080, and you should see:

Congrats, you’re using a provisioning tool to manage your infrastructure as code!

One of the big advantages of provisioning tools is that they support not just deploying infrastructure, but also updating and destroying it. For example, now that you’ve deployed an EC2 instance using OpenTofu, make a change to the configuration, such as adding a new Test tag with the value "update," as shown in Example 17:

Run the apply command again, and you should see output that looks like this:

Every time you run OpenTofu, it records information about what infrastructure it created in an OpenTofu state file. OpenTofu manages state using backends; if you don’t specify a backend, the default is to use the local backend, which stores state locally in a terraform.tfstate file in the same folder as the OpenTofu module (you’ll see how to use other backends in Part 5). This file contains a custom JSON format that records a mapping from the OpenTofu resources in your configuration files to the representation of those resources in the real world.

When you run apply the first time on the ec2-instance module, OpenTofu records in the state file the IDs of the EC2 instance, security group, security group rules, and any other resources it created. When you run apply again, you can see "Refreshing state" in the log output, which is OpenTofu updating itself on the latest status of the world. As a result, the plan output that you see is the diff between what’s currently deployed in the real world and what’s in your OpenTofu code. The preceding diff shows that OpenTofu wants to create a single tag called Test, which is exactly what you want, so type yes and hit Enter, and you’ll see OpenTofu perform an update operation, updating the EC2 instance with your new tag.

When you’re done testing, you can run tofu destroy to have OpenTofu undeploy everything it deployed earlier:

When you run destroy, OpenTofu shows you a destroy plan, which tells you about all the resources it’s about to delete. This gives you one last chance to check that you really want to delete this stuff before you actually do it. It goes without saying that you should rarely, if ever, run destroy in a production environment—there’s no "undo" for the destroy command. If everything looks good, type yes and hit Enter, and in a minute or two, OpenTofu will clean up everything it deployed.

One of OpenTofu’s more powerful features is that the modules are reusable. In a general purpose programming language (e.g., JavaScript, Python, Java), you put reusable code in a function; in OpenTofu, you put reusable code in a module. You can then use that module multiple times to spin up many copies of the same infrastructure, without having to copy/paste the code.

So far, you’ve been using the ec2-instance module as a root module, which is any module on which you run apply directly. However, you can also use it as a reusable module, which is a module meant to be included in other modules (e.g., in other root modules) as a means of code re-use.

Let’s give it a shot. First, create a folder called modules to store your reusable modules:

Next, move the ec2-instance module into the modules folder:

Create a folder called live to store your root modules:

Inside the live folder, create a new folder called sample-app, which will house the new root module you’ll use to deploy the sample app:

In the live/sample-app folder, create main.tf with the contents shown in Example 18:

To use one module from another, all you need is the following:

If you were to run apply on this code, it would use the ec2-instance module code to create a single EC2 instance. But the beauty of code reuse is that you can use the module multiple times, as shown in Example 19:

This code has two module blocks, so if you run apply on it, it will create two EC2 instances, one with all the resources named "sample-app-tofu-1" and one with all the resources named "sample-app-tofu-2." If you had three module blocks, it would create three EC2 instances; and so on. And, of course, you can mix and match different modules, include modules in other modules, and so on. It’s not unusual for modules to be reused dozens or hundreds of times across a company, so that you put in the work to create a module that meets your company’s needs once, and then use it over and over again.

Before you run apply, there are two small changes to make. The first change is to move the provider block from the ec2-instance (reusable) module to the sample-app (root) module, as shown in Example 20:

You typically don’t define provider blocks in reusable modules, and instead inherit those provider configurations from the root module, which allows users to configure them however they prefer (e.g., use different regions, accounts, and so on). The second change is to create an outputs.tf file in the sample-app folder with the contents shown in Example 21:

The preceding code "proxies" the output variables from the underlying ec2-instance module so that you can see those outputs when you run apply on the sample-app root module. OK, you’re finally ready to run this code:

When apply completes, you should have two EC2 instances running, and the output variables should show their IPs and instance IDs. If you wait a minute or two for the instances to boot up, and open http://<IP>:8080 in your browser, where <IP> is the public IP of either instance, you should see the familiar "Hello, World!" text. When you’re done experimenting, run tofu destroy to clean everything up again.

There’s one more trick with OpenTofu modules: the source parameter can be set to not only a local file path, but also to a URL. For example, the blog post series’s sample code repo in GitHub includes an ec2-instance module that is more or less identical to your own ec2-instance module. You can use the module directly from the series’s GitHub repo by setting the source parameters to a GitHub URL, as shown in Example 22:

The preceding code sets the source URL to a GitHub URL. Take note of two things:

OpenTofu supports not only GitHub URLs for module sources, but also GitLab URLs, BitBucket URLs, and so on. One particularly convenient option is to publish your modules to a module registry, which is a centralized way to share, find, and use modules. OpenTofu and Terraform each provide a public registry you can use for open source modules; you can also run private registries within your company. I’ve published all the reusable modules in this blog post series to the OpenTofu and Terraform public registries. Note that the registries have specific requirements on how the repo must be named and its folder structure, so to publish these reusable modules, I had to copy them to another repo called https://github.com/brikis98/terraform-book-devops, into the folder structure modules/<MODULE_NAME>, which allows you to consume the modules using registry URLs of the form brikis98/devops/book//modules/<MODULE>. For example, instead of the GitHub URL in Example 22, you can use the more convenient source URL shown in Example 23:

Registry URLs are a bit shorter, and they allow you to use the version parameter to specify the version, which is a bit cleaner than appending a ref parameter, and supports version constraints.

Run init on this code one more time:

The init command is responsible for downloading provider code and module code, and you can see in the preceding output that, this time, it downloaded the module code from the OpenTofu registry. If you now run apply, you should get the exact same two EC2 instances as before. When you’re done experimenting, run destroy to clean everything up.

You’ve now seen the power of reusable modules. A common pattern at many companies is for the Ops team to define and manage a library of vetted, reusable OpenTofu modules—e.g., one module to deploy servers, another to deploy databases, another to configure networking, and so on—and for the Dev teams to use these modules as a self-service way to deploy and manage the infrastructure they need for their apps.

This blog post series will make use of this pattern in future blog posts. Instead of writing every line of code from scratch, you’ll be able to use modules directly from this series’s sample code repo to deploy the infrastructure you need for each post.

How do provisioning tools stack up using the IaC category criteria from before?

Provisioning tools should be your go-to option for managing infrastructure. Moreover, many provisioning tools can be used to not only manage traditional infrastructure (e.g., servers), but many other aspects of software delivery as well. For example, you can use OpenTofu to manage your version control system (e.g., using the GitHub provider), metrics (e.g., using the Grafana provider), and your on-call rotation (e.g., using the PagerDuty provider), tying them all together with code.

Although I’ve been comparing IaC tools this entire blog post, the reality is that you’ll probably need to use multiple IaC tools together, as discussed in the next section.

Example: OpenTofu and Ansible. You use OpenTofu to deploy the underlying infrastructure, including the network topology, data stores, load balancers, and servers, and you use Ansible to deploy apps on top of those servers, as depicted in Figure 20:

This is an easy approach to get started with and there are many ways to get Ansible and OpenTofu to work together (e.g., OpenTofu adds tags to your servers, and Ansible uses an inventory plugin to automatically discover servers with those tags). The main downside is that using Ansible typically means mutable infrastructure, rather than immutable, so as your codebase, infrastructure, and team grow, maintenance and debugging can become more difficult.

Example: OpenTofu and Packer. You use Packer to package your apps as VM images, and you use OpenTofu to deploy your infrastructure, including servers that run these VM images, as illustrated in Figure 21:

This is also an easy approach to get started with. In fact, you already had a chance to try this combination out earlier in this post. Moreover, this is an immutable infrastructure approach, which will make maintenance easier. The main drawback is that VMs can take a long time to build and deploy, which slows down iteration speed.

Example: OpenTofu, Packer, Docker, and Kubernetes. You use Packer to create a VM image that has Docker and Kubernetes installed, you use OpenTofu to deploy your infrastructure, including servers that run this VM image, and when the servers boot up, they form a Kubernetes cluster that you use to run your Dockerized applications, as shown in Figure 22:

You’ll get to try this out in Part 3. The advantage of this approach is that you get the power of an IaC tool (OpenTofu) for managing your infrastructure, the power of server templating (Docker) for configuring your servers (with fast builds and the ability to run images on your local computer), and the power of an orchestration tool (Kubernetes) for managing your apps (including scheduling, auto healing, auto scaling, and service communication). The drawback is the added complexity, both in terms of extra infrastructure to run (the Kubernetes cluster) and in terms of several extra layers of abstraction (Kubernetes, Docker, Packer) to learn, manage, and debug.