Part 7. How to Set Up Networking

Just about the entire Internet runs on top of the Internet Protocol (IP), which is a set of rules for how to route and address data across networks. The first major version of IP, IPv4, which has been around since the 1980s, is the dominant protocol used on the Internet today; its successor, IPv6, started rolling out around 2006, and is gradually gaining adoption.

IP addresses are a central part of IP: each address (a) identifies one host on the network and (b) specifies the location of the host on the network so you can route traffic to it. IPv4 addresses are 32-bit numbers which are typically displayed as four groups of two decimal digits, such as 11.22.33.44. With only 32-bits, the number of possible unique IPv4 addresses is 2³², or roughly 4 billion, which is a problem, as we’ve had far more than 4 billion Internet-connected devices for a long time^[35]. Running out of IPs is one of the reasons the world is moving to IPv6, which uses 128-bit addresses that are typically displayed as eight groups of four hexadecimal digits, such as 2001:0db8:85a3:0000:0000:8a2e:0370:7334. With 128-bit addresses, the number of possible unique addresses is 2¹²⁸, or roughly 340 undecillion (340 followed by 36 zeros), which is unlikely to ever run out. Unfortunately, IPv6 adoption world-wide is still well under 50%^[36]: many older networking devices don’t support IPv6, so adoption takes a long time, as it requires updating software and hardware across millions of devices and networks around the world. Therefore, most of what you do with networking for now, as well as most of what this blog post will focus on, will be IPv4.

How do you get a public IP address? The Internet Assigned Numbers Authority (IANA) owns all public IP addresses and it assigns them in a hierarchical manner: at the top level, IANA delegates blocks of IP addresses to Internet registries that cover specific regions of the world. These registries, in turn, delegate blocks of IP addresses to network operators within their region, such as Internet Service Providers (ISPs), cloud providers (e.g., AWS, Azure, Google Cloud), enterprise companies, and so on. Finally, these network operators assign IPs to specific devices: for example, when you sign up for an Internet connection at home with an ISP, that ISP assigns you an IP address from its block of IPs; and when you deploy EC2 instances in AWS, AWS assigns you an IP address from its block of IPs.^[37]

IP addresses are a fundamental building block of the Internet, and they work very well for computers talking to other computers, but they aren’t particularly human-friendly: if the only way to access your servers was to memorize a bunch of random numbers that may change from time to time, the Internet and World Wide Web probably wouldn’t have made it very far. What you want instead is to use memorable, human-friendly, consistent names. This is precisely the role of the Domain Name System, which is the topic of the next section.

The Domain Name System (DNS) is a service that allows you to use a human-friendly domain name to access a web service instead of an IP address: for example, you can use www.google.com instead of 172.253.116.139 to access Google’s servers. DNS stores the mapping from names to IP addresses in a globally-distributed hierarchy of nameservers. When you enter www.google.com into your web browser, your computer doesn’t talk to the nameservers directly, but instead sends a request to a local DNS resolver: at home, your ISP typically configures itself as the DNS resolver; in the cloud, the cloud provider typically configures itself as the DNS resolver. The DNS resolver takes the domain name and processes the parts in reverse order by making a series of queries to the hierarchy of name servers in Figure 74:

The DNS resolver’s first query goes to the root nameservers, which run at 13 IP addresses that are managed by IANA and hard-coded into most DNS resolvers, and these servers return the IP addresses of the top-level domain (TLD) nameservers for the TLD you requested (.com). The DNS resolver’s second query goes to these TLD nameservers, which are also managed by IANA, and they return the IP addresses to use for the authoritative nameservers for the domain you requested (google.com). Finally, the DNS resolver’s third query goes to these authoritative nameservers, which are operated by a variety of companies, such as Amazon Route 53, Azure DNS, GoDaddy, Namecheap, CloudFlare DNS, and so on, and these servers return the DNS records that contain the information that is associated with the domain name you requested (www.google.com).

There are many types of DNS records, each of which stores different kinds of information: for example, DNS A records and DNS AAAA records are "address" records that store IPv4 addresses and IPv6 addresses, respectively; DNS CNAME records are "canonical name" records that store aliases for a domain name; DNS TXT records are "text records" that can store arbitrary text; and so on. When your browser looks up www.google.com, it typically requests A or AAAA records. Three rounds of requests to get some DNS records may seem like a lot, but DNS is typically pretty fast, and there is a lot of caching along the way: e.g., your browser, your operating system, and the DNS resolver may all cache records for some period of time to reduce the number of lookups.

So that’s how DNS records are looked up, but how do they get there in the first place? Who decides who owns what domain? As with most things related to the Internet, this also goes back to IANA, which owns and manages all domain names. IANA delegates the management of these domain names to accredited registrars, who are allowed to sell domain names to end users. The registrars are often (but not always) the same companies that run authoritative name servers, such as Amazon Route 53, Azure DNS, GoDaddy, and so on. Note that, technically, you never own a domain name: you can only lease it, for which you pay an annual fee. If you stop paying that fee, the registrar can lease it to someone else.

Once you lease a domain name, you then have permissions to configure the DNS records for that domain in its authoritative nameservers, which allows users all over the world to access your servers via that domain name. DNS is a beautiful, scalable system, and getting your first domain name working can feel magical. Let’s try out an example of this magic by registering and configuring a domain name in Route 53.

In this section, you’ll deploy a web app, and set up a domain name for it. We’ll use Amazon’s Route 53 as the domain name registrar and the web app will be a simple HTTP server running on several EC2 instances that respond with "Hello, World!" This involves three steps: register a domain name, deploy EC2 instances, and configure DNS records. Let’s start by registering a domain name.

In this section, I’m going to walk you through an overview of how physical networks work, to help build your intuition for why private networks are useful and what’s happening under the hood.

Let’s start by thinking through how you’d connect computers together. Connecting two computers is easy: all it takes is a single cable, as shown in Figure 79.

Connecting N computers is more complicated: if you had to connect every computer to every other computer, you’d need N² cables, which would be very messy and expensive. The typical solution is to connect all the computers to a single switch, which is a device that can forward data between computers, which only requires N cables, as shown in Figure 80:

These connected computers form a network. Connecting two networks together is easy: you typically do it using routers, as shown in Figure 81.

Connecting N networks together is hard: you’d have that N² problem again. The typical solution is to connect those routers together using the Internet, as shown in Figure 82:

The term "Internet" is derived from interconnected networks: it’s a network of networks. Many of those networks are private networks, which is the focus of this section.

Let’s look at two common private networks: a private network in your house and a private network in a data center. For your home network, you probably got a router from your ISP, which is actually both a switch and a router, and it creates a private network which allows the devices you have at home (e.g., your computer, laptop, phone, tablet, printer, TV) to talk to each other. For a data center network, the data center technicians set up various switches and routers, and this creates a private network which allows the servers in that data center to talk to each other.

Each of these private networks has several key characteristics:

The following sections go into detail on each of these, starting with only allowing authorized devices to connect to the private network.

If you deploy into the cloud, the cloud provider has already taken care of all the physical networking for you: all the servers, switches, routers, and cables are already hooked up, largely in a way you can’t see or control. What you can control is a virtual network, which is a network you configure entirely in software (which is why it’s sometimes referred to as software-defined networking). In the following several sections, you’ll learn about virtual networks in the cloud, virtual networks in orchestration tools, and then go through an example of creating a virtual network in AWS.

The traditional approach used at many companies for managing access to private networks is the castle-and-moat model, based on the analogy to a castle with an extremely secure perimeter (walls, moat, drawbridge, etc.), which makes it hard to get into the castle, but a soft interior, which allows you free rein to move around once you’re inside. The equivalent with a private network is one that doesn’t allow you to access anything from outside the network, but once you’re "in" the network, you can access everything.

In a physical network, with the castle-and-moat model, merely being connected to the network means you’re "in." For example, with many corporate office networks, if you are plugged into the network via a physical cable, you can access everything in that network: all the wiki pages, the issue tracker, the IT help desk, and so on. However, if you’re outside the physical network, how do you connect to it? For example, if you’re working from home, how do you get access to your corporate office network? Or if you have infrastructure deployed in a VPC in the cloud, how do you get access to the private subnets of that VPC?

A common solution is to deploy a bastion host. In a fortress, a bastion is a structure that is designed to stick out of the wall, allowing for more reinforcement and extra armaments, so that it can better withstand attacks. In a network, a bastion host is a server that is designed to be visible outside the network (i.e., it’s in the DMZ), and this server has extra security hardening and monitoring, so it can better withstand attacks. The idea is that you keep the vast majority of your servers private, with the network acting as a secure perimeter (like a wall and moat), and you use the bastion host as the sole entrypoint to that network. Since there’s just one bastion, you can put a lot of effort into making it as secure as possible. You then allow users to connect to the bastion host using protocols such as SSH, RDP, or VPN, each of which we’ll dive into later in this blog post. Since the bastion host is "in" the network, once you’ve successfully connected to the bastion host, you’re now also "in," and you can freely access everything else in the network, as shown in Figure 84:

For example, if you are able to connect to the bastion host in Figure 84, you can then access everything in the private subnets of that VPC, including the private servers and database with IPs 10.0.0.20, 10.0.0.21, and 10.0.0.22. This approach worked well-enough in the past, but in the modern world, the castle-and-moat approach leads to security concerns, as discussed in the next section.

The castle-and-moat approach originated in a world where:

In short, your location on the network mattered: some locations could be trusted, while others could not. This is increasingly not the world we live in. In fact, these days:

As a result, for many companies, the idea of a secure perimeter and soft interior no longer makes sense. There’s no clear "perimeter" or "interior" anymore, and there’s no location on the network that can be implicitly trusted. This has led to the rise of the zero-trust architecture (ZTA), which is based on the concept of "never trust, always verify," where you never trust a user or device just because they have access to some location on the network. The core principles of ZTA can be summarized as follows:

The zero-trust model has been evolving for many years. Some of the major publications on it include No More Chewy Centers: Introducing The Zero Trust Model Of Information Security by John Kindervag, where he coins the term "Zero Trust Model," Zero Trust Architecture by NIST, and BeyondCorp: A New Approach to Enterprise Security by Google. Google’s BeyondCorp paper is arguably what popularized the zero trust model, even though the paper doesn’t ever use that term explicitly, but the principles are largely the same.

One of the more controversial principles in the BeyondCorp paper is that Google no longer requires employees working remotely to use VPN to access internal resources: instead, those resources are accessible directly via the public Internet. At first, this feels like a paradox: how can exposing internal resources to the public be more secure? Google’s take is that exposing internal tools publicly forces you to put much more effort into securing them than if you merely relied on the network perimeter for security. Figure 85 shows a simplified version of the type of architecture Google described in BeyondCorp:

The idea is that you expose all your internal resources to the public Internet via an access proxy which uses your user database, device registry, and access policies to authenticate, authorize, and encrypt every single connection. From a quick glance, the zero-trust approach in Figure 85 might not look all that different from the moat-and-castle approach in Figure 84: both rely on a single entrypoint to the network (a bastion host or an access proxy) that authorizes connections before granting access to private resources. However, the key difference is that in the moat-and-castle approach, only the bastion host is protected, and all the private resources are open, so if you can get past the bastion, you get access to all the private resources, whereas with the zero-trust approach, every single private resource is protected, and each one requires you to go through the authorization process with the access proxy. Instead of a single, strong perimeter around all the resources in your network, the zero-trust approach is a bit like putting a separate strong perimeter around each individual resource.

That means that zero-trust isn’t a single tool you adopt, but something you integrate into every part of your architecture, including the following:

Implementing a true zero-trust architecture is a tremendous amount of work. The reality is that very few companies are able to fully pull it off. It’s a good goal for all companies to strive for, but how far down the ZTA path you go depends on your scale of company: smaller startups will typically use the castle-and-moat approach; mid-sized companies will often adopt a handful of ZTA principles, such as using SSO and securing microservice communication; large enterprises will try to go for all the ZTA principles. As you saw in Section 1.1.2, you need to adapt your architecture to the needs and capabilities of your company.

Whether you use the castle-and-moat model, the zero-trust model, or something in between, there is a common set of tooling that you will typically use to access any private network:

Note that these options are not mutually exclusive, as it’s common to combine VPN with SSH and RDP. The following sections will go through each of these options in detail, starting with SSH.

Secure Shell (SSH) is a protocol that allows you to connect to a computer over the network to execute commands. It uses a client-server architecture, as shown in Figure 86: for example, the client could be the computer of a developer on your team named Alice and the server could be the bastion host. When Alice connects to the bastion host over SSH, she gets a remote terminal where she can run commands and access the private network as if she was using the bastion host directly.

Let’s take a quick look at how to use SSH, followed by its advantages and drawbacks.

Remote Desktop Protocol (RDP) is a way to connect to a Windows server remotely and to manage it via the full Windows user interface, as shown in Figure 88. It’s just like being at the computer: you can use the mouse, keyboard, and all the desktop apps.

Let’s take a quick look at how to use RDP, followed by its advantages and drawbacks.

A Virtual Private Network (VPN) is a way to extend a private network across multiple other networks or devices. One of the main goals is for the VPN to be transparent to software running on those devices: that is, that software should be able to communicate with the private network as if the device was plugged physically into the network, without the software being aware of the VPN or having to do anything differently. Another main goal is to keep all traffic going over the VPN secure through the use of encryption.

These days, there are three common use cases for VPNs:

Let’s take a quick look at how to use VPN, followed by its advantages and drawbacks.

As soon as you have one service, A, that needs to talk to another service, B, you have to figure out service discovery: how does A figure out the right IP addresses to use to talk to B? This can be a challenging problem as each service may have multiple replicas running on multiple servers, and the number of replicas and which servers they are running on may change frequently as you deploy new versions, replicas crash and are replaced, or you scale the number of replicas up or down in response to load.

In the next section, we’ll go over some of the tools you can use to solve this problem, and in the section after that, we’ll compare the tools to help you figure out which one is the right fit for your use cases.

As you saw in Part 6, a big part of breaking your code into services is defining an API for the service, and maintaining it over the long term. One of the key decision you’ll have to make is the protocol you will use for that API, which actually consists of two primary decisions:

In the next section, we’ll go over some of the most common protocols in use today, and in the section after that, we’ll go over the key factors to consider when picking a protocol.

A service mesh is a networking layer that is designed to help manage communication between applications in a (micro)service architecture by providing a single, unified solution to the following problems:

Almost all of these are problems of scale: if you only have two or three services, a small team, and not a lot of load, these problems are not likely to affect you, and a service mesh may be an unnecessary overhead; however, if you have hundreds of services owned by dozens of teams, processing tens of thousands of requests per second, these are problems you’ll be dealing with every day. If you try to solve these problems individually, one at a time, you’ll find it is a huge amount of work, and that the solution to one has an impact on the other: e.g., how you manage encryption has a big impact on your ability to do tracing and traffic mirroring. Moreover, the simple solutions you’re likely to try first may require you to make code changes to every single app, and as you learned in Part 6, rolling out global changes across many services can take a very long time.

This is where a service mesh can be of use: it gives you an integrated, all-in-one solution to these problems, and just as importantly, it can solve most of these problems in a way that is transparent, and does not require you to make changes to your application code.

When things are working, a service mesh can feel like a magical way to dramatically upgrade the security and debuggability of your (micro)service architecture. However, when things aren’t working, the service mesh itself can be difficult to debug. Service meshes solve many problems, which is great, but at the cost of introducing many new moving parts that can be the source of new problems: e.g., encryption, authentication, authorization, routing, firewalls, tracing, and so on. Understanding, installing, configuring, and managing a service mesh can be a lot of overhead. If you’re at the scale where you need solutions to the problems listed earlier, it’s worth it; if you’re a tiny startup, it’ll only slow you down.

Service mesh tools can be broken down into three buckets. The first bucket is the service mesh tools designed for use with Kubernetes:

The second bucket is managed service mesh tools from cloud providers:

The third bucket is service mesh tools that can be used with any orchestration approach in any deployment environment (e.g., Kubernetes, EC2, on-prem servers, etc.):

The best way to get a feel for what a service mesh does is to try one out, so let’s go through an example of using Istio with Kubernetes.

Istio is a popular service mesh for Kubernetes that was originally created by Google, IBM and Lyft, and open sourced in 2017. Let’s see how Istio can help you manage the two microservices you deployed with Kubernetes in Part 6: one of those microservices was a backend app that exposed a simple JSON-over-HTTP REST API and the other microservice was a frontend app that made service calls to the backend, using the service discovery mechanism built into Kubernetes, and then rendered the data it got back using HTML. Make a copy of those two sample apps into the folder you’re using for this blog post’s examples:

Next, let’s install Istio. We’ll use the same local Kubernetes cluster that’s part of Docker Desktop, just as you did in Part 3. Make sure you’re authenticated to that cluster as follows:

Download Istio using curl:

This will download and extract Istio into a folder called istio-<VERSION>, where <VERSION> is the latest Istio version number, such as 1.22.3. Head into that folder:

Inside the bin folder, you’ll find istioctl, which is a CLI tool that has useful helper functions for working with Istio. Add the bin folder to your PATH:

Note that the preceding command will only add bin to your PATH in the current terminal window; if you want to make this a permanent change available in all terminal windows, add the same code to your profile configuration: i.e., ~/.profile, ~/.bash_profile or ~/.zprofile.

Next, install Istio as follows:

This uses a minimal profile to install Istio, which is good enough for learning and testing. If you’re installing Istio for production, see the install instructions for other profiles you can use.

The way Istio works is to inject its own sidecar into every Pod you deploy into Kubernetes. That sidecar is what provides all the security, observability, resiliency, and traffic management features, without you having to change your application code. To configure Istio to inject its sidecar into all Pods you deploy into the default namespace, run the following command:

Istio supports a number of integrations with observability tools. For this example, let’s use the sample add-ons that come with the Istio installer, which include a dashboard for Istio called Kiali, a database for monitoring data called Prometheus, a UI for visualizing monitoring data called Grafana, and a distributed tracing tool called Jaeger:

At this point, you can verify everything is installed correctly by running the verify-install command:

If everything looks good, deploy the frontend and backend as you did before:

After a few seconds, you should be able to make a request to the frontend as follows:

At this point, everything should be working exactly as before. So is Istio doing anything? One way to find out is to open up the Kiali dashboard you installed earlier:

This command should pop open the dashboard in your web browser. Click the Traffic Graph link in the nav on the left, and you should see something similar to Figure 89:

If the Traffic Graph doesn’t show you anything, run curl localhost several more times, and then click the refresh button in the top right of the dashboard. You should see a visualization of the path your requests take through your microservices, including through the Services and Pods. This is just one of the observability tools you get with Istio. To see another one, click on Workloads in the left nav, select sample-app-backend, and click the Logs tab. You should see logs for your backend, including logs from the Node.js app, as well as access logs from Istio components, such as the Envoy proxy, as shown in Figure 90:

Right away, you see one of the key benefits of a service mesh: observability. Istio gives you distributed tracing, logs, and metrics.

Another key benefit of service meshes is security, including support for automatically encrypting, authenticating, and authorizing all requests within the service mesh. By default, to make it possible to install Istio without breaking everything, Istio initially allows unencrypted, unauthenticated, and unauthorized requests to go through. However, you can change this by configuring policies in Istio.

Create a file called istio/istio-auth.yml with the policies shown in Example 140:

This code does the following:

Hit CTRL+C to shut down Grafana and deploy these policies as follows:

Now, look what happens if you try to access the frontend app again:

Since your request to the frontend wasn’t using mTLS, Istio rejected the connection immediately. Enforcing mTLS makes sense for backends, as they should only be accessible to other services, but your frontend should be accessible to users outside your company, so you should disable the mTLS requirement for just the frontend, as shown in Example 141:

This is an authentication policy that works as follows:

You can put the YAML in Example 141 into a new YAML file, but dealing with too many YAML files for the frontend is tedious and error-prone. Let’s instead use the --- divider to combine the frontend’s sample-app-deployment.yml, sample-app-service.yml, and the YAML you just saw in Example 141 into a single file called kubernetes-config.yml, with the structure shown in Example 142:

With all of your YAML in a single kubernetes-config.yml, you can delete the sample-app-deployment.yml and sample-app-service.yml files, and deploy changes to the frontend app with a single call to apply:

Try accessing the frontend again, adding the --write-out flag so that curl prints the HTTP response code after the response body:

This time, Istio did not close the connection immediately, as authentication and encryption with mTLS are no longer required. However, you got a 403 response code (Forbidden) and "access denied" in the response body because the allow-nothing authorization policy is still blocking all requests. To fix this, you need to add authorization policies to the backend and the frontend.

This requires that Istio has some way to identify the frontend and backend (authentication). Istio uses Kubernetes service accounts as identities, providing a TLS certificate to each application based on its service identity, and using mTLS to provide mutual authentication: that is, it’ll not only have the backend verify the request is coming from the frontend, but before sending that request, it’ll have the frontend verify it is really talking to the backend. Istio will handle all the TLS details for you, so all you need to do is associate the frontend and backend with their own service accounts and add an authorization policy to each one.

Let’s start with the frontend. Update its kubernetes-config.yml as shown in Example 143:

Here are the updates to make to the frontend:

Run apply to deploy these changes to the frontend:

Next, head over to the backend, and combine its Deployment and Service definitions into a single kubernetes-config.yml file, separated by ---, just like you did for the frontend. Once that’s done, update the backend’s kubernetes-config.yml as shown in Example 144:

Here are the updates to make to the backend:

Run apply to deploy these changes to the backend:

And now test the frontend one more time:

Congrats, you got a 200 (OK) response code and the expected HTML response body, which means you now have microservices running in Kubernetes, using service discovery, and communicating securely via a service mesh! With the authentication and authorization policies you have in place, you have significantly improved your security posture: all communication between services (such as the request the frontend successfully made to the backend) is now encrypted, authenticated, and authorized—all without you having to modify the Node.js source code of either app. Moreover, you have access to all the other service mesh benefits, too: observability, resiliency, and traffic management.

When you’re done testing, you can run delete on the kubernetes-config.yml files of the frontend and backend to clean up the apps. If you wish to uninstall Istio, first, remove the global authorization and authentication policies:

Next uninstall the addons:

And finally, uninstall Istio itself, including deleting its namespace, and removing the default labeling behavior:

One of the benefits of software-defined networking is that it’s fast and easy to try out different networking approaches. Instead of having to spend hours or days setting up physical routers, switches, and cables, you can try out a tool like Istio in minutes, and if it doesn’t work for you, it only takes a few more minutes to uninstall Istio, and try something else. And, of course, you can save your networking configuration (e.g., Istio policies) as code, which allows you to go back and forth as much as you want without losing any of your past work. Keep experimenting until you find the solution that best fits your needs.