Part 7. How to Set Up Networking

Just about the entire Internet runs on top of the Internet Protocol (IP), which is a set of rules for how to route and address data across networks. The first major version of IP, IPv4, which has been around since the 1980s, is the dominant protocol used on the Internet today; its successor, IPv6, started rolling out around 2006, and is gradually gaining adoption.

IP addresses are a central part of IP: each address (a) identifies one host on the network and (b) specifies the location of the host on the network, so you can route traffic to it. IPv4 addresses are 32-bit numbers which are typically displayed as four groups of two decimal digits, such as 11.22.33.44. With only 32-bits, the number of possible unique IPv4 addresses is 2³², or roughly 4 billion, which is a problem, as we’ve had far more than 4 billion Internet-connected devices for a long time.^[29] Running out of IPs is one of the reasons the world is moving to IPv6, which uses 128-bit addresses that are typically displayed as eight groups of four hexadecimal digits, such as 2001:0db8:85a3:0000:0000:8a2e:0370:7334. With 128-bit addresses, the number of possible unique addresses is 2¹²⁸, or roughly 340 undecillion (340 followed by 36 zeros), which is unlikely to ever run out. Unfortunately, IPv6 adoption world-wide is still well under 50%.^[30] Many older networking devices don’t support IPv6, so adoption takes a long time, as it requires updating software and hardware across millions of devices and networks around the world. Therefore, most of what you do with networking for now, as well as most of what this blog post will focus on, will be IPv4.

How do you get a public IP address? The Internet Assigned Numbers Authority (IANA) owns all public IP addresses and it assigns them in a hierarchical manner. At the top level, IANA delegates blocks of IP addresses to Internet registries that cover specific regions of the world. These registries, in turn, delegate blocks of IP addresses to network operators within their region, such as Internet Service Providers (ISPs), cloud providers (e.g., AWS, Azure, Google Cloud), enterprise companies, and so on. Finally, these network operators assign IPs to specific devices. For example, when you sign up for an Internet connection at home with an ISP, that ISP assigns you an IP address from its block of IPs; when you deploy EC2 instances in AWS, AWS assigns you an IP address from its block of IPs.^[31]

IP addresses are a fundamental building block of the Internet, and they work very well for computers talking to other computers, but they aren’t particularly human-friendly. If the only way to access your servers was to memorize a bunch of random numbers that may change from time to time, the Internet and World Wide Web probably wouldn’t have made it very far. What you want instead is to use memorable, human-friendly, consistent names. This is precisely the role of the Domain Name System, which is the topic of the next section.

The Domain Name System (DNS) is a service that allows you to use a human-friendly domain name to access a web service instead of an IP address: for example, you can use www.google.com instead of 172.253.116.139 to access Google’s servers. DNS stores the mapping from names to IP addresses in a globally-distributed hierarchy of nameservers. When you enter www.google.com into your web browser, your computer doesn’t talk to the nameservers directly, but instead sends a request to a local DNS resolver. At home, your ISP typically configures itself as the DNS resolver; in the cloud, the cloud provider typically configures itself as the DNS resolver. The DNS resolver takes the domain name and processes the parts in reverse order by making a series of queries to the hierarchy of name servers in Figure 68:

The DNS resolver’s first query goes to the root nameservers, which run at 13 IP addresses that are managed by IANA and hard-coded into most DNS resolvers, and these servers return the IP addresses of the top-level domain (TLD) nameservers for the TLD you requested (.com). The DNS resolver’s second query goes to these TLD nameservers, which are also managed by IANA, and they return the IP addresses to use for the authoritative nameservers for the domain you requested (google.com). Finally, the DNS resolver’s third query goes to these authoritative nameservers, which are operated by a variety of companies, such as Amazon Route 53, Azure DNS, GoDaddy, Namecheap, CloudFlare DNS, and so on, and these servers return the DNS records that contain the information that is associated with the domain name you requested (www.google.com).

There are many types of DNS records, each of which stores different kinds of information. For example, DNS A records and DNS AAAA records are "address" records that store IPv4 addresses and IPv6 addresses, respectively; DNS CNAME records are "canonical name" records that store aliases for a domain name; DNS TXT records are "text records" that can store arbitrary text; and so on. When your browser looks up www.google.com, it typically requests A or AAAA records. Three rounds of requests to get some DNS records may seem like a lot, but DNS is typically pretty fast, and there is a lot of caching along the way (e.g., your browser, OS, and DNS resolver may cache records for some period of time to reduce the number of lookups).

So that’s how DNS records are looked up, but how do they get there in the first place? Who decides who owns what domain? As with most things related to the Internet, this also goes back to IANA, which owns and manages all domain names. IANA delegates the management of these domain names to accredited registrars, who are allowed to sell domain names to end users. The registrars are often (but not always) the same companies that run authoritative name servers, such as Amazon Route 53, Azure DNS, GoDaddy, and so on. Note that, technically, you never own a domain name. You can only lease it, for which you pay an annual fee. If you stop paying that fee, the registrar can lease it to someone else.

Once you lease a domain name, you then have permissions to configure the DNS records for that domain in its authoritative nameservers, which allows users all over the world to access your servers via that domain name. DNS is a beautiful, scalable system, and getting your first domain name working can feel magical. Let’s try out an example of this magic by registering and configuring a domain name in Route 53.

In this section, you’ll deploy a web app, and set up a domain name for it. We’ll use Amazon’s Route 53 as the domain name registrar and the web app will be a simple HTTP server running on several EC2 instances that respond with "Hello, World!" This involves three steps: register a domain name, deploy EC2 instances, and configure DNS records. Let’s start by registering a domain name.

In this section, I’m going to walk you through an overview of how physical networks work. Note that networking is a huge topic, so I’ve had to compress a lot of information, and what you’re seeing here is a simplified picture that skips over some of the nuances. Let’s start by thinking through how you’d connect computers together. Connecting two computers is easy: all it takes is a single cable, as shown in Figure 73.

Connecting N computers is more complicated: if you had to connect every computer to every other computer, you’d need N² cables, which would be messy and expensive. The solution is to connect all the computers to a single switch, a device that can forward data between computers, which only requires N cables, as shown in Figure 74:

These connected computers form a network. Connecting two networks is easy: you typically do it using routers, as shown in Figure 75.

Connecting N networks is hard, as you have that N² problem again. The solution is to connect those routers using the Internet, as shown in Figure 76:

The term "Internet" is derived from interconnected networks: a network of networks. Many of those networks are private networks, which is the focus of this section.

Let’s look at two common private networks: a private network in your house and a private network in a data center. For your home network, you probably got a router from your ISP, which is actually both a switch and a router, and it creates a private network that allows the devices you have at home (e.g., your computer, laptop, phone, tablet, printer, TV) to talk to each other. For a data center network, the data center technicians set up various switches and routers, and this creates a private network that allows the servers in that data center to talk to each other.

Each of these private networks has several key characteristics:

The following sections go into detail on each of these, starting with only allowing authorized devices to connect to the private network.

If you deploy into the cloud, the cloud provider has already taken care of all the physical networking for you: all the servers, switches, routers, and cables are already hooked up, largely in a way you can’t see or control. What you can control is a virtual network, which is a network you configure entirely in software (which is why it’s sometimes referred to as software-defined networking). In the following several sections, you’ll learn about virtual networks in the cloud, virtual networks in orchestration tools, and then go through an example of creating a virtual network in AWS.

The traditional approach used at many companies for managing access to private networks is the castle-and-moat model, based on the analogy to a castle with a secure perimeter (walls, moat, drawbridge, etc.), and a soft interior; it’s hard to get into the castle, but once you’re inside, you have you free rein to move around. The equivalent with a private network is one that doesn’t allow you to access anything from outside the network, but once you’re "in" the network, you can access everything.

In a physical network, with the castle-and-moat model, merely being connected to the network means you’re "in." For example, with many corporate office networks, if you are plugged into the network via a physical cable, you can access everything in that network: all the wiki pages, the issue tracker, the IT help desk, and so on. However, if you’re outside the physical network, how do you connect to it? For example, if you’re working from home, how do you get access to your corporate office network? Or if you have infrastructure deployed in a VPC in the cloud, how do you get access to the private subnets of that VPC?

A common solution is to deploy a bastion host. In a fortress, a bastion is a structure that is designed to stick out of the wall, allowing for more reinforcement and extra armaments, so that it can better withstand attacks. In a network, a bastion host is a server that is designed to be visible outside the network (i.e., it’s in the DMZ), and this server has extra security hardening and monitoring, so it can better withstand attacks. The idea is that you keep the vast majority of your servers private, with the network acting as a secure perimeter (like a wall and moat), and you use the bastion host as the sole entrypoint to that network. Since there’s just one bastion, you can put a lot of effort into making it secure. You then allow users to connect to the bastion host using protocols such as SSH, RDP, or VPN, each of which we’ll dive into later in this blog post. Since the bastion host is "in" the network, once you’ve successfully connected to the bastion host, you’re now also "in," and you can freely access everything else in the network, as shown in Figure 78:

For example, if you are able to connect to the bastion host in Figure 78, you can then access everything in the private subnets of that VPC, including the private servers and database with IPs 10.0.0.20, 10.0.0.21, and 10.0.0.22. This approach worked well-enough in the past, but in the modern world, the castle-and-moat approach leads to security concerns, as discussed in the next section.

The castle-and-moat approach originated in a world where:

In short, your location on the network mattered: some locations could be trusted, while others could not. This is increasingly not the world we live in, as these days:

As a result, for many companies, the idea of a secure perimeter and soft interior no longer makes sense. There’s no clear "perimeter" or "interior" anymore, and there’s no location on the network that can be implicitly trusted. This has led to the rise of the zero-trust architecture (ZTA), which is based on the concept of "never trust, always verify," where you never trust a user or device just because they have access to some location on the network. The core principles of ZTA can be summarized as follows:

The zero-trust model has been evolving for many years. Some of the major publications on it include No More Chewy Centers: Introducing The Zero Trust Model Of Information Security by John Kindervag, where he coins the term "Zero Trust Model," Zero Trust Architecture by NIST, and BeyondCorp: A New Approach to Enterprise Security by Google. Google’s BeyondCorp paper is arguably what popularized the zero trust model, even though the paper doesn’t ever use that term.

A surprising principle in the BeyondCorp paper is that Google no longer requires employees working remotely to use VPN to access internal resources; instead, those resources are accessible directly via the public Internet. This may seem like a paradox: how can exposing internal resources to the public be more secure? Google’s take is that exposing internal tools publicly forces you to put more effort into securing them than if you merely relied on the network perimeter for security. Figure 79 shows a simplified version of the architecture Google described in BeyondCorp:

The idea is that you expose your internal resources to the public Internet via an access proxy, which uses the user database, device registry, and access policies to authenticate, authorize, and encrypt every connection. From a quick glance, the zero-trust approach in Figure 79 might not look all that different from the castle-and-moat approach in Figure 78: both rely on a single entrypoint to the network (a bastion host or an access proxy) that grants access to private resources. The key difference is that in the castle-and-moat approach, only the bastion host is protected, and all the private resources are open, so if you can get past the bastion, you get access to all the private resources, whereas with the zero-trust approach, every single private resource is protected, and each one requires you to go through an authorization process with the access proxy. Instead of a single perimeter around all the resources in your network, the zero-trust approach is like putting a separate perimeter around each individual resource.

That means that zero-trust isn’t a single tool you adopt, but something you integrate into every part of your architecture, including the following:

Implementing a true zero-trust architecture is a tremendous amount of work, and few companies pull it off fully. It’s a good goal for all companies to strive for, but how far down the ZTA path you go depends on your scale of company. Smaller startups will typically use the castle-and-moat approach; mid-sized companies will often adopt a handful of ZTA principles, such as using SSO and securing microservice communication; large enterprises will try to go for most of the ZTA principles. As you saw in Section 1.1.2, you need to adapt your architecture to the needs and capabilities of your company.

Now that you’ve seen the castle-and-moat and zero-trust models, let’s look at some of the most common tools you use to access private networks:

The following sections will go through each of these options, starting with SSH.

Secure Shell (SSH) is a protocol that allows you to connect to a computer over the network to execute commands. It uses a client-server architecture, as shown in Figure 80. For example, the client could be the computer of a developer on your team named Alice and the server could be the bastion host. When Alice connects to the bastion host over SSH, she gets a remote terminal where she can run commands and access the private network as if she was using the bastion host directly.

SSH is ubiquitous: just about all modern Linux, Unix, and macOS distributions support SSH natively, and there are multiple clients for Windows. SSH is also generally considered a mature and secure protocol, as it’s an open standard with open source implementations, it has been around for around 30 years, and due to its ubiquity, it has a massive community around it.

Under the hood, SSH uses public-key cryptography for authentication and encryption; you’ll learn more about these topics in Part 8. For now, all you need to know is that SSH relies on a key pair, which consists of a public key and a private key. Configuring one server to accept one user’s public key is no problem, but at scale, this becomes a challenge. If you need to support a large number of servers and developers, key rotation and revocation (e.g., when a developer leaves the company), and different levels of permissions and access (including temporary access), things get a lot more complicated. One solution is to use managed services from cloud providers, such as EC2 instance connect and Systems Manager in AWS and metadata-managed SSH connections in Google Cloud. Another solution is to use the general-purpose connectivity tools I mentioned earlier: e.g., Teleport, TailScale, etc.

Let’s take a quick look at how to use SSH.

Remote Desktop Protocol (RDP) is a way to connect to a Windows server remotely and to manage it via the full Windows user interface, as shown in Figure 82. It’s just like being at the computer: you can use the mouse, keyboard, and all the desktop apps.

Being able to use the full Windows UI makes RDP accessible to all roles at a company (not just developers), and it can be a nicer experience than being limited to a terminal (as with SSH). However, RDP only works with Windows servers, and it is somewhat notorious for security vulnerabilities, so you can’t expose it directly to the public Internet (as you’ll see shortly).

Let’s take a quick look at how to use RDP.

A Virtual Private Network (VPN) is a way to extend a private network across multiple networks or devices. The fact that you’re on VPN is transparent to the software running on those devices: that is, that software can communicate with the private network as if the device was plugged physically into the network, without the software being aware of the VPN or having to do anything differently.

There are VPN clients for almost every operating system (including smartphones), allowing you to access private networks from your own devices in a way that’s accessible to all roles at a company (and not just developers). Most VPN tools are built around either IPsec (Internet Protocol Security) or TLS (Transport Layer Security), two protocols that are generally considered mature and secure, as they have been around for more than 30 years, are ubiquitous, and have massive communities around them. These protocols are used keep all traffic going over the VPN secure through the use of encryption.

Note that IPsec and TLS typically rely on certificates, which are based on public-key cryptography (like SSH), but allow for mutual authentication, where the client can verify the VPN server is really who it says it is using the server’s certificate, and the server can verify the user is really who they say they are using the client’s certificate. This is great for security, but managing certificates at scale can sometimes be challenging (you’ll learn more about IPsec, TLS, and certificates in Part 8). Another challenge with VPN is that routing all your network traffic through VPN servers can increase latency, and, depending on usage, degrade throughput.

These days, there are three common use cases for VPNs:

Let’s take a quick look at how to use VPN for the first two use cases.

As soon as you have one service, A, that needs to talk to another service, B, you have to figure out service discovery: how does A figure out the right IP addresses to use to talk to B? This can be a challenging problem, as each service may have multiple replicas running on multiple servers, and the number of replicas and which servers they are running on may change frequently as you deploy new versions, replicas crash and are replaced, or you scale the number of replicas up or down in response to load.

The next section will go over some of the tools you can use for service discovery, and the section after that will compare service discovery tools to help you pick the right tool for the job.

As you saw in Part 6, a big part of breaking your code into services is defining an API for the service, and maintaining it over the long term. One of the key decision you’ll have to make is the protocol you will use for that API, which actually consists of two primary decisions:

The next section will go over some of the most common protocols in use today, and the section after that will go over the key factors to consider when picking a protocol.

A service mesh is a networking layer that is designed to help manage communication between applications in a microservice architecture by providing a single, unified solution to the following problems:

Almost all of these are problems of scale. If you only have two or three services, a small team, and not a lot of load, these problems are not likely to affect you, and a service mesh may be an unnecessary overhead; however, if you have hundreds of services owned by dozens of teams and high load, these are problems you’ll be dealing with every day. If you try to solve these problems one at a time, you’ll find it is a huge amount of work, and that the solution to one has an impact on the other: e.g., how you manage encryption affects your ability to do tracing and traffic mirroring. Moreover, the simple solutions you’re likely to try first may require you to make code changes to every single app, and as you learned in Part 6, rolling out global changes across many services can take a long time.

This is where a service mesh can be of use: it gives you an integrated, all-in-one solution to these problems, and just as importantly, it can solve most of these problems in a way that is transparent, and does not require you to change your app code.

When things are working, a service mesh can feel like a magical way to dramatically upgrade the security and debuggability of your microservice architecture. However, when things aren’t working, the service mesh itself can be difficult to debug. Service meshes solve many problems, which is great, but at the cost of introducing many new moving parts that can be the source of new problems: e.g., encryption, authentication, authorization, routing, firewalls, tracing, and so on. Understanding, installing, configuring, and managing a service mesh can be a lot of overhead. If you’re at the scale where you need solutions to the problems listed earlier, it’s worth it; if you’re a tiny startup, it’ll only slow you down.

Service mesh tools can be broken down into three buckets. The first bucket is the service mesh tools designed for use with Kubernetes, which includes Linkerd (the project that coined the term "service mesh"), Istio, Cilium, and Traefik Mesh. The second bucket is managed service mesh tools from cloud providers, such as AWS App Mesh and Google Cloud Service Mesh. The third bucket is service mesh tools that can be used with any orchestration approach (e.g., Kubernetes, EC2, on-prem servers, etc.), such as Consul service mesh, Kuma, and Kong Mesh.

The best way to get a feel for what a service mesh does is to try one out, so let’s go through an example of using Istio with Kubernetes.

Istio is a popular service mesh for Kubernetes that was originally created by Google, IBM and Lyft, and open sourced in 2017. Let’s see how Istio can help you manage the two microservices you deployed with Kubernetes in Part 6. One of those microservices was a backend app that exposed a simple JSON-over-HTTP REST API and the other microservice was a frontend app that made service calls to the backend, using the service discovery mechanism built into Kubernetes, and then rendered the data it got back using HTML. Make a copy of those two sample apps into the folder you’re using for this blog post’s examples:

Next, let’s install Istio. Use the Kubernetes cluster that’s part of Docker Desktop, just as you did in Part 3, making sure you’re authenticated to that cluster as follows:

Download Istio using curl:

This will download and extract Istio into a folder called istio-<VERSION>, where <VERSION> is the latest Istio version number, such as 1.22.3. Head into that folder:

Inside the bin folder, you’ll find istioctl, which is a CLI tool that has useful helper functions for working with Istio. Add the bin folder to your PATH:

Note that the preceding command will only add bin to your PATH in the current terminal window; if you want to make this a permanent change available in all terminal windows, add the same code to your profile configuration: i.e., ~/.profile, ~/.bash_profile or ~/.zprofile.

Next, install Istio as follows:

This uses a minimal profile to install Istio, which is good enough for learning and testing (see the install instructions for profiles you can use for production). The way Istio works is to inject its own sidecar into every Pod you deploy into Kubernetes. That sidecar is what provides all the security, observability, resiliency, and traffic management features, without you having to change your application code. To configure Istio to inject its sidecar into all Pods you deploy into the default namespace, run the following command:

Istio supports a number of integrations with observability tools. For this example, let’s use the sample add-ons that come with the Istio installer, which include a dashboard for Istio called Kiali, a database for monitoring data called Prometheus, a UI for visualizing monitoring data called Grafana, and a distributed tracing tool called Jaeger:

At this point, you can verify everything is installed correctly by running the verify-install command:

If everything looks good, deploy the frontend and backend as you did before:

After a few seconds, you should be able to make a request to the frontend as follows:

At this point, everything should be working exactly as before. So is Istio doing anything? One way to find out is to open up the Kiali dashboard you installed earlier:

This command will open the dashboard in your web browser. Click Traffic Graph in the nav on the left, and you should see something similar to Figure 83:

If the Traffic Graph doesn’t show you anything, run curl localhost several more times, and then click the refresh button in the top right of the dashboard. You should see a visualization of the path your requests take through your microservices, including through the Services and Pods. Right away, you see one of the key benefits of a service mesh: observability. You get not only this service mesh visualization, but also aggregated logs (click on Workloads in the left nav, select sample-app-backend, and click the Logs tab), metrics (run istioctl dashboard grafana), and distributed traces (run istioctl dashboard jaeger).

Another key benefit of service meshes is security, including support for automatically encrypting, authenticating, and authorizing all requests within the service mesh. By default, to make it possible to install Istio without breaking everything, Istio initially allows unencrypted, unauthenticated, and unauthorized requests to go through. However, you can change this by configuring policies in Istio.

Create a file called istio/istio-auth.yml with the policies shown in Example 132:

This code does the following:

Hit CTRL+C to shut down Kiali and deploy these policies as follows:

Now, look what happens if you try to access the frontend app again:

Since your request to the frontend wasn’t using mTLS, Istio rejected the connection immediately. Enforcing mTLS makes sense for backends, as they should only be accessible to other services, but your frontend should be accessible to users outside your company, so you can disable the mTLS requirement for the frontend as shown in Example 133:

This is an authentication policy that works as follows:

You can put the YAML in Example 133 into a new YAML file, but dealing with too many YAML files for the frontend is tedious and error-prone. Let’s instead use the --- divider to combine the frontend’s sample-app-deployment.yml, sample-app-service.yml, and the YAML you just saw in Example 133 into a single file called kubernetes-config.yml, with the structure shown in Example 134:

With all of your YAML in a single kubernetes-config.yml, you can delete the sample-app-deployment.yml and sample-app-service.yml files, and deploy changes to the frontend app with a single call to kubectl apply:

Try accessing the frontend again, adding the --write-out flag so that curl prints the HTTP response code after the response body:

This time, Istio did not close the connection immediately, as authentication and encryption with mTLS are no longer required. However, you got a 403 response code (Forbidden) and "access denied" in the response body because the allow-nothing authorization policy is still blocking all requests. To fix this, you need to add authorization policies to the backend and the frontend.

This requires that Istio has some way to identify the frontend and backend. Istio uses Kubernetes service accounts as identities, automatically providing a TLS certificate to each application based on its service account, and using mTLS to provide mutual authentication (i.e., the backend will verify the request is coming from the frontend, and the frontend will verify it is really talking to the backend). Istio will handle all the TLS details for you, so all you need to do is associate the frontend and backend with their own service accounts and add an authorization policy to each one.

Start with the frontend, updating its kubernetes-config.yml as shown in Example 135:

Here are the updates to make to the frontend:

Run apply to deploy these changes to the frontend:

Next, head over to the backend, and combine its Deployment and Service definitions into a single kubernetes-config.yml file, separated by ---, just like you did for the frontend. Once that’s done, update the backend’s kubernetes-config.yml as shown in Example 136:

Here are the updates to make to the backend:

Run apply to deploy these changes to the backend:

And now test the frontend one more time:

Congrats, you got a 200 (OK) response code and the expected HTML response body, which means you now have microservices running in Kubernetes, using service discovery, and communicating securely via a service mesh! With the authentication and authorization policies you have in place, you have significantly improved your security posture: all communication between services (such as the request the frontend successfully made to the backend) is now encrypted, authenticated, and authorized—all without you having to modify the Node.js source code of either app. Moreover, you have access to all the other service mesh benefits, too: observability, resiliency, and traffic management.

When you’re done testing, you can run delete on the kubernetes-config.yml files of the frontend and backend to clean up the apps. If you wish to uninstall Istio, first, remove the global authorization and authentication policies:

Next uninstall the addons:

And finally, uninstall Istio itself, including deleting its namespace, and removing the default labeling behavior:

One of the benefits of software-defined networking is that it’s fast and easy to try out different networking approaches. Instead of having to spend hours or days setting up physical routers, switches, and cables, you can try out a tool like Istio in minutes, and if it doesn’t work for you, it only takes a few more minutes to uninstall Istio, and try something else. And, of course, you can save your networking configuration (e.g., Istio policies) as code, which allows you to go back and forth as much as you want without losing any of your past work. Keep experimenting until you find the solution that best fits your needs.