Introduction to Kubernetes Networking

This article extends my notes from an Udemy course "Kubernetes for the Absolute Beginners - Hands-on". All course content rights belong to course creators. 

The previous article in the series was Kubernetes Deployments | My Public Notepad.

Networking within a single node

Let's first consider a single node Kubernetes cluster where node contains a single pod inside which runs a container.

The node has an IP address e.g. 192.168.1.2. We use it to access the Kubernetes node, SSH into it, etc.. 

In a Minikube setup, this is the IP address of the Minikube virtual machine inside the hypervisor. Our laptop's local network IP address might be 192.168.1.10. (...and VM is probably using bridged network adapter: VirtualBox Network Settings: All You Need to Know)

Unlike in the Docker world where an IP address is always assigned to a Docker container, in the Kubernetes world, the IP address is assigned to a pod. Each pod in the Kubernetes gets its own internal IP address. This IP address can be in the range e.g. 10.244.*.* series, and the IP assigned to the pod is e.g. 10.244.0.2.

So how is it getting this IP address?

When Kubernetes is initially configured, an internal private network gets created with the address e.g.  10.244.0.0 and all the pods are attached to it. When we deploy multiple pods, they all get a separate IP assigned from this network e.g.: 10.244.0.2, 10.244.0.3, 10.244.0.4....

The pods can communicate to each other through this IP, but accessing the other pods using this internal

IP address may not be a good idea as IP address can change when pods are recreated. There are better ways to establish communication between pods

Cluster Networking (multiple nodes)

Let's assume we have two nodes running Kubernetes and they have IP addresses 192.168.1.2 and 192.168.1.3 assigned to them. They are not part of the cluster yet.

Each of them has a single pod deployed as discussed in the previous section above. These pods are attached to an internal network (each own internal network, withing their host node) and they have their own IP addresses assigned. As their internal networks are independent, both networks might be 10.244.0.0 and both pods might end up having the same IP address assigned e.g. 10.244.0.2.

This would not be working well if nodes would be part of the same cluster as the pods have the same IP addresses assigned to them and that would lead to IP conflicts in the network.

When a Kubernetes cluster is set up, Kubernetes does not automatically set up any kind of networking to handle these issues. As a matter of fact, Kubernetes expects us to set up networking to meet certain fundamental requirements. Some of these are that:

•  All the containers or pods in a Kubernetes cluster must be able to communicate with one another without having to configure NAT
• All nodes must be able to communicate with containers and all containers must be able to communicate with the nodes in the cluster without NAT

Kubernetes expects us to set up a networking solution that meets these criteria. Fortunately, we don't have to set it up all on our own as there are multiple pre-built solutions available. Some of them are:

• Cisco ACI Networks [Cisco ACI - Application Centric Infrastructure - Cisco]
• Cilium [Cilium - Cloud Native, eBPF-based Networking, Observability, and Security]
• Big Cloud Fabric [Big Cloud Fabric - Red Hat Ecosystem Catalog]
• Flannel [flannel-io/flannel: flannel is a network fabric for containers, designed for Kubernetes]
• VMware [VMware Networking Solutions]
• Zest
• Calico [About Calico | Calico Documentation]
• Weave Net [weaveworks/weave: Simple, resilient multi-host containers networking and more.]

The choice depends on the platform we're deploying our Kubernetes cluster on. For example, if we are setting up a Kubernetes cluster from scratch on our own systems we may use any of the solutions like Calico or Flannel, etc. If we were deploying on a VMware environment, NSX may be a good option. 

In case of our cluster with the custom networking either Flannel or Calico setup could be used. It would manage the networks and IPs in our nodes and assign a different network address for each network in its node. For example, one node would have network 10.244.0.0 (and one pod in it might have IP 10.244.0.2) while another node might get network 10.244.1.0 (and one pod in it might hav IP 10.244.1.2). 

These networking solutions provision a routing between nodes' internal networks. This creates a virtual network of all pods and nodes where they are all assigned a unique IP address. By using simple routing techniques, the cluster networking enables communication between the different pods or nodes to meet the networking requirements of Kubernetes. Thus, all the pods now can communicate to each other using the assigned IP address.

---