AWS EKS Cluster Networking

If we select a cluster and go to Networking tab, we'll see the following settings: 

• VPC
• Cluster IP address family
• Service IPv4 range
• Subnets
• Cluster security group
• Additional security groups
• API server endpoint access

Manage drop down groups them into following:

• VPC Resources (Network environment)
• Subnets
• Additional security groups - optional
• Endpoint access (API server endpoint access)
• Remote networks

We'll describe here the meaning and purpose for each of them.

VPC

Amazon Virtual Private Cloud (Amazon VPC) enables you to launch AWS resources into a virtual network that you have defined. This virtual network closely resembles a traditional network that you would operate in your own data center, with the benefits of using the scalable infrastructure of AWS. 

A virtual private cloud (VPC) is a virtual network dedicated to your AWS account. A subnet is a range of IP addresses in your VPC. 

Each Managed Node Group requires you to specify one of more subnets that are defined within the VPC used by the Amazon EKS cluster. Nodes are launched into subnets that you provide. The size of your subnets determines the number of nodes and pods that you can run within them. 

You can run nodes across multiple AWS availability zones by providing multiple subnets that are each associated different availability zones. Nodes are distributed evenly across all of the designated Availability Zones. 

If you are using the Kubernetes Cluster Autoscaler and running stateful pods, you should create one Node Group for each availability zone using a single subnet and enable the -\-balance-similar-node-groups feature in cluster autoscaler.

EKS suggests using Private Subnets for worker nodes.

Cluster IP address family

Select the IP address type that pods and services in your cluster will receive: IPv4 or IPv6.

Amazon EKS does not support dual stack clusters. However, if you worker nodes contain an IPv4 address, Amazon EKS will configure IPv6 pod routing so that pods can communicate with cluster external IPv4 endpoints.

Service IPv4 range

The IP address range from which cluster services will receive IP addresses. Manually configuring this range can help prevent conflicts between Kubernetes services and other networks peered or connected to your VPC.

Service CIDR is only configurable when choosing IPv4 as your cluster IP address family. With IPv6, the service CIDR will be an auto generated unique local address (ULA) range.

Subnets

Choose the subnets in your VPC where the control plane may place elastic network interfaces (ENIs) to facilitate communication with your cluster. The specified subnets must span at least two availability zones.

To control exactly where the ENIs will be placed, specify only two subnets, each from a different AZ, and Amazon EKS will make cross-account ENIs in those subnets. The Amazon EKS control plane creates up to 4 cross-account ENIs in your VPC for each cluster.

You may choose one set of subnets for the control plane that are specified as part of cluster creation, and a different set of subnets for the worker nodes.

EKS suggests using private subnets for worker nodes.

If you select IPv6 cluster address family, the subnets specified as part of cluster creation must contain an IPv6 CIDR block.

Cluster security group & Additional security groups

Amazon VPC Security groups control communications within the Amazon EKS cluster including between the managed Kubernetes control plane and compute resources in your AWS account such as worker nodes and Fargate pods.

The Cluster Security Group is a unified security group that is used to control communications between the Kubernetes control plane and compute resources on the cluster. The cluster security group is applied by default to the Kubernetes control plane managed by Amazon EKS as well as any managed compute resources created by Amazon EKS. 

EKS automatically creates a cluster security group on cluster creation to facilitate communication between worker nodes and control plane. Description of such SG is: EKS created security group applied to ENI that is attached to EKS Control Plane master nodes, as well as any managed workloads. The name of this SG is in form: eks-cluster-sg-<cluster_name>-1234567890. It's attached to the same VPC that cluster is in. Its rules are:

• Inbound: allow all traffic (all protocols and ports) from itself (see https://stackoverflow.com/questions/66917854/aws-security-group-source-of-inbound-rule-same-as-security-group-name)
• Outbound: allow all IPv4 and IPv6 traffic

Optionally, choose additional security groups to apply to the EKS-managed Elastic Network Interfaces that are created in your control plane subnets. To create a new security group, go to the corresponding page in the VPC console.

Additional cluster security groups control communications from the Kubernetes control plane to compute resources in your account. Worker node security groups are security groups applied to unmanaged worker nodes that control communications from worker nodes to the Kubernetes control plane.

You can apply additional cluster security groups to control communications from the Kubernetes control plane to compute resources in your account.

Worker node security groups are security groups applied to unmanaged worker nodes that control communications from worker nodes to the Kubernetes control plane.

Example:

• Description: EKS cluster security group
• Inbound rules:
• IP version: IPv4
• Type: HTTPS 
• Protocol: TCP
• Port range: 443
• Source: 192.168.1.0/24
• Description: Office LAN CIDR (for acccess via Site-to-site VPN)

API server endpoint access

You can limit, or completely disable, public access from the internet to your Kubernetes cluster endpoint.

Amazon Amazon EKS creates an endpoint for the managed Kubernetes API server that you use to communicate with your cluster (using Kubernetes management tools such as kubectl). By default, this API server endpoint is public to the internet, and access to the API server is secured using a combination of AWS Identity and Access Management (IAM) and native Kubernetes Role Based Access Control (RBAC).

You can, optionally, limit the CIDR blocks that can access the public endpoint. If you limit access to specific CIDR blocks, then it is recommended that you also enable the private endpoint, or ensure that the CIDR blocks that you specify include the addresses that worker nodes and Fargate pods (if you use them) access the public endpoint from.

You can enable private access to the Kubernetes API server so that all communication between your worker nodes and the API server stays within your VPC. You can limit the IP addresses that can access your API server from the internet, or completely disable internet access to the API server.

Cluster endpoint access Info

Configure access to the Kubernetes API server endpoint.

• Public - The cluster endpoint is accessible from outside of your VPC. Worker node traffic will leave your VPC to connect to the endpoint.
• Public and private - The cluster endpoint is accessible from outside of your VPC. Worker node traffic to the endpoint will stay within your VPC.
• Private - The cluster endpoint is only accessible through your VPC. Worker node traffic to the endpoint will stay within your VPC.

If we choose Public or Public and private we get Advanced settings with option to Add/edit sources to public access endpoint. We can add here up to 40 CIDR blocks

Public access endpoint sources - Determines the traffic that can reach the Kubernetes API endpoint of this cluster.

Use CIDR notation to specify an IP address range (for example, 203.0.113.5/32).

If connecting from behind a firewall, you'll need the IP address range used by the client computers.

By default, your public endpoint is accessible from anywhere on the internet (0.0.0.0/0).

If you restrict access to your public endpoint using CIDR blocks, it is strongly recommended to also enable private endpoint access so worker nodes and/or Fargate pods can communicate with the cluster. Without the private endpoint enabled, your public access endpoint CIDR sources must include the egress sources from your VPC. For example, if you have a worker node in a private subnet that communicates to the internet through a NAT Gateway, you will need to add the outbound IP address of the NAT Gateway as part of a allowlisted CIDR block on your public endpoint.

---