Horizontal Pod Autoscaler (HPA)

 

The Horizontal Pod Autoscaler (HPA) serves to automatically align your application's capacity with its real-time demand by adjusting the number of pod replicas. Its operation depends on several critical components and configurations within an EKS or Kubernetes cluster. 

Kubernetes Horizontal Pod Autoscaler (HPA) is a built-in Kubernetes controller

• Standard Kubernetes autoscaling mechanism
• HPA API is available out of the box. It is a part of the Core Kubernetes and does not need installing third-party controllers or addons.
• But it is NOT fully operational "by default" in a standard Amazon EKS cluster. API definitions for HPA resources exist within Kubernetes, but they require a Metrics Server to function—which AWS does not install for you automatically during cluster creation.

It's "standard" in the sense that the feature is built into the Kubernetes control plane, but it isn't "automatic" in the sense that it guesses which of your apps need scaling.

Think of it like a Thermostat: The thermostat (HPA Controller) is already installed on the wall (EKS Control Plane), but it won't turn on the AC until you tell it what the Target Temperature (CPU/Memory threshold) is and which room (Deployment) to monitor.

Here is why a manifest is required for every app:

1. The Controller vs. The Resource

The Controller (The "How"): This is a loop running inside the EKS Control Plane. It is always active, waiting for instructions. Kubernetes HPA Documentation explains this loop.

The Resource (The "What"): The HPA Manifest is that instruction. It tells the controller: "Watch Deployment X, keep CPU at 50%, and don't go above 10 pods."

2. Manual Intent

Kubernetes follows a Declarative Model. It never assumes you want to scale. If it scaled every pod automatically, a single bug in your code (like an infinite loop) could scale your cluster to 1,000 nodes and drain your AWS budget instantly. You must explicitly opt-in by creating the HPA resource.

3. Unique Criteria for Every App

Not all apps scale the same way:

• Web API: Might scale when CPU hits 70%.
• Background Worker: Might scale based on Memory usage.
• Data Processor: Might scale based on a Custom Metric like SQS queue depth.

Summary: What is "Standard"?

What is standard is the API definition and the Controller. What is not standard is your specific application's scaling logic.

To see what the HPA Controller is looking for, you can check your Deployment's resource requests via kubectl:

kubectl get deployment <name> -o yaml | grep resources -A 5

Key Features:

• Pod Scaling: Adjusts the number of pod replicas to match the demand.
• Automatically scales up/down the number of pods in a deployment, replication controller, or replica set based on observed CPU utilization, memory or other selected custom/external metrics.

Purpose

• Dynamic Scalability: Automatically adds pods during traffic surges to maintain performance and removes them during low-traffic periods to reduce waste.
• Cost Optimisation: Ensures you only pay for the compute resources currently needed rather than over-provisioning for peak loads.
• Resilience & Availability: Prevents application crashes and outages by proactively scaling out before resources are fully exhausted.
• Operational Efficiency: Replaces manual intervention with "architectural definition," allowing infrastructure to manage itself based on predefined performance rules. 

Dependencies

HPA cannot function on its own; it requires the following "links" and infrastructure: 

• Metrics Server (The Aggregator): This is the most critical infrastructure dependency. The HPA controller queries the Metrics API (typically provided by the Metrics Server) to get real-time CPU and memory usage data.
• Both HPA and VPA rely on the metrics.k8s.io API to retrieve CPU and memory data. Because EKS is a managed control plane, AWS keeps it "lean" by leaving the choice of metrics provider to you.
• HPA: Without Metrics Server, HPA will show a status of <unknown> for its targets.
• VPA: Without Metrics Server, the VPA Recommender cannot analyze resource usage to suggest changes
• Resource Requests (The Baseline): For the HPA to calculate percentage-based utilization (e.g., "scale at 50% CPU"), the target Deployment must have resources.requests defined. Without these, the HPA has no 100% baseline to measure against and will show an unknown status.
• Controller Manager: The HPA logic runs as a control loop within the Kubernetes kube-controller-manager, which periodically (every 15 seconds by default) evaluates the metrics and updates the desired replica count.
• Scalable Target: The HPA must be linked to a resource that supports scaling, such as a Deployment, ReplicaSet, or StatefulSet.
• Cluster Capacity (Node Scaling): While HPA scales pods, it depends on an underlying node scaler (like Karpenter or Cluster Autoscaler) to provide new EC2 instances if the cluster runs out of physical space for the additional pods. 

Installation and Configuration

To use HPA ensure the Metrics Server is installed in your cluster to provide resource metrics.

kubectl apply -f https://github.com/kubernetes-sigs/metrics-server/releases/latest/download/components.yaml

Once the Metrics Server is installed, HPA is ready to go. It is a core Kubernetes controller, so you don't need to install any additional software beyond the metrics provider. We can simply create an HPA resource for our deployment:

kubectl autoscale deployment our-deployment \

    --cpu=50% \

    --min=1 \

    --max=10

We can also use the manifest:

apiVersion: autoscaling/v2

kind: HorizontalPodAutoscaler

metadata:

  name: my-app-hpa

spec:

  scaleTargetRef:

    apiVersion: apps/v1

    kind: Deployment

    name: my-app

  minReplicas: 2

  maxReplicas: 10

  metrics:

  - type: Resource

    resource:

      name: cpu

      target:

        type: Utilization

        averageUtilization: 50

How to check if HPA is enabled in the cluster?

% kubectl api-resources -o wide | grep autoscaling

NAME SHORTNAMES APIVERSION NAMESPACED KIND

VERBS CATEGORIES

...

horizontalpodautoscalers  hpa  autoscaling/v2 true HorizontalPodAutoscaler  create,delete,deletecollection,get,list,patch,update,watch all

In which namespace do HorizontalPodAutoscalers reside in?

In AWS EKS, HorizontalPodAutoscalers (HPA) are namespaced resources, meaning they belong in the same namespace as the workload (e.g., Deployment or StatefulSet) they are intended to scale. 

While there is no single "HPA namespace," here is how they are distributed and where related components live:

1. The HPA Resource Namespace 

• Application Namespace: When you create an HPA, you define it within the specific namespace where your application is running (e.g., default, production, or demo).
• Constraint: An HPA can only scale a target resource (like a Deployment) that exists in that same namespace. 

2. Infrastructure & Metrics Namespaces 

While the HPA resource lives with your app, the supporting infrastructure often resides in system namespaces: 

• Metrics Server: This is a mandatory prerequisite for HPA on EKS. It is typically deployed in the kube-system namespace.
• Custom Metrics Adapters: If you are scaling based on custom metrics (like Prometheus or CloudWatch), components like the prometheus-adapter or k8s-cloudwatch-adapter may be installed in kube-system or a dedicated namespace like custom-metrics.
• Cluster Autoscaler: Often confused with HPA, the Cluster Autoscaler (which scales EC2 nodes rather than pods) also typically resides in the kube-system namespace. 

To find all HPAs across your entire EKS cluster, you can run:

kubectl get hpa -A

We might have an output like this:

% kubectl get horizontalpodautoscalers -A    

No resources found

It is possible to get "No resources found" for several reasons, despite the resource being namespaced. This usually means that while the API type exists, no actual instances of that resource have been created in your EKS cluster yet.

Why you see "No resources found":

• HPA is not yet created: By default, EKS clusters do not come with any HorizontalPodAutoscalers pre-configured. You must explicitly create one for your application.
• Metrics Server Missing: HPAs rely on the Kubernetes Metrics Server to function. While the HPA object can be created without it, it will show a status of <unknown> and may not appear if you are looking for active scaling.
• Namespace Context: Even with -A (all namespaces), if no user or system service has defined an HPA resource, the list will be empty. 

How to Verify and Fix:

• Check if Metrics Server is running:
• Run kubectl get deployment metrics-server -n kube-system. If it’s missing, you can install it via the AWS EKS Add-ons in the console or via kubectl apply.

        kubectl get all -A | grep metrics-server 

• Check API availability:
• Run kubectl api-resources | grep hpa to confirm the cluster recognizes the resource type.
• Create a test HPA:
• If you have a deployment named my-app, try creating one: 

        kubectl autoscale deployment my-app \

            --cpu=50% \

            --min=1 \

            --max=10

Note: If you are using a newer version of EKS (like 1.31) with Auto Mode, some autoscaling is handled automatically by the control plane, but standard HPAs still need to be manually defined if you want pod-level scaling based on custom metrics. 

% kubectl get all -A | grep metrics-server 

default pod/metrics-server-5db5f64c66-sjd2p        1/1     Running     0          205d

default service/metrics-server  ClusterIP       172.21.76.224    <none>  443/TCP  95d

default deployment.apps/metrics-server             1/1     1            1         295d

default replicaset.apps/metrics-server-5db5f64c66   1         1         1         295d

This behavior occurs because no instances of HorizontalPodAutoscaler (HPA) have been created yet, even though the supporting infrastructure (Metrics Server) and API are active. 

In Kubernetes, the presence of the metrics-server and the autoscaling/v2 API resource does not mean an HPA is automatically running for your apps. You must manually define an HPA for each deployment you want to scale. 

Why kubectl get hpa -A is empty

• Workloads are not yet auto-scaled: By default, EKS (and Kubernetes) does not apply HPAs to your deployments. You must explicitly create an HPA object that references your target Deployment or StatefulSet.
• kubectl get all exclusion: Standard kubectl get all does not include HPAs in its output, which is why your previous command didn't show them even if they existed.
• Namespace Location: While your metrics-server is in the default namespace (though typically it's in kube-system), HPAs must be created in the same namespace as the app they are scaling. 

How to create your first HPA

If you have a deployment (e.g., named my-deployment) in the default namespace, you can create an HPA for it using this command:

kubectl autoscale deployment my-deployment \

    --cpu=60%

    --min=1 \

    --max=10

--cpu string

Target CPU utilization over all the pods. When specified as a percentage (e.g."70%" for 70% of requested CPU) it will target average utilization. When specified as quantity (e.g."500m" for 500 milliCPU) it will target average value. Value without units is treated as a quantity with miliCPU being the unit (e.g."500" is "500m").

--memory string

Target memory utilization over all the pods. When specified as a percentage (e.g."60%" for 60% of requested memory) it will target average utilization. When specified as quantity (e.g."200Mi" for 200 MiB, "1Gi" for 1 GiB) it will target average value. Value without units is treated as a quantity with mebibytes being the unit (e.g."200" is "200Mi").