Monday, 29 July 2024

Kubernetes Ingress Service


Ingress is a more flexible and powerful solution for managing external access to services within a Kubernetes cluster than Kubernetes LoadBalancer Service

It provides:
  • load balancing
  • SSL termination
  • name-based virtual hosting

Ingress controllers can be configured to handle traffic more efficiently and securely.

A full Ingress deployment manifest in Kubernetes defines how external traffic is routed to services within the cluster. It uses an Ingress controller (like NGINX) to manage the rules, and the manifest specifies which services should be exposed, based on hostnames, paths, or other criteria. The manifest typically includes details about the Ingress class, rules, and the services it exposes. 

Manifest example:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: my-ingress
  namespace: default
spec:
  ingressClassName: nginx # Or gce, aws, etc.
  rules:
  - host: example.com # or myapp.example.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: my-service
            port:
              number: 80
  - host: myapp.example.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: my-service
            port:
              number: 80
  - host: myapp2.example.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: my-service2
            port:
              number: 80

Here is the meaning of all fields in the manifest:
  • apiVersion: Kubernetes API version
  • kind: Kubernetes object type (Ingress)
  • metadata: Metadata about the Ingress, including its name, namespace, and labels. 
  • spec: The heart of the manifest, defining the Ingress rules and behavior. 
    • ingressClassName: Specifies the Ingress controller to use (e.g., nginx, gce, aws).
    • rules: An array of rules that define how traffic should be routed. Each rule includes: 
      • host: The hostname to match (e.g., example.com). 
      • http: A section defining HTTP traffic routing.
        • paths: An array of paths to match within the HTTP request.
          • backend: The service to forward traffic to based on the path.
            • serviceName: The name of the service.
            • servicePort: The port of the service.



host

Removing the host (host-specific rules) from the ingress will make it accept traffic for any hostname, potentially exposing the service more broadly than intended. These changes could lead to unintended access to services if the ALB receives requests with different or no host headers.

serviceName

What is this service and what are its requirements so it can serve the traffic?


In Kubernetes (K8s), an Ingress manifest by itself does not directly create an AWS load balancer — but it can indirectly result in one being created depending on how your cluster is configured.

Ingress is just a K8s object that defines HTTP routing rules (e.g., "requests to /foo go to Service A"). It requires an Ingress Controller to actually implement those rules and expose the traffic. If we just apply an Ingress manifest and no Ingress Controller is installed, nothing will happen — no load balancer will be created.

To get an AWS Load Balancer via Ingress we need to install and use an Ingress Controller that integrates with AWS, such as:

  • AWS ALB Ingress Controller (now called AWS Load Balancer Controller)
  • Nginx Ingress Controller (but then you'd still need a Service of type LoadBalancer to expose it)

If we're using the AWS Load Balancer Controller and define an Ingress object with proper annotations, then it will create an AWS Application Load Balancer (ALB) and configure it with listeners and target groups according to the Ingress rules. This is the most direct way to have an Ingress lead to the creation of an AWS load balancer.

NOTE: LB creation takes some time e.g. 30 seconds.


Example: Ingress manifest that works with AWS Load Balancer Controller

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: example-ingress
  namespace: default
  annotations:
    alb.ingress.kubernetes.io/scheme: internet-facing
    alb.ingress.kubernetes.io/target-type: ip
    alb.ingress.kubernetes.io/listen-ports: '[{"HTTP": 80}]'
spec:
  ingressClassName: alb
  rules:
    - http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: my-service
                port:
                  number: 80


Annotations

For full list of AWS ALB annotations, see Annotations - AWS Load Balancer Controller.

alb.ingress.kubernetes.io/scheme


The annotation alb.ingress.kubernetes.io/scheme in AWS Load Balancer Controller specifies the visibility and network exposure of the Application Load Balancer (ALB) created for your Kubernetes Ingress.

Possible values:
  • internet-facing
    • ALB is publicly accessible over the internet.
    • Use for services that need to be accessed publicly — e.g., websites, APIs, dashboards
  • internal
    • ALB is private, only accessible within your VPC (e.g., internal apps).
    • Use for internal microservices, admin interfaces, or when you only want access within a private network (e.g., from a VPN or other VPC resources).
Example (internet facing):

metadata:
  name: my-ingress
  annotations:
    alb.ingress.kubernetes.io/scheme: internet-facing


This will create a public-facing ALB with a public DNS name like:

my-ingress-1234567890.us-west-2.elb.amazonaws.com


The subnets in your cluster must be correctly tagged:
  • For internet-facing: subnets must be public and tagged with: kubernetes.io/role/elb = 1
  • For internal: subnets must be private and tagged with: kubernetes.io/role/internal-elb = 1
If you don't set the scheme explicitly, it defaults to internet-facing.


Example: Internal Ingress 

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: internal-ingress
  namespace: default
  annotations:
    kubernetes.io/ingress.class: alb
    alb.ingress.kubernetes.io/scheme: internal
    alb.ingress.kubernetes.io/subnets: subnet-abc123,subnet-def456  # Optional: for precise control
    alb.ingress.kubernetes.io/group.name: internal-apps
    alb.ingress.kubernetes.io/tags: "env=dev,app=my-internal-service"
spec:
  rules:
    - http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: my-service
                port:
                  number: 80


This configuration instructs the AWS Load Balancer Controller to:
  • Create an internal ALB (i.e., not exposed to the public internet).
  • Place the ALB in private subnets (must be tagged: kubernetes.io/role/internal-elb = 1).
  • The ALB is reachable only within the VPC or from connected networks (like via VPN or Direct Connect).

The ALB will get a DNS name that looks like this:

internal-k8s-<name>-<hash>.<region>.elb.amazonaws.com

e.g.

internal-k8s-internalingr-abc123456789.us-west-2.elb.amazonaws.com

The internal- prefix on the DNS name indicates it is not public. It will not resolve outside the VPC (e.g., from your home network or the public internet).

We can verify the internal ALB:

  • AWS Console:
            Go to EC2 → Load Balancers → Look at the Scheme column (internal).
  • CLI:
     aws elbv2 describe-load-balancers \
         --names internal-k8s-internalingr-abc123456789


alb.ingress.kubernetes.io/listen-ports


The annotation alb.ingress.kubernetes.io/listen-ports in AWS Load Balancer Controller is used to customize the listener ports that are created on the Application Load Balancer (ALB) for a Kubernetes Ingress resource.

By default, the AWS Load Balancer Controller creates listeners for port 80 (HTTP) and/or port 443 (HTTPS) depending on your Ingress TLS settings.
If you need to override this behavior—for example, to support only HTTPS or use non-default ports—you can use this annotation.

Example: Default Behavior (HTTP and HTTPS):

alb.ingress.kubernetes.io/listen-ports: '[{"HTTP": 80}, {"HTTPS": 443}]'


Creates both:
  • an HTTP listener on port 80
  • an HTTPS listener on port 443 (requires TLS configuration)

It must be a JSON array of objects, where each object specifies a protocol and port.


Example: HTTPS Only (no HTTP listener):

alb.ingress.kubernetes.io/listen-ports: '[{"HTTPS": 443}]'


Useful if you want to force HTTPS and avoid exposing port 80. Make sure you also configure tls: section in the Ingress and associate an ACM certificate using:

alb.ingress.kubernetes.io/certificate-arn


Example: Custom Ports (e.g., HTTP on 8080):

alb.ingress.kubernetes.io/listen-ports: '[{"HTTP": 8080}]'

ALB will listen on port 8080 for HTTP traffic instead of the default 80.

We can only use ports supported by ALB, which are typically: 1–65535, but port 80 and 443 are standard and expected by most clients.. Custom ports may not work well unless you control the client environment.


Example: Internal Service with Only HTTPS

annotations:
  alb.ingress.kubernetes.io/scheme: internal
  alb.ingress.kubernetes.io/listen-ports: '[{"HTTPS": 443}]'
  alb.ingress.kubernetes.io/certificate-arn: arn:aws:acm:...

If we specify only HTTPS, you must provide a valid certificate via alb.ingress.kubernetes.io/certificate-arn.

If we omit this annotation, the controller decides based on TLS block:
  • If TLS is configured, both HTTP and HTTPS are created
  • If not, only HTTP on port 80 is created



alb.ingress.kubernetes.io/tags


Specifies which tags will be applied onto Application Load Balancer that will be created by LB Controller. Example:

alb.ingress.kubernetes.io/tags: "kubernetes.io/ingress-name=${local.ingress_name},Environment=${local.workspace},Team=${local.team}"

This means that LB will have these tags:
  • kubernetes.io/ingress-name, set to value ${local.ingress_name}
  • Environment set to value ${local.workspace}
  • Team, set to value ${local.team}

Note that by default, AWS LB Controller applies the following tags:
  • ingress.k8s.aws/resource = LoadBalancer
  • ingress.k8s.aws/stack = <namespace>/<ingress_name>



alb.ingress.kubernetes.io/group.name

One of the possible annotations is alb.ingress.kubernetes.io/group.name. It is used in AWS Load Balancer Controller. It specifies the target group name for grouping multiple Ingress resources under a single Application Load Balancer (ALB). 

By default, each Ingress resource gets its own ALB, which can be costly or unnecessary. Setting the alb.ingress.kubernetes.io/group.name annotation allows you to share a single ALB across multiple Ingress resources.

Example:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: my-ingress
  annotations:
    alb.ingress.kubernetes.io/group.name: my-apps
    alb.ingress.kubernetes.io/scheme: internet-facing
spec:
  rules:
    - host: app1.example.com
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: app1-service
                port:
                  number: 80


If we have another Ingress resource with the same group.name, it will be attached to the same ALB.

Benefits of using group.name:
  • Cost efficiency: Fewer ALBs to manage and pay for.
  • Simplified architecture: Group related services behind one ALB.
  • Custom routing: Combine multiple paths or hostnames on one load balancer.

All Ingresses in the same group must share the same ALB settings, such as:
  • alb.ingress.kubernetes.io/scheme (e.g. internet-facing or internal)
  • alb.ingress.kubernetes.io/load-balancer-name (optional but can be used)

The ALB is shared, but each rule/path is managed separately based on the Ingress definitions.



external-dns.alpha.kubernetes.io/ingress-hostname-source


 annotations-only: This annotation configures External DNS to only use explicit hostname annotations when determining the DNS records to create


When do we need to use Ingress?


Whether you need one depends entirely on who needs to reach the service/application.

We DO NOT need an Ingress (most common) if:
  • Service runs inside the cluster
  • Service clients run inside the cluster
  • Everything talks via Kubernetes service DNS e.g. http://tempo.grafana-tempo.svc.cluster.local

Then Service should stay ClusterIP only.

Why We Generally Should NOT Expose Application/Service?

If application/service is:
  • Not an end-user UI
  • Not meant for browser access
  • Not something external systems should hit directly
  • Not something we want publicly reachable
Exposing it increases attack surface unnecessarily.

We only need an Ingress if:

1) External write clients send requests directly to cluster

Example:
  • Apps outside cluster
  • On-prem services
  • External environments

Then they need: https://my-app.company.com


In that case, yes — create an Ingress or LoadBalancer.

2) External read clients runs outside the cluster

If App UI is external and querying App inside Kubernetes,
then we must expose App via:
  • Ingress
  • LoadBalancer
  • Private networking

Example: Grafana backend service has this URL:  http://tempo.grafana-tempo.svc.cluster.local:3200

That is:
  • Internal Kubernetes DNS
  • ClusterIP service
  • Port 3200 (Tempo HTTP API)
  • Namespace: grafana-tempo

So our Grafana Tempo is internal-only, and being queried from inside the cluster.

That means: we do NOT need an Ingress.
---


How can multiple ingresses share the same ALB?                                                                                                                                                

It's a feature of the AWS Load Balancer Controller called IngressGroup.
  
Default: one Ingress = one ALB

Out of the box, the controller provisions a dedicated ALB per Ingress resource. That's wasteful — an ALB is ~$16/mo + LCUs, and you'd get one per service.

IngressGroup: the group.name annotation


Ingresses that declare the same alb.ingress.kubernetes.io/group.name are merged by the controller into a single ALB:

# staging-ingress      → group.name: example-staging-default-k8s
# service3-staging     → group.name: example-staging-default-k8s   ← same group

The controller watches all Ingresses, buckets them by group.name, and for each bucket provisions one ALB whose listener rules are the union of every member Ingress's rules. That's exactly why, after our apply, kubectl get ingress -n staging showed both service3-staging-ingress and staging-ingress with the same ADDRESS (k8s-examplestagingdefau-74e6ea742a-…) — same ALB.

How routing works on the shared ALB


- Each backend service referenced by any member Ingress gets its own target group.
- Each host/path rule becomes a listener rule on the shared listener (443), with a host condition forwarding to that service's target group:

  service1.staging.example.com            → service1-api TG      (instance / round_robin)
  service2.staging.example.com            → service2-api TG      (instance / round_robin)
  service3.staging.example.com            → service3 TG            (ip / least_outstanding_requests)

- DNS: external-dns points every host at the one shared ALB DNS name.

What's shared vs what's per-Ingress — the key to our split


This is the crucial distinction that let us change only service3:
  • Scope: ALB-level (shared, must be consistent across the group) 
  • Examples: scheme, listener ports, certificates, security groups, WAF ACL, access-logs/idle-timeout, subnets
  • In our change: Replicated identically on the new Ingress so the group stays consistent (conflicting values make the controller error)   
  • Scope: Per-Ingress → per-target-group (can differ)
  • Examples: target-type (instance/ip), target-group-attributes (LOR/round_robin, stickiness), healthcheck, backend-protocol   
  • In our change: service3 TG gets ip + LOR; the other services' TGs stay instance + round_robin — same ALB

So the ALB is one shared object, but each service's target group is independent — which is precisely how we gave service3 ip-mode + least_outstanding_requests without touching the other 5 services. The controller confirmed it merged cleanly with SuccessfullyReconciled and no annotation conflict.

Why do it (and the trade-off you already met)


Benefits: one ALB instead of N → big cost saving, one DNS/cert/WAF/log stream to manage, consolidated metrics.

Trade-off: a shared ALB means shared metrics and a shared alarm. The target-response-time alarm is ALB-wide (request-weighted across all 6 services), so service3's latency drags the shared average and trips the alarm even though the other 5 are healthy. A dedicated ALB would isolate that — but as we discussed, it costs ~$25–35/mo and doesn't fix the latency, so we kept the shared ALB and fixed the routing instead.

Two operational notes

  • Rule ordering: when member Ingresses could have overlapping host/path rules, alb.ingress.kubernetes.io/group.order (an integer, lower = higher priority) controls listener-rule precedence. For distinct hosts (our case) there's no conflict, so we didn't need it.
  • ALB lifecycle is tied to the group, not any one Ingress: removing an Ingress just drops its rules/TG from the shared ALB (what we did — service3 moved from the shared Ingress's rules to its own); the ALB is only deprovisioned when the last member Ingress in the group is deleted.


Does k8s ingress gives us a fine grained control over alb target groups?


Yes, exactly. The Kubernetes Ingress resource—driven by the AWS Load Balancer Controller—provides immense control over underlying target group attributes natively through its annotation ecosystem. 

However, it is important to distinguish where the boundaries of this control lie and when a TargetGroupBinding custom resource (CRD) becomes the better choice.

What Ingress Can Control Natively


By using the alb.ingress.kubernetes.io/target-group-attributes annotation, you can pass comma-separated strings to adjust almost any native AWS Target Group feature directly from your YAML:
  • Routing Algorithms: Forcing load_balancing.algorithm.type=least_outstanding_requests or changing the algorithm on the fly.
  • Connection Draining: Adjusting deregulation delays via deregistration_delay.timeout_seconds=30.
  • Sticky Sessions: Configuring cookie-based persistence using stickiness.enabled=true,stickiness.type=lb_cookie.
  • Slow Start Mode: Gracefully warming up targets with slow_start.duration_seconds=30.

The "Catch" with Ingress-Driven Configuration


While highly granular, configuring everything inside the Ingress manifest ties the lifecycle of the AWS infrastructure strictly to the lifecycle of your Kubernetes objects.

  • Destructive Updates: If someone deletes or misconfigures the Ingress manifest, the AWS Load Balancer Controller will actively delete or alter the actual AWS Target Groups and listeners.
  • Monolithic Manifests: If you have dozens of backend services, your Ingress file becomes a massive block of text crammed with annotations, introducing formatting risk.
  • Blast Radius (Shared ALBs): If you use group.name to combine multiple Ingresses onto a single ALB, a bad change by one team can inadvertently overwrite shared settings or cause priority conflicts.

Moving to Ultimate Control: TargetGroupBinding

If your platform requires strict infrastructure isolation, you should decouple the load balancer from the Ingress object entirely using the TargetGroupBinding custom resource.

With this pattern, your cloud/Terraform team provisions the ALB, listeners, and Target Groups natively inside AWS (applying Least Outstanding Requests, certificates, and strict IAM rules there). You then hand the Target Group ARN to the Kubernetes team.

apiVersion: elbv2.k8s.aws/v1beta1
kind: TargetGroupBinding
metadata:
  name: decoupled-app-binding
  namespace: production
spec:
  # The controller watches this and syncs your Pod IPs straight to this AWS resource
  targetGroupARN: arn:aws:elasticloadbalancing:us-east-1:123456789012:targetgroup/my-pre-made-tg/abcdef123456
  targetType: ip
  serviceRef:
    name: my-app-service
    port: 80


Why Use TargetGroupBinding?

  • Zero Lifecycle Risk: Deleting the Kubernetes deployment or service will never delete the actual AWS infrastructure. The controller only manages the IP target list.
  • Multi-Cluster Routing: You can apply the exact same TargetGroupBinding file in Cluster A and Cluster B, allowing a single AWS ALB target group to seamlessly balance traffic across two entirely separate Kubernetes clusters.
  • Clean GitOps separation: Network teams manage AWS infrastructure via Terraform, while application developers handle standard K8s objects without touching infrastructure code.

How to implement blue-green deployment routing using TargetGroupBindings?


To implement a blue-green deployment using TargetGroupBindings, you leverage the absolute decoupling of the data plane.

Instead of updating Kubernetes manifests or re-routing traffic inside the cluster, you provision two independent AWS Target Groups (Blue and Green) via Terraform or the AWS Console. Your active traffic flip happens at the AWS ALB Listener rule level, completely safely, while the Kubernetes controller handles updating the actual pod IPs in the background.

Here is the step-by-step architecture and implementation pattern.

Step 1: Deploy Two TargetGroupBindings in Kubernetes


You deploy both your Blue and Green application deployments and services concurrently. Then, you map each Kubernetes service directly to its dedicated AWS Target Group ARN using two separate TargetGroupBinding manifests.

apiVersion: elbv2.k8s.aws/v1beta1
kind: TargetGroupBinding
metadata:
  name: app-blue-binding
  namespace: production
spec:
  # Points to the BLUE target group in AWS
  targetGroupARN: arn:aws:elasticloadbalancing:us-east-1:123456789012:targetgroup/app-blue-tg/111111111111
  targetType: ip
  serviceRef:
    name: app-blue-service
    port: 80
---
apiVersion: elbv2.k8s.aws/v1beta1
kind: TargetGroupBinding
metadata:
  name: app-green-binding
  namespace: production
spec:
  # Points to the GREEN target group in AWS
  targetGroupARN: arn:aws:elasticloadbalancing:us-east-1:123456789012:targetgroup/app-green-tg/222222222222
  targetType: ip
  serviceRef:
    name: app-green-service
    port: 80


At this point, the AWS Load Balancer Controller populates both target groups with the direct IPs of your pods.  AWS handles health checks for both environments independently.

Step 2: The Infrastructure State (Steady State)


Your AWS Application Load Balancer listener rule is configured to route 100% of live traffic to the Blue target group.

AWS ALB Listener (Port 443) 
     │
     └───► [Rule 1]: Forward to:
              ├──► app-blue-tg  (Weight: 100) ◄─── LIVE TRAFFIC
              └──► app-green-tg (Weight: 0)   ◄─── IDLE (Deploying New Code)


Step 3: Executing the Canary/Weighted Shift

When you want to roll out a new version, you deploy it to your app-green pods in Kubernetes. Once you verify the Green pods are healthy inside Kubernetes, you update your AWS ALB Listener Rule (using Terraform, AWS CLI, or an external progressive delivery tool like Argo Rollouts).You can execute a 10% canary test purely at the AWS level:

AWS ALB Listener (Port 443)
     │
     └───► [Rule 1]: Forward to:
              ├──► app-blue-tg  (Weight: 90)
              └──► app-green-tg (Weight: 10)  ◄─── Testing new release


Because both target groups inherit your Least Outstanding Requests (LOR) setting independently, the ALB smoothly evaluates traffic distribution across the two pools based on active requests without crashing your new containers.


Step 4: The Final Promotion (The "Flip")

If metrics, error rates, and latencies remain healthy on the green environment, you update the AWS ALB listener weights one final time to shift all users over:

AWS ALB Listener (Port 443)
     │
     └───► [Rule 1]: Forward to:
              ├──► app-blue-tg  (Weight: 0)   ◄─── OLD VERSION (Hold for rollback)
              └──► app-green-tg (Weight: 100) ◄─── NEW LIVE PRODUCTION


Why this pattern beats traditional Kubernetes Ingress flips:Zero DNS or K8s API Lag: Modifying an AWS ALB listener rule happens almost instantly at the cloud provider level. You do not have to wait for Kubernetes API reconciliation loops or internal routing tables to update.Instant Rollback: If the new Green release starts throwing 500 errors, you instantly change the AWS ALB weight back to app-blue-tg: 100. The rollback takes seconds and drops zero active connections.True Out-of-Band Testing: You can create a private AWS ALB listener (e.g., port 8080) that routes 100% to app-green-tg. This allows QA teams to run end-to-end integration tests on the new infrastructure using the live load balancer before production traffic ever sees it.If you want, I can:Provide the Terraform code to manage these weighted AWS ALB listener rulesShow how Argo Rollouts automates this specific TargetGroupBinding logic


Here is the complete architectural layout showing how requests flow through your infrastructure, alongside how the AWS Load Balancer Controller handles control plane configurations behind the scenes.


┌────────────────────────────────────────────────────────────────────────┐
│                          1. REQUEST DATA PLANE                         │
└────────────────────────────────────────────────────────────────────────┘

    [ External Clients ]               [ Internal Clients (VPC/VPN) ]
             │                                        │
             │ (HTTPS Request)                        │ (HTTP/HTTPS Request)
             ▼                                        ▼
   ┌──────────────────────────────────────────────────────────────────────┐
   │ ──► [ Route 53 DNS Lookup ] ──────────────────────────────────────── │
   │     Resolves domain name to ALB DNS name / IP addresses              │
   └──────────────────────────────────────────────────────────────────────┘
                                      │
                                      ▼
   ┌──────────────────────────────────────────────────────────────────────┐
   │ ──► [ AWS Application Load Balancer (ALB) ] ──────────────────────── │
   │     - Terminates TLS/SSL                                             │
   │     - Evaluates Listener Rules (Weights, Paths, Host headers)        │
   └──────────────────────────────────────────────────────────────────────┘
                                      │
                 ┌────────────────────┴────────────────────┐
                 │ (Routes based on Rule Weights / LOR)    │
                 ▼                                         ▼
   ┌──────────────────────────────┐          ┌──────────────────────────────┐
   │ [ AWS Target Group: BLUE ]   │          │ [ AWS Target Group: GREEN ]  │
   │ (Algorithm: LOR)             │          │ (Algorithm: LOR)             │
   └──────────────────────────────┘          └──────────────────────────────┘
                 │                                         │
                 │ (Direct Pod IP Routing Bypass)          │
                 ▼                                         ▼
   ┌──────────────────────────────────────────────────────────────────────┐
   │ [ AWS EC2 Worker Nodes ]                                             │
   │                                                                      │
   │   Pod A (Blue Application)               Pod C (Green Application)   │
   │   [IP: 10.0.1.45] ◄─────────┐            [IP: 10.0.1.92] ◄────────┐  │
   │                             │                                     │  │
   │   Pod B (Blue Application)  │            Pod D (Green Application)│  │
   │   [IP: 10.0.2.11] ◄─────────┘            [IP: 10.0.2.84] ◄────────┘  │
   │                                                                      │
   └──────────────────────────────────────────────────────────────────────┘


┌────────────────────────────────────────────────────────────────────────┐
│                        2. K8s KINETIC CONTROL PLANE                    │
└────────────────────────────────────────────────────────────────────────┘

   ┌──────────────────────────────────────────────────────────────────────┐
   │ [ Kubernetes API Server ]                                            │
   │                                                                      │
   │   Manifests: [ Ingress / TargetGroupBinding ]                        │
   │       Contains configuration properties like:                        │
   │       - load_balancing.algorithm.type=least_outstanding_requests    │
   └──────────────────────────────────────────────────────────────────────┘
             ▲
             │ Watches resources for changes
             ▼
   ┌──────────────────────────────────────────────────────────────────────┐
   │ [ AWS Load Balancer Controller Pod ] (Runs inside EKS/EC2 cluster)   │
   └──────────────────────────────────────────────────────────────────────┘
             │
             │ Reconciles state via AWS APIs
             ▼
   ┌──────────────────────────────────────────────────────────────────────┐
   │ [ AWS Cloud Control Plane ]                                          │
   │ Modifies Listener Rules, Target Group Attributes, and Target IPs     │
   └──────────────────────────────────────────────────────────────────────┘


Architectural Highlights

Unified Entrance Network: Both External Clients coming from the public internet and Internal Clients originating from inside your corporate VPC use Amazon Route 53 to find the entry point. Route 53 directs them straight to the single AWS Application Load Balancer (ALB).Separation of Concerns: Notice that the Kubernetes Ingress / TargetGroupBinding manifests and the AWS Load Balancer Controller do not handle actual application traffic. They operate strictly on the Control Plane to build and update the AWS components.Bypassing Node-Port Constraints (The IP Target Type): When traffic leaves the ALB Target Groups, it routes directly to individual Pod IPs inside the EC2 Worker Nodes. By utilizing the AWS VPC CNI network plugin, the traffic bypasses the Kubernetes kube-proxy layer entirely. This bypass ensures the ALB retains distinct visibility into the active connection metrics of each pod, keeping Least Outstanding Requests (LOR) calculations precise.



No comments: