Monday, 29 July 2024

Kubernetes Ingress Service


Ingress is a more flexible and powerful solution for managing external access to services within a Kubernetes cluster than Kubernetes LoadBalancer Service

It provides:
  • load balancing
  • SSL termination
  • name-based virtual hosting

Ingress controllers can be configured to handle traffic more efficiently and securely.

A full Ingress deployment manifest in Kubernetes defines how external traffic is routed to services within the cluster. It uses an Ingress controller (like NGINX) to manage the rules, and the manifest specifies which services should be exposed, based on hostnames, paths, or other criteria. The manifest typically includes details about the Ingress class, rules, and the services it exposes. 

Manifest example:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: my-ingress
  namespace: default
spec:
  ingressClassName: nginx # Or gce, aws, etc.
  rules:
  - host: example.com # or myapp.example.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: my-service
            port:
              number: 80
  - host: myapp.example.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: my-service
            port:
              number: 80
  - host: myapp2.example.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: my-service2
            port:
              number: 80

Here is the meaning of all fields in the manifest:
  • apiVersion: Kubernetes API version
  • kind: Kubernetes object type (Ingress)
  • metadata: Metadata about the Ingress, including its name, namespace, and labels. 
  • spec: The heart of the manifest, defining the Ingress rules and behavior. 
    • ingressClassName: Specifies the Ingress controller to use (e.g., nginx, gce, aws).
    • rules: An array of rules that define how traffic should be routed. Each rule includes: 
      • host: The hostname to match (e.g., example.com). 
      • http: A section defining HTTP traffic routing.
        • paths: An array of paths to match within the HTTP request.
          • backend: The service to forward traffic to based on the path.
            • serviceName: The name of the service.
            • servicePort: The port of the service.



host

Removing the host (host-specific rules) from the ingress will make it accept traffic for any hostname, potentially exposing the service more broadly than intended. These changes could lead to unintended access to services if the ALB receives requests with different or no host headers.

serviceName

What is this service and what are its requirements so it can serve the traffic?


In Kubernetes (K8s), an Ingress manifest by itself does not directly create an AWS load balancer — but it can indirectly result in one being created depending on how your cluster is configured.

Ingress is just a K8s object that defines HTTP routing rules (e.g., "requests to /foo go to Service A"). It requires an Ingress Controller to actually implement those rules and expose the traffic. If we just apply an Ingress manifest and no Ingress Controller is installed, nothing will happen — no load balancer will be created.

To get an AWS Load Balancer via Ingress we need to install and use an Ingress Controller that integrates with AWS, such as:

  • AWS ALB Ingress Controller (now called AWS Load Balancer Controller)
  • Nginx Ingress Controller (but then you'd still need a Service of type LoadBalancer to expose it)

If we're using the AWS Load Balancer Controller and define an Ingress object with proper annotations, then it will create an AWS Application Load Balancer (ALB) and configure it with listeners and target groups according to the Ingress rules. This is the most direct way to have an Ingress lead to the creation of an AWS load balancer.

NOTE: LB creation takes some time e.g. 30 seconds.


Example: Ingress manifest that works with AWS Load Balancer Controller

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: example-ingress
  namespace: default
  annotations:
    alb.ingress.kubernetes.io/scheme: internet-facing
    alb.ingress.kubernetes.io/target-type: ip
    alb.ingress.kubernetes.io/listen-ports: '[{"HTTP": 80}]'
spec:
  ingressClassName: alb
  rules:
    - http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: my-service
                port:
                  number: 80


Annotations


alb.ingress.kubernetes.io/scheme


The annotation alb.ingress.kubernetes.io/scheme in AWS Load Balancer Controller specifies the visibility and network exposure of the Application Load Balancer (ALB) created for your Kubernetes Ingress.

Possible values:
  • internet-facing
    • ALB is publicly accessible over the internet.
    • Use for services that need to be accessed publicly — e.g., websites, APIs, dashboards
  • internal
    • ALB is private, only accessible within your VPC (e.g., internal apps).
    • Use for internal microservices, admin interfaces, or when you only want access within a private network (e.g., from a VPN or other VPC resources).
Example (internet facing):

metadata:
  name: my-ingress
  annotations:
    alb.ingress.kubernetes.io/scheme: internet-facing


This will create a public-facing ALB with a public DNS name like:

my-ingress-1234567890.us-west-2.elb.amazonaws.com


The subnets in your cluster must be correctly tagged:
  • For internet-facing: subnets must be public and tagged with: kubernetes.io/role/elb = 1
  • For internal: subnets must be private and tagged with: kubernetes.io/role/internal-elb = 1
If you don't set the scheme explicitly, it defaults to internet-facing.


Example: Internal Ingress 

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: internal-ingress
  namespace: default
  annotations:
    kubernetes.io/ingress.class: alb
    alb.ingress.kubernetes.io/scheme: internal
    alb.ingress.kubernetes.io/subnets: subnet-abc123,subnet-def456  # Optional: for precise control
    alb.ingress.kubernetes.io/group.name: internal-apps
    alb.ingress.kubernetes.io/tags: "env=dev,app=my-internal-service"
spec:
  rules:
    - http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: my-service
                port:
                  number: 80


This configuration instructs the AWS Load Balancer Controller to:
  • Create an internal ALB (i.e., not exposed to the public internet).
  • Place the ALB in private subnets (must be tagged: kubernetes.io/role/internal-elb = 1).
  • The ALB is reachable only within the VPC or from connected networks (like via VPN or Direct Connect).

The ALB will get a DNS name that looks like this:

internal-k8s-<name>-<hash>.<region>.elb.amazonaws.com

e.g.

internal-k8s-internalingr-abc123456789.us-west-2.elb.amazonaws.com

The internal- prefix on the DNS name indicates it is not public. It will not resolve outside the VPC (e.g., from your home network or the public internet).

We can verify the internal ALB:

  • AWS Console:
            Go to EC2 → Load Balancers → Look at the Scheme column (internal).
  • CLI:
     aws elbv2 describe-load-balancers \
         --names internal-k8s-internalingr-abc123456789


alb.ingress.kubernetes.io/listen-ports


The annotation alb.ingress.kubernetes.io/listen-ports in AWS Load Balancer Controller is used to customize the listener ports that are created on the Application Load Balancer (ALB) for a Kubernetes Ingress resource.

By default, the AWS Load Balancer Controller creates listeners for port 80 (HTTP) and/or port 443 (HTTPS) depending on your Ingress TLS settings.
If you need to override this behavior—for example, to support only HTTPS or use non-default ports—you can use this annotation.

Example: Default Behavior (HTTP and HTTPS):

alb.ingress.kubernetes.io/listen-ports: '[{"HTTP": 80}, {"HTTPS": 443}]'


Creates both:
  • an HTTP listener on port 80
  • an HTTPS listener on port 443 (requires TLS configuration)

It must be a JSON array of objects, where each object specifies a protocol and port.


Example: HTTPS Only (no HTTP listener):

alb.ingress.kubernetes.io/listen-ports: '[{"HTTPS": 443}]'


Useful if you want to force HTTPS and avoid exposing port 80. Make sure you also configure tls: section in the Ingress and associate an ACM certificate using:

alb.ingress.kubernetes.io/certificate-arn


Example: Custom Ports (e.g., HTTP on 8080):

alb.ingress.kubernetes.io/listen-ports: '[{"HTTP": 8080}]'

ALB will listen on port 8080 for HTTP traffic instead of the default 80.

We can only use ports supported by ALB, which are typically: 1–65535, but port 80 and 443 are standard and expected by most clients.. Custom ports may not work well unless you control the client environment.


Example: Internal Service with Only HTTPS

annotations:
  alb.ingress.kubernetes.io/scheme: internal
  alb.ingress.kubernetes.io/listen-ports: '[{"HTTPS": 443}]'
  alb.ingress.kubernetes.io/certificate-arn: arn:aws:acm:...

If we specify only HTTPS, you must provide a valid certificate via alb.ingress.kubernetes.io/certificate-arn.

If we omit this annotation, the controller decides based on TLS block:
  • If TLS is configured, both HTTP and HTTPS are created
  • If not, only HTTP on port 80 is created



alb.ingress.kubernetes.io/tags


Specifies which tags will be applied onto Application Load Balancer that will be created by LB Controller. Example:

alb.ingress.kubernetes.io/tags: "kubernetes.io/ingress-name=${local.ingress_name},Environment=${local.workspace},Team=${local.team}"

This means that LB will have these tags:
  • kubernetes.io/ingress-name, set to value ${local.ingress_name}
  • Environment set to value ${local.workspace}
  • Team, set to value ${local.team}

Note that by default, AWS LB Controller applies the following tags:
  • ingress.k8s.aws/resource = LoadBalancer
  • ingress.k8s.aws/stack = <namespace>/<ingress_name>



alb.ingress.kubernetes.io/group.name

One of the possible annotations is alb.ingress.kubernetes.io/group.name. It is used in AWS Load Balancer Controller. It specifies the target group name for grouping multiple Ingress resources under a single Application Load Balancer (ALB). 

By default, each Ingress resource gets its own ALB, which can be costly or unnecessary. Setting the alb.ingress.kubernetes.io/group.name annotation allows you to share a single ALB across multiple Ingress resources.

Example:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: my-ingress
  annotations:
    alb.ingress.kubernetes.io/group.name: my-apps
    alb.ingress.kubernetes.io/scheme: internet-facing
spec:
  rules:
    - host: app1.example.com
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: app1-service
                port:
                  number: 80


If we have another Ingress resource with the same group.name, it will be attached to the same ALB.

Benefits of using group.name:
  • Cost efficiency: Fewer ALBs to manage and pay for.
  • Simplified architecture: Group related services behind one ALB.
  • Custom routing: Combine multiple paths or hostnames on one load balancer.

All Ingresses in the same group must share the same ALB settings, such as:
  • alb.ingress.kubernetes.io/scheme (e.g. internet-facing or internal)
  • alb.ingress.kubernetes.io/load-balancer-name (optional but can be used)

The ALB is shared, but each rule/path is managed separately based on the Ingress definitions.



external-dns.alpha.kubernetes.io/ingress-hostname-source


 annotations-only: This annotation configures External DNS to only use explicit hostname annotations when determining the DNS records to create




---

No comments: