Pages

Tuesday 7 May 2024

Kubernetes Deployments

These are custom notes that extend my notes from an Udemy course "Kubernetes for the Absolute Beginners - Hands-on". All course content rights belong to course creators. 

This is the fourth article in the series, the previous one being Kubernetes Controllers | My Public Notepad

Let's look at the process of deploying our application in a production environment. 

Let's assume we have a web server that needs to be deployed in a production environment. For high availability reasons, we need multiple instances of the web server running.

Also, whenever newer versions of application builds become available on the Docker registry, we would like to upgrade our Docker instances seamlessly. When upgrading the instances we do not want to upgrade all of them at once. This may impact users accessing our applications so we might want to upgrade them one after the other. That kind of upgrade is known as rolling updates.

If one of the upgrades we performed resulted in an unexpected error, we'd need to undo the
recent change. We should be able to roll back the changes that were recently carried out.

We might want to make multiple changes to our environment such as upgrading the underlying Web Server versions as well as scaling our environment and also modifying the resource allocations etc. We don't want to apply each change immediately after the command is run, instead we'd like to apply a pause to our environment, make the changes and then resume so that all the changes are rolled out together.

All of these capabilities are available with the Kubernetes Deployments

Pods deploy single instances of our application such as the web application. Each container is encapsulated in pods. Multiple pods are deployed using Replication Controllers or Replica Sets and then comes Deployment which is a Kubernetes object that comes higher in the hierarchy. 

Deployment provides us with the capability to:
  • upgrade the underlying instances seamlessly using rolling updates
  • undo changes
  • pause and resume changes as required

How to create a deployment?


As with pods and replica sets, for this new Kubernetes object type, we first create the deployment definition file. Its contents is the same to the replica set definition file, except for the kind which is now going to be Deployment.

deployment_definition.yaml:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: myapp-deployment
  labels:
    app: myapp
    type: front-end
spec:
  template:
    metadata:
      name: myapp-pod
      labels:
        app: my-app
        type: front-end
    spec:
      containers:
      - name: nginx-container
        image: nginx:1.7.0
  replicas: 3
  selector:
    matchLabels:
      type: front-end


template has a pod definition inside it.

To create deployment:

$ kubectl create -f deployment_definition.yaml 
deployment.apps/myapp-deployment created


If number of deployment definition attributes is small, we can create a deployment without the full-blown definition file:

$ kubectl create deployment <deployment_name> --image=<image_name> --replicas=<replicas_count>


If we try to create a deployment that has the name same as some already created deployment, we'll get the error:

$ kubectl create -f ./minikube/deployment/deployment.yaml 
Error from server (AlreadyExists): error when creating "./minikube/deployment/deployment.yaml": deployments.apps "myapp-deployment" already exists

This applies to any other Kubernetes object type.

Kubernetes object type that we put as the kind value is case-sensitive. If we use the name which does not start with capital letter, we'll get an error like this:

$ kubectl create -f /root/deployment-definition-1.yaml 
Error from server (BadRequest): error when creating "/root/deployment-definition-1.yaml": deployment in version "v1" cannot be handled as a Deployment: no kind "deployment" is registered for version "apps/v1" in scheme "k8s.io/apimachinery@v1.29.0-k3s1/pkg/runtime/scheme.go:100"

To see created deployments:

$ kubectl get deployment
NAME               READY   UP-TO-DATE   AVAILABLE   AGE
myapp-deployment   3/3     3            3           105s

or, version with the plural:

$ kubectl get deployments
NAME               READY   UP-TO-DATE   AVAILABLE   AGE
myapp-deployment   3/3     3            3           107s


Deployment automatically creates a replica set:

$ kubectl get rs
NAME                          DESIRED   CURRENT   READY   AGE
myapp-deployment-6bddbfd569   3         3         3       3m35s


Deployment contains a single pod template, and generates one replicaset per revision. 

To get more details about ReplicaSet:

$ kubectl get rs -o wide
NAME                          DESIRED   CURRENT   READY   AGE     CONTAINERS        IMAGES   SELECTOR
myapp-deployment-6bddbfd569   3         3         3       3m58s   nginx-container   nginx    pod-template-hash=6bddbfd569,type=front-end


The replica sets ultimately create pods:

$ kubectl get pod
NAME                                READY   STATUS    RESTARTS   AGE
myapp-deployment-6bddbfd569-n7j4z   1/1     Running   0          6m6s
myapp-deployment-6bddbfd569-qzpgl   1/1     Running   0          6m6s
myapp-deployment-6bddbfd569-xlnz8   1/1     Running   0          6m6s

or 

$ kubectl get pods
NAME                                READY   STATUS    RESTARTS   AGE
myapp-deployment-6bddbfd569-n7j4z   1/1     Running   0          6m8s
myapp-deployment-6bddbfd569-qzpgl   1/1     Running   0          6m8s
myapp-deployment-6bddbfd569-xlnz8   1/1     Running   0          6m8s


Note that:

replica_set_name = <deployment_name>-<pod_template_hash>

pod_name = <replica_set_name>-<arbitrary_id>=<deployment_name>-<pod_template_hash>-<arbitrary_id>

To see all the created Kubernetes objects at once run:

$ kubectl get all
NAME                                    READY   STATUS    RESTARTS   AGE
pod/myapp-deployment-6bddbfd569-n7j4z   1/1     Running   0          14m
pod/myapp-deployment-6bddbfd569-qzpgl   1/1     Running   0          14m
pod/myapp-deployment-6bddbfd569-xlnz8   1/1     Running   0          14m

NAME                 TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)   AGE
service/kubernetes   ClusterIP   10.96.0.1    <none>        443/TCP   19d

NAME                               READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/myapp-deployment   3/3     3            3           14m

NAME                                          DESIRED   CURRENT   READY   AGE
replicaset.apps/myapp-deployment-6bddbfd569   3         3         3       14m


In this output above, observe how naming of Kubernetes objects unveils their hierarchy: 
deployment >> 
    replicaset >> 
        pods

To get extensive information about the deployment:

$ kubectl describe deployment myapp-deployment
Name:                   myapp-deployment
Namespace:              default
CreationTimestamp:      Tue, 07 May 2024 23:44:32 +0100
Labels:                 app=myapp
                        type=front-end
Annotations:            deployment.kubernetes.io/revision: 1
Selector:               type=front-end
Replicas:               3 desired | 3 updated | 3 total | 3 available | 0 unavailable
StrategyType:           RollingUpdate
MinReadySeconds:        0
RollingUpdateStrategy:  25% max unavailable, 25% max surge
Pod Template:
  Labels:  app=my-app
           type=front-end
  Containers:
   nginx-container:
    Image:         nginx
    Port:          <none>
    Host Port:     <none>
    Environment:   <none>
    Mounts:        <none>
  Volumes:         <none>
  Node-Selectors:  <none>
  Tolerations:     <none>
Conditions:
  Type           Status  Reason
  ----           ------  ------
  Available      True    MinimumReplicasAvailable
  Progressing    True    NewReplicaSetAvailable
OldReplicaSets:  <none>
NewReplicaSet:   myapp-deployment-6bddbfd569 (3/3 replicas created)
Events:
  Type    Reason             Age   From                   Message
  ----    ------             ----  ----                   -------
  Normal  ScalingReplicaSet  6s    deployment-controller  Scaled up replica set myapp-deployment-6bddbfd569 to 3



Updates and rollbacks in a deployment



Before we look at how we upgrade our application, let's try to understand rollouts and versioning in a deployment.


Rollouts and Versioning


After deployment is created, it triggers a rollout. A new rollout creates a new deployment revision. Let's call it Revision 1.

In the future when the application is upgraded, meaning when the container version is updated to a new one, a new rollout is triggered and a new deployment revision is created named Revision 2.

This helps us keep track of the changes made to our deployment and enables us to roll back to a previous version of deployment if necessary.

You can see the status of your rollout by running:

$ kubectl rollout status deployment/deployment_name

For example:

$ kubectl rollout status deployment/myapp-deployment
deployment "myapp-deployment" successfully rolled out

Note that it's mandatory to include the name of the Kubernetes object type - deployment in this case. If we don't include it, we'll get an error:

$ kubectl rollout status myapp-deployment
error: the server doesn't have a resource type "myapp-deployment"

To see the revisions and history of rollouts run:

$ kubectl rollout history deployment/myapp-deployment
deployment.apps/myapp-deployment 
REVISION  CHANGE-CAUSE
1         <none>


Deployment Strategies


There are two types of deployment strategies: 
  • recreate
  • rolling update

Let's assume that we have five replicas of our web application instance deployed.

One way to upgrade these to a newer version is to destroy all of these and then create newer versions of application instances, meaning first destroy the five running instances, and then deploy five new instances of the new application version. The problem with this, as you can imagine, is that during the period after the older versions are down and before any newer version is up, the application is down and inaccessible to users. This strategy is known as the recreate strategy and thankfully this is not the default deployment strategy.

The second strategy is where we do not destroy all of them at once. Instead, we take down the older version and bring up a newer version one by one. This way the application never goes down and the upgrade is seamless. This strategy is called rolling update.

If we do not specify a strategy while creating the deployment, it will assume it to be rolling update (rolling update is the default deployment strategy).


How to update deployment?


Update can mean updating our application version,  updating the version of Docker containers used, updating their labels or updating the number of replicas, etc.

We modify a deployment definition file (e.g. we change the version/tag of the container image) and then run:

$ kubectl apply -f deployment_definition.yaml


Running this command applies the changes, a new rollout is triggered and a new revision of the deployment is created.

But there is another way to do the same thing:

$ kubectl set image deployment/myapp-deployment nginx-container=nginx:1.7.1

You can use this command to update the image of our application but doing it this way will result in the deployment definition file having a different configuration.


How a deployment performs an upgrade (update)?


The difference between the recreate and rolling update strategies can be seen when we look at the deployments in detail:

kubectl describe deployment myapp-deployment

When the recreate strategy was used, the events indicate that the old replica set was scaled down to zero first and then the new replica set scaled up to five.

However, when the rolling update strategy was used, the old replica set was scaled down one at a time, simultaneously scaling up the new replica set one at a time.

We can see that during the update we have two replica sets, the old and a new one.

When a brand new deployment is created, say, to deploy five replicas, it first creates a replica set automatically, which in turn creates the number of pods required to meet the number of replicas. 

When we upgrade our application, the Kubernetes deployment object creates a new replica set under the hood and starts deploying the containers there at the same time taking down the pods in the old replica set following a rolling update strategy.

This can be seen when we list the replica sets:

$ kubectl get replicasets 


Here we'd see the old replica set with zero pods and the new replica set with five pods.


How a deployment performs a roll back?


Let's assume that once we upgrade our application, we realize something isn't right, something's wrong with the new version of build we used to upgrade. So we would like to roll back our update.

Kubernetes deployments allow us to roll back to a previous revision.

To undo a change run:

$ kubectl rollout undo deployment/myapp-deployment


The deployment will then destroy the pods in the new replica set and bring the older ones up in the
old replica set.

When we compare the output of the kubectl get replica sets command before and after the rollback, we will be able to notice this difference before the rollback.

The first replica set had zero pods and new replica set had five pods and this is reversed after the rollback is finished.

To summarize the commands:

kubectl create -  to create the deployment
kubectl get deployments - to list the deployments
kubectl apply - to update the deployments
kubectl set image - to update the deployments
kubectl rollout status -  to see the status of rollouts 
kubectl rollout undo - to roll back a deployment operation
---

Sunday 5 May 2024

Kubernetes Controllers

These are custom notes that extend my notes from an Udemy course "Kubernetes for the Absolute Beginners - Hands-on". All course content rights belong to course creators. 

This is the third article in the series, the previous one being Managing Pods In A Minikube Cluster | My Public Notepad.




Controllers are the processes that monitor Kubernetes objects and respond accordingly. One of them is the Replication Controller.

Let's consider a case where we have a single pod running our application. We'll show here two reasons why we want to have pod replicas running.

1) High Availability


If application crashes and the pod fails, users will no longer be able to access our application. To prevent this we need to have more than one instance of pod (pod replicas) running at the same time. If one fails, we still have our application running on the other one.

Replication Controller helps us run multiple instances of a single pod in the Kubernetes cluster, thus providing high availability. Even if we want to have a single pod, the Replication Controller can help by automatically bringing up a new pod when the existing one fails.

Replication Controller ensures that the specified number of pods are running at all times, no matter if it's one or more pods.

2) Load Balancing 


So we have a single pod serving a set of users. When the number of users increase, we deploy an additional pod to share and balance the load across the two pods. If the demand further increases and if we were to run out of resources on the first node, we could deploy additional pods across the other nodes in the cluster.

Replication Controller spans across multiple nodes in the cluster. It helps us balance the load across multiple pods on different nodes as well as scale our application when the demand increases.

Replication Controller vs Replica Set


Replication Controller and Replica Set are similar terms, both have the same purpose, but they are not the same.

Replication Controller is the older technology that is being replaced by Replica Set which is the new recommended way to set up replication. There are minor differences in the way each work.

How to create a Replication Controller?


Just as we used definition files to create pod objects (see Managing Pods In A Minikube Cluster | My Public Notepad), we will use a Replication Controller definition file which, as any Kubernetes definition file, has four sections: apiVersion, kind, metadata and spec.

rc-definition.yaml:

apiVersion: v1
kind: ReplicationController
metadata:
  name: myapp-rc
  labels:
    app: myapp
    type: front-end
spec:
  template:
    metadata:
      name: myapp-pod
      labels:
        app: myapp
        type: frontend
    spec:
      containers:
      - name: nginx-container
        image: nginx
  replicas: 3 


Replication Controller is supported in Kubernetes API version v1.

In any Kubernetes definition file the spec section defines what's inside the object we are creating.

Replication Controller needs to create multiple instances of a pod. We need to specify a blueprint of those pod instances so we'll create a template section under spec to provide a pod template to be used by the Replication Controller to create replicas.

For pod template definition we can reuse the contents of the pod definition file pod-definition.yml that we created in Managing Pods In A Minikube Cluster | My Public Notepad so we'll simply move all the contents of the pod definition file into the template section of the replication controller, except for the apiVersion (it is already specified) and kind (no need to specify kind as it can only be a pod). Make sure that metadata and spec are children of the template and are properly indented.

To specify how many replicas we want to have we use property replicas under the spec (at the same level as template).

To create this Kubernetes object let's first start minikube (which a single cluster on the local host and configures kubectl to talk to that minikube cluster):

$ minikube start
😄  minikube v1.33.0 on Ubuntu 22.04
✨  Using the docker driver based on existing profile
👍  Starting "minikube" primary control-plane node in "minikube" cluster
🚜  Pulling base image v0.0.43 ...
🏃  Updating the running docker "minikube" container ...
🐳  Preparing Kubernetes v1.30.0 on Docker 26.0.1 ...
🔎  Verifying Kubernetes components...
    ▪ Using image gcr.io/k8s-minikube/storage-provisioner:v5
🌟  Enabled addons: storage-provisioner, default-storageclass
🏄  Done! kubectl is now configured to use "minikube" cluster and "default" namespace by default

We can check kubectl config:

$ cat  ~/.kube/config 
apiVersion: v1
clusters:
- cluster:
    certificate-authority: /home/bojan/.minikube/ca.crt
    extensions:
    - extension:
        last-update: Sun, 05 May 2024 12:54:52 BST
        provider: minikube.sigs.k8s.io
        version: v1.33.0
      name: cluster_info
    server: https://192.168.49.2:8443
  name: minikube
contexts:
- context:
    cluster: minikube
    extensions:
    - extension:
        last-update: Sun, 05 May 2024 12:54:52 BST
        provider: minikube.sigs.k8s.io
        version: v1.33.0
      name: context_info
    namespace: default
    user: minikube
  name: minikube
current-context: minikube
kind: Config
preferences: {}
users:
- name: minikube
  user:
    client-certificate: /home/bojan/.minikube/profiles/minikube/client.crt
    client-key: /home/bojan/.minikube/profiles/minikube/client.key


To find more details about ReplicaSet Kubernetes object we can use:

$ kubectl explain replicaset
GROUP:      apps
KIND:       ReplicaSet
VERSION:    v1

DESCRIPTION:
    ReplicaSet ensures that a specified number of pod replicas are running at
    any given time.
    
FIELDS:
  apiVersion    <string>
    APIVersion defines the versioned schema of this representation of an object.
    Servers should convert recognized schemas to the latest internal value, and
    may reject unrecognized values. More info:
    https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources

  kind  <string>
    Kind is a string value representing the REST resource this object
    represents. Servers may infer this from the endpoint the client submits
    requests to. Cannot be updated. In CamelCase. More info:
    https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds

  metadata      <ObjectMeta>
    If the Labels of a ReplicaSet are empty, they are defaulted to be the same
    as the Pod(s) that the ReplicaSet manages. Standard object's metadata. More
    info:
    https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#metadata

  spec  <ReplicaSetSpec>
    Spec defines the specification of the desired behavior of the ReplicaSet.
    More info:
    https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#spec-and-status

  status        <ReplicaSetStatus>
    Status is the most recently observed status of the ReplicaSet. This data may
    be out of date by some window of time. Populated by the system. Read-only.
    More info:
    https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#spec-and-status


Let's now create replication controller based on its definition file:

$ kubectl create -f rc-definition.yaml 
replicationcontroller/myapp-rc created


When the replication controller is created, it first creates the pods using the pod definition template, as many as required, which is 3 in our case.

To view the list of created replication controllers:

$ kubectl get replicationcontrollers
NAME       DESIRED   CURRENT   READY   AGE
myapp-rc   3         3         3       6m51s


The output also shows the number of desired, current and ready pod replicas.

To see all the pods and among them the pods that were created by the replication controller:

$ kubectl get pods
NAME             READY   STATUS    RESTARTS   AGE
myapp-rc-4mcjh   1/1     Running   0          9m58s
myapp-rc-5jrm2   1/1     Running   0          9m58s
myapp-rc-rg6jm   1/1     Running   0          9m58s

Unlike standalone, independent pods, all pods created automatically by the replication controller have names that start with the name of the replication controller, which is myapp-rc in our case. 

RC, indicating that they are all created automatically by the replication controller.

To delete objects defined in a definition file:

$ kubectl delete -f ./minikube/rc-definition.yaml 
replicationcontroller "myapp-rc" deleted

$ kubectl get pods
No resources found in default namespace.

How to create a Replica Set?


Let's create a definition file, which is very similar to the one above, for replication controller:

replicaset-definition.yaml:

apiVersion: apps/v1
kind: ReplicaSet
metadata:
  name: myapp-replicaset
  labels:
    app: myapp
    type: front-end
spec:
  template:
    metadata:
      name: myapp-pod
      labels:
        app: myapp
        type: front-end
    spec:
      containers:
      - name: nginx-container
        image: nginx
  replicas: 3
  selector:
    matchLabels:
      type: front-end


Kubernetes API version v1 has no support for Replica Sets but app/v1 does. ReplicaSets moved to apps/v1 in 1.9 version of Kubernetes.

One major difference between replication controller and replica set: replica set requires a selector definition.

The selector section helps the replica set identify what pods fall under it. We need to specify it although we specify what pods fall under it via the template because replica set can also manage pods that were not created as part of the replica set creation.

There might be pods created before the creation of the replica set that match labels specified in the selector. The replica set will also take (the number of) those pods into consideration when creating the replicas.

The selector is not a required field in case of a replication controller, but it is still available. When we skip it, as we did in the rc-definition.yaml, Kubernetes assumes it to be the same as the labels provided in the pod definition file.

In case of replica set, selector needs explicitly to be stated in the definition file.

matchLabels selector simply matches the labels specified under it to the labels on the pod.

The replica set selector also provides many other options for matching labels that were not available in a replication controller.

To create a replica set and then verify it and also number of pods:

$ kubectl create -f replicaset-definition.yaml 
replicaset.apps/myapp-replicaset created

$ kubectl get replicasets
NAME               DESIRED   CURRENT   READY   AGE
myapp-replicaset   3         3         3       9s

There is also a shorter version of the same command, using alias rs instead of replicaset(s):

$ kubectl get rs
$ kubeclt delete rs my-replicaset
 
Let's check the pods:

$ kubectl get pods
NAME                     READY   STATUS    RESTARTS   AGE
myapp-replicaset-47xr4   1/1     Running   0          13s
myapp-replicaset-9bq9l   1/1     Running   0          13s
myapp-replicaset-p6q7m   1/1     Running   0          13s


To find out more details about ReplicaSets e.g. the name of the container running in pods and which Docker image has been used for pod creation we can use:

$ kubectl get replicasets -o wide
NAME              DESIRED   CURRENT   READY   AGE   CONTAINERS          IMAGES       SELECTOR
new-replica-set   4         4         0       57s   busybox-container   busybox777   name=busybox-pod

(The example above is not related to the execution of previous commands, it's just a generic example)

In the example above, we see that there are 4 current pods but none of them is READY. To find out the reason we can check pods events with:

$ kubectl describe pods

or, if we want to check the specific pod:

$ kubectl get pods
NAME                    READY   STATUS             RESTARTS   AGE
new-replica-set-2kp8n   0/1     ImagePullBackOff   0          6m19s
new-replica-set-4mrf4   0/1     ImagePullBackOff   0          6m19s
new-replica-set-nxggw   0/1     ImagePullBackOff   0          6m19s
new-replica-set-hh4j9   0/1     ImagePullBackOff   0          6m19s


$ kubectl describe pod new-replica-set-2kp8n
Name:             new-replica-set-2kp8n
Namespace:        default
Priority:         0
Service Account:  default
Node:             controlplane/192.21.178.6
Start Time:       Tue, 07 May 2024 11:05:48 +0000
Labels:           name=busybox-pod
Annotations:      <none>
Status:           Pending
IP:               10.42.0.9
IPs:
  IP:           10.42.0.9
Controlled By:  ReplicaSet/new-replica-set
Containers:
  busybox-container:
    Container ID:  
    Image:         busybox777
    Image ID:      
    Port:          <none>
    Host Port:     <none>
    Command:
      sh
      -c
      echo Hello Kubernetes! && sleep 3600
    State:          Waiting
      Reason:       ImagePullBackOff
    Ready:          False
    Restart Count:  0
    Environment:    <none>
    Mounts:
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-8lgtt (ro)
Conditions:
  Type                        Status
  PodReadyToStartContainers   True 
  Initialized                 True 
  Ready                       False 
  ContainersReady             False 
  PodScheduled                True 
Volumes:
  kube-api-access-8lgtt:
    Type:                    Projected (a volume that contains injected data from multiple sources)
    TokenExpirationSeconds:  3607
    ConfigMapName:           kube-root-ca.crt
    ConfigMapOptional:       <nil>
    DownwardAPI:             true
QoS Class:                   BestEffort
Node-Selectors:              <none>
Tolerations:                 node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
                             node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
  Type     Reason     Age                   From               Message
  ----     ------     ----                  ----               -------
  Normal   Scheduled  13m                   default-scheduler  Successfully assigned default/new-replica-set-2kp8n to controlplane
  Normal   Pulling    11m (x4 over 13m)     kubelet            Pulling image "busybox777"
  Warning  Failed     11m (x4 over 13m)     kubelet            Failed to pull image "busybox777": failed to pull and unpack image "docker.io/library/busybox777:latest": failed to resolve reference "docker.io/library/busybox777:latest": pull access denied, repository does not exist or may require authorization: server message: insufficient_scope: authorization failed
  Warning  Failed     11m (x4 over 13m)     kubelet            Error: ErrImagePull
  Warning  Failed     11m (x6 over 13m)     kubelet            Error: ImagePullBackOff
  Normal   BackOff    3m22s (x42 over 13m)  kubelet            Back-off pulling image "busybox777"


In this case, the specified Docker image was wrong, non-existing. 

To fix the replica set so it uses the correct Docker image we need to update ReplicaSet definition after which we can: 
  • either delete and recreate the ReplicaSet 
  • or delete all pods, so new ones with the correct image will be created
If we fix/edit ReplicaSet definition, new pods, using the updated information, will not be created automatically.


To check events for all ReplicaSets:

$ kubectl describe replicasets

or, for specific one:

$ kubectl describe replicaset new-replica-set
Name:         new-replica-set
Namespace:    default
Selector:     name=busybox-pod
Labels:       <none>
Annotations:  <none>
Replicas:     4 current / 4 desired
Pods Status:  0 Running / 4 Waiting / 0 Succeeded / 0 Failed
Pod Template:
  Labels:  name=busybox-pod
  Containers:
   busybox-container:
    Image:      busybox777
    Port:       <none>
    Host Port:  <none>
    Command:
      sh
      -c
      echo Hello Kubernetes! && sleep 3600
    Environment:  <none>
    Mounts:       <none>
  Volumes:        <none>
Events:
  Type    Reason            Age    From                   Message
  ----    ------            ----   ----                   -------
  Normal  SuccessfulCreate  7m46s  replicaset-controller  Created pod: new-replica-set-hh4j9
  Normal  SuccessfulCreate  7m46s  replicaset-controller  Created pod: new-replica-set-2kp8n
  Normal  SuccessfulCreate  7m46s  replicaset-controller  Created pod: new-replica-set-nxggw
  Normal  SuccessfulCreate  7m46s  replicaset-controller  Created pod: new-replica-set-4mrf4



Scaling In Action


Case #1: There are less pods than specified in replicas


Let's delete one of the pods that are managed by ReplicaSet:

$ kubectl delete pod myapp-replicaset-47xr4
pod "myapp-replicaset-47xr4" deleted


As a side note, we can delete multiple pods with:

$ kubectl delete pods my-replica-set-4mrf4 my-replica-set-hh4j9 my-replica-set-nxggw my-replica-set-tgs9d 
pod "my-replica-set-4mrf4" deleted
pod "my-replica-set-hh4j9" deleted
pod "my-replica-set-nxggw" deleted
pod "my-replica-set-tgs9d" deleted

But let's assume we only deleted that one pod.

If we now check pods, we'll see that we still have 3 pods, but one has a new name. That is the pod that ReplicaSet created once it realized that one pod went down:

kubectl get pods
NAME                     READY   STATUS    RESTARTS   AGE
myapp-replicaset-ne54m   1/1     Running   0          12s
myapp-replicaset-9bq9l   1/1     Running   0          1m23s
myapp-replicaset-p6q7m   1/1     Running   0          1m233s


All events related to the specific ReplicaSet can be seen in the output of this command:

$ kubectl describe replicaset myapp-replicaset
Name:         myapp-replicaset
Namespace:    default
Selector:     env=production
Labels:       app=myapp
Annotations:  <none>
Replicas:     3 current / 3 desired
Pods Status:  3 Running / 0 Waiting / 0 Succeeded / 0 Failed
Pod Template:
  Labels:  env=production
  Containers:
   nginx:
    Image:         nginx
    Port:          <none>
    Host Port:     <none>
    Environment:   <none>
    Mounts:        <none>
  Volumes:         <none>
  Node-Selectors:  <none>
  Tolerations:     <none>
Events:
  Type    Reason            Age    From                   Message
  ----    ------            ----   ----                   -------
  Normal  SuccessfulCreate  17m    replicaset-controller  Created pod: myapp-replicaset-47xr4
  Normal  SuccessfulCreate  17m    replicaset-controller  Created pod: myapp-replicaset-9bq9l
  Normal  SuccessfulCreate  17m    replicaset-controller  Created pod: myapp-replicaset-p6q7m
  Normal  SuccessfulCreate  6m33s  replicaset-controller  Created pod: myapp-replicaset-ne54m


Case #2: There are more pods than specified in replicas


Let's consider situation when we have more pods, all with the same label as specified in ReplicaSet's matchLabels selector. Let's  test what happens if we have one extra such pod:

$ kubectl create -f nginx.yaml 
pod/nginx created

We would expect now to have 4 pods running but we actually have 3 as ReplicaSet terminated one extra pod:

$ kubectl get pods
NAME                     READY   STATUS    RESTARTS   AGE
myapp-replicaset-jnm6z   1/1     Running   0          28m
myapp-replicaset-nk5n9   1/1     Running   0          18m
myapp-replicaset-nwwmh   1/1     Running   0          28m

If we executed the last command as soon as new pod was created, we would have seen all four pods here but this last one would be in Terminating state.

Labels and Selectors


What is the use case for labels and selectors? Why do we label our pods and objects in Kubernetes?

Let's assume we deployed three instances of our front end web application as three pods. We would like to create a replication controller or replica set to ensure that we have three active pods at any time.

We can use replica set to monitor these existing pods if we have them already created as it is in this example. In case they were not created, the replica set will create them for us.

The role of the replica set is to monitor the pods and if any of them were to fail deploy new ones.
The replica set is in fact a process that monitors the pods.

Now how does the replica set know what pods to monitor? There could be hundreds of other pods in the cluster running different applications. This is where labelling our pods during creation comes in handy. We could now provide these labels as a filter for replica set. Under the selector section we use the matchLabels filter and provide the same label that we used while creating the pods. This way the replica set knows which pods to monitor.

What happens if label in template does not match the label in the matchLabels? kubectl create fails and reports an error related to mismatched labels.

Why labels are not enough and why we need templates?



Let's assume we have three existing pods that were created already and we need to create a replica set to monitor the pods to ensure there are a minimum of three running at all times. When the replica set is created, it is not going to deploy a new instance of POD as three of them with matching labels are already created. We need to provide a template section in the replica set specification although we are not expecting the replica set to create a new port on deployment because in case one of the pods were to fail in the future, the replica set needs to create a new one to maintain the desired number of pods. And for the replica set to create a new pod, the template definition section is required.


How to scale the Replica Set?


Let's assume we started with three replicas and then we decide to scale to six. There are multiple ways to update our replica set to scale to six replicas.

1) Update the number of replicas in the definition file to six:

replicaset-definition.yaml:

...
  replicas: 6
...


 Then run:

$ kubectl replace -f replicaset-definition.yaml


2) Run kubectl scale command by specifying replica set definition file:

$ kubectl scale --replicas=6 -f replicaset-definition.yaml

or by specifying object type name (replicaset) and object name (myapp-replicaset):

$ kubectl scale --replicas=6 replicaset myapp-replicaset
replicaset.apps/myapp-replicaset scaled


Using the file name as input will not result in the number of replicas being updated automatically in the file. The number of replicas in the replica set definition file will still be three, even though we scaled our replica set to have six replicas using the kubectl scale command and the file as input.

3) Run:

$ kubectl edit replicaset myapp-replicaset

This commands opens a copy of the ReplicaSet definition in a text editor (e.g. vi, vim):

# Please edit the object below. Lines beginning with a '#' will be ignored,
# and an empty file will abort the edit. If an error occurs while saving this file will be
# reopened with the relevant failures.
#
apiVersion: apps/v1
kind: ReplicaSet
metadata:
  creationTimestamp: "2024-05-05T22:47:22Z"
  generation: 1
  labels:
    app: myapp
    type: front-end
  name: myapp-replicaset
  namespace: default
  resourceVersion: "138523"
  uid: 6237c310-29f9-4c13-8f03-34fa70aa4307
spec:
  replicas: 3
  selector:
    matchLabels:
      type: front-end
  template:
    metadata:
      creationTimestamp: null
      labels:
        type: front-end
      name: nginx
    spec:
      containers:
      - image: nginx
        imagePullPolicy: Always
        name: nginx
        resources: {}
        terminationMessagePath: /dev/termination-log
        terminationMessagePolicy: File
      dnsPolicy: ClusterFirst
      restartPolicy: Always
      schedulerName: default-scheduler
      securityContext: {}
      terminationGracePeriodSeconds: 30
status:
  availableReplicas: 3
  fullyLabeledReplicas: 3
  observedGeneration: 1
  readyReplicas: 3
  replicas: 3
"/tmp/kubectl-edit-352613877.yaml" 45L, 1135B    

As we can see that file is stored in /tmp directory. Changes in this file are applied to ReplicaSet as soon as this file is saved.

If we change number of replicas from 3 to 4 and save this file:

$ kubectl edit replicaset myapp-replicaset
replicaset.apps/myapp-replicaset edited

...number of pods will automatically be increased to 4:

$ kubectl get pods
NAME                     READY   STATUS    RESTARTS   AGE
myapp-replicaset-5r8vj   1/1     Running   0          4s
myapp-replicaset-jnm6z   1/1     Running   0          7h25m
myapp-replicaset-nk5n9   1/1     Running   0          7h15m
myapp-replicaset-nwwmh   1/1     Running   0  

The same approach can be used to scale down.

We can use kubectl edit replicaset to change any of the attributes in its definition, not just replicas number. That can be e.g. name of the Docker image in the template. 

There are also options available for automatically scaling the replica set based on load.


How to delete Replica Set?



To delete replica set (and underlying pods):

$ kubectl delete replicaset myapp-replicaset

It is possible to specify multiple ReplicaSets for deletion:

$ kubectl delete replicaset replicaset-1 replicaset-2
replicaset.apps "replicaset-1" deleted
replicaset.apps "replicaset-2" deleted
---

Working with files in Linux

 


Creating a file


To create a file use touch:

touch filename


Writing into a file


It is possible to use redirection operators > and >> to achieve this:

  • > will overwrite existing file or crate a new file
  • >> will append text to existing file or create a new file

> file.txt

What does “>” do vs “>>”?


Next two commands have the same output - if file.txt does not exist, it will be created and string test (without quotes will be written into it):

$ echo "test" > file.txt
$ echo test > file.txt
$ cat test.txt 
test


tee command


If we want command's output to be written in the terminal but also into a file, we can pipe the  command into the tee command which takes stdin and writes it to two outputs, one being a file and another stdout. tee will crate the file if it does not exist:

$ echo test | tee test3.txt
test
$ cat test3.txt 
test



tee command can also help when we need to write into a file that only superuser has write permissions. Our current bash terminal might be running as non-privileged user and so e.g. this command will fail:

echo line >> /etc/apt/sources.list

Using

sudo echo line >> /etc/apt/sources.list

...will not help as only echo will be executed as sudo.

The solution is:

echo line | sudo tee /etc/apt/sources.list > /dev/null

> /dev/null makes stdout being dumped to /dev/null.


Ending file with new line character


[No newline at end of file]

It is a good style to always put the newline as a last character if it is allowed by the file format.

Unix historically had a convention of all human-readable text files ending in a newline. Reasons:
Practically, because many Unix tools require or expect it for proper display.
Philosophically, because each line in a text file terminates with an "end-of-line" character--the last line shouldn't be any exception.

To write into file a set of lines which end with a new line character:

$ echo $'first line\nsecond line\nthirdline\n' > foo.txt

$'...' construct expands embedded ANSI escape sequences

How to put a newline special character into a file using the echo command and redirection operator?

Difference between printf and echo:

printf "hello \n"
hello 
printf "hello " // note that new line is not appended automatically
hello $ echo "hello \n"
hello \n
echo "hello"
hello
$  // new line is appended automatically

Getting the information about a file


To get the number of lines (well, newline characters) in the file:

$ wc -l myfile.txt
23 myfile.txt

(This is why it's important to follow the convention and end each line with newline character.)

To get the number of words on some webpage:

$ curl "https://example.com/" 2>/dev/null | grep -i "word1|word2" | wc -l


To see the last couple of lines in the file use command tail:

tail myfile

To find various hash sums of a file:

md5sum file_name
sha1sum file_name
sha256sum file_name


If file is Windows executable, it is possible to examine it with:

exiftool somefile.exe 

To install exiftool:

$ sudo apt install libimage-exiftool-perl

linux - viewing dll and exe file properties/attributes via the command line - Unix & Linux Stack Exchange

Checking whether file exists


if test -f "$symlink_file"; then
   echo "$symlink_file" exists and is regular file.
else
   echo "$symlink_file" does not exist or is not a regular file.
fi

if test -L "$regular_file"; then
   echo "$regular_file" exists and is symlink file.
else
   echo "$regular_file" does not exist or is not a symlink file.
fi

How to Check if a File or Directory Exists in Bash

Copying files


cp - copy

cp [OPTIONS] SOURCE DEST 
SOURCE - file or directory
DEST - file or directory

An error is reported if directory is specified as source and file as destination.

$ cp -r test test.txt
cp: cannot overwrite non-directory 'test.txt' with directory 'test'


Copying files and directories to/from remote machine


 

Moving files


mv *.{jpg,gif,png} ~/Pictures

Renaming files


To rename all .new files in the current directory to *.old:

rename -f -v 's/.new/.old/' *

-f = force; allows overwriting existing *.old files
-v = verbose

File viewing and editing

To simply view the content of some file, use cat:

cat filename

To edit some file, you can use vi editor. Example:

vi ~/.profile 

gedit can also be used as graphic editor:

sudo gedit ~/.profile

To enter some special character (e.g. bulletpoint) press CTRL+SHIFT+U and underscored "u" should appear (u). Then use numeric keyboard to type in the Unicode code of the character (e.g. 2022) and press Enter. [source]

To see the content of the file as binary and hexadecimal:

xxd -b file
xxd file


Searching for Files


To search file from the root directory use /:

$ find / -name "file_name.ext"

To find any file or directory which contains some string in their name, recursively, starting from the current directory:

$ find . -name "*STRING*" 

Searching for text across files


How do I find all files containing specific text on Linux?

man grep

grep -rnw '/path/to/somewhere/' -e 'pattern'

-r or -R = recursive,
-n = line number
-w = match the whole word.
-l (lower-case L) = just give the file name of matching files
--include=\*.{c,h} =  search through those files which have .c or .h extensions
--exclude=*.o = exclude searching all the files ending with .o extension
--exclude-dir={dir1,dir2,*.dst} = exclude a particular directory(ies)
-e PATTERN = string pattern to be searched
-i = ignore the case

Example:

$ grep -r /var/lib/go/src/ -e "CodeDecode"
/var/lib/go/src/encoding/json/bench_test.go:func BenchmarkCodeDecoder(b *testing.B) {


Example:

$ find ./go/src/pkg -type f -name "*.go" | xargs egrep '^type.*(er|or) interface {'

xargs manual - xargs builds and executes command lines from standard input
egrep manual - egrep prints lines matching a pattern

Comparing Files


How to ignore line endings when comparing files?

$ diff --strip-trailing-cr file1 file2


How to detect file ends in newline?


Working with executable files


Running a command prefixed by the time command will tell us how long our code took to execute.

time myapp
real 0m13.761s
user 0m0.262s
sys 0m0.039s

If an executable is present but some of its dependencies are missing bash (or sh) might display an error messages stating that main executable is not found (which might be a bit misleading).

Example:

/ # ls
bin       myapp      data-vol  dev       etc       home      lib       media     mnt       opt       proc      root      run       sbin      srv       sys       tmp       usr       var
/ #  myapp
/bin/sh:  myappnot found