These are custom notes that extend my notes from an Udemy course "Kubernetes for the Absolute Beginners - Hands-on". All course content rights belong to course creators.
For starting with Kubernetes and local tests we can use VMs running on a local machine so our choice is Minikube.
A Kubernetes cluster can be deployed on either physical or virtual machines. To get started with Kubernetes development, we can use Minikube. Minikube is a lightweight Kubernetes implementation that creates a VM on our local machine and deploys a simple cluster containing only one node. Minikube is available for Linux, macOS, and Windows systems. The Minikube CLI provides basic bootstrapping operations for working with your cluster, including start, stop, status, and delete. [Using Minikube to Create a Cluster | Kubernetes]
 |
| source: https://kubernetes.io/ |
In Introduction to Kubernetes | My Public Notepad we listed components that need to be installed on the master and worker nodes. Minikube blends together master and worker into a single node which is run on a local machine, in VM. Minikube packages together all these components into a single ISO image which can be downloaded and run in VM.
- Our goal is to install / set up a a basic Kubernetes cluster on our local machine using the minikube utility
- minikube creates one Node cluster, where the master and worker processes are on the same machine
- minikube creates a non-production sandbox environment that we can use to test out Kubernetes
- minikube is used for quick tests within the single cluster, in a single host on the local machine before launching the full blown Kubernetes multi-node cluster.
 |
| source: Kubernetes Local Setup Using Minikube and Kubectl | by Ankit Maheshwari | AWS in Plain English |
Minikube Installation Prerequisites
Before installing minikube:
- kubectl command line tool is what we will use to manage our Kubernetes resources and our cluster after it is set up using minikube
- Installing the kubectl utility before installing minikube will allow minikube to configure the kubectl utility to work with the cluster when it provisions it
- kubectl utility can work with multiple clusters, local or remote, at the same time
- if kubectl is not installed locally, minikube already includes kubectl which can be used like this: minikube kubectl -- <kubectl commands>
- if kubectl is already installed, minikube will automatically take care of configuring it when it starts, when it provisions a Kubernetes cluster
- To install kubectl (on Linux), follow instructions from here: Install and Set Up kubectl on Linux | Kubernetes
kubectl needs to be configured so it knows to which cluster it needs to talk to (to which master node). $KUBECONFIG environment variable contains the path to the directory that contains minikube configuration and which is usually ~/.kube/.
Upon fresh kubectl installation, before it's configured, ~/.kube/ is not created and KUBECONFIG is not specified.
Default kubectl config file is ~/.kube/config. It initially does not exist and before we use kubectl to talk to Kubernetes cluster we need to create it or it can be created by Minikube on its startup (be careful as Minikube might overwrite existing kubeconfig file!)
To learn more about kubeconfig file: What is a kubeconfig file? | Enable Sysadmin
2) We need to make sure that virtualization is enabled.
On Linux, it is enabled if the following command returns non-empty output:
$ grep -E --color 'vmx|svm' /proc/cpuinfo
If it's not enabled, there should be an option in BIOS to enable virtualization.
3) We need to install a virtual machine manager (hypervisor) or container manager/engine such as: Docker, QEMU, Hyperkit, Hyper-V, KVM, Parallels, Podman, VirtualBox, or VMware Fusion/Workstation.
We can use e.g. VirtualBox virtualization solution but minikube can also run without a hypervisor, directly on the host using Docker.
The Docker driver allows you to install Kubernetes into an existing Docker install. On Linux, this does not require virtualization to be enabled. [docker | minikube]
VirtualBox is minikube’s original driver. It may not provide the fastest start-up time, but it is the most stable driver available for users of Microsoft Windows Home. [virtualbox | minikube]
VirtualBox installation: Linux_Downloads – Oracle VM VirtualBox
When we provision a cluster using minikube, it will automatically create a virtual machine as required.
Minikube Installation
Installation on Linux (Ubuntu)
$ sudo apt update && sudo apt upgrade
$ curl -LO https://storage.googleapis.com/minikube/releases/latest/minikube-linux-amd64
$ sudo install minikube-linux-amd64 /usr/local/bin/minikube && rm minikube-linux-amd64
$ curl -LO https://storage.googleapis.com/minikube/releases/latest/minikube-linux-amd64
$ sudo install minikube-linux-amd64 /usr/local/bin/minikube && rm minikube-linux-amd64
Installation verification:
$ which minikube
/usr/local/bin/minikube
Minikube Configuration
Upon minikube installation, we have a new directory that minikube uses for state/configuration: .minikube. By default, it is created in home directory (~/) and it contains several sub-directories, all of which are initially empty:
$ ls -la ~/.minikube/
total 36
drwxr-xr-x .
drwxr-xr-x ..
drwxr-xr-x addons
drwxr-xr-x cache
drwxr-xr-x certs
drwxr-xr-x config
drwxr-xr-x files
drwxr-xr-x logs
drwxr-xr-x machines
MINIKUBE_HOME environment variable contains the path to the .minikube directory but if it's not specified, it defaults to ~/.minikube. For more details, see: Configuration | minikube.
Minikube CLI
Let's explore minikube CLI:
$ minikube --help
minikube provisions and manages local Kubernetes clusters optimized for development workflows.
Basic Commands:
start Starts a local Kubernetes cluster
status Gets the status of a local Kubernetes cluster
stop Stops a running local Kubernetes cluster
delete Deletes a local Kubernetes cluster
dashboard Access the Kubernetes dashboard running within
the minikube cluster
pause pause Kubernetes
unpause unpause Kubernetes
Images Commands:
docker-env Provides instructions to point your terminal's docker-cli to the Docker Engine inside minikube.
(Useful for building docker images directly inside minikube)
podman-env Configure environment to use minikube's Podman service
cache Manage cache for images
image Manage images
Configuration and Management Commands:
addons Enable or disable a minikube addon
config Modify persistent configuration values
profile Get or list the current profiles (clusters)
update-context Update kubeconfig in case of an IP or port change
Networking and Connectivity Commands:
service Returns a URL to connect to a service
tunnel Connect to LoadBalancer services
Advanced Commands:
mount Mounts the specified directory into minikube
ssh Log into the minikube environment (for debugging)
kubectl Run a kubectl binary matching the cluster version
node Add, remove, or list additional nodes
cp Copy the specified file into minikube
Troubleshooting Commands:
ssh-key Retrieve the ssh identity key path of the specified node
ssh-host Retrieve the ssh host key of the specified node
ip Retrieves the IP address of the specified node
logs Returns logs to debug a local Kubernetes cluster
update-check Print current and latest version number
version Print the version of minikube
options Show a list of global command-line options (applies to all commands).
Other Commands:
completion Generate command completion for a shell
license Outputs the licenses of dependencies to a directory
Use "minikube <command> --help" for more information about a given command.
Creating the Minikube Cluster
To test minikube installation we can provision a Kubernetes cluster:
$ minikube start
😄 minikube v1.33.0 on Ubuntu 22.04
🆕 Kubernetes 1.30.0 is now available. If you would like to upgrade, specify: --kubernetes-version=v1.30.0
✨ Using the virtualbox driver based on existing profile
💿 Downloading VM boot image ...
> minikube-v1.33.0-amd64.iso....: 65 B / 65 B [---------] 100.00% ? p/s 0s
> minikube-v1.33.0-amd64.iso: 314.16 MiB / 314.16 MiB 100.00% 3.72 MiB p/
👍 Starting "minikube" primary control-plane node in "minikube" cluster
🔄 Restarting existing virtualbox VM for "minikube" ...
❗ Image was not built for the current minikube version. To resolve this you can delete and recreate your minikube cluster using the latest images. Expected minikube version: v1.32.0 -> Actual minikube version: v1.33.0
🐳 Preparing Kubernetes v1.28.3 on Docker 24.0.7 ...| Bad local forwarding specification '0:localhost:8443'
🔗 Configuring bridge CNI (Container Networking Interface) ...
🔎 Verifying Kubernetes components...
▪ Using image gcr.io/k8s-minikube/storage-provisioner:v5
🌟 Enabled addons: storage-provisioner, default-storageclass
❗ /usr/local/bin/kubectl is version 1.30.0, which may have incompatibilities with Kubernetes 1.28.3.
▪ Want kubectl v1.28.3? Try 'minikube kubectl -- get pods -A'
🏄 Done! kubectl is now configured to use "minikube" cluster and "default" namespace by default
We can also explicitly specify the driver (virtualization tool) to be used e.g. VirtualBox:
$ minikube start --driver=virtualbox
minikube downloaded the minikube ISO image for mini cube. This image is then used to provision a VM on VirtualBox. It downloaded Kubernetes version 1.28.3 and any other required binaries.
We can open VirtualBox UI we can see that a virtual machine by the name minikube has been created and it is in a running state:
We can open VirtualBox UI we can see that a virtual machine by the name minikube has been created and it is in a running state:
Note the last line in the minikube start output. Minikube set up ~/.kube/config to point to minikube cluster. kubectl utility is now configured to use the Kubernetes cluster provisioned using minikube.
So, before we run kubectl for the first time and we want to use Minikube cluster, we first need to run minikube start.
To check kubectl configuration:
$ kubectl config view
apiVersion: v1
clusters:
- cluster:
certificate-authority: /home/bojan/.minikube/ca.crt
extensions:
- extension:
last-update: Sun, 12 May 2024 22:31:20 BST
provider: minikube.sigs.k8s.io
version: v1.33.0
name: cluster_info
server: https://192.168.59.100:8443
name: minikube
contexts:
- context:
cluster: minikube
extensions:
- extension:
last-update: Sun, 12 May 2024 22:31:20 BST
provider: minikube.sigs.k8s.io
version: v1.33.0
name: context_info
namespace: default
user: minikube
name: minikube
current-context: minikube
kind: Config
preferences: {}
users:
- name: minikube
user:
client-certificate: /home/bojan/.minikube/profiles/minikube/client.crt
client-key: /home/bojan/.minikube/profiles/minikube/client.key
If we run some kubectl command but we didn't start minikube we'll get the error:
$ kubectl get deployment
E0513 12:37:29.927893 8136 memcache.go:265] couldn't get current server API group list: Get "https://192.168.59.100:8443/api?timeout=32s": dial tcp 192.168.59.100:8443: i/o timeout
192.168.59.100 is the IP address of the apiserver node (a singe node acting both as master and worker) in case of Minikube) and 8443 is the default listening port on it.
Solution here is to run minikube start.
On one of the subsequent runs of minikube start I got the following output:
$ minikube start
😄 minikube v1.33.0 on Ubuntu 22.04
🆕 Kubernetes 1.30.0 is now available. If you would like to upgrade, specify: --kubernetes-version=v1.30.0
✨ Using the virtualbox driver based on existing profile
👍 Starting "minikube" primary control-plane node in "minikube" cluster
🔄 Restarting existing virtualbox VM for "minikube" ...
❗ Image was not built for the current minikube version. To resolve this you can delete and recreate your minikube cluster using the latest images. Expected minikube version: v1.32.0 -> Actual minikube version: v1.33.0
🐳 Preparing Kubernetes v1.28.3 on Docker 24.0.7 ...- Bad local forwarding specification '0:localhost:8443'
🔗 Configuring bridge CNI (Container Networking Interface) ...
▪ Using image docker.io/kubernetesui/dashboard:v2.7.0
▪ Using image docker.io/kubernetesui/metrics-scraper:v1.0.8
▪ Using image gcr.io/k8s-minikube/storage-provisioner:v5
╭───────────────────────────────────────────────────────────────────────────────────────────────────╮
│ │
│ You have selected "virtualbox" driver, but there are better options ! │
│ For better performance and support consider using a different driver: │
│ - kvm2 │
│ - qemu2 │
│ │
│ To turn off this warning run: │
│ │
│ $ minikube config set WantVirtualBoxDriverWarning false │
│ │
│ │
│ To learn more about on minikube drivers checkout https://minikube.sigs.k8s.io/docs/drivers/ │
│ To see benchmarks checkout https://minikube.sigs.k8s.io/docs/benchmarks/cpuusage/ │
│ │
╰───────────────────────────────────────────────────────────────────────────────────────────────────╯
🔎 Verifying Kubernetes components...
💡 Some dashboard features require the metrics-server addon. To enable all features please run:
minikube addons enable metrics-server
🌟 Enabled addons: storage-provisioner, default-storageclass, dashboard
❗ /usr/local/bin/kubectl is version 1.30.0, which may have incompatibilities with Kubernetes 1.28.3.
▪ Want kubectl v1.28.3? Try 'minikube kubectl -- get pods -A'
🏄 Done! kubectl is now configured to use "minikube" cluster and "default" namespace by default
If we already have kubeconfig file set for some non-Minikube cluster, we can make Minikube create its own kubeconfig at some different location or with different name by changing the value of KUBECONFIG env variable:
$ KUBECONFIG=/home/user/.kube/minikube-config minikube start
To ensure that everything has been set up correctly we'll run:
$ minikube status
minikube
type: Control Plane
host: Running
kubelet: Running
apiserver: Running
kubeconfig: Configured
Our cluster is now set up. We will deploy some applications on the cluster in order to make sure it's working as expected.
To check if kubectl commands are working and also check how many nodes are in the cluster we can run:
To check if kubectl commands are working and also check how many nodes are in the cluster we can run:
$ kubectl get node
NAME STATUS ROLES AGE VERSION
minikube Ready control-plane 10d v1.28.3
We can see that it is a single node cluster with node running Kubernetes v1.28.3.
ROLES can also have a value master.
$ kubectl get node
NAME STATUS ROLES AGE VERSION
controlplane Ready control-plane,master 19m v1.29.0+k3s1
Some more examples:
$ kubectl get nodes
NAME STATUS ROLES AGE VERSION
controlplane Ready control-plane,master 30m v1.29.0+k3s1
If we specify output format via -o (--output) argument as wide, we'll get a bit more info about the nodes:
$ kubectl get nodes -o wide
NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME
controlplane Ready control-plane,master 30m v1.29.0+k3s1 192.10.75.9 <none> Alpine Linux v3.16 5.4.0-1106-gcp containerd://1.7.11-k3s2
To get the more details about the cluster:
$ kubectl cluster-info
Kubernetes control plane is running at https://127.0.0.1:6443
CoreDNS is running at https://127.0.0.1:6443/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy
Metrics-server is running at https://127.0.0.1:6443/api/v1/namespaces/kube-system/services/https:metrics-server:https/proxy
To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.
To get more detailed information about cluster (and here we filter out info about node(s)):
$ kubectl cluster-info dump | grep node
"node-role.kubernetes.io/control-plane": "true",
"node-role.kubernetes.io/master": "true",
"node.kubernetes.io/instance-type": "k3s"
"alpha.kubernetes.io/provided-node-ip": "192.10.75.9",
"k3s.io/node-args": "[\"server\",\"--advertise-address\",\"192.10.75.9\",\"--tls-san\",\"controlplane\",\"--flannel-iface\",\"eth0\"]",
"k3s.io/node-config-hash": "QSE3CODHPKSU74VVY6TG2XX6OZXX5OMS37YI54QW2G4MHIETIL2Q====",
"k3s.io/node-env": "{\"K3S_DATA_DIR\":\"/var/lib/rancher/k3s/data/e5efd5aeb0cb8e4c1d802582bd7085968576e89d8f34ca22a450b4f4ae4d4c15\"}",
"node.alpha.kubernetes.io/ttl": "0",
"wrangler.cattle.io/node"
"nodeInfo": {
"nodePort": 30812
"nodePort": 31811
...
To get more information about the node of interest:
$ kubectl describe node controlplane
Name: controlplane
Roles: control-plane,master
Labels: beta.kubernetes.io/arch=amd64
beta.kubernetes.io/instance-type=k3s
beta.kubernetes.io/os=linux
kubernetes.io/arch=amd64
kubernetes.io/hostname=controlplane
kubernetes.io/os=linux
node-role.kubernetes.io/control-plane=true
node-role.kubernetes.io/master=true
node.kubernetes.io/instance-type=k3s
Annotations: alpha.kubernetes.io/provided-node-ip: 192.10.75.9
flannel.alpha.coreos.com/backend-data: {"VNI":1,"VtepMAC":"02:0e:eb:5a:ab:97"}
flannel.alpha.coreos.com/backend-type: vxlan
flannel.alpha.coreos.com/kube-subnet-manager: true
flannel.alpha.coreos.com/public-ip: 192.10.75.9
k3s.io/hostname: controlplane
k3s.io/internal-ip: 192.10.75.9
k3s.io/node-args: ["server","--advertise-address","192.10.75.9","--tls-san","controlplane","--flannel-iface","eth0"]
k3s.io/node-config-hash: QSE3CODHPKSU74VVY6TG2XX6OZXX5OMS37YI54QW2G4MHIETIL2Q====
k3s.io/node-env: {"K3S_DATA_DIR":"/var/lib/rancher/k3s/data/e5efd5aeb0cb8e4c1d802582bd7085968576e89d8f34ca22a450b4f4ae4d4c15"}
node.alpha.kubernetes.io/ttl: 0
volumes.kubernetes.io/controller-managed-attach-detach: true
CreationTimestamp: Mon, 06 May 2024 06:19:39 +0000
Taints: <none>
Unschedulable: false
Lease:
HolderIdentity: controlplane
AcquireTime: <unset>
RenewTime: Mon, 06 May 2024 06:45:50 +0000
Conditions:
Type Status LastHeartbeatTime LastTransitionTime Reason Message
---- ------ ----------------- ------------------ ------ -------
MemoryPressure False Mon, 06 May 2024 06:45:11 +0000 Mon, 06 May 2024 06:19:39 +0000 KubeletHasSufficientMemory kubelet has sufficient memory available
DiskPressure False Mon, 06 May 2024 06:45:11 +0000 Mon, 06 May 2024 06:19:39 +0000 KubeletHasNoDiskPressure kubelet has no disk pressure
PIDPressure False Mon, 06 May 2024 06:45:11 +0000 Mon, 06 May 2024 06:19:39 +0000 KubeletHasSufficientPID kubelet has sufficient PID available
Ready True Mon, 06 May 2024 06:45:11 +0000 Mon, 06 May 2024 06:19:39 +0000 KubeletReady kubelet is posting ready status
Addresses:
InternalIP: 192.10.75.9
Hostname: controlplane
Capacity:
cpu: 36
ephemeral-storage: 1016057248Ki
hugepages-1Gi: 0
hugepages-2Mi: 0
memory: 214587056Ki
pods: 110
Allocatable:
cpu: 36
ephemeral-storage: 988420490080
hugepages-1Gi: 0
hugepages-2Mi: 0
memory: 214587056Ki
pods: 110
System Info:
Machine ID:
System UUID: 82ad48cc-cf0c-c0bf-7b55-2c21495826a1
Boot ID: 6914c0b2-ef4d-4ff1-9180-75737388e9af
Kernel Version: 5.4.0-1106-gcp
OS Image: Alpine Linux v3.16
Operating System: linux
Architecture: amd64
Container Runtime Version: containerd://1.7.11-k3s2
Kubelet Version: v1.29.0+k3s1
Kube-Proxy Version: v1.29.0+k3s1
PodCIDR: 10.42.0.0/24
PodCIDRs: 10.42.0.0/24
ProviderID: k3s://controlplane
Non-terminated Pods: (5 in total)
Namespace Name CPU Requests CPU Limits Memory Requests Memory Limits Age
--------- ---- ------------ ---------- --------------- ------------- ---
kube-system coredns-6799fbcd5-hl6zl 100m (0%) 0 (0%) 70Mi (0%) 170Mi (0%) 26m
kube-system local-path-provisioner-84db5d44d9-qgvm2 0 (0%) 0 (0%) 0 (0%) 0 (0%) 26m
kube-system svclb-traefik-f39a148c-87mtw 0 (0%) 0 (0%) 0 (0%) 0 (0%) 25m
kube-system metrics-server-67c658944b-5dsjt 100m (0%) 0 (0%) 70Mi (0%) 0 (0%) 26m
kube-system traefik-f4564c4f4-4qvgx 0 (0%) 0 (0%) 0 (0%) 0 (0%) 25m
Allocated resources:
(Total limits may be over 100 percent, i.e., overcommitted.)
Resource Requests Limits
-------- -------- ------
cpu 200m (0%) 0 (0%)
memory 140Mi (0%) 170Mi (0%)
ephemeral-storage 0 (0%) 0 (0%)
hugepages-1Gi 0 (0%) 0 (0%)
hugepages-2Mi 0 (0%) 0 (0%)
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Starting 26m kube-proxy
Normal NodePasswordValidationComplete 26m k3s-supervisor Deferred node password secret validation complete
Normal Synced 26m cloud-node-controller Node synced successfully
Normal Starting 26m kubelet Starting kubelet.
Warning InvalidDiskCapacity 26m kubelet invalid capacity 0 on image filesystem
Normal NodeHasSufficientMemory 26m (x2 over 26m) kubelet Node controlplane status is now: NodeHasSufficientMemory
Normal NodeHasNoDiskPressure 26m (x2 over 26m) kubelet Node controlplane status is now: NodeHasNoDiskPressure
Normal NodeHasSufficientPID 26m (x2 over 26m) kubelet Node controlplane status is now: NodeHasSufficientPID
Normal NodeAllocatableEnforced 26m kubelet Updated Node Allocatable limit across pods
Normal NodeReady 26m kubelet Node controlplane status is now: NodeReady
Normal RegisteredNode 26m node-controller Node controlplane event: Registered Node controlplane in Controller
To create a deployment using this cluster:
$ kubectl create deployment hello-minikube --image=kicbase/echo-server:1.0
deployment.apps/hello-minikube created
To check it:
$ kubectl get deployments
NAME READY UP-TO-DATE AVAILABLE AGE
hello-minikube 1/1 1 1 68s
To expose this deployment as a service on port 8080:
$ kubectl expose deployment hello-minikube --type=NodePort --port=8080
service/hello-minikube exposed
To get info on the service:
$ kubectl get services hello-minikube
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
hello-minikube NodePort 10.104.92.181 <none> 8080:30235/TCP 4m5s
To get the URL of the exposed service:
$ minikube service hello-minikube --url
http://192.168.59.100:30235
Cleanup:
$ kubectl delete services hello-minikube
service "hello-minikube" deleted
$ kubectl delete deployment hello-minikube
deployment.apps "hello-minikube" deleted
$ kubectl get pods
No resources found in default namespace.
Dashboard
Kubernetes provides a Web-based UI Dashboard which can also be launched for Minikube.
We need to make sure that Minikube cluster is running:
$ minikube dashboard --url
🤷 The control-plane node minikube host is not running: state=Stopped
👉 To start a cluster, run: "minikube start"
Fix:
$ minikube start
Let's try again:
$ minikube dashboard --url
🤔 Verifying dashboard health ...
🚀 Launching proxy ...
🤔 Verifying proxy health ...
http://127.0.0.1:36767/api/v1/namespaces/kubernetes-dashboard/services/http:kubernetes-dashboard:/proxy/
We can now open this URL in the browser:
To continue your Kubernetes learning journey, read the next article in these series: Managing Pods In A Minikube ClusterMy Public Notepad |
No comments:
Post a Comment