Core components of a Kubernetes Cluster

Kubernetes is an open source platform for managing containerized workloads and services that facilitates declarative configuration and automation. The name Kubernetes had originated from Greek, meaning helmsman or pilot. It is portable as well as extensible and has a rapidly growing ecosystem. Kubernetes’ services and tools are widely available.

In this article, we will be going through a 10,000-foot view of the main Kubernetes components, from what each container is composed of, to how a container in a pod is deployed and scheduled across each of the workers. It is crucial to understand full particulars of the Kubernetes cluster in order to be able to deploy and design a solution based on Kubernetes as an orchestrator for containerized applications.

Here is a brief regarding the things that we are going to cover in this article:

Control panel components
The Kubernetes worker’s components
Pods as basic building blocks
Kubernetes services, load balancers, and Ingress controllers
Kubernetes deployments and Daemon Sets
Persistent storage in Kubernetes

The Kubernetes Control Plane

The Kubernetes master nodes are where the core control plane services live; not all services have to reside on the same node; however, for centralization and practicality, they are often deployed this way. This obviously raises services availability questions; however, they can easily be overcome by having several nodes and providing load balancing requests to achieve a highly available set of master nodes.

The master nodes are composed of four basic services:

The kube-apiserver
The kube-scheduler
The kube-controller-manager
The etcd database

Master nodes can either run on bare metal servers, virtual machines, or a private or public cloud, but it is not recommended to run container workloads on them. We will see more on this later.

The following diagram shows the Kubernetes master nodes components:

Kubernetes master nodes components

The kube-apiserver

The API server is what ties everything together. It is the frontend REST API of the cluster that receives manifests to create, update, and delete API objects such as services, pods, Ingress, and others.

The kube-apiserver is the only service that we should be talking to; it is also the only one that writes and talks to the etcd database for registering the cluster state. With the kubectl command, we will send commands to interact with it. This will be our Swiss Army knife when it comes to Kubernetes.

The kube-controller-manager

The kube-controller-manager daemon, in a nutshell, is a set of infinite control loops that are shipped for simplicity in a single binary. It watches for the defined desired state of the cluster and it makes sure that it is accomplished and satisfied by moving all the bits and pieces necessary to achieve it. The kube-controller-manager is not just one controller; it contains several different loops that watch different components in the cluster. Some of them are the service controller, the namespace controller, the service account controller, and many others. You can find each controller and its definition in the Kubernetes GitHub repository: https://github.com/kubernetes/kubernetes/tree/master/pkg/controller.

The kube-scheduler

The kube-scheduler schedules your newly created pods to nodes with enough space to satisfy the pods' resource needs. It basically listens to the kube-apiserver and the kube-controller-manager for newly created pods that are put into a queue and then scheduled to an available node by the scheduler. The kube-scheduler definition can be found here: https://github.com/kubernetes/kubernetes/blob/master/pkg/scheduler.

Besides compute resources, the kube-scheduler also reads the nodes' affinity and anti-affinity rules to find out whether a node can or cannot run that pod.

The etcd database

The etcd database is a very reliable consistent key-value store that's used to store the state of the Kubernetes cluster. It contains the current status of the pods in which the node is running on, how many nodes the cluster currently has, what is the state of those nodes, how many replicas of deployment are running, services names, and others.

As we mentioned before, only the kube-apiserver talks to the etcd database. If the kube-controller-manager needs to check the state of the cluster, it will go through the API server in order to get the state from the etcd database, instead of querying the etcd store directly. The same happens with the kube-scheduler if the scheduler needs to make it known that a pod has been stopped or allocated to another node; it will inform the API server, and the API server will store the current state in the etcd database.

With etcd, we have covered all the main components for our Kubernetes master nodes so that we are ready to manage our cluster. But a cluster is not only composed of masters; we still require the nodes that will be performing the heavy lifting by running our applications.

Kubernetes Worker Nodes

The worker nodes that perform this task in Kubernetes are simply called nodes. Previously, around 2014, they were called minions, but this term was later replaced with just nodes, as the name was confusing with Salt's terminologies and made people think that Salt was playing a major role in Kubernetes.

These nodes are the only place that you will be running workloads, as it is not recommended to have containers or loads on the master nodes, as they need to be available to manage the entire cluster. The nodes are very simple in terms of components; they only require three services to fulfil their task:

Kubelet
Kube-proxy
Container runtime

Let's explore these three components in a little bit more depth.

The kubelet

The kubelet is a low-level Kubernetes component and one of the most important ones after the kube-apiserver; both of these components are essential for the provisioning of pods/containers in the cluster. The kubelet is a service that runs on the Kubernetes nodes and listens to the API server for pod creation. The kubelet is only in charge of starting/ stopping and making sure that containers in pods are healthy; the kubelet will not be able to manage any containers that were not created by it.

The kubelet achieves the goals by talking to the container runtime via container runtime interface (CRI). The CRI provides pluggability to the kubelet via a gRPC client, which is able to talk to different container runtimes. As we mentioned earlier, Kubernetes supports multiple container runtimes to deploy containers, and this is how it achieves such diverse support for different engines.

You can check for kubelet’s source code via https://github.com/kubernetes/kubernetes/tree/master/pkg/kubelet.

The kube-proxy

The kube-proxy is a service that resides on each node of the cluster and is the one that makes communications between pods, containers, and nodes possible. This service watches the kube-apiserver for changes on defined services (service is a sort of logical load balancer in Kubernetes; we will dive deeper into services later on in this article) and keeps the network up to date via iptables rules that forward traffic to the correct endpoints. Kube-proxy also sets up rules in iptables that do random load balancing across pods behind a service.

Here is an example of an iptables rule that was made by the kube-proxy:

-A KUBE-SERVICES -d 10.0.162.61/32 -p tcp -m comment --comment "default/example: has no endpoints" -m tcp --dport 80 -j REJECT --reject-with icmp-port-unreachable

Note that this is a service with no endpoints (no pods behind it).

Container runtime

To be able to spin up containers, we require a container runtime. This is the base engine that will create the containers in the nodes kernel for our pods to run. The kubelet will be talking to this runtime and will spin up or stop our containers on demand.

Currently, Kubernetes supports any OCI-compliant container runtime, such as Docker, rkt, runc, runsc, and so on.

You can refer this https://github.com/opencontainers/runtime-spec to learn more about all the specifications from the OCI Git-Hub page.

Now that we have explored all the core components that form a cluster, let’s now take a look at what can be done with them and how Kubernetes is going to help us orchestrate and manage our containerized applications.

Kubernetes Objects

Kubernetes objects are exactly that: they are logical persistent objects or abstractions that will represent the state of your cluster. You are the one in charge of telling Kubernetes what your desired state of that object is so that it can work to maintain it and make sure that the object exists.

To create an object, there are two things that it needs to have: a status and its spec. The status is provided by Kubernetes, and it is the current state of the object. Kubernetes will manage and update that status as needed to be in accordance with your desired state. The spec field, on the other hand, is what you provide to Kubernetes, and is what you tell it to describe the object you desire. For example, the image that you want the container to be running, the number of containers of that image that you want to run, and so on.

Each object has specific spec fields for the type of task that they perform, and you will be providing these specifications on a YAML file that is sent to the kube-apiserver with kubectl, which that transforms it into JSON and sends it as an API request. We will dive deeper into each object and its spec fields later in this article.

Here is an example of a YAML that was sent to kubectl:

cat << EOF | kubectl create -f -kind: ServiceapiVersion: v1metadata: Name: frontend-servicespec: selector: web: frontend ports: - protocol: TCP port: 80 targetPort: 9256EOF

The basic fields of the object definition are the very first ones, and these ones will not vary from object to object and are very self-explanatory. Let's take a quick look at them:

kind: The kind field tells Kubernetes what type of object you are defining: a pod, a service, a deployment, and so on
apiVersion: Because Kubernetes supports multiple API versions, we need to specify a REST API path that we want to send our definition to
metadata: This is a nested field, which means that you have several more subfields to metadata, where you will write basic definitions such as the name of your object, assigning it to a specific namespace, and also tag a label to it to relate your object to other Kubernetes objects

So, we have now been through the most-used fields and their contents; you can learn more about the Kuberntes API conventions at https://github.com/kubernetes/community/blob/master/contributors/devel/api-conventions.md

Some of the fields of the object can later be modified after the object has been created, but that will depend on the object and the field that you want to modify.

The following is a short list of the various Kubernetes objects that you can create:

Pod
Volume
Service
Deployment
Ingress
Secret
ConfigMap

And there are many more.

Let's take a closer look at each one of these items.

Pods – the basis of Kubernetes

Pods are the most basic objects in Kubernetes and also the most important ones. Everything revolves around them; we can say that Kubernetes is for the pods! All of the other objects are here to serve them, and all the tasks that they do are to make the pods achieve your desired state.

So, what is a pod and why are pods so important?

A pod is a logical object that runs one or more containers together on the same network namespace, the same inter-process communication (IPC), and, sometimes, depending on the version of Kubernetes, the same process ID (PID) namespace. This is because they are the ones that are going to run our containers and hence will be the center of attention. The whole point of Kubernetes is to be a container orchestrator, and with pods, we make orchestration possible.

As we mentioned before, containers on the same pod live in a "bubble" where they can talk to one another via localhost, as they are local to one another. One container in a pod has the same IP address as the other container because they are sharing a network namespace, but in most cases, you will be running on a one-on-one basis, that is to say, a single container per pod. Multiple containers per pod are only used on very specific scenarios, such as when an application requires a helper such as a data pusher or a proxy that needs to communicate in a fast and resilient way with the primary application.

The way you define a pod is the same way you would do so for any other Kubernetes object: via a YAML that contains all the pod specs and definitions:

kind: PodapiVersion: v1metadata:name: hello-podlabels: hello: podspec: containers: - name: hello-container image: alpine args: - echo - "Hello World"

Let's go through the basic pod definitions needed under the spec field to create our pod:

Containers: A container is an array; therefore, we have a set of several subfields under it. Basically, it's what defines the containers that are going to be running on the pod. We can specify a name for the container, the image that is going to be a spin-off from, and the arguments or command that we need it to run. The difference between arguments and commands is the same as the difference between CMDand ENTRYPOINT. Take note that all the fields that we just went through are for the containers array. They are not directly part of the spec of the pod.
restartPolicy: This field is exactly that: it tells Kubernetes what to do with a container, and it applies to all the containers in the pod in the case of a zero or non-zero exit code. You can choose from either option, Never, OnFailure or Always. Always will be the default in case a restartPolicy is not defined.

These are the most basic specs that you are going to declare on a pod; other specs will require you have a little bit more background knowledge on how to use them and how they interact with various other Kubernetes objects. We will revisit them later in this article; some of them are as follows:

Volume
Env
Ports
dnsPolicy
initContainers
nodeSelector
Resource limits and requests

To view the pods that are currently running in your cluster, you can run kubectl get pods:

dsala@MININT-IB3HUA8:~$ kubectl get podsNAME READY STATUS RESTARTS AGEbusybox 1/1 Running 120 5d

Alternatively, you can run kubectl describe pods without specifying any pod. This will print out a description of every pod running in the cluster. In this case, it will be only the busybox pod, as it is the only one that's currently running:

dsala@MININT-IB3HUA8:~$ kubectl describe podsName: busyboxNamespace: defaultPriority: 0PriorityClassName: <none>Node: aks-agentpool-10515745-2/10.240.0.6Start Time: Wed, 19 Sep 2018 14:23:30 -0600Labels: <none>Annotations: <none>Status: RunningIP: 10.244.1.7Containers: busybox:[...] (Output truncated for readability)Events:Type Reason Age From Message---- ------ ---- ---- -------Normal Pulled 45s (x121 over 5d) kubelet, aks-agentpool-10515745-2 Container image "busybox" already present on machineNormal Created 44s (x121 over 5d) kubelet, aks-agentpool-10515745-2 Created containerNormal Started 44s (x121 over 5d) kubelet, aks-agentpool-10515745-2 Started container

Pods are mortal. Once it dies or is deleted, it cannot be recovered. Its IP and the containers that were running on it will be gone; they are totally ephemeral. The data on the pods that is mounted as a volume may or may not survive, depending on how you set it up. If our pods die and we lose them, how do we ensure that all our microservices are running? Well, deployments are the answer.

Deployments

Pods by themselves are not very useful since it is not very efficient to have more than a single instance of our application running in a single pod. Provisioning hundreds of copies of our application on different pods without having a method to look for them all will get out of hand really quickly.

This is where deployments come into play. With deployments, we can manage our pods with a controller. This allows us to not only decide how many we want to run, but we can also manage updates by changing the image version or the image itself that our containers are running. Deployments are what you will be working with most of the time. With deployments as well as pods and any other objects that we mentioned before, they have their own definition inside a YAML file:

apiVersion: apps/v1kind: Deploymentmetadata: name: nginx-deployment labels: deployment: nginxspec: replicas: 3 selector: matchLabels: app: nginx template: metadata: labels: app: nginx spec: containers: - name: nginx image: nginx:1.7.9 ports: - containerPort: 80

Let's start exploring their definition.

At the beginning of the YAML, we have more general fields, such as apiVersion, kind, and metadata. But under spec is where we will find the specific options for this API Object.

Under spec, we can add the following fields:

Selector: With the Selector field, the deployment will know which pods to target when changes are applied. There are two fields that you will be us

> Core components of a Kubernetes Cluster