Understanding Kubernetes Operators

Almost every Kubernetes tutorial speaks about how to quickly deploy a container in a pod and expose it as a service. Mostly these tutorials focus on stateless services and ignore a deeper explanation of state management in Kubernetes. But Kubernetes supports both types of deployments, the stateless deployments and stateful deployments and they have somewhat different operational requirements.

In stateless deployments, the order of deploying pods, their labels, network address or port, storage class, or volume are not important. You keep them if they are healthy and serving, you dispose of them when they become unhealthy, outdated, or just no longer needed and replace them as necessary.

In stateful deployments, there’s usually some order to the way pods are created. They maintain a state in the network such as their address or port, and they also depend upon connections to storage volumes to ensure data persistence.

Stateful Kubernetes Orchestration

It gets a bit difficult to manage stateful services and deployments on Kubernetes. A typical Kubernetes stateless deployment can be made as:

$ kubectl create deployment ...
$ kubectl expose deployment ...

Since these pods and their services would be stateless, it does not matter how Kubernetes creates them, what matters is that they are created and the “state” of the cluster is managed for us.

In the case of deployments with stateful components, there is more to consider. For example, we need to provide a storage class or the persistent volume claim for the services to utilize the storage, which of course needs to be self-managed.

This is not only an issue with storage. Kubernetes does not expose the deployments as services automatically by itself. This gets cumbersome to manage by hand, or even by declarative scripts.

For example, a typical deployment process for a stateful application follows something like this,

NAME     READY     STATUS              RESTARTS   AGE

pod-0   1/1       Running             0          1m

pod-1   0/1       ContainerCreating   0          8s

This shows that your objects are created one after the other, and these labels and identifiers are used to backup and restore the cluster, since Kubernetes labels them in the order that they are created. This doesn’t permit us to perform a usual Kubernetes hit-and-forget deployment and requires special care to be taken during the deployment and creation of resources. Kubernetes also requires that, in addition to network exposure services, a correctly labeled headless service exists to orchestrate stateful application operations.

Operators Simplify Stateful Deployments

This is where Kubernetes Operators come into play. A Kubernetes Operator is essentially software that wraps manual tasks into Kubernetes functions for you and allows you to control how stateful or complex deployments are executed (even multi-container deployments). The key concept is that an Operator wraps any necessary logic for deploying and operating an application using Kubernetes constructs.

The job of the Kubernetes Operator is to manage and provision the resources that you have to maintain manually, and since it is provided with enough authorization in the cluster, the operator is able to perform cluster-keeping tasks itself, such as rescheduling the pods if they fail or scaling the replica sets in the cluster as needed.

You can develop Kubernetes Operators using the open sourced Operator Framework SDK and the Go programming language. An Operator is the automation script that watches over the resources and executes the manual tasks that are needed to be performed by your engineering team. However, the Operator SDK is for the development of Operators, not managing your Kubernetes objects and resources. The Operator itself can then control the services.

For example, Kublr supports Microsoft Azure and Amazon Web Services for cluster deployment. In these environments your Operators can help you in managing storage, networking and security by automatically managing the underlying infrastructure for you, claiming the storage from Azure Disks and Files, for instance, in the case of Azure.

A Kubernetes Operator can help you in a cloud-native and cloud-first development and deployment scenario. Operators can aid you in the selection of resources from the cloud environment, without your team having to request and manage resources manually.

Operators may also serve as access mediators, so that users who don’t have and don’t need full privileges to manage infrastructure can still get mediated access to it.

One example is the open source etcd operator that manages and provisions the etcd clusters on Kubernetes. This Operator manages pod creation and deletion, failovers, restoration and much more that would have required intervention of an engineer. Since they aid us in being cloud-native, an Operator can also optimize the storage claims for your pods or dynamically allocate volume and provision it for your pods on its own, one of such is the open sourced PVC Operator by Banzai Cloud that you can explore on your own.

Another example is Rook — a cloud-native storage operator for Kubernetes — that can manage various storage and data management systems running on Kubernetes, such as Ceph, EdgeFS, Cassandra, CocroachDB, Minio, and NFS.

Learning more About Kubernetes Operators

You can get more detail about Operators and the Operator SDK at project GitHub repository. And while you can write your own Operators with the SDK, there is also a growing library of downloadable, pre-coded Operators available from OperatorHub.io. Browsing through these Operators will give you an idea of what’s possible.