A collection of awesome Kubernetes tools and projects to deploy, secure, and monitor your Kubernetes clusters.

today I’m listing my top Kubernetes tools for testing and development.

Lens

For a while, I was an exclusive GKE user, spoiled by its simplicity and the managed dashboard on the GCP console. For local development, running the Kubernetes Dashboard maintained by the Kubernetes community fit most of my needs. However, as I transitioned into managing multi- and hybrid-cloud environments, I needed a single tool to interact with GKE, EKS, and local minikube clusters.

Lens is an open-source Kubernetes IDE that makes it easy to switch between multiple clusters and interact with deployed applications. Aqua Security also has an extension for Starboard that integrates vulnerability testing, which makes the DevSecOps workflow easier to manage.

Fairwinds Tools

Fairwinds and Aqua Securrity provides several useful Kubernetes tools to validate, scan, and optimize applications:

• Polaris: deployed as a dashboard, admission controller, or a CLI tool to audit and validate deployment best practices (e.g. does your deployment have health checks defined, are resource requests and limits configured)
• Kube-hunter: looks for security weaknesses in running clusters
• Kube-bench: runs the CIS Kubernetes Benchmark and flags failed configurations
• Trivy: vulnerability scanning tool for containers that you can integrate into the CI pipeline
• Goldilocks: queries resource utilization in each namespace by using the Kubernetes vertical-pod autoscaler in recommendation mode. For more information, check out “Ultimate Kubernetes Resource Planning Guide”

Kyverno

Kyverno is a CNCF sandbox project aimed at simplifying policy management in Kubernetes. While Kubernetes has PodSecurityPolicies and NetworkPolicies that admins and operators can configure, it’s often hard to configure correctly, test the policies, and validate resources. Kyverno solves this problem by creating a ClusterPolicy CRD to validate and enforce rules.

Some example policies include:

• Disallowing root and privileged containers
• Disallowing hostNetwork, hostPort, hostPIID, and hostIPC
• Disallowing latest image tag
• Enforcing labels on deployments

CLI Tools

While there are a plethora of CLI tools for Kubernetes, these are the tools found to be most useful:

• kube-ps1: adds the current Kubernetes context and namespace to bash/zsh prompts to avoid interacting with the wrong cluster
• kubectx + kubens: kubectx helps you switch between clusters, whereas kubens is used for switching between namespaces
• krew: package manager for kubectl plugins (equivalent of brew, apt)
• kubectl-neat: removes redundant information added by Kubernetes when running kubectl get . This is helpful for debugging Kubernetes manifests without most lines we don’t usually need (e.g. creationTimestamp, uid, dnsPolicy,terminationMessagePolicy).
• kube-no-trouble: detects the use of deprecated APIs in your cluster. Very useful to run a sanity check prior to upgrading your clusteer
• helm-mapkubeapis: updates Helm releases that contain deprecated or removed Kubernetes APIs. Combine this with kube-no-trouble to not only fix the applications you control, but also third-party Helm charts.
• kube-diff + helm-diff: diff checker to preview what’s changed between deployed versions of Kubernetes manifests/Helm releases

Kube Forwarder

Most likely during development, you will have databases deployed in your Kubernetes cluster that is not exposed via a Kubernetes service (e.g. NodePort, Ingress). Kubectl allows port-forwarding to map those services to localhost to interact locally. This is fine for one or two applications, but when you have multiple applications, Kube Forwarder becomes much easier to use than managing multiple terminal tabs. It also handles auto-reconnect and multi-clusters, which is nice if you need to switch between dev and qa clusters for testing.

Kubecost

Like with any cloud computing, running Kubernetes on AWS/GCP/Azure can get expensive very quickly. Even after fine-tuning the cluster with spot/pre-emptible instances and using autoscalers to scale down unused resources, it’s easy to miss idle or over-provisioned resources. Kubecost monitors Kubernetes spend and provides a granular report on cost allocation. For large-scale projects, there’s probably already a team monitoring cost spend closely, but for personal projects, Kubecost is excellent if you want to avoid fumbling with cloud billing tools yourself.

Kubespy

Kubespy is a tool from Pulumi to observe how Kubernetes resources change by tracing the Kubernetes API. Kubespy provides more detailed information than kubectl get -w and shows all the changes in the command line. I found this tool to be useful in debugging cert-manager and TLS certification issues, but I’m sure it can be used for all other Kubernetes debugging sessions.

If there are other useful Kubernetes tools that I missed, please let me know in the comments and I’ll make sure to check it out.