Table of Contents
Developing a Kubernetes controller/operator can be a stressful experience. You have to come to terms with eventual consistency (i.e. “trust the cluster, it will work itself out eventually”), frequent retries and conflicts because you are not the only application that is running and might make modifications to Kubernetes resources (just because you created a ConfigMap does not mean some other, unknown to you application will try to modify that ConfigMap). Things get even more funky when your controllers begin to operator on multiple clusters at same time.
One of the biggest issues for me was having a mental trace of how my Kubernetes resources evolved over time. Which controller at what point made what modification to it? If a resource ends up stuck in deletion because of some finalizers, how did that happen exactly?
kubectl get can already be used to keep a tap on resources by using the
--watch flag, but it’s limited and
annoying to use when you have to deal with a whole number of resources at the same time. Plus all you would
kubectl get --watch is the full resource, no matter how little was changed.
stalk can help tremendously in these situations. Instead of writing the entire resources to stdout,
keeps their latest version in memory and can then present you with a diff view.
stalk allows you to hide
irrelevant things (
managedFields, anyone?) or only show specific paths of your resources.
$ stalk --help Usage of stalk: -c, --context-lines int Number of context lines to show in diffs (default 3) -w, --diff-by-line Compare entire lines and do not highlight changes within words -h, --hide stringArray Path expression to hide in output (can be given multiple times) --hide-managed Do not show managed fields (default true) -j, --jsonpath string JSON path expression to transform the output (applied before the --show paths) --kubeconfig string Kubeconfig file to use (uses $KUBECONFIG by default) -l, --labels string Label-selector as an alternative to specifying resource names -n, --namespace stringArray Kubernetes namespace to watch resources in (supports glob expression) (can be given multiple times) -s, --show stringArray Path expression to include in output (can be given multiple times) (applied before the --hide paths) -e, --show-empty Do not hide changes which would produce no diff because of --hide/--show/--jsonpath -v, --verbose Enable more verbose output -V, --version Show version info and exit immediately
Ever wondered how a Deployment progresses?
A Kubernetes Deployment has fields like
availableReplicas etc. in its status.
But when you create a Deployment, how do these fields actually change over time as Pods are started/deleted?
stalk can tell you! The command below will stalk all Deployments in your cluster (in all namespaces) and
will only show the status subresource:
$ stalk deployments -s status
Find out why your changes won’t stick
Sometimes you try to edit a Kubernetes resource and your changes are seemingly lost/overwritten. The root cause
is usually some other controller that, for whatever reason, undoes your changes immediately. Sometimes you are
not even involved, but instead you have two or more components in your cluster that fight over the state of
a single resource (this should of course not happen in production systems). If you ever tried to quickly hammer
kubectl get somethingsomething foo -o yaml over and over and over again and hoped to see the stray change,
stalk can also help you. Just run…
$ stalk somethingsomething foo
and you can quickly see what’s going on.