How to Monitor Advanced Metrics for Kubernetes Pod Deployment Progress and Availability

DigitalOcean Kubernetes includes metrics visualizations to provide insight into the health of your Kubernetes clusters and deployments.

  • Basic metrics include CPU usage, load averages, bandwidth, and disk I/O. Basic metrics are useful for capacity planning and identifying unhealthy worker nodes. For example, you can set up alerting for worker node metrics.

    Basic metrics are available by default for clusters, node pools, and worker nodes because all Kubernetes clusters come with the DigitalOcean metrics agent, a lightweight, open-source service that forwards metrics to the DigitalOcean’s Prometheus backend.

  • Advanced metrics include deployment status metrics, like DaemonSet pod scheduling and availability. Advanced metrics are useful for in-depth views into Kubernetes-specific metrics.

    To use advanced monitoring, you need to install the sidecar agent kube-state-metrics.

To complement DigitalOcean advanced Kubernetes monitoring, you can also install Linkerd 2, a third-party tool that provides service-level observability.

Set Up the kube-state-metrics Sidecar Agent

Installation

To enable DigitalOcean’s optional Kubernetes deployment metrics, you need to install kube-state-metrics, a sidecar metrics agent that collects object state metrics from the Kubernetes API server.

From the command line, first clone the kube-state-metrics repo:

git clone git@github.com:kubernetes/kube-state-metrics.git

Next, deploy kube-state-metrics with kubectl:

kubectl create -f kube-state-metrics/examples/standard/

Once kube-state-metrics is installed, the new metrics will appear along with your Kubernetes basic metrics for cluster-level views. It may take a few minutes for the graphs to populate.

Removal

To remove kube-state-metrics, run the following kubectl delete command from the directory where you installed the repo.

kubectl delete -f kube-state-metrics/kubernetes/

View Advanced Kubernetes Metrics

You can view advanced performance metrics after you install kube-state-metrics. Click the name of the cluster to go to its Overview page, then click the Insights tab.

The Insights tab of a Kubernetes cluster

Use the Select object drop-down to choose a cluster.

Use the Select period drop-down to change between metrics time frames, from 1 hour to 30 days. Each graph will display approximately 300 data points.

By default, the summary to the right shows the most recent metrics values. If you hover over a point in time on one of the graphs, the summary will display the values from that time point instead.

If you recently installed the sidecar agent, it may take a few minutes for the metrics data to finish processing before you see it on the Insights page.

Kubernetes Advanced Metrics Details

The Kubernetes deployment metrics include:

  • Pod deployment progress and availability
  • DaemonSet deployment progress and pod availability, if using DaemonSets
  • StatefulSet deployment progress, if using StatefulSets

All advanced metrics plots represent metrics aggregated across the cluster.

Pod Deployment Status

The pod deployment status plot displays the number of pods expected at deployment start, and the count of the pods that are available and unavailable after deployment.

Pod deployment status across the cluster

DaemonSet Deployment Status

If you are using DaemonSets, the DaemonSet deployment status plot displays the number of nodes that should be running a daemon pod, the number of nodes have been updated and have available daemon pods, and the number of nodes that have been updated but have unavailable daemon pods.

DaemonSet deployment status across the cluster

StatefulSet Deployment Status

If you are using StatefulSets, the StatefulSet deployment status plot displays the requested number of StatefulSet pods and the number created and ready for use.

StatefulSet deployment status across the cluster