20,000 Upgrades Later: Lessons From a Year of Managed Kubernetes Upgrades

Video

About the Talk

Upgrading to a new release is one of the most disruptive operations we regularly inflict on our Kubernetes clusters. There are multiple strategies for doing an upgrade, but they all require rescheduling workloads and restarting cluster components.

We started offering upgrades on our managed Kubernetes platform, DigitalOcean Kubernetes Service (DOKS), in May 2019. Since then, our customers have kicked off about 20,000 automated patch and minor release upgrades on their clusters. Most of those upgrades went well, but some didn’t and we’ve learned a few things from the ones that went wrong.

In this talk, we will share lessons from a year of automated Kubernetes upgrades: what we got right, what we got wrong, workloads that caused us trouble, and changes we’ve made to make the process smoother. We hope these lessons will help others avoid pain in their Kubernetes upgrades.

What You’ll Learn

• How DigitalOcean coordinates Kubernetes upgrades for managed clusters.
• What Kubernetes users, especially those using DigitalOcean managed Kubernetes, can do to ensure their workloads tolerate upgrades.

This Talk is Designed For

• Kubernetes administrators/operators who are interested in the details of how DigitalOcean performs Kubernetes upgrades, what we’ve seen go wrong, and what we’ve learned.
• Developers running applications on Kubernetes who are interested in how to configure your workloads to avoid problems during upgrades.

Prerequisites

Knowledge of the components of a Kubernetes cluster and how applications are deployed.

About the Presenter

Adam Wolfe Gordon is the tech lead for managed Kubernetes and container registry at DigitalOcean. He previously worked on block storage at DigitalOcean and EMC. Adam is a regular conference speaker and a frequent attendee of and presenter at local meetups in Edmonton, Alberta, Canada. He likes building and debugging microservices, observability, and occasional forays into lower-level software.