Kubernetes on DigitalOcean

DigitalOcean Kubernetes (DOKS) is a managed Kubernetes service that lets you deploy Kubernetes clusters without the complexities of handling the control plane and containerized infrastructure. Clusters are compatible with standard Kubernetes toolchains and integrate natively with DigitalOcean Load Balancers and block storage volumes.

Plans and Pricing

The cost of a DigitalOcean Kubernetes cluster is based on the cluster's resources:

All charges for Kubernetes clusters appear in the Kubernetes section of monthly invoices.

Bandwidth

Outbound data transfer is shared between all Droplets, including Kubernetes worker nodes, so bandwidth for Kubernetes cluster worker nodes is charged at the same rate as Droplet bandwidth pricing.

You can view your accumulated monthly transfer allowance on your account's billing page in the Droplet transfer section. For an in-depth description of how data transfer accrual works, read our detailed bandwidth billing page.

Regional Availability

At least one datacenter in every region supports Kubernetes. Kubernetes will not be offered in NYC2, AMS2, or SFO1.

Learn more in the regional availability matrix

Features

Kubernetes is a powerful open-source system for managing containerized applications in a clustered environment. Its focus is to improve how you manage related, distributed components and services across varied infrastructure.

DigitalOcean Kubernetes is a managed Kubernetes service lets you deploy scalable and secure Kubernetes clusters without the complexities of administrating the control plane. We manage the Kubernetes master and the underlying containerized infrastructure.

DigitalOcean Kubernetes provides administrator access to the cluster and full access to the Kubernetes API with no restrictions on which API objects you can create. We manage key services and settings on your behalf that you cannot or should not modify.

You retain full access to the cluster with existing toolchains. You have cluster-level administrative rights to create and delete any Kubernetes API objects through the DigitalOcean API and doctl.

There are no restrictions on the API objects you can create as long as the underlying Kubernetes version supports them. We offer the latest version of Kubernetes as well as earlier patch levels of the latest minor version for special use cases. You can also install popular tools like Helm, metrics-server, and Istio.

Conformance Certification

DOKS conforms to the Cloud Native Computing Foundation's Kubernetes Software Conformance Certification program and is proud to be a CNCF Certified Kubernetes product.

In addition, we run our own extended suite of end-to-end tests on every DOKS release to ensure stability, performance, and upgradability.

Worker Nodes and Node Pools

Worker nodes are built on Droplets, but unlike standalone Droplets, worker nodes are managed with the Kubernetes command-line client kubectl and are not accessible with SSH. On both the master nodes and the worker nodes, DigitalOcean maintains the system updates, security patches, operating system configuration and installed packages.

All the worker nodes within a node pool have identical resources, but each node pool can have a different worker configuration. This lets you have different services on different node pools, where each pool has the RAM, CPU, and attached storage resources the service requires.

You can create and modify node pools at any time. Worker nodes are automatically deleted and respawned when needed, and you can manually recycle worker nodes. Nodes in the node pool will inherit the node pool's naming scheme when you first create a node pool, however, renaming a node pool will not rename the nodes. Nodes will inherit the new naming scheme only when they are recycled or the node pool is resized, creating new nodes.

Kubernetes role-based access control (RBAC) is enabled by default. See Using RBAC Authorization for details.

Persistent Data

You can persist data in DigitalOcean Kubernetes clusters to block storage volumes using the DigitalOcean CSI plugin. (See the feature overview page to learn which block storage volume features are available on DigitalOcean Kubernetes.) We recommend against using HostPath volumes because nodes are frequently replaced and all data stored on the nodes will be lost.

You can also persist data to DigitalOcean object storage by using the Spaces API to interact with Spaces from within your application.

Load Balancing

The DigitalOcean Kubernetes Cloud Controller supports provisioning DigitalOcean Load Balancers.

VPC Networks

Clusters are added to a VPC network for the datacenter region by default. This keeps traffic between clusters and other applicable resources from being routed outside the datacenter over the public internet.

Cluster networking is preconfigured with Cilium. Overlay networking is preconfigured with Cilium and supports network policies.

Log Rotation

Cluster logs are rotated when they reach 10 MB in size. The last 2 copies are retained in addition to the current active log.

Tags

Clusters are automatically tagged with k8s and the specific cluster ID, like k8s:EXAMPLEc-3515-4a0c-91a3-2452eEXAMPLE. Worker nodes are additionally tagged with k8s:worker. You can add custom tags to the cluster and worker nodes in the Tags field.

Limits

  • The master configuration is managed by DigitalOcean. You cannot modify the master files, feature gates, or admission controllers. See The Managed Elements of DigitalOcean Kubernetes for more specifics.

  • Because DigitalOcean Kubernetes clusters provision a single master node, they are not highly available. During upgrades or maintenance, the control plane may be temporarily unavailable. This does not affect running clusters and does not make the cluster workers or workloads unavailable.

  • Worker nodes are subject to Droplet limits. Block storage volumes are similarly subject to volumes limits.

  • You cannot manually resize DOKS nodes by using the control panel to edit the Droplets. The reconciler will view this as aberrant and revert such changes. To resize DOKS nodes, create a node pool of the desired size, and once it is fully provisioned, remove the old one.

  • The manual deletion of nodes using kubectl delete is not supported, and will put your cluster in an unpredictable state. Instead, resize the node pool to the desired number of nodes, or use doctl kubernetes cluster node-pool delete node.

  • The control panel does not support assigning a cluster to a project. Instead, do this with doctl projects resources assign:

    doctl projects resources assign <project-id> --resource=do:kubernetes:<cluster-id>
    

Resource Limits

  • Clusters must be in a single datacenter region.

  • Clusters can have up to 512 nodes.

  • Nodes can have up to 110 pods.

  • Network throughput is capped at 2 Gbps.

For general information on the upper limits of Kubernetes cluster sizes and how large cluster sizes affect scaling behavior, see the official Kubernetes documentation on building large clusters and scalability validation of the release

Allocatable Memory

The size of DOKS nodes determines the maximum amount of memory you can allocate to Pods. Because of this, we recommend using nodes with less than 2GB of allocatable memory only for development purposes and not production. These distinctions are visible during the cluster creation process.

The following table describes the maximum allocatable memory that will be available for scheduling pods.

Size Slugs Node Memory (GiB) Maximum Pod Allocatable Memory
s-1vcpu-2gb, s-2vcpu-2gb 2 1 GiB
s-1vcpu-3gb 3 1.66 GiB
s-2vcpu-4gb, c-2 4 2.5 GiB
s-4vcpu-8gb, g-2vcpu-8gb, gd-2vcpu-8gb, c-4 8 6 GiB
s-6vcpu-16gb, g-4vcpu-16gb, gd-4vcpu-16gb, c-8 16 13 GiB
s-8vcpu-32gb, g-8vcpu-32gb, gd-8vcpu-32gb, c-16 32 28 GiB
s-12vcpu-48gb 48 43 GiB
s-16vcpu-64gb, g-16vcpu-64gb, gd-16vcpu-64gb, c-32 64 58 GiB
s-20vcpu-96gb 96 88.5 GiB
s-24vcpu-128gb, g-32vcpu-128gb, gd-32vcpu-128gb 128 119.5 GiB
g-40vcpu-160gb, gd-40vcpu-160gb 160 151 GiB
s-32vcpu-192gb 192 182 GiB

This memory reservation is due to the following processes that are running on DOKS nodes:

  • kubelet
  • kube-proxy
  • docker
  • cilium
  • cilium-operator
  • coredns
  • do-node-agent
  • kubelet-rubber-stamp
  • The OS

In clusters running Kubernetes 1.16 or higher, the allocatable memory is encoded in the “Kube Reserved” and “System Reserved” values in kubelet. For more information, see Reserve Compute Resources for System Daemons in the Kubernetes Documentation.

Names and Tags

  • At creation time, the k8s prefix is reserved for system tags and cannot be used at the beginning of custom tags.

  • You cannot tag load balancers or block storage volumes.

  • Although it's currently possible, we will not support tagging individual worker nodes in the future.

Feature Support

In DigitalOcean Kubernetes clusters, we do not yet support:

  • Automatic Let's Encrypt certificates for DigitalOcean Load Balancers. Instead, you can generate a certificate yourself.

  • Assigning Kubernetes clusters or the underlying Droplets in a cluster to a project.

Known Issues

  • In the DigitalOcean Control Panel, cluster resources (worker nodes, load balancers, and block storage volumes) are listed outside the Kubernetes page. If you rename or otherwise modify these resources in the control panel, you may render them unusable to the cluster or cause the reconciler to provision replacement resources. To avoid this, manage your cluster resources exclusively with kubectl or from the control panel's Kubernetes page.

  • The DigitalOcean autoscaler does not support a min_node size of 0, therefore the mininum node size for an autoscaling group is 1.

  • Installing webhooks targeted at services within the cluster can cause Kubernetes version upgrades to fail because internal services may not be accessible during upgrade.

  • Load balancers and block storage volumes created by Kubernetes manifests are not deleted when a cluster is deleted. You are billed for them until you delete them explicitly.

  • The certificate authority, client certificate, and client key data in the kubeconfig.yaml file displayed in the control panel expire every seven days after download. If you use this file, you will need to download a new certificate every week. To avoid this, we strongly recommend using doctl.

  • Kubernetes 1-Click Apps can be installed multiple times to a cluster and will be installed in the same namespace each time. This means that subsequent installations of a given 1-Click App will overwrite the previous instance of that 1-Click App, as well as the data that was associated with it.

  • If a Kubernetes 1-Click App is currently installing and a subsequent install request for the same App is made, the subsequent request will not be processed. Only once the 1st request is completed (Done or Failed) may a subsequent request be made to install the same Kubernetes 1-Click App on the same cluster.

  • Kubernetes 1-Click Apps that are deleted from a cluster still appear in the history of installed 1-Click Apps on the cluster's Overview page. If a 1-Click App was installed on a cluster multiple times, it will be listed as installed multiple times regardless of whether the 1-Click App is currently present on the cluster.

Latest Updates

6 October 2020

12 May 2020

28 April 2020

  • The DigitalOcean Virtual Private Cloud (VPC) service is now available for all customers. VPC replaces the private networking service. Existing private networks will continue to function as normal but with the enhanced security and features of the VPC service. See the description of VPC features for more information.

7 April 2020

For more information, see all Kubernetes release notes.