Fresh features and 1-Click Apps for DigitalOcean Kubernetes
Scalable compute services
Managed Kubernetes clusters
Worry-free setup & maintenance
Simple object storage
Connect, share and learn
DevOps and development guides
Questions and Answers
Development and systems Q&A
How to Deploy a PHP Application with Kubernetes on Ubuntu 16.04
Understanding Database Sharding
How To Build a Node.js Application with Docker
Prepared for DigitalOcean by Cloud Spectator LLC
DigitalOcean commissioned Cloud Spectator to evaluate the performance of
virtual machines (VMs) from three different Cloud Service Providers (CSPs):
Amazon Web Services, Google Compute Engine and DigitalOcean.
Cloud Spectator tested the various VMs to evaluate the CPU performance,
Random Access Memory (RAM) and storage read and write performance of each
provider’s VMs. The purpose of the study was to understand
Cloud service performance among major Cloud providers with similarly-sized
VMs using a standardized, repeatable testing methodology. Based on the analysis,
DigitalOcean’s VM performance was superior in nearly all measured VM performance
dimensions, and DigitalOcean provides some of the most compelling
performance per dollar available in the industry.
Figure 1 - CPU Performance Per Dollar of all tested VMs
The primary drivers for DigitalOcean’s strong performance were three factors:
Key findings and observations from this analysis are highlighted below,
with more detailed analysis following in the body of the report.
The following summary findings were gleaned from the testing performed by Cloud
Spectator during this engagement:
Compute and memory performance were tested using the GeekBench4 test suite.
The following highlights emerged from these tests:
Storage performance was tested using the FIO tool to perform a variety of tests.
The 4K random read and random write results are summarized below.
Price-Performance, or the value of a given Cloud service, derived by dividing
performance by price (compute/memory and storage), is summarized below:
The details of the testing setup, design and methodology along with full results,
are explained in the body of the report.
DigitalOcean commissioned Cloud Spectator to assess the performance of virtual
machines (VMs) from three different Cloud Service Providers (CSP) or providers:
DigitalOcean (DO), Amazon Web Services (AWS), and Google Compute Engine (GCE).
Cloud Spectator tested the various VMs from these providers to evaluate the CPU
performance, RAM and storage for each provider’s VMs. The purpose of the study
was to understand the VM performance between Cloud providers with similarly-sized
and classed VMs using standardized and repeatable testing methodology. Performance
information for specified standard 1 and 2 vCPU 2GB and compute-optimized 2 vCPU
4GB VMs was gathered using Geekbench4 and FIO benchmarking tools for this analysis.
Each VM type was provisioned with four duplicate VMs to limit sampling error.
Data was then collected during 100 iteration tests.
This project focused on comparison of performance data for CPU, RAM and storage
IOPs for random read and write. The CPU-Memory composite scores and storage scores
were evaluated on their own, and then were used to calculate the price-performance
value for all provider VM offerings. The price-performance value for each VM was
calculated by dividing performance averages by monthly cost in USD, with separate
scores calculated for storage read and write. This simple price-performance formula
allows the universal comparison of VMs offered by the respective Cloud Service
Providers included in this analysis.
Using this proven Cloud sampling and testing methodology, Cloud Spectator evaluated
the Cloud services based on price-performance calculations, while detailing specific
strengths and weakness of each provider’s VMs based on the objective performance
results. Given the inherent variability of Cloud services, these methods are necessary
to provide reliable and comparable analyses of Cloud-based infrastructure-as-aService
Virtual machines (VMs) for this engagement focused on 1 and 2 vCPU VMs. They were
grouped and classified as standard 1 vCPU 2GB, standard 2 vCPU 2GB and compute-optimized
2 vCPU 4GB machines. All instances were deployed with a current release of Ubuntu 18.04
from the respective providers. Local SSDs were used for DigitalOcean’s Droplet VMs, while
elastic block storage (EBS) general purpose (gp2) block storage was used for AWS, and
persistent solid state (SSD) block storage was used with GCE. Standard VMs are targeted
for general purpose workloads or application testing, and are configured with 1:2 and
1:1 RAM to vCPU ratios for standard 1 vCPU and 2 vCPU instances, respectively. Standard
instances are configured to share cores with other VMs. Some may “burst” temporarily
(AWS T3.small), while others opportunistically use available resources on the parent
server, e.g. GCE g1-small, for performance boosts when needed for a particular
workload. On the other hand, compute-optimized VMs are configured with dedicated vCPU’s
operating at full capacity with a 1:2 vCPU to RAM ratio for more demanding workloads.
The VMs selected for this engagement are listed in the tables below:
Figure 6.1 - DigitalOcean, AWS and GCP VMs tested and compared
The test design and methodology used in this analysis are described in the following sections.
The test design and methodology are described below for each of the VM performance
dimensions evaluated, as follows: CPU and RAM, and storage random read/write.
Synthetic testing was performed on the selected VMs to enable objective comparisons
CPU and memory testing were conducted with the Geekbench4 benchmarking suite,
which allows modern testing scenarios such as floating-point computations, encryption
and decryption, as well as image encoding, life-science algorithms and other use cases.
Storage results were obtained using FIO (Flexible I/O tester) using 4KB blocks and
threads corresponding to vCPU count. Several thousand 60-second random iterations
were conducted to compensate for the high variability often seen when stressing
storage volumes. Results were gathered and represented in IOPs (input/output operations
Testing was conducted on specific VM types for each provider. Provider VM configurations
may yield different results based on underlying infrastructure, virtualization technology
and settings (e.g. shared resources), and other factors. Furthermore, factors such as
user contention or physical hardware malfunctions can also cause suboptimal performance.
Cloud Spectator therefore provisioned multiple VMs with the same configuration to better
sample the underlying hardware and enabling technology, as well as improve testing
accuracy and limit the effects of underlying environmental variables.
The VMs selected for this engagement were generally-available specified offerings from
the various providers. While better performance can often be attained from providers
when additional features or support services are purchased, the selected VMs used in
Cloud Spectator’s testing do not leverage such value-added services. This helps provide
data and test results that are indicative of real-world customer choices from the tested
Four duplicate VMs were deployed during testing to minimize sources of error prevalent
in a Cloud hosting environment. The most notable challenge is the Noisy Neighbor Effect.
Testing duplicate VMs mitigates most non-specific errors that could be attributed to a
singular parent instance or storage volume. By minimizing possible sources of error, more
accurate and precise performance samples can be collected during testing.
In the sections below, the performance results from Cloud Spectator’s testing is
presented. Graphs of the performance results, along with interpretive narratives,
are provided for all tests.
Price-performance, also known as value, compares the performance of a Cloud service
to the price of that service. Thus, price-performance offers a universal metric for
comparing Cloud service value. Priceperformance is calculated from the average Geekbench4
multi-core score divided by the monthly price in US Dollars (USD). A higher
price-performance score indicates greater value for a given VM configuration.
Generally, standard, burstable or smaller VMs tend to achieve higher price-performance
values than computeoptimized or larger VMs with static vCPU allocations. Larger VMs with
specifically provisioned resources are typically used for more large production use cases,
and therefore have higher prices. Thus, this evaluation was limited to comparable size
categories to illustrate the differences between these types of machines.
Multi-core CPU test averages tend to display large differences based on vCPU count,
as opposed to singlecore tests. The results section below provides an overview of the
single-core performance, while multi-core performance is used for price-performance
The chart below depicts the CPU single-core performance of all VMs evaluated in this
study. The GeekBench4 single-core score represents the processing speed of one CPU
(or core) processing a single stream of instructions rather than multiple parallel
streams per core. Many consumer applications, although they are multithreaded, rarely
utilize more than one CPU thread at a time. Thus, the single-core CPU test is a reasonable
real world test for typical consumer workloads.
Figure 9.1 - Single-core Averages
In most production scenarios, all vCPUs will be utilized in multi-core VMs if a workload
or application is able to exploit them. Though not discussed in detail, single-core scores
provide a universal performance baseline that is useful when comparing multi-core
The chart below depicts the CPU multi-core performance of all VMs covered in this study.
As core count, or vCPU quantity, increases VMs generally produce higher scores as depicted
in the graph below. Computeoptimized machines are shown below with solid colored bars,
while standard VMs are displayed with pattern filled bars.
Figure 10.1 - Multi-core Average
Figure 10.2 - Multi-core Coefficient of Variation
*AWS unlimiting bursting may incur additional charges.
In the next section, the analysis will focus on Cloud storage performance.
Storage performance results are summarized in the sections below. The testing
methodology for storage ensured that all machines were tested for several thousand
iterations using FIO with a block size of 4KB, queue depth of 32 running at 1 thread
per vCPU for random read and write.
Below, storage random read results for all providers and VM types are displayed.
Summary details are provided below. The chart displays compute-optimized VMs as solid
colors while standard VMs are pattern filled.
Figure 11.1 - Storage 4K Read IOPs
Figure 11.2 - Storage Read Coefficient of Variation
VM size, as described by vCPU count alone has some role in storage random read
performance in this test though RAM has a much greater affect as can be seen from the
results above with 2 vCPU 2GB instances vs compute-optimized 2CPU 4GB VMs from the same
providers. Below, random write results are presented.
Unlike random read performance, random write performance requires numerous background
tasks between operations. This includes data allocation and redundancy checking across
the large storage arrays typically used in Cloud-base block storage. VM vCPU counts are
less important factors in random write performance than vendor storage types. Below,
storage write performance are presented. The data compares all VM sizes and confirms
that vCPU core count is a less relevant factor for storage performance than CPU-Memory
testing. Summary observations of these test results can be found below the charts.
Figure 12.1 - 4K Write IOPs
Figure 12.2 - 4K Write Coefficient of Variation
To summarize, DigitalOcean performed well for random write performance, while dominating
random read tests. Google provided consistency for both read and write operations,
while AWS displayed significantly lower write speeds compared to read along with
universally high storage variation due to EBS burst.
This section focuses on the compute and memory price-performance, or value. The values
shown are linear and unweighted, using the multi-core performance scores (figures 10.1)
and monthly price (tables 6.1 - 6.3). Higher scores indicate higher value for each USD
spent by the tested VMs.
The chart below summarizes all VM sizes evaluated for price-performance. Compute-optimized
VMs are found in solid colors, while standard VMs are shown with a pattern fill.
Figure 14.1 - CPU Price-Performance
Figure 13.2 - VM Total Monthly Price
In summary, DigitalOcean VMs provide excellent value across all sizes and both VM
groups (standard and compute-optimized), with standard class machines having a
price-performance advantage over the computeoptimized machines. With the lowest
prices and competitive CPU scores, DigitalOcean achieved superior performance scores,
even against some of the newest hyperscale offerings. The following section details
This section focuses on storage price-performance value. Storage performance can be a
major bottleneck for certain applications, and often storage pricing can become a budgeting
challenge for enterprises. The values shown in the sections below are linear and unweighted,
using the average performance scores for random read (Figure 11.1) and random write
(Figure 12.1), as well as monthly prices (tables 6.1-6.3). Higher scores indicate a better
price-performance value per USD spent compared to competing VMs in the same size category.
In the chart below, price-performance is compared for the specified VMs based on random
read speed. This is most useful when considering read-intensive use-cases. Summary
observations are provided below.
Figure 14.1 - Read Price-Performance
Overall, DigitalOcean’s offerings were dominant in read speeds, as their local
storage provided a significant edge over the block storage offered by other tested
Write speed, no matter the drive type or configuration, has always been a technology
challenge. The following summary observations were gleaned from the analysis.
Compute-optimized VMs are displayed in solid colors, while the standard VMs are
shown in pattern fills.
Figure 15.1 - Write Price-Performance
In summary, DigitalOcean provides excellent storage price-performance for both read
and write for all VM configurations tested, regardless of machine classification or size.
For this engagement, Cloud Spectator tested Amazon Web Services, Google Compute
Engine and DigitalOcean VMs head to head across three categories: Standard 1vCPU
2GB RAM, Standard 2vCPU 2GB RAM and compute-optimized 2vCPU 4GB RAM. The testing
and data collection were performed by running exhaustive computational and storage
benchmarks on specified VM configurations. From these results, priceperformance values
for each VM type were determined.
DigitalOcean’s Droplets displayed excellent performance values in all areas. Their
CPU-Memory performance, particularly within the standard class, achieved performance
values exceeding those of comparable offerings by a factor of three. Additionally,
DigitalOcean’s read speed was vastly superior across all VM types. Storage write
price-performance value was less dominant than read results. However, DigitalOcean’s
priceperformance value remains excellent, with all of its tested VMs leading in the
Raw performance testing of CPU and memory demonstrated that DigitalOcean’s standard
2CPU 2GB offering is a stand out configuration, even when compared with the T3.small,
Amazon’s most recent series released on August 21st, 2018. In the 2 vCPU compute-optimized
category, Amazon’s current VM, the C5.large, provided competitive results, although at
38% higher cost. Both AWS and DigitalOcean offerings surpassed Google’s custom Skylake
VM by a margin greater than 20%. While DigitalOcean’s 1vCPU machine was the leading VM,
it was challenged by the opportunistic, burstable GCE g1-small in performance. However,
the DigitalOcean VM exhibits far less performance variability, which makes it an advantageous
choice for small workloads with known requirements.
Based on Cloud Spectator’s testing, DigitalOcean’s local SSD raw read performance was
exceptional as compared to Amazon’s EBS and Google’s Persistent SSD Block storage. Raw
write performance results were also very good, slightly exceeding Google’s offerings.
However, DigitalOcean achieved the highest scores across all categories.
While Google provided consistent performance for both read and write operations,
Amazon’s EBS performance was approximately 10% of rival offerings during write tests.
As noted, Amazon’s Elastic Block Storage has a “burst” feature, which was demonstrated
in the pointedly high variation seen in the test results. The high-performance variability
can be remedied by purchasing larger EBS volumes with higher base IOPs and longer burst
periods. Higher capacity EBS volumes, however, will add substantially to the price of
The Cloud Spectator analysis reveals DigitalOcean is competitive with the hyperscale
providers such Amazon Web Services and Google Compute Engine. DigitalOcean combines an
aggressive pricing strategy with outstanding storage performance and excellent computational
performance. This positions DigitalOcean as a world class Cloud Service Provider.
DigitalOcean offers machines for a variety of use cases, meeting requirements that span
individual end-users, developers as well as large-scale enterprises.