Report this

What is the reason for this report?

How To Protect Your Server Against the Meltdown and Spectre Vulnerabilities

Published on January 10, 2018
How To Protect Your Server Against the Meltdown and Spectre Vulnerabilities

What are Meltdown and Spectre?

On January 4, 2018, multiple vulnerabilities in the design of modern CPUs were disclosed. Taking advantage of certain processor performance optimizations, these vulnerabilities—named Meltdown and Spectre—make it possible for attackers to coerce applications into revealing the contents of system and application memory when manipulated correctly. These attacks work because the normal privileges checking behavior within the processor is subverted through the interaction of features like speculative execution, branch prediction, out-of-order execution, and caching.

Meltdown was disclosed in CVE-2017-5754. Spectre was disclosed in CVE-2017-5753 and CVE-2017-5715.

For more detailed information, check out the how does meltdown work? and how does spectre work? sections below.

Am I Affected by Meltdown and Spectre?

Meltdown and Spectre affect the majority of modern processors. The processor optimizations that are used in these vulnerabilities are a core design feature of most CPUs, meaning that most systems are vulnerable until specifically patched. This includes desktop computers, servers, and compute instances operating in Cloud environments.

Patches to protect against Meltdown are being released from operating system vendors. While updates are also being released for Spectre, it represents an entire class of vulnerabilities, so it will likely require more extensive ongoing remediation.

In cloud and virtualized environments, providers will need to update the underlying infrastructure to protect their guests. Users will need to update their servers to mitigate the impact within guest operating systems.

How Can I Protect Myself?

Full protection against this class of vulnerability will likely require changes in CPU design. In the interim, software updates can provide mitigation against exploits by disabling or working around some of the optimized behavior that leads to these vulnerabilities.

Unfortunately, because these patches affect the optimization routines within the processor, mitigation patches may decrease the performance of your server. The extent of the slowdown is highly dependent on the type of work being performed, with I/O intensive processes experiencing the largest impact.

Current Mitigation Patch Status

At the time of writing (January 9, 2018), Linux distributions have started to distribute patches, but no distributions are yet fully patched.

Distributions that have released kernel updates with partial mitigation (patched for Meltdown AND variant 1 of Spectre) include:

  • CentOS 7: kernel 3.10.0-693.11.6
  • CentOS 6: kernel 2.6.32-696.18.7

Distributions that have released kernel updates with partial mitigation (patched for Meltdown) include:

  • Fedora 27: kernel 4.14.11-300
  • Fedora 26: kernel 4.14.11-200
  • Ubuntu 17.10: kernel 4.13.0-25-generic
  • Ubuntu 16.04: kernel 4.4.0-109-generic
  • Ubuntu 14.04: kernel 3.13.0-139-generic
  • Debian 9: kernel 4.9.0-5-amd64
  • Debian 8: kernel 3.16.0-5-amd64
  • Debian 7: kernel 3.2.0-5-amd64
  • Fedora 27 Atomic: kernel 4.14.11-300.fc27.x86_64
  • CoreOS: kernel 4.14.11-coreos

If your kernel is updated to at least the version corresponding to the above, some updates have been applied.

Operating systems that have not yet released kernels with mitigation include:

  • FreeBSD 11.x
  • FreeBSD 10.x

Ubuntu 17.04, which is reaching end of life on January 13, 2018 will not receive patches. Users are strongly encouraged to update or migrate.

Warning: We strongly recommend that you update or migrate off of any release that has reached end of life. These releases do not receive critical security updates for vulnerabilities like Meltdown and Spectre, which can put your systems and users at risk.

Because of the severity of this vulnerability, we recommend applying updates as they become available instead of waiting for a full patch set. This may require you to upgrade the kernel and reboot more than once in the coming days and weeks.

How Do I Apply the Updates?

To update your servers, you need to update your system software once patches are available for your distribution. You can update by running your regular package manager to download the latest kernel version and then rebooting your server to switch over to the patched code.

Note: This article was written to be generally applicable and platform agnostic. If you are using DigitalOcean as your hosting provider and are running an older Droplet, you may have to perform an extra step before getting started.

DigitalOcean’s legacy kernel management system used externally managed kernels that could be changed in the control panel. If your Droplet uses this system, you will need to configure it to use internal kernel management before continuing (newer Droplets use this system automatically). To check whether you need to update to internal kernels and to learn how to make the switch, read our How To Update a DigitalOcean Server’s Kernel article.

For Ubuntu and Debian servers, you can update your system software by refreshing your local package index and then upgrading your system software:

  1. sudo apt-get update
  2. sudo apt-get dist-upgrade

For CentOS servers, you can download and install updated software by typing:

  1. sudo yum update

For Fedora servers, use the dnf tool instead:

  1. sudo dnf update

Regardless of the operating system, once the updates are applied, reboot your server to switch to the new kernel:

  1. sudo reboot

Once the server is back online, log in and check the active kernel against the list above to ensure that your kernel has been upgraded. Check for new updates frequently to ensure that you receive further patches as they become available.

Additional Context

The Meltdown and Spectre family of vulnerabilities exploit performance-enhancing features within modern processors. A combination of processor features like speculative execution, privilege checking, out-of-order execution, and CPU caching allows read access to memory locations that should be out-of-bounds. The result is that unprivileged programs can be coerced into revealing sensitive data from their memory or accessing privileged memory from the kernel or other applications.

How Does Meltdown Work?

The Meltdown vulnerability works by tricking a processor into reading an out-of-bounds memory location by taking advantage of flaws in a CPU optimization called speculative execution. The general idea works like this:

  • A request is made for an illegal memory location.
  • A second request is made to conditionally read a valid memory location if the first request contained a certain value.
  • Using speculative execution, the processor completes the background work for both requests before checking that the initial request is invalid. Once the processor understands that the requests involve out-of-bounds memory, it correctly denies both requests. Though the results are not returned by the processor after the privilege checking code identifies the memory access as invalid, both of the accessed locations remain in the processor’s cache.
  • A new request is now made for the valid memory location. If it returns quickly, then the location was already in the CPU cache, indicating that the conditional request earlier was executed. Iterative use of these conditionals can be used to understand the value in out-of-bounds memory locations.

Meltdown represents a specific vulnerability that can be patched against.

How Does Spectre Work?

Spectre also works by tricking a processor to misuse speculative execution to read restricted values. The disclosure notices describe two variants with different levels of complexity and impact.

For variant 1 of Spectre, the processor is tricked into speculatively executing a read before a bounds check is enforced. First, the attacker encourages the processor to speculatively reach for a memory location beyond its valid boundaries. Then, like Meltdown, an additional instruction conditionally loads a legal address into cache based on the out-of-bounds value. Timing how long it takes to retrieve the legal address afterwards reveals whether it was loaded into cache. This, in turn, can reveal the value of the out-of-bounds memory location.

Variant 2 of Spectre is the most complicated both to exploit and mitigate against. Processors often speculatively execute instructions even when they encounter a conditional statement that cannot be evaluated yet. They do this by guessing the most likely result of the conditional using a mechanism called branch prediction.

Branch prediction uses the history of previous runs through a code path to pick a path to speculatively execute. This can be used by attackers to prime a processor to make an incorrect speculative decision. Because the branch selection history does not store absolute references to the decision, a processor can be fooled into choosing a branch in one part of the code even when trained in another. This can be exploited to reveal memory values outside of the acceptable range.

Conclusion

Spectre and Meltdown represent serious security vulnerabilities; the full potential of their possible impact is still developing.

To protect yourself, be vigilant in updating your operating system software as patches are released by vendors and continue to monitor communications related to the Meltdown and Spectre vulnerabilities.

Thanks for learning with the DigitalOcean Community. Check out our offerings for compute, storage, networking, and managed databases.

Learn more about our products

About the author

Justin Ellingwood
Justin Ellingwood
Author
See author profile

Former Senior Technical Writer at DigitalOcean, specializing in DevOps topics across multiple Linux distributions, including Ubuntu 18.04, 20.04, 22.04, as well as Debian 10 and 11.

Category:

Still looking for an answer?

Was this helpful?
Leave a comment...

This textbox defaults to using Markdown to format your answer.

You can type !ref in this text area to quickly search our full set of tutorials, documentation & marketplace offerings and insert the link!

Hi all, for all Ubuntu 16.04 users please be aware that the kernel 4.4.0-108 is buggy!! https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1742323

Be sure to upgrade to kernel kernel 4.4.0-109 which was also released.

@jellingwood maybe you should update that in the post.

Even after doing this on a CentOS 7.4 Droplet, spectre-meltdown-checker reports:

  • Mitigation 2
  • Kernel compiled with retpoline option: NO
  • Kernel compiled with a retpoline-aware compiler: NO

STATUS: VULNERABLE (IBRS hardware + kernel support OR kernel with retpoline are needed to mitigate the vulnerability)

How do we fix this?

This comment has been deleted

If your droplet won’t switch to the latest kernel after the described updates and your droplet uses “external kernel management” (ie. changing the kernel via the control panel), you might want to consider changing to “Internal Kernel Management” following https://www.digitalocean.com/community/tutorials/how-to-update-a-digitalocean-server-s-kernel

After that you’ll be able to explicitly change to the kernel you want to use.

Ubuntu 16.04 kernel is updated again, or at least before a link to this post was emailed to me today, I had just completed an upgrade to 4.4.0-112

i have no clue what to do… for someone who is not IT at all - this is mind blowing.

Yum update seems to “remove” the old kernel and installs the new one on Centos 7

yet a uname -r shows the old one still.

a yum update shows no new packages waiting.

No idea what went wrong here.

Just to be clear. Is simply changing the Kernel from the admin console good enough? Or do we still have to update the droplet after changing the kernel? Also some of my droplets have internally managed Kernels. What does that mean exactly?

$ sudo apt-get dist-upgrade >>> is failing on Ubuntu 16.04, I get this error:

Err:1 http://download.owncloud.org/download/repositories/stable/Ubuntu_16.04 owncloud-files 10.0.5-1.1 404 Not Found [IP: 138.201.139.178 80]

let us know when we can try again, thanks!

I’m having a bit of a tough time upgrading my kernels. Running CentOS 6, these are pretty old droplets so had to be changed to the GrubLoader kernel in the control panel. After doing that, they boot to a newer kernel, which is great. But it’s not the newest kernel.

If I try running yum upgrade kernel it seems to work, but throws out grubby fatal error: unable to find a suitable template and doesn’t have a new kernel on reboot. The grub conf file doesn’t even reference the kernel that I am running, so it’s clearly being ignored. Any suggestions? Any hints on how CentOS is actually deciding what kernel to run?

I don’t want to UPGRADE my whole OS

Just give required update please

And please TIME to offer AMD CPUs, enough of this Intel crap

I’m confused about my situation. My host is running on CentOS 6.9 provided by you. But I’ve changed the kernel from 2.6.32-696.18.7.el6.x86_64 to 4.14.14-1.el6.elrepo.x86_64 recently. What should I do ?

I have Ubuntu 16.04 and when running the upgrade I am getting the following message:

"A new version of /boot/grub/menu.lst is available, but the version installed currently has been locally modified.

What would you like to do about menu.lst?"

When I select the option “show side-byside difference between the versions” I get the following:

Line by line differences between versions                                                                  │
                                 │                                                                                                            │
                                 │ Old file: /run/grub/menu.lst root.root 0644 2018-01-27 12:08:50                                            │
                                 │ New file: /tmp/fileCrfBIN root.root 0600 2018-01-27 12:08:50                                               │
                                 │                                                                                                            │
                                 │ ## ## End Default Options ## ## ## End Default Options ##                                                  │
                                 │                                                                                                            │
                                 │ title Ubuntu 16.04.3 LTS, kernel 4.4.0-97-generic title Ubuntu 16.04.3 LTS, kernel 4.4.0-97-generic        │
                                 │ root (hd0) root (hd0)                                                                                      │
                                 │ kernel /boot/vmlinuz-4.4.0-97-generic root=LABEL=clo kernel /boot/vmlinuz-4.4.0-97-generic root=LABEL=clo  │
                                 │ initrd /boot/initrd.img-4.4.0-97-generic initrd /boot/initrd.img-4.4.0-97-generic                          │
                                 │                                                                                                            │
                                 │ title Ubuntu 16.04.3 LTS, kernel 4.4.0-97-generic ( title Ubuntu 16.04.3 LTS, kernel 4.4.0-97-generic (    │
                                 │ root (hd0) root (hd0)                                                                                      │
                                 │ kernel /boot/vmlinuz-4.4.0-97-generic root=LABEL=clo kernel /boot/vmlinuz-4.4.0-97-generic root=LABEL=clo  │
                                 │ initrd /boot/initrd.img-4.4.0-97-generic initrd /boot/initrd.img-4.4.0-97-generic                          │
                                 │                                                                                                            │
                                 │ ### END DEBIAN AUTOMAGIC KERNELS LIST ### END DEBIAN AUTOMAGIC KERNELS LIST

What do I select now?

I have Droplet with Ubuntu server 17.10. Here I can read that patched version is “kernel 4.13.0-25-generic”. I have updated server with “apt-get update”, “apt-get upgrade”, “apt-get install linux-image-generic” and everything is finished fine.

But, when I check with “uname -a”, I get information the “4.13.0-19-generic” is used. How can I upgrade to latest kernel?

For Ubuntu 16.04 is “dist-upgrade” necessary ? I simply did “upgrade” instead and rebooted and my kernel version is “4.4.0-112-generic” which is higher than the recommended “4.4.0-109-generic” Isn’t a simple “upgrade” sufficient and less risky than a “dist-upgrade” ?

I’m currently running: Ubuntu 14.04.5 LTS (GNU/Linux 3.13.0-57-generic x86_64)

I ran the update commands:

sudo apt-get update && sudo apt-get dist-upgrade

I saw this show up at the end of the dist-upgrade:

update-initramfs: Generating /boot/initrd.img-3.13.0-141-generic

but after I restarted, I’m still seeming my old kernel version in the welcome message and when running uname -a

Am I doing something wrong here?

I have Ubuntu 14.04 and I do not want to update to 16.04 using “dist-upgrade” since this will cause some of my programs to stop working.

How can I update the kernel of 14.04 without upgrading to 16.04?

If it helps, I’ve just run through this process with lots of CentOS 6 boxes. The yum update kernel* worked great, but had to do two things to get the instance to pick up the new kernel 1 - in the DO control panel, under Kernel, select their GRUB loader version 0.2 2 - rpm -q kernel lists all the kernels installed. I used rpm -e to remove all the older versions - and the instance then booted into the latest kernel

Hope this helps - this is a brief summary of quite a few hours work updating tons of boxes!

I did follow the steps, rebooted the system and everything, but looks like my kernel version hasn’t been updated.

~$ sudo apt-get dist-upgrade Reading package lists… Done Building dependency tree Reading state information… Done Calculating upgrade… Done The following packages were automatically installed and are no longer required: libc-ares2 libv8-3.14.5 linux-headers-3.13.0-68 linux-headers-3.13.0-68-generic linux-image-3.13.0-68-generic linux-image-extra-3.13.0-68-generic Use ‘apt-get autoremove’ to remove them. 0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.

~$ hostnamectl status Static hostname: dc-01 Icon name: computer-vm Chassis: vm Boot ID: 6658869fb9574f3186f8201ab51d44b8 Operating System: Ubuntu 14.04.5 LTS Kernel: Linux 3.13.0-57-generic Architecture: x86_64

Help! I am on a CentOS 6 droplet that is from 2013 (very old), and it has been on kernel 2.6.32-431.1.2.0.1.el6.x86_64. I did sudo yum update which was fine, but after sudo reboot it stayed on the old kernel. (I manage another newer CentOS 6 droplet from 2017, which was on 2.6.32-696.1.1.el6, and after reboot it went up to2.6.32-696.20.1.el6 so that is fine.)

To solve this, I tried to follow this: https://www.digitalocean.com/community/tutorials/how-to-update-a-digitalocean-server-s-kernel and did sudo rpm -e 2.6.32-431.1.2.0.1.el6.x86_64. After that, my instance no longer booted!

I had to restore from backup. Now, I’m back on older kernel, and I have no way to upgrade it.

Linode for example took care of this for their users. I don’t know why you ask your users to do this at their end. You should take care of it, because this is serious and can break up things. I noticed that it’s a trend to ask your users to do everything, even if it’s something that belongs to your core responsibilities. I followed the instructions above and I don’t know if this upgraded the kernel to 4.4.0-109 or not. I just followed what it’s above.

As expected, my sites went down and I had to restore my backup now. The backup restore worked fine though. I will submit a ticket. Apparently, the instructions above are not enough.

I was referred to Digital by a trusted techy colleague Because everything just worked and they have good support. Well, i have client website down and my techy colleague is overseas. Also, i have 20,000 email campaign launching in 8 hours with a push to social. I’m not techy and have no idea how to fix! HEEEEEEELP!!!

Will I get a newer kernel if I resize the droplets? I resized one droplet last time and now I am planning to update it’s kernel but when I checked it, it was already updated.

I don’t see any of the latest Ubuntu kernels in the dropdown list. I tried to upgrade to the latest one that is on the list (Ubuntu 16.04 x64 vmlinuz-4.4.0-28-generic), but my ssh keys were no longer on the server. I could not hit the I could even hit the website it runs. It just times out. I went to the help pages for upgrading to Ubuntu and there is literally nothing there. It shows the title and there is no article.

Hi all,

I’m looking for a straight forward (plain talking) answer to the (unanswered) question under the above Contents “Am I Affected by the Meltdown and Spectre?”

The best offering I’ve been able to find from the ongoing comments is to use: https://github.com/speed47/spectre-meltdown-checker

After having run this script, it seems my tested droplet is indeed vulnerable to one of the variants - despite having taken a snapshot, apt-get update, apt-get dist-grade, reboot!

I also noticed (above) comments relating to running ‘uname -ir’ but then failed to state what to expect in the response. Mine states: ‘4.4.0-112-generic x86_64’ is that okay? I’m guessing not, since the script still shows the vulnerability.

So, let’s say we’ve now determined that our droplet has one (or more of ) the vulns, what are the next steps? What happens if you’ve carried out all the update advice but still get the vuln, what next? Watch this space?

I appreciate the ‘Current Mitigation Patch Status’ section states my distribution (Ubuntu 16:04) has a partial mitigation, but it kind of feels like I’ve been left hanging and don’t know what to do next.

However, that could just be me and I could have very easily missed something. Would appreciate any advice.

Running apt-get dist-upgrade on an Ubuntu 14.04 droplet brings me to a dialog on configuring grub-pc. What should I select? #2 is the default selection.

Configuring grub-pc  
A new version of configuration file /etc/default/grub is available, but the version installed currently has been locally modified.

What do you want to do about modified configuration file grub?

1) install the package maintainer's version
2) keep the local version currently installed
3) show the differences between the versions
4) show a side-by-side difference between the versions
5) show a 3-way difference between available versions
6) do a 3-way merge between available versions (experimental)
7) start a new shell to examine the situation

Thanks for your help! This is a busy production site so I don’t want to mess things up :)

I’ve run this: sudo apt-get update && sudo apt-get dist-upgrade And on reboot it’s showing my kernel as Ubuntu 16.04.4 LTS (GNU/Linux 4.4.0-116-generic x86_64)

Does this include the patches? Is there a command I can run to verify the patches were installed?

I see updating kernel inside droplet protects only against hostile users inside that droplet.

But how about other droplets on the same physical CPU? Is it safe to assume that same CPU “neighbor droplet” owner can’t access the memory space of our droplet via the meltdown bug?

Thanks for clarifications!

Creative CommonsThis work is licensed under a Creative Commons Attribution-NonCommercial- ShareAlike 4.0 International License.
Join the Tech Talk
Success! Thank you! Please check your email for further details.

Please complete your information!

The developer cloud

Scale up as you grow — whether you're running one virtual machine or ten thousand.

Get started for free

Sign up and get $200 in credit for your first 60 days with DigitalOcean.*

*This promotional offer applies to new accounts only.