Having application issues after spectre/meltdown reboots?
(Written by Nate, Mike, and Jonathan from DO Support Team)
Due to reboots as a result of our efforts to mitigate the Spectre and Meltdown vulnerabilities, we’ve seen a number of customers reporting issues including failure to boot, services failing to start, etc. While in most cases, these issues are not caused by the reboot itself, or DigitalOcean’s platform, our goal is to help our customers troubleshoot as quickly and as efficiently as possible.
Below we’ve provided a few links to guides that will help troubleshoot common issues that will help you, as a customer, and us on the support side better understand where the issue may lie. If you’re unable to determine the root cause after troubleshooting, please don’t hesitate to reach out and our support team will be happy to further assist!
Troubleshooting Network Access and Connectivity
Networking can fail if you are running an older droplet which has a kernel selection box in the control panel. This selection box chooses the kernel that we try to boot, and just because the system boots does not mean that the correct one is selected. In this case, you will want to follow this guide to make sure you are booting under the right kernel:
A valid test may simply be to set the kernel to our Grub Loader kernel and see if this restores your networking. If you are already on the Grub Loader kernel, it can be beneficial to try setting the original kernel used on the droplet. You may not be aware of what that kernel was, and our support team may be able to assist you with this via support ticket.
Attempting to ping your Droplet will allow you to confirm that your Droplet is reachable over network from either your local environment, or another Droplet or server. If you’ve disabled ICMP requests, ping will fail. This is common if you’re using the default ruleset provided by our Cloud Firewall service, or if you’ve modified
firewalld, or your firewall of choice to do the same.
ping -c 5 126.96.36.199
The above will send 5 packages to the IP
188.8.131.52 and return packet loss (if any) and round-trip time. The response times for ping will vary based on your location and where your Droplet is located, thus for example, pinging a Droplet in NYC3 from India will likely be higher than someone pinging a Droplet in NYC3 from within the United States.
- How To Use Nmap to Scan for Open Ports on your VPS
- How To Test your Firewall Configuration with Nmap and Tcpdump
nmap will allow you to run a port scan against your Droplets IP to confirm whether a port is open, filtered, or closed. Most commonly, we see that ports 80 (HTTP) or 443 (HTTPS/SSL) have been firewalled off, thus preventing web access to your website or application.
nmap -p 22,80,443
The above command will check ports 22 (SSH), 80 (HTTP), and 443 (HTTPS/SSL) for a response and return their current status.
mtr allows you to troubleshooting connectivity issues between Droplets, or from your local environment to a Droplet. It provides details such as latency, hops between routes, etc.
Using the DigitalOcean Web Console
If you’re unable to access your Droplet via SSH, whether due to the firewall blocking access, or due to a larger issue, DigitalOcean provides access to a web based console. The web Console functions much like a physical attached console in that even if SSH isn’t working, you should be able to access your Droplet.
The web Console does require a root password to be used as it does not currently provide support for SSH keys. If you’ve locked the root user, or you’ve forgot the password for the root user, you may need to do a password reset before you’ll be able to access console. To reset your root password, please:
- Visit https://cloud.digitalocean.com
- Click on the name of the Droplet you wish to reset the root password on.
- Click on Access from the left side menu.
- Under Reset Root Password, please click on the “Reset Root Password” button.
Note: This will reboot your Droplet.
Troubleshooting Your Web Server (Apache/NGINX)
One of the most common web server issues is a 40x or 50x error in the browser. This indicates that the Droplet itself is working properly, however the web server may not be able to serve the request as expected. For information on web server status codes, please see:
The most popular or often seen error codes are:
- 200 - OK
- 404 - File Not Found
- 500 - Internal Server Error
- 503 - Service Unavailable
- 504 - Gateway Timeout
When it comes to errors, the
error log for the web server is going to be the best file to refer to. This file will often detail additional information associated with the error received and in some cases, application specific information that can also be helpful in determining the root cause. We can use the
tail command to check the last X lines of the log (where X is any number you choose).
tail -20 /var/log/apache2/error.log
tail -20 /var/log/nginx/error.log
Troubleshooting Virtual Hosts (Apache) and Server Blocks (NGINX)
When it comes to Virtual Hosts and Server Blocks, a missing
} or and extra line or space can often be the difference between the web server starting and failing. While the guides below do target Ubuntu, the configuration for Virtual Hosts and Server Blocks is not OS-specific and applies globally to all distributions.
- How To Set Up Apache Virtual Hosts on Ubuntu 14.04 LTS
- How To Set Up Nginx Server Blocks (Virtual Hosts) on Ubuntu 16.04
Common Linux Troubleshooting
Troubleshooting Logs (ex: system)
- How To View and Configure Linux Logs on Ubuntu and Centos
- How To Manage Logfiles with Logrotate on Ubuntu 16.04
Reaching Out to DigitalOcean Support
If the above guides do not aid in resolving the issue that you’re currently experiencing, please feel free to reach out to our support team by submitting a support ticket within our support portal:
Please keep in mind, our support team does not have access to the internals of your Droplet, thus we’re unable to execute commands or pull information directly. Any details that you can provide from the troubleshooting steps above will assist us in troubleshooting the issues you’re experiencing and working with you towards a resolution.