No internet/connection after droplet reboot

April 23, 2019 732 views
Networking
lukasea17033bb7e7ecf80fc21
By:
lukasea17033bb7e7ecf80fc21

After rebooting Ubuntu 18.04 Droplet I couldn’t access it via SSH. Then I realized that I don’t receive any response when pinging its IP. I connected to my Droplet through DigitalOcean built-in terminal and tried to ping other remote servers but this also didn’t work. So it seems that my Droplet is disconnected from the internet. I tried rebooting, shutting off/turning on. Nothing helped. I tried to Google for possible reasons and searched DigitalOcean’s QA section but didn’t find any working solution. Here is what I found so far:

  • if i run “sudo ip link set eth0 up” it returns “Cannot find device "eth0”
  • if i run “ifconfig -a”, there is no eth0, only ens3 and lo
  • if i run “dmesg | grep -i eth”, i get “virtio_net virtio0 ens3: renamed from eth0”. if i do the same on other Droplets (where internet is working) i get

“virtionet virtio0 ens3: renamed from eth0;
virtionet virtio0 eth0: renamed from ens3”

i.e. eth0 is renamed to ens3 and then back to eth0. So maybe if I managed to rename ens3 to eth0 on my broken Droplet I would be able to restore internet connection but I don’t know how to do this. Also I am not sure why it got broken in the first place. I created a support ticket for this but i didn’t get any response yet.

Log In to Comment
3 Answers
lukasea17033bb7e7ecf80fc21 April 30, 2019
Accepted Answer

After almost two weeks since I raised this issue, I was finally able to resolve it. The problem occurred in the first place because different packages got uninstalled somehow when I rebooted. This includes cloud-init, ufw and landscape-common. There is probably more that I haven’t noticed yet. Let’s start with the internet connection, this is how I fixed it:

Step 1. I created file “/etc/udev/rules.d/70-persistent-net.rules” and added

SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="b2:fe:09:35:6e:57", NAME="eth0"

as I mentioned previously. Note that MAC address used above can be found here: “/etc/netplan/50-cloud-init.yaml”

Step 2. Run “sudo reboot” and check that eth0 interface persists by running “ifconfig -a”

Step 3. Run the following command:

sudo ifconfig eth0 207.154.193.4 netmask 255.255.240.0 up

Your IP address and netmask can be found if you go to your Droplet’s page and then go to Networking tab. It is important that both these are correct.

Step 4. Run another command:

ip route add default via 207.154.192.1

This is your gateway address which again can be found on Networking tab on your Droplet’s page. At this point you should be able to access your server through SSH. If you try to run “ping www.google.com” it will fail to resolve the domain. Also if you do “sudo reboot”, the configuration won’t persist and you will need to repeat steps 3-4 again. So let’s continue.

Step 5. Run command:

sudo nano /etc/resolv.conf

and change nameserver to “8.8.8.8”, i.e. Google’s DNS, and save the file. Now you should be able to run “ping www.google.com” and receive response.

Step 6. Now I ran these:

sudo apt update
sudo apt upgrade

not sure if this was necessary though.

Step 7. Let’s install the missing cloud-init so that network configuration persists next time we reboot. We can do this by running:

sudo apt install cloud-init

Now you should be able to run “sudo reboot” without losing network connection.

Step 8. Finally, I installed some missing packages by running these:

sudo apt install ufw
sudo apt install landscape-common

landscape-common will enable “landscape-sysinfo” command which is used to show server information on your welcome screen when you connect to your server.

There is probably more stuff that is missing and my Droplet is not exactly the same as it was when I first created it but at least I was able to restore network connection, missing packages that I use and continue running my scripts are before. I hope this hasn’t left any security hole in my system. Probably it would be wise to create new Droplet from scratch at some point.

Reply
jarland MOD April 23, 2019

Greetings!

In recent cases where I’ve seen this occur, somehow cloudinit had been uninstalled on the server. I never figured out how, but it might be worth checking the apt logs with something like:

grep "cloud" /var/log/apt/history.log

A few of us on the support team put this together to try to help as well:
https://www.digitalocean.com/community/questions/having-trouble-with-the-network-on-your-droplet

Checking /etc/netplan/50-cloud-init.yaml may offer some insight. Here’s a clean example from a droplet I just created a few minutes ago:

root@django-s-1vcpu-2gb-nyc3-01:~# cat /etc/netplan/50-cloud-init.yaml
# This file is generated from information provided by
# the datasource.  Changes to it will not persist across an instance.
# To disable cloud-init's network configuration capabilities, write a file
# /etc/cloud/cloud.cfg.d/99-disable-network-config.cfg with the following:
# network: {config: disabled}
network:
    version: 2
    ethernets:
        eth0:
            addresses:
            - 134.209.38.178/20
            - 10.17.0.19/16
            gateway4: 134.209.32.1
            match:
                macaddress: ce:22:f3:b0:93:75
            nameservers:
                addresses:
                - 67.207.67.2
                - 67.207.67.3
                search: []
            set-name: eth0

If all else fails you can use the recovery ISO to bypass anything OS specific and test, as well as transfer data out if needed:
https://www.digitalocean.com/docs/droplets/resources/recovery-iso/

Jarland

Reply
  • lukasea17033bb7e7ecf80fc21 April 23, 2019

    When i run

    grep "cloud" /var/log/apt/history.lo

    this all I see:

    https://imgur.com/JQpBwAy

    What does this tell me about cloudinit?

    50-cloud-init.yaml file seems correct. The problem is when I run

    ifconfig -a

    https://imgur.com/g8l8Fqg

    Here the first interface is named ens3 and not eth0 like it is when you create a new Droplet. Also, I don’t want to use recovery tools or create new Droplet because I would like to figure out a clean way to fix this. This shouldn’t be happening in production servers and if it is, there must be a better way to fix it than using recovery tools.

lukasea17033bb7e7ecf80fc21 April 25, 2019

Here is some more info:

sudo netplan apply

does not work and gives “ sudo: netplan: command not found”. I also tried this:

  • i created /etc/udev/rules.d/70-persistent-net.rules file and added this info:

SUBSYSTEM==“net”, ACTION==“add”, DRIVERS==“?*”, ATTR{address}==“b2:fe:09:35:6e:57”, NAME=“eth0”

then i ran

ifconfig -a

and ens3 changed to eth0 but still no internet connection. also inet, netmask and broadcast line was missing.

  • then i ran sudo ifconfig eth0 207.154.193.4 netmask 255.255.240.0 the missing inet, netmask and broadcast now appeared. also, now it said RUNNING. I tried to ping www.google.com but still there was no internet. So the issue remains unsolved.
Reply
Have another answer? Share your knowledge.

Related