I’m having issues with DNS on my Droplet. It cannot connect to outside servers, even ping doesn’t work. Not even dnf update.

So I followed an earlier suggestion found in this forum to persist the resolv.conf file.

  1. I changed the /etc/resolv.conf to use Google’s servers.
# Generated by NetworkManager
nameserver 8.8.8.8
nameserver 8.8.4.4

  1. Then to make it persistent, I added “dns=none” in the [main] section of /etc/NetworkManager/NetworkManager.conf
[main]
plugins = ifcfg-rh,
dns=none

  1. Then I restarted NetworkManager:
sudo systemctl restart NetworkManager.service

I tried ping after this. No go. Still not working.

So I rebooted this droplet.

Still the same, ping not working. AND the /etc/resolv.conf file got changed to the default again:

nameserver 127.0.0.53
options edns0 trust-ad

What can I do? Is this related to cloud.init? Something unique in Fedora? Never happened to me in CentOS. RedHat suggests to make resolv.conf a symlink instead, is that something we need to do in Fedora??

These answers are provided by our Community. If you find them useful, show some love by clicking the heart. If you run into issues leave a comment, or add your own answer to help others.

×
5 answers

TL;DR: The simplest solution is to force use of your preferred DNS servers in /etc/systemd/resolved.conf. For example:

DNS=67.207.67.2 67.207.67.3

Then restart systemd-resolved:

$ sudo systemctl restart systemd-resolved.service

Now you’re good to go! Ensure it’s working withresolvectl query example.com (or with dig).

Now, for the longer answer: I’m afraid looking at RHEL 8 documentation is going to mislead you here. In RHEL 8 and Fedora 32, /etc/resolv.conf was managed by NetworkManager. But in Fedora 33, it is now managed by systemd-resolved, and is mostly ignored by default: software that uses glibc for name resolution – which is most software – will not read /etc/resolv.conf at all. More info here.

DigitalOcean’s freshly installed Fedora 33 droplets default to using Cloudflare’s 1.1.1.1 as fallback DNS because no other DNS servers are configured. Problem is, this is not really ever supposed to happen in practice: the assumption is that users would almost always receive a more specific DNS server via DHCP that would override the fallback DNS servers. But it seems that does not happen with DigitalOcean’s droplets. That is arguably a misconfiguration to be addressed – for instance, I notice that DigitalOcean’s Ubuntu droplets default to using DigitalOcean’s 67.207.67.2 and 67.207.67.3 DNS servers instead, which is better – but it’s not initially a big deal because Cloudflare is fine. Problem comes after your first system update. Next time you reboot (or restart systemd-resolved) after your first update, you’ll have no more DNS servers configured at all. Why? Because some users complained about fallback to Cloudflare and Google, so the fallback was removed in systemd-246.7-1.fc33. Consequence of that is that anyone relying on fallback DNS – including DigitalOcean’s F33 droplets – now no longer has any DNS configured at all after you update systemd. There are several possible ways to fix this, but since cloud servers don’t need any fancy dynamic configuration like desktops do, I suggest simply editing /etc/systemd/resolved.conf as shown above.

My opinion is that this problem is a bug on DigitalOcean’s end. Although I’m not a fan of Fedora removing the fallback DNS servers – and in retrospect, changing the behavior post-release was a clear mistake, oops! – it’s surely not intended for the fallback servers to be used in the first place. I see that DigitalOcean’s Fedora 32 droplets create a hardcoded /etc/resolv.conf using cloud-init:

; Created by cloud-init on instance boot automatically, do not edit.
;
nameserver 67.207.67.2
nameserver 67.207.67.3

Clearly that doesn’t work anymore. Ideally DigitalOcean would figure out how its Ubuntu droplets get configured with DigitalOcean’s DNS servers, then try to replicate that with Fedora’s droplets, since Ubuntu and Fedora 33 both use systemd-resolved and both use NetworkManager to configure systemd-resolved, so configuration should be almost identical (with the exception that Ubuntu still reads /etc/resolv.conf, but that shouldn’t matter for this).

  • Note that my proposed solution will stop working when the underlying bug is resolved (hehe), so this is only a temporary workaround. Hence if you are finding this answer in 2025, you probably want to look elsewhere. Configuring DNS settings with dns= only works when there are no link-specific settings to override them. And once this problem is fixed, there will be link-specific settings again. But until then, it’s a great easy way to get DNS working without having to fiddle with anything else. You can use resolvectl to see what your current DNS settings are. With my workaround, you should have:

    $ resolvectl
    Global
             Protocols: LLMNR=resolve -mDNS -DNSOverTLS DNSSEC=no/unsupported
      resolv.conf mode: stub                                                 
    Current DNS Server: 67.207.67.2                                          
           DNS Servers: 67.207.67.2 67.207.67.3                              
    
    Link 2 (eth0)
    Current Scopes: LLMNR/IPv4 LLMNR/IPv6                                       
         Protocols: -DefaultRoute +LLMNR -mDNS -DNSOverTLS DNSSEC=no/unsupported
    
    Link 3 (eth1)
    Current Scopes: LLMNR/IPv4 LLMNR/IPv6                                       
         Protocols: -DefaultRoute +LLMNR -mDNS -DNSOverTLS DNSSEC=no/unsupported
    

    But a proper fix would have DNS servers configured on Link 2 or Link 3 instead, with +DefaultRoute, all by default so that it’s working without you needing to do anything to your droplet. I’m not certain, but I’m increasingly confident this is a bug in cloud-init. I’ve failed to figure out how the DNS settings are getting set properly for Ubuntu, though.

    • As of today (2021-02-24), your solution has been somehow implemented by DO. I see DigitalOcean[...]: writing resolvers to /etc/systemd/resolved.conf in my logs.

      Unfortunately, they placed a comma between the IP addresses rather than a blank which breaks the resolver again. Even worse, it’s done on every boot. For now, I rewrite /etc/systemd/resolved.conf in the bootcmd section of the cloud init user data. I’ve created a ticket with DO…

Found the answer, I think. The issue indeed is that /etc/resolv.conf does not stay persistent in Fedora. Changing it to symlink does work for now.

https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/8/html/configuring_and_managing_networking/manually-configuring-the-etc-resolv-conf-file_configuring-and-managing-networking

Hope this helps someone else.

PS. Just to add, this has nothing to do with my firewall.

This one fixed my problem. Fedora 33.

After reboot DNS will get lost. Run this command.

sudo nmcli con mod System\ eth0 ipv4.dns "8.8.8.8 8.8.4.4"

And then reboot the box again. Things got fixed after that.

I contacted the DO support. This is a known issue and they are working on it. With the default settings in Fedora 33, they suggest this temporary workaround:

sudo resolvectl dns eth0 8.8.8.8 4.4.4.4

This command must be run after each reboot. I put the following lines in the droplet’s user data:

#cloud-config
bootcmd:
- [ resolvectl, dns, eth0, "8.8.8.8", "4.4.4.4" ]

This solved the issue for me. You can check the status with resolvectl. See also mcatanzaro’s answer above :-)

Kudos to the DO team for the quick response!

Submit an Answer