Related

Join the DigitalOcean Community Community

Join 1M+ other developers and:

  • Get help and share knowledge in Q&A
  • Subscribe to topics of interest
  • Get courses & tools that help you grow as a developer or small business owner
 Join Now
Can't install php-soap to my PHP 7.0 centos Question Question
How to connect a DigitalOcean domain to Wix by Pointing Question Question

Question

Unable to boot from my droplet's hard drive (CentOS)

Posted December 15, 2020 1.4k views
CentOSConfiguration ManagementSystem ToolsDigitalOcean Droplets

I have been at this for days without any solution. I’ve been back and forth with DO support, but they have been unable to help me.

My droplet is using CentOS 7 (RHEL). In the Recovery section in my Droplet’s settings, when I try to boot from the hard drive (selecting the ‘boot from hard drive’ option), I checked my DO web console and I get this error:

error: symbol 'grub_calloc' not found.
Entering rescue mode...
grub rescue>

I’ve had a dozen browser tabs open with websites and articles, and have exhausted Google trying every suggested solution I could find. A lot of the solutions were for Ubuntu, so I was trying to find the CentOS equivalent, or just try the steps anyway and hoped they worked. I have not worked with Ubuntu as much. These are the things I’ve done:

  • Powered off/on the droplet
  • Power-cycled the droplet
  • Went into recovery mode, mounted the disk (option 1), then went to the chroot option (option 5)

From there I did these things that I’ve found all over the internet:

  1. Checked my kernel by typing umame -sr
    It returned:
    Linux 4.15.0-23-generic

  2. Updated/upgraded the kernel (dnf update kernel, dnf upgrade kernel)

  3. Tried installing an older kernel

  4. Updated/upgraded grub (Grub2 on CentOS 7)

  5. Reinstalled Grub

  6. Ran this command to see all of the CentOS versions that are installed on the disk:
    awk -F\' '$1=="menuentry " {print $2}' /etc/grub2.cfg

These are the two entries I have:

CentOS Linux 7 Rescue [long alphanumeric string] (3.10.0-693.17.1.el7.x86
CentOS Linux (0-rescue-[long alphanumeric string]) 7 (Core)

  1. Attempted to mount the boot partition (it’s located on /dev/vda15).
    I typed:
    mount /dev/vda15
    I got an error:
    mount: can't find /dev/vda15 in /etc/fstab

  2. Set the Boot Default in Grub. I only had two versions to choose from (see step 6).
    I typed:
    grub2-set-default 0
    the powered down the droplet, switched to boot from the hard disk, then powered it up again. I still got the 'Grub Rescue’ screen. So I powered the droplet back down, switched to Recovery mode again, powered up, and tried the other boot option:
    grub2-set-default 1
    Rinse and repeat with the power up/down the droplet, and I still got the Grub Rescue screen.

  3. Tried grub2-mkconfig which was a popular solution everywhere, but this was the error I kept getting when I typed that: 

    /usr/sbin/grub2-probe: error: failed to get canonical path of 'overlay'

I couldn’t for the life of me find that error anywhere on Google or any searches where other people had that same error. There was a similar one about it not being able to get the canonical path of ’/cow’, but I couldn’t determine if it was the same issue I was having.

I’m at a loss of what to do. I can’t seem to get my droplet to successfully boot from the hard drive. I am able to get into my files and things when I boot through the Recovery Mode, but I don’t want to keep my droplet set on Recovery Mode forever. I have a lot of websites connected to this droplet.

Is there anything I can do to fix this? Or will I have to completely destroy the droplet and rebuild? I just don’t want to lose everything. I have databases and things that I need to preserve. I already took a snapshot of my droplet as soon as this problem happened. I am hoping to be able to fix the hard disk’s boot sector and migrate my droplet data back.

Thank you in advance for all your help!!

Add a comment
knuxchan
By:
knuxchan

Related

Join the DigitalOcean Community Community

Join 1M+ other developers and:

  • Get help and share knowledge in Q&A
  • Subscribe to topics of interest
  • Get courses & tools that help you grow as a developer or small business owner
 Join Now
Can't install php-soap to my PHP 7.0 centos Question Question
How to connect a DigitalOcean domain to Wix by Pointing Question Question

These answers are provided by our Community. If you find them useful, show some love by clicking the heart. If you run into issues leave a comment, or add your own answer to help others.

×
Submit an Answer
1 answer
KFSys December 15, 2020

Hi @knuxchan,

You are in a though situation. It seems like you’ll need to write your grub config manually if everything else fails.

Additionally, you should focus on the error you’ve see when trying to regenerate your grub config

/usr/sbin/grub2-probe: error: failed to get canonical path of 'overlay'

To solve your current problem, you’ll have to change the UUID(s) in “/etc/fstab” and “/boot/grub/grub.cfg”. First get the proper UUID, you’ll need that to type in your /etc/fstab

blkid

You can use the above to get it. Next the fstab should look like the following

UUID=${UUID_HERE} /                       xfs     defaults        0 0

Additionally, you’ll need to edit the boot grub.conf as well /boot/grub/grub.conf

This should look like

default=0
timeout=0


title CentOS Linux 8 (4.18.0-240.1.1.el8_3.x86_64)
    root (hd0)
    kernel /boot/vmlinuz-4.18.0-240.1.1.el8_3.x86_64 ro root=UUID={UUID_HERE} console=hvc0 LANG=en_US.UTF-8
    initrd /boot/initramfs-4.18.0-240.1.1.el8_3.x86_64.img

Please note that the kernel will be different than the one provided above^^. You can see your vmlinuz and initramfs in your /boot partition and actually replace it with that.

In regards to typing the kernel manually, you can re-type what you see in your /boot/grub/grub.conf while in rescue mode as well.

Regards,
KFSys

Reply Report
  • knuxchan December 15, 2020

    Hi. Thanks for your response.

    When I go /etc/fstab, I noticed the UUID is currently set to the /dev/disk system that is currently mounted (/dev/vda1). The boot disk system is located in /dev/vda15. Should I still change the UUID to the /dev/vda15/ in here even though it’s not the disk that’s currently mounted?

    This is what my /etc/fstab looks like:

    #
# /etc/fstab
# Created by anaconda on Mon May 22 18:47:37 2017
#
# Accessible filesystems, by reference, are maintained under 'dev/disk'
# See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info
#
UUID=[uuid of /dev/vda1] /
ts
/swapfile none swap sw 0 0

    I don’t have the file /boot/grub/grub.conf
    The only file that is similar is /boot/grub2/grub.cfg however, it says at the top of the file: ‘DO NOT EDIT THIS FILE. It is automatically generated by grub2-mkconfig using templates from /etc/grub.d and settings from /etc/default/grub/’.

    When I go to those files, I’m not getting what is shown in your example above from /boot/grub/grub.conf

    /etc/grub.d is a directory and contains these files (I assume these are the template files):

    00_header
00_tuned
01_users
10_linux
20_linux_xen
20_ppc_terminfo
30_os-prober
40_custom
41_custom

    /etc/default/grub is a file that contains this:

    GRUB_TIMEOUT=5
GRUB_DISTRIBUTOR="$(sed 's, release .*$,,g' /etc/system-release)
GRUB_DEFAULT=saved
GRUB_DISABLE_SUBMENU=true
GRUB_TERMINAL_OUTPUT="console"
GRUB_CMDLINE_LINUS="crashkernel=auto rhgb quiet"
GRUB_DISABLE_RECOVERY="true"

    Any ideas?

    Reply Report
    • KFSys December 20, 2020

      Hi @knuxchan,

      First let’s begin with the FStab, you should set the fstab as the system that’s mounted your /.

      As for the /boot/grub/grub.conf try to create it following the exmaple I gave above. Of course, you can check your boot folder to see which kernel to use. Maybe you’ll have the same as the one I’ve posted in the example.

      As for the file /etc/default/grub I can confirm, that’s alright and normal!

      Reply Report
      • knuxchan December 21, 2020

        Hi, @KFSys thank you so much for your help and replies!
        I managed to finally get this issue fixed, after going back and forth with DO Support. This is what I had to do in case others have this same issue:

        I restarted the droplet, and then without mounting the filesystem, went into the interactive shell (option #6) and ran the following commands:

        mount /dev/vda1 /mnt
mount --bind /dev /mnt/dev
mount --bind /dev/pts /mnt/dev/pts
mount --bind /proc /mnt/proc
mount --bind /sys /mnt/sys
chroot /mnt/
grub2-mkconfig -o /boot/grub2/grub.cfg
grub2-install /dev/vda

        The error I got on the first command was because of mentioning xfs type, while the current FS type here is ext4.

        After I ran these commands, Grub was installed successfully, and I was able to boot from my droplet’s hard drive again!

        Reply Report

Related

Looking for something else?

Ask a new question Search for more help