Resizing my Droplet made it unbootable

February 3, 2018 138 views
Storage CoreOS

I resized my Droplet from the old 512MB RAM plan, to the new 5USD/mo 1GB RAM plan, and now it's not booting. When I view the screen via the QEMU console, it says "Booting from Hard Disk... Boot failed: not a bootable disk".

I opened a ticket 9 hours ago and I haven't gotten any response.

I've also tried taking a snapshot of the broken Droplet and creating a new droplet from that snapshot, but it just hangs at the "Booting from Hard Disk..." step and never errors out or boots. I've even tried this in another region, just in case NYC1 is having problems, but the exact same issue happens.

I can't download my snapshot, so my data is trapped in Digital Ocean's data center and I have no idea how to get it out. I also can't mount the recovery kernel because the CoreOS image doesn't have this option enabled.

What can I do?

2 Answers

I had a similar issue when I resized my droplet. It was a ubuntu 14.04 8GB droplet and when the resize was finished I saw the following error:

since I can't attach in image I'll type the error by hand:
error: no such device: xxxxx-xxxxxxx-xxxxx-xxxxx-xxxxx.

Press any key to continue...

Gave up waiting for root device. Common problems:

  • Boot args (cat /proc/cmdline)
  • Check rootdelay= (did the system wait long enough?)
  • Check root= (did the system wait for the right device?)
  • Missing modules (cat /proc/modules; ls /dev) ALERT! /dev/disk/by-uuid/xxxxx-xxxxxxx-xxxxx-xxxxx-xxxxx does not exist. Dropping to a shell!

I also got the same problem if I tried to restore from a backup.

However, I discovered that going to the 'kernel' tab and changing the kernel from the recovery kernel that it was set to to using an ubuntu 14.04 kernel seemed to fix the problem. Hopefully this helps.

CoreOS isn't able to change kernels, so I just waited a full day for the Digital Ocean support people to enable the Recovery ISO. From there I used dd and gzip to transfer the entire disk over ssh and recover files from it on my desktop machine.

The issue ended up being that the resize tool had destroyed the partition table of the disk and I needed to manually look for ext4 superblocks with xxd and grep. Hint: xxd disk.img | grep -E 'ff ?ff ?53 ?ef' will give you a reasonably sized list to look through. From there I managed to mount the disk with an offset I calculated based on the position of the superblock, recover the files, and move them over to a new VPS.

Digital Ocean support provided basically no help and they haven't even explained why resizing a droplet destroyed my filesystem.

Have another answer? Share your knowledge.