Question

Dropblet filesystem wont mount

Posted August 30, 2020 266 views
CentOSBackups

Hello,

i’m facing a new problem with my droplet that’s said:

mounted filesystem with ordered data mode. opts: discard

That don’t allow me to login into my droplet at all. i tried to use the droplet console in recovery mode but i couldn’t figure out what is the problem !!

any suggestion! my droplet is fully down right now.

These answers are provided by our Community. If you find them useful, show some love by clicking the heart. If you run into issues leave a comment, or add your own answer to help others.

×
6 answers

Is that will be mounted by the digital ocean support team ?? because i didn’t find a way to do it !!!!

Hi,
I must admit I have never met the message (and situation) like that. If the data held on your droplet is crucial, I recommend you to contact DO support team, and to not perform any action unless they instruct you to do that.

But, let’s assume that your data is backed up, and you have nothing to loose. So, what I would do, is:
1. Using DO control panel take a snapshot of the droplet, just in case.

2. In control panel switch to recovery environment (Boot from Recovery ISO), and launch recovery console.

3. From the menu choose Check Filesystem. Some messages should be displayed then. Note from them what is your file system, for further purposes.

4. Shutdown recovery environment, in control panel switch to your regular droplet’s environment (Boot from Hard Drive), and turn the droplet on.

5. If the problem still persists, switch to recovery console again to use more powerful filesystem checking tool.

6. From the menu choose Interactive Shell.

7. Let’s assume that your droplet’s filesystem is XFS (as it is in Centos 8 droplet). You could figure out the filesystem type when you did Check Filesystem in step 3.

8. To find out what is your droplet’s disk, execute command

fdisk -l

It will be a message displayed, looking like that:

Disk /dev/vda: 25 GiB, 26843545600 bytes, 52428800 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: 8A5128D6-37BC-4C45-918E-A322C04589E1

Device     Start      End  Sectors Size Type
/dev/vda1   6144 52428766 52422623  25G Linux filesystem
/dev/vda2   2048     4095     2048   1M BIOS boot

Find the section of a disk which size equals your droplet’s size. In above case it is /dev/vda1

9. Run xfs_repair utility. This utility is intended for XFS filesystem only. If your droplet’s disk have different filesystem than XFS, you have to use the other utility.

xfs_repair device

Accordingly to above, examplary configuration, it would be:

xfs_repair /dev/vda1

10. When filesystem checking is finished, shutdown recovery environment, switch to your regular droplet’s environment, and turn the droplet on.

11. Check if droplet works correctly. Update us with the current situation :-)

  • Hello, after executing xfs_repair /dev/vda1 i got this:

    ERROR: The filesystem has valuable metadata changes in a log which needs to be replayed. Mount the filesystem to replay the log, and unmount it before re-running xfs_repair. If you are unable to mount the filesystem, then use the -L option to destroy the log and attempt a repair. Note that destroying the log may cause corruption – please attempt a mount of the filesystem before doing this.

    I guess the i have to mount something but i don’t know how to do it?

Hi,

It means XFS has got dirty log, and xfs_repair could not be ran. So,…

1. Launch recovery console, choose Interactive Shell in menu.

2. Create directory for mounting disk.

mkdir /mnt/sysimage

3. Try to mount your disk.

mount /dev/vda1 /mnt/sysimage

4. Update us how it went.

  • 1- i did what did you suggest and i turn off/on the Droplet into the Boot from Hard Drive mode and try to console again but i got a lot of messages ... denied { open } ..

    2- i re used the interactive Shell and re mout the image as you suggest and i run the xfs_repair /dev/vda1, i got the following message:

    /dev/vda1 contains a mounted filesystem
    /dev/vda1 contains a mounted and ritable filesystem

    fata error – couldn’t initialize XFS library

    • OK. You did too many steps; xfs_repair does not work on disk which is mounted. But the good news is you could mount your droplet’s disk. It is quite likely that XFS log is clean now, so follow these steps:

      1. Launch recovery console, choose Interactive Shell in menu.

      2. Run XFS repair utility.

      xfs_repair /dev/vda1
      

      It may take time, so be patient, do not interrupt it.

      Provide us an update how it went.

      • i run this command and it run normally i guess till phase 7 and i got a done in the end.

        what to do ?

        1- close the interactive shell directly or powered off the droblet directly ?

        2- turn it on Hard Drive mode an re try to connect ?

        • The safest way (but not necessary) is to shutdown rescue environment in Interactive Shell

          shutdown -h now
          

          You can close rescue environment terminal window now (it is browser, actually).

          Switch to Hard drive mode. Turn your droplet on. Launch console. Let us know how it went.

      • 1- I exit the interactive Shell.
        2- Switch t Hard Drive mode.
        3- Powred off and on the Droplet.
        4- Access to the Droblet Console.
        5- nothing change i guess and i got this messages:

        EXT4-fs (sda): INFO: recovery required on readonly filesystem
        EXT4-fs (sda): write access will be enabled during recovery
        EXT4-fs (sda): recovery complete
        EXT4-fs (sda): mounted filesystem with ordered data mode. (null)
        random: crng init done
        auditprintkskb: 42 callbacks suppresed.

        After this i got this repeted same message in a loop:

        avc: denied { read } for pid=929 comm=“ftdc” name=“netstat” dev=“proc” …

  • when i try to use the droblet console i got a lot of messages that contained … denied { open } from pid …

    i don’t know what does it mean that ?

Show answer This answer has been marked as resolved by zoompress2009.

Hello @Yannek,

when i tried to enable the SELinux=enforcing again i got the same problem and the droplet wont be accessible.

i don’t know why.

PS: i tried ti enabled it from command line.

Hi @zoompress2009,

You need to troubleshoot it. Change SELinux mode to permissive. In that mode security policies are not enforced. You will see just warns, and all the actions will be logged.
Here is the link to the manual describing SELinux concept and how to troubleshoot the problems (paragraph no. 5) in CentOS context.
https://wiki.centos.org/HowTos/SELinux

Submit an Answer