fsck - read only file system

December 5, 2013 3.2k views
For the second time in 3 months for vm went down - a fsck was necessary. No warning. Just down. Has this happened to anyone else? What do you do to mitigate the problem? Has support been helpful?
1 comment
  • Hi All,
    I faced the same issue in 3 months it get down 2nd time, 1 time i loss my data.
    i have open ticket but they are just replacing the boot option and nothing more that.
    my server down more than 36 hours i have migrate sites but mysql data still there.

    anyone can help me how to backup mysql data,
    we don't have access in terminal, only VNC is access but showing error: read only file system

9 Answers
Please open up a support ticket so we can investigate this. Thanks.
Thank Kamal, I've opened one. Support has been somewhat helpful - letting me know what to do to fix the problem - even though the fix thus far has not worked. What I've not gotten is how to keep it from happening - why it is happening - and how to keep it from happening. Not having that guidance/feedback - NONE - has lead me to believe I need to find another host.
"... has lead me to believe I need to find another host."

Why? It's not like you're paying for guidance/feedback. DigitalOcean advertises/sells "unmanaged" cloud servers. That's what you pay for; that's what you get.
I guess we have a different perspective.
It's unfair for me to ask how/why my server goes down because it is not managed? Yes, I'm afraid we definitely have a different perspective.
"It's unfair for me to ask how/why my server goes down because it is not managed?"

That's not what I was trying to say. Perhaps I focused too much on the literal meaning of "guidance/feedback."

Totally agree that questions limited to the availability of the unmanaged VPS you're paying for are fair game. I guess the scenario I had in my mind -- although, in retrospect, I realize that I have no clue whether this pertains to your current situation or not -- is that it's unrealistic to expect a provider of "unmanaged" cloud servers to fix problems on a VPS caused by user error; for a whopping $5/mo.

Case-in-point: Google reviews of GoDaddy's technical support (and GoDaddy charges waaaaay more than $5/mo. for a VPS!).
Here is a follow-up - it may be a little long - maybe it will help others make informed decisions regarding using DO or VMs in general.

So, ultimately, at the end of the day, my server became hosed and I was forced to provision another one to bring my app back up. It was down for 16*+ hours. We (the owner who pays me) got 80 bucks to compensate and some friendly guesses regarding what might of happened - 'probably because I had shut down improperly' - unlikely because I simply had not shutdown. More likely, after doing some research, the need for fsck was caused by faulty (or faltering) hardware - or faulty disk mounting (of SSDs). Would I bet my life on it? No. They did mention no others had experienced the same problem - which would blow that theory out of the water. Which is interesting since they have a kernel mode specifically for booting and running fsck.

In any case this was the second time in 3 months that my app was left down. I've been running this same app for *years* without issues both on hardware(a mac mini which had a hybrid persistance hd/ssd, which I do miss sometimes) and on other VMs.

I've moved my app server to Linode - leaving my asset host in place (which thank goodness also had backups from the server that was hosed). I should have done this before to mitigate risk anyway.

So my lesson, learned:
1) Make damn sure you are making backups. Do them yourself the old fashioned cron/rsync way. Use 2+ DCs. (yea, I had done this)
2) Have a fallback. Like I set I got set up in Linode - and kept my DO asset host/backup andI *might* come back someday and setup an app server again - and fail over to it if Linode craps out
4) Experiment with 'managed' plans. It *might* be worth it. It *might* have helped in this case
5) Reboot and run fsck periodically - investigate what it finds, examining specifically for possible faulty sectors/drives/mounts

My theory: SSDs are awesome. But may not be ideal for servers (that write often).

I won't be severing ties completely with DO, but I won't be recommending them either.

Hope that helps someone.

Been experiencing the same thing, only thing I can pin it down to is a lot of read/writes, exact same operations I carry out daily on multiple VPS machines with other companies and my own servers. This is the second time it's happened in a month so at least I have been backing up daily just in case. I've done a lot of sysadmin work and development and this isn't normal, there's nothing that should've caused the disk to remount itself as read only.
ray: have you opened up a support ticket?
Have another answer? Share your knowledge.