MySQL stopped working after system hard reset

Question

I have a droplet with Ubuntu 22.04 LTS installed. I followed some tutorials and installed Apache, PHP, MySQL and Wordpress. Everything was working flawlessly. I set up a few websites in this droplet with Wordpress, everything ok. But then, I had to perform a hard reset (via Digitalocean panel, forcing the droplet to shutdown manually and then booting it up manually again). After that, my Wordpress sites were broken with the message in the browser: “Error establishing a database connection”. It seems MySQL is broken. I have a simple droplet with 1GB ram and 2GB swap file. Here are a few console outputs that might be of interest:

# apt update
Hit:1 http://security.ubuntu.com/ubuntu jammy-security InRelease
Hit:2 http://mirrors.digitalocean.com/ubuntu jammy InRelease                          
Get:3 http://mirrors.digitalocean.com/ubuntu jammy-updates InRelease [119 kB]         
Hit:4 http://mirrors.digitalocean.com/ubuntu jammy-backports InRelease                           
Hit:5 https://repos-droplet.digitalocean.com/apt/droplet-agent main InRelease         
Fetched 119 kB in 4s (32.7 kB/s)
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
All packages are up to date.
root@ubuntu-22:~# apt upgrade
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
Calculating upgrade... Done
The following packages were automatically installed and are no longer required:
  libflashrom1 libftdi1-2
Use 'apt autoremove' to remove them.
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
2 not fully installed or removed.
After this operation, 0 B of additional disk space will be used.
Do you want to continue? [Y/n] 
Setting up mysql-server-8.0 (8.0.36-0ubuntu0.22.04.1) ...
mysqld will log errors to /var/log/mysql/error.log
2024-02-06T12:10:15.633786Z 0 [ERROR] [MY-010946] [Server] Failed to start mysqld daemon. Check mysqld error log.
Warning: Unable to start the server.
Job for mysql.service failed because the control process exited with error code.
See "systemctl status mysql.service" and "journalctl -xeu mysql.service" for details.
invoke-rc.d: initscript mysql, action "restart" failed.
● mysql.service - MySQL Community Server
     Loaded: loaded (/lib/systemd/system/mysql.service; enabled; vendor preset: enabled)
     Active: activating (auto-restart) (Result: exit-code) since Tue 2024-02-06 12:10:29 UTC; 14ms ago
    Process: 1220056 ExecStartPre=/usr/share/mysql/mysql-systemd-start pre (code=exited, status=0/SUCCESS)
    Process: 1220064 ExecStart=/usr/sbin/mysqld (code=exited, status=1/FAILURE)
   Main PID: 1220064 (code=exited, status=1/FAILURE)
     Status: "Server shutdown complete"
      Error: 22 (Invalid argument)
        CPU: 2.664s

Feb 06 12:10:29 ubuntu-22 systemd[1]: mysql.service: Failed with result 'exit-code'.
Feb 06 12:10:29 ubuntu-22 systemd[1]: Failed to start MySQL Community Server.
Feb 06 12:10:29 ubuntu-22 systemd[1]: mysql.service: Consumed 2.664s CPU time.
Feb 06 12:10:29 ubuntu-22 systemd[1]: mysql.service: Scheduled restart job, restart counter is at 1.
Feb 06 12:10:29 ubuntu-22 systemd[1]: Stopped MySQL Community Server.
Feb 06 12:10:29 ubuntu-22 systemd[1]: mysql.service: Consumed 2.664s CPU time.
Feb 06 12:10:29 ubuntu-22 systemd[1]: Starting MySQL Community Server...
dpkg: error processing package mysql-server-8.0 (--configure):
 installed mysql-server-8.0 package post-installation script subprocess returned error exit status 1
dpkg: dependency problems prevent configuration of mysql-server:
 mysql-server depends on mysql-server-8.0; however:
  Package mysql-server-8.0 is not configured yet.

dpkg: error processing package mysql-server (--configure):
 dependency problems - leaving unconfigured
No apport report written because the error message indicates its a followup error from a previous failure.
                                                                                                          Errors were encountered while processing:
 mysql-server-8.0
 mysql-server
needrestart is being skipped since dpkg has failed
E: Sub-process /usr/bin/dpkg returned an error code (1)

# systemctl restart mysql.service
Job for mysql.service failed because the control process exited with error code.
See "systemctl status mysql.service" and "journalctl -xeu mysql.service" for details.

# journalctl -xeu mysql.service
Feb 06 12:17:17 ubuntu-22 systemd[1]: mysql.service: Main process exited, code=exited, status=1/FAILURE
░░ Subject: Unit process exited
░░ Defined-By: systemd
░░ Support: http://www.ubuntu.com/support
░░ 
░░ An ExecStart= process belonging to unit mysql.service has exited.
░░ 
░░ The process' exit code is 'exited' and its exit status is 1.
Feb 06 12:17:17 ubuntu-22 systemd[1]: mysql.service: Failed with result 'exit-code'.
░░ Subject: Unit failed
░░ Defined-By: systemd
░░ Support: http://www.ubuntu.com/support
░░ 
░░ The unit mysql.service has entered the 'failed' state with result 'exit-code'.
Feb 06 12:17:17 ubuntu-22 systemd[1]: Failed to start MySQL Community Server.
░░ Subject: A start job for unit mysql.service has failed
░░ Defined-By: systemd
░░ Support: http://www.ubuntu.com/support
░░ 
░░ A start job for unit mysql.service has finished with a failure.
░░ 
░░ The job identifier is 56114662 and the job result is failed.
Feb 06 12:17:17 ubuntu-22 systemd[1]: mysql.service: Consumed 2.679s CPU time.
░░ Subject: Resources consumed by unit runtime
░░ Defined-By: systemd
░░ Support: http://www.ubuntu.com/support
░░ 
░░ The unit mysql.service completed and consumed the indicated resources.
Feb 06 12:17:17 ubuntu-22 systemd[1]: mysql.service: Scheduled restart job, restart counter is at 8.
░░ Subject: Automatic restarting of a unit has been scheduled
░░ Defined-By: systemd
░░ Support: http://www.ubuntu.com/support
░░ 
░░ Automatic restarting of the unit mysql.service has been scheduled, as the result for
░░ the configured Restart= setting for the unit.
Feb 06 12:17:17 ubuntu-22 systemd[1]: Stopped MySQL Community Server.
░░ Subject: A stop job for unit mysql.service has finished
░░ Defined-By: systemd
░░ Support: http://www.ubuntu.com/support
░░ 
░░ A stop job for unit mysql.service has finished.
░░ 
░░ The job identifier is 56114746 and the job result is done.
Feb 06 12:17:17 ubuntu-22 systemd[1]: mysql.service: Consumed 2.679s CPU time.
░░ Subject: Resources consumed by unit runtime
░░ Defined-By: systemd
░░ Support: http://www.ubuntu.com/support
░░ 
░░ The unit mysql.service completed and consumed the indicated resources.
Feb 06 12:17:17 ubuntu-22 systemd[1]: Starting MySQL Community Server...
░░ Subject: A start job for unit mysql.service has begun execution
░░ Defined-By: systemd
░░ Support: http://www.ubuntu.com/support
░░ 
░░ A start job for unit mysql.service has begun execution.
░░ 
░░ The job identifier is 56114746.

Could someone please help me? I need to fix this to get some important pages I created on Wordpress, it’s not an option to delete this droplet and create another one.

Bobby · Answer

Hi there,

The error messages from the system logs and systemctl status indicate that there’s an issue with MySQL starting up, but the exact cause isn’t clear from the provided output. However, here are a few more things that you could check to get to the actual error that is causing the problem:

1. Check MySQL Error Logs

The first step should be to check the MySQL error log for more detailed information about why the service is failing to start. The error log is mentioned in your output as /var/log/mysql/error.log. This file should contain specific error messages that can guide us to the root cause.

sudo cat /var/log/mysql/error.log

2. Ensure Enough Disk Space and Memory

Given that your droplet has 1GB of RAM and a 2GB swap file, it’s essential to ensure that you have enough free disk space and memory. MySQL can fail to start if the system doesn’t have enough resources.

Check disk space using df -h.
Check memory usage with free -m.

3. Check MySQL Configuration

If there were recent changes to MySQL’s configuration files (e.g., /etc/mysql/my.cnf or any other file under /etc/mysql/mysql.conf.d/), those changes might be causing issues. Review any recent changes and ensure there are no syntax errors or incompatible settings.

Feel free to share the errors here so I can try to advise you further!

Best,

Bobby

KFSys · Answer

Heya @renatov , Looking at the provided information, I can pinpoint the following as the error: dpkg: error processing package mysql-server-8.0 (--configure): installed mysql-server-8.0 package post-installation script subprocess returned error exit status 1 dpkg: dependency problems prevent configuration of mysql-server: mysql-server depends on mysql-server-8.0; however: Package mysql-server-8.0 is not configured yet. Have you tried to install any MySQL after the issue or before the issue happened? This error usually appears when you have two MySQL versions for instance.

alexdo · Answer

I will definitely recommend you to add a swap file if you haven’t done this yet and also use the MySQL tuner script and see if the MySQL configuration needs an improvement.

You can also create a simple bash script to check if MySQL is running and if not to restart it.

#!/bin/bash

# Check if MySQL is running
sudo service mysql status > /dev/null 2>&1

# Restart the MySQL service if it's not running.
if [ $? != 0 ]; then
    sudo service mysql restart
fi

Run this script every 5 minutes using a cron job like this one:

 */5 * * * * /home/user/scripts/monitor.sh > /dev/null 2>&1

Hope that this helps!

Report this

MySQL stopped working after system hard reset

1. Check MySQL Error Logs

2. Ensure Enough Disk Space and Memory

3. Check MySQL Configuration

Become a contributor for community

DigitalOcean Documentation

Resources for startups and AI-native businesses

The developer cloud

Start building today

1. Check MySQL Error Logs

2. Ensure Enough Disk Space and Memory

3. Check MySQL Configuration