Question

centos 8 mysql keeps stoped status

mysql craahed tarting today, and it keeps down. what I get if I want to restar mysql service

systemctl restart mysqld
Job for mysqld.service failed because the control process exited with error code.
See "systemctl status mysqld.service" and "journalctl -xe" for details.

my error log

2020-10-19T09:10:14.068125Z 0 [System] [MY-010116] [Server] /usr/libexec/mysqld (mysqld 8.0.21) starting as process 320061
2020-10-19T09:10:14.088157Z 1 [System] [MY-013576] [InnoDB] InnoDB initialization has started.
2020-10-19T09:10:14.145172Z 0 [ERROR] [MY-012681] [InnoDB] mmap(137363456 bytes) failed; errno 12
2020-10-19T09:10:14.145376Z 1 [ERROR] [MY-012956] [InnoDB] Cannot allocate memory for the buffer pool
2020-10-19T09:10:14.145460Z 1 [ERROR] [MY-012930] [InnoDB] Plugin initialization aborted with error Generic error.
2020-10-19T09:10:14.145573Z 1 [ERROR] [MY-010334] [Server] Failed to initialize DD Storage Engine
2020-10-19T09:10:14.145831Z 0 [ERROR] [MY-010020] [Server] Data Dictionary initialization failed.
2020-10-19T09:10:14.145973Z 0 [ERROR] [MY-010119] [Server] Aborting
2020-10-19T09:10:14.146611Z 0 [System] [MY-010910] [Server] /usr/libexec/mysqld: Shutdown complete (mysqld 8.0.21)  Source distribution.

journalctl -xe

Oct 19 09:11:03 centos-s-1vcpu-1gb-sgp1-01 sshd[320118]: Invalid user 111111 from 185.153.196.230 port 32742
Oct 19 09:11:03 centos-s-1vcpu-1gb-sgp1-01 sshd[320118]: pam_unix(sshd:auth): check pass; user unknown
Oct 19 09:11:03 centos-s-1vcpu-1gb-sgp1-01 sshd[320118]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=185.153.196.230
Oct 19 09:11:05 centos-s-1vcpu-1gb-sgp1-01 sshd[320118]: Failed password for invalid user 111111 from 185.153.196.230 port 32742 ssh2
Oct 19 09:11:06 centos-s-1vcpu-1gb-sgp1-01 sshd[320118]: pam_unix(sshd:auth): check pass; user unknown
Oct 19 09:11:08 centos-s-1vcpu-1gb-sgp1-01 sshd[320118]: Failed password for invalid user 111111 from 185.153.196.230 port 32742 ssh2
Oct 19 09:11:09 centos-s-1vcpu-1gb-sgp1-01 sshd[320118]: Disconnecting invalid user 111111 185.153.196.230 port 32742: Change of username or service not allowed: (111111,ssh-connection) -> (admin,ssh-connectio>
Oct 19 09:11:09 centos-s-1vcpu-1gb-sgp1-01 sshd[320118]: PAM 1 more authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=185.153.196.230

from the error log looks like a memory issue so i run free -mh command

free -mh
              total        used        free      shared  buff/cache   available
Mem:          818Mi       296Mi       291Mi       101Mi       230Mi       297Mi
Swap:            0B          0B          0B

I used of 296m but i cache 230m i should supposed to have decent amount of memory I can used am I right ? https://www.digitalocean.com/community/questions/out-of-memory-is-it-possible

from the answer of this post , this is the what he said

To expand on the “used” RAM above, simply because I’ve “used” 1.9GB out of the 2GB I’ve been allocated does not mean that I’ve used that much. The 1.6GB’s from the cache column is included in that 1.9GB’s, so my actual usage is only ~300MB. The other 1.3GB is cached and used by current processes until they are either stopped, killed or restarted

but once i take a close look at journalctl -xe log , there is a user 111111 trying to login with wrong password ??

Failed password for invalid user 111111 from 185.153.196.230 port 32742 ssh2

I dont know if this is actually a memory problem or another issues ?? hopefully someone have infra knowledge can advise me on this .thanks now all of my site is down , I am using the cheapest droplet instance. which have 1G memory. btw , I am using centos8 nginx, php , mysql stack.thanks again.

Subscribe
Share

Submit an answer
You can type!ref in this text area to quickly search our full set of tutorials, documentation & marketplace offerings and insert the link!

These answers are provided by our Community. If you find them useful, show some love by clicking the heart. If you run into issues leave a comment, or add your own answer to help others.

Hello there,

In addition to what has already been mentioned, I wanted to point out that CentOS Linux 8 has reached End Of Life (EOL) on December 31st, 2021.

I would strongly recommend planning either migration to another OS like Debian or Ubuntu, or upgrading to CentOS Stream.

I personally believe that the safest way to do this is to follow these steps here:

  • Spin up a new Droplet with the desired OS
  • Install the necessary services (webserver, sql server, etc.)
  • After that using rsync for example copy over your files and migrate your databases
  • Then test your website and server fully to make sure that everything is working as expected
  • Finally, change your DNS to point to the new Droplet

Regards, Alex

Hello,

The key error here is:

InnoDB: Fatal error: cannot allocate memory for the buffer pool

This crash is due to your system running out of memory. I’d suggest that you add a swap file to give yourself a bit more of a buffer. Check out this tutorial:

https://www.digitalocean.com/community/tutorials/how-to-add-swap-space-on-ubuntu-20-04

What you can also do is to use the MySQLTuner script.

The MySQLTuner is a script written in Perl and allows you to quickly test your MySQL configuration and it gives you suggestions for adjustments to increase performance and stability.

According to the official GitHub page, it supports 300 indicators for MySQL/MariaDB/Percona Server in this last version.

To run the script you could do the following:

  • SSH to your Droplet
  • Download the script:
wget http://mysqltuner.pl/ -O mysqltuner.pl
  • Then execute it:
perl mysqltuner.pl

The script would run multiple checks against your MySQL instance, all checks done by MySQLTuner are documented here.

Also as stated in the official documentation, it is still extremely important for you to fully understand each change you make to a MySQL database server. If you don’t understand portions of the script’s output, or if you don’t understand the recommendations, you should consult a knowledgeable DBA or system administrator that you trust.

As a good practice make sure to always test your changes on staging environments before implementing them on your production database.

On the same note, if you want to have a worry-free MySQL hosting and focus on your application, I would recommend trying out the DigitalOcean Managed Databases:

https://www.digitalocean.com/products/managed-databases-mysql/

This was mini tutorial was posted from bobbyiliev in this question in our community: https://www.digitalocean.com/community/questions/how-to-tweak-mysql-mariadb-configuration-for-increased-performance-and-stability

You can also create a simple bash script to check if MySQL is running and if not to restart it.

#!/bin/bash

# Check if MySQL is running
sudo service mysql status > /dev/null 2>&1

# Restart the MySQL service if it's not running.
if [ $? != 0 ]; then
    sudo service mysql restart
fi

Run this script every 5 minutes using a cron job like this one:

 */5 * * * * /home/user/scripts/monitor.sh > /dev/null 2>&1

Hope that this helps! Regards, Alex

The answer depends on the question System OS (CentOS) and MySQL Mariadb service

Solution: Just add some seconds before make service start - in my case I’ve added 45 seconds

Add the following lines under [service] section in {/etc/systemd/system/multi-user.target.wants/mariadb.service} - Of course the path depends on your system OS and MySQL service name (not all have mariadb.service)

Restart=always RestartSec=45s Do NOT forget to run the following commands

sudo systemctl daemon-reload

sudo systemctl restart mariadb.service

Hi @stupidboy,

It’s possible you have some misconfiguration in your configuration file - /etc/my.cnf.

Can you please post try with a default configuration file to start MySQL to see if the same errors will be presented?

Another solution would be to try and configure a SWAP file. I know you have enough memory but more often than not it will help out in such situations.