Job for mysqld.service failed See “systemctl status mysqld.service , journalctl -xe & tail -30 /var/log/mysql/error.log "

August 23, 2017 41.9k views
MySQL WordPress Ubuntu 16.04
mformainuddin
By:
mformainuddin

I don't know what i have to do now seeing this error(mysql.service won't start) bcz i'm that much linux guy.....
Pls give some solution asap

systemctl status mysqld.service

● mysql.service - MySQL Community Server
   Loaded: loaded (/lib/systemd/system/mysql.service; enabled; vendor preset: enabled)
   Active: activating (start-post) (Result: exit-code) since Wed 2017-08-23 19:17:34 UTC; 24s ago
  Process: 26606 ExecStart=/usr/sbin/mysqld (code=exited, status=1/FAILURE)
  Process: 26597 ExecStartPre=/usr/share/mysql/mysql-systemd-start pre (code=exited, status=0/SUCCESS)
 Main PID: 26606 (code=exited, status=1/FAILURE);         : 26607 (mysql-systemd-s)
    Tasks: 2
   Memory: 324.0K
      CPU: 520ms
   CGroup: /system.slice/mysql.service
           └─control
             ├─26607 /bin/bash /usr/share/mysql/mysql-systemd-start post
             └─26676 sleep 1

Aug 23 19:17:34 ubuntu-1gb-nyc3-01 systemd[1]: Starting MySQL Community Server...
Aug 23 19:17:35 ubuntu-1gb-nyc3-01 systemd[1]: mysql.service: Main process exited, code=exited, status=1/FAILURE

journalctl -xe

-- Unit mysql.service has begun starting up.
Aug 23 19:19:05 ubuntu-1gb-nyc3-01 audit[26888]: AVC apparmor="DENIED" operation="open" profile="/usr/sbin/mysqld" name="/proc/26888/status" pid=2688
Aug 23 19:19:05 ubuntu-1gb-nyc3-01 audit[26888]: AVC apparmor="DENIED" operation="open" profile="/usr/sbin/mysqld" name="/sys/devices/system/node/" p
Aug 23 19:19:05 ubuntu-1gb-nyc3-01 audit[26888]: AVC apparmor="DENIED" operation="open" profile="/usr/sbin/mysqld" name="/proc/26888/status" pid=2688
Aug 23 19:19:05 ubuntu-1gb-nyc3-01 kernel: audit: type=1400 audit(1503515945.954:748): apparmor="DENIED" operation="open" profile="/usr/sbin/mysqld"
Aug 23 19:19:05 ubuntu-1gb-nyc3-01 kernel: audit: type=1400 audit(1503515945.954:749): apparmor="DENIED" operation="open" profile="/usr/sbin/mysqld"
Aug 23 19:19:05 ubuntu-1gb-nyc3-01 kernel: audit: type=1400 audit(1503515945.954:750): apparmor="DENIED" operation="open" profile="/usr/sbin/mysqld"
Aug 23 19:19:06 ubuntu-1gb-nyc3-01 systemd[1]: mysql.service: Main process exited, code=exited, status=1/FAILURE
Aug 23 19:19:23 ubuntu-1gb-nyc3-01 sshd[26930]: Connection closed by 116.31.116.40 port 40327 [preauth]
Aug 23 19:19:36 ubuntu-1gb-nyc3-01 systemd[1]: Failed to start MySQL Community Server.
-- Subject: Unit mysql.service has failed
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- Unit mysql.service has failed.
--
-- The result is failed.
Aug 23 19:19:36 ubuntu-1gb-nyc3-01 systemd[1]: mysql.service: Unit entered failed state.
Aug 23 19:19:36 ubuntu-1gb-nyc3-01 systemd[1]: mysql.service: Failed with result 'exit-code'.
Aug 23 19:19:36 ubuntu-1gb-nyc3-01 systemd[1]: mysql.service: Service hold-off time over, scheduling restart.
Aug 23 19:19:36 ubuntu-1gb-nyc3-01 systemd[1]: Stopped MySQL Community Server.
-- Subject: Unit mysql.service has finished shutting down
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- Unit mysql.service has finished shutting down.
Aug 23 19:19:36 ubuntu-1gb-nyc3-01 systemd[1]: Starting MySQL Community Server...
-- Subject: Unit mysql.service has begun start-up
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- Unit mysql.service has begun starting up.
Aug 23 19:19:36 ubuntu-1gb-nyc3-01 audit[26981]: AVC apparmor="DENIED" operation="open" profile="/usr/sbin/mysqld" name="/proc/26981/status" pid=2698
Aug 23 19:19:36 ubuntu-1gb-nyc3-01 audit[26981]: AVC apparmor="DENIED" operation="open" profile="/usr/sbin/mysqld" name="/sys/devices/system/node/" p
Aug 23 19:19:36 ubuntu-1gb-nyc3-01 audit[26981]: AVC apparmor="DENIED" operation="open" profile="/usr/sbin/mysqld" name="/proc/26981/status" pid=2698
Aug 23 19:19:36 ubuntu-1gb-nyc3-01 kernel: audit: type=1400 audit(1503515976.446:751): apparmor="DENIED" operation="open" profile="/usr/sbin/mysqld"
Aug 23 19:19:36 ubuntu-1gb-nyc3-01 kernel: audit: type=1400 audit(1503515976.446:752): apparmor="DENIED" operation="open" profile="/usr/sbin/mysqld"
Aug 23 19:19:36 ubuntu-1gb-nyc3-01 kernel: audit: type=1400 audit(1503515976.446:753): apparmor="DENIED" operation="open" profile="/usr/sbin/mysqld"
Aug 23 19:19:37 ubuntu-1gb-nyc3-01 systemd[1]: mysql.service: Main process exited, code=exited, status=1/FAILURE
Aug 23 19:19:48 ubuntu-1gb-nyc3-01 sudo[27025]:     root : TTY=pts/0 ; PWD=/root ; USER=root ; COMMAND=/bin/journalctl -xe
Aug 23 19:19:48 ubuntu-1gb-nyc3-01 sudo[27025]: pam_unix(sudo:session): session opened for user root by root(uid=0)

tail -30 /var/log/mysql/error.log


2017-08-23T19:22:08.951104Z 0 [Warning] Changed limits: max_open_files: 1024 (requested 5000)
2017-08-23T19:22:08.951207Z 0 [Warning] Changed limits: table_open_cache: 431 (requested 2000)
2017-08-23T19:22:09.177161Z 0 [Warning] TIMESTAMP with implicit DEFAULT value is deprecated. Please use --explicit_defaults_for_timestamp server option (see documentation for more details).
2017-08-23T19:22:09.181318Z 0 [Note] /usr/sbin/mysqld (mysqld 5.7.19-0ubuntu0.16.04.1) starting as process 27514 ...
2017-08-23T19:22:09.190896Z 0 [Note] InnoDB: PUNCH HOLE support available
2017-08-23T19:22:09.190945Z 0 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
2017-08-23T19:22:09.190960Z 0 [Note] InnoDB: Uses event mutexes
2017-08-23T19:22:09.190971Z 0 [Note] InnoDB: GCC builtin __atomic_thread_fence() is used for memory barrier
2017-08-23T19:22:09.190980Z 0 [Note] InnoDB: Compressed tables use zlib 1.2.8
2017-08-23T19:22:09.190990Z 0 [Note] InnoDB: Using Linux native AIO
2017-08-23T19:22:09.191768Z 0 [Note] InnoDB: Number of pools: 1
2017-08-23T19:22:09.192044Z 0 [Note] InnoDB: Using CPU crc32 instructions
2017-08-23T19:22:09.195523Z 0 [Note] InnoDB: Initializing buffer pool, total size = 128M, instances = 1, chunk size = 128M
2017-08-23T19:22:09.213977Z 0 [Note] InnoDB: Completed initialization of buffer pool
2017-08-23T19:22:09.219177Z 0 [Note] InnoDB: If the mysqld execution user is authorized, page cleaner thread priority can be changed. See the man page of setpriority().
2017-08-23T19:22:09.233451Z 0 [Note] InnoDB: Highest supported file format is Barracuda.
2017-08-23T19:22:09.252993Z 0 [Note] InnoDB: Creating shared tablespace for temporary tables
2017-08-23T19:22:09.253125Z 0 [Note] InnoDB: Setting file './ibtmp1' size to 12 MB. Physically writing the file full; Please wait ...
2017-08-23T19:22:09.264465Z 0 [Warning] InnoDB: 1048576 bytes should have been written. Only 851968 bytes written. Retrying for the remaining bytes.
2017-08-23T19:22:09.264519Z 0 [Warning] InnoDB: Retry attempts for writing partial data failed.
2017-08-23T19:22:09.264539Z 0 [ERROR] InnoDB: Write to file ./ibtmp1failed at offset 5242880, 1048576 bytes should have been written, only 851968 were written. Operating system error number 28. Check that your OS and file s2017-08-23T19:22:09.866980Z 0 [Note] InnoDB: Removed temporary tablespace data file: "ibtmp1"
2017-08-23T19:22:09.867026Z 0 [ERROR] Plugin 'InnoDB' init function returned error.
2017-08-23T19:22:09.867033Z 0 [ERROR] Plugin 'InnoDB' registration as a STORAGE ENGINE failed.
2017-08-23T19:22:09.867040Z 0 [ERROR] Failed to initialize plugins.
2017-08-23T19:22:09.867043Z 0 [ERROR] Aborting

2017-08-23T19:22:09.867049Z 0 [Note] Binlog end
2017-08-23T19:22:09.867098Z 0 [Note] Shutting down plugin 'MyISAM'
2017-08-23T19:22:09.867485Z 0 [Note] /usr/sbin/mysqld: Shutdown complete

and i'm using ubuntu 16.04(1gb) Droplet

Thanks in advance

19 comments
  • iteagle03 August 23, 2017

    Error was here (in bold) in the sysout:
    2017-08-23T19:22:09.253125Z 0 [Note] InnoDB: Setting file './ibtmp1' size to 12 MB. Physically writing the file full; Please wait ...
    2017-08-23T19:22:09.264465Z 0 [Warning] InnoDB: 1048576 bytes should have been written. Only 851968 bytes written. Retrying for the remaining bytes.
    2017-08-23T19:22:09.264519Z 0 [Warning] InnoDB: Retry attempts for writing partial data failed.
    2017-08-23T19:22:09.264539Z 0 [ERROR] InnoDB: Write to file ./ibtmp1failed at offset 5242880, 1048576 bytes should have been written, only 851968 were written. Operating system error number 28. Check that your OS and file s2017-08-23T19:22:09.866980Z 0 [Note] InnoDB: Removed temporary tablespace data file: "ibtmp1"

    How much free space do you have on your droplet?

  • mformainuddin August 23, 2017

    @iteagle03

    Here it is:

    Filesystem      Size  Used Avail Use% Mounted on
udev            992M     0  992M   0% /dev
tmpfs           201M   11M  190M   6% /run
/dev/vda1        30G   29G  4.8M 100% /
tmpfs          1001M     0 1001M   0% /dev/shm
tmpfs           5.0M     0  5.0M   0% /run/lock
tmpfs          1001M     0 1001M   0% /sys/fs/cgroup
tmpfs           201M     0  201M   0% /run/user/0
  • iteagle03 August 23, 2017 
    Filesystem      Size  Used Avail Use% Mounted on
udev            992M     0  992M   0% /dev
tmpfs           201M   11M  190M   6% /run
**/dev/vda1        30G   29G  4.8M 100% /**
tmpfs          1001M     0 1001M   0% /dev/shm
tmpfs           5.0M     0  5.0M   0% /run/lock
tmpfs          1001M     0 1001M   0% /sys/fs/cgroup
tmpfs           201M     0  201M   0% /run/user/0

    You appear to be out of space. From '/' if you run this command it will tell you your top 10 directories in terms of disk space used.

    du -aBM 2>/dev/null | sort -nr | head -n 10

    You can change the '10' into any value you want ... 20, 30, etc.

    There is another more "pretty" tool I found but you may not have enough space to install the utility.

  • mformainuddin August 23, 2017

    @iteagle03

    ok i typed:

    du -aBM 2>/dev/null | sort -nr | head -n 10

    and it shows: 

    root@ubuntu-1gb-nyc3-01:~# du -aBM 2>/dev/null | sort -nr | head -n 30
5M      .
4M      ./images
3M      ./images/itv
2M      ./images/itv.zip
1M      ./.xsession-errors
1M      ./.Xauthority
1M      ./.vnc/xstartup
1M      ./.vnc/ubuntu-1gb-nyc3-01:1.log
1M      ./.vnc/passwd
1M      ./.vnc
1M      ./.viminfo
1M      ./Videos
1M      ./Templates
1M      ./.ssh
1M      ./Public
1M      ./.profile
1M      ./player/yendifplayer.js
1M      ./player/yendifplayer.css
1M      ./player/subtitles.vtt
1M      ./player/player.swf
1M      ./player/LICENSE.md
1M      ./player/index.html
1M      ./player/images/play.png
1M      ./player/images/icons_white.png
1M      ./player/images/icons_black.png
1M      ./player/images
1M      ./player/embed.js
1M      ./player
1M      ./Pictures
1M      ./.nano/search_history

    Then what i need to do???

  • iteagle03 August 23, 2017

    And you ran this command from the root of the directory ( '/' )? If you run 'pwd' and get something other than '/' you are not at the root directory. Once you are at root ('cd /') and run that du command, you should get something like this:

    1936M   .
1209M   ./usr
361M    ./usr/src
327M    ./usr/lib
264M    ./usr/share
246M    ./var
222M    ./usr/bin
212M    ./lib
173M    ./var/lib
159M    ./lib/modules

    Most of my disk usage is in the '/usr' directory, which is fine. I'm only using about 2GB out of 30GB for my droplet. :)

  • mformainuddin August 23, 2017

    @iteagle03
    It's showing that:

    root@ubuntu-1gb-nyc3-01:~# pwd
/root
root@ubuntu-1gb-nyc3-01:~# /
-bash: /: Is a directory
root@ubuntu-1gb-nyc3-01:~# cd /
root@ubuntu-1gb-nyc3-01:/# du -aBM 2>/dev/null | sort -nr | head -n 10
sort: write failed: /tmp/sort5Rl6rV: No space left on device
  • iteagle03 August 23, 2017

    Ouch. You have so little space you have literally no wiggle room to work with.

    Well, let's wander into the /var/log directory and run the command 'ls -halt' and see if we have any large log files we could possibly clear out.

    What are you using this droplet for that has used up so much disk space?

  • mformainuddin August 23, 2017

    @iteagle03

    it's showing that:

    root@ubuntu-1gb-nyc3-01:/var/log# ls -halt
total 204M
-rw-r-----  1 syslog adm    1003K Aug 23 20:51 syslog
-rw-r-----  1 syslog adm     497K Aug 23 20:51 kern.log
-rw-r-----  1 syslog adm     359K Aug 23 20:51 auth.log
-rw-------  1 root   utmp     73M Aug 23 20:49 btmp
-rw-rw-r--  1 root   utmp     34K Aug 23 20:25 lastlog
-rw-rw-r--  1 root   utmp     28K Aug 23 20:25 wtmp
-rw-------  1 root   root    1.2K Aug 23 20:24 php7.0-fpm.log
-rw-r--r--  1 root   root    715K Aug 23 20:24 cloud-init-output.log
-rw-r--r--  1 syslog adm      22M Aug 23 20:24 cloud-init.log
-rw-r-----  1 syslog adm        0 Aug 23 06:25 syslog.1.gz
drwxrwxr-x 11 root   syslog  4.0K Aug 23 06:25 .
drwxr-x---  2 mysql  adm     4.0K Aug 23 06:25 mysql
drwxr-x---  2 root   adm     4.0K Aug 23 06:25 apache2
-rw-r-----  1 syslog adm     8.0K Aug 22 06:25 syslog.1
-rw-r-----  1 syslog adm      14M Aug 21 06:25 auth.log.1
-rw-r-----  1 syslog adm     764K Aug 21 06:25 kern.log.1
-rw-r-----  1 syslog adm     8.9K Aug 21 06:25 syslog.2.gz
-rw-r-----  1 syslog adm        0 Aug 20 06:25 syslog.1.gz-2017082106.backup
drwxr-xr-x  2 root   adm     4.0K Aug 19 06:25 nginx
-rw-r-----  1 syslog adm      69K Aug 19 06:25 syslog.3.gz
-rw-------  1 root   root    1.6K Aug 18 19:40 php7.0-fpm.log.1
-rw-r-----  1 syslog adm        0 Aug 18 06:25 syslog.1.gz-2017081906.backup
-rw-r-----  1 syslog adm      24K Aug 17 06:25 syslog.4.gz
-rw-r-----  1 syslog adm     2.3K Aug 16 06:25 syslog.5.gz
-rw-r--r--  1 root   root     59K Aug 16 04:53 dpkg.log
-rw-r-----  1 syslog adm     2.4K Aug 15 06:25 syslog.6.gz
-rw-r-----  1 syslog adm      50K Aug 14 06:25 syslog.7.gz
-rw-r-----  1 syslog adm     1.2M Aug 13 06:25 auth.log.2.gz
-rw-r--r--  1 root   root       0 Aug 12 06:25 alternatives.log
-rw-r-----  1 syslog adm      154 Aug 11 14:46 kern.log.2.gz
-rw-------  1 root   root     275 Aug 11 10:16 php7.0-fpm.log.2.gz
-rw-r--r--  1 root   root    1.5K Aug 11 10:16 alternatives.log.1
-rw-r-----  1 syslog adm     2.0M Aug  7 06:25 auth.log.3.gz
-rw-r-----  1 root   adm        0 Aug  3 06:25 apport.log
-rw-r-----  1 root   adm      510 Aug  2 15:42 apport.log.1
-rw-r-----  1 syslog adm      11K Aug  2 15:42 kern.log.3.gz
drwxr-x---  2 root   adm     4.0K Aug  1 06:25 unattended-upgrades
drwxr-xr-x  2 root   root    4.0K Aug  1 06:25 apt
-rw-------  1 root   utmp     89M Aug  1 06:25 btmp.1
-rw-r-----  1 syslog adm     1.5M Jul 30 06:25 auth.log.4.gz
-rw-------  1 root   root      76 Jul 30 06:25 php7.0-fpm.log.3.gz
-rw-r-----  1 syslog adm      33K Jul 29 15:49 kern.log.4.gz
-rw-r-----  1 root   adm      321 Jul 29 15:49 apport.log.2.gz
-rw-r--r--  1 root   root     87K Jul 28 08:35 dpkg.log.1
-rw-r-----  1 root   adm      282 Jul 26 08:40 apport.log.3.gz
-rw-r-----  1 root   adm      289 Jul 25 11:45 apport.log.4.gz
-rw-------  1 root   root      76 Jul 23 06:25 php7.0-fpm.log.4.gz
-rw-r--r--  1 root   root     127 Jul 21 05:20 alternatives.log.2.gz
-rw-r-----  1 root   adm      835 Jul 18 03:18 apport.log.5.gz
-rw-------  1 root   root      76 Jul 17 06:25 php7.0-fpm.log.5.gz
-rw-rw-r--  1 root   utmp     14K Jul 13 04:58 wtmp.1
-rw-------  1 root   root     283 Jul 13 04:58 php7.0-fpm.log.6.gz
-rw-r-----  1 root   adm     6.9K Jul  9 18:46 apport.log.6.gz
-rw-------  1 root   root     275 Jul  4 18:19 php7.0-fpm.log.7.gz
-rw-------  1 root   root      76 Jun 25 06:25 php7.0-fpm.log.8.gz
-rw-r--r--  1 root   root    2.1K Jun 20 03:00 dpkg.log.2.gz
-rw-------  1 root   root      76 Jun 19 06:25 php7.0-fpm.log.9.gz
-rw-r-----  1 root   adm     8.4K Jun 18 15:35 apport.log.7.gz
-rw-------  1 root   root     278 Jun 15 09:56 php7.0-fpm.log.10.gz
-rw-------  1 root   root     311 Jun  9 17:44 php7.0-fpm.log.11.gz
-rw-------  1 root   root    1022 Jun  1 14:01 php7.0-fpm.log.12.gz
-rw-r--r--  1 root   root     25K May 31 01:41 dpkg.log.3.gz
-rw-r--r--  1 root   root    1.2K May 20 06:58 alternatives.log.3.gz
-rw-r--r--  1 root   root    1.1K May 17 14:51 fontconfig.log
-rw-r--r--  1 root   root    2.4K Apr 28 05:05 dpkg.log.4.gz
-rw-r--r--  1 root   root     126 Apr 28 05:05 alternatives.log.4.gz
-rw-r--r--  1 root   root     12K Mar 30 07:39 dpkg.log.5.gz
drwxr-xr-x  2 root   root    4.0K Mar 21 10:22 dbconfig-common
-rw-r--r--  1 root   root     810 Mar 21 09:21 alternatives.log.5.gz
drwxr-xr-x 14 root   root    4.0K Mar 21 08:37 ..
drwxr-xr-x  2 root   root    4.0K Mar  8 02:17 fsck
drwxr-xr-x  2 root   root    4.0K Feb 23 18:05 lxd
drwxr-xr-x  2 root   root    4.0K Jan  5  2017 dist-upgrade

    I installed a wordpress site there and it's using for technology blog

  • iteagle03 August 23, 2017

    I'd delete any log file that is a .gz or .1 file that is larger than 5 MB.

    auth.log.1 & btmp.1 are both pretty big.

    I think you need to determine whether you need to keep around any log files older than two months as well (that is a .gz file).

    Then go into either apache2 or nginx (whichever is your webserver) and lets see how big those log files are and see what we can do about them.

  • mformainuddin August 23, 2017

    am i need to delete auth.log.1 & btmp.1 or not???

    apache2:

    
root@ubuntu-1gb-nyc3-01:/var/log/apache2# ls -halt
total 310M
-rw-r-----  1 root adm     83M Aug 23 21:25 access.log
-rw-r-----  1 root adm     25M Aug 23 21:23 error.log
drwxrwxr-x 11 root syslog 4.0K Aug 23 06:25 ..
-rw-r-----  1 root adm       0 Aug 23 06:25 error.log.1.gz
drwxr-x---  2 root adm    4.0K Aug 23 06:25 .
-rw-r-----  1 root adm    276K Aug 23 06:25 access.log.1.gz
-rw-r-----  1 root adm       0 Aug 22 06:25 error.log.1.gz-2017082306.backup
-rw-r-----  1 root adm     16K Aug 22 06:25 access.log.1.gz-2017082306.backup
-rw-r-----  1 root adm       0 Aug 21 06:25 error.log.1.gz-2017082206.backup
-rw-r-----  1 root adm       0 Aug 21 06:25 access.log.1.gz-2017082206.backup
-rw-r-----  1 root adm       0 Aug 20 06:25 error.log.1.gz-2017082106.backup
-rw-r-----  1 root adm    268K Aug 20 06:25 access.log.1.gz-2017082106.backup
-rw-r-----  1 root adm       0 Aug 19 06:25 error.log.1.gz-2017082006.backup
-rw-r-----  1 root adm    276K Aug 19 06:25 access.log.1.gz-2017082006.backup
-rw-r-----  1 root adm       0 Aug 18 06:25 error.log.1.gz-2017081906.backup
-rw-r-----  1 root adm    1.3M Aug 18 06:25 access.log.1.gz-2017081906.backup
-rw-r-----  1 root adm     85M Aug 17 06:30 access.log.1
-rw-r-----  1 root adm     23M Aug 17 06:25 error.log.1
-rw-r-----  1 root adm    4.8M Aug 16 06:30 access.log.2.gz
-rw-r-----  1 root adm    1.1M Aug 16 06:25 error.log.2.gz
-rw-r-----  1 root adm    4.8M Aug 15 06:25 access.log.3.gz
-rw-r-----  1 root adm    1.1M Aug 15 06:25 error.log.3.gz
-rw-r-----  1 root adm    8.7M Aug 14 06:30 access.log.4.gz
-rw-r-----  1 root adm    1.5M Aug 14 06:25 error.log.4.gz
-rw-r-----  1 root adm    6.8M Aug 13 06:30 access.log.5.gz
-rw-r-----  1 root adm    1.4M Aug 13 06:25 error.log.5.gz
-rw-r-----  1 root adm    5.4M Aug 12 06:30 access.log.6.gz
-rw-r-----  1 root adm    1.2M Aug 12 06:25 error.log.6.gz
-rw-r-----  1 root adm    4.3M Aug 11 06:30 access.log.7.gz
-rw-r-----  1 root adm    967K Aug 11 06:25 error.log.7.gz
-rw-r-----  1 root adm    4.3M Aug 10 06:30 access.log.8.gz
-rw-r-----  1 root adm    905K Aug 10 06:25 error.log.8.gz
-rw-r-----  1 root adm    5.9M Aug  9 06:30 access.log.9.gz
-rw-r-----  1 root adm    1.1M Aug  9 06:25 error.log.9.gz
-rw-r-----  1 root adm    5.0M Aug  8 06:25 access.log.10.gz
-rw-r-----  1 root adm    1.1M Aug  8 06:25 error.log.10.gz
-rw-r-----  1 root adm    6.7M Aug  7 06:30 access.log.11.gz
-rw-r-----  1 root adm    1.3M Aug  7 06:25 error.log.11.gz
-rw-r-----  1 root adm    9.3M Aug  6 06:30 access.log.12.gz
-rw-r-----  1 root adm    1.8M Aug  6 06:25 error.log.12.gz
-rw-r-----  1 root adm    6.7M Aug  5 06:30 access.log.13.gz
-rw-r-----  1 root adm    1.4M Aug  5 06:30 error.log.13.gz
-rw-r-----  1 root adm    6.6M Aug  4 06:30 access.log.14.gz
-rw-r-----  1 root adm    1.3M Aug  4 06:25 error.log.14.gz
-rw-r-----  1 root adm       0 Mar 21 08:37 other_vhosts_access.log
  • iteagle03 August 23, 2017

    It's ultimately up to you what you delete but do you see what I am trying to do? I'm trying to go through your logs and clear up some space. Hopefully after clear out some big log files we'll have enough disk space to run the 'sudo du -aBM 2>/dev/null | sort -nr | head -n 10' command (lets add sudo to include some directories that might only be accessible as a sudo or root user). From there you can really dive into where your disk usage is and attempt to resolve that roadblock.

    Another command you can run after clearing out your log files might be 'sudo apt-get autoremove' which will remove any packages that aren't needed by your system.

    It's that or we add some block storage. You will know your sever better than anyone but something has eaten up all your available disk space.

  • mformainuddin August 23, 2017

    @iteagle03
    ok then we remove that btmp.1 file... i think it's a size(89mb) to run 'sudo du -aBM 2>/dev/null | sort -nr | head -n 10'...
    is it ok??? basically i don't have that much skill on that.......& that's why i asked repeatedly...sorry

    root@ubuntu-1gb-nyc3-01:/var/log# ls -halt
total 205M
-rw-r-----  1 syslog adm    1.3M Aug 23 22:00 syslog
-rw-r-----  1 syslog adm    655K Aug 23 22:00 kern.log
-rw-r-----  1 syslog adm    466K Aug 23 21:59 auth.log
-rw-------  1 root   utmp    73M Aug 23 21:59 btmp
-rw-rw-r--  1 root   utmp    34K Aug 23 21:40 lastlog
-rw-rw-r--  1 root   utmp    32K Aug 23 21:40 wtmp
-rw-------  1 root   root   1.6K Aug 23 21:38 php7.0-fpm.log
-rw-r--r--  1 root   root   716K Aug 23 21:38 cloud-init-output.log
-rw-r--r--  1 syslog adm     22M Aug 23 21:38 cloud-init.log
-rw-r-----  1 syslog adm       0 Aug 23 06:25 syslog.1.gz
drwxrwxr-x 11 root   syslog 4.0K Aug 23 06:25 .
drwxr-x---  2 mysql  adm    4.0K Aug 23 06:25 mysql
drwxr-x---  2 root   adm    4.0K Aug 23 06:25 apache2
-rw-r-----  1 syslog adm    8.0K Aug 22 06:25 syslog.1
-rw-r-----  1 syslog adm     14M Aug 21 06:25 auth.log.1
-rw-r-----  1 syslog adm    764K Aug 21 06:25 kern.log.1
-rw-r-----  1 syslog adm    8.9K Aug 21 06:25 syslog.2.gz
-rw-r-----  1 syslog adm       0 Aug 20 06:25 syslog.1.gz-2017082106.backup
drwxr-xr-x  2 root   adm    4.0K Aug 19 06:25 nginx
-rw-r-----  1 syslog adm     69K Aug 19 06:25 syslog.3.gz
-rw-------  1 root   root   1.6K Aug 18 19:40 php7.0-fpm.log.1
-rw-r-----  1 syslog adm       0 Aug 18 06:25 syslog.1.gz-2017081906.backup
-rw-r-----  1 syslog adm     24K Aug 17 06:25 syslog.4.gz
-rw-r-----  1 syslog adm    2.3K Aug 16 06:25 syslog.5.gz
-rw-r--r--  1 root   root    59K Aug 16 04:53 dpkg.log
-rw-r-----  1 syslog adm    2.4K Aug 15 06:25 syslog.6.gz
-rw-r-----  1 syslog adm     50K Aug 14 06:25 syslog.7.gz
-rw-r-----  1 syslog adm    1.2M Aug 13 06:25 auth.log.2.gz
-rw-r--r--  1 root   root      0 Aug 12 06:25 alternatives.log
-rw-r-----  1 syslog adm     154 Aug 11 14:46 kern.log.2.gz
-rw-------  1 root   root    275 Aug 11 10:16 php7.0-fpm.log.2.gz
-rw-r--r--  1 root   root   1.5K Aug 11 10:16 alternatives.log.1
-rw-r-----  1 syslog adm    2.0M Aug  7 06:25 auth.log.3.gz
-rw-r-----  1 root   adm       0 Aug  3 06:25 apport.log
-rw-r-----  1 root   adm     510 Aug  2 15:42 apport.log.1
-rw-r-----  1 syslog adm     11K Aug  2 15:42 kern.log.3.gz
drwxr-x---  2 root   adm    4.0K Aug  1 06:25 unattended-upgrades
drwxr-xr-x  2 root   root   4.0K Aug  1 06:25 apt
****-rw-------  1 root   utmp    89M Aug  1 06:25 btmp.1****
-rw-r-----  1 syslog adm    1.5M Jul 30 06:25 auth.log.4.gz
-rw-------  1 root   root     76 Jul 30 06:25 php7.0-fpm.log.3.gz
-rw-r-----  1 syslog adm     33K Jul 29 15:49 kern.log.4.gz
-rw-r-----  1 root   adm     321 Jul 29 15:49 apport.log.2.gz
-rw-r--r--  1 root   root    87K Jul 28 08:35 dpkg.log.1
-rw-r-----  1 root   adm     282 Jul 26 08:40 apport.log.3.gz
-rw-r-----  1 root   adm     289 Jul 25 11:45 apport.log.4.gz
-rw-------  1 root   root     76 Jul 23 06:25 php7.0-fpm.log.4.gz
-rw-r--r--  1 root   root    127 Jul 21 05:20 alternatives.log.2.gz
-rw-r-----  1 root   adm     835 Jul 18 03:18 apport.log.5.gz
-rw-------  1 root   root     76 Jul 17 06:25 php7.0-fpm.log.5.gz
-rw-rw-r--  1 root   utmp    14K Jul 13 04:58 wtmp.1
-rw-------  1 root   root    283 Jul 13 04:58 php7.0-fpm.log.6.gz
-rw-r-----  1 root   adm    6.9K Jul  9 18:46 apport.log.6.gz
-rw-------  1 root   root    275 Jul  4 18:19 php7.0-fpm.log.7.gz
-rw-------  1 root   root     76 Jun 25 06:25 php7.0-fpm.log.8.gz
-rw-r--r--  1 root   root   2.1K Jun 20 03:00 dpkg.log.2.gz
-rw-------  1 root   root     76 Jun 19 06:25 php7.0-fpm.log.9.gz
-rw-r-----  1 root   adm    8.4K Jun 18 15:35 apport.log.7.gz
-rw-------  1 root   root    278 Jun 15 09:56 php7.0-fpm.log.10.gz
-rw-------  1 root   root    311 Jun  9 17:44 php7.0-fpm.log.11.gz
-rw-------  1 root   root   1022 Jun  1 14:01 php7.0-fpm.log.12.gz
-rw-r--r--  1 root   root    25K May 31 01:41 dpkg.log.3.gz
-rw-r--r--  1 root   root   1.2K May 20 06:58 alternatives.log.3.gz
-rw-r--r--  1 root   root   1.1K May 17 14:51 fontconfig.log
-rw-r--r--  1 root   root   2.4K Apr 28 05:05 dpkg.log.4.gz
-rw-r--r--  1 root   root    126 Apr 28 05:05 alternatives.log.4.gz
-rw-r--r--  1 root   root    12K Mar 30 07:39 dpkg.log.5.gz
drwxr-xr-x  2 root   root   4.0K Mar 21 10:22 dbconfig-common
-rw-r--r--  1 root   root    810 Mar 21 09:21 alternatives.log.5.gz
drwxr-xr-x 14 root   root   4.0K Mar 21 08:37 ..
drwxr-xr-x  2 root   root   4.0K Mar  8 02:17 fsck
drwxr-xr-x  2 root   root   4.0K Feb 23 18:05 lxd
drwxr-xr-x  2 root   root   4.0K Jan  5  2017 dist-upgrade
  • mformainuddin August 24, 2017

    @iteagle03

    i just delete btmp.1 file and and i use this command "sudo du -aBM 2>/dev/null | sort -nr | head -n 10" & i see there is a file in this directory "/var/www/html/wp-content/log/oooo here is file named dbcache(24gb) " and deleted that file now disk space is:

    Filesystem      Size  Used Avail Use% Mounted on
udev            992M     0  992M   0% /dev
tmpfs           201M  3.1M  197M   2% /run
/dev/vda1        30G  5.0G   25G  18% /
tmpfs          1001M     0 1001M   0% /dev/shm
tmpfs           5.0M     0  5.0M   0% /run/lock
tmpfs          1001M     0 1001M   0% /sys/fs/cgroup
tmpfs           201M     0  201M   0% /run/user/0

    but till mysql won't start.... what i need to do now???

  • mformainuddin August 24, 2017

    @iteagle03
    now my apache2 and mysql both service is not starting it shows
    apache2.service:

    root@ubuntu-1gb-nyc3-01:~# sudo service apache2 restart
Job for apache2.service failed because the control process exited with error code. See "systemctl status apache2.service" and "journalctl -xe" for details.
root@ubuntu-1gb-nyc3-01:~# systemctl status apache2.service
● apache2.service - LSB: Apache2 web server
   Loaded: loaded (/etc/init.d/apache2; bad; vendor preset: enabled)
  Drop-In: /lib/systemd/system/apache2.service.d
           └─apache2-systemd.conf
   Active: failed (Result: exit-code) since Thu 2017-08-24 13:04:48 UTC; 25s ago
     Docs: man:systemd-sysv-generator(8)
  Process: 2770 ExecStart=/etc/init.d/apache2 start (code=exited, status=1/FAILURE)

Aug 24 13:04:48 ubuntu-1gb-nyc3-01 apache2[2770]: Output of config test was:
Aug 24 13:04:48 ubuntu-1gb-nyc3-01 apache2[2770]: (2)No such file or directory: AH02291: Cannot access directory '/var/log/apache2/' for main error log
Aug 24 13:04:48 ubuntu-1gb-nyc3-01 apache2[2770]: (2)No such file or directory: AH02291: Cannot access directory '/var/log/apache2/' for error log of vhost defined at
Aug 24 13:04:48 ubuntu-1gb-nyc3-01 apache2[2770]: AH00014: Configuration check failed
Aug 24 13:04:48 ubuntu-1gb-nyc3-01 apache2[2770]: Action 'configtest' failed.
Aug 24 13:04:48 ubuntu-1gb-nyc3-01 apache2[2770]: The Apache error log may have more information.
Aug 24 13:04:48 ubuntu-1gb-nyc3-01 systemd[1]: apache2.service: Control process exited, code=exited status=1
Aug 24 13:04:48 ubuntu-1gb-nyc3-01 systemd[1]: Failed to start LSB: Apache2 web server.
Aug 24 13:04:48 ubuntu-1gb-nyc3-01 systemd[1]: apache2.service: Unit entered failed state.
Aug 24 13:04:48 ubuntu-1gb-nyc3-01 systemd[1]: apache2.service: Failed with result 'exit-code'.
lines 1-18/18 (END)
root@ubuntu-1gb-nyc3-01:~# journalctl -xe
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- Unit apache2.service has failed.
--
-- The result is failed.
Aug 24 13:03:04 ubuntu-1gb-nyc3-01 systemd[1]: apache2.service: Unit entered failed state.
Aug 24 13:03:04 ubuntu-1gb-nyc3-01 systemd[1]: apache2.service: Failed with result 'exit-code'.
Aug 24 13:03:04 ubuntu-1gb-nyc3-01 sudo[2739]: pam_unix(sudo:session): session closed for user root
Aug 24 13:04:48 ubuntu-1gb-nyc3-01 sudo[2764]:     root : TTY=pts/0 ; PWD=/root ; USER=root ; COMMAND=/usr/sbin/service apache2 restart
Aug 24 13:04:48 ubuntu-1gb-nyc3-01 sudo[2764]: pam_unix(sudo:session): session opened for user root by root(uid=0)
Aug 24 13:04:48 ubuntu-1gb-nyc3-01 systemd[1]: Stopped LSB: Apache2 web server.
-- Subject: Unit apache2.service has finished shutting down
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- Unit apache2.service has finished shutting down.
Aug 24 13:04:48 ubuntu-1gb-nyc3-01 systemd[1]: Starting LSB: Apache2 web server...
-- Subject: Unit apache2.service has begun start-up
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- Unit apache2.service has begun starting up.
Aug 24 13:04:48 ubuntu-1gb-nyc3-01 apache2[2770]:  * Starting Apache httpd web server apache2
Aug 24 13:04:48 ubuntu-1gb-nyc3-01 apache2[2770]:  *
Aug 24 13:04:48 ubuntu-1gb-nyc3-01 apache2[2770]:  * The apache2 configtest failed.
Aug 24 13:04:48 ubuntu-1gb-nyc3-01 apache2[2770]: Output of config test was:
Aug 24 13:04:48 ubuntu-1gb-nyc3-01 apache2[2770]: (2)No such file or directory: AH02291: Cannot access directory '/var/log/apache2/' for main error log
Aug 24 13:04:48 ubuntu-1gb-nyc3-01 apache2[2770]: (2)No such file or directory: AH02291: Cannot access directory '/var/log/apache2/' for error log of vhost defined at
Aug 24 13:04:48 ubuntu-1gb-nyc3-01 apache2[2770]: AH00014: Configuration check failed
Aug 24 13:04:48 ubuntu-1gb-nyc3-01 apache2[2770]: Action 'configtest' failed.
Aug 24 13:04:48 ubuntu-1gb-nyc3-01 apache2[2770]: The Apache error log may have more information.
Aug 24 13:04:48 ubuntu-1gb-nyc3-01 systemd[1]: apache2.service: Control process exited, code=exited status=1
Aug 24 13:04:48 ubuntu-1gb-nyc3-01 systemd[1]: Failed to start LSB: Apache2 web server.
-- Subject: Unit apache2.service has failed
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- Unit apache2.service has failed.
--
-- The result is failed.
Aug 24 13:04:48 ubuntu-1gb-nyc3-01 systemd[1]: apache2.service: Unit entered failed state.
Aug 24 13:04:48 ubuntu-1gb-nyc3-01 systemd[1]: apache2.service: Failed with result 'exit-code'.
Aug 24 13:04:48 ubuntu-1gb-nyc3-01 sudo[2764]: pam_unix(sudo:session): session closed for user root

    and mysql.service:

    root@ubuntu-1gb-nyc3-01:~# sudo service mysql restart
Job for mysql.service failed because the control process exited with error code. See "systemctl status mysql.service" and "journalctl -xe" for details.
root@ubuntu-1gb-nyc3-01:~# systemctl status mysql.service
● mysql.service - MySQL Community Server
   Loaded: loaded (/lib/systemd/system/mysql.service; enabled; vendor preset: enabled)
   Active: activating (start-post) (Result: exit-code) since Thu 2017-08-24 13:12:58 UTC; 18s ago
  Process: 3672 ExecStart=/usr/sbin/mysqld (code=exited, status=1/FAILURE)
  Process: 3663 ExecStartPre=/usr/share/mysql/mysql-systemd-start pre (code=exited, status=0/SUCCESS)
 Main PID: 3672 (code=exited, status=1/FAILURE);         : 3673 (mysql-systemd-s)
    Tasks: 2
   Memory: 324.0K
      CPU: 383ms
   CGroup: /system.slice/mysql.service
           └─control
             ├─3673 /bin/bash /usr/share/mysql/mysql-systemd-start post
             └─3713 sleep 1

Aug 24 13:12:58 ubuntu-1gb-nyc3-01 systemd[1]: Starting MySQL Community Server...
Aug 24 13:12:58 ubuntu-1gb-nyc3-01 systemd[1]: mysql.service: Main process exited, code=exited, status=1/FAILURE
root@ubuntu-1gb-nyc3-01:~# journalctl -xe
Aug 24 13:12:58 ubuntu-1gb-nyc3-01 systemd[1]: Starting MySQL Community Server...
-- Subject: Unit mysql.service has begun start-up
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- Unit mysql.service has begun starting up.
Aug 24 13:12:58 ubuntu-1gb-nyc3-01 audit[3672]: AVC apparmor="DENIED" operation="open" profile="/usr/sbin/mysqld" name="/proc/3672/status" pid=3672 comm="mysqld" reque
Aug 24 13:12:58 ubuntu-1gb-nyc3-01 audit[3672]: AVC apparmor="DENIED" operation="open" profile="/usr/sbin/mysqld" name="/sys/devices/system/node/" pid=3672 comm="mysql
Aug 24 13:12:58 ubuntu-1gb-nyc3-01 audit[3672]: AVC apparmor="DENIED" operation="open" profile="/usr/sbin/mysqld" name="/proc/3672/status" pid=3672 comm="mysqld" reque
Aug 24 13:12:58 ubuntu-1gb-nyc3-01 kernel: audit: type=1400 audit(1503580378.245:85): apparmor="DENIED" operation="open" profile="/usr/sbin/mysqld" name="/proc/3672/st
Aug 24 13:12:58 ubuntu-1gb-nyc3-01 kernel: audit: type=1400 audit(1503580378.245:86): apparmor="DENIED" operation="open" profile="/usr/sbin/mysqld" name="/sys/devices/
Aug 24 13:12:58 ubuntu-1gb-nyc3-01 kernel: audit: type=1400 audit(1503580378.245:87): apparmor="DENIED" operation="open" profile="/usr/sbin/mysqld" name="/proc/3672/st
Aug 24 13:12:58 ubuntu-1gb-nyc3-01 systemd[1]: mysql.service: Main process exited, code=exited, status=1/FAILURE
Aug 24 13:13:28 ubuntu-1gb-nyc3-01 systemd[1]: Failed to start MySQL Community Server.
-- Subject: Unit mysql.service has failed
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- Unit mysql.service has failed.
--
-- The result is failed.
Aug 24 13:13:28 ubuntu-1gb-nyc3-01 systemd[1]: mysql.service: Unit entered failed state.
Aug 24 13:13:28 ubuntu-1gb-nyc3-01 systemd[1]: mysql.service: Failed with result 'exit-code'.
Aug 24 13:13:28 ubuntu-1gb-nyc3-01 systemd[1]: mysql.service: Service hold-off time over, scheduling restart.
Aug 24 13:13:28 ubuntu-1gb-nyc3-01 systemd[1]: Stopped MySQL Community Server.
-- Subject: Unit mysql.service has finished shutting down
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- Unit mysql.service has finished shutting down.
Aug 24 13:13:28 ubuntu-1gb-nyc3-01 systemd[1]: Starting MySQL Community Server...
-- Subject: Unit mysql.service has begun start-up
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- Unit mysql.service has begun starting up.
Aug 24 13:13:28 ubuntu-1gb-nyc3-01 audit[3754]: AVC apparmor="DENIED" operation="open" profile="/usr/sbin/mysqld" name="/proc/3754/status" pid=3754 comm="mysqld" reque
Aug 24 13:13:28 ubuntu-1gb-nyc3-01 audit[3754]: AVC apparmor="DENIED" operation="open" profile="/usr/sbin/mysqld" name="/sys/devices/system/node/" pid=3754 comm="mysql
Aug 24 13:13:28 ubuntu-1gb-nyc3-01 audit[3754]: AVC apparmor="DENIED" operation="open" profile="/usr/sbin/mysqld" name="/proc/3754/status" pid=3754 comm="mysqld" reque
Aug 24 13:13:28 ubuntu-1gb-nyc3-01 kernel: audit: type=1400 audit(1503580408.737:88): apparmor="DENIED" operation="open" profile="/usr/sbin/mysqld" name="/proc/3754/st
Aug 24 13:13:28 ubuntu-1gb-nyc3-01 kernel: audit: type=1400 audit(1503580408.741:89): apparmor="DENIED" operation="open" profile="/usr/sbin/mysqld" name="/sys/devices/
Aug 24 13:13:28 ubuntu-1gb-nyc3-01 kernel: audit: type=1400 audit(1503580408.741:90): apparmor="DENIED" operation="open" profile="/usr/sbin/mysqld" name="/proc/3754/st
Aug 24 13:13:28 ubuntu-1gb-nyc3-01 systemd[1]: mysql.service: Main process exited, code=exited, status=1/FAILURE
  • iteagle03 August 24, 2017

    So, in regards to your apache2 error it can't write to the directory '/var/log/apache2/'. I have no idea what the problem is with mysql.

  • mformainuddin August 26, 2017

    @iteagle03
    write to the directory '/var/log/apache2/'..... what i need to do???
    Thanks

  • iteagle03 August 26, 2017

    Is the directory present? If so, is the access.log, error.log, and othervhostsaccess.log log files present in the directory?

  • mformainuddin August 26, 2017

    @iteagle03
    I solved all issue...Thanks a lot
    last.... can you help how to use any tool that can help to optimize or delete unnecessary file in ubuntu or any command???

  • oplaze July 8, 2018

    Thanks not letting rest of the people know how did you solve the issue?

Log In to Comment
3 Answers
Aprexer August 23, 2017

Could you let us know what you've already tried to do to resolve your issue?
Have you googled your error? If so have you tried those solutions?
Have you tried purging mysql-server and start again?

Reply
mformainuddin August 23, 2017

@Aprexer

I tried start and stop mysql.service and install mysql tuner... and try to open my.cnf .... that's it

Reply
emmaakin November 22, 2017

You should first setup a swapfile. and also add line like this
innodb_buffer_pool_size = 20M to the end of the line of your /etc/mysql/my.cnf

!includedir /etc/mysql/conf.d/
!includedir /etc/mysql/mysql.conf.d/
[mysqld]
innodb_buffer_pool_size = 20M```
Reply
Have another answer? Share your knowledge.

Related Questions