The Community Forums

Interact with an entire community of cPanel & WHM users!
  1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

MySQL Randomly Fails then Recovers

Discussion in 'Database Discussions' started by jabelone, Jun 18, 2015.

  1. jabelone

    jabelone Registered

    Joined:
    Jun 18, 2015
    Messages:
    3
    Likes Received:
    0
    Trophy Points:
    1
    Location:
    Brisbane, Queensland
    cPanel Access Level:
    Root Administrator
    Hi,

    I'm running a few websites and databases on a VPS I manage. It's running CentOS 6.6 and has been doing it for a while but it's starting to get annoying. :/ MySQL will randomly just "fail" then be recovered. Most sites are WordPress based and I have a mobile app I created that relies on accessing a database so it's quite important. The problem is I have no idea what is causing this to happen. Here is an example email I get from the server every time it happens.

    Code:
    [B]Server:[/B] gallifrey.********.com.au
    [B]Primary IP Address:[/B] ***.***.***.***
    [B]Service Name:[/B] mysql
    [B]Service Status:[/B] failed
    [B]Notification:[/B] The service “mysql” appears to be down.
    [B]Service Check Method:[/B] The system’s command to check or to restart this service failed.
    [B]Number of Restart Attempts:[/B] 1
    [B]Startup Log:[/B]
    Starting MySQL...... SUCCESS!
    [B]Log Messages:[/B]
    150617 12:09:47 mysqld_safe mysqld from pid file /var/lib/mysql/gallifrey.********.com.au.pid ended
    150617 12:09:47 [Note] /usr/sbin/mysqld: Shutdown complete
    150617 12:09:46 mysqld_safe mysqld restarted
    150617 12:09:46 mysqld_safe Number of processes running now: 0
    150617 3:26:20 [Note] /usr/sbin/mysqld: ready for connections.
    150617 03:26:15 mysqld_safe Starting mysqld daemon with databases from /var/lib/mysql
    150617 03:25:04 mysqld_safe mysqld from pid file /var/lib/mysql/gallifrey.********.com.au.pid ended
    150617 3:25:04 [Note] /usr/sbin/mysqld: Shutdown complete
    150617 03:25:03 mysqld_safe mysqld restarted
    150617 03:25:03 mysqld_safe Number of processes running now: 0
    150616 12:13:12 [Note] /usr/sbin/mysqld: ready for connections.
    150616 12:13:08 mysqld_safe Starting mysqld daemon with databases from /var/lib/mysql
    150616 12:09:44 mysqld_safe mysqld from pid file /var/lib/mysql/gallifrey.********.com.au.pid ended
    150616 12:09:44 [Note] /usr/sbin/mysqld: Shutdown complete
    150616 12:09:43 mysqld_safe mysqld restarted
    150616 12:09:43 mysqld_safe Number of processes running now: 0
    [B]Memory Information:[/B]
    [LIST]
    [*][LIST]
    [*]Used: 303 MB
    [*]Available: 313 MB
    [*]Installed: 616 MB
    [/LIST]
    [/LIST]
    [INDENT]Load Information: 0.20 0.30 0.15
    Uptime: 7 days, 19 hours, 33 minutes, and 9 seconds
    IOStat Information:
    avg-cpu: %user %nice %system %iowait %steal %idle
    2.43 0.44 0.65 3.89 0.03 92.56
    Device: tps Blk_read/s Blk_wrtn/s Blk_read Blk_wrtn
    vda 11.58 521.90 115.61 351719556 77913264
    ChkServd Version: 15.9[/INDENT]
    
    Then I get one of these:

    Code:
    [B]Server:[/B] gallifrey.********.com.au
    [B]Primary IP Address:[/B] ***.***.***.***
    [B]Service Name:[/B] mysql
    [B]Service Status:[/B] recovered
    [B]Notification:[/B] The service “mysql” is now operational.
    [B]Service Check Raw Output:[/B]
    The 'mysql' service passed the check.
    [B]Startup Log:[/B]
    Starting MySQL...... SUCCESS!
    [B]Log Messages:[/B]
    150617 3:26:20 [Note] /usr/sbin/mysqld: ready for connections.
    150617 03:26:15 mysqld_safe Starting mysqld daemon with databases from /var/lib/mysql
    150617 03:25:04 mysqld_safe mysqld from pid file /var/lib/mysql/gallifrey.********.com.au.pid ended
    150617 3:25:04 [Note] /usr/sbin/mysqld: Shutdown complete
    150617 03:25:03 mysqld_safe mysqld restarted
    150617 03:25:03 mysqld_safe Number of processes running now: 0
    150616 12:13:12 [Note] /usr/sbin/mysqld: ready for connections.
    150616 12:13:08 mysqld_safe Starting mysqld daemon with databases from /var/lib/mysql
    150616 12:09:44 mysqld_safe mysqld from pid file /var/lib/mysql/gallifrey.********.com.au.pid ended
    150616 12:09:44 [Note] /usr/sbin/mysqld: Shutdown complete
    150616 12:09:43 mysqld_safe mysqld restarted
    150616 12:09:43 mysqld_safe Number of processes running now: 0
    150616 3:24:52 [Note] /usr/sbin/mysqld: ready for connections.
    [B]Memory Information:[/B]
    [LIST]
    [*][LIST]
    [*]Used: 308 MB
    [*]Available: 308 MB
    [*]Installed: 616 MB
    [/LIST]
    [/LIST]
    [INDENT]Load Information: 0.22 0.61 0.40
    Uptime: 7 days, 10 hours, 51 minutes, and 16 seconds
    IOStat Information:
    avg-cpu: %user %nice %system %iowait %steal %idle
    2.45 0.44 0.65 3.95 0.03 92.47
    Device: tps Blk_read/s Blk_wrtn/s Blk_read Blk_wrtn
    vda 11.68 524.46 117.45 337052860 75479880
    ChkServd Version: 15.9
    
    
     
    #1 jabelone, Jun 18, 2015
    Last edited by a moderator: Jul 14, 2015
  2. cPanelMichael

    cPanelMichael Forums Analyst
    Staff Member

    Joined:
    Apr 11, 2011
    Messages:
    30,854
    Likes Received:
    676
    Trophy Points:
    113
    cPanel Access Level:
    Root Administrator
  3. jabelone

    jabelone Registered

    Joined:
    Jun 18, 2015
    Messages:
    3
    Likes Received:
    0
    Trophy Points:
    1
    Location:
    Brisbane, Queensland
    cPanel Access Level:
    Root Administrator
    Hi, sorry for the late reply - for some reason the forum didn't email me when you replied.

    Here is the last bit of the log from this morning, when it happened again. I have included a lot of the "before it crashed" if that helps. I got a cPanel service monitor email saying it had failed right around 11:22.

    P.S. Sorry for the terrible formatting in the OP, I don't know what happened and it won't let me edit it.


    - Removed - No Need For That Much Output in a Thread -
     
    #3 jabelone, Jul 13, 2015
    Last edited by a moderator: Jul 14, 2015
  4. jabelone

    jabelone Registered

    Joined:
    Jun 18, 2015
    Messages:
    3
    Likes Received:
    0
    Trophy Points:
    1
    Location:
    Brisbane, Queensland
    cPanel Access Level:
    Root Administrator
    This has been resolved after opening a support ticket with cPanel. As can be seen by these two entries in the log file, the server was running out of memory.

    Code:
    Jul 14 18:50:15 gallifrey kernel: Out of memory: Kill process 22589 (mysqld) score 159 or sacrifice child
    Jul 14 18:50:15 gallifrey kernel: Killed process 22589, UID 498, (mysqld) total-vm:1349600kB, anon-rss:161992kB, file-rss:244kB
     
  5. cPanelMichael

    cPanelMichael Forums Analyst
    Staff Member

    Joined:
    Apr 11, 2011
    Messages:
    30,854
    Likes Received:
    676
    Trophy Points:
    113
    cPanel Access Level:
    Root Administrator
Loading...

Share This Page