Please whitelist cPanel in your adblocker so that you’re able to see our version release promotions, thanks!

The Community Forums

Interact with an entire community of cPanel & WHM users!

MySQL down - chkserv.d never catch this.

Discussion in 'General Discussion' started by ispro, Oct 28, 2004.

  1. ispro

    ispro Well-Known Member

    Joined:
    Apr 8, 2004
    Messages:
    628
    Likes Received:
    1
    Trophy Points:
    168
    I have a problem.
    chkservd.d doesn't catch if mysql is really responding.

    For example, I'm removing /var/lib/mysql/HOSTNAME.pid while mysql not stopped.
    Then starting mysql (another copy!) service mysql start
    Then check it:
    mysql
    ERROR 2002: Can't connect to local MySQL server through socket '/var/lib/mysql/mysql.sock' (2)
    It's nice.

    But chkserv.d not catch that MySQL is already down.
    service chkservd restart
    doesn't help.

    I've tuned /etc/chkserv.d/mysql to the following (to remove old mysql process):
    killall -TERM mysql;sleep 2;killall -9 -g mysql;killall -TERM mysqld;sleep 2;killall -9 -g mysqld;/scripts/restartsrv mysql

    However looks like a problem in how chkserv.d threat its checks. Not in command to execute.

    May anyone help.
     
    Stop hovering to collapse... Click to collapse... Hover to expand... Click to expand...
  2. Sinewy

    Sinewy Well-Known Member

    Joined:
    May 15, 2004
    Messages:
    367
    Likes Received:
    1
    Trophy Points:
    168
    Location:
    Sydney, Australia
    cPanel Access Level:
    DataCenter Provider
    as a rule of thumb: removing the mysql pid file kills the socket. not recommended.
     
  3. ispro

    ispro Well-Known Member

    Joined:
    Apr 8, 2004
    Messages:
    628
    Likes Received:
    1
    Trophy Points:
    168
    Damn! I knew this!
    It was the modelled situation which chkserv.d cannot to handle.

    I'm asking about how to make chkserv.d to deal with such a situations!
     
    Stop hovering to collapse... Click to collapse... Hover to expand... Click to expand...
  4. netlook

    netlook Well-Known Member
    PartnerNOC

    Joined:
    Mar 25, 2004
    Messages:
    335
    Likes Received:
    0
    Trophy Points:
    166
    Yeah, good question. Can anybody say something about this? How to solve the problem? I also have this situation - there are opened too much mysqld procesess on my server and then mysql stops working with information "Too many connections" - but chkservd didn't catch this... Any idea?
     
    #4 netlook, Oct 28, 2004
    Last edited: Oct 29, 2004
  5. MattDr2

    MattDr2 Well-Known Member
    PartnerNOC

    Joined:
    Feb 19, 2003
    Messages:
    84
    Likes Received:
    0
    Trophy Points:
    156
    Location:
    Norman, OK
    We are also looking for a resolve. This has become a big problem lately.

    Regards,
    Matt
     
  6. ispro

    ispro Well-Known Member

    Joined:
    Apr 8, 2004
    Messages:
    628
    Likes Received:
    1
    Trophy Points:
    168
    I would like to share the simple piece of code that helps me to at least restart MySQL (and chkservd additionally). Remember to have /root/.my.cnf with MySQL access info! (cPanel servers has it already).

    Put in the cron under root user and you will be happy :)

    Code:
    #!/usr/bin/perl
    
    print "Checking if the common services up & running\n";
    mysql();
    chkservd();
    
    # MySQL
    sub mysql {
    print "\n*** Checking if the MySQLrunning... ***\n\n";
    $mysql_status = `mysqladmin status`;
    print $mysql_status;
    if ($mysql_status =~ m/Uptime/)
        {
        print "\nMySQL uptime ok.\n";
        }
    else
        {
        print "\nMySQL down/unreachable! Going to hard-reboot it...\n";
        `killall -TERM mysql;sleep 2;killall -9 -g mysql;killall -TERM mysqld;sleep 2;killall -9 -g mysqld;/scripts/restartsrv mysql`;
        }
    }
    
    # chkservd
    sub chkservd {
    print "\n*** Checking if the chkservd running... ***\n\n";
    $chkservd_status = `service chkservd status`;
    print $chkservd_status;
    if ($chkservd_status =~ m/running/)
        {
        print "\nchkservd running.\n";
        }
    else
        {
        print "\nchkservd stopped! Going to restart it...\n";
        `service chkservd restart`;
        }
    }
    
    
     
    Stop hovering to collapse... Click to collapse... Hover to expand... Click to expand...
  7. raventec

    raventec Well-Known Member

    Joined:
    Apr 19, 2003
    Messages:
    120
    Likes Received:
    0
    Trophy Points:
    166
    Neat!

    :) Neat! Thanks for that!
     
  8. netlook

    netlook Well-Known Member
    PartnerNOC

    Joined:
    Mar 25, 2004
    Messages:
    335
    Likes Received:
    0
    Trophy Points:
    166
    Great!

    :) :) :)

    Very Very Very Thank You!
     
  9. cPanelNick

    cPanelNick Administrator
    Staff Member

    Joined:
    Mar 9, 2015
    Messages:
    3,480
    Likes Received:
    29
    Trophy Points:
    158
    cPanel Access Level:
    DataCenter Provider
    We are going to switch back to the method of checking the mysql server status instead of checking if its running or not in the next release of /scripts.
     
  10. cPanelNick

    cPanelNick Administrator
    Staff Member

    Joined:
    Mar 9, 2015
    Messages:
    3,480
    Likes Received:
    29
    Trophy Points:
    158
    cPanel Access Level:
    DataCenter Provider
    You can get this update by running /scripts/updatenow
     
  11. ispro

    ispro Well-Known Member

    Joined:
    Apr 8, 2004
    Messages:
    628
    Likes Received:
    1
    Trophy Points:
    168
    What exactly updated by this?
    chkserv.d or something else?

    Btw, I've noticed that chkserv.d constantly restarts MySQL (I have removed all the other monitoring scripts already!). I was insisted to stop chkservd service!

    What can be the case?..
     
    Stop hovering to collapse... Click to collapse... Hover to expand... Click to expand...
    #11 ispro, Oct 29, 2004
    Last edited: Oct 29, 2004
  12. netlook

    netlook Well-Known Member
    PartnerNOC

    Joined:
    Mar 25, 2004
    Messages:
    335
    Likes Received:
    0
    Trophy Points:
    166
    How is it called and how it works? Or how to make it working?
     
  13. cPanelNick

    cPanelNick Administrator
    Staff Member

    Joined:
    Mar 9, 2015
    Messages:
    3,480
    Likes Received:
    29
    Trophy Points:
    158
    cPanel Access Level:
    DataCenter Provider
    check /var/log/chkservd.log
     
  14. manghooli

    manghooli Well-Known Member

    Joined:
    Aug 22, 2004
    Messages:
    55
    Likes Received:
    0
    Trophy Points:
    156
    released?!
     
  15. BenThomas

    BenThomas Well-Known Member

    Joined:
    Feb 12, 2004
    Messages:
    598
    Likes Received:
    0
    Trophy Points:
    166
    Location:
    Houston, Texas USA
    cPanel Access Level:
    Root Administrator
    Yes, it is released. Check /scripts/restartsrv_mysql for the actual code. Checksrvd calls /scripts/restartsrv_mysql --check to determine is mysql is running. Before it was just checking for mysql processes, now it makes a connection to the server.
     
  16. netlook

    netlook Well-Known Member
    PartnerNOC

    Joined:
    Mar 25, 2004
    Messages:
    335
    Likes Received:
    0
    Trophy Points:
    166
    Ok, everything is working fine when I manually run this script, but when I put it into cron with:

    */5 * * * * root /path/to/script

    I recive an e-mail notification with:

    mysqladmin: connect to server at 'localhost' failed
    error: 'Access denied for user: 'root@localhost' (Using password: NO)'


    How to prevent this?
     
  17. ispro

    ispro Well-Known Member

    Joined:
    Apr 8, 2004
    Messages:
    628
    Likes Received:
    1
    Trophy Points:
    168
    Add "HOME=/root" before cron job

    e.g.:
    HOME=/root
    */5 * * * * root /pathto/script >/dev/null 2>&1

    Then it will use /root/.my.cnf to connect to MySQL
     
    Stop hovering to collapse... Click to collapse... Hover to expand... Click to expand...
  18. netlook

    netlook Well-Known Member
    PartnerNOC

    Joined:
    Mar 25, 2004
    Messages:
    335
    Likes Received:
    0
    Trophy Points:
    166
    Of course :p I forgot... Shame on me :eek:

    But another question - also after running script from cron with fixed issue .my.cnf, in confirmation mail there is wrote that chkservd is restarted, because it wasn't running. Its not true. When I run the script manually form SSH everything is ok. Any idea?

    Thank you
     
  19. ispro

    ispro Well-Known Member

    Joined:
    Apr 8, 2004
    Messages:
    628
    Likes Received:
    1
    Trophy Points:
    168
    I have no idea on that...
    Add the $chkservd_status to the mail body to see if it has got a reply of "service chkservd status".

    We have no problems with that. Even if chkservd failed, our script restart it smoothly.
     
    Stop hovering to collapse... Click to collapse... Hover to expand... Click to expand...
  20. netlook

    netlook Well-Known Member
    PartnerNOC

    Joined:
    Mar 25, 2004
    Messages:
    335
    Likes Received:
    0
    Trophy Points:
    166
    Ok, thank you :)
     
Loading...

Share This Page

  1. This site uses cookies to help personalise content, tailor your experience and to keep you logged in if you register.
    By continuing to use this site, you are consenting to our use of cookies.
    Dismiss Notice