Please whitelist cPanel in your adblocker so that you’re able to see our version release promotions, thanks!

The Community Forums

Interact with an entire community of cPanel & WHM users!

[CPANEL-21627] Chkservd reports service failures during graceful reboots

Discussion in 'General Discussion' started by leadwatch, Jun 11, 2018.

Tags:
  1. leadwatch

    leadwatch Registered

    Joined:
    Jun 11, 2018
    Messages:
    4
    Likes Received:
    0
    Trophy Points:
    1
    Location:
    NJ, USA
    cPanel Access Level:
    Root Administrator
    Every time I do a graceful reboot in WHM, a few minutes later, I receive a cPanel alert email for each service that says it failed. The message is the same for all services. The alerts typically only come in once, but sometimes a few will be sent a second time. After that they stop.

    For example:
    Service Check Raw Output:
    The “mysql” service is down.
    The subprocess “/usr/local/cpanel/scripts/restartsrv_mysql” reported error number 255 when it ended.

    Server is running CENTOS 7.5 kvm, cPanel/WHM v70.0.48 (though the issue has been happening for months, seemingly independent of version). The setup is default from the host. I do updates and the associated graceful restarts (which is when I experience the issue) as needed.

    I'm relatively new to server administration. I've been searching for a solution for this but have come up empty other than turning off failure alerts altogether, which I don't want to do.

    Does anyone know why this happens and how I can fix it without turning off alerts?
     
  2. cPanelMichael

    cPanelMichael Technical Support Community Manager
    Staff Member

    Joined:
    Apr 11, 2011
    Messages:
    45,442
    Likes Received:
    1,961
    Trophy Points:
    363
    cPanel Access Level:
    Root Administrator
    Twitter:
    Hello @leadwatch,

    This can happen after a reboot when the service monitoring process (Chksrvd) starts before the other services. You can try increasing the default value of "3" to a value such as "4" for the following option under the System tab in WHM >> Tweak Settings:

    ChkServd TCP check failure threshold

    Per it's description:

    The number of times a ChkServd TCP check must fail before notification is sent and the service is restarted. On heavily loaded systems these types of service checks fail occasionally, producing erroneous indications that services are down. A value of 3 or higher is recommended for most systems.

    Thank you.
     
    Stop hovering to collapse... Click to collapse... Hover to expand... Click to expand...
  3. leadwatch

    leadwatch Registered

    Joined:
    Jun 11, 2018
    Messages:
    4
    Likes Received:
    0
    Trophy Points:
    1
    Location:
    NJ, USA
    cPanel Access Level:
    Root Administrator
    Hi Michael,

    I set the ChkServd TCP check failure threshold to 5 and rebooted the server to test. It did not fix the issue. I still got a series of alert emails for failed services right after reboot. Do you have any other suggestions?

    Thanks.
     
  4. cPanelMichael

    cPanelMichael Technical Support Community Manager
    Staff Member

    Joined:
    Apr 11, 2011
    Messages:
    45,442
    Likes Received:
    1,961
    Trophy Points:
    363
    cPanel Access Level:
    Root Administrator
    Twitter:
    Stop hovering to collapse... Click to collapse... Hover to expand... Click to expand...
  5. leadwatch

    leadwatch Registered

    Joined:
    Jun 11, 2018
    Messages:
    4
    Likes Received:
    0
    Trophy Points:
    1
    Location:
    NJ, USA
    cPanel Access Level:
    Root Administrator
    I opened a ticket as you suggested.
     
  6. cPanelMichael

    cPanelMichael Technical Support Community Manager
    Staff Member

    Joined:
    Apr 11, 2011
    Messages:
    45,442
    Likes Received:
    1,961
    Trophy Points:
    363
    cPanel Access Level:
    Root Administrator
    Twitter:
    Hello @leadwatch,

    To update, per support ticket 9730779, it looks like the services were not actually failing. Instead the service checks from Chkservd occurred while the server was in the process of shutting down and thus the notifications were sent when the server booted back up.

    Thank you.
     
    Stop hovering to collapse... Click to collapse... Hover to expand... Click to expand...
  7. leadwatch

    leadwatch Registered

    Joined:
    Jun 11, 2018
    Messages:
    4
    Likes Received:
    0
    Trophy Points:
    1
    Location:
    NJ, USA
    cPanel Access Level:
    Root Administrator
    Yes, this is the problem I'm having. I suspected the services were not actually failing, but it's nice to have confirmation. However, that still leaves me with the same problem - a flood of unnecessary and unwanted failure emails every time I restart the server.

    Response from cPanel support:
    Has anyone in the community experienced this issue? Does anyone know of a workaround or a setting that may be causing this?
     
  8. cPanelMichael

    cPanelMichael Technical Support Community Manager
    Staff Member

    Joined:
    Apr 11, 2011
    Messages:
    45,442
    Likes Received:
    1,961
    Trophy Points:
    363
    cPanel Access Level:
    Root Administrator
    Twitter:
    Hello @leadwatch,

    We do have an internal case open (CPANEL-21627) that would address this issue by suspending Chkservd (the service monitoring daemon) upon initiating a graceful reboot through Web Host Manager. I'll monitor this case and update this thread with more information on it's status as it becomes available. There's no workaround to report at this time, but you can safely ignore the notifications that are sent during the time the server is rebooting.

    Thank you.
     
    Stop hovering to collapse... Click to collapse... Hover to expand... Click to expand...
Loading...

Share This Page

  1. This site uses cookies to help personalise content, tailor your experience and to keep you logged in if you register.
    By continuing to use this site, you are consenting to our use of cookies.
    Dismiss Notice