System Terminated Sub-Process Error

[venomdog]

Guys,
Excuse me if this question is asked thousands of time but I am getting this emails starting this month and I cannot find any solution for it.
I tried to search solution on this forum but looks like I have something else altogether. I managed to fix few other problems but this one is taking sometime for me.

The chkservd sub-process with PID “17521” ran for “5 minutes and 1 second”. The system terminated this sub-process when it exceeded the time allowed between checks, which is “5 minutes”. To determine why, check the “/var/log/chkservd.log ” and “ /usr/local/cpanel/logs/tailwatchd_log ” files.

You likely received this notification as a symptom of a larger problem. If your server is experiencing a high load, we recommend that you investigate the cause. If you continue to receive this notification, it is likely that your system is unable to handle demand or there is a misconfiguration that delays restarts.

If you are sure that no misconfigurations exist, you should consider gradually increasing the following options in WHM’s “Tweak Settings” feature: “The number of times chkservd will allow a previous check to complete before terminating the check”, “The number of seconds between chkservd service checks”, or both.


Notification Type hang ⚠
Service chkservd
Memory Information
Used 2.94 GB
Available 2.84 GB
Installed 3.02 GB
Load Information 23.16 15.08 9.10
Uptime 10 hours, 1 minute, and 19 seconds
IOStat Information avg-cpu:  %user  %nice %system %iowait  %steal  %idle           6.91    0.35    1.06    0.44    0.00  91.24 Device:            tps  Blk_read/s  Blk_wrtn/s  Blk_read  Blk_wrtn
ChkServd Version 17.0
Top Processes
PID Owner CPU % Memory % Command
17997 nobody 45.83 0.30 /usr/local/cpanel/3rdparty/perl/514/bin/perl /etc/munin/plugins/apache_accesses
17991 root 36.17 0.37 /usr/local/cpanel/scripts/restartsrv_sshd - waiting for lock
1358 mysql 21.11 1.59 /usr/sbin/mysqld --basedir=/usr --datadir=/var/lib/mysql --plugin-dir=/usr/lib64/mysql/plugin --user=mysql --log-error=/var/lib/mysql/shagari.satsom.in.err --open-files-limit=10000 --pid-file=/var/lib/mysql/shagari.satsom.in.pid
17973 munin 1.17 0.46 /usr/local/cpanel/3rdparty/perl/514/bin/perl /usr/local/cpanel/3rdparty/share/munin/munin-update
17951 root 0.44 0.58 tailwatchd - chkservd - sshd check


Preview of “cpanel_chkservd_log_tail.txt”
[2016-01-25 22:14:04 -0500] Disk check .... simfs (/) [10.68%] ... {status:eek:k} ... Done
Loading services .....clamd....cpanellogd....cpdavd....cphulkd....cpsrvd....crond....dnsadmin....exim....ftpd....httpd....imap....ipaliases....lfd....mailman....mysql....named....nscd....p0f....pop....queueprocd....rsyslogd....spamd....sshd..Done
Service Check Started
Service Check Finished
[2016-01-25 22:09:04 -0500] Service check ....sshd [[check command:+][socket connect:N/A]]...spamd [[check command:+][socket connect:N/A]]...rsyslogd [[check command:+][socket connect:N/A]]...queueprocd [[check command:+][socket connect:N/A]]...pop [[check command:+][socket connect:+]]...p0f [[check command:+][socket connect:N/A]]...nscd [[check command:+][socket connect:N/A]]...named [[check command:+][socket connect:N/A]]...mysql [[check command:+][socket connect:N/A]]...mailman [[check command:+][socket connect:N/A]]...lfd [[check command:+][socket connect:N/A]]...ipaliases [[check command:+][socket connect:N/A]]...imap [[socket_service_auth:1][check command:+][socket connect:+]]...httpd [[check command:N/A][socket connect:+]]...ftpd [[socket_service_auth:1][check command:+][socket connect:+]]...exim [[check command:+][socket connect:+]]...entropychat [[check command:N/A][socket connect:N/A]]...dnsadmin [[check command:+][socket connect:+]]...crond [[check command:+][socket connect:N/A]]...cpsrvd [[http_service_auth:1][check command:N/A][socket connect:+]]...cphulkd [[check command:+][socket connect:+]]...cpgreylistd [[check command:N/A][socket connect:N/A]]...cpdavd [[http_service_auth:1][check command:+][socket connect:+]]...cpanellogd [[check command:+][socket connect:N/A]]...clamd [[check command:+][socket connect:N/A]]...Done
[2016-01-25 22:09:04 -0500] OOM check ....Done
[2016-01-25 22:09:04 -0500] Disk check .... simfs (/) [10.68%] ... {status:eek:k} ... Done
Loading services .....clamd....cpanellogd....cpdavd....cphulkd....cpsrvd....crond....dnsadmin....exim....ftpd....httpd....imap....ipaliases....lfd....mailman....mysql....named....nscd....p0f....pop....queueprocd....rsyslogd....spamd....sshd..Done
Service Check Started
Service Check Finished
Preview of “cpanel_tailwatchd_log_tail.txt”
[10647] [2016-01-25 13:22:26 -0500] [Cpanel::TailWatch] [START] 10647 1453746145
[10633] [2016-01-25 13:22:25 -0500] [Cpanel::TailWatch] The tailwatchd driver 'Cpanel::TailWatch::JailManager' is not enabled.
[10633] [2016-01-25 13:22:25 -0500] [Cpanel::TailWatch::Eximstats] Loading email sending limits from 1453744800 - 1453748400
[10633] [2016-01-25 13:22:25 -0500] [Cpanel::TailWatch] [INFO] Configured to conserve memory; skipped load of Linux::Inotify2.
[10633] [2016-01-25 13:22:25 -0500] [Cpanel::TailWatch] [INFO] Opened /usr/local/cpanel/logs/tailwatchd_log in append mode
[2060] [2016-01-25 13:22:24 -0500] [Cpanel::TailWatch] tailwatch exiting on SIGTERM
[2060] [2016-01-25 13:22:24 -0500] [Cpanel::TailWatch] [2060] [2016-01-25 13:22:24 -0500] [Cpanel::TailWatch] tailwatch exiting on SIGTERM
[10619] [2016-01-25 13:22:24 -0500] [Cpanel::TailWatch] The tailwatchd driver 'Cpanel::TailWatch::JailManager' is not enabled.
[10619] [2016-01-25 13:22:24 -0500] [Cpanel::TailWatch::Eximstats] Loading email sending limits from 1453744800 - 1453748400
[10619] [2016-01-25 13:22:24 -0500] [Cpanel::TailWatch] [INFO] Configured to conserve memory; skipped load of Linux::Inotify2.

 

[cPanelMichael]

Load Information 23.16 15.08 9.10

Hello

The issue likely stems from the load average on the system. The following thread will guide you through troubleshooting high loads:

Troubleshooting high server loads on Linux servers

Thank you.

 

[venomdog]

Thanks Michael. And sorry to reply you late, that threads seems to be too much for me to investigate, it talks about a script that will again give me some pointers. But for me this issue occurs every 3rd day and it effects my site, luckily I have also used cloudflare so I am online always, what I do is I restart my VPS and then it all works fine until I hit this issue again on 3rd day precisely.

Any other pointers ? I also wonder why mysql process takes 22.9% of cpu when this occurs.

 

[cPanelMichael]

The thread also references some log files you can check to verify the cause of the issue. You may want to consult with your provider or a qualified system administrator if this beyond the scope of your abilities.

Thank you.