chkservd and supervisord

[ExtremeGuy]

I am having an issue with running supervisord as a service monitored by chkservd. The chkservd service keeps restarting the service every iteration as the check appears to be failing, but I don't know why

The chkservd log file is reporting the following

supervisord [[check command:-][socket connect:N/A][fail count:19]Restarting supervisord....

My chkserv.d file located at /etc/chkserv.d/supervisord

service[supervisord]=9123,GET / HTTP/1.0,HTTP/1..,service supervisord restart,supervisord,supervisord

and my supervisor configuration file (commented entries removed for brevity)

[supervisord]
;http_port=/var/tmp/supervisor.sock ; (default is to run a UNIX domain socket server)
http_port=127.0.0.1:9123 ; (alternately, ip_address:port specifies AF_INET)

logfile=/var/log/supervisor/supervisord.log ; (main log file;default $CWD/supervisord.log)
logfile_maxbytes=50MB ; (max main logfile bytes b4 rotation;default 50MB)
logfile_backups=10 ; (num of main logfile rotation backups;default 10)
loglevel=info ; (logging level;default info; others: debug,warn)
pidfile=/var/run/supervisord.pid ; (supervisord pidfile;default supervisord.pid)
nodaemon=false ; (start in foreground if true;default false)
minfds=1024 ; (min. avail startup file descriptors;default 1024)
minprocs=200 ; (min. avail process descriptors;default 200)

[supervisorctl]
;serverurl=unix:///var/tmp/supervisor.sock ; use a unix:// URL for a unix socket
serverurl=http://127.0.0.1:9123 ; use an http:// url to specify an inet socket


I changed it to an http service rather than a socket to see if I can address but it has not worked. My original chksrvd file for supervisord was, as per Cpanel docs

service[supervisord]=x,x,x,service supervisord restart,supervisord,supervisord

I understand that supervisord is an unsupported service, but I would really like to know if I have anything incorrectly configured that would cause chksrvd to be constantly failing to read the service status.

Jason

 

[cPanelMichael]

Hello Jason,

Can you confirm which Operating System and cPanel & WHM versions are installed on the system so we can attempt to reproduce the issue?

Thank you.

 

[ExtremeGuy]

Hello Jason,

Can you confirm which Operating System and cPanel & WHM versions are installed on the system so we can attempt to reproduce the issue?

Thank you.

The server is a VPS running CENTOS 6.10 with WHM v76.0.14. supervisor is currently running version 2.1

 

[cPanelMichael]

Hello @ExtremeGuy,

I tested this using the steps documented on the link below for process-based monitoring:

TailWatch - cPanel Knowledge Base - cPanel Documentation

# grep supervisord /etc/chkserv.d/chkservd.conf
supervisord:1

# cat /etc/chkserv.d/supervisord
service[supervisord]=x,x,x,service supervisord restart,supervisord,root

Once Chkservd was setup, I killed the supervisord process with the following command:

# killall -9 supervisord
# ps aux|grep supervisord
root      9229  0.0  0.0 112708   980 pts/1    S+   12:33   0:00 grep --color=auto supervisord

Upon the next Chkservd service check, /var/log/chkservd.log showed supervisord was detected as down and successfully restarted:

supervisord [[check command:-][socket connect:N/A][fail count:1]Restarting supervisord....
system: service supervisord restart
[notify:failed service:supervisord]]..

# ps aux|grep supervisord
root      9282  0.0  0.6 219408 11880 ?        Ss   12:35   0:00 /usr/bin/python /usr/bin/supervisord -c /etc/supervisord.conf
root      9366  0.0  0.0 112708   980 pts/1    S+   12:36   0:00 grep --color=auto supervisord

Can you let me know if the issue persists on your system using these same steps and file contents?

Thank you.