SOLVED [CPANEL-21312] The cpgreylistd service is down

martin MHC

Well-Known Member
Sep 14, 2016
216
35
78
UK
cPanel Access Level
Root Administrator
Over the last few days on WHM 70.0.48 I've been getting daily reports that :

================
(XID ewurwr) The “cpgreylistd” service is down.

The subprocess “/usr/local/cpanel/scripts/restartsrv_cpgreylistd” reported error number 255 when it ended.

Startup Log [*] cPGreyList is running with PID: '38473'
[+] Successfully sent 'HUP' signal to daemon.

This happens at the same time every night; 11pm (+/- 5 minutes) and for approximately 5 minutes until I get a second notice that CPGreylist has restarted.

================

It's not a big deal in itself; but it's persistently every night. This hadn't occured previously.

I have looked at various logs but can't find reference to "cPGreyList" so clarification on which log file to explore would also be good. I have checked for a Checkservd.log (see here) but there is none.

I am curious if reinstalling greylist would help, however I'm not overly familiar with yum and can't find google results as to best ways of running yum to reinstall (rather than fresh install or update) greylist.

I would like some advice on:

1) How can I find out what specifically causes this issue? (error 255 sounds like a generic fail)

2) If possible, how do I solve this greylist downtime?

3) If nessecary, how would I correctly re-install Greylist on my server?

Many thanks !
 

cPanelLauren

Product Owner
Staff member
Nov 14, 2017
13,296
1,271
313
Houston
Hi @martin MHC

It sounds like it's staying unresponsive to chkservd checks of the service. Can you run the following via SSH I'd like to see how long the process has been running:
Code:
 ps faux |grep cpgreylist[d]
 

martin MHC

Well-Known Member
Sep 14, 2016
216
35
78
UK
cPanel Access Level
Root Administrator
Hi @martin MHC

It sounds like it's staying unresponsive to chkservd checks of the service. Can you run the following via SSH I'd like to see how long the process has been running:
Code:
 ps faux |grep cpgreylist[d]
Hello Lauren,

The output at 5:19pm my time is:

root 58944 0.0 0.1 36680 4708 ? S Jun06 0:08 cpgreylistd - processor

Just incase it relates, since asking the question I did find the WHM --> Software --> Rebuild RPM Database and run that earlier today. (Not sure if it would make a difference but thought it was worth a shot)

Cheers
 

cPanelLauren

Product Owner
Staff member
Nov 14, 2017
13,296
1,271
313
Houston
Hi @martin MHC

It doesn't look like it's been running too long since 12am yesterday. If you stop it with the following:

Code:
/scripts/restartsrv_cpgreylistd --stop
Ensure no further cpgreylistd processes are running by running the earlier command again:
Code:
ps faux |grep cpgreylist[d]
Then restart it by running:

Code:
/scripts/restartsrv_cpgreylistd --start
Can you let me know if the issue persists?

Thanks!
 
  • Like
Reactions: martin MHC

martin MHC

Well-Known Member
Sep 14, 2016
216
35
78
UK
cPanel Access Level
Root Administrator
Hello @cPanelLauren .
Thanks for the advice.

Everything ran as expected; output below just for clarity:

(I will let you know tomorrow if the issue returns tonight)

In the meantime how would I go about establishing what caused this?

[email protected] [~]# /scripts/restartsrv_cpgreylistd --stop
Waiting for “cpgreylistd” to stop ………finished.

Startup Log
[+] No running cPGreyList process found.
[*] Starting cPGreyListd ...
[*] Found cPGreyList running with PID: '58944'. Stopping...
[+] cPGreyList stopped successfully.

cpgreylistd stopped successfully.
[email protected] [~]# ps faux |grep cpgreylist[d]
[email protected] [~]# /scripts/restartsrv_cpgreylistd --start
Waiting for “cpgreylistd” to start ……waiting for “cpgreylistd” to initialize ……… finished.

Service Status
cpgreylistd (/usr/local/cpanel/libexec/cpgreylistd --restart) running as root with PID 42952 (process table check method)

Startup Log
[+] No running cPGreyList process found.

cpgreylistd started successfully.
 
Last edited:

cPanelLauren

Product Owner
Staff member
Nov 14, 2017
13,296
1,271
313
Houston
HI @martin MHC

Great, now will you let us know if you continue to receive the same notifications that it failed to restart? I'd give it until tomorrow to be sure.

Thanks!
 

martin MHC

Well-Known Member
Sep 14, 2016
216
35
78
UK
cPanel Access Level
Root Administrator
Hi @martin MHC
I'm so happy to hear that! I'm glad I could help!

Thanks!
Hello @cPanelLauren -- unfortunately the issue has returned again; exactly the same as before! Nothng else has been changed as far as I am aware!

I have run the commands given above to restart the service but would like to find out what's causing it to break at 11pm every night?!

I guess this may need to be a ticket?
 
Last edited:

martin MHC

Well-Known Member
Sep 14, 2016
216
35
78
UK
cPanel Access Level
Root Administrator
HI @martin MHC

Is there anything logged to
Code:
/usr/local/cpanel/logs/cpgreylistd.log
Hello
The only vaguely interesting notes in that file are pretty minor:

Code:
    [2018-06-08 22:57:18 +0100] info [cpgreylistd] SIGHUP received: re-execing daemon 
    [2018-06-08 22:57:18 +0100] info [cpgreylistd] cPGreyListd Processor startup with PID '27551'

    [2018-06-09 22:57:20 +0100] info [cpgreylistd] SIGHUP received: re-execing daemon
    [2018-06-09 22:59:07 +0100] info [cpgreylistd] cPGreyListd Processor startup with PID '50033'

    [2018-06-10 22:57:31 +0100] info [cpgreylistd] SIGHUP received: re-execing daemon
    [2018-06-10 23:02:13 +0100] info [cpgreylistd] cPGreyListd Processor startup with PID '58621'
PID 58621 is currently running ok. Another notice of the above was received again this morning....
 

cPanelLauren

Product Owner
Staff member
Nov 14, 2017
13,296
1,271
313
Houston
Hi @martin MHC

It looks like the same issue is persisting, for some reason chkservd isn't seeing the greylistd service restart within the allotted time frame. There's no issue with it's restarting it's just not doing it before chkservd times out. You can probably confirm this by checking /var/log/chkservd.log for restarts of the cpgreylistd service.

It may be useful to open a ticket at this point so we can look at the configuration closer and determine why it's not being recognized by chkservd.

Thanks!
 

cPanelLauren

Product Owner
Staff member
Nov 14, 2017
13,296
1,271
313
Houston
Hi @martin MHC

No worries! I just checked in on the ticket and it looks like it's already being worked on by one of our analysts. I'll update here when there's more information!

Thanks!
 

martin MHC

Well-Known Member
Sep 14, 2016
216
35
78
UK
cPanel Access Level
Root Administrator
Hello @cPanelLauren and all, the problem appeared to be related to having a SELinux installation running on the server. Rather strangely the SELinux had been installed since 2015 and appeared to have run without issues iro WHM but recently this had caused backup and UPCP faults includng this one.

Having disabled SELinux the system seems to be running smoothly (touch wood!) over the last 2 days.
 

cPanelLauren

Product Owner
Staff member
Nov 14, 2017
13,296
1,271
313
Houston

cPanelLauren

Product Owner
Staff member
Nov 14, 2017
13,296
1,271
313
Houston
Hi @martin MHC

I wanted to add to this thread for anyone coming across this later, as a result of this thread we did end up opening a case CPANEL-21312 - cpgreylistd dies if it gets SIGHUP while starting MERGE this is currently being tested and I'll update here again once it's completed.

Thanks!
 
  • Like
Reactions: martin MHC

cPanelMichael

Administrator
Staff member
Apr 11, 2011
47,908
2,216
463
Hello,

To update, CPANEL-21312 is included with cPanel & WHM version 76:

Fixed case CPANEL-21312: Prevent cpgreylistd from being killed by SIGHUP when restarting.

The full change log is available at:

76 Change Log - Change Logs - cPanel Documentation

Thank you.


 
  • Like
Reactions: martin MHC