SOLVED The service cphulkd appears to be down

[Ryan Hartwick]

Hello Everyone,

First time posting in these forums. We're getting the following message on a daily basis it seems. Normally a message follows immediately afterwards that the service has recovered. I'm not a Cpanel expert but I've tried looking through the logs but I'm not finding anything that tells me what's going on. This could just be because I don't know what I'm looking for though.

Time of the message seems to vary, normally the late evening or early morning:

The service “cphulkd” appears to be down.


Server <hiding this>
Primary IP Address <hiding this>
Service Name cphulkd
Service Status failed [IMG]
Notification The service “cphulkd” appears to be down.
Service Check Method The system’s command to check or to restart this service failed.
Number of Restart Attempts 1
Service Check Raw Output (XID muvqw6) The “cphulkd” service is down.

The subprocess “/usr/local/cpanel/scripts/restartsrv_cphulkd” reported error number 3 when it ended.
Startup Log Apr 06 00:15:02 <hiding this> systemd[1]: Starting cPanel brute force detector services...
Apr 06 00:15:02 <hiding this> systemd[1]: PID file /var/run/cphulkd_processor.pid not readable (yet?) after start.
Apr 06 00:15:02 <hiding this> systemd[1]: Started cPanel brute force detector services.
Apr 06 01:40:02 <hiding this> systemd[1]: cphulkd.service: main process exited, code=killed, status=14/ALRM
Apr 06 01:40:02 <hiding this> systemd[1]: Unit cphulkd.service entered failed state.
Apr 06 01:40:02 <hiding this> systemd[1]: cphulkd.service failed.
Log Messages The system could not provide log messages for “cphulkd” because it failed to read all of the potential log files with the following errors: Error while attempting to read “/var/log/messages”: “Empty File”, Error while attempting to read “/var/log/secure”: “Empty File”, Error while attempting to read “/var/log/maillog”: “Empty File”
Memory Information
Used 2.66 GB
Available 14.96 GB
Installed 16 GB
Load Information 0.08 0.02 0.01
Uptime 9 days, 14 hours, 41 minutes, and 31 seconds
Top Processes
PID Owner CPU % Memory % Command
8401 dovecot 0.15 0.02 dovecot/auth
3403 root 0.08 0.63 /usr/local/cpanel/3rdparty/perl/528/bin/perl -T -w /usr/local/cpanel/3rdparty/bin/spamd --max-spare=1 --max-children=3 --allowed-ips=127.0.0.1,::1 --pidfile=/var/run/spamd.pid --listen=5
1255 cpanelsolr 0.07 1.83 /usr/lib/jvm/jre-1.8.0/bin/java -server -Xms512m -Xmx512m -XX:NewRatio=3 -XX:SurvivorRatio=4 -XX:TargetSurvivorRatio=90 -XX:MaxTenuringThreshold=8 -XX:+UseConcMarkSweepGC -XX:+UseParNewGC -XX:ConcGCThreads=4 -XX:ParallelGCThreads=4 -XX:+CMSScavengeBeforeRemark -XX:PretenureSizeThreshold=64m -XX:+UseCMSInitiatingOccupancyOnly -XX:CMSInitiatingOccupancyFraction=50 -XX:CMSMaxAbortablePrecleanTime=6000 -XX:+CMSParallelRemarkEnabled -XX:+ParallelRefProcEnabled -XX:-OmitStackTraceInFastThrow -verbose:gc -XX:+PrintHeapAtGC -XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+PrintGCTimeStamps -XX:+PrintTenuringDistribution -XX:+PrintGCApplicationStoppedTime -Xloggc:/home/cpanelsolr/server/logs/solr_gc.log -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=9 -XX:GCLogFileSize=20M -Dsolr.log.dir=/home/cpanelsolr/server/logs -Djetty.port=8984 -DSTOP.PORT=7984 -Dhost=127.0.0.1 -Duser.timezone=UTC -Djetty.home=/home/cpanelsolr/server -Dsolr.solr.home=/home/cpanelsolr/server/solr -Dsolr.install.dir=/home/cpanelsolr -Xss256k -Dsolr.autoSoftCommit.maxTime=3000 -Dsolr.log.muteconsole -XX:OnOutOfMemoryError=/home/cpanelsolr/bin/oom_solr.sh 8984 /home/cpanelsolr/server/logs -jar start.jar --module=http
516 mysql 0.05 0.54 /usr/sbin/mysqld
3815 root 0.04 0.61 spamd child


The chkservd process attempts to connect to “127.0.0.1:579” in order to validate that this service is functioning. If you blocked connections with iptables or the “Host Access Control” interface in WHM, this failure may be a false positive.

To resolve this issue, either open the firewall to allow connections as the root user to “127.0.0.1:579” or disable checks for this service in WHM’s “Service Manager” interface with the “Configure Monitor Settings” link below.

Configure Monitor Settings:
<hiding this>

Configure chkservd:
<hiding this>

Disable HTML notifications:
<hiding this>

The system generated this notice on Saturday, April 6, 2019 at 5:45:24 AM UTC.

“Service failures (ChkServd)” notifications are currently configured to have an importance of “High”. You can change the importance or disable this type of notification in WHM’s Contact Manager at:
<hiding this>

Do not reply to this automated message.

 

[cPanelMichael]

Hello @Ryan Hartwick,

It's possible this relates to the issue described on the following thread:

[CPANEL-24832] Multiple service failure reports for cpsrvd, IMAP, and Exim

Can you verify if disabling cpsrvd and cphulkd in the Dormant services section under the Software tab in WHM >> Tweak Settings addresses the issue?

Thank you.

 

[Ryan Hartwick]

Thanks Michael!

I'll give that a try and report back my findings.

 

[Ryan Hartwick]

Hello everyone,

Just thought I'd update my findings after a week or so of monitoring and this appears to have been fixed. A big thanks for your assistance Michael on figuring this one out. My faith in user forums has been restored

 

[cPanelMichael]

Hello Ryan,

I'm glad to see the workaround from the linked thread helped. Thanks for letting us know!