Please whitelist cPanel in your adblocker so that you’re able to see our version release promotions, thanks!

The Community Forums

Interact with an entire community of cPanel & WHM users!

The service “cpanel-dovecot-solr”/"clamd" appears to be down

Discussion in 'E-mail Discussion' started by Olufemi Lawal, Oct 25, 2018.

  1. Olufemi Lawal

    Olufemi Lawal Registered

    Joined:
    Oct 25, 2018
    Messages:
    3
    Likes Received:
    0
    Trophy Points:
    1
    Location:
    Beijing, China
    cPanel Access Level:
    Root Administrator
    I've been receiving notification emails saying that these services are down and then after about 4 minutes I receive emails sayings: The service “cpanel-dovecot-solr”/"clamd" is now operational.

    Here's some more information:

    For Clamd:
    Service Name
    clamd
    Service Status failed
    Notification The service “clamd” appears to be down.
    Service Check Method The system’s command to check or to restart this service failed.
    Number of Restart Attempts 1
    Service Check Raw Output (XID 34q7h5) The “clamd” service is down.

    The subprocess “/usr/local/cpanel/scripts/restartsrv_clamd” reported error number 255 when it ended.
    Startup Log No startup log
    Memory Information
    Used 2.52 GB
    Available 8.48 GB
    Installed 11 GB
    Load Information 3.33 4.36 2.67
    Uptime 91 days, 9 hours, 43 minutes, and 31 seconds
    IOStat Information avg-cpu: %user %nice %system %iowait %steal %idle 1.95 0.01 0.17 0.01 0.00 97.86 Device: tps Blk_read/s Blk_wrtn/s Blk_read Blk_wrtn

    For Dovecot_solr:
    Number of Restart Attempts
    1
    Service Check Raw Output (XID nu9p7p) The “cpanel_dovecot_solr” service is down.

    The subprocess “/usr/local/cpanel/scripts/restartsrv_cpanel_dovecot_solr” reported error number 255 when it ended.
    Startup Log No startup log
    Memory Information
    Used 2.08 GB
    Available 8.92 GB
    Installed 11 GB
    Load Information 2.96 4.43 2.63
    Uptime 91 days, 9 hours, 42 minutes, and 53 seconds
    IOStat Information avg-cpu: %user %nice %system %iowait %steal %idle 1.95 0.01 0.17 0.01 0.00 97.86 Device: tps Blk_read/s Blk_wrtn/s Blk_read Blk_wrtn

    I'm not sure if this is associated but we've also been receiving spam emails that falsely appear to have been send from an email address on our server.

    Any help on how best to proceed will be very much appreciated.
     
  2. GOT

    GOT Get Proactive! PartnerNOC

    Joined:
    Apr 8, 2003
    Messages:
    1,484
    Likes Received:
    187
    Trophy Points:
    193
    Location:
    Chesapeake, VA
    cPanel Access Level:
    DataCenter Provider
    Hard to say with any certainty. These are both mail related services and if you are having a massive spam outbreak as you post implies they could be crashing under load.

    How big is your mail queue?

    Might look in /var/log/maillig and messages around the time of the notice to see if theres any messages about crashes or terminated processes
     
    Stop hovering to collapse... Click to collapse... Hover to expand... Click to expand...
  3. Olufemi Lawal

    Olufemi Lawal Registered

    Joined:
    Oct 25, 2018
    Messages:
    3
    Likes Received:
    0
    Trophy Points:
    1
    Location:
    Beijing, China
    cPanel Access Level:
    Root Administrator
    Thanks for the reply GOT.

    I looked into the maillog and and then exim_paniclog and saw a lot of OOM crash message. The processes spamd, clamd and java were being killed due to a lack of memory.

    I'm looking into ways to limit the amount of RAM that ClamAV takes. There doesn't seem to be an easy fix. Any other anti-virus suggestions or other solutions?
     
  4. GOT

    GOT Get Proactive! PartnerNOC

    Joined:
    Apr 8, 2003
    Messages:
    1,484
    Likes Received:
    187
    Trophy Points:
    193
    Location:
    Chesapeake, VA
    cPanel Access Level:
    DataCenter Provider
    You post says you have 11 GB ram. That is an odd amount. And its unlikely that these services are actually what are taking up all your ram. I would start looking at your ram utilization in other areas. Prime suspects would be mysql, apahce and php typically.
     
    Stop hovering to collapse... Click to collapse... Hover to expand... Click to expand...
  5. cPanelLauren

    cPanelLauren Forums Analyst II Staff Member

    Joined:
    Nov 14, 2017
    Messages:
    6,464
    Likes Received:
    505
    Trophy Points:
    263
    Location:
    Houston
    cPanel Access Level:
    DataCenter Provider
    Hi @Olufemi


    Can you show me an example of the oom message? Are these present in just /var/log/maillog or in /var/log/messages as well?
     
    Stop hovering to collapse... Click to collapse... Hover to expand... Click to expand...
  6. Olufemi Lawal

    Olufemi Lawal Registered

    Joined:
    Oct 25, 2018
    Messages:
    3
    Likes Received:
    0
    Trophy Points:
    1
    Location:
    Beijing, China
    cPanel Access Level:
    Root Administrator
    Hi Lauren,

    I've only found these OOM messages in the /var/log/messages. Here is an excerpt.

    Code:
    Oct 24 04:52:57  PAM-hulk[5445]: Brute force detection active: 550 LOGIN DENIED -- TOO MANY FAILURES
    
    Oct 24 04:53:06 2 PAM-hulk[5445]: Brute force detection active: 550 LOGIN DENIED -- TOO MANY FAILURES
    
    Oct 24 05:38:44 : [7824298.758448] Out of memory in UB 125883: OOM killed process 9235 (clamd) score 0 vm:949668kB, rss:20452kB, swap:524776kB
    
    Oct 24 05:38:50 kernel: [7824304.760221] Out of memory in UB 125883: OOM killed process 9078 (java) score 0 vm:6212272kB, rss:61516kB, swap:286936kB
    
    Oct 24 05:38:53  kernel: [7824308.202905] Out of memory in UB 125883: OOM killed process 5944 (spamd child) score 0 vm:241512kB, rss:2740kB, swap:108428kB
    
    Oct 24 05:38:54 n kernel: [7824309.132341] Out of memory in UB 125883: OOM killed process 27727 (spamd) score 0 vm:223584kB, rss:4840kB, swap:93488kB
    
    Oct 24 05:38:55  kernel: [7824310.329803] Out of memory in UB 125883: OOM killed process 26635 (mysqld) score 0 vm:6794980kB, rss:34864kB, swap:59984kB
    
    Oct 24 05:38:57  kernel: [7824311.478852] Out of memory in UB 125883: OOM killed process 8123 (php) score 0 vm:246188kB, rss:45852kB, swap:28876kB
    
    Oct 24 05:39:03  kernel: [7824318.147747] Out of memory in UB 125883: OOM killed process 8166 (php) score 0 vm:247616kB, rss:41084kB, swap:36680kB
    
    Oct 24 05:39:05  kernel: [7824319.621564] Out of memory in UB 125883: OOM killed process 8190 (php) score 0 vm:246916kB, rss:48248kB, swap:29012kB
    
    Oct 24 05:39:08  kernel: [7824323.041045] Out of memory in UB 125883: OOM killed process 8155 (php) score 0 vm:246916kB, rss:41908kB, swap:35284kB
    
    Oct 24 05:39:12  kernel: [7824326.960922] Out of memory in UB 125883: OOM killed process 8185 (php) score 0 vm:247320kB, rss:53124kB, swap:24508kB
    
    Oct 24 05:39:14  kernel: [7824328.533322] Out of memory in UB 125883: OOM killed process 8186 (php) score 0 vm:246912kB, rss:50936kB, swap:26396kB
    
    Oct 24 05:39:15  kernel: [7824329.412481] Out of memory in UB 125883: OOM killed process 8184 (php) score 0 vm:244384kB, rss:45620kB, swap:29516kB
    
    Oct 24 05:52:46 PAM-hulk[9779]: Brute force detection active: 550 LOGIN DENIED -- TOO MANY FAILURES
    
    Oct 24 05:52:50 PAM-hulk[9787]: Brute force detection active: 550 LOGIN DENIED -- TOO MANY FAILURES
    
    Oct 24 05:52:53  PAM-hulk[9787]: Brute force detection active: 550 LOGIN DENIED -- TOO MANY FAILURES
    
    

    Here's an example of the /var/log/exim_paniclog entry:

    Code:
    2018-10-25 00:37:12 1gFaC9-0004zB-G7 spam acl condition: all spamd servers failed
    2018-10-25 00:37:33 1gFaCU-0004ze-UI spam acl condition: all spamd servers failed
    2018-10-25 00:37:52 1gFaCm-00051L-Gj spam acl condition: all spamd servers failed
    2018-10-25 00:37:54 1gFaDK-0004ze-1D malware acl condition: clamd /var/clamd : unable to connect to UNIX socket (/var/clamd): Connection refused
    2018-10-25 00:38:24 1gFaDK-0004ze-1D spam acl condition: all spamd servers failed
    2018-10-25 00:40:04 1gFaFP-0005C1-SN malware acl condition: clamd /var/clamd : unable to connect to UNIX socket (/var/clamd): Connection refused
    2018-10-25 00:40:34 1gFaFP-0005C1-SN spam acl condition: all spamd servers failed
    d
    Thanks for your help!
     
  7. cPanelLauren

    cPanelLauren Forums Analyst II Staff Member

    Joined:
    Nov 14, 2017
    Messages:
    6,464
    Likes Received:
    505
    Trophy Points:
    263
    Location:
    Houston
    cPanel Access Level:
    DataCenter Provider
    These do show you're running out of memory plain and simple:

    Code:
    Oct 24 05:39:12  kernel: [7824326.960922] Out of memory in UB 125883: OOM killed process 8185 (php) score 0 vm:247320kB, rss:53124kB, swap:24508kB
    
    Oct 24 05:39:14  kernel: [7824328.533322] Out of memory in UB 125883: OOM killed process 8186 (php) score 0 vm:246912kB, rss:50936kB, swap:26396kB
    
    Oct 24 05:39:15  kernel: [7824329.412481] Out of memory in UB 125883: OOM killed process 8184 (php) score 0 vm:244384kB, rss:45620kB, swap:29516kB
    
    I was hoping to see that this turned out to process memory limits but it's not - this is the system memory and you might want to check with your provider to see if there's anything they can do to assist you with this issue further.
     
    Stop hovering to collapse... Click to collapse... Hover to expand... Click to expand...
  8. behzad neissari

    behzad neissari Registered

    Joined:
    Feb 13, 2018
    Messages:
    1
    Likes Received:
    0
    Trophy Points:
    0
    Location:
    UAE
    cPanel Access Level:
    Root Administrator
    I update WHM in my centos server to CENTOS 7.6 andwhm 78.0.13 and after that all my email stay in Delivery Queue and cpanel-dovecot-solr is down and not start.

    i try to restart service and get following message:
    Code:
    [[email protected] ~]# systemctl status cpanel-dovecot-solr
    ● cpanel-dovecot-solr.service - Solr for cPanel Dovecot
    Loaded: loaded (/etc/systemd/system/cpanel-dovecot-solr.service; disabled; ve ndor preset: disabled)
    Active: inactive (dead)
    [[email protected] ~]# systemctl start cpanel-dovecot-solr
    [[email protected] ~]# systemctl status cpanel-dovecot-solr
    ● cpanel-dovecot-solr.service - Solr for cPanel Dovecot
    Loaded: loaded (/etc/systemd/system/cpanel-dovecot-solr.service; disabled; ve ndor preset: disabled)
    Active: failed (Result: exit-code) since Thu 2019-02-28 06:03:09 UTC; 4s ago
    Process: 20134 ExecStop=/home/cpanelsolr/bin/solr stop (code=exited, status=1/ FAILURE)
    Process: 19666 ExecStart=/home/cpanelsolr/bin/solr start -noprompt -h ${SOLR_H OSTNAME} -p ${SOLR_PORT} -m ${SOLR_MEMORY} -a ${SOLR_JVM_OPTS} (code=exited, sta tus=0/SUCCESS)
    Process: 19655 ExecStartPre=/usr/local/cpanel/3rdparty/scripts/cpanel_dovecot_ solr_firewall start (code=exited, status=0/SUCCESS)
    Main PID: 19861 (code=exited, status=143)
    
    Feb 28 06:02:39 centos.example.com systemd[1]: Starting Solr for cPanel Dov....
    Feb 28 06:02:50 centos.example.com solr[19666]: [194B blob data]
    Feb 28 06:02:50 centos.example.com solr[19666]: Started Solr server on port...!
    Feb 28 06:02:50 centos.example.com systemd[1]: Started Solr for cPanel Dovecot.
    Feb 28 06:03:09 centos.example.com systemd[1]: cpanel-dovecot-solr.service:...a
    Feb 28 06:03:09 centos.example.com systemd[1]: cpanel-dovecot-solr.service:...1
    Feb 28 06:03:09 centos.example.com systemd[1]: Unit cpanel-dovecot-solr.ser....
    Feb 28 06:03:09 centos.example.com systemd[1]: cpanel-dovecot-solr.service ....
    Hint: Some lines were ellipsized, use -l to show in full.
    
    Please help me
     
    #8 behzad neissari, Feb 28, 2019
    Last edited by a moderator: Feb 28, 2019
  9. GOT

    GOT Get Proactive! PartnerNOC

    Joined:
    Apr 8, 2003
    Messages:
    1,484
    Likes Received:
    187
    Trophy Points:
    193
    Location:
    Chesapeake, VA
    cPanel Access Level:
    DataCenter Provider
    I would first start with removing the Solr plug in. See if that resolves the issues and if so then you could try re-installing it.
     
    Stop hovering to collapse... Click to collapse... Hover to expand... Click to expand...
  10. cPanelLauren

    cPanelLauren Forums Analyst II Staff Member

    Joined:
    Nov 14, 2017
    Messages:
    6,464
    Likes Received:
    505
    Trophy Points:
    263
    Location:
    Houston
    cPanel Access Level:
    DataCenter Provider
    Hi @behzad neissari

    This doesn't look like there's an error it appears that Dovecot Solr is disabled:
    Code:
    Loaded: loaded (/etc/systemd/system/cpanel-dovecot-solr.service; disabled; ve ndor preset: disabled)
    
    You can enable it by going to WHM>>Service Configuration>>Service Manager -> Check Enabled + Monitor next to cpanel-dovecot-solr

    This is most likely not the cause of the mail queue issues, what's present in /var/log/exim_mainlog for the mail in the queue?


    Thanks!
     
    Stop hovering to collapse... Click to collapse... Hover to expand... Click to expand...
Loading...

Share This Page

  1. This site uses cookies to help personalise content, tailor your experience and to keep you logged in if you register.
    By continuing to use this site, you are consenting to our use of cookies.
    Dismiss Notice