[Case 103893] Clamdscan Process Hanging with File Manager

GuruDavid

Member
Mar 25, 2014
5
1
3
cPanel Access Level
Root Administrator
To the tremendous team here at cPanel,

A number of us have noticed with the latest update [ cPanel Version: 11.44.0.17 ] an apparent issue with the File Manager tool within cPanel. If the ClamAV plugin is installed via WHM > Plugins, upon performing a file upload via cPanel's File Manager, an auto-scan of the file is performed, rather attempted. The issue appears that if the clamd services is not enabled the upload hangs or stalls, as you can see here:
--

Code:
servplus 30836 99.2  0.0   6260  1540 ?        R    19:33   3:46          \_ /usr/local/cpanel/3rdparty/bin/clamdscan --stdout --no-summary /home/servplus/tmp/Cpanel_Form_file.upload.sKbbsKASDlqb_PTu
--
Is there a way to disable this 'autoscan' feature?

Further, for the upload to actually succeed, the process must be killed, otherwise all uploads 'show as successful' however the process continues to hang and no file is actually uploaded.

Would it be possible to implement a check to ensure that clamd is enabled before the 'autoscan' is performed?

Any/all feedback is welcomed, thank you.
 

wlittle

Registered
Mar 3, 2014
3
0
1
cPanel Access Level
Root Administrator
Hi,

I can confirm this here on my side as well. Processes hang and load spikes tremendously. The "solution" here is to pkill clamdscan and enable clamd service or uninstall the WHM ClamAV plugin.

I disagree with users having to enable clamd in conjunction with the WHM ClamAV plugin as this plugin alone still provides functionality to the user. I also disagree with uninstalling the WHM ClamAV plugin and re-installing via the package manager, as this is not optimal/efficient. The WHM ClamAV plugin allows for finer control, and users may simply "flip" the clamd service "switch" for more functions. If ClamAV is installed via the package manager and the user wishes to add more functionality, the user must re-install back to the plugin.

Switching between the two installations is not only inefficient, but would also imply there is no functional use of the WHM ClamAV plugin without clamd service enabled, so this should be auto-enabled upon plugin installation, which I disagree with.

Requested Solutions:
-----
1) Add clamd service status check prior to scan to avoid upload hangs
2) Add tweak setting to enable/disable auto-scanning of cPanel > File Manager uploads.

Thanks,
 

cPanelMichael

Administrator
Staff member
Apr 11, 2011
47,910
2,213
363
Hello :)

Allow me to quote some information from internal case number 103893, which is open to address this issue.

Internal case number 103893 is open to address the issue where if ClamAV is installed but clamd is disabled in Service Manager, file uploads via File Manager result in a hung clamdscan process and the upload doesn't complete. This hung clamdscan process does not occur if ClamAV is not installed *or* if it's installed and clamd is enabled in Service Manager.
Known Workarounds:

1) Enable clamd in WHM -> Service Manager
2) Uninstall ClamAV in WHM -> Manage Plugins
A resolution is scheduled for inclusion in a future build of cPanel. Please monitor the change log for this case number to see when it's been released:

11.44 - Change Log

Thank you.
 

dpearson

Registered
Apr 9, 2013
2
0
1
cPanel Access Level
DataCenter Provider
Hello :)

Allow me to quote some information from internal case number 103893, which is open to address this issue.


Thank you.
If you would this would be a great thing to push to the top of the priority list. We've got thousands of cPanel VPS's deployed and most of them have Clamd disabled in WHM so At the moment I'm constantly scanning and killing hung clamdscan processes to keep things in check.

The sooner you can get that bug patched the better.
 

WiredChris

Member
Aug 7, 2013
8
0
1
cPanel Access Level
DataCenter Provider
Here's a one liner which resolves this issue:

Code:
ps axu | grep clamdscan | grep -v grep | awk '{print $2}' | xargs kill; rm -f /etc/clamddisable; /scripts/restartsrv_clamd;
It wouldn't be so bad if it just broke the file manager but the load issues it's causing it a real headache. I hope it's fixed soon.
 

Vandalite

Registered
May 7, 2008
3
0
51
Does this also affect clamdscan when the 'clamd' service is enabled? I have a slightly different although just as disturbing set of symptoms:

Code:
 486855 root      20   0 35884 1336 1088 R 100.0  0.0  18:03.54 /usr/local/cpanel/3rdparty/bin/clamdscan --quiet --no-summary /etc/passwd
 380692 root      20   0 35884 1340 1088 R 97.1  0.0  52:28.64 /usr/local/cpanel/3rdparty/bin/clamdscan --quiet --no-summary /etc/passwd
 513546 root      20   0 35884 1336 1088 R 92.2  0.0   8:27.00 /usr/local/cpanel/3rdparty/bin/clamdscan --quiet --no-summary /etc/passwd
 329637 root      20   0 35884 1336 1088 R 77.7  0.0  62:22.67 /usr/local/cpanel/3rdparty/bin/clamdscan --quiet --no-summary /etc/passwd
Or, as it appears in ps axjf:

Code:
     1  329634  621170  502009 ?             -1 S        0   0:00 /usr/local/cpanel/3rdparty/bin/perl /usr/local/cpanel/scripts/restartsrv_clamd --check
 329634  329637  621170  502009 ?             -1 R        0  63:23  \_ /usr/local/cpanel/3rdparty/bin/clamdscan --quiet --no-summary /etc/passwd
      1  380686  621170  502009 ?             -1 S        0   0:00 /usr/local/cpanel/3rdparty/bin/perl /usr/local/cpanel/scripts/restartsrv_clamd --check
 380686  380692  621170  502009 ?             -1 R        0  53:29  \_ /usr/local/cpanel/3rdparty/bin/clamdscan --quiet --no-summary /etc/passwd
      1  486848  621170  502009 ?             -1 S        0   0:00 /usr/local/cpanel/3rdparty/bin/perl /usr/local/cpanel/scripts/restartsrv_clamd --check
 486848  486855  621170  502009 ?             -1 R        0  19:04  \_ /usr/local/cpanel/3rdparty/bin/clamdscan --quiet --no-summary /etc/passwd
      1  513518  621170  502009 ?             -1 S        0   0:00 /usr/local/cpanel/3rdparty/bin/perl /usr/local/cpanel/scripts/restartsrv_clamd --check
 513518  513546  621170  502009 ?             -1 R        0   9:27  \_ /usr/local/cpanel/3rdparty/bin/clamdscan --quiet --no-summary /etc/passwd
We're not certain here what's causing this but if left unchecked, it starts eating CPU power... all of it. Linux is generally smart enough to task-switch around this, but unlike the clamav resident scanner, it doesn't seem to honor process nice levels.

Any idea what's going on here?
 

Vandalite

Registered
May 7, 2008
3
0
51
I've noticed something similar but not quite identical occurring on a few of our servers here.

First, a snippet of ps axjf:

Code:
     1  329634  621170  502009 ?             -1 S        0   0:00 /usr/local/cpanel/3rdparty/bin/perl /usr/local/cpanel/scripts/restartsrv_clamd --check
 329634  329637  621170  502009 ?             -1 R        0  63:23  \_ /usr/local/cpanel/3rdparty/bin/clamdscan --quiet --no-summary /etc/passwd
      1  380686  621170  502009 ?             -1 S        0   0:00 /usr/local/cpanel/3rdparty/bin/perl /usr/local/cpanel/scripts/restartsrv_clamd --check
 380686  380692  621170  502009 ?             -1 R        0  53:29  \_ /usr/local/cpanel/3rdparty/bin/clamdscan --quiet --no-summary /etc/passwd
      1  486848  621170  502009 ?             -1 S        0   0:00 /usr/local/cpanel/3rdparty/bin/perl /usr/local/cpanel/scripts/restartsrv_clamd --check
 486848  486855  621170  502009 ?             -1 R        0  19:04  \_ /usr/local/cpanel/3rdparty/bin/clamdscan --quiet --no-summary /etc/passwd
      1  513518  621170  502009 ?             -1 S        0   0:00 /usr/local/cpanel/3rdparty/bin/perl /usr/local/cpanel/scripts/restartsrv_clamd --check
 513518  513546  621170  502009 ?             -1 R        0   9:27  \_ /usr/local/cpanel/3rdparty/bin/clamdscan --quiet --no-summary /etc/passwd
And then a view from top:

Code:
 380692 root      20   0 35884 1340 1088 R 99.3  0.0  60:17.85 /usr/local/cpanel/3rdparty/bin/clamdscan --quiet --no-summary /etc/passwd
 486855 root      20   0 35884 1336 1088 R 99.3  0.0  25:52.77 /usr/local/cpanel/3rdparty/bin/clamdscan --quiet --no-summary /etc/passwd
 329637 root      20   0 35884 1336 1088 R 97.4  0.0  70:12.59 /usr/local/cpanel/3rdparty/bin/clamdscan --quiet --no-summary /etc/passwd
 513546 root      20   0 35884 1336 1088 R 93.8  0.0  16:16.24 /usr/local/cpanel/3rdparty/bin/clamdscan --quiet --no-summary /etc/passwd
This was captured from one of our servers with the clamav software installed from the 'manage plugins' section of WHM, and with the 'clamd' service enabled in our service manager.

It's using a very high amount of CPU, and we really can't tell what it's doing with it all, and that has us concerned. Is this related? Will it get fixed with that planned update or should this be a different thread?
 

kisonay

Member
Mar 21, 2013
20
0
1
cPanel Access Level
Root Administrator
root spawning clamdscan which is consuming all resources

For the past week or so I have been getting High Load alerts via email from my WHM VPS. While investigating it appears root keeps on spawning clamdscans which consume a high amount of CPU.

This happens almost hourly, sometimes it is worst than others.

Has anyone experienced this before? How can I determine what is spawning these via root and how can I stop it from happening?

Code:
Pid	Owner	Priority	CPU %	Memory %	Command
9692 (Trace) (Kill)	root	-5	  53.9	0.0	/usr/local/cpanel/3rdparty/bin/clamdscan --quiet --no-summary /etc/passwd
3004 (Trace) (Kill)	root	-5	  43.1	0.0	/usr/local/cpanel/3rdparty/bin/clamdscan --quiet --no-summary /etc/passwd
11609 (Trace) (Kill)	root	-5	  39.1	0.0	/usr/local/cpanel/3rdparty/bin/clamdscan --quiet --no-summary /etc/passwd
30857 (Trace) (Kill)	root	-5	  36.2	0.0	/usr/local/cpanel/3rdparty/bin/clamdscan --quiet --no-summary /etc/passwd
5215 (Trace) (Kill)	root	-5	  32.6	0.0	/usr/local/cpanel/3rdparty/bin/clamdscan --quiet --no-summary /etc/passwd
15655 (Trace) (Kill)	root	-5	  24.8	0.1	/usr/local/cpanel/3rdparty/bin/clamdscan --quiet --no-summary /etc/passwd
CENTOS 6.5 x86_64 xenpv – s1 WHM 11.44.0 (build 18)
 

wlittle

Registered
Mar 3, 2014
3
0
1
cPanel Access Level
Root Administrator
I've noticed something similar but not quite identical occurring on a few of our servers here.
Your output and symptoms look similar to the issue we are seeing, but you say that clamd is enabled in WHM > Manage Services? Are you sure that you killed the hung clamdscan processes as indicated above? You may try WiredChris' command above. You may also kill off those hung clamd restart processes as well.

Thanks,
 

Vandalite

Registered
May 7, 2008
3
0
51
Your output and symptoms look similar to the issue we are seeing, but you say that clamd is enabled in WHM > Manage Services? Are you sure that you killed the hung clamdscan processes as indicated above? You may try WiredChris' command above. You may also kill off those hung clamd restart processes as well.

Thanks,
I can confirm clamd is definitely enabled on our service manager in WHM.

That command from above does three things:
ps axu | grep clamdscan | grep -v grep | awk '{print $2}' | xargs kill; <-- does the killing of the hung processes
rm -f /etc/clamddisable; <-- tries to remove a file that isn't on our server.
/scripts/restartsrv_clamd; <-- spawns a new hung thread almost immediately.

At least for us, it's the actual attempt to restart clamd using that restartsrv_clamd or even the --check version that triggers another copy of this clamdscan thread to run, and it just sits there, using the full power of a CPU core for no apparent reason.
 

st0rm

Member
Jun 27, 2014
12
0
1
cPanel Access Level
Root Administrator
Problem with clamdscan

I have been getting a lot of heavy load reports from my servers , and by checking i found out that clamav cpanel plugin is consuming a LOT of resources scanning /etc/password , spawning a lot of processes for the same purpose which is scanning /etc/password .. what seems to be the problem ?
 

Del Drago

Member
Mar 2, 2012
12
1
53
cPanel Access Level
Root Administrator
Re: root spawning clamdscan which is consuming all resources

For the past week or so I have been getting High Load alerts via email from my WHM VPS. While investigating it appears root keeps on spawning clamdscans which consume a high amount of CPU.
I'm having the exact same issue. Is there a fix in the works?
 

cPanelMichael

Administrator
Staff member
Apr 11, 2011
47,910
2,213
363
Hello :)

This issue is addressed in cPanel version 11.44.0.19:

Fixed case 103893: Update cpanel-clamav to 0.98.4-1.cp1140.

Feel free to update cPanel to this version if you do not have automatic updates enabled and let us know if the issue persists.

Thank you.