Optimizing SpamAssassin for quickness/CPU-efficiency

[lbeachmike]

Hi there -

Has anybody tried to optimize SpamAssassin for processing speed?

I've got lots of RAM to work with and would like to optimize so emails are processed most efficiently.

Currently I have 12 max children and the default for Max Connections Per Child. I'm not sure if there are many moves to be made here, given that cpanel offers no means of changing the bulk of SpamAssassin default settings.

I see that my spam emails average about 12-14 seconds for processing. This is on an 8-core hyper-threaded box and on a 12-core hyper-threaded box!

Are there any productive changes I can make to improve upon that? I'd not mind throwing a few more gig of RAM at this, and I also wonder if there's a way to have SA make more efficient use of a multi-threaded machine similar to what PIGZ has done for compression.

Thanks.

Mike

 

[cPanelTristan]

You could try to modify settings for SpamAssassin in /var/lib/spamassassin/3.003001/updates_spamassassin_org directory. Most of the settings you have for SpamAssassin should be housed there.

I don't have suggestions for changes, but this is being mentioned per the "cpanel offers no means of changing the bulk of SpamAssassin default settings" comment.

 

[lbeachmike]

Thanks Tristan - I took a look inside that directory and there are a large number of files. It is not apparent to me as to how I would be able to modify a setting like min-children. In fact when I grep that directory for "child" or "Child" there are no matches.

Is modification of the content in this directory supported? I was told via ticket that the settings I am seeking to change are not supported.

Thanks.

 

[cPanelTristan]

We have an option in WHM > Manage Plugins that has "spamdconf" that can be enabled. When it is enabled, you can modify some of the settings for children as those settings appear in WHM > Plugins > Setup Spamd Startup Configuration area. There's no option for min-children, though.

As for modification of the directory being supported, it isn't supported but it is available. Any modifications outside WHM that cannot be done via scripts either are basically custom. We don't support customizations, but that doesn't mean customizations don't work either. That's why the "no means" isn't actually how I would have phrased it. There's a means, it just is going to require customizations. There are going to probably be startup values you cannot modify, yes.

 

[lbeachmike]

Hi Tristan - Thanks. I already do use the Spamd Startup Settings plugin, which only allows four options of a huge number of SpamAssassin configuration options, to be set.

Paul explained via ticket back in May why the additional options are not setable without modifying restartsrv.pl -

This option cannot be set. I looked at /usr/local/cpanel/Cpanel/RestartSrv.pm and cPanel is only reading certain setting which are set in a function there and the options that it reads are set at Main >> Spamd Startup Configuration, and get written out to /etc/cpspamd.conf, but only those options you can set are read, in other words you cannot set your own options and if you did they would be ignored by the restart code.

And then provided further details in a subsequent response -

Extra parameters cannot be set in a config file. *Some* parameters can, but only those that are parsed by Restartsrv.pm which is the module that /scripts/restartsrv_* use when doing a service restart:

sub getspamdopts {
my $cpspamdconf = '/etc/cpspamd.conf';
my $spamdoptions = '';
my $socketpath = '';
my $allowedips = '--allowed-ips=127.0.0.1';
my $maxconnperchild = '';
my $maxchildren = '--max-children=5';
my $maxspare = '';
my $pidfile = '--pidfile=/var/run/spamd.pid';
my $localonly = '';
my $timeouttcp = '';
my $timeoutchild = '';
if ( -e $cpspamdconf ) {
open( SPAMD, "<", $cpspamdconf );
while (<SPAMD>) {
if ( !(/^[\s\t]*$/) && !(/^[\s\t]*\#.*$/) ) {
chomp();
my ( $option, $value ) = split( '=', $_ );
next if ( !defined $value || $value eq '' );
if ( $option eq 'allowedips' ) {
$allowedips = "--allowed-ips=${value}";
}
elsif ( $option eq 'socketpath' ) {
$socketpath = "--socketpath=${value}";
}
elsif ( $option eq 'maxconnperchild' ) {
$maxconnperchild = "--max-conn-per-child=${value}";
}
elsif ( $option eq 'maxspare' ) {
$maxspare = "--max-spare=${value}";
}
elsif ( $option eq 'maxchildren' ) {
$maxchildren = "--max-children=${value}";
}
elsif ( $option eq 'pidfile' ) {
$pidfile = "--pidfile=${value}";
}
elsif ( $option eq 'local' ) {
$localonly = '--local';
}
elsif ( $option eq 'timeouttcp' ) {
$timeouttcp = "--timeout-tcp=${value}";
}
elsif ( $option eq 'timeoutchild' ) {
$timeoutchild = "--timeout-child=${value}";
}
}
}
}
close(SPAMD);
$spamdoptions = $localonly . ' ' . $timeoutchild . ' ' . $timeouttcp . ' ' . ( $socketpath ? $socketpath : $allowedips ) . ' ' . $maxconnperchild . ' ' . $pidfile . ' ' . $maxchildren . ' ' . $maxspare;
return $spamdoptions;
}

Only the options you see in that function can be set as those are the ones returned to $spamdoptions. While yes it technically would be possible to hack that routine in Restartsrv.pm, that is outside the scope of support and very much not recommended as Restartsrv.pm is used by not just SpamAssassin.

So, it appears that getting additional settings modified requires hacking of restartsrv.pl, which I'm surpised is hard-coded like that - the code must be very ancient code because that seems very un-cpanel-like

I've not yet sat down and tried to walk through the code to figure it out, so if you have any insight, it would be most appreciated.

 

[lbeachmike]

Incidentally, my feature-request is here -

http://forums.cpanel.net/f145/config-file-spamassassin-ability-set-all-available-options-283811.html

Amazingly, there is zero interest thus far. Perhaps my subject line is not very catchy I'd think this would be a hot item, given the amount of resource SpamAssassin utilizes and the plethora of currently unreachable settings that exist as shown here - best SpamAssassin options documentation - 032411

 

[lbeachmike]

So absolutely nobody in cpanel land has optimized spamassassin?

 

[semseoymas]

I discovered the cpanel parser does not take into account the contents of the variables, so you could:

[email protected] [~]# cat /etc/cpspamd.conf
allowedips=127.0.0.1
maxchildren=4
maxconnperchild=100 -l

[email protected] [~]# ps aux | grep spamd
root 8904 1.4 0.1 157584 49604 ? Ss 17:58 0:00 /usr/local/cpanel/3rdparty/perl/514/bin/spamd -d --allowed-ips=127.0.0.1 --max-conn-per-child=100 -l --pidfile=/var/run/spamd.pid --max-children=4

See that "-l" option??

 

[semseoymas]

Cannot modify... so....

Cannot use "=" if inserting new variables.... so mine is:

/usr/local/cpanel/3rdparty/perl/514/bin/spamd -d --allowed-ips=127.0.0.1 --max-conn-per-child=40 --min-spare 2 --max-spare 2 --min-children 2 --pidfile=/var/run/spamd.pid --max-children=6

As you see, my conf file:

allowedips=127.0.0.1
maxchildren=6
maxconnperchild=40 --min-spare 2 --max-spare 2 --min-children 2

 

[myusername]

So absolutely nobody in cpanel land has optimized spamassassin?

I'm just wondering if you ever made it into a real feature request instead of just requesting it in the forums. I didn't see it over there when I looked earlier.