SOLVED Why was this outgoing email marked as spam

[jnyr5478]

Hi,

Our client's server uses PHP Mail to send email. We primarily use SpamAssassin to filter outgoing email. I need to understand why some emails are incorrectly being flagged as spam and therefore end up in exim_rejectlog.

Here are some relevant settings in Home > Service Configuration > Exim Configuration Manager:

ACL Options:

• Apache SpamAssassin™ reject spam score threshold: 5

Apache SpamAssassin™ Options:

• Apache SpamAssassin™: Forced Global ON: On
• Apache SpamAssassin™: message size threshold to scan: 200KB
• (this option is greyed out) Scan outgoing messages for spam and reject based on defined Apache SpamAssassin™ score (Minimum: 0.1; Maximum: 99.9): 2
• Do not forward mail to external recipients based on the defined Apache SpamAssassin™ score (Minimum: 0.1; Maximum: 99.9): 2

All other options are set to Default. I see in this post that outgoing settings are stored in a different place. In /etc/exim.conf.localopts, I see this at the top:

acl_outgoing_spam_scan_over_int=20

Here's an example of an email that is logged in exim_rejectlog:

2019-12-05 13:35:59 1icvyo-000Pdx-IO F=<[email protected]> rejected by non-SMTP ACL: "SpamAssassin as cpaneleximscanner detected OUTGOING not smtp message as spam (2.2/20)"
Envelope-from: <[email protected]>
Envelope-to: <[email protected]>
P Received: from cpaneluseraccount by host.clientdomain.tld with local (Exim 4.92)
(envelope-from <[email protected]>)
id 1icvyo-000Pdx-IO
for [email protected]; Thu, 05 Dec 2019 13:35:58 -0500
T To: [email protected]ld
Subject: New Warranty Reimbursement Claim
X-PHP-Script: www.clientdomain.tld/index.php for { IP Address }
X-PHP-Originating-Script: 1004:class.phpmailer.php
Date: Thu, 5 Dec 2019 18:35:58 +0000
F From: Client Company <[email protected]>
I Message-ID: <[email protected]>
MIME-Version: 1.0
Content-Type: text/html; charset=UTF-8

The user submitted the following comments, which don't seem spammy to me:

"On/ Off switch would not operate. Replaced faulty wiring harness."

Questions:

1. To confirm, when scanning outgoing emails, is SA using a threshold of 20 (2 * 10) and not 5?
2. If the above is correct, which option controls the outgoing threshold score?
   • Scan outgoing messages for spam and reject based on defined Apache SpamAssassin™ score
   • Do not forward mail to external recipients based on the defined Apache SpamAssassin™ score
3. Does anyone have thoughts on what parts of this email are flagging as spam? FWIW, this same user (same email address) was successfully able to send a very similar email roughly 30 minutes later.

Thanks!

 

[cPanelLauren]

Hi @jnyr5478

First of all, I just want to thank you for the detailed information. It helps everyone understand where specifically the issue lies and shows what you've looked at thus far.

To confirm, when scanning outgoing emails, is SA using a threshold of 20 (2 * 10) and not 5?

Based on what you've shown it would appear that it should be 2.0 or 2*10

• Why is the following greyed out?

• (this option is greyed out) Scan outgoing messages for spam and reject based on defined Apache SpamAssassin™ score (Minimum: 0.1; Maximum: 99.9): 2

• The only instance in which this should be greyed out is if you've selected

Scan outgoing messages for spam and reject based on the Apache SpamAssassin™ internal spam_score setting

• only one of these is able to be selected at a time. For example, looking at my settings on my test server:

• The same goes for forwarding mail, but based on what you note you've defined the score as 2:

• Do not forward mail to external recipients based on the defined Apache SpamAssassin™ score (Minimum: 0.1; Maximum: 99.9): 2

• Apache SpamAssassin™ reject spam score threshold: 5

This threshold is for inbound mail only, SpamAssassin will reject mail at SMTP time if the threshold is 5 or higher.

You can see the score an email must be under in order to be considered NOT spam in the message exim outputs as well:

rejected by non-SMTP ACL: "SpamAssassin as cpaneleximscanner detected OUTGOING not smtp message as spam (2.2/20)"

Specifically (2.2/2.0)

f the above is correct, which option controls the outgoing threshold score?

• Scan outgoing messages for spam and reject based on defined Apache SpamAssassin™ score
• Do not forward mail to external recipients based on the defined Apache SpamAssassin™ score

These are two different functions for outbound mail, one of these is solely for mail that is being forwarded and the other is for mail that is being sent from your server without a forward.

For example:

• I send an email to [email protected] from my gmail account.
• [email protected] has a forwarder in place to an external source [email protected]
• On user1's server they have set the following:
  • Scan outgoing messages for spam and reject based on defined Apache SpamAssassin™ score - 5.0 (5*10)
  • Do not forward mail to external recipients based on the defined Apache SpamAssassin™ score - 2.0 (2*10)
  • user1 has inbound scoring set to flag as SPAM at 2.0
  • user1 has the reject threshold set to 5 or autodelete enabled and set to 5
• My message scores a 2.5 with SpamAssassin when it comes into the server (inbound SpamAssassin Scoring)
• Based on the score the message is flagged as SPAM but deliverable to [email protected]
• Exim then looks for a forwarder, since one is found exim will attempt to forward the mail
• Based on the forward mail to external recipients setting of 2.0 my message will not be forwarded to the external server
• If user1 were to directly send my mail to their external server manually and it scored a 2.0 based on the Scan outgoing messages for spam setting of 5.0 the message would be able to be sent.

oes anyone have thoughts on what parts of this email are flagging as spam? FWIW, this same user (same email address) was successfully able to send a very similar email roughly 30 minutes later.

You can see what rules are flagged for an outbound email in /var/log/maillog

I tested this with some emails earlier today and here are the results I got:

This is a test using SpamAssassin's GTUBE test:

Dec 17 16:33:12 server spamd[11879]: spamd: connection from localhost [127.0.0.1]:47922 to port 783, fd 6
Dec 17 16:33:12 server spamd[11879]: spamd: setuid to cpaneleximscanner succeeded
Dec 17 16:33:12 server spamd[11879]: generic: trusted_networks doesn't contain internal_networks entry '0/0'
Dec 17 16:33:12 server spamd[11879]: spamd: checking message <[email protected]> for cpaneleximscanner:990
Dec 17 16:33:12 server spamd[11879]: spamd: identified spam (1001.1/5.0) for cpaneleximscanner:990 in 0.5 seconds, 795 bytes.
Dec 17 16:33:12 server spamd[11879]: spamd: result: Y 1001 - ALL_TRUSTED,FSL_BULK_SIG,GTUBE,PYZOR_CHECK,TVD_SPACE_RATIO scantime=0.5,size=795,user=cpaneleximscanner,uid=990,required_score=5.0,rhost=localhost,raddr=127.0.0.1,rport=47922,mid=<[email protected]>,autolearn=no autolearn_force=no,shortcircuit=no

I also tested with some keywords widely used by spammers:

Dec 17 16:31:39 server spamd[11879]: spamd: setuid to cpaneleximscanner succeeded
Dec 17 16:31:39 server spamd[11879]: generic: trusted_networks doesn't contain internal_networks entry '0/0'
Dec 17 16:31:39 server spamd[11879]: spamd: checking message <[email protected]> for cpaneleximscanner:990
Dec 17 16:31:39 server spamd[11879]: spamd: identified spam (4.3/2.0) for cpaneleximscanner:990 in 0.5 seconds, 767 bytes.
Dec 17 16:31:39 server spamd[11879]: spamd: result: Y 4 - ALL_TRUSTED,DRUGS_ERECTILE,KAM_VIAGRA6 scantime=0.5,size=767,user=cpaneleximscanner,uid=990,required_score=2.0,rhost=localhost,raddr=127.0.0.1,rport=47882,mid=<[email protected]>,autolearn=no autolearn_force=no,shortcircuit=no

This doesn't give me the weight for each score but it does give me the rules that matched. I can look these up in cPanel (or over CLI)

Rule	Weight
ALL_TRUSTED	-1
FSL_BULK_SIG	0.118
GTUBE	1000
PYZOR_CHECK	1.985
TVD_SPACE_RATIO	.0001

Total Weight: 1001.1

CLI (not pretty) output:

[[email protected] spamassassin]# egrep -r 'ALL_TRUSTED|FSL_BULK_SIG|GTUBE|PYZOR_CHECK|TVD_SPACE_RATIO' /var/lib/spamassassin |grep score
3.004001/updates_spamassassin_org/50_scores.cf:score ALL_TRUSTED -1.000
3.004001/updates_spamassassin_org/50_scores.cf:score TVD_SPACE_RATIO 0.001
3.004001/updates_spamassassin_org/50_scores.cf:# score ALL_TRUSTED -1.360 -1.440 -1.665 -1.800
3.004001/updates_spamassassin_org/50_scores.cf:# GTUBE - Generic Test for Unsolicited Bulk Email
3.004001/updates_spamassassin_org/50_scores.cf:score GTUBE 1000.000
3.004001/updates_spamassassin_org/50_scores.cf:score PYZOR_CHECK 0 1.985 0 1.392 # n=0 n=2
3.004003/updates_spamassassin_org/72_active.cf:#score    FSL_BULK_SIG          3.000    # limit
3.004003/updates_spamassassin_org/72_active.cf:#score       TVD_SPACE_RATIO_MINFP  2.500   # limit
3.004003/updates_spamassassin_org/72_scores.cf:score FSL_BULK_SIG                          0.001 0.118 0.001 0.118
3.004003/updates_spamassassin_org/72_scores.cf:score TVD_SPACE_RATIO_MINFP                 2.499 2.499 2.499 2.499
3.004003/updates_spamassassin_org/50_scores.cf:score ALL_TRUSTED -1.000
3.004003/updates_spamassassin_org/50_scores.cf:score TVD_SPACE_RATIO 0.001
3.004003/updates_spamassassin_org/50_scores.cf:# score ALL_TRUSTED -1.360 -1.440 -1.665 -1.800
3.004003/updates_spamassassin_org/50_scores.cf:# GTUBE - Generic Test for Unsolicited Bulk Email
3.004003/updates_spamassassin_org/50_scores.cf:score GTUBE 1000.000
3.004003/updates_spamassassin_org/50_scores.cf:score PYZOR_CHECK 0 1.985 0 1.392 # n=0 n=2
3.004002/updates_spamassassin_org/72_active.cf:#score    FSL_BULK_SIG          3.000    # limit
3.004002/updates_spamassassin_org/72_active.cf:#score       TVD_SPACE_RATIO_MINFP  2.500   # limit
3.004002/updates_spamassassin_org/72_scores.cf:score FSL_BULK_SIG                          0.001 0.110 0.001 0.110
3.004002/updates_spamassassin_org/72_scores.cf:score TVD_SPACE_RATIO_MINFP                 2.499 2.499 2.499 2.499
3.004002/updates_spamassassin_org/50_scores.cf:score ALL_TRUSTED -1.000
3.004002/updates_spamassassin_org/50_scores.cf:score TVD_SPACE_RATIO 0.001
3.004002/updates_spamassassin_org/50_scores.cf:# score ALL_TRUSTED -1.360 -1.440 -1.665 -1.800
3.004002/updates_spamassassin_org/50_scores.cf:# GTUBE - Generic Test for Unsolicited Bulk Email
3.004002/updates_spamassassin_org/50_scores.cf:score GTUBE 1000.000
3.004002/updates_spamassassin_org/50_scores.cf:score PYZOR_CHECK 0 1.985 0 1.392 # n=0 n=2

For the second one:

Rule	Weight
ALL_TRUSTED	-1
DRUGS_ERECTILE	2.221
KAM_VIAGRA6	3.1

Total Weight: 4.3

From CLI (not pretty)

[[email protected] spamassassin]# egrep -r 'ALL_TRUSTED|DRUGS_ERECTILE|KAM_VIAGRA6' /var/lib/spamassassin|grep score
3.004001/updates_spamassassin_org/50_scores.cf:score ALL_TRUSTED -1.000
3.004001/updates_spamassassin_org/50_scores.cf:score DRUGS_ERECTILE 1.778 2.221 1.299 1.994
3.004001/updates_spamassassin_org/50_scores.cf:score DRUGS_ERECTILE_OBFU 1.324 1.309 2.935 1.109
3.004001/updates_spamassassin_org/50_scores.cf:# score ALL_TRUSTED -1.360 -1.440 -1.665 -1.800
3.004003/updates_spamassassin_org/72_active.cf:#score    DRUGS_ERECTILE_SHORT_SHORTNER 1.5 # limit
3.004003/updates_spamassassin_org/72_scores.cf:score DRUGS_ERECTILE_SHORT_SHORTNER         1.079 1.499 1.079 1.499
3.004003/updates_spamassassin_org/50_scores.cf:score ALL_TRUSTED -1.000
3.004003/updates_spamassassin_org/50_scores.cf:score DRUGS_ERECTILE 1.778 2.221 1.299 1.994
3.004003/updates_spamassassin_org/50_scores.cf:score DRUGS_ERECTILE_OBFU 1.324 1.309 2.935 1.109
3.004003/updates_spamassassin_org/50_scores.cf:# score ALL_TRUSTED -1.360 -1.440 -1.665 -1.800
3.004002/updates_spamassassin_org/72_active.cf:#score    DRUGS_ERECTILE_SHORT_SHORTNER 1.5 # limit
3.004002/updates_spamassassin_org/72_scores.cf:score DRUGS_ERECTILE_SHORT_SHORTNER         0.607 0.059 0.607 0.059
3.004002/updates_spamassassin_org/50_scores.cf:score ALL_TRUSTED -1.000
3.004002/updates_spamassassin_org/50_scores.cf:score DRUGS_ERECTILE 1.778 2.221 1.299 1.994
3.004002/updates_spamassassin_org/50_scores.cf:score DRUGS_ERECTILE_OBFU 1.324 1.309 2.935 1.109
3.004002/updates_spamassassin_org/50_scores.cf:# score ALL_TRUSTED -1.360 -1.440 -1.665 -1.800

 

[jnyr5478]

Thanks @cPanelLauren, this is helpful.

I'm not sure why, but both of these Apache SpamAssassin™ Options are greyed out (screenshot attached):

• Scan outgoing messages for spam and reject based on the Apache SpamAssassin™ internal spam_score setting: On
• Scan outgoing messages for spam and reject based on defined Apache SpamAssassin™ score (Minimum: 0.1; Maximum: 99.9): 2

It seems that I need to increase the outgoing spam threshold to something greater than 2.2, but I can't do that with the option greyed out. I've toggled the "Apache SpamAssassin™: Forced Global ON" option to Off and saved, but that didn't ungrey either of them.

Looking in /var/log/maillog shows the following for one of the emails rejected as spam:

Dec  5 11:01:47 host spamd[29812]: spamd: connection from localhost [127.0.0.1]:46780 to port 783, fd 6
Dec  5 11:01:47 host spamd[29812]: spamd: setuid to cpaneleximscanner succeeded
Dec  5 11:01:47 host spamd[29812]: generic: trusted_networks doesn't contain internal_networks entry '0/0'
Dec  5 11:01:47 host spamd[29812]: spamd: checking message <[email protected]> for cpaneleximscanner:991
Dec  5 11:01:48 host spamd[29812]: spamd: clean message (2.2/5.0) for cpaneleximscanner:991 in 0.8 seconds, 7725 bytes.
Dec  5 11:01:48 host spamd[29812]: spamd: result: . 2 - BITCOIN_SPAM_01,HTML_MESSAGE,HTML_MIME_NO_HTML_TAG,MIME_HTML_ONLY,PDS_BTC_ID scantime=0.8,size=7725,user=cpaneleximscanner,uid=991,required_score=5.0,rhost=localhost,raddr=127.0.0.1,rport=46780,mid=<[email protected]>,autolearn=no autolearn_force=no,shortcircuit=no
Dec  5 11:01:48 host spamd[50425]: prefork: child states: II

This doesn't give me the weight for each score but it does give me the rules that matched. I can look these up in cPanel (or over CLI)

^^ I don't know how to do this.

 

[cPanelLauren]

Hi @jnyr5478

I'm not sure why, but both of these Apache SpamAssassin™ Options are greyed out (screenshot attached):

• Scan outgoing messages for spam and reject based on the Apache SpamAssassin™ internal spam_score setting: On
• Scan outgoing messages for spam and reject based on defined Apache SpamAssassin™ score (Minimum: 0.1; Maximum: 99.9): 2

It seems that I need to increase the outgoing spam threshold to something greater than 2.2, but I can't do that with the option greyed out. I've toggled the "Apache SpamAssassin™: Forced Global ON" option to Off and saved, but that didn't ungrey either of them.

What's in /var/cpanel/cpanel.config for this? Mine, for example looks like this:

grep acl_outgoing_spam_scan /etc/exim.conf.localopts

^^ I don't know how to do this.

I apologize, I gave you all of that info on everything else but didn't tell you how to do that. You can find the scores in cPanel>>Email>>Spam Filters>>Show Additional Configurations -> Configure Calculated Spam Scores Settings -> Add a New "scores" item - you won't actually add this you'll just use it to search for the rules.

The raw output like I ran for mine is:

[[email protected] /]# egrep -r 'BITCOIN_SPAM_01|HTML_MESSAGE|HTML_MIME_NO_HTML_TAG|MIME_HTML_ONLY|PDS_BTC_ID' /var/lib/spamassassin|grep score
/var/lib/spamassassin/3.004001/updates_spamassassin_org/50_scores.cf:score HTML_MIME_NO_HTML_TAG 0.001 0.635 0.001 0.377
/var/lib/spamassassin/3.004001/updates_spamassassin_org/50_scores.cf:score MIME_HTML_ONLY_MULTI 0.000 0.001 0.001 0.001
/var/lib/spamassassin/3.004001/updates_spamassassin_org/50_scores.cf:score HTML_MESSAGE 0.001
/var/lib/spamassassin/3.004001/updates_spamassassin_org/50_scores.cf:score MIME_HTML_ONLY 2.199 1.105 1.199 0.723
/var/lib/spamassassin/3.004003/updates_spamassassin_org/72_active.cf:#score          BITCOIN_SPAM_01  2.500    # limit
/var/lib/spamassassin/3.004003/updates_spamassassin_org/72_active.cf:#score    PDS_BTC_ID 0.5
/var/lib/spamassassin/3.004003/updates_spamassassin_org/72_scores.cf:score BITCOIN_SPAM_01                       1.000 1.000 1.000 1.000
/var/lib/spamassassin/3.004003/updates_spamassassin_org/72_scores.cf:score PDS_BTC_ID                            0.500 0.499 0.500 0.499
/var/lib/spamassassin/3.004003/updates_spamassassin_org/50_scores.cf:score HTML_MIME_NO_HTML_TAG 0.001 0.635 0.001 0.377
/var/lib/spamassassin/3.004003/updates_spamassassin_org/50_scores.cf:score MIME_HTML_ONLY_MULTI 0.000 0.001 0.001 0.001
/var/lib/spamassassin/3.004003/updates_spamassassin_org/50_scores.cf:score HTML_MESSAGE 0.001
/var/lib/spamassassin/3.004003/updates_spamassassin_org/50_scores.cf:score MIME_HTML_ONLY 0.1
/var/lib/spamassassin/3.004002/updates_spamassassin_org/72_active.cf:#score          BITCOIN_SPAM_01  2.500    # limit
/var/lib/spamassassin/3.004002/updates_spamassassin_org/72_active.cf:#score    PDS_BTC_ID 0.5
/var/lib/spamassassin/3.004002/updates_spamassassin_org/72_scores.cf:score BITCOIN_SPAM_01                       1.000 1.000 1.000 1.000
/var/lib/spamassassin/3.004002/updates_spamassassin_org/72_scores.cf:score PDS_BTC_ID                            0.499 0.499 0.499 0.499
/var/lib/spamassassin/3.004002/updates_spamassassin_org/50_scores.cf:score HTML_MIME_NO_HTML_TAG 0.001 0.635 0.001 0.377
/var/lib/spamassassin/3.004002/updates_spamassassin_org/50_scores.cf:score MIME_HTML_ONLY_MULTI 0.000 0.001 0.001 0.001
/var/lib/spamassassin/3.004002/updates_spamassassin_org/50_scores.cf:score HTML_MESSAGE 0.001
/var/lib/spamassassin/3.004002/updates_spamassassin_org/50_scores.cf:score MIME_HTML_ONLY 0.1

Rule	Weight
BITCOIN_SPAM_01	1
HTML_MESSAGE	0.001
HTML_MIME_NO_HTML_TAG	0.635
MIME_HTML_ONLY	0.1
PDS_BTC_ID	0.499

Total: 2.2

 

[jnyr5478]

$ grep acl_outgoing_spam_scan /etc/exim.conf.localopts

shows

acl_outgoing_spam_scan=1
acl_outgoing_spam_scan_over_int=20

 

[SamuelM]

Hello @jnyr5478,

It seems that I need to increase the outgoing spam threshold to something greater than 2.2, but I can't do that with the option greyed out. I've toggled the "Apache SpamAssassin™: Forced Global ON" option to Off and saved, but that didn't ungrey either of them.

I'm not certain if you ever figured this out. In order to define your own value for the Do not forward mail to external recipients based on the defined Apache SpamAssassin™ score option, you need to first toggle the Do not forward mail to external recipients if it matches the Apache SpamAssassin™ internal spam_score setting option to Off, then scroll to the bottom of the page and click Save. Then navigate back to the Exim Configuration Manager and you should now see that the option is no longer greyed out, and you can define a score of your choice.

 

[jnyr5478]

@cPSamuel this is resolved. I opened a ticket with the hosting company who ultimately replied with the following:

It seems that the both options were enabled manually which caused configuration descripencies with cPanel scripts.

After disabling it manually in advanced configuration I was able to switch in between the 2 mentioned options.

Thanks for checking in.

 

[SamuelM]

Thank you @jnyr5478 for getting back to us. I will now mark this thread as solved. Feel free to reply if you have any further questions in the future.