Exim Filter does not work (or only for local delivery)

hub2000

Active Member
Dec 18, 2017
29
3
3
germany
cPanel Access Level
Root Administrator
Hello,

I use the separate cpanel_system_filter_new file

I added xls and xlsx filetypes for email appendices.

Its working for both xls and xlsx, for local and external email boxes sending these filetypes to my server, BUT, with one exception:

I received emails from our Chinese supplier with an excel file with filename "SL-W1-190916DOC.xls" and there its not working, repeatedly.

So, how does this filter "really" "work" ?

Hubertus

here is the info of that xls file:

.png
 

hub2000

Active Member
Dec 18, 2017
29
3
3
germany
cPanel Access Level
Root Administrator
Hello,

its stranger, i receive xlsx files, but cannot send xlsx files myself out.

i created copy of /etc/cpanel_exim_system_filter to /etc/cpanel_system_filter_new

manual Setting in WHM >> Home >> Service Configuration >> Exim Configuration Manager: "/etc/cpanel_system_filter_new"
There i added:


Code:
# Exim filter
# # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # #
#  cPanel System Filter for EXIM                                                                                #
#  VERSION = 2.0                                                                                                #
# # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # #
#   !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!DO NOT MODIFY THIS FILE DIRECTLY!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!      #
# # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # #
#   Direct modifications to the /etc/cpanel_exim_system_filter file will be lost when the configuration is      #
#   next rebuilt. To have modifications retained, please use one of the following options:                      #
#                                                                                                               #
#    1)                                                                                                         #
#      * Place each sysfilter block you wish to include in a unique file at:                                    #
#            /usr/local/cpanel/etc/exim/sysfilter/options/                                                      #
#      * Enable or disable the custom block in WHM using:                                                       #
#          Service Configuration => Exim Configuration Manager => Filters => Custom Filter: [your unique file]  #
#                                                                                                               #
#    2)                                                                                                         #
#      * Create a custom sysfilter file in /etc/                                                                #
#      * Change the location of the sysfilter file in WHM using:                                                #
#          Service Configuration => Exim Configuration Manager => Filters => System Filter File                 #
#                                                                                                               #
# # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # #
#   !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!DO NOT MODIFY THIS FILE DIRECTLY!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!      #
# # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # #
#                                                                                                               #
#  Only process once                                                                                            #
#                                                                                                               #
# # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # #
if not first_delivery
then
  finish
endif
# # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # #
#                                                                                                               #
#  Ignore "real" errors                                                                                         #
#                                                                                                               #
# # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # #
if error_message and $header_from: contains "[email protected]"
then
  finish
endif
# # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # #

# BEGIN - Included from /usr/local/cpanel/etc/exim/sysfilter/options/attachments
# (Use the Basic Editor in the Exim Configuration Manager in WHM to change)
# or manually edit /etc/exim.conf.localopts and run /scripts/buildeximconf
## -----------------------------------------------------------------------
# Look for single part MIME messages with suspicious name extensions
# Check Content-Type header using quoted filename [content_type_quoted_fn_match]
if $header_content-type: matches "(?:file)?name=(\"[^\"]+\\\\.(?:ad[ep]|ba[st]|chm|cmd|com|cpl|crt|eml|doc|docx|docm|xls|xlsx|xlsm|exe|hlp|hta|in[fs]|isp|jse?|lnk|md[be]|ms[cipt]|pcd|pif|reg|scr|sct|shs|url|vb[se]|ws[fhc])\")"
then
  fail text "This message has been rejected because it has\n\
         potentially executable content: \n\
       \n\
       $1\n\
       \n\
         This form of attachment has been used by\n\
             recent viruses or other malware.\n\
         If you meant to send this file then please\n\
         package it up as a zip file and resend it.\n\
       Office files should be sent as PDF files.\n\
       \n\
       Deutsch: Bitte senden Sie Office Files als PDF."
  seen finish
endif
# same again using unquoted filename [content_type_unquoted_fn_match]
if $header_content-type: matches "(?:file)?name=(\\\\S+\\\\.(?:ad[ep]|ba[st]|chm|cmd|com|cpl|crt|eml|doc|docx|docm|xls|xlsx|xlsm|exe|hlp|hta|in[fs]|isp|jse?|lnk|md[be]|ms[cipt]|pcd|pif|reg|scr|sct|shs|url|vb[se]|ws[fhc]))([\\\\s;]|\\$)"
then
  fail text "This message has been rejected because it has\n\
       potentially executable content: \n\
       \n\
       $1\n\
       \n\
       This form of attachment has been used by\n\
             recent viruses or other malware.\n\
       If you meant to send this file then please\n\
       package it up as a zip file and resend it.\n\
       Office files should be sent as PDF files.\n\
       \n\
       Deutsch: Bitte senden Sie Office Files als PDF."
  seen finish
endif


## -----------------------------------------------------------------------
# Attempt to catch embedded VBS attachments
# in emails.   These were used as the basis for
# the ILOVEYOU virus and its variants - many many varients
# Quoted filename - [body_quoted_fn_match]
if $message_body matches "(?:Content-(?:Type:(?>\\\\s*)[\\\\w-]+/[\\\\w-]+|Disposition:(?>\\\\s*)attachment);(?>\\\\s*)(?:file)?name=|begin(?>\\\\s+)[0-7]{3,4}(?>\\\\s+))(\"[^\"]+\\\\.(?:ad[ep]|ba[st]|chm|cmd|com|cpl|crt|eml|doc|docx|docm|xls|xlsx|xlsm|exe|hlp|hta|in[fs]|isp|jse?|lnk|md[be]|ms[cipt]|pcd|pif|reg|scr|sct|shs|url|vb[se]|ws[fhc])\")[\\\\s;]"
then
  fail text "This message has been rejected because it has\n\
         a potentially executable attachment: \n\
       \n\
       $1\n\
       \n\
         This form of attachment has been used by\n\
             recent viruses or other malware.\n\
         If you meant to send this file then please\n\
         package it up as a zip file and resend it.\n\
       Office files should be sent as PDF files.\n\
       \n\
       Deutsch: Bitte senden Sie Office Files als PDF."
  seen finish
endif
# same again using unquoted filename [body_unquoted_fn_match]
if $message_body matches "(?:Content-(?:Type:(?>\\\\s*)[\\\\w-]+/[\\\\w-]+|Disposition:(?>\\\\s*)attachment);(?>\\\\s*)(?:file)?name=|begin(?>\\\\s+)[0-7]{3,4}(?>\\\\s+))(\\\\S+\\\\.(?:ad[ep]|ba[st]|chm|cmd|com|cpl|crt|eml|doc|docx|docm|xls|xlsx|xlsm|exe|hlp|hta|in[fs]|isp|jse?|lnk|md[be]|ms[cipt]|pcd|pif|reg|scr|sct|shs|url|vb[se]|ws[fhc]))[\\\\s;]"
then
  fail text "This message has been rejected because it has\n\
         a potentially executable attachment: \n\
       \n\
       $1\n\
       \n\
         This form of attachment has been used by\n\
             recent viruses or other malware.\n\
         If you meant to send this file then please\n\
         package it up as a zip file and resend it.\n\
       Office files should be sent as PDF files.\n\
       \n\
       Deutsch: Bitte senden Sie Office Files als PDF."
  seen finish
endif
## -----------------------------------------------------------------------



#### Version history
#
# 0.01 5 May 2000
#    Initial release
# 0.02 8 May 2000
#    Widened list of content-types accepted, added WSF extension
# 0.03 8 May 2000
#    Embedded the install notes in for those that don't do manuals
# 0.04 9 May 2000
#    Check global content-type header.  Efficiency mods to REs
# 0.05 9 May 2000
#    More minor efficiency mods, doc changes
# 0.06 20 June 2000
#    Added extension handling - thx to Douglas Gray Stephens & Jeff Carnahan
# 0.07 19 July 2000
#    Latest MS Outhouse bug catching
# 0.08 19 July 2000
#    Changed trigger length to 80 chars, fixed some spelling
# 0.09 29 September 2000
#    More extensions... its getting so we should just allow 2 or 3 through
# 0.10 18 January 2001
#    Removed exclusion for error messages - this is a little nasty
#    since it has other side effects, hence we do still exclude
#    on unix like error messages
# 0.11 20 March, 2001
#    Added CMD extension, tidied docs slightly, added RCS tag
#    ** Missed changing version number at top of file :-(
# 0.12 10 May, 2001
#    Added HTA extension
# 0.13 22 May, 2001
#    Reformatted regexps and code to build them so that they are
#    shorter than the limits on pre exim 3.20 filters.  This will
#    make them significantly less efficient, but I am getting so
#    many queries about this that requiring 3.2x appears unsupportable.
# 0.14 15 August,2001
#    Added .lnk extension - most requested item :-)
#    Reformatted everything so its now built from a set of short
#    library files, cutting down on manual duplication.
#    Changed \w in filename detection to . - dodges locale problems
#    Explicit application of GPL after queries on license status
# 0.15 17 August, 2001
#    Changed the . in filename detect to \S (stops it going mad)
# 0.16 19 September, 2001
#    Pile of new extensions including the eml in current use
# 0.17 19 September, 2001
#    Syntax fix
#
#### Install Notes
#
# Exim filters run the exim filter language - a very primitive
# scripting language - in place of a user .forward file, or on
# a per system basis (on all messages passing through).
# The filtering capability is documented in the main set of manuals
# a copy of which can be found on the exim web site
#    http://www.exim.org/
#
# To install, copy the filter file (with appropriate permissions)
# to /etc/exim/system_filter.exim and add to your exim config file
# [location is installation depedant - typicaly /etc/exim/config ]
# in the first section the line:-
#    message_filter = /etc/exim/system_filter.exim
#    message_body_visible = 5000
#
# You may also want to set the message_filter_user & message_filter_group
# options, but they default to the standard exim user and so can
# be left untouched.  The other message_filter_* options are only
# needed if you modify this to do other functions such as deliveries.
# The main exim documentation is quite thorough and so I see no need
# to expand it here...
#
# Any message that matches the filter will then be bounced.
# If you wish you can change the error message by editing it
# in the section above - however be careful you don't break it.
#
# After install exim should be restarted - a kill -HUP to the
# daemon will do this.
#
#### LIMITATIONS
#
# This filter tries to parse MIME with a regexp... that doesn't
# work too well.  It will also only see the amount of the body
# specified in message_body_visible
#
#### BASIS
#
# The regexp that is used to pickup MIME/uuencoded body parts with
# quoted filenames is replicated below (in perl format).
# You need to remember that exim converts newlines to spaces in
# the message_body variable.
#
#      (?:Content-                    # start of content header
#      (?:Type: (?>\s*)                # rest of c/t header
#        [\w-]+/[\w-]+                # content-type (any)
#        |Disposition: (?>\s*)            # content-disposition hdr
#        attachment)                    # content-disposition
#      ;(?>\s*)                    # ; space or newline
#      (?:file)?name=                # filename=/name=
#      |begin (?>\s+) [0-7]{3,4} (?>\s+))         # begin octal-mode
#      (\"[^\"]+\.                    # quoted filename.
#        (?:ad[ep]                # list of extns
#        |ba[st]
#        |chm
#        |cmd
#        |com
#        |cpl
#        |crt
#        |eml
#        |exe
#        |hlp
#        |hta
#        |in[fs]
#        |isp
#        |jse?
#        |lnk
#        |md[be]
#        |ms[cipt]
#        |pcd
#        |pif
#        |reg
#        |scr
#        |sct
#        |shs
#        |url
#        |vb[se]
#        |ws[fhc])
#      \"                        # end quote
#      )                        # end of filename capture
#      [\s;]                        # trailing ;/space/newline

#
#
### [End]
# END - Included from /usr/local/cpanel/etc/exim/sysfilter/options/attachments

# BEGIN - Included from /usr/local/cpanel/etc/exim/sysfilter/options/spam_rewrite
# (Use the Basic Editor in the Exim Configuration Manager in WHM to change)
# or manually edit /etc/exim.conf.localopts and run /scripts/buildeximconf
if "${if def:header_X-Spam-Subject: {there}}" is there
then
    headers remove Subject
    headers add "Subject: $rh_X-Spam-Subject:"
    headers remove X-Spam-Subject
endif
# END - Included from /usr/local/cpanel/etc/exim/sysfilter/options/spam_rewrite
 
  • Like
Reactions: cPanelLauren

hub2000

Active Member
Dec 18, 2017
29
3
3
germany
cPanel Access Level
Root Administrator
Hello,
in case of misunderstanding i want to clarify:

I simply need a solution, where i can reject incoming emails from other servers which do have one of the listet attachements like "xls".

With the above config file i still receive emails with xls attachements.

How is this achieved?

Hubertus
 
Last edited:

cPanelLauren

Product Owner
Staff member
Nov 14, 2017
13,304
1,250
313
Houston
This activity is already performed in the exim filter with the Exim Configuration -> Filters Setting:
Attachments: Filter messages with dangerous attachments

This looks like the same filter you're using essentially. Is the setting in the exim configuration manager for this enabled?
 

hub2000

Active Member
Dec 18, 2017
29
3
3
germany
cPanel Access Level
Root Administrator
Hello,

yes, its nearly the same, i added some text in German language and some file types.

AND, big change, i added xls, xlsm, doc, docm. These file types are not in you standard "/etc/cpanel_exim_system_filter" file, but are nowadays the most problematic file types (emotet).

here the setting:

1583478111390.png
1583478721372.png

Am i correct: This should work for incoming external emails, or both outbound and inbound?

I still receive emails from China with .xls attachement


1583478329587.png


When i sent me the same xls file to my server from a free external email account, then the filter works and the email gets rejected!
 
Last edited:

cPanelLauren

Product Owner
Staff member
Nov 14, 2017
13,304
1,250
313
Houston
Am i correct: This should work for incoming external emails, or both outbound and inbound?
This should be effective for all mail - that also wouldn't explain why you receive external email with attachments from one location over another

message_body an header_content_type are the two things you want to look for which you're doing already. This is an example we have based on if you wanted to block the .foo extension:
Code:
if $header_content-type: matches "(?:file)?name=(\"[^\"]+\\\\.(?:ad[ep]|ba[st]|chm|cmd|com|cpl|crt|eml|exe|foo|hlp|hta|in[fs]|isp|jse?|lnk|md[be]|ms[cipt]|pcd|pif|reg|scr|sct|shs|url|vb[se]|ws[fhc])\")"

if $header_content-type: matches "(?:file)?name=(\\\\S+\\\\.(?:ad[ep]|ba[st]|chm|cmd|com|cpl|crt|eml|exe|foo|hlp|hta|in[fs]|isp|jse?|lnk|md[be]|ms[cipt]|pcd|pif|reg|scr|sct|shs|url|vb[se]|ws[fhc]))"

if $message_body matches "(?:Content-(?:Type:(?>\\\\s*)[\\\\w-]+/[\\\\w-]+|Disposition:(?>\\\\s*)attachment);(?>\\\\s*)(?:file)?name=|begin(?>\\\\s+)[0-7]{3,4}(?>\\\\s+))(\"[^\"]+\\\\.(?:ad[ep]|ba[st]|chm|cmd|com|cpl|crt|eml|exe|foo|hlp|hta|in[fs]|isp|jse?|lnk|md[be]|ms[cipt]|pcd|pif|reg|scr|sct|shs|url|vb[se]|ws[fhc])\")[\\\\s;]"

if $message_body matches "(?:Content-(?:Type:(?>\\\\s*)[\\\\w-]+/[\\\\w-]+|Disposition:(?>\\\\s*)attachment);(?>\\\\s*)(?:file)?name=|begin(?>\\\\s+)[0-7]{3,4}(?>\\\\s+))(\\\\S+\\\\.(?:ad[ep]|ba[st]|chm|cmd|com|cpl|crt|eml|exe|foo|hlp|hta|in[fs]|isp|jse?|lnk|md[be]|ms[cipt]|pcd|pif|reg|scr|sct|shs|url|vb[se]|ws[fhc]))[\\\\s;]"
 

hub2000

Active Member
Dec 18, 2017
29
3
3
germany
cPanel Access Level
Root Administrator
Hello,
so everything is correct configured, but does not work in all cases? The email with .xls attachement was accepted and delivered:

1.jpg

2.jpg

3.jpg


May it help, if i send you via private message the original email from /cur mail folder?
 

hub2000

Active Member
Dec 18, 2017
29
3
3
germany
cPanel Access Level
Root Administrator
Here is the next one from today:

Anmerkung 2020-03-07 084128.jpg

Strange: this time, when i look into the email on the server, there is no attachment shown with the name "Document02.xlsm"

I could send you also this email.
 

cPanelLauren

Product Owner
Staff member
Nov 14, 2017
13,304
1,250
313
Houston
There's no reason why this shouldn't be getting filtered/stripped unless this is a local domain. Is the domain the attachment is originating from a domain that is local to the server?

If not I believe It warrants some further testing of this as well, as the Content-Type and disposition match what should be getting stripped. Do you have any other customizations in place?
 

hub2000

Active Member
Dec 18, 2017
29
3
3
germany
cPanel Access Level
Root Administrator
Hello,

no, these are all external emails, originating from external servers.

I tested it also from an external server, and it was succesfully filtered.

No other customizations in place.
 

cPanelLauren

Product Owner
Staff member
Nov 14, 2017
13,304
1,250
313
Houston
Can you please open a ticket using the link in my signature? Once open please reply with the Ticket ID here so that we can update this thread with the resolution once the ticket is resolved.


Thanks!
 

hub2000

Active Member
Dec 18, 2017
29
3
3
germany
cPanel Access Level
Root Administrator
Root cause:
The entire email is not scanned as the cost would be very high overhead. Only the first 4k of an email is scanned and unfortunately, there is no way to increase this setting in Exim.

Conclusion: This rule or method does not work (for bigger emails), as spammers know about this limitation. i still get spam emails.

Way forward:
I noticed, the desired functionality can be achieved through:

Global Email Filters
Body Contains .xls …..
Discard Email

Perhaps this „cpanel_system_filter_new“ help section in cpanel is a bit misleading for people want to achieve a better protection and should be updated? (The helpfile) makes the impression “it works” but its not reliable as we see now….)

https://docs.cpanel.net/knowledge-base/email/how-to-customize-the-exim-system-filter-file/

Nice to have:
Is there perhaps a professional plug-in for cpanel which filters emails
 

cPanelLauren

Product Owner
Staff member
Nov 14, 2017
13,304
1,250
313
Houston
I just looked into this, it does look like the issue is the message_body_visible. If you want to scan ALL email for attachments despite the size you could modify this. I don't think the SA rule would work for this without scanning the entire message since SA has a threshold which would need to be increased as well.

I don't know where it was identified that this setting is not increasable. You can indeed increase this, the support analysts just indicated that they couldn't do it for you though they should have let you know that you can do it by going here: WHM>>Service Configuration>>Exim Configuration Manager -> Advanced Editor -> message_body_visible

Exim's default is 500 lines where ours is 5000

I highly doubt the overhead would be hugely noticeable if you increased it unless you receive email with huge attachments frequently.

I can suggest that kb article be updated to include a note about message body size, I do think this is worth noting. I don't think the article is misleading, exim's standard function doesn't have an issue though does have a size limitation.

There are a ton of plugins available for exim. SpamAssassin is integrated into exim which will perform this functionality and one of our analysts provided a rule for it for you. But there are a number of other applications that will do this - some examples can be found here: cPanel App Catalog :: Result for "spam"
 

hub2000

Active Member
Dec 18, 2017
29
3
3
germany
cPanel Access Level
Root Administrator
Hello,

thank you so much for your help.
What do you mean with "SA Filter": Spam assassin filter? But this is not "Global Email Filter" i tried now, right?

The situation is not fully solved with "Spam assassin Filter" nor "Exim Filter", due to the "big email sizes".
Unfortunately, all spammers are aware of this problem ;) and are sending "big emails" :rolleyes:.

So i tried next option "Global Email Filter" . There I also still get .doc files with malicious spam.

I used the global filter:

Anmerkung 2020-03-18 091755.jpg

But id did not work for an email of 445kb this morning:
Anmerkung 2020-03-18 092105.jpg

- is there also a "size scan limit" for the Global Email Filter?

To face the reality, somehow the whole email needs to be scanned in order to get more security?? The question is how efficient/intelligent the scanner is.

Question: Do you recommend for a 99% solution (now 20% solution):

a) increasing Exims message_body_visible (our server: only 2200 emails/day, 128GB 16 Core )
b) increasing Spamd scan size and make a spamd rule
c) Using Global Email Filters (how?)
d) other working solution
 
Last edited:

cPanelLauren

Product Owner
Staff member
Nov 14, 2017
13,304
1,250
313
Houston
- is there also a "size scan limit" for the Global Email Filter?
These all use exim's directives for this. So it'd be subject to the same issues I'd assume, you could increase logging to determine what *exactly* is failing. I added the following in my exim configuration at /etc/exim.conf.local:

Code:
@[email protected]
log_selector = +all
If you do this you'd need to rebuild the exim configuration by running /scripts/buildeximconf or just update it in WHM's Exim Configuration Manager -> Advanced Editor.

This is discussed here: 53. Log files - Section 15. Reducing or increasing what is logged.
The following may also be helpful:

There's also a way to test filters before installing them as well as already installed filters:

1. Forwarding and filtering in Exim - Section 3 Testing a New Filter File and Section 5 Testing an installed filter file.


a) increasing Exims message_body_visible (our server: only 2200 emails/day, 128GB 16 Core )
So if you sufficiently increased the message_body_visible variable so that all email's were being parsed through the filter, I don't see how that would be a 99% resolution?
Another thing to note is how many of those emails would actually be part of the group that isn't scanned fully.

b) increasing Spamd scan size and make a spamd rule
This solution would be on par with the filter added through exim. Essentially it's going to be the same process to allow for this to happen on larger emails.