SOLVED [CPANEL-23751] Exim Custom filter no longer working

[Shood]

Hello there,
I have many filters in /usr/local/cpanel/etc/exim/sysfilter/options and working good for a while, today I received messages from a blocked emails in those filters.

I reviewed a custom filter file, it looks good.
To make sure: I've added my own email on gmail to this filter, I've sent a test from my Gmail to my email on my server and I received it!

This is the script in filter file:

if ("$h_from:" contains "mygmail")
then fail
endif

if ("$h_from:" contains "anothermyGmail")
then fail
endif

Note: mygmail and anothermyGmail above are examples, I wrote the actual addresses in the filter file

I tried all the ways I know, e.g.
I left blank line at the top and at the bottom (I've read that somewhere in cPanel forums)
I changed then fail to then save "/dev/null" 660
I deleted the file and recreated it again but still not working

After each try I've made I saved and restarted Exim through: Basic Editor -> Filter, even I rebooted whole system.

While all filters haven't changed and were working fine, what happened?
cPanel V76
Any advice please? Thanks in advance.

 

[rpvw]

You could try using seen finish instead of fail eg

then
seen finish
endif

From the exim filer docs 3. Exim filter files

16. The finish command
The command finish, which has no arguments, causes Exim to stop interpreting the filter file. This is not a significant action unless preceded by “seen”. A filter file containing only “seen finish” is a black hole.

And do make sure your filters are correctly written into the /etc/cpanel_exim_system_filter file.

 

[Shood]

Thank you @rpvw for your reply
Unfortunately doesn't work, Filters are correctly written into two directories:
/etc/cpanel_exim_system_filter and /usr/local/cpanel/etc/exim/sysfilter/options

if ("$h_from:" contains "myGmail")
then seen finish
endif

 

[rpvw]

Try rebuilding exim in a terminal with /scripts/buildeximconf - if you see any errors or calls to old or non existent files, look in /etc/exim.conf.localopts in the filters section for references to non existent files or filters, and remove them, and then run /scripts/buildeximconf again to get a clean build.

After building you will reed to restart exim with /usr/local/cpanel/scripts/restartsrv_exim

 

[Shood]

Thank you again @rpvw
Also not working, all settings look good

 

[cPanelLauren]

Hi @Shood


Do you have a custom filter file or are you adding to the existing one? If you have a custom one are you sure that the custom file is being referenced in tweak settings? Also when you test what is logged in the exim logs?


Thanks!

 

[Shood]

Hi @cPanelLauren
I'm adding to an existing one, however today I've added a new filter file to avoid any potential conflicted in settings.
New filter is referenced in Exim Configuration Filter, the same result

if ("$h_from:" contains "mygmail")
then fail
endif

I sent a message from my Gmail to my email on server and I received it, this is exim logs:

[..goole.com..] "SpamAssassin as [cPanelUSER] detected message as NOT spam (1.1)"
[..goole.com..] Message has been scanned: no virus or other harmful content was found
[..my gamil ..] [..goole.com..]  P=esmtps ... [my message subject] for [my email on my server]
[my email on my server] R=virtual_user T=dovecot_virtual_delivery C="250 2.0.0 [my email on my server] Saved"
2018-11-28 07:52:42 Completed

Finally: what do you mean please:

custom file is being referenced in tweak settings

Custom filter is referenced in Exim Configuration Manager-> Basic ->Filters, if should be registered somewhere else please guide me how.
Thank you

 

[cPanelLauren]

Custom filter is referenced in Exim Configuration Manager-> Basic ->Filters

That's exactly what I meant I apologize I mis-typed with tweak settings.

I sent a message from my Gmail to my email on server and I received it, this is exim logs:

[..goole.com..] "SpamAssassin as [cPanelUSER] detected message as NOT spam (1.1)"
[..goole.com..] Message has been scanned: no virus or other harmful content was found
[..my gamil ..] [..goole.com..] P=esmtps ... [my message subject] for [my email on my server]
[my email on my server] R=virtual_user T=dovecot_virtual_delivery C="250 2.0.0 [my email on my server] Saved"
2018-11-28 07:52:42 Completed

I don't show the filter in this even being checked

You could test it using exim -bhc but you'd need to know the exact IP address the message was originating from and the domain would need to be added to the filter. This will output all of Exim's checks, ACL's and filters in order of when they are applied during the transaction though it won't actually deliver a message. Use instructions on this can be found here: Exim/exim

 

[cPanelLauren]

Another item that has been brought to my attention is that there is a case open regarding exim filters CPANEL-23751

In order to determine if this case is affecting your issue could you please provide me with a list of ALL present custom filters and their status in

/etc/exim.conf.localopts

and a list of all custom filters at

/usr/local/cpanel/etc/exim/sysfilter/options/

 

[Shood]

Thank you very much @cPanelLauren
You're always doing the best.

A- I tested exim -bhc, it returns a complex report from Google which in fact I couldn't understand, it's contains 440 lines so, can I pass you privately? there's a lot of private information within. Or even if opening a ticket to keep privacy.
B- exim.conf.localopts is attached, I hope that it doesn't contain some private information
C- list of all custom filters at:
z_test_28-11
zshood_block_rcve_14_9_18
zshood_blok_rcve_ml_demn_26_11_18
Also the default filters: fail_spam_score_over_..

Thank you again.

 

[rpvw]

Does the filter_test_block_14_9_18 exist as a file in the /usr/local/cpanel/etc/exim/sysfilter/options/ folder ?

 

[Shood]

Hi @rpvw
Yes it's

 

[rpvw]

I give up. To help you requires you to give us complete and accurate information, and every post you make has some important information missing, and is just wasting time.

For the last time, assuming your filters syntax are all correct, and that exim can actually process them, here are the steps you need to take to ensure that your filters are loaded and working:

1. Compare the list of filter files in your /usr/local/cpanel/etc/exim/sysfilter/options/ folder with the filter lines that are included in the /etc/exim.conf.localopts file, and delete any lines in the /etc/exim.conf.localopts file that refers to non-existing filter files.
2. In a shell terminal, run the script /scripts/buildeximconf to rebuild exim and make sure there are no errors or warnings.
3. Restart exim with /usr/local/cpanel/scripts/restartsrv_exim
4. Check the content of the /etc/cpanel_exim_system_filter file, this file should contain the content of ALL the custom filters that are contained in the /usr/local/cpanel/etc/exim/sysfilter/options/ folder. If any are missing - start over.
5. Check in your WHM >> Service Configuration >> Exim Configuration Manager > Filters that the filters you you want are correctly enabled or disabled - if you make any changes here, click the SAVE button and rebuild and restart exim and check step 4 again.

Good luck.

 

[cPanelLauren]

A- I tested exim -bhc, it returns a complex report from Google which in fact I couldn't understand, it's contains 440 lines so, can I pass you privately? there's a lot of private information within. Or even if opening a ticket to keep privacy.

It's actually not a report from google it's every single action exim takes during a message transaction. What you're looking for in that report is if the filter is ever processed.

B- exim.conf.localopts is attached, I hope that it doesn't contain some private information

It doesn't and thank you for providing it

C- list of all custom filters at:

I think the easiest way to do this would just be to run:

ls -lah  /usr/local/cpanel/etc/exim/sysfilter/options

and paste the results in your response to me, I can suss out which are the custom ones.


Thanks!

 

[Shood]

Dears, I really appreciate the big time you've spent with me.
While everything looks correct, I thing that is really a wasting of time as @rpvw said
This is the current situation:
- I deleted all custom filter files from /usr/local/cpanel/etc/exim/sysfilter/options directory
- Rebuild and restart Exim
- /etc/exim.conf.localopts didn't modified automatically so I removed all lines refer to old files manually.
- Rebuild and restart Exim. I created a new fresh filter file-> Save and restart Exim configuration, Basic -> Filter
- Now: /etc/exim.conf.localopts contains one line for the new filter name, And /usr/local/cpanel/etc/exim/sysfilter/options as below
- I still receiving this crazy message from myGmail. so what? rebuild and restart again... still receiving.

@cPanelLauren: to save your time all I thing that opening a ticket is a good choice, or continue from this point?. However we will update this thread after fixing

This below ...sysfilter/options directory contents:

drwxr-xr-x 2 root root 4.0K Dec  3 08:21 .
drwxr-xr-x 3 root root 4.0K Dec  3 08:21 ..
-rw-r--r-- 1 root root 7.3K Aug 22 21:36 attachments
-rw-r--r-- 1 root root  183 Mar 21  2018 fail_spam_score_over_100
-rw-r--r-- 1 root root  183 Mar 21  2018 fail_spam_score_over_125
-rw-r--r-- 1 root root  183 Mar 21  2018 fail_spam_score_over_150
-rw-r--r-- 1 root root  183 Mar 21  2018 fail_spam_score_over_175
-rw-r--r-- 1 root root  183 Mar 21  2018 fail_spam_score_over_200
-rw-r--r-- 1 root root  320 Mar 21  2018 fail_spam_score_over_int
-rw-r--r-- 1 root root  174 Jun 20 17:09 spam_rewrite
-rw-r--r-- 1 root root  105 Dec  3 08:20 zshood_test_ccm_3-12-18

Thank you again

 

[cPanelLauren]

Hi @Shood

Always feel free to open a support ticket with us. In this case, if you'd like to just click the link in my signature. Once it's open please update this thread with the ticket ID so that we can continue following up with updates.

Thanks!

 

[Shood]

Thank you @cPanelLauren
Support Request ID is: 10879577

 

[cPanelLauren]

Hi @Shood


Great I'm watching the ticket and I'll reply here with any updates.

Thanks!

 

[cPanelLauren]

Hi @Shood

I just checked in on your ticket and it looks like the analyst has confirmed that this is related to CPANEL-23751

There was a filter defined manually in the /etc/exim.conf.localopts file that did not exist in the /usr/local/cpanel/etc/exim/sysfilter/options/ directory.

This was causing a build error when creating the main system filter file and due to the issue in the case if a custom filter file no longer exists and filter <name> is set to 1 in /etc/exim.conf.localopts, custom filters will no longer be compiled into /etc/cpanel_exim_system_filter - meaning none of them will work.


This issue is fixed in cPanel v78 and we'll update this thread when it's moved to RELEASE. Until then the workaround for this of course would be to ensure that all of the filters set to 1 in /etc/exim.conf/localopts actually do exist.

Thanks @rpvw as well for the heads up on the case!

Thanks!

 

[Shood]

Thank you very much @cPanelLauren and @rpvw for this rich information and time you have spent in this long thread to assist me.