In Progress [CPANEL-23091] Regex filtering rule over 1024 characters issue

[delboy007]

There is a very serious bug in the regex filtering. In the past if I created a rule over 1024 characters long when I tried to create the rule I got an error message. No problem I used to reduce the characters and create another rule and could do that as many times as I wanted.

I haven't used that filtering for a year or two but now I came back to it andwas happy to see there was no error message so assume there was no longer a character limitation. A day later I was surprised to see that I hadn't received a single email, not even spam. I deleted the rule and then started to receive emails.

In short... if you enter more than 1024 characters a rule all emails sent to you and any email address will be discarded. This could be disastrous for many people and I have no idea what important emails I might have received during the time the rule was active.

 

[cPanelLauren]

Hi @delboy007

Can you provide an example of the rule/s you're creating?

Thanks!

 

[delboy007]

My host provider ran tests and agreed there was a bug. Here is the relevant part of the log

== [email protected] R=virtual_user_filter defer (-17): error in filter file: string is too long in line 13 of filter file (max = 1024 chars)

- Removed -

 

[cPanelLauren]

Hi @delboy007


I created a similar filter used 1136 characters, set it to forward to a specific email and tested it - not only did it work as expected it didn't impact any other accounts nor did I receive a similar error in the exim logs. This leads me to wonder if the issue isn't in the regex you're using or some specific customization you or your provider have employed.

 exigrep 1g0sWy-0009PP-26 /var/log/exim_mainlog
2018-09-14 13:09:25 cwd=/var/spool/exim 3 args: /usr/sbin/exim -Mc 1g0sWy-0009PP-26

2018-09-14 13:09:24 1g0sWy-0009PP-26 H=mail-lf1-f43.google.com [209.85.167.43]:46609 Warning: "SpamAssassin as myuser detected message as NOT spam (4.3)"
2018-09-14 13:09:25 1g0sWy-0009PP-26 H=mail-lf1-f43.google.com [209.85.167.43]:46609 Warning: Message has been scanned: no virus or other harmful content was found
2018-09-14 13:09:25 1g0sWy-0009PP-26 <= [email protected] H=mail-lf1-f43.google.com [IPREMOVED]:46609 P=esmtps X=TLSv1.2:ECDHE-RSA-AES128-GCM-SHA256:128 CV=no S=4192 [email protected]l.com T="Hi there" for [email protected]
2018-09-14 13:09:25 1g0sWy-0009PP-26 => myuser <[email protected]> R=virtual_user T=dovecot_virtual_delivery C="250 2.0.0 <[email protected]> IHslA1X5m1t5jQAAgi2POw Saved"
2018-09-14 13:09:25 1g0sWy-0009PP-26 Completed

 

[delboy007]

This is way beyond my area of expertise but I can't see anything in this log to say it has been through the global filter rules, only the spam assasin filter. Certainly having done any customisations as I wouldn't know how to start and I doubt whether my host providers "inmotion" would have done so either.

Just as a matter of interest can you please try again but disabling spam assasin

 

[cPanelLauren]

Hi @delboy007

It doesn't indicate that it's been even filtered because it didn't get flagged by the rule. That was actually the point - you indicated that enabling the rule caused it to fail all mail - I've done the same and can show it's not doing that. If it hits on a rule you'll get a notification in the exim log which is why I believe the issue is in your regex.
Thanks!

 

[delboy007]

I have contacted my host provider who say they have not made any customizations. Can you please test again with my regex below. As you can see the format complies with regex syntax. If there is a problem with my regex why does that it only ever fail when it contains more that 1024 characters ? If I split the regex below in half then both of them will work so I don't see how the format can be wrong.

subject = matches regex
redirect to email
discard
stop processing rules

20/20|A free sample|Amazon Offer|Amazon Review Invitation|Amazon Reward|and still going|Asian Dating|Asian Women|Auto Insurance|Baldness|belly bulge|Best Wine|Blood Sugar|brain power|Burial Insurance|cannabis|car insurance|car insurance.|Car Repair|Cash Loans|Citrix|Claim your reward|credi report|Credit Card Debt|dark web|Dead Batteries|diabetes|dress sizes|drink will make you|Ductwork|Earn Money|e-Commerce|energy efficient|equifax,|Erectile|Explicit:|fan repairs|Fight Cancer|Flash Sale|Flash Sale:|flexible loan|flight simulator|For men over|Get Hard|Get rid of|Gift Card|got stiff|Hair gel|hair growth|Hair Loss|has been searching|Health Insurance?|Healthcare|HOME INVASIONS|Home Warranty|homeowners?|How I Lost|Injury Claim|interest debt|Keto|keto diet|Ketosis?|Labor Day|learn to fly|Legal in All|life coverage|life insurance|Life Insurance|Load Documents|load offer|Loan Documents|Loans|Lose Weight|Manhood|Marijuana|Mark Cuban|match.com|Meet Singles|Miracle Cure|MK Handbags|National Debt|need replacement|Pain Killer|Pain Relief|Perfect Match|Pharma|Plumbing repairs|Printer Ink|Prostate|Quara|regrow|Renewal by Anderson|repair bills|Request A Loan|Risk Free|Russian Beauty|Russian Women|Sam's Club|Save Money|search engine results|SEO Report|Shark Tank|shocking ways|skinny|Smart Aircool!|Start Earning|Sunglasses|Tesla Said:|testosterone|Timeshare|tinnitus|Tinnitus|Toe and Nail|Trumpcare|Try new product|Vision|Wall Street|Weight Loss|Woodworking|Woolworths|WSJ|Your Family

 

[delboy007]

I can confirm that emails are not delivered for as long as the above regex is enabled. If I delete the filter then the emails start to come through around half and hour later.

 

[cPanelLauren]

Hi @delboy007

Perfect, thanks for providing that. I'll do some testing with your rule and I'll let you know what the outcome is.


Thanks!

 

[cPanelLauren]

I tried this as a user level filter initially and received no errors on it. I then added your filter rule as a global level filter and immediately started receiving errors:

R=central_filter defer (-17): error in filter file: string is too long in line 13 of filter file (max = 1024 chars)

I think my initial issue was that I had been adding it as a user level filter rather than a global filter as well. I'm opening an internal case for this as I agree you should be warned that the limitation exists rather than the behavior that is occurring now. I'll update you with the case ID as soon as it's ready.

Thanks!

 

[cPanelLauren]

Hi @delboy007

I've created CPANEL-23091 for this issue. I'll update here with progress as it becomes available.

Thanks!