Global Email Filters - Body Bug

Howdy,
I'm so welcome to the forum
I'm here to ask if I'm not the only one having this issue:
Creating 2 global filters, Russian & Chinese
In each filter I enter the most used chars in each language and set if Subject or Body contains any of the, fail with a message!
For Russian I enter both lower & upper case in separate fields.
BTW, they all work just fine for Subject but NOT for Body!
You can fill the UTF-8 email body with all of blocked characters but it's not filtered and I receive those damn SPAM emails!
Seems bug to me! Advise?! cPanel 78.0.21. Site owner, no root access.

 

I found something, ain't wrong, just don’t know if this is a bug in cPanel or a bad implemented feature.
Looking at the HTML message source, those words are not inserted as they are, Base64 encoding transform your HTML message like:

И this is a test и
>>
=D0=98 this is a test =D0=B8

I don’t think I should enter the encoded form of each character in cPanel.
I do believe developers should update their underlying code and cover this encoding.
Just some staff, please let me know we will have such update in our future?
No any estimated date needed, just if we will get this covered or we should insert each encoded char as well?

Bug in forum as well:
Can't Edit original post, got error: Your message contains elements that seems like kina SPAM!!!
If so, why can make a new post?!

 

Hello and thanks for response
1. So the error being shown is misleading caused me to think it's a bug!
2. Snapshot attached, but as I told, in HTML Base64 encoded email messages these characters are encoded in message and that's why it does not find them, in your code, you should not just check:
If Message.Body.Contains(varMsgBody) = True Then Fail with Message
But you should also check for the encoded varMsgBody as well.
Cause for instance: И is converted to =D0=98 and и to =D0=B8 etc...
I don't know Perl (which I think cPanel is written in) so can't help with code snippet, but it's pretty simple.
Thanks anyway.

 

Hi dear Mike,
In filter test it works as expected, because it is Plain Text email message.
In real world most of people use HTML email message with Base64 encoding format.
So as I said for example, Russian Й character inside the source of an HTML email is NOT entered as is.
Й is converted to =D0=99 in HTML.
So if we have Й in our HTML body, your code search in the HTML source and cannot find it because it simply use the InStr (in Perl should be String > Index) to search for Й in the body source.
That's not wrong, but just not enough, in your source you should both:
1. Check Й for example, in Body for example, you already do it
2. (Add) And also check if the encoded form of Й which is =D0=99 also exists in the message source?
The lack of second check break many of user filters, in many languages/characters in HTML Body check filters.
I'm sure that's simple. And snapshot is attached

 

Please replace Base64 with Quote Printable, my bad, Base64 is usually used on attachments only, not body.

 

And please enable Edit Post, or edit the misleading error it is showing!

 

Thanks, that was just an advise, for example, I posted a reply, then wanted to add something else, and had to do it in a new post.
If the policy is to disable edits, I understand, but the error is shows in misleading: Error says that your post triggers SPAM filter!
Then user starts an endless edit to make it possible to post!
Either remove edit button, or show a proper error message.
That was just a suggestion to improve the forum, I don't need edits.
But thanks for the kind response.

 

I already asked them before posting here, and they confirmed this is by design of cPanel so no one can do something about it.
I'm also a .NET developer since 2003 and most of my work was on email systems, so I'm sure about it.
Your filter system is simply like this:
When I select: If Body Contains "Й" (for example) Fail with message
In your code you simply check:

varMsgBody is the variable holding the whole email message body
Й is the Russian character I entered in the filter list (among with some others)

If varMsgBody.Contains("Й") = True Then
'Fail with message
End If

While you should also:

If varMsgBody.Contains("Й") = True Or varMsgBody.Contains(Uri.EscapeUriString("Й")) = True Then
'Fail with message
End If

Because in the inner message source Й is converted to =D0=99 so you should check both for the entered character and it's HTML encoded form so your code logic works in all conditions with all characters.
Please send an email to my address registered on my profile and I allow you to access my cPanel and make a test account, then send an HTML email to it containing my blocked characters and you see that the filter does not work as expected.

The reason this section seems important is that we are getting many many SPAM everyday in Chinese/Russian and we prefer not to receive any email with those characters in Body.
But this filter while is excellent but is not working as expected.

*** Please note that Uri.EscapeUriString("Й") in .NET converts Й to =D0=99
Unfortunately I don't know the equivalent in Perl

 

Please note that usually Base64 is used on attachments, which are not my concern here.
I am talking about Body based filtering, that one is encoded as Quoted Printable, for example we set a new global rule:
If Body Contains "Й" Fail with message
This will not work even if you fill the body with hundreds of "Й" characters
Because in HTML email "Й" is converted to =D0=99
So the filter just works for ancient plain text emails, not HTML ones!
I also told the very quick way to fix it, just add one condition to the IF, check for its encoded form as well.

If varMsgBody.Contains("Й") = True Then
'Fail with message
End If

update to:

If varMsgBody.Contains("Й") = True Or varMsgBody.Contains(Uri.EscapeUriString("Й")) = True Then
'Fail with message
End If

That's as easy as ABC to add support for it in a future update, just please forward my whole messages to a real person who is a developer in charge.
-
I will try that other method by then

 

OK I asked my hosting provider to install TextCat plugin as suggested, after checking with your support.
I was almost sure it will not work that way, the only thing in this world I'm sure of, are email systems.

There is no certain way of detecting an email language but "language" or "encoding" tag!
Which is useless most of the times, since a Russian email can be UTF-8!
Even if you have some other algorithm, it is not working, my email body was all Russian characters and delivered perfectly.

I even did provide code samples to show your code logic fault but no one cares about the bug or code malfunction!

 

Well thanks, done, 12236283 & 12236375, I hope someone understand me finally, and care about those bugs! End of story