Please whitelist cPanel in your adblocker so that you’re able to see our version release promotions, thanks!

The Community Forums

Interact with an entire community of cPanel & WHM users!

Global Email Filters - Body Bug

Discussion in 'E-mail Discussion' started by omidsolo, Apr 29, 2019.

Tags:
  1. omidsolo

    omidsolo Member

    Joined:
    Apr 29, 2019
    Messages:
    16
    Likes Received:
    0
    Trophy Points:
    1
    Location:
    US
    cPanel Access Level:
    Website Owner
    Howdy,
    I'm so welcome to the forum :)
    I'm here to ask if I'm not the only one having this issue:
    Creating 2 global filters, Russian & Chinese
    In each filter I enter the most used chars in each language and set if Subject or Body contains any of the, fail with a message!
    For Russian I enter both lower & upper case in separate fields.
    BTW, they all work just fine for Subject but NOT for Body!
    You can fill the UTF-8 email body with all of blocked characters but it's not filtered and I receive those damn SPAM emails!
    Seems bug to me! Advise?! cPanel 78.0.21. Site owner, no root access.
     
    Stop hovering to collapse... Click to collapse... Hover to expand... Click to expand...
  2. omidsolo

    omidsolo Member

    Joined:
    Apr 29, 2019
    Messages:
    16
    Likes Received:
    0
    Trophy Points:
    1
    Location:
    US
    cPanel Access Level:
    Website Owner
    I found something, ain't wrong, just don’t know if this is a bug in cPanel or a bad implemented feature.
    Looking at the HTML message source, those words are not inserted as they are, Base64 encoding transform your HTML message like:

    И this is a test и
    >>
    =D0=98 this is a test =D0=B8

    I don’t think I should enter the encoded form of each character in cPanel.
    I do believe developers should update their underlying code and cover this encoding.
    Just some staff, please let me know we will have such update in our future?
    No any estimated date needed, just if we will get this covered or we should insert each encoded char as well?

    Bug in forum as well:
    Can't Edit original post, got error: Your message contains elements that seems like kina SPAM!!!
    If so, why can make a new post?!
     
    Stop hovering to collapse... Click to collapse... Hover to expand... Click to expand...
    #2 omidsolo, Apr 29, 2019
    Last edited: Apr 29, 2019
  3. Infopro

    Infopro cPanel Sr. Product Evangelist Staff Member

    Joined:
    May 20, 2003
    Messages:
    16,899
    Likes Received:
    482
    Trophy Points:
    583
    Location:
    Pennsylvania
    cPanel Access Level:
    Root Administrator
    Twitter:
    This has to do with you being a new forum user. Editing of posts is not permitted by new users.

    As for your filter, you might want to post a screenshot (upload it to a post, do not link to an image please) or simply post it here, within code tags, for review, to see whats breaking it.
     
    Stop hovering to collapse... Click to collapse... Hover to expand... Click to expand...
  4. omidsolo

    omidsolo Member

    Joined:
    Apr 29, 2019
    Messages:
    16
    Likes Received:
    0
    Trophy Points:
    1
    Location:
    US
    cPanel Access Level:
    Website Owner
    Hello and thanks for response :)
    1. So the error being shown is misleading caused me to think it's a bug!
    2. Snapshot attached, but as I told, in HTML Base64 encoded email messages these characters are encoded in message and that's why it does not find them, in your code, you should not just check:
    If Message.Body.Contains(varMsgBody) = True Then Fail with Message
    But you should also check for the encoded varMsgBody as well.
    Cause for instance: И is converted to =D0=98 and и to =D0=B8 etc...
    I don't know Perl (which I think cPanel is written in) so can't help with code snippet, but it's pretty simple.
    Thanks anyway.
     

    Attached Files:

    Stop hovering to collapse... Click to collapse... Hover to expand... Click to expand...
  5. cPanelMichael

    cPanelMichael Technical Support Community Manager Staff Member

    Joined:
    Apr 11, 2011
    Messages:
    47,325
    Likes Received:
    2,157
    Trophy Points:
    363
    cPanel Access Level:
    Root Administrator
    Twitter:
    Hello @omidsolo,

    Can you use the "Filter Test" option in cPanel >> Global Email Filters and upload a screenshot of the results?

    Thank you.
     
    Stop hovering to collapse... Click to collapse... Hover to expand... Click to expand...
  6. omidsolo

    omidsolo Member

    Joined:
    Apr 29, 2019
    Messages:
    16
    Likes Received:
    0
    Trophy Points:
    1
    Location:
    US
    cPanel Access Level:
    Website Owner
    Hi dear Mike,
    In filter test it works as expected, because it is Plain Text email message.
    In real world most of people use HTML email message with Base64 encoding format.
    So as I said for example, Russian Й character inside the source of an HTML email is NOT entered as is.
    Й is converted to =D0=99 in HTML.
    So if we have Й in our HTML body, your code search in the HTML source and cannot find it because it simply use the InStr (in Perl should be String > Index) to search for Й in the body source.
    That's not wrong, but just not enough, in your source you should both:
    1. Check Й for example, in Body for example, you already do it
    2. (Add) And also check if the encoded form of Й which is =D0=99 also exists in the message source?
    The lack of second check break many of user filters, in many languages/characters in HTML Body check filters.
    I'm sure that's simple. And snapshot is attached :)
     

    Attached Files:

    Stop hovering to collapse... Click to collapse... Hover to expand... Click to expand...
    #6 omidsolo, May 1, 2019
    Last edited: May 2, 2019
  7. omidsolo

    omidsolo Member

    Joined:
    Apr 29, 2019
    Messages:
    16
    Likes Received:
    0
    Trophy Points:
    1
    Location:
    US
    cPanel Access Level:
    Website Owner
    Please replace Base64 with Quote Printable, my bad, Base64 is usually used on attachments only, not body.
     
    Stop hovering to collapse... Click to collapse... Hover to expand... Click to expand...
  8. omidsolo

    omidsolo Member

    Joined:
    Apr 29, 2019
    Messages:
    16
    Likes Received:
    0
    Trophy Points:
    1
    Location:
    US
    cPanel Access Level:
    Website Owner
    And please enable Edit Post, or edit the misleading error it is showing!
     
    Stop hovering to collapse... Click to collapse... Hover to expand... Click to expand...
  9. Infopro

    Infopro cPanel Sr. Product Evangelist Staff Member

    Joined:
    May 20, 2003
    Messages:
    16,899
    Likes Received:
    482
    Trophy Points:
    583
    Location:
    Pennsylvania
    cPanel Access Level:
    Root Administrator
    Twitter:
    Please provide more details of what you need edited. After 24 hours you're unable to edit posts, but we'll be happy to help with that if needed.
     
    Stop hovering to collapse... Click to collapse... Hover to expand... Click to expand...
  10. omidsolo

    omidsolo Member

    Joined:
    Apr 29, 2019
    Messages:
    16
    Likes Received:
    0
    Trophy Points:
    1
    Location:
    US
    cPanel Access Level:
    Website Owner
    Thanks, that was just an advise, for example, I posted a reply, then wanted to add something else, and had to do it in a new post.
    If the policy is to disable edits, I understand, but the error is shows in misleading: Error says that your post triggers SPAM filter!
    Then user starts an endless edit to make it possible to post!
    Either remove edit button, or show a proper error message.
    That was just a suggestion to improve the forum, I don't need edits.
    But thanks for the kind response.
     
    Stop hovering to collapse... Click to collapse... Hover to expand... Click to expand...
  11. cPanelMichael

    cPanelMichael Technical Support Community Manager Staff Member

    Joined:
    Apr 11, 2011
    Messages:
    47,325
    Likes Received:
    2,157
    Trophy Points:
    363
    cPanel Access Level:
    Root Administrator
    Twitter:
    Hello @omidsolo,

    It would help our investigation if we could access the affected system and review the live filter. Can you ask your hosting provider to open a support ticket with us on your behalf so we can take a closer look? You can ask them for the ticket number they receive from us and post it here once they let you know what it is.

    Thank you.
     
    Stop hovering to collapse... Click to collapse... Hover to expand... Click to expand...
  12. omidsolo

    omidsolo Member

    Joined:
    Apr 29, 2019
    Messages:
    16
    Likes Received:
    0
    Trophy Points:
    1
    Location:
    US
    cPanel Access Level:
    Website Owner
    I already asked them before posting here, and they confirmed this is by design of cPanel so no one can do something about it.
    I'm also a .NET developer since 2003 and most of my work was on email systems, so I'm sure about it.
    Your filter system is simply like this:
    When I select: If Body Contains "Й" (for example) Fail with message
    In your code you simply check:

    varMsgBody is the variable holding the whole email message body
    Й is the Russian character I entered in the filter list (among with some others)

    If varMsgBody.Contains("Й") = True Then
    'Fail with message
    End If


    While you should also:

    If varMsgBody.Contains("Й") = True Or varMsgBody.Contains(Uri.EscapeUriString("Й")) = True Then
    'Fail with message
    End If


    Because in the inner message source Й is converted to =D0=99 so you should check both for the entered character and it's HTML encoded form so your code logic works in all conditions with all characters.
    Please send an email to my address registered on my profile and I allow you to access my cPanel and make a test account, then send an HTML email to it containing my blocked characters and you see that the filter does not work as expected.

    The reason this section seems important is that we are getting many many SPAM everyday in Chinese/Russian and we prefer not to receive any email with those characters in Body.
    But this filter while is excellent but is not working as expected.

    *** Please note that Uri.EscapeUriString("Й") in .NET converts Й to =D0=99
    Unfortunately I don't know the equivalent in Perl
     
    Stop hovering to collapse... Click to collapse... Hover to expand... Click to expand...
    #12 omidsolo, May 2, 2019
    Last edited: May 3, 2019
  13. cPanelMichael

    cPanelMichael Technical Support Community Manager Staff Member

    Joined:
    Apr 11, 2011
    Messages:
    47,325
    Likes Received:
    2,157
    Trophy Points:
    363
    cPanel Access Level:
    Root Administrator
    Twitter:
    Hello @omidsolo,

    Thank you for providing us with additional information.

    You are correct about the Base64-encoded content. Exim will not automatically decode the Base64-encoded content during the filter rule processing with HTML emails. You can find some additional discussion of this topic on the following thread:

    Block custom attachments EXIM issue

    As an alternative, you can install and use the SpamAssassin TextCat plugin to filter out emails using specific languages. You can read more about this plugin at:

    Mail::SpamAssassin::Plugin::TextCat - TextCat language guesser

    To install it, ask your hosting provider to add the following line to the /etc/mail/spamassassin/init.pre file:

    Code:
    loadplugin Mail::SpamAssassin::Plugin::TextCat
    Your provider will also need to restart SpamAssassin with the following command after adding the above entry:

    Code:
    /scripts/restartsrv_spamd
    Once this plugin is installed, you can add and customize the "ok_languages" SpamAssassin configuration option (documentation on this option is found on the TextCat plugin link above) to your account's /home/username/.spamassassin/user_prefs file.

    Let me know if this helps.

    Thank you.
     
    Stop hovering to collapse... Click to collapse... Hover to expand... Click to expand...
  14. omidsolo

    omidsolo Member

    Joined:
    Apr 29, 2019
    Messages:
    16
    Likes Received:
    0
    Trophy Points:
    1
    Location:
    US
    cPanel Access Level:
    Website Owner
    Please note that usually Base64 is used on attachments, which are not my concern here.
    I am talking about Body based filtering, that one is encoded as Quoted Printable, for example we set a new global rule:
    If Body Contains "Й" Fail with message
    This will not work even if you fill the body with hundreds of "Й" characters
    Because in HTML email "Й" is converted to =D0=99
    So the filter just works for ancient plain text emails, not HTML ones!
    I also told the very quick way to fix it, just add one condition to the IF, check for its encoded form as well.

    If varMsgBody.Contains("Й") = True Then
    'Fail with message
    End If

    update to:

    If varMsgBody.Contains("Й") = True Or varMsgBody.Contains(Uri.EscapeUriString("Й")) = True Then
    'Fail with message
    End If


    That's as easy as ABC to add support for it in a future update, just please forward my whole messages to a real person who is a developer in charge.
    -
    I will try that other method by then :)
     
    Stop hovering to collapse... Click to collapse... Hover to expand... Click to expand...
  15. omidsolo

    omidsolo Member

    Joined:
    Apr 29, 2019
    Messages:
    16
    Likes Received:
    0
    Trophy Points:
    1
    Location:
    US
    cPanel Access Level:
    Website Owner
    OK I asked my hosting provider to install TextCat plugin as suggested, after checking with your support.
    I was almost sure it will not work that way, the only thing in this world I'm sure of, are email systems.

    There is no certain way of detecting an email language but "language" or "encoding" tag!
    Which is useless most of the times, since a Russian email can be UTF-8!
    Even if you have some other algorithm, it is not working, my email body was all Russian characters and delivered perfectly.

    I even did provide code samples to show your code logic fault but no one cares about the bug or code malfunction!
     
    Stop hovering to collapse... Click to collapse... Hover to expand... Click to expand...
  16. cPanelMichael

    cPanelMichael Technical Support Community Manager Staff Member

    Joined:
    Apr 11, 2011
    Messages:
    47,325
    Likes Received:
    2,157
    Trophy Points:
    363
    cPanel Access Level:
    Root Administrator
    Twitter:
    Stop hovering to collapse... Click to collapse... Hover to expand... Click to expand...
    #16 cPanelMichael, May 8, 2019
    Last edited: May 29, 2019
  17. omidsolo

    omidsolo Member

    Joined:
    Apr 29, 2019
    Messages:
    16
    Likes Received:
    0
    Trophy Points:
    1
    Location:
    US
    cPanel Access Level:
    Website Owner
    Well thanks, done, 12236283 & 12236375, I hope someone understand me finally, and care about those bugs! End of story :)
     
    Stop hovering to collapse... Click to collapse... Hover to expand... Click to expand...
Loading...

Share This Page

  1. This site uses cookies to help personalise content, tailor your experience and to keep you logged in if you register.
    By continuing to use this site, you are consenting to our use of cookies.
    Dismiss Notice