Please whitelist cPanel in your adblocker so that you’re able to see our version release promotions, thanks!

The Community Forums

Interact with an entire community of cPanel & WHM users!

Max Defer Percentage

Discussion in 'General Discussion' started by sparek-3, Jun 5, 2018.

Tags:
  1. sparek-3

    sparek-3 Well-Known Member

    Joined:
    Aug 10, 2002
    Messages:
    1,762
    Likes Received:
    116
    Trophy Points:
    343
    cPanel Access Level:
    Root Administrator
    What's the story with

    Cannot validate setting for “max_defer_fail_percentage”: “125”.

    When trying to create a new account or add a new package?

    Where is this validating at? Seemed to work with previous versions of cPanel. Doesn't work with cPanel 11.70.0.48
     
  2. sparek-3

    sparek-3 Well-Known Member

    Joined:
    Aug 10, 2002
    Messages:
    1,762
    Likes Received:
    116
    Trophy Points:
    343
    cPanel Access Level:
    Root Administrator
    Seems to work when you create an account through the API. But if you go through the WHM interface, it doesn't work.

    Does anybody test anything any more?
     
  3. cPanelLauren

    cPanelLauren Forums Analyst II
    Staff Member

    Joined:
    Nov 14, 2017
    Messages:
    3,137
    Likes Received:
    222
    Trophy Points:
    173
    Location:
    Houston
    cPanel Access Level:
    DataCenter Provider
    Hi @sparek-3

    I've been trying to replicate this actually. I had someone else mention it on the forums but I wasn't able to. Can you give me the exact steps you're taking that prompt the error?
     
    Stop hovering to collapse... Click to collapse... Hover to expand... Click to expand...
  4. sparek-3

    sparek-3 Well-Known Member

    Joined:
    Aug 10, 2002
    Messages:
    1,762
    Likes Received:
    116
    Trophy Points:
    343
    cPanel Access Level:
    Root Administrator
    It may be something within the configuration, but it's odd that it works through the API but not through the WHM interface. That would seem to indicate that someone forgot to dot their t's and cross their i's.

    Create a package - /var/cpanel/packages/pack1

    BWLIMIT=102400
    CGI=y
    CPMOD=paper_lantern
    DIGESTAUTH=n
    FEATURELIST=default
    FRONTPAGE=n
    HASSHELL=n
    IP=n
    LANG=en
    MAXADDON=unlimited
    MAXFTP=unlimited
    MAXLST=0
    MAXPARK=unlimited
    MAXPOP=unlimited
    MAXSQL=unlimited
    MAXSUB=unlimited
    MAX_DEFER_FAIL_PERCENTAGE=125
    MAX_EMAIL_PER_HOUR=100
    QUOTA=10240


    Now log into root's WHM.

    Click Create a New Account

    Domain: mytestdomain.com
    Username: mytest
    Password: myextremelyweakpassword
    Package: pack1


    Click Create

    Account Creation Status: failed
    Cannot validate setting for “max_defer_fail_percentage”: “125”.


    Now try it through the API:

    From root's shell:

    /usr/local/cpanel/bin/whmapi1 createacct username=mytest domain=mytestdomain.com plan=pack1 password=myextremelyweakpassword useregns=0 reseller=0

    It creates without a hitch.

    It's entirely possible that I have something not set correctly in /var/cpanel/cpanel.config or where ever else configuration files might be laying. But wouldn't that stop it from working on the API as well?

    (Note: I didn't actually use myextremelyweakpassword as my password)

    Coincidentally, you can't add a package through the WHM with a Maximum percentage of failed or deferred messages a domain may send per hour higher than 100.

    Log into the WHM

    Click Add a Package

    and try to set the Maximum percentage of failed or deferred messages a domain may send per hour value higher than 100. It gives the same Cannot validate setting for “max_defer_fail_percentage”: “XXX”. error.
     
  5. cPanelLauren

    cPanelLauren Forums Analyst II
    Staff Member

    Joined:
    Nov 14, 2017
    Messages:
    3,137
    Likes Received:
    222
    Trophy Points:
    173
    Location:
    Houston
    cPanel Access Level:
    DataCenter Provider
    What I'm thinking happened is the ability to set this over 100% has been removed (but just through the UI)

    Based on what you mentioned here:

    and here:

    I went ahead at checked this value in tweak settings and sure enough I can't set it to a value over 100%:

    I also was able to replicate when creating a package. Since you can't create a package with this value set to 125% (except through the API) I can assert that the account creation process is failing due to previous package values having that set.

    I will get a case opened for this @sparek-3 thanks for getting me pointed in the right direction in order to replicate it.


    Well darnit there goes that idea :/
     
    Stop hovering to collapse... Click to collapse... Hover to expand... Click to expand...
  6. sparek-3

    sparek-3 Well-Known Member

    Joined:
    Aug 10, 2002
    Messages:
    1,762
    Likes Received:
    116
    Trophy Points:
    343
    cPanel Access Level:
    Root Administrator
    That's kind of what I thought might've been happening.

    The defer settings has always been an over 100% value, far as I can recall. If you set the max sending limit to 100 and the defer limit to 125, then this essentially meant it would queue up 25% of the messages over your 100% limit, or in this case, 25 messages.

    If this is changing to having to be set at just 25%, then that would seem to break a lot of backward compatibility.

    Kind of frustrating that this bug managed to make it this far in the release cycle of cPanel 70 - although I suppose it could have been in 68 - I skipped over 68. It just seems like this is something that should have been caught a bit earlier on. Is anybody actually using EDGE or CURRENT?
     
  7. cPanelLauren

    cPanelLauren Forums Analyst II
    Staff Member

    Joined:
    Nov 14, 2017
    Messages:
    3,137
    Likes Received:
    222
    Trophy Points:
    173
    Location:
    Houston
    cPanel Access Level:
    DataCenter Provider
    I know I didn't receive any support requests on it until v70 so I don't believe it was occurring in v68.

    It does seem similar to some of the changes that were done for the SpamAssassin revamp (as far as the integer errors) and I wonder if it's related, none the less I know they work really hard to find issues with the product before it gets released. Our QA department is pretty great at that, sometimes I think the perception of what is meant to be there/impacts users and what they're testing against is a bit different as it's hard to know some of those things beforehand.

    I'll get you the case number as soon as I have it opened though and I'll follow up here once it's complete.
     
    Stop hovering to collapse... Click to collapse... Hover to expand... Click to expand...
  8. sparek-3

    sparek-3 Well-Known Member

    Joined:
    Aug 10, 2002
    Messages:
    1,762
    Likes Received:
    116
    Trophy Points:
    343
    cPanel Access Level:
    Root Administrator
    Well, I know from a developer's standpoint, it can be difficult to see issues that you yourself create. I suspect the people that are actually developing cPanel and writing the code, probably don't run web hosting companies or have time to do so. I'm sure they try to catch as many bugs as they can, but some are going to slip through simply because a developer isn't going to know the real world click by click procedure that typical users are going to use.

    The QA department may also be affected by this. I suspect none of them actually run a web hosting company or have time.

    But I thought the point of having EDGE and CURRENT (and to a lesser extent RELEASE) was to get real world insight into what may or may not need to be addressed. I suspect EDGE is mostly in use by 3rd party plugin developers who want their hands on the next version of cPanel as quickly as possible. I doubt anybody runs EDGE in a production environment or any where close to a normal web hosting business, so I'm willing to give EDGE users a pass on not finding bugs/issues. So that leaves CURRENT. Is anybody actually using CURRENT in a production environment? To find bugs and issues that need to be addressed before that version of cPanel reaches more of the masses in RELEASE? I suppose you could make the same argument for RELEASE -> STABLE, but I suspect a large majority of cPanel users run RELEASE.

    This, to me, would seem to mean that having CURRENT is mostly pointless. If nobody is using it, and nobody is finding bugs and issues before that version of cPanel hits RELEASE, then what's the point of having it? Or perhaps you need to incentivize more users to use CURRENT and find bugs/issues before that version of cPanel reaches the masses in RELEASE. Or perhaps abolish CURRENT, make STABLE the recommended tier and use RELEASE as your en masse testing.

    I'm completely off-topic with this rant, but it's just been something that has always puzzled me.
     
  9. cPanelLauren

    cPanelLauren Forums Analyst II
    Staff Member

    Joined:
    Nov 14, 2017
    Messages:
    3,137
    Likes Received:
    222
    Trophy Points:
    173
    Location:
    Houston
    cPanel Access Level:
    DataCenter Provider
    Hi @sparek-3

    I just wanted to let you know that I've opened CPANEL-20909 for this issue. I've also noted in the case we should be making changes like this uniformly - if it's unable to be used in the UI it shouldn't be able to be done through WHMAPI1

    There are actually quite a few servers running EDGE in production as well as CURRENT (by quite a few I mean a lot more than you'd think).

    I found an inquiry to our development that suggested someone else asked if this change was intentional and it appears that it was with the explanation
    I think the symptoms you're seeing are an unintended consequence that may have been brought up previously but ruled out as an issue due to the fact the change was intentional, though I don't think the WHMAPI1 should allow account creation.
     
    Stop hovering to collapse... Click to collapse... Hover to expand... Click to expand...
  10. sparek-3

    sparek-3 Well-Known Member

    Joined:
    Aug 10, 2002
    Messages:
    1,762
    Likes Received:
    116
    Trophy Points:
    343
    cPanel Access Level:
    Root Administrator
    That's worrisome.

    Keep in mind, or at least as far as I know, all packages from previous versions of cPanel have had deferred percentages set over 100%, i.e. 125% meaning 25% of messages over the limit are deferred. I'll admit that could have been poor logic to begin with, but it's the way it has been for quite some time. If you move away from that, it's going to break a lot of backwards compatibility. At least that's my thoughts.

    While it's impossible to always think of every consequence and hindsight is always 20/20, but this is also a learning experience. Why were values over 100% ever allowed? What was 125% the default or recommended setting for so long?
     
  11. cPanelNick

    cPanelNick Administrator
    Staff Member

    Joined:
    Mar 9, 2015
    Messages:
    3,480
    Likes Received:
    30
    Trophy Points:
    158
    cPanel Access Level:
    DataCenter Provider
    Hi sparek-3,

    The email_send_limits_defer_cutoff which defaults to 125% (server wide, not configurable by package) is not related to the max_defer_fail_percentage (configurable by package/account/domain) setting. email_send_limits_defer_cutoff controls the point at where the system starts rejecting email instead of queuing it once a user reaches the maximum allowed emails per hour. I understand this is a bit confusing because both settings have the word "defer" in them.

    For example:
    When email_send_limits_defer_cutoff is set to 125% when the allowed emails per hour reach 100%, the outgoing email will be queued. Once allowed emails per hour reach 125%, the outgoing email will be rejected.

    When email_send_limits_defer_cutoff is set to 100% when the allowed emails per hour reach 100% outgoing email will be rejected.

    As for the max_defer_fail_percentage setting, this should never be more than 100% as its calculated as a percentage of the total email sent in an hour:
    Code:
    max_defer_fail_percentage = CURRENT_DEFER_FAIL_PER_HOUR / (CURRENT_SUCCESSFUL_EMAILS_PER_HOUR + CURRENT_DEFER_FAIL_PER_HOUR)
     
  12. sparek-3

    sparek-3 Well-Known Member

    Joined:
    Aug 10, 2002
    Messages:
    1,762
    Likes Received:
    116
    Trophy Points:
    343
    cPanel Access Level:
    Root Administrator
    Yep! definitely confusing.

    What's the difference between email_send_limits_defer_cutoff in /var/cpanel/cpanel.config
    and
    MAX_DEFER_FAIL_PERCENTAGE in packages in /var/cpanel/packages ?

    What does MAX_DEFER_FAIL_PERCENTAGE control?

    If email_send_limits_defer_cutoff is controlling the point at which messages get deferred, then what is a per package MAX_DEFER_FAIL_PERCENTAGE doing?

    If email_send_limits_defer_cutoff is set at 125% and a specific account with a package MAX_DEFER_FAIL_PERCENTAGE set to 50% starts sending out mail, which percentage of messages will be deferred?

    I just know that we have packages and resellers have packages that have MAX_DEFER_FAIL_PERCENTAGE set at 125 from previous versions of cPanel.

    This seems more like a case of "let's provide configure the hell out of it options" which has drained usability. Sure you can configure the hell out of it, but nobody knows what the hell they are configuring.
     
  13. cPanelNick

    cPanelNick Administrator
    Staff Member

    Joined:
    Mar 9, 2015
    Messages:
    3,480
    Likes Received:
    30
    Trophy Points:
    158
    cPanel Access Level:
    DataCenter Provider
    email_send_limits_defer_cutoff controls what happens when maxemailsperhour is reached

    MAX_DEFER_FAIL_PERCENTAGE is the value that will be used in /var/cpanel/users/<username> when a user is created with that package

    This maximum percentage of a domain’s outgoing mail that can consist of failed or deferred messages before messages will be rejected. The newer email_outbound_spam_detect_enable (Monitor the number of unique recipients per hour to detect potential spammers.) feature is likely far more useful.

    MAX_DEFER_FAIL_PERCENTAGE is another way to detect outgoing spamming by looking at which percentage of messages are failures. It is not related to email_send_limits_defer_cutoff and does not use this setting as once it is reached it will block all outbound messages for the rest of the hour.

    The email_send_limits_defer_cutoff setting controls what happens when the maxemailsperhour setting is reached. When the maximum emails per hour is reached messages will be deferred (queued). When the maximum emails per hour reaches 125% messages will be rejected.

    As for MAX_DEFER_FAIL_PERCENTAGE: If more than 50% of messages sent in the hour are failures or deferrals email will be rejected for the rest of the hour.

    We are planning on automatically normalizing values > 100 to be 100 with case CPANEL-20909 in order to address this.

    The outbound spam monitoring system was designed to replace the MAX_DEFER_FAIL_PERCENTAGE system as it proved to be too confusing. There will likely be a few more incremental improvements to the outbound spam monitoring system before we consider deprecating the MAX_DEFER_FAIL_PERCENTAGE system.
     
    #13 cPanelNick, Jun 6, 2018
    Last edited: Jun 6, 2018
    cPanelLauren likes this.
  14. sparek-3

    sparek-3 Well-Known Member

    Joined:
    Aug 10, 2002
    Messages:
    1,762
    Likes Received:
    116
    Trophy Points:
    343
    cPanel Access Level:
    Root Administrator
    I have to say, you've made all of this thoroughly confusing with all the different configuration options. I'll just normalize these values to 100 myself and call it a day.
     
  15. cPanelLauren

    cPanelLauren Forums Analyst II
    Staff Member

    Joined:
    Nov 14, 2017
    Messages:
    3,137
    Likes Received:
    222
    Trophy Points:
    173
    Location:
    Houston
    cPanel Access Level:
    DataCenter Provider
    HI @sparek-3

    I just wanted to let you know that we resolved this issue with a fix for this in v70.0.51 which was moved to RELEASE last night. Please let us know if you notice any issues with it!

    Thanks!
     
    Stop hovering to collapse... Click to collapse... Hover to expand... Click to expand...
  16. phil99

    phil99 Member

    Joined:
    Jun 10, 2018
    Messages:
    7
    Likes Received:
    1
    Trophy Points:
    3
    Location:
    UK
    cPanel Access Level:
    Root Administrator
    Could someone please clarify how the "The percentage of email messages (above the account’s hourly maximum) to queue and retry for delivery" setting is calculated?

    The help text beneath the setting says:

    "When an account exceeds the maximum number of emails it is allowed to send per hour, by default, any additional messages are queued for delivery and sent in the next hour. This setting allows you to limit the number of messages that will be queued by the system. For example, if you set this value to 125%, once the account reaches its hourly limit, Exim will queue any additional messages, up to 125% of the Max hourly emails per domain value. Once the account reaches 125% of the Max hourly emails per domain value, any additional outgoing messages are discarded."

    I read this as meaning that if you have the max emails per hour at 100, exim will queue a further 125 emails for delivery, and discard any more.

    However reference [1] gives the following example:

    "You set the Max hourly emails per domain option to 100, and you set the The percentage of email messages (above the account's hourly maximum) to queue and retry for delivery option to 200. The mail server applies the following rules to the outgoing messages from each domain:

    The mail server sends the first 100 outgoing messages from the domain.
    The mail server queues the next 100 outgoing messages from the domain.
    The mail server discards any additional outgoing messages from the domain.
    In the following hour, the mail server attempts to send all queued outgoing messages from the domain."

    Or using the same figures as above, 100 emails would be sent in the first hour and only 25 queued.

    So is my initial reading flawed, and the example given correct? The wording is quite confusing. Thanks.

    [1] How to Prevent Spam with Mail Limiting Features - cPanel Knowledge Base - cPanel Documentation
     
  17. 24x7server

    24x7server Well-Known Member

    Joined:
    Apr 17, 2013
    Messages:
    1,880
    Likes Received:
    89
    Trophy Points:
    78
    Location:
    India
    cPanel Access Level:
    Root Administrator
    Hi,

    -> I think it goes like this, hourly limit will make sure that mails under that limit are delivered. Mails going beyond that limit upto 125% will be queued.
    If 100 is the hourly limit, 100 mails will be delivered and 125% of 100 means 125 mails will be queued for further delivery for the next hour and 126 onwards for that hour will be discarded.
    225 mails will trigger for the given hour 100 will be delivered and 125 will be queued for delivery, but that will be delivered only in the next hour..
     
    Stop hovering to collapse... Click to collapse... Hover to expand... Click to expand...
  18. phil99

    phil99 Member

    Joined:
    Jun 10, 2018
    Messages:
    7
    Likes Received:
    1
    Trophy Points:
    3
    Location:
    UK
    cPanel Access Level:
    Root Administrator
    Well it seems I'm not the only one to find this confusing. But the answer to my question appears to be that yes, I read it wrong. From post #13:

    "The email_send_limits_defer_cutoff setting controls what happens when the maxemailsperhour setting is reached. When the maximum emails per hour is reached messages will be deferred (queued). When the maximum emails per hour reaches 125% messages will be rejected."

    So using my example, 100 mails will be delivered, a further 25 will be queued, and then any further messages will be rejected.
     
Loading...

Share This Page

  1. This site uses cookies to help personalise content, tailor your experience and to keep you logged in if you register.
    By continuing to use this site, you are consenting to our use of cookies.
    Dismiss Notice