Please whitelist cPanel in your adblocker so that you’re able to see our version release promotions, thanks!

The Community Forums

Interact with an entire community of cPanel & WHM users!

Max Defer Percentage

Discussion in 'General Discussion' started by sparek-3, Jun 5, 2018.

  1. sparek-3

    sparek-3 Well-Known Member

    Joined:
    Aug 10, 2002
    Messages:
    1,719
    Likes Received:
    98
    Trophy Points:
    328
    cPanel Access Level:
    Root Administrator
    What's the story with

    Cannot validate setting for “max_defer_fail_percentage”: “125”.

    When trying to create a new account or add a new package?

    Where is this validating at? Seemed to work with previous versions of cPanel. Doesn't work with cPanel 11.70.0.48
     
  2. sparek-3

    sparek-3 Well-Known Member

    Joined:
    Aug 10, 2002
    Messages:
    1,719
    Likes Received:
    98
    Trophy Points:
    328
    cPanel Access Level:
    Root Administrator
    Seems to work when you create an account through the API. But if you go through the WHM interface, it doesn't work.

    Does anybody test anything any more?
     
  3. cPanelLauren

    cPanelLauren Forums Analyst
    Staff Member

    Joined:
    Nov 14, 2017
    Messages:
    2,230
    Likes Received:
    160
    Trophy Points:
    143
    Location:
    Houston
    cPanel Access Level:
    DataCenter Provider
    Hi @sparek-3

    I've been trying to replicate this actually. I had someone else mention it on the forums but I wasn't able to. Can you give me the exact steps you're taking that prompt the error?
     
    Stop hovering to collapse... Click to collapse... Hover to expand... Click to expand...
  4. sparek-3

    sparek-3 Well-Known Member

    Joined:
    Aug 10, 2002
    Messages:
    1,719
    Likes Received:
    98
    Trophy Points:
    328
    cPanel Access Level:
    Root Administrator
    It may be something within the configuration, but it's odd that it works through the API but not through the WHM interface. That would seem to indicate that someone forgot to dot their t's and cross their i's.

    Create a package - /var/cpanel/packages/pack1

    BWLIMIT=102400
    CGI=y
    CPMOD=paper_lantern
    DIGESTAUTH=n
    FEATURELIST=default
    FRONTPAGE=n
    HASSHELL=n
    IP=n
    LANG=en
    MAXADDON=unlimited
    MAXFTP=unlimited
    MAXLST=0
    MAXPARK=unlimited
    MAXPOP=unlimited
    MAXSQL=unlimited
    MAXSUB=unlimited
    MAX_DEFER_FAIL_PERCENTAGE=125
    MAX_EMAIL_PER_HOUR=100
    QUOTA=10240


    Now log into root's WHM.

    Click Create a New Account

    Domain: mytestdomain.com
    Username: mytest
    Password: myextremelyweakpassword
    Package: pack1


    Click Create

    Account Creation Status: failed
    Cannot validate setting for “max_defer_fail_percentage”: “125”.


    Now try it through the API:

    From root's shell:

    /usr/local/cpanel/bin/whmapi1 createacct username=mytest domain=mytestdomain.com plan=pack1 password=myextremelyweakpassword useregns=0 reseller=0

    It creates without a hitch.

    It's entirely possible that I have something not set correctly in /var/cpanel/cpanel.config or where ever else configuration files might be laying. But wouldn't that stop it from working on the API as well?

    (Note: I didn't actually use myextremelyweakpassword as my password)

    Coincidentally, you can't add a package through the WHM with a Maximum percentage of failed or deferred messages a domain may send per hour higher than 100.

    Log into the WHM

    Click Add a Package

    and try to set the Maximum percentage of failed or deferred messages a domain may send per hour value higher than 100. It gives the same Cannot validate setting for “max_defer_fail_percentage”: “XXX”. error.
     
  5. cPanelLauren

    cPanelLauren Forums Analyst
    Staff Member

    Joined:
    Nov 14, 2017
    Messages:
    2,230
    Likes Received:
    160
    Trophy Points:
    143
    Location:
    Houston
    cPanel Access Level:
    DataCenter Provider
    What I'm thinking happened is the ability to set this over 100% has been removed (but just through the UI)

    Based on what you mentioned here:

    and here:

    I went ahead at checked this value in tweak settings and sure enough I can't set it to a value over 100%:

    I also was able to replicate when creating a package. Since you can't create a package with this value set to 125% (except through the API) I can assert that the account creation process is failing due to previous package values having that set.

    I will get a case opened for this @sparek-3 thanks for getting me pointed in the right direction in order to replicate it.


    Well darnit there goes that idea :/
     
    Stop hovering to collapse... Click to collapse... Hover to expand... Click to expand...
  6. sparek-3

    sparek-3 Well-Known Member

    Joined:
    Aug 10, 2002
    Messages:
    1,719
    Likes Received:
    98
    Trophy Points:
    328
    cPanel Access Level:
    Root Administrator
    That's kind of what I thought might've been happening.

    The defer settings has always been an over 100% value, far as I can recall. If you set the max sending limit to 100 and the defer limit to 125, then this essentially meant it would queue up 25% of the messages over your 100% limit, or in this case, 25 messages.

    If this is changing to having to be set at just 25%, then that would seem to break a lot of backward compatibility.

    Kind of frustrating that this bug managed to make it this far in the release cycle of cPanel 70 - although I suppose it could have been in 68 - I skipped over 68. It just seems like this is something that should have been caught a bit earlier on. Is anybody actually using EDGE or CURRENT?
     
  7. cPanelLauren

    cPanelLauren Forums Analyst
    Staff Member

    Joined:
    Nov 14, 2017
    Messages:
    2,230
    Likes Received:
    160
    Trophy Points:
    143
    Location:
    Houston
    cPanel Access Level:
    DataCenter Provider
    I know I didn't receive any support requests on it until v70 so I don't believe it was occurring in v68.

    It does seem similar to some of the changes that were done for the SpamAssassin revamp (as far as the integer errors) and I wonder if it's related, none the less I know they work really hard to find issues with the product before it gets released. Our QA department is pretty great at that, sometimes I think the perception of what is meant to be there/impacts users and what they're testing against is a bit different as it's hard to know some of those things beforehand.

    I'll get you the case number as soon as I have it opened though and I'll follow up here once it's complete.
     
    Stop hovering to collapse... Click to collapse... Hover to expand... Click to expand...
  8. sparek-3

    sparek-3 Well-Known Member

    Joined:
    Aug 10, 2002
    Messages:
    1,719
    Likes Received:
    98
    Trophy Points:
    328
    cPanel Access Level:
    Root Administrator
    Well, I know from a developer's standpoint, it can be difficult to see issues that you yourself create. I suspect the people that are actually developing cPanel and writing the code, probably don't run web hosting companies or have time to do so. I'm sure they try to catch as many bugs as they can, but some are going to slip through simply because a developer isn't going to know the real world click by click procedure that typical users are going to use.

    The QA department may also be affected by this. I suspect none of them actually run a web hosting company or have time.

    But I thought the point of having EDGE and CURRENT (and to a lesser extent RELEASE) was to get real world insight into what may or may not need to be addressed. I suspect EDGE is mostly in use by 3rd party plugin developers who want their hands on the next version of cPanel as quickly as possible. I doubt anybody runs EDGE in a production environment or any where close to a normal web hosting business, so I'm willing to give EDGE users a pass on not finding bugs/issues. So that leaves CURRENT. Is anybody actually using CURRENT in a production environment? To find bugs and issues that need to be addressed before that version of cPanel reaches more of the masses in RELEASE? I suppose you could make the same argument for RELEASE -> STABLE, but I suspect a large majority of cPanel users run RELEASE.

    This, to me, would seem to mean that having CURRENT is mostly pointless. If nobody is using it, and nobody is finding bugs and issues before that version of cPanel hits RELEASE, then what's the point of having it? Or perhaps you need to incentivize more users to use CURRENT and find bugs/issues before that version of cPanel reaches the masses in RELEASE. Or perhaps abolish CURRENT, make STABLE the recommended tier and use RELEASE as your en masse testing.

    I'm completely off-topic with this rant, but it's just been something that has always puzzled me.
     
  9. cPanelLauren

    cPanelLauren Forums Analyst
    Staff Member

    Joined:
    Nov 14, 2017
    Messages:
    2,230
    Likes Received:
    160
    Trophy Points:
    143
    Location:
    Houston
    cPanel Access Level:
    DataCenter Provider
    Hi @sparek-3

    I just wanted to let you know that I've opened CPANEL-20909 for this issue. I've also noted in the case we should be making changes like this uniformly - if it's unable to be used in the UI it shouldn't be able to be done through WHMAPI1

    There are actually quite a few servers running EDGE in production as well as CURRENT (by quite a few I mean a lot more than you'd think).

    I found an inquiry to our development that suggested someone else asked if this change was intentional and it appears that it was with the explanation
    I think the symptoms you're seeing are an unintended consequence that may have been brought up previously but ruled out as an issue due to the fact the change was intentional, though I don't think the WHMAPI1 should allow account creation.
     
    Stop hovering to collapse... Click to collapse... Hover to expand... Click to expand...
  10. sparek-3

    sparek-3 Well-Known Member

    Joined:
    Aug 10, 2002
    Messages:
    1,719
    Likes Received:
    98
    Trophy Points:
    328
    cPanel Access Level:
    Root Administrator
    That's worrisome.

    Keep in mind, or at least as far as I know, all packages from previous versions of cPanel have had deferred percentages set over 100%, i.e. 125% meaning 25% of messages over the limit are deferred. I'll admit that could have been poor logic to begin with, but it's the way it has been for quite some time. If you move away from that, it's going to break a lot of backwards compatibility. At least that's my thoughts.

    While it's impossible to always think of every consequence and hindsight is always 20/20, but this is also a learning experience. Why were values over 100% ever allowed? What was 125% the default or recommended setting for so long?
     
  11. cPanelNick

    cPanelNick Administrator
    Staff Member

    Joined:
    Mar 9, 2015
    Messages:
    3,480
    Likes Received:
    30
    Trophy Points:
    158
    cPanel Access Level:
    DataCenter Provider
    Hi sparek-3,

    The email_send_limits_defer_cutoff which defaults to 125% (server wide, not configurable by package) is not related to the max_defer_fail_percentage (configurable by package/account/domain) setting. email_send_limits_defer_cutoff controls the point at where the system starts rejecting email instead of queuing it once a user reaches the maximum allowed emails per hour. I understand this is a bit confusing because both settings have the word "defer" in them.

    For example:
    When email_send_limits_defer_cutoff is set to 125% when the allowed emails per hour reach 100%, the outgoing email will be queued. Once allowed emails per hour reach 125%, the outgoing email will be rejected.

    When email_send_limits_defer_cutoff is set to 100% when the allowed emails per hour reach 100% outgoing email will be rejected.

    As for the max_defer_fail_percentage setting, this should never be more than 100% as its calculated as a percentage of the total email sent in an hour:
    Code:
    max_defer_fail_percentage = CURRENT_DEFER_FAIL_PER_HOUR / (CURRENT_SUCCESSFUL_EMAILS_PER_HOUR + CURRENT_DEFER_FAIL_PER_HOUR)
     
  12. sparek-3

    sparek-3 Well-Known Member

    Joined:
    Aug 10, 2002
    Messages:
    1,719
    Likes Received:
    98
    Trophy Points:
    328
    cPanel Access Level:
    Root Administrator
    Yep! definitely confusing.

    What's the difference between email_send_limits_defer_cutoff in /var/cpanel/cpanel.config
    and
    MAX_DEFER_FAIL_PERCENTAGE in packages in /var/cpanel/packages ?

    What does MAX_DEFER_FAIL_PERCENTAGE control?

    If email_send_limits_defer_cutoff is controlling the point at which messages get deferred, then what is a per package MAX_DEFER_FAIL_PERCENTAGE doing?

    If email_send_limits_defer_cutoff is set at 125% and a specific account with a package MAX_DEFER_FAIL_PERCENTAGE set to 50% starts sending out mail, which percentage of messages will be deferred?

    I just know that we have packages and resellers have packages that have MAX_DEFER_FAIL_PERCENTAGE set at 125 from previous versions of cPanel.

    This seems more like a case of "let's provide configure the hell out of it options" which has drained usability. Sure you can configure the hell out of it, but nobody knows what the hell they are configuring.
     
  13. cPanelNick

    cPanelNick Administrator
    Staff Member

    Joined:
    Mar 9, 2015
    Messages:
    3,480
    Likes Received:
    30
    Trophy Points:
    158
    cPanel Access Level:
    DataCenter Provider
    email_send_limits_defer_cutoff controls what happens when maxemailsperhour is reached

    MAX_DEFER_FAIL_PERCENTAGE is the value that will be used in /var/cpanel/users/<username> when a user is created with that package

    This maximum percentage of a domain’s outgoing mail that can consist of failed or deferred messages before messages will be rejected. The newer email_outbound_spam_detect_enable (Monitor the number of unique recipients per hour to detect potential spammers.) feature is likely far more useful.

    MAX_DEFER_FAIL_PERCENTAGE is another way to detect outgoing spamming by looking at which percentage of messages are failures. It is not related to email_send_limits_defer_cutoff and does not use this setting as once it is reached it will block all outbound messages for the rest of the hour.

    The email_send_limits_defer_cutoff setting controls what happens when the maxemailsperhour setting is reached. When the maximum emails per hour is reached messages will be deferred (queued). When the maximum emails per hour reaches 125% messages will be rejected.

    As for MAX_DEFER_FAIL_PERCENTAGE: If more than 50% of messages sent in the hour are failures or deferrals email will be rejected for the rest of the hour.

    We are planning on automatically normalizing values > 100 to be 100 with case CPANEL-20909 in order to address this.

    The outbound spam monitoring system was designed to replace the MAX_DEFER_FAIL_PERCENTAGE system as it proved to be too confusing. There will likely be a few more incremental improvements to the outbound spam monitoring system before we consider deprecating the MAX_DEFER_FAIL_PERCENTAGE system.
     
    #13 cPanelNick, Jun 6, 2018
    Last edited: Jun 6, 2018
    cPanelLauren likes this.
  14. sparek-3

    sparek-3 Well-Known Member

    Joined:
    Aug 10, 2002
    Messages:
    1,719
    Likes Received:
    98
    Trophy Points:
    328
    cPanel Access Level:
    Root Administrator
    I have to say, you've made all of this thoroughly confusing with all the different configuration options. I'll just normalize these values to 100 myself and call it a day.
     
  15. cPanelLauren

    cPanelLauren Forums Analyst
    Staff Member

    Joined:
    Nov 14, 2017
    Messages:
    2,230
    Likes Received:
    160
    Trophy Points:
    143
    Location:
    Houston
    cPanel Access Level:
    DataCenter Provider
    HI @sparek-3

    I just wanted to let you know that we resolved this issue with a fix for this in v70.0.51 which was moved to RELEASE last night. Please let us know if you notice any issues with it!

    Thanks!
     
    Stop hovering to collapse... Click to collapse... Hover to expand... Click to expand...
Loading...

Share This Page

  1. This site uses cookies to help personalise content, tailor your experience and to keep you logged in if you register.
    By continuing to use this site, you are consenting to our use of cookies.
    Dismiss Notice