The Community Forums

Interact with an entire community of cPanel & WHM users!
  1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Email over quota errors even after email is deleted more than 24 hours ago.

Discussion in 'E-mail Discussions' started by gkgcpanel, Dec 21, 2012.

  1. gkgcpanel

    gkgcpanel Well-Known Member

    Joined:
    Jun 6, 2007
    Messages:
    217
    Likes Received:
    0
    Trophy Points:
    16
    cPanel Access Level:
    DataCenter Provider
    I know that cPanel updates mail quota every 4 hours, (or is supposed to), and for the most part this is working just fine.

    But occasionally we get a user who is over quota on email, and goes into webmail, deletes/purges everything and even deletes purged files. Then 24 hours later, they are still over quota.

    There is nothing left in any of their mail folders, and they can not receive new email (everything bounces with over quota errors). Running:

    Code:
    /scripts/generate_maildirsize -confirm USERNAME
    
    solves the problem. Mail space goes back to None and the users can receive email again.

    Like I said, most of the time this works just fine, and only every once in a while do I have to force the maildirsize regenteration.

    So why is this not clearing itself out after 4 hours?
     
  2. gkgcpanel

    gkgcpanel Well-Known Member

    Joined:
    Jun 6, 2007
    Messages:
    217
    Likes Received:
    0
    Trophy Points:
    16
    cPanel Access Level:
    DataCenter Provider
    Re: Email over quota errors even after email is deleted more than 24 hours

    Anyone??? This is still happening on a daily basis, but only on 3 of our 20 servers and only certain accounts.
    No idea why!
     
  3. ruzbehraja

    ruzbehraja Well-Known Member

    Joined:
    May 19, 2011
    Messages:
    383
    Likes Received:
    7
    Trophy Points:
    18
    cPanel Access Level:
    Root Administrator
    I experienced the same thing with 1 user on my server who was continuously hitting the mailbox limit.

    The solution (temporary) is to delete the maildirsize file for that email id and reset the quota from cpanel.
     
  4. gkgcpanel

    gkgcpanel Well-Known Member

    Joined:
    Jun 6, 2007
    Messages:
    217
    Likes Received:
    0
    Trophy Points:
    16
    cPanel Access Level:
    DataCenter Provider
    Thanks, but I believe that running this:

    Code:
    /scripts/generate_maildirsize -confirm USERNAME
    
    will accomplish the same task.



     
  5. WhiteDog

    WhiteDog Well-Known Member

    Joined:
    Feb 19, 2008
    Messages:
    118
    Likes Received:
    0
    Trophy Points:
    16
    I've had the same problem with a user yesterday (on cPanel 11.36.0.11).
    His mailbox has been cleared for weeks yet cPanel still clmaimed 400+ MB was in it.

    This command solves the issue but i'm wondering why it was not correct in the first place.
    Which process / script is run every 4 hours to recalculate the size? Perhaps that one is not set up correctly?

    Can someine from cPanel comment?
     
  6. gkgcpanel

    gkgcpanel Well-Known Member

    Joined:
    Jun 6, 2007
    Messages:
    217
    Likes Received:
    0
    Trophy Points:
    16
    cPanel Access Level:
    DataCenter Provider
    Come on cPanel!!! Why does this happen and not recalculate the mail spool by itself? Why do we have to constantly run this generate_maildirsize command for random customers?? What is it that is not clearing the quota by itself...?

    We are getting complaints on this and it's very annoying to the customer to have to contact us to get a resolution that should be 100% automatic...

    If a customer clears out their mail, the quota should regenerate automatically, but in some cases it does NOT do that for hours or even days/weeks.

    Please provide a solution.
     
  7. cPanelMichael

    cPanelMichael Forums Analyst
    Staff Member

    Joined:
    Apr 11, 2011
    Messages:
    30,762
    Likes Received:
    662
    Trophy Points:
    113
    cPanel Access Level:
    Root Administrator
    Disk space usage for email accounts is recalculated every four hours. If this is not happening automatically, please open a support ticket so we can take a closer look:

    Submit A Ticket

    You can provide the ticket number here so we can update this thread with the outcome.

    Thank you.
     
  8. syno

    syno Member

    Joined:
    Jun 20, 2013
    Messages:
    14
    Likes Received:
    0
    Trophy Points:
    1
    cPanel Access Level:
    Root Administrator
    gkgcpanel I have the same problem in process of solving... Try to restart dovecot. (Like once per day.) Please tell if it helped to decrease amount of quota errors. My suspiction that it's bug in dovecot.
     
  9. cPanelMichael

    cPanelMichael Forums Analyst
    Staff Member

    Joined:
    Apr 11, 2011
    Messages:
    30,762
    Likes Received:
    662
    Trophy Points:
    113
    cPanel Access Level:
    Root Administrator
    Feel free to submit a ticket so we can attempt to reproduce the behavior on your server and open an internal case if necessary.

    Thank you.
     
  10. syno

    syno Member

    Joined:
    Jun 20, 2013
    Messages:
    14
    Likes Received:
    0
    Trophy Points:
    1
    cPanel Access Level:
    Root Administrator
    cPanelMichael. I don't know how to reproduce it. It sometimes appears and users complain. What is curiously similar to gkgcpanel's case is that some users seems to have it more frequently, while others never. I don't see how to report it to the ticket as this is not immediate issue, but when user/client complain I'm not gonna wait undefined time when issue will be possible 'solved' and do `generate_maildirsize' immediately. So there is nothign material to submit to the ticket... It's weird poblem and annoying.

    And I even think that when problem reappears and being submitted to the ticket CP person will just run generate_maildirsize and report problem solved. Isn't it? I remember I have that before. But it needs deeper investigation than that (and I can run generate_maildirsize myself).
     
  11. gkgcpanel

    gkgcpanel Well-Known Member

    Joined:
    Jun 6, 2007
    Messages:
    217
    Likes Received:
    0
    Trophy Points:
    16
    cPanel Access Level:
    DataCenter Provider
    I'm with Syno... This problem is so intermittent that it is very difficult to pin down. It happens on a few servers, and a few accounts, but not all accounts. It happens more frequently with a few of the accounts. There's no telling. And I did once open up a ticket on this problem (don't recall the number) and all that happened was the tech ran generate_maildirsize and responded with issue resolved. I already know that will solve the problem. What I'm trying to figure out is how does cPanel (or Dovecot) report the quota and how does it recalculate every 4 hours? what happens every 4 hours (a command/script is run or what?)

    We also can't wait for it to happen and then open a ticket and wait. Customers get really unhappy very quickly when they can't get email, so fixing it quickly is extrememly important and opening a ticket just means the customer has to wait that much longer.

    Since this problem has been reported by several people, it must be something that cPanel can test on their end... Set up an email account with a 1M mail spool, fill it up and clear it out and see what happens...

    Thanks
     
  12. syno

    syno Member

    Joined:
    Jun 20, 2013
    Messages:
    14
    Likes Received:
    0
    Trophy Points:
    1
    cPanel Access Level:
    Root Administrator
    I was not able to get info from the support on how cpanel regenerate `maildirsize' every 4 hours. (Problem is we don't contact developers directly, and support people just don't know such details.)
     
  13. cPanelMichael

    cPanelMichael Forums Analyst
    Staff Member

    Joined:
    Apr 11, 2011
    Messages:
    30,762
    Likes Received:
    662
    Trophy Points:
    113
    cPanel Access Level:
    Root Administrator
    The "maildirsize" files themselves are not something that are "generated" normally. The IMAP and Exim processes should adjust these files in real time as messages are added/removed. While the cPanel interface does state that usage is only calculated every four hours, updates to the email accounts page (pops.html) should be instantaneous in most cases.

    It is worth opening a support ticket if you notice email accounts are still behaving as if they are over their disk space quota after emails have been deleted. While reproducing an issue can be helpful, you can explain the issue continues and we can attempt to setup a test account on the server or review the existing accounts to see if we notice any specific problems.

    Thank you.
     
  14. syno

    syno Member

    Joined:
    Jun 20, 2013
    Messages:
    14
    Likes Received:
    0
    Trophy Points:
    1
    cPanel Access Level:
    Root Administrator
    I don't feel satisfied when filing ticket which will not be resolved. What I would like instead, is to contact developer who is motivated to solve problem. Right now I have maildirsize being wrong for one client since 12:15 (now 16:35) which is more than 4 hours. I wonder how maildirsize is supposedly recalculated every 4 hours? There is nothing in crontab, etc.

    Client deleted mail via pop3:

    Jul 5 12:15:47 x dovecot: POP3(x@x): Disconnected: Logged out top=0/0, retr=1/22536813, del=1/1, size=22536792, bytes=40/22536971

    Current maildirsize content:

    # cat /home/x/mail/x/x/maildirsize
    20971520S,0C
    294697 1
    151791 1
    -446488 -2
    177931 1
    -177931 -1
    293381 1
    -293381 -1
    3783 1
    -3783 -1
    229855 1
    -229855 -1
    10412 1
    -10412 -1
    670249 1
    -670249 -1
    7195 1
    -7195 -1
    14538 1
    -14538 -1
    612035 1
    -612035 -1
    22247802 1

    There is no /home/x/mail/x/x/maildirsize.lock

    That big mail is arrived at 12:14

    2013-07-05 12:14:21 [1003] 1Uv19i-0000GB-CK <= y@y H=y [y]:35216 P=esmtp S=22247684 id=1213641373012048@y T="y" from <y@y> for x@x

    So exim wrote maildirsize correctly, but dovecot failed to modify it when user deleted mail that was over quota.

    ps. I also added some debug output into dovecot but it so obscure and incomprehensible. If you or some developer want to know bit more details: basically dovecot-1.2.17/src/plugins/quota/quota-maildir.c::maildirsize_write() function got EAGAIN after calling file_dotlock_open_group() so it abandon attempt to write maildirsize and also don't log any error.

    But, probably that was not related to problem of not recalculating maildisize every 4 hours. And additional annoyance is that user who is over quota is not able to send mail too... as his email check return error. But why not let him/her at least send mails?


    root@y:# telnet 0 25
    Trying 0.0.0.0...
    Connected to 0 (0.0.0.0).
    Escape character is '^]'.
    220-y ESMTP Exim 4.80 #2 Fri, 05 Jul 2013 17:12:02 +0400
    220-We do not authorize the use of this system to transport unsolicited,
    220 and/or bulk e-mail.
    ehlo y
    250-y Hello localhost [127.0.0.1]
    250-SIZE 36700160
    250-8BITMIME
    250-PIPELINING
    250-AUTH PLAIN LOGIN
    250-STARTTLS
    250 HELP
    mail from:<x@x> <- this is mailbox that is over quota
    250 OK
    rcpt to:<root@y> <- there is valid email address w/o quota
    550-Verification failed for <x@x>
    550-Mailbox quota exceeded
    550 Sender verify failed
     
  15. gkgcpanel

    gkgcpanel Well-Known Member

    Joined:
    Jun 6, 2007
    Messages:
    217
    Likes Received:
    0
    Trophy Points:
    16
    cPanel Access Level:
    DataCenter Provider
    cPanel team... I think Syno is onto something with the bug on Dovecot.

    Just had another user complain that their mail quota is over and they had cleared out all email hours ago. Messages to them were bouncing with mailbox is over quota. I looked and pops.html did definitely show it was full.

    Now instead of running the generate-maildirsize script, I decided to restart dovecot with /scripts/restartsrv_dovecot

    Then I refreshed the email page (pops.html) and it now shoed 0 quota used. I sent a test message and it delivered.

    So what Syno mentioned above may very well be the culprit.

    Thanks
    Peter
     
  16. cPanelMichael

    cPanelMichael Forums Analyst
    Staff Member

    Joined:
    Apr 11, 2011
    Messages:
    30,762
    Likes Received:
    662
    Trophy Points:
    113
    cPanel Access Level:
    Root Administrator
    It's important to open a support ticket for an issue such as this. This will give us a ticket to flag to our development team if necessary, and it's reason to setup a test environment to reproduce the issue.

    Thank you.
     
  17. syno

    syno Member

    Joined:
    Jun 20, 2013
    Messages:
    14
    Likes Received:
    0
    Trophy Points:
    1
    cPanel Access Level:
    Root Administrator
    1. Sometimes when Exim deliver new mail there appear stale (i.e. not deleted) maildirsize.lock file with content like "29740<line break>/usr/local/cpanel/bin/eximadmin - waiting for lockfile" which is obviously cpanel related (and eximadmin source is not available). You can find a lot of such maildirsize.lock files on your system with similar content and very different timestamps using `locate maildirsize.lock' command. Try this:

    # locate maildirsize.lock | wc -l
    260

    # locate maildirsize.lock | xargs ls -ltr

    My oldest maildirsize.lock is May 23 12:59, and newest almost current time. This is probably a bug in cpanel.

    2. In some situations Dovecot abandon writing to maildirsize if such maildirsize.lock file is present. This maybe of interest for cpanel developers: dovecot-2.2: changeset 16591:24d6708cea76 Alas, this patch can not be applied straightforwardly to cpanel's dovecot because of version difference (cpanel's dovecot version is 1.2.17 and patch is for 2.2). But I currently test back-port of it on my system and will report after a week if there is any improvement.
     
  18. gkgcpanel

    gkgcpanel Well-Known Member

    Joined:
    Jun 6, 2007
    Messages:
    217
    Likes Received:
    0
    Trophy Points:
    16
    cPanel Access Level:
    DataCenter Provider
    Some more info. I don't believe this has anything to do with dovecot.

    I just had another user who was over quota, and messages to him were bouncing with same error.
    He does NOT use any email client, only Horde webmail. checked all folders (including trash) and everything was empty.
    Restarting dovecot did NOT have any effect this time. Regenerating the maildirsize however did clear it up as expected.

    I have several test accounts active, looking for this problem. If any of them get over quota and it does NOT resolve by itself, I will open up a ticket and let cPanel investigate. But until I can duplicate it with one of my test accounts, I can't wait on cPanel when a customer calls to complain...
     
  19. syno

    syno Member

    Joined:
    Jun 20, 2013
    Messages:
    14
    Likes Received:
    0
    Trophy Points:
    1
    cPanel Access Level:
    Root Administrator
    gkgcpanel, Webmail is using IMAP and IMAP is served by Dovecot. Yes, restarting Dovecot does not help resolving issue. I think here we have multiple problems at once, that's why it's so complicated issue. 1) Stale maildirsize.lock left by cpanel's eximadmin, 2) Dovecot unfortunate behavior, 3) Even with all that why cpanel does not correct mailboxes sizes every 4 hours as support say?

    ps. Two days since patch and I don't have single issue with maildirsize, so that experimental solution looks promising as of yet.
     
    #19 syno, Jul 12, 2013
    Last edited: Jul 12, 2013
  20. syno

    syno Member

    Joined:
    Jun 20, 2013
    Messages:
    14
    Likes Received:
    0
    Trophy Points:
    1
    cPanel Access Level:
    Root Administrator
    I tried this on many servers and found that non-zero value is only on single server. gkgcpanel can you check several of your servers?
     
Loading...

Share This Page