Please whitelist cPanel in your adblocker so that you’re able to see our version release promotions, thanks!

The Community Forums

Interact with an entire community of cPanel & WHM users!

Mailbox Compression and Maildir-->MDBox conversion

Discussion in 'E-mail Discussion' started by mtindor, Mar 2, 2018.

Tags:
  1. mtindor

    mtindor Well-Known Member

    Joined:
    Sep 14, 2004
    Messages:
    1,342
    Likes Received:
    57
    Trophy Points:
    178
    Location:
    inside a catfish
    cPanel Access Level:
    Root Administrator
    Hello,

    I just enabled Mailbox compression today. 99% of the mailboxes on said server are using Maildir. I want to gradually migrate, per account, users' email from the maildir format to the mdbox format.

    New emails coming in are automatically being compressed. Old emails are not automatically compressed (and yes, I realized this is documented).

    I wanted to convert an account from maildir to mdbox and have all messages compressed. So I ran the maildir-to-mdbox conversion, then ran an mdbox-to-maildir conversion, and then ran a maildir-to-mdbox conversion.

    No matter what I do, the conversion process does not seem to be compressing all of the old email for an account. But, from what I've read of others on the forum, simply converting from maildir to mdbox (or vice versa) and back should cause all of the email for a specific account to be compressed.

    What am I doing wrong?

    Mike
     
    Stop hovering to collapse... Click to collapse... Hover to expand... Click to expand...
  2. cPanelMichael

    cPanelMichael Forums Analyst
    Staff Member

    Joined:
    Apr 11, 2011
    Messages:
    43,961
    Likes Received:
    1,821
    Trophy Points:
    363
    cPanel Access Level:
    Root Administrator
    Hi Mike,

    It's important to note that we don't offer support for the compression of existing messages at this time. The feature request for this is open at:

    Ability to compress existing email

    I encourage you to vote and add your feedback to this request. As far as a temporary workaround to compress existing messages, some users have reported success with this by switching from the Maildir format to the Mdbox format, back to the Maildir format, and then to the Mdbox format again. However, it's important to note the following warning from our Mailbox Conversion document if you choose to do this:

    If the temporary workaround isn't working for you, could you verify how you are checking the results?

    Thank you.
     
    Stop hovering to collapse... Click to collapse... Hover to expand... Click to expand...
  3. mtindor

    mtindor Well-Known Member

    Joined:
    Sep 14, 2004
    Messages:
    1,342
    Likes Received:
    57
    Trophy Points:
    178
    Location:
    inside a catfish
    cPanel Access Level:
    Root Administrator
    I recognize that cPanel doesn't support the ability to compress existing email at this time. I have followed exactly that process which others have outlined in the forums. Yes I'm aware of message indexes getting reset for POP3 users.

    I verified that it is not actually compressing all mail when converting back to maildir and then converting to mdbox again by attempting to view the various m.* files in the storage folder. I can read all of the messages in all of the m.* files using cat / less without issue. In the last m.* file (in this case m.91), there is a mixture of uncompressed and compressed messages because that particular file contains messages from prior to this morning + a couple from this morning [after I enabled compression].

    If the conversion back to maildir and then forward to mdbox were actually compressing all of the email, I would not be able to view all of the messages in any of the m.* files using cat / less.

    Mike
     
    Stop hovering to collapse... Click to collapse... Hover to expand... Click to expand...
  4. cPanelMichael

    cPanelMichael Forums Analyst
    Staff Member

    Joined:
    Apr 11, 2011
    Messages:
    43,961
    Likes Received:
    1,821
    Trophy Points:
    363
    cPanel Access Level:
    Root Administrator
    Could you open a support ticket using the link in my signature so we can take a closer look at the system?

    Thank you.
     
    Stop hovering to collapse... Click to collapse... Hover to expand... Click to expand...
  5. mtindor

    mtindor Well-Known Member

    Joined:
    Sep 14, 2004
    Messages:
    1,342
    Likes Received:
    57
    Trophy Points:
    178
    Location:
    inside a catfish
    cPanel Access Level:
    Root Administrator
    I did upvote the feature request. Thanks for showing me that. Of course, unless a lot of people upvote it, it'll be useless.

    At any rate, after further searching, I see that even now after dovecot has supported compression for years there is still no simple, tried-and-true reliable way of compressing existing mail.

    I have to wonder if those who report successful compression of all emails have really checked and confirmed that every indidivual email file (maildir) or m.* file (mdbox) is indeed compressed. Perhaps the space savings they are seeing was simply a savings due to conversion from maildir to mdbox. Of course, I saw one person mention going from 5.5 GB to 3.5 GB of space used. I can't imagine that one could realize that kind of space savings in any other way but having full compressed mail files. I can't imagine saving that amount of space purely by changing from maildir to mdbox (without compression of existing email).

    Mike
     
    Stop hovering to collapse... Click to collapse... Hover to expand... Click to expand...
  6. mtindor

    mtindor Well-Known Member

    Joined:
    Sep 14, 2004
    Messages:
    1,342
    Likes Received:
    57
    Trophy Points:
    178
    Location:
    inside a catfish
    cPanel Access Level:
    Root Administrator
    Hi Michael,

    I've opened a support ticket.

    Thanks

    Mike
     
    Stop hovering to collapse... Click to collapse... Hover to expand... Click to expand...
  7. mtindor

    mtindor Well-Known Member

    Joined:
    Sep 14, 2004
    Messages:
    1,342
    Likes Received:
    57
    Trophy Points:
    178
    Location:
    inside a catfish
    cPanel Access Level:
    Root Administrator
    As I suspected, cPanel support has verified that switching back and forth between the maildir / mdbox formats is not going to magically make all of your mail become compressed, even if you are sure compression is enabled in dovecot, since the conversion scripts do not pass anything through Dovecot and that is the only way Dovecot would compress them.

    Mike
     
    Stop hovering to collapse... Click to collapse... Hover to expand... Click to expand...
  8. Hedloff

    Hedloff Well-Known Member

    Joined:
    Jun 7, 2004
    Messages:
    135
    Likes Received:
    4
    Trophy Points:
    168
    Location:
    Up north!
    cPanel Access Level:
    DataCenter Provider
    I cannot seem to find this feature request, was it removed? Why?

    Did you find anyway to compress old emails Mike?
     
  9. cPanelMichael

    cPanelMichael Forums Analyst
    Staff Member

    Joined:
    Apr 11, 2011
    Messages:
    43,961
    Likes Received:
    1,821
    Trophy Points:
    363
    cPanel Access Level:
    Root Administrator
    Hello @Hedloff,

    Thanks for the report. It looks to have been removed as part of the feature request website's hardware failure in March:

    I've re-created the feature request, and it should be approved out of moderation shortly. Once it's approved, please take the time to vote and re-add any feedback to the request:

    https://features.cpanel.net/topic/ability-to-compress-existing-email

    We apologize for the inconvenience.

    Thank you.
     
    Stop hovering to collapse... Click to collapse... Hover to expand... Click to expand...
Loading...

Share This Page

  1. This site uses cookies to help personalise content, tailor your experience and to keep you logged in if you register.
    By continuing to use this site, you are consenting to our use of cookies.
    Dismiss Notice