SOLVED How is mdbox pruning / rotation handled?

[brt]

I've searched and found no answers relating to how cPanel servers handle deleted emails from mdbox storage.

I know there are settings for rotation, both in size and in time, but it appears that just relates to how often a new file is started; not anything to do with how existing files are treated.

So.

If you go back and delete a bunch of email that is stored in previous mdbox files, how/when is that space freed up? Are those files ever re-written/pruned to free up space?

 

[kdean]

When UPCP runs each day, email maintenance is run which prunes deleted emails and frees up space. You can manually run with:

/scripts/dovecot_maintenance

 

[brt]

When UPCP runs each day, email maintenance is run which prunes deleted emails and frees up space. You can manually run with:

/scripts/dovecot_maintenance

This applies to mdbox storage too, or just maildir?

With maildir it could just delete the individual email file(s) but with mdbox you've got -many- messages saved in a single file.

 

[cPanelMichael]

If you go back and delete a bunch of email that is stored in previous mdbox files, how/when is that space freed up? Are those files ever re-written/pruned to free up space?

Hello @brt,

The following settings in WHM >> Mailserver Configuration are relevant to the automatic deletion of emails in the Trash folders when using the mdbox format:

Auto Expunge Trash
Trash Expire Time

For deleted emails stored in the Trash folder, the emails are pruned from the mdbox files based on the Trash Expire Time value when Auto Expunge Trash is enabled. This happens when the following dovecot maintenance script runs during /scripts/upcp:

/usr/local/cpanel/scripts/dovecot_maintenance

As far as the general deletion of emails (e.g. when a user fully deletes the email and doesn't use the Trash folder), the mdbox files are updated to reflect the change when /usr/local/cpanel/scripts/dovecot_maintenance runs during /scripts/upcp.

Thank you.

 

[brt]

Hello @brt,

The following settings in WHM >> Mailserver Configuration are relevant to the automatic deletion of emails in the Trash folders when using the mdbox format:

Auto Expunge Trash
Trash Expire Time

For deleted emails stored in the Trash folder, the emails are pruned from the mdbox files based on the Trash Expire Time value when Auto Expunge Trash is enabled. This happens when the following dovecot maintenance script runs during /scripts/upcp:

/usr/local/cpanel/scripts/dovecot_maintenance

As far as the general deletion of emails (e.g. when a user fully deletes the email and doesn't use the Trash folder), the mdbox files are updated to reflect the change when /usr/local/cpanel/scripts/dovecot_maintenance runs during /scripts/upcp.

Thank you.

As far as I know, Auto Expunge Trash and Trash Expire Time just deal with how long messages are retained/visible in the Trash folder; not when they're actually purged from the message store / filesystem. I may not be correct, but I'd like concrete evidence either way.

My understanding is that new messages coming in get written into a single file until MDBOX rotation size (MB) or MDBOX rotation interval (Weeks or Days) occurs, at which point a new file is started.

Specifically, I'm wanting to know that if someone deletes a large amount of old messages, say from last year, in old, established, long-rotated mdbox files, is / when is that space actually freed up in the filesystem?

I am under the impression that the old mdbox file(s) need to be re-written before any space is actually freed, no?

 

[cPanelMichael]

As far as I know, Auto Expunge Trash and Trash Expire Time just deal with how long messages are retained/visible in the Trash folder; not when they're actually purged from the message store / filesystem. I may not be correct, but I'd like concrete evidence either way.

Yes, this is correct.

My understanding is that new messages coming in get written into a single file until MDBOX rotation size (MB) or MDBOX rotation interval (Weeks or Days) occurs, at which point a new file is started.

Specifically, I'm wanting to know that if someone deletes a large amount of old messages, say from last year, in old, established, long-rotated mdbox files, is / when is that space actually freed up in the filesystem?

I am under the impression that the old mdbox file(s) need to be re-written before any space is actually freed, no?

The /usr/local/cpanel/scripts/dovecot_maintenance script (it runs nightly as part of the cPanel update cron job) automatically detects deleted emails and removes them from their corresponding mdbox files. For example, here's an example of an email account with several messages on a system with the default values (10M and 0) for MDBOX rotation size (MB) and MDBOX rotation interval (Weeks or Days):

# pwd
/home/user123/mail/domain.tld/test1/storage/
# du -s *
4   dovecot.map.index.log
2380   m.2

As you can see, it's not large enough to get rotated by the referenced settings. However, upon logging in to the email account and deleting a few messages, the mdbox files are reduced in space the next time /usr/local/cpanel/scripts/dovecot_maintenance runs:

# /usr/local/cpanel/scripts/dovecot_maintenance
[dovecot_maintenance] Setting I/O priority to reduce system load: best-effort: prio 7
Purging deleted messages for “user123” …Done
# du -s *
4    dovecot.map.index.log
2240    m.3

Thank you.

 

[brt]

Perfect. Thank you!