Broken virtual size in mailbox

[gRobertsTVW]

Hi all,

We had to use the transfer tool to move an account from one server to another and unfortunately since then, one mailbox of the account is having issues receiving via IMAP.

Jun  9 09:49:15 NEW_SERVER dovecot: imap([email protected])<15349><SHgOca79mp0zm4YX>: Error: Mailbox INBOX: Deleting corrupted cache record uid=160560: UID 160560: Broken virtual size in mailbox INBOX: read(/home/redacted/mail/redacted.com/orders/cur/1686150116.M856969P19174.OLD_SERVER.co.uk,S=20664,W=21034:2,): FETCH BODY[] got too little data: 20693 vs 21034

Multiple sources suggest it's a corrupt dovecot cache/index file, which we've used a combination of tools to try and fix with no luck:

tools/maildir-size-fix.pl at main · dovecot/tools

Dovecot tools. Contribute to dovecot/tools development by creating an account on GitHub.

github.com

tools/maildir-size-check.sh at main · dovecot/tools

Dovecot tools. Contribute to dovecot/tools development by creating an account on GitHub.

github.com

/usr/local/cpanel/scripts/remove_dovecot_index_files

The maildir-size* tools did originally flag a number of files

/home/redacted/mail/.orders@redacted_com/.Trash/cur/1686235035.M771008P40608.teamvaa17.teamvalleyserver.co.uk,S=1543,W=1577:2,S: Wrong W=1577 value, should be W=1552
/home/redacted/mail/.orders@redacted_com/.Trash/cur/1686235035.M771005P40608.teamvaa17.teamvalleyserver.co.uk,S=1556,W=1590:2,S: Wrong W=1590 value, should be W=1557
/home/redacted/mail/.orders@redacted_com/.Trash/cur/1686235035.M771006P40608.teamvaa17.teamvalleyserver.co.uk,S=1749,W=1786:2,S: Wrong W=1786 value, should be W=1749
/home/redacted/mail/.orders@redacted_com/.Sent/cur/1685539940.M897022P28294.OLD_SERVER.co.uk,S=499794,W=506610:2,S: Wrong W=506610 value, should be W=501016
/home/redacted/mail/.orders@redacted_com/.Sent/cur/1685447530.M952248P10948.OLD_SERVER.co.uk,S=3804,W=3910:2,S: Wrong W=3910 value, should be W=3847
/home/redacted/mail/.orders@redacted_com/.Sent/cur/1685514243.M22507P31990.OLD_SERVER.co.uk,S=9082,W=9317:2,S: Wrong W=9317 value, should be W=9099
/home/redacted/mail/.orders@redacted_com/.Junk/cur/1685426315.M676120P24704.OLD_SERVER.co.uk,S=12132,W=12436:2,: Wrong W=12436 value, should be W=12143
/home/redacted/mail/.orders@redacted_com/.Junk/cur/1685426319.M849991P24704.OLD_SERVER.co.uk,S=18136,W=18581:2,: Wrong W=18581 value, should be W=18162
/home/redacted/mail/.orders@redacted_com/.Junk/cur/1685608960.M931188P18000.OLD_SERVER.co.uk,S=16740,W=17152:2,: Wrong W=17152 value, should be W=16928
/home/redacted/mail/.orders@redacted_com/.Junk/cur/1685426316.M590314P24704.OLD_SERVER.co.uk,S=12207,W=12514:2,: Wrong W=12514 value, should be W=12235
/home/redacted/mail/.orders@redacted_com/.Junk/cur/1685426319.M614856P24704.OLD_SERVER.co.uk,S=10148,W=10408:2,: Wrong W=10408 value, should be W=10178
/home/redacted/mail/.orders@redacted_com/.Junk/cur/1685441550.M414748P35065.OLD_SERVER.co.uk,S=10112,W=10370:2,: Wrong W=10370 value, should be W=10183
/home/redacted/mail/.orders@redacted_com/.Junk/cur/1685612366.M152865P33161.OLD_SERVER.co.uk,S=12076,W=12380:2,: Wrong W=12380 value, should be W=12089
/home/redacted/mail/.orders@redacted_com/.Junk/cur/1685426319.M369937P24704.OLD_SERVER.co.uk,S=13855,W=14200:2,: Wrong W=14200 value, should be W=13901
/home/redacted/mail/.orders@redacted_com/.Junk/cur/1685426317.M230095P24704.OLD_SERVER.co.uk,S=10591,W=10860:2,: Wrong W=10860 value, should be W=10607
/home/redacted/mail/.orders@redacted_com/.Junk/cur/1685433560.M44485P2513.OLD_SERVER.co.uk,S=14837,W=15203:2,: Wrong W=15203 value, should be W=14855
/home/redacted/mail/.orders@redacted_com/.Junk/cur/1685447559.M771407P10948.OLD_SERVER.co.uk,S=12154,W=12459:2,: Wrong W=12459 value, should be W=12291
/home/redacted/mail/.orders@redacted_com/.Junk/cur/1685426317.M456179P24704.OLD_SERVER.co.uk,S=14290,W=14643:2,: Wrong W=14643 value, should be W=14323
/home/redacted/mail/.orders@redacted_com/.Junk/cur/1685426317.M930247P24704.OLD_SERVER.co.uk,S=16324,W=16724:2,: Wrong W=16724 value, should be W=16337
/home/redacted/mail/.orders@redacted_com/.Junk/cur/1685426320.M91974P24704.OLD_SERVER.co.uk,S=12191,W=12497:2,: Wrong W=12497 value, should be W=12205
/home/redacted/mail/.orders@redacted_com/.Junk/cur/1685617997.M841332P9514.OLD_SERVER.co.uk,S=16526,W=16932:2,: Wrong W=16932 value, should be W=16780
/home/redacted/mail/.orders@redacted_com/.Junk/cur/1685426316.M135069P24704.OLD_SERVER.co.uk,S=10258,W=10520:2,: Wrong W=10520 value, should be W=10279
/home/redacted/mail/.orders@redacted_com/.Junk/cur/1685426318.M652976P24704.OLD_SERVER.co.uk,S=10670,W=10942:2,: Wrong W=10942 value, should be W=10690
/home/redacted/mail/.orders@redacted_com/.Junk/cur/1685426316.M996182P24704.OLD_SERVER.co.uk,S=10113,W=10371:2,: Wrong W=10371 value, should be W=10130
/home/redacted/mail/.orders@redacted_com/.Junk/cur/1685426316.M375243P24704.OLD_SERVER.co.uk,S=19154,W=19620:2,: Wrong W=19620 value, should be W=19178
/home/redacted/mail/.orders@redacted_com/.Junk/cur/1685426317.M694984P24704.OLD_SERVER.co.uk,S=10176,W=10435:2,: Wrong W=10435 value, should be W=10201
/home/redacted/mail/.orders@redacted_com/.Junk/cur/1685426318.M410113P24704.OLD_SERVER.co.uk,S=12197,W=12503:2,: Wrong W=12503 value, should be W=12210
/home/redacted/mail/.orders@redacted_com/.Junk/cur/1685426319.M131173P24704.OLD_SERVER.co.uk,S=16347,W=16748:2,: Wrong W=16748 value, should be W=16361
/home/redacted/mail/.orders@redacted_com/.Junk/cur/1685426318.M172997P24704.OLD_SERVER.co.uk,S=12115,W=12420:2,: Wrong W=12420 value, should be W=12156
/home/redacted/mail/.orders@redacted_com/.Junk/cur/1685433560.M264704P2513.OLD_SERVER.co.uk,S=12169,W=12475:2,: Wrong W=12475 value, should be W=12196
/home/redacted/mail/.orders@redacted_com/.Junk/cur/1685426315.M909110P24704.OLD_SERVER.co.uk,S=14563,W=14924:2,: Wrong W=14924 value, should be W=14588
/home/redacted/mail/.orders@redacted_com/.Junk/cur/1685426318.M913155P24704.OLD_SERVER.co.uk,S=12114,W=12418:2,: Wrong W=12418 value, should be W=12147
/home/redacted/mail/.orders@redacted_com/.Junk/cur/1685426320.M330025P24704.OLD_SERVER.co.uk,S=10111,W=10369:2,: Wrong W=10369 value, should be W=10127
/home/redacted/mail/.orders@redacted_com/.Junk/cur/1685426316.M790013P24704.OLD_SERVER.co.uk,S=14495,W=14853:2,: Wrong W=14853 value, should be W=14513

And when passing the -f flag on maildir-size-check.sh, it renamed these to reflect the correct sizes and no longer reported any issues.

Unfortunately though, this did not resolve the issue and we are still seeing errors in the maillog.

Does anyone have any suggestions?

 

[cPRex]

Hey there! Can you try running the following commands to see if that clears the issue? This moves the dovecot indexes for the user out of the way and then restarts the mail server, but they can easily be put back if something goes wrong:

mv /home/username/mail/dovecot.index{,.bak-`date +%Y%m%d`}
mv /home/username/mail/dovecot.index.cache{,.bak-`date +%Y%m%d`}
mv /home/username/mail/dovecot.index.log{,.bak-`date +%Y%m%d`}
/scripts/restartsrv_dovecot

 

[gRobertsTVW]

Hi cPRex,

Thanks for getting back to me.

I have been through all dovecot.index files for the affected mailbox (trash etc is affected too) and renamed them as described and restarted dovecot and unfortunately, we're still seeing the same errors.

Jun  9 13:31:21 NEW_SERVER dovecot: imap([email protected])<24456><rUTci7H9dY4zm4YX>: Error: Mailbox INBOX.Junk: Deleting corrupted cache record uid=82925: UID 82925: Broken virtual size in mailbox INBOX.Junk: read(/home/redacted/mail/redacted.com/orders/.Junk/cur/1685426319.M614856P24704.OLD_SERVER.co.uk,S=10148,W=10178:2,): FETCH BODY[] got too little data: 10177 vs 10178

Strangely other mailboxes on the same account are fine and this mailbox on the original server was fine also, but we had to move it and now experiencing issues?

 

[vanessa]

I've seen this issue before, and for me, deleting the "maildirsize" files corrected the issue. There's a file in each account so if you want to be bold about it:

rm -fv /home/*/mail/$domain/$user/maildirsize

then run:

/scripts/generate_maildirsize --allaccounts

 

[gRobertsTVW]

Thanks Vanessa,

Unfortunately that had no effect either

Jun  9 14:39:49 NEW_SERVER dovecot: imap([email protected])<4358><RW67gLL9i/4zm4YX>: Error: Mailbox INBOX.Junk: Deleting corrupted cache record uid=82925: UID 82925: Broken virtual size in mailbox INBOX.Junk: read(/home/redacted/mail/redacted.com/orders/.Junk/cur/1685426319.M614856P24704.OLD_SERVER.co.uk,S=10148,W=10178:2,): FETCH BODY[] got too little data: 10177 vs 10178

Note, there are multiple errors in varying folders across the account. I was contemplating deleting the erroneous files but this would be deleting customers emails .

 

[cPRex]

Sounds like it's time to create a ticket for this one.

 

[vanessa]

Maybe try this too, according to cpanel docs (which might be similar to what cprex was offering)

/scripts/remove_dovecot_index_files --user user

 

[gRobertsTVW]

Thanks Vanessa, we have already tried this and unfortunately this didn't resolve the issue.

I'll escalate this and will let you know the outcome.

 

[gRobertsTVW]

Maybe try this too, according to cpanel docs (which might be similar to what cprex was offering)

/scripts/remove_dovecot_index_files --user user

Sounds like it's time to create a ticket for this one.

Hi Both,

Just thought I'd update you on this.

Turns out that a number of emails were corrupt and coming back as data (using file) which caused issues.

After removing these, it appears the issue was resolved.

Thanks again for your help