Mailboxes corrupting (dovecot.index file)

camble

Member
Aug 13, 2012
7
0
1
cPanel Access Level
Root Administrator
Hello,
I have recently had a few of my clients saying that their email accounts aren't working and when I've investigated by logging onto Webmail (Roundcube) it shows the following warning/error: "Server Error (OK)".

I paid server admins to look into the issue and for one instance they said that the connection hadn't been closed properly by Outlook Express for a client, however I'm pretty sure that some of the other clients it's happened to are not using Outlook Express so I don't think it's just cause by that program.


The way I've been fixing the corrupt mailboxes is to delete/rename the dovecot.index file for their email account, however I'd like to know how I can prevent this from happening.


Does anybody have any ideas?


Kind regards
Liam
 

camble

Member
Aug 13, 2012
7
0
1
cPanel Access Level
Root Administrator
Hi Peter,
Here is the message (with email addresses changed to prevent spam) that I got from my server admins:

Feb 11 13:04:20 ip-1**-2*5-33-59 dovecot: imap(kevin.s****@loc8********gents.co.uk): Error: broken sync positions in index file /home/loc8/mail/loc8**********nts.co.uk/kevin.s****/dovecot.index
Feb 11 13:04:20 ip-1**-2*5-33-59 dovecot: imap(kevin.s****@loc8********gents.co.uk): Warning: fscking index file /home/loc8/mail/loc8********ents.co.uk/kevin.s****/dovecot.index
 

domeneas

Well-Known Member
Sep 20, 2013
48
13
58
cPanel Access Level
Root Administrator
Hi,

this is happening to me too. It was one incident 6 months or so ago. Then a couple here and there, but now I have had atleast 10 within the last few weeks telling me it has something to to with load or activity. Seems only to happen to very active users.

I started looking through the logs and found this beauty of someone whom I just deleted the dovecot*.* files from their mail dir to get it working again:

Code:
Apr  3 22:21:17 tor dovecot: pop3-login: Login: user=<ellingsen@*****.com>, method=PLAIN, rip=*.*.*.205, lip=*.*.*.207, mpid=23929, TLS, session=<zJjDJCn2lwAClhLN>
Apr  3 22:21:19 tor dovecot: pop3(ellingsen@*****.com): Error: broken sync positions in index file /home/brensel/mail/*****.com/ellingsen/dovecot.index
Apr  3 22:21:19 tor dovecot: pop3(ellingsen@*****.com): Warning: fscking index file /home/brensel/mail/*****.com/ellingsen/dovecot.index
Apr  3 22:21:19 tor dovecot: pop3(ellingsen@*****.com): Error: Fixed index file /home/brensel/mail/*****.com/ellingsen/dovecot.index: log_file_tail_offset 176 -> 144
Apr  3 22:21:19 tor dovecot: pop3(ellingsen@*****.com): Panic: file mail-transaction-log.c: line 350 (mail_transaction_log_set_mailbox_sync_pos): assertion failed: (file_offset >= log->head->saved_tail_offset)
Apr  3 22:21:19 tor dovecot: pop3(ellingsen@*****.com): Error: Raw backtrace: /usr/lib64/dovecot/libdovecot.so.0 [0x2b73fe993a7a] -> /usr/lib64/dovecot/libdovecot.so.0 [0x2b73fe993ae6] -> /usr/lib64/dovecot/libdovecot.so.0 [0x2b73fe993473] -> /usr/lib64/dovecot/libdovecot-storage.so.0 [0x2b73fe6ea81b] -> /usr/lib64/dovecot/libdovecot-storage.so.0(mail_index_sync_commit+0x89) [0x2b73fe6e2069] -> /usr/lib64/dovecot/libdovecot-storage.so.0 [0x2b73fe66e20d] -> /usr/lib64/dovecot/libdovecot-storage.so.0 [0x2b73fe66d717] -> /usr/lib64/dovecot/libdovecot-storage.so.0 [0x2b73fe66da58] -> /usr/lib64/dovecot/libdovecot-storage.so.0(maildir_storage_sync_init+0xbb) [0x2b73fe66dbfb] -> /usr/lib64/dovecot/libdovecot-storage.so.0(mailbox_sync_init+0x31) [0x2b73fe69a391] -> /usr/lib64/dovecot/libdovecot-storage.so.0(mailbox_sync+0x27) [0x2b73fe69cae7] -> dovecot/pop3(client_create+0x332) [0x405722] -> dovecot/pop3 [0x40414b] -> dovecot/pop3 [0x404708] -> /usr/lib64/dovecot/libdovecot.so.0 [0x2b73fe9512f2] -> /usr/lib64/d
Apr  3 22:21:19 tor dovecot: pop3(ellingsen@*****.com): Fatal: master: service(pop3): child 23929 killed with signal 6 (core dumps disabled)
 

cPanelMichael

Administrator
Staff member
Apr 11, 2011
47,880
2,268
463
It looks like the dovecot.index file for that email account might be corrupted. Please try moving it out of the way to see if that resolves the issue. EX:

Code:
mv /home/username/mail/domain.com/email-account/dovecot.index /root/
The file should automatically regenerate upon logging in. I know you mentioned this issue continues to reoccur, so feel free to open a support ticket if this keeps happening so we can take a closer look. You can post the ticket number here so we can update this thread with the outcome.

Thank you.
 

domeneas

Well-Known Member
Sep 20, 2013
48
13
58
cPanel Access Level
Root Administrator
Yes, that works every time, but the issue keep recurring and while the customer waits for it to be resolved by us his email service is unusable.

I will open a support case.
 

vestigium

Registered
Apr 14, 2014
4
0
1
cPanel Access Level
Website Owner
Yes, that works every time, but the issue keep recurring and while the customer waits for it to be resolved by us his email service is unusable.

I will open a support case.
Hi! I have same problem with my clients.
I already tried to clean dovecot files, but doesn't works.
If you found a solution, can you update this topic?

Thanks!
 

vestigium

Registered
Apr 14, 2014
4
0
1
cPanel Access Level
Website Owner
I solved my problem. In my case occurred when trying to delete any email.
I was deleting the dovecot files from wrong folder (excluding the root folder).
In my error log specified the directory that the problem was occurring:

Code:
Apr 14 12:49:13 vps dovecot: imap([email protected]): Warning: fscking index file /home/domain/mail/domain.com/account/.Trash/dovecot.index
Only took into this directory and remove the dovecot files.
 

cPanelMichael

Administrator
Staff member
Apr 11, 2011
47,880
2,268
463
To update, the customer was advised to increase the "Spare Authentication Processes" value for Dovecot to see if that makes a difference in the corruption of the dovecot.index files.

Thank you.
 

domeneas

Well-Known Member
Sep 20, 2013
48
13
58
cPanel Access Level
Root Administrator
To update, the customer was advised to increase the "Spare Authentication Processes" value for Dovecot to see if that makes a difference in the corruption of the dovecot.index files.
That didn't help, as I still got errors later that day, about 30 minutes after changing the number of available authentication daemons, but oddly do not have any the last 2 days. I have a little hope that maybe the last errors really were the last ones and some log processing time or the likes accounts for that 30-45 min gap.

Cpanel now has core dumps from the server when the error occurred, so hopefully that will tell them something.
 

irefuse

Registered
Jun 11, 2006
1
0
151
That didn't help, as I still got errors later that day, about 30 minutes after changing the number of available authentication daemons, but oddly do not have any the last 2 days. I have a little hope that maybe the last errors really were the last ones and some log processing time or the likes accounts for that 30-45 min gap.

Cpanel now has core dumps from the server when the error occurred, so hopefully that will tell them something.
Has there been any update on your ticket? I am having this exact same issue. larger/more frequently used mailboxes are having the index files become corrupt. At least 1-2 times a month
 

cPanelMichael

Administrator
Staff member
Apr 11, 2011
47,880
2,268
463
Has there been any update on your ticket? I am having this exact same issue. larger/more frequently used mailboxes are having the index files become corrupt. At least 1-2 times a month
The customer has not reported additional problems since updating their kernel.

Thank you.
 

domeneas

Well-Known Member
Sep 20, 2013
48
13
58
cPanel Access Level
Root Administrator
Sorry, I forgot about this thread.

Unfortunately I am not able to update the Kernel, so I cannot test that. I would have to migrate the whole VPS to a new platform, and I'm just not willing to do that with as many users as are on the box in question.

I am seeing improvements from just tweaking the authentication daemons as stated above though. At least I assume it must be from that. The issue is much rarer again now.
 

camble

Member
Aug 13, 2012
7
0
1
cPanel Access Level
Root Administrator
I've not visited this thread for a while but I've consistently had the same problem time and time again, however it does seem to be limited to only a few of my clients (which I know use the email a lot).

I noticed that it was suggested to increase the "Spare Authentication Processes" value - my default value is 2, so what would be an appropriate value to increase to?


Kind regards
Liam
 

cPanelMichael

Administrator
Staff member
Apr 11, 2011
47,880
2,268
463
I noticed that it was suggested to increase the "Spare Authentication Processes" value - my default value is 2, so what would be an appropriate value to increase to?
You could try 3 or 4 to see if that makes a difference with this issue.

Thank you.
 

camble

Member
Aug 13, 2012
7
0
1
cPanel Access Level
Root Administrator
You could try 3 or 4 to see if that makes a difference with this issue.

Thank you.
A couple of weeks ago I changed it to 5 after speaking to a rep from GoDaddy who said 5 would be ok.
Fingers crossed nothing has corrupted since but that doesn't necessarily mean it's solved it as it happened every so often anyway.

The client who it was mainly happening to has gone elsewhere because of the issue so it might not be so easy to know if it does happen.


Kind regards
Liam
 

camble

Member
Aug 13, 2012
7
0
1
cPanel Access Level
Root Administrator
// Update //

Since changing the Spare Authentication Processes to 5, it doesn't seem to have happened "as" often, although it's hard to gauge. However in the last few weeks it seems to have started happening a lot more regularly.

I am having my developer write a script that will allow my clients to go to a url which will remove/rename all dovecot.index files within their account that should fix the issue, although this is obviously not ideal because I'd rather it didn't happen in the first place.


When I looked at the log files previously and asked my server admin to overview them, they said that it was because the connection wasn't closed properly by the email client. My hypothesis is that it's when a user is running a program such as Outlook and doesn't close the program before shutting down their machine, and Outlook doesn't close properly (Not Responding) and they force close it. My reason behind this idea is because I always shut my Outlook down before shutting down and it's never once happened to me, however I know that several of my clients will not do this and will just shut down the machine with programs running.