cPanel wrong disk usage report

thesufi

Member
Dec 5, 2018
9
2
3
Dhaka
cPanel Access Level
Root Administrator
Hello,

I am having some unusual trouble with one of my cPanel accounts. I have a reseller account from a provider, where I have some 8-9GB email hosted under one cPanel account.
On the quick stats on cPanel home page (first image), it shows Disk Usage is 8.03 GB of 11.72 GB (account is limited to 12000MB).quick_stat.png

However, in "cPanel > Disk Usage", it shows "34,122.91 MB total disk space used." (second image).
disk_usage.png

This problem is mostly with email accounts and their wrong disk usage data. When I sort email accounts by "Storage used" from high to low (third image), counting only first page of 5 pages of disk usage manually gives around 14.3 GB usage, that is after leaving "Default" account from the counting, which is 15.82 GB itself!

email_accounts.png

Now, I have contacted with my hosting provider, who after contacting with cPanel told me that (quoted from my hosting provider support ticket reply) "this is intentional / known behavior and unfortunately not something they consider a bug" - here "they" = cPanel.

My hosting provider also quoted from cPanel:
"If you convert your mailboxes to the maildir format, you may notice a discrepancy between reported and actual disk usage for the default email account. This occurs because a maildir mailbox adds the total default email account usage to the INBOX directory usage of every other email address on the cPanel account. It does not add the subfolders for any other mailbox."

When I have asked my hosting provider about any recent conversion to maildir format, they have mentioned that they have always used "maildir", and this is what my hosting provider have mentioned:
"We've always ran maildir because if we did not then the ability to restore individual email messages would no longer be possible. cPanel now has a case open CPANEL-26525 to improve the reporting of the disk usage pf email so it's not misleading/incorrect but there is no estimate on when this will be fixed."

My hosting provider gave me a SSH command (du -h --max-depth=1 mail/) to check the disk usage of mail directory. Unfortunately, for my end user, who is using this cPanel account for only email purpose, is not usable and/or not user friendly.

Now, I need a solution for this problem. Since my hosting provider could not solve this, even after contacting cPanel support, I have no other option but to open a thread here. I am paying my hosting provider, who is paying cPanel, for a end product, which we all expect bug free/usable at it's best. I understand that there will be some bugs in a software. But what I am facing here is not any minor bug, and is kind of unusable for an finished product, in my opinion.

If my hosting provider is following this thread, to them: I am not here to complain, but to look for a solution.

I am sorry for the long post, and if my post is not clear or doesn't make any sense. And thanks for reading it in full.

I am unable to edit the original post. So here are the missing information:
cPanel Version: 78.0.21, running on CLOUDLINUX 7.6 kvm
 
Last edited by a moderator:

thesufi

Member
Dec 5, 2018
9
2
3
Dhaka
cPanel Access Level
Root Administrator
Hello @cPanelMichael

Thanks for your reply. I have checked the CPANEL-26448 and your provided thread. However, I can confirm that it doesn't matched with my scenario entirely. Only default email disk usage part matches. On my case, every email accounts are providing false disk usage data. That's the major problem. If it was only "default" account, I could have easily avoid it. But every email accounts are showing wrong disk usage data. Kindly read my post again and match with my attached images.

Thanks.
 

cPanelMichael

Administrator
Staff member
Apr 11, 2011
47,910
2,212
363
Hello @thesufi,

We'd need to take a look at your system to see why the disk space reported for your virtual email accounts are inaccurate. Can you ask your hosting provider to open a support ticket with us on your behalf so we can take a closer look? You can ask them for the ticket number they receive from us and post it here once they let you know what it is.

Thank you.
 

cPanelMichael

Administrator
Staff member
Apr 11, 2011
47,910
2,212
363
Hello @thesufi,

To update, the disk space usage discrepancy for the individual email accounts is associated with Compress Messages feature documented on the link below:

Mailserver Configuration - Version 78 Documentation - cPanel Documentation

This is related to the following notice in the cPanel UI:

Email account storage may occupy less space on the disk if you use compression or hard-link optimizations designed to save space. Email account storage does not include the metadata that the system uses to store email.
The following feature request is open to provide a supported method of compressing existing email (this would address the issue you have noticed):

Ability to compress existing email

I encourage you to vote and add feedback to this feature request if this is something you'd like to see added to cPanel & WHM in the future.

Thank you.
 

thesufi

Member
Dec 5, 2018
9
2
3
Dhaka
cPanel Access Level
Root Administrator
Hello @cPanelMichael,

Yes, this seems like the problem. This feature request has been there for around 10 months now. So, I don't think this is going to see daylight anytime soon. Any possible workaround? I can deploy a quick VPS and do some work, if there is any hope to fix this! Because the disk usage data end user is getting is very misleading and causing problem.

Thanks.
 

mathx

Active Member
Jan 16, 2017
26
3
3
Toronto
cPanel Access Level
DataCenter Provider
Some of the statements here are incorrect. We're on a ZFS filesystem which compresses automatically, and we see a 7775MB report for the user, however because of the email reported usage it's putting the user over 11,000MB.

I compared du -xm . | sort -n | tail -1 (which reports 7775MB of actual disk) vs

$find . -ls -type f | awk '{a+=$7} END {print a/1000/1000;}'
8171.7

you can see this isnt very far off, vs the "email disk usage of 11,600MB" in email accounts reported under the disk usage tab of the user's cpanel panel.

Note what i've done here is tallied the actual LISTED disk usage from ls of the file, which is the correct file size uncompressed. So compression isnt the issue here specifically. Something is completely wrong in the disk usage calculation for email.

Where is the script that cpanel uses to calculate this so that I may investigate and see what it is doing internally, and possibly modify it for our uses?
 

cPanelMichael

Administrator
Staff member
Apr 11, 2011
47,910
2,212
363
Any possible workaround? I can deploy a quick VPS and do some work, if there is any hope to fix this! Because the disk usage data end user is getting is very misleading and causing problem.
The workaround noted on the following feature request comments remains applicable:

Ability to compress existing email comment 1
Ability to compress existing email comment 2

Where is the script that cpanel uses to calculate this so that I may investigate and see what it is doing internally, and possibly modify it for our uses?
Here's the path to the script that's used to generate the maildirsize files:

Code:
/usr/local/cpanel/scripts/generate_maildirsize
Feel free to open a support ticket if you'd like us to take a closer look and investigate where the discrepancy is coming from.

Thank you.
 

cPanelMichael

Administrator
Staff member
Apr 11, 2011
47,910
2,212
363
Ticket #12315117
The issue reported in this ticket was determined to relate to case CPANEL-26448:

Pending Publication - [CPANEL-26448] Default Email Account Showing Incorrect Usage

The workaround provided in the support ticket was to execute the /scripts/generate_maildirsize command and move aside the email account cache file from the affected user's /home/username/.cpanel/ directory. EX:

Code:
/scripts/generate_maildirsize --confirm --allaccounts
mv -v /home/username/.cpanel/email_accounts.json /home/username/.cpanel/backup.email_accounts.json.backup1
Thank you.