Quotas issue: unable to receive emails or work with cPanel (VPS)

[Hieroglyf]

Hello,

I have an virtuozzo VPS server (CentOS 5.11 / 2.6.32-042stab113.11) using WHM 54.0 (build 21) and last week or so I started experiencing problems like this:

1. Certain accounts are unable to receive e-mails. If I look in the e-mail queue, I see the following error:

  == [email protected] R=virtual_user T=virtual_userdelivery defer (122): Disk quota exceeded: failed to open tmp/1460382424.H22664P18135.host.domain.com (10 tries)

But the quota is set to Unlimited and there is enough free space on the VPS.

I have checked also the inodes and they seem fine. I tried changing the plan to a different one, setting quota to different values, but to no avail (cpanel still reports disk usage as 0 and quota as unlimited).

Interestingly enough, I am able to receive emails to some accounts that also have Unlimited quota set.

2. When I login to cPanel with the aforementioned problematic account, I see the following errors:

On the top, there is a box that says:

(XID 648qan) The system failed to open the file “/var/cpanel/bandwidth.cache/username" for reading because of an error: Permission denied at /usr/local/cpanel/Cpanel/API/StatsBar.pm line 49.

folllowed by many, many lines of debug code

On the bottom of the page, there is yet another box with this error:

(XID 44t8qd) Too many (1024) failed attempts to create a temp file based on “/home/username/.cpanel/caches/statcache_persistant”! The last tried file was “/home/username/.cpanel/caches/statcache_persistant.tmp.86518652”, and the last error was: Disk quota exceeded at /usr/local/cpanel/Cpanel/FileUtils/Write.pm line 96.

Again, followed by the debug code or whatnot.

There appears to be some problem with quotas. I asked my VPS provider if second-level quotas are enabled and the reply was positive, that they are. So I went on a quest to resolve the obvious problem with quotas (went through options suggested in this forum and beyond), but unsuccessfully, so far.

I suspect the problem here might be the kernel version, since I have stumbled upon this thread:

[OVZ-6661] `quota` is not working on latest kernel if running command not from / on simfs - bugs.openvz.org

… and the behaviour of setting/checking quotas is the same on my system.

I am unable to set the quota through cPanel/whm, I am unable to do so via ssh, so I wonder, is there a workaround for that? Assuming the aforementioned kernel is the actual issue and I cannot get the VPS provider to upgrade it, what are my choices? Will disabling the quotas altogether help? Or perhaps disable them only in cPanel? How? Does anybody know?

Any help will be highly appreciated. Please note, that I have also opened a ticket on the cPanel customer portal, but I am afraid that I will not receive any reply before my clients kill me, or perhaps themselves. I would like to avoid that.

Thank you!
Michal

 

[cPanelMichael]

I have checked also the inodes and they seem fine.

Hello

What percentage of inodes are available on this VPS? Is it possible an account or application is temporarily using up your inodes, and the levels go back down by the time you check the usage? Do you notice any fail counts for your VPS resource limits when using the "cat /proc/user_beancounters" command?


Thank you.

 

[Hieroglyf]

Thank you for your reply.

The inodes (through df -i) look fine:

Filesystem  Inodes  IUsed  IFree IUse% Mounted on
simfs  83886080 1051857 82834223  2% /
none  524288  110  524178  1% /dev
none  524288  3889  520399  1% /tmp

the cat /proc/user_beancounters returns the following:

Version: 2.5
  uid  resource  held  maxheld  barrier  limit  failcnt
  2222243:  kmemsize  1007176200  1090928640  2147483648  2147483648  0
  lockedpages  0  5651  524288  524288  0
  privvmpages  270542  341663  9223372036854775807  9223372036854775807  0
  shmpages  22448  38363  9223372036854775807  9223372036854775807  0
  dummy  0  0  9223372036854775807  9223372036854775807  0
  numproc  112  198  9223372036854775807  9223372036854775807  0
  physpages  991390  1052495  0  1048576  0
  vmguarpages  0  0  9223372036854775807  9223372036854775807  0
  oomguarpages  123102  168325  9223372036854775807  9223372036854775807  0
  numtcpsock  63  133  9223372036854775807  9223372036854775807  0
  numflock  13  25  9223372036854775807  9223372036854775807  0
  numpty  1  1  9223372036854775807  9223372036854775807  0
  numsiginfo  0  30  9223372036854775807  9223372036854775807  0
  tcpsndbuf  1972688  8545992  9223372036854775807  9223372036854775807  0
  tcprcvbuf  1032192  2333728  9223372036854775807  9223372036854775807  0
  othersockbuf  189320  897184  9223372036854775807  9223372036854775807  0
  dgramrcvbuf  0  890008  9223372036854775807  9223372036854775807  0
  numothersock  107  157  9223372036854775807  9223372036854775807  0
  dcachesize  991718767  1073741824  1073741824  1073741824  0
  numfile  3115  3960  9223372036854775807  9223372036854775807  0
  dummy  0  0  9223372036854775807  9223372036854775807  0
  dummy  0  0  9223372036854775807  9223372036854775807  0
  dummy  0  0  9223372036854775807  9223372036854775807  0
  numiptent  1313  1316  9223372036854775807  9223372036854775807  0

(All failcnt are zeros)

Since the issue is persistent, I don't think there is any specific application that would "spike" out and eat the inodes.

 

[Hieroglyf]

Just a small update: I am now unable to even create a new user :-(. While displaying "Creating bandwidth datastore" overlay, it shows some error underneath the overlay, so I cannot see the details. There is definitely something rotten in there, but I just cannot figure it out. I even tried downloading and storing random files, just to make sure the VPS can actually store new files. And it can. So there is free space. But somehow cPanel thinks there isn't and it all revolves around those quotas…

 

[Hieroglyf]

I have received a response from the cPanel technical support, basically stating the above, that the quotas don't work properly and that it seems like the VPS node might be somehow misconfigured, second level quotas being turned off etc. So I am preparing to migrate elsewhere, since the current VPS provider doesn't really communicate with me. Pitiful.

 

[cPanelMichael]

I have received a response from the cPanel technical support, basically stating the above, that the quotas don't work properly and that it seems like the VPS node might be somehow misconfigured, second level quotas being turned off etc. So I am preparing to migrate elsewhere, since the current VPS provider doesn't really communicate with me. Pitiful.

Thank you for taking the time to update this thread with the outcome of the support ticket. Let us know if you have any additional questions.