SOLVED Server Crashing due to /dev/mapper/cloudlinux-root full

[matthewmalk248]

Hey gang, kinda freaking out here as our server keeps crashing with the 50G root partiton saying its 100% full, but using du -sh *, all I'm able to see actually being used is about 13G. I've already moved mysql data to /home/ partition to save a few GB but can't seem to find where this ghostly 35GB+ is hiding. Is something broken on the partition?

Results of du -sch * --exclude=home | sort -n

du: cannot access ‘proc/1312/task/1459/fd/489’: No such file or directory
du: cannot access ‘proc/12164’: No such file or directory
du: cannot access ‘proc/12285/task/12285/fd/4’: No such file or directory
du: cannot access ‘proc/12285/task/12285/fdinfo/4’: No such file or directory
du: cannot access ‘proc/12285/fd/4’: No such file or directory
du: cannot access ‘proc/12285/fdinfo/4’: No such file or directory
du: cannot access ‘proc/12300’: No such file or directory
du: cannot access ‘proc/12306’: No such file or directory
0       bin
0       dev
0       lib
0       lib64
0       proc
0       sbin
0       scripts
0       sys
3.3G    var
4.0K    backup
4.0K    home1
4.0K    homerust
4.0K    media
4.0K    razor-agent.log
4.0K    srv
8.0K    quota.user
8.9M    run
9.1G    usr
16K     lost+found
18M     root
37M     etc
187M    boot
252K    ca
368K    tmp
422G    mnt
435G    total
875M    opt
1012K   public_html

Results of df

Filesystem                    1K-blocks        Used  Available Use% Mounted on
/dev/mapper/cloudlinux-root    51474912    49777560          0 100% /
devtmpfs                        1926276           0    1926276   0% /dev
tmpfs                           1936808           0    1936808   0% /dev/shm
tmpfs                           1936808        9048    1927760   1% /run
tmpfs                           1936808           0    1936808   0% /sys/fs/cgroup
/dev/sda1                        487634      192983     264955  43% /boot
/dev/mapper/cloudlinux-home   245132620    16927088  215730348   8% /home
[email protected]_IP_ADDRESS:/  20404107248 16169225424 4234881824  80% /mnt/sbs
tmpfs                            387364           0     387364   0% /run/user/0

Results of cat /etc/fstabs

#
# /etc/fstab
# Created by anaconda on Sun Feb 21 19:38:37 2016
#
# Accessible filesystems, by reference, are maintained under '/dev/disk'
# See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info
#
/dev/mapper/cloudlinux-root     /       ext4    usrjquota=quota.user,jqfmt=vfsv0        1       1
UUID=0aee59fd-4cab-4b78-ac8d-529e944226c1 /boot                   ext4    defaults        1 2
/dev/mapper/cloudlinux-home     /home   ext4    usrjquota=quota.user,jqfmt=vfsv0        1       2
/dev/mapper/cloudlinux-swap swap                    swap    defaults        0 0

#/dev/sdb1      /homerust       ext4    usrjquota=quota.user,jqfmt=vfsv0        0       0

#nonssd=/homerust /homerust ext4 defaults,usrquota,noatime 0 0
[email protected]_IP_ADDRESS:/ /mnt/sbs fuse.sshfs  defaults,_netdev,port=3922,nonempty  0  0

 

[cPanelMichael]

Hello,

Do you notice any hidden directories (directories that start with a period) in the / partition? These aren't scanned when using the "*" entry in the du command.

Thank you.