/dev/mapper/centos-root is full! but is it?

OrpheusCCC

Registered
Dec 21, 2021
2
1
3
Michigan
cPanel Access Level
Root Administrator
Hi, so I have a situation that is confounding me. after a df -h it shows quite obviously that the /dev/mapper/centos-root is 99% full.


>>>
[[email protected] /]# df -h
Filesystem Size Used Avail Use% Mounted on
devtmpfs 2.9G 0 2.9G 0% /dev
tmpfs 2.9G 8.0K 2.9G 1% /dev/shm
tmpfs 2.9G 314M 2.6G 11% /run
tmpfs 2.9G 0 2.9G 0% /sys/fs/cgroup
/dev/mapper/centos-root 50G 50G 564M 99% /
/dev/sda1 1014M 188M 827M 19% /boot
/dev/mapper/centos-home 845G 232G 614G 28% /home
/dev/loop0 2.3G 3.8M 2.2G 1% /tmp
tmpfs 579M 0 579M 0% /run/user/0
/dev/sdb1 1008G 676G 282G 71% /backup-2021
>>>

through other forum posts, I have attempted to figure out where the data hog is, however, there are no directories that correlate with this info from the df command.

>>>

[[email protected] /]# du -sch * --exclude=home
0 backup
676G backup-2021
0 bin
155M boot
0 dev
76M etc
0 lib
0 lib64
0 media
0 mnt
1.6M ncdu-1.16
838M opt
du: cannot access ‘proc/31710/task/31710/fd/4’: No such file or directory
du: cannot access ‘proc/31710/task/31710/fdinfo/4’: No such file or directory
du: cannot access ‘proc/31710/fd/4’: No such file or directory
du: cannot access ‘proc/31710/fdinfo/4’: No such file or directory
0 proc
4.0K razor-agent.log
31M root
314M run
0 sbin
0 scripts
0 srv
0 sys
964K tmp
6.2G usr
2.4G var
686G total
>>>

the "backup" doesn't count, as it is separate. but according to this output the /root is only 31M, and the biggest data users only account for 8.6 G with usr and var put together.

Many of these "fixes" for the issue end up being "how can I resize the drive". But I want to know why it reads as "full" without any corresponding directories or files that appear to be the culprit, and how can I fix it, and prevent it from filling up in the future. Having a larger drive would just allow it to fill up again later.

further note: Logs are being managed, so /var/log isn't the culprit. /tmp is not an issue, as it appears to only be 964K. Any usual sources for this don't appear to be the issue here. I am perplexed.

The most I can see is that the /dev/mapper/centos-root is what is referenced as full, which maps back to /dev/dm-0

I am not super familiar with the LVM mapper, but could there be a file that is a part of the LVM system between the linking of /dev/mapper/centos-root and dev/dm-0 that wouldn't show up as part of /root?

Thank you for any help.
 

OrpheusCCC

Registered
Dec 21, 2021
2
1
3
Michigan
cPanel Access Level
Root Administrator
Hi, so I have a situation that is confounding me. after a df -h it shows quite obviously that the /dev/mapper/centos-root is 99% full.


>>>
[[email protected] /]# df -h
Filesystem Size Used Avail Use% Mounted on
devtmpfs 2.9G 0 2.9G 0% /dev
tmpfs 2.9G 8.0K 2.9G 1% /dev/shm
tmpfs 2.9G 314M 2.6G 11% /run
tmpfs 2.9G 0 2.9G 0% /sys/fs/cgroup
/dev/mapper/centos-root 50G 50G 564M 99% /
/dev/sda1 1014M 188M 827M 19% /boot
/dev/mapper/centos-home 845G 232G 614G 28% /home
/dev/loop0 2.3G 3.8M 2.2G 1% /tmp
tmpfs 579M 0 579M 0% /run/user/0
/dev/sdb1 1008G 676G 282G 71% /backup-2021
>>>

through other forum posts, I have attempted to figure out where the data hog is, however, there are no directories that correlate with this info from the df command.

>>>

[[email protected] /]# du -sch * --exclude=home
0 backup
676G backup-2021
0 bin
155M boot
0 dev
76M etc
0 lib
0 lib64
0 media
0 mnt
1.6M ncdu-1.16
838M opt
du: cannot access ‘proc/31710/task/31710/fd/4’: No such file or directory
du: cannot access ‘proc/31710/task/31710/fdinfo/4’: No such file or directory
du: cannot access ‘proc/31710/fd/4’: No such file or directory
du: cannot access ‘proc/31710/fdinfo/4’: No such file or directory
0 proc
4.0K razor-agent.log
31M root
314M run
0 sbin
0 scripts
0 srv
0 sys
964K tmp
6.2G usr
2.4G var
686G total
>>>

the "backup" doesn't count, as it is separate. but according to this output the /root is only 31M, and the biggest data users only account for 8.6 G with usr and var put together.

Many of these "fixes" for the issue end up being "how can I resize the drive". But I want to know why it reads as "full" without any corresponding directories or files that appear to be the culprit, and how can I fix it, and prevent it from filling up in the future. Having a larger drive would just allow it to fill up again later.

further note: Logs are being managed, so /var/log isn't the culprit. /tmp is not an issue, as it appears to only be 964K. Any usual sources for this don't appear to be the issue here. I am perplexed.

The most I can see is that the /dev/mapper/centos-root is what is referenced as full, which maps back to /dev/dm-0

I am not super familiar with the LVM mapper, but could there be a file that is a part of the LVM system between the linking of /dev/mapper/centos-root and dev/dm-0 that wouldn't show up as part of /root?

Thank you for any help.

Solution was:
the back-up drive apparently had been unmounted when a couple back-ups were being made, and the backup was still being stored, even though it didn't have the separate drive partition mounted, leaving the backups in the root file, but appearing to be in the backup file. Deleting the rogue backups cleared up root, and creating an auto-mount scenario for the backup drive is supposedly going to solve the issue in the future.
 
  • Like
Reactions: cPanelAnthony