/dev/loop0 (/tmp) is 100% full

[SoN9ne]

I have found many articles about this but have yet to see one that solves this problem...

I have been receiving this message almost every month now.

/dev/loop0 (/tmp) is 100% full

I have already increased my /tmp directory to 1GB and am thinking about increasing it to 2GB... I am a little concerned since I noticed there are a few files that are unusually large and I have no clue as to what they are, I am hoping someone here will know.

The files that are unusually large are: (filename - size)
34bd161eccfacc7d-odbJw2 - 75582
getID37NK1qg.7&brs\=32745 - 566079488 (This is huge...)
getID3IUn0Af.7&brs\=6529 - 263851490
getID3ljXWoI.6&brs\=4470 - 137348853

Does anyone know what these files are from? Are these cpanel updates? Is it safe to delete these files?

I hope someone can give me some insight and can help me isolate the issue and prevent it from happening again...

Thanks,
Jeremy

[SoN9ne]

Nvm I had been able to isolate the issue. It was getID3 from a wordpress install.

[vanessa]

One of your users is doing this. If you're running suPHP and it's a PHP script, the ownership of the files should tell you who is causing the problem. You can band-aid it all you want by increasing the tmp disk size, but you need to figure out who on your server is filling it up.

[chapsrulez]

I have already increased my /tmp directory to 1GB and am thinking about increasing it to 2GB... I am a little concerned since I noticed there are a few files that are unusually large and I have no clue as to what they are, I am hoping someone here will know.Thanks,
Jeremy

Jeremy.

I have the same problem, i mean im running out of space in /tmp but havent found a way to unmount it add more space and mount it again.

Could you please tell me how you did this.
Thanks a lot
/cas

Nevermind, i already found how.
http://adamstechblog.com/2009/11/03/...ion-on-cpanel/

[postcd]

I have same problem Getting these messages quite frequently. Not sure what is the cause and how to find it?

[Sparrow-Sean]

You can remove the files alternatively whilst you manage a mount:

rm -rf *

That should remove everything in the TMP file, you can always manage a CRONTAB -E command to delete the files of this directory, the TMP will eventually fill up and this is a possible solution to size increase.

[GaryT]

You can remove the files alternatively whilst you manage a mount:

rm -rf *

I would not recommend that as if the user ends up in the wrong directory.. then... well I can only imagen the outcome you do know MySql is there ?

/tmp directory is more of less your session files and such, You can pretty much remove all, If you do the following:

cd /tmp

then

ls -la

you may see tons of files like sess_fea3b004782e544393d9453550ced004

if you do then just run this:

rm -f -r ses*

[cPanelJeff]

There is a utility called "tmpwatch" that should exist in a default install of CentOS. tmpwatch exists to handle the removal of "old" files from /tmp and other locations on the server. From the manpage for tmpwatch:

DESCRIPTION

       tmpwatch  recursively  removes files which haven’t been accessed for a given number of hours. Normally, it’s 
       used to clean up directories which are used for temporary holding space such as /tmp.

tmpwatch runs via cron:

[root@cpj ~]# cat /etc/cron.daily/tmpwatch 
flags=-umc

/usr/sbin/tmpwatch "$flags" -x /tmp/.X11-unix -x /tmp/.XIM-unix \
        -x /tmp/.font-unix -x /tmp/.ICE-unix -x /tmp/.Test-unix 240 /tmp

/usr/sbin/tmpwatch "$flags" 720 /var/tmp

for d in /var/{cache/man,catman}/{cat?,X11R6/cat?,local/cat?}; do
    if [ -d "$d" ]; then
        /usr/sbin/tmpwatch "$flags" -f 720 "$d"
    fi
done

The first command without the excluded paths essentially looks like this:

/usr/sbin/tmpwatch -umc 240 /tmp

The "240" is the number of hours, which would be 10 days. If you want to have files removed from /tmp more frequently, you may try changing 240 to a lesser value. This would be more preferred than running a cron job that runs "rm -rf" on everything inside /tmp, as that could interfere with the MySQL and PostgreSQL sockets, and will interfere with EasyApache while it's running.

[GaryT]

Or you can use your custom one and set it to run when you like which also cleans other things, Runs very good.

Do the following:

nano /scripts/cleantmp

Add:

# This script cleans out /tmp of empty, root, cpanel and nobody session files in /tmp

# if --test is passed, we just show the results
if [ "$1" == "--test" ]
then
CMD="-exec ls -la"
echo "$0: test mode"
else
CMD="-exec rm -rf"
fi

# remove empty session files that are over 2 hours old
find /tmp -name "sess*" -empty -mmin +120 -maxdepth 1 $CMD {} \;

# remove root owned session files
find /tmp -name "sess*" -user root -maxdepth 1 $CMD {} \;

# remove nobody session files
find /tmp -name "*sess*" -user nobody -maxdepth 1 $CMD {} \;

# remove cpanel owned session files
find /tmp -name "sess*" -user cpanel -maxdepth 1 $CMD {} \;

# remove any session file over 4 hours old
find /tmp -name "sess*" -mmin +240 -maxdepth 1 $CMD {} \;

# remove any spamassassin file over 4 hours old
find /tmp -name ".spamassassin*" -mmin +240 -maxdepth 1 $CMD {} \;

Save and close:


run this:

chmod 755 /scripts/cleantmp

run:

crontab -e

add:

0 0 * * * /scripts/cleantmp > /dev/null 2>&1

save and close:

reload / restart crond service:

/etc/init.d/crond restart

thats it

This cron will then run the script every day and clear out the /tmp folder.

Hope this helps

[postcd]

Thank You for nice example and tutorial.

I realised that issue of quick flooding space of this /dev/loop0 is caused by sending high amount of emails using SMTP protocol.

It reaches value like 99%+- and i get email, but its quickyl cleaned back to 3% usage..

Can be affected anyohow webserver or websites hosted on this server?
How to prevent it when we know SMTP is the cause?

[twhiting9275]

cPanel's minimal tmp setup is horrific at best. With stuff climbing up there constantly, it should be increased from 512mb to at least a 2-3 gig partition. Here's how to do this:

Step 1: Remove the old tmp space

service httpd stop
service mysql stop
rm -rf /usr/tmpDSK

Step 2, edit the script used to create tmp space

nano /scripts/securetmp

In there, look for

my $tmpdsksize     = 512000;    # Must be larger than 250000

and replace it with

my $tmpdsksize     = 3072000;    # Must be larger than 250000

Save (ctrl-X , then y when asked), and reboot the server

shutdown -r now

The system will adjust temp sizes when it comes back up.

For whatever reason, unmounting tmp is close to impossible, even if mysql and http aren't running. Your best bet is to follow the above (which I've done many times and never had a problem with)

[postcd]

I have just followed above guide to resize /temp size from 512MB to 4GB. My server has 8 core cpu and 8GB RAM.

After restart, it is 5 minutes and load of the server is very high and websites show very slow:

LOAD: 45.89 (8 CPUs)

After 10 minutes load is 8.12 (8 CPUs) and goes down......

What can be the cause? PS: Is it good idea to have such big /tmp on this configuration? (Server is used for webhosting, there is plenty of free RAM, but requires alot of CPU)

[twhiting9275]

The first reboot will cause your server to have to create the new partition, which is going to mean that it's going to be loaded down a bit. This will pass, just ride it out.

[postcd]

Thanks. What happend is that LOAD went down under value 8, then i deduplicated one large file using shell. After it was done there was running full cpanel backup, that load is 40-50 more than one hour!

Then i retrminated cpbackup, but loand do not went down.

Restart of Apache and Mysql dont help, load is up. High was 80.

What should i do please?

Considering making smaller /tmp ?



it gets better

usually before change CPU was 3.00 (8) and RAM 12% used

Edit: Load is back at 70 (((

[twhiting9275]

following the steps I posted above won't lead to massive increased load repeatedly, on a normally functioning hard drive. If, however your hard drive has issues, you're going to see that, no matter what. Check your server logs