Drive Critical: /dev/loop0 (/var/tmp) is 97% full

[cPanelTristan]

Did you try running the script a second time after receiving that initial failure?

 

[ronald.prism]

Did you try running the script a second time after receiving that initial failure?

Yes. Doing so outputs the same notice -


*** Notice *** No loop module detected
If the loopback block device is built as a module, try running `modprobe loop` as root via ssh and running this script again.
If the loopback block device is built into the kernel itself, you can ignore this message.
*** Notice *** No working loopback device files found. Try running `modprobe loop` as root via ssh and running this script again.

Nothing else changes.


Thanks

 

[cPanelTristan]

Is your system Virtuozzo or OpenVZ? If so, then /scripts/securetmp will not work on those systems due to not having a loopback device on them.

 

[ronald.prism]

Is your system Virtuozzo or OpenVZ? If so, then /scripts/securetmp will not work on those systems due to not having a loopback device on them.

It's a straight-up dedicated server.

I did an -

# /sbin/lsmod | grep loop
loop 18633 2

but /scripts/securetmp terminates on that notice regardless.

Looking at the securetmp script, both checks below fail to detect the loop device.

    if ( !grep /loop/, @modules ) {
        print "*** Notice *** No loop module detected\n";    # could be built into kernel, so don't bail out yet
        print "If the loopback block device is built as a module, try running `modprobe loop` as root via ssh and running this script again.\n";
        print "If the loopback block device is built into the kernel itself, you can ignore this message.\n";
    }
    if ( !$has_loop_device ) {
        print "*** Notice *** No working loopback device files found. Try running `modprobe loop` as root via ssh and running this script again$

How risky would it be to run the script with those checks disabled?

 

[cPanelTristan]

I wouldn't try running it without the checks. Could you please submit a ticket to us about this issue? You can use WHM > Support Center > Contact cPanel or use the link in my signature.

 

[DreamPhysix]

Today I noticed that my tmp folder /usr/tmpDSK is relatively high today. In crontab I have 0 * * * * /usr/sbin/tmpwatch --mtime --all 24 /tmp so I figured I could run /usr/sbin/tmpwatch --mtime --all 24 /tmp in cli to clean up the partition, but the disk usage didn't change. Does this mean that tmpwatch is not working properly? No output was given when issuing the "/usr/sbin/tmpwatch --mtime --all 24 /tmp" command

 

[cPanelMichael]

No output was given when issuing the "/usr/sbin/tmpwatch --mtime --all 24 /tmp" command

You can use the verbose flag when running the command manually. EX:

/usr/sbin/tmpwatch --verbose --mtime --all 24 /tmp

This will let you know which files and directories are removed. You can try decreasing the "mtime" value if you want to remove newer files and directories.

Thank you.

 

[TapanB]

Hello,

What size is your /tmp partition? You can resize it if it isn't large enough. Many servers have only 500MB for /tmp which isn't large enough and should be around 2GB instead. You could edit /scripts/securetmp for this line:

my $tmpdsksize     = 512000;    # Must be larger than 250000

Change to this line for 2GB:

my $tmpdsksize     = 2097152;    # Must be larger than 250000

Save the file wq), then run these commands to resize /tmp partition:

umount -l /tmp
umount -l /var/tmp
rm -fv /usr/tmpDSK
/scripts/securetmp

Unfortunately, if you aren't able to catch it when it has the high usage, it's difficult for us to state why it's alerting. It could be high session activity (those sessions do clear on their own) for some site during the times it hits the high disk space level. It could be MySQL having the temporary MyISAM tables filling it up. It could be EAccelerator writing files to it (if you use that, it writes some temporary files there). There are many reasons it could be happening. If you aren't certain, you can simply resize it to be larger. Otherwise, you'll have to watch it closely until you do catch it and then provide the details on what you are seeing.

There are also some scripts that will periodically clear tmp such as tmpwatch, but you need to be careful not to cron clear /tmp as EasyApache writes to /tmp during build, so you cannot simply clear files every 15 minutes or something, since you'll then break EasyApache from recompiling properly.

Thanks.

Hi,

I got same email as the op and I followed your instructions and now my server is down. I am getting error:

/tmp/crontab.XXXXJRVkLb: Read-only file system

How to fix this ???

Please help my server is down.

Thanks.

 

[tank]

I would suggest submitting a ticket if this is the case. Servers down =

 

[cPanelTristan]

Running the commands indicated in this thread would not cause the machine to go down and submitting a ticket to cPanel would be unable to fix this error:

/tmp/crontab.XXXXJRVkLb: Read-only file system

The /tmp partition is read only and indicates file system corruption or disk issues. You would need to contact your provider, datacenter or NOC to reboot the machine and have them run an fsck on it. This is outside of anything to do with cPanel and is a hardware level issue.

 

[gkgcpanel]

Tristan,

Ok, so I followed the instructions below and moved mysqltmp to our backup drive (which has plenty of space).

Only question I have is, what is to prevent a user from escaping to a MySQL Shell prompt (which can be easily done), and be able to list all of our customers backups?

You can use the following steps to move the MySQL /tmp directory location:

1. Edit /etc/my.cnf file to set new tmpdir for MySQL

vi /etc/my.cnf

Place the following line into the file under the [mysqld] section:

tmpdir=/mysqltmp

Save the file with :wq

2. Create the /mysqltmp directory

mkdir /mysqltmp
chmod 1777 /mysqltmp

3. Restart MySQL and check that the tmpdir is now /mysqltmp

/etc/init.d/mysql restart
mysqladmin var | grep tmpdir

You should see the following return:

# mysqladmin var | grep tmpdir
| slave_load_tmpdir                       | /mysqltmp
| tmpdir                                  | /mysqltmp

This presumes / partition itself has sufficient space. I would like to hope / partition would have more space than /tmp was provided on most systems.

 

[SitexMedia]

I have decided not to install tempwatch for now as I hope the advice here will cure the tmp directory getting full

After changing my /scripts/securetmp to

my $tmpdsksize = 2097152; # Must be larger than 250000

I then

umount -l /tmp
umount -l /var/tmp
rm -fv /usr/tmpDSK
/scripts/securetmp

and received this warning notice

*** Notice *** No loop module detected
If the loopback block device is built as a module, try running `modprobe loop` as root via ssh and running this script again.
If the loopback block device is built into the kernel itself, you can ignore this message.
*** Notice *** No working loopback device files found. Try running `modprobe loop` as root via ssh and running this script again.

Would you like to secure /tmp & /var/tmp at boot time? (y/n) y
Would you like to secure /tmp & /var/tmp now? (y/n) y

Securing /tmp & /var/tmp
/tmp is already secure
/var/tmp is already secure
Checking fstab for entries ...Done
Logrotate TMPDIR already configured
Process Complete

Can anybody tell me does the above look right? as I did not notice on other posts the two lines asking to secure the tmp folders

I ran the above a couple of times and always the same. I did not run using modprobe loop as unsure what that is

Thanks

 

[SitexMedia]

Once again, since I did the update above i have had an email

The file system /usr/tmpDSK, which is mounted at /tmp, has reached warn status because it is 89% full

Used: 1.66GB
Available: 0.21GB
Total: 1.97GB

I can increase the tmp size to 4gb and see what happens, I have looked at the files and see

drwxrwxrwt. 4 root root 110592 Mar 19 00:49 ./
dr-xr-xr-x. 22 root root 4096 Mar 17 11:49 ../
drwxr-xr-x. 2 mijasfor mijasfor 4096 Mar 18 16:47 drush-env/
-rw-------. 1 costapro costapro 0 Mar 18 09:40 file0U0O1C - <<<---- lots of these

drwx------. 2 root root 16384 Mar 17 11:49 lost+found/


-rw-------. 1 ukcityes ukcityes 359 Mar 19 00:28 sess_04b4c368056bdade9cf162ffaaebb8d9
-rw-------. 1 bedinbri bedinbri 105 Mar 19 00:39 sess_04cad4e85272e3c82cc878b79029efa9
-rw-------. 1 ukcityes ukcityes 248 Mar 19 00:28 sess_055946f36d4b89ae09c52d74c558dcc5

various of the above type lines

I will increase the size of the tmp to see what happens but may need to do more than that

Any thoughts?

 

[DomineauX]

Try the following to see what large files are in /tmp:
ls -lahS /tmp/ | head

If disk space is fine when you check it though it might have been a MySQL tmp table that was quite large but gone once it was no longer needed (query completed or failed) by MySQL.

Better to find what is consuming space and resolve it than just increasing /tmp until it doesn't run low.

 

[irwebco]

Hello,

What size is your /tmp partition? You can resize it if it isn't large enough. Many servers have only 500MB for /tmp which isn't large enough and should be around 2GB instead. You could edit /scripts/securetmp for this line:

my $tmpdsksize     = 512000;    # Must be larger than 250000

Change to this line for 2GB:

my $tmpdsksize     = 2097152;    # Must be larger than 250000

Save the file wq), then run these commands to resize /tmp partition:

umount -l /tmp
umount -l /var/tmp
rm -fv /usr/tmpDSK
/scripts/securetmp

Unfortunately, if you aren't able to catch it when it has the high usage, it's difficult for us to state why it's alerting. It could be high session activity (those sessions do clear on their own) for some site during the times it hits the high disk space level. It could be MySQL having the temporary MyISAM tables filling it up. It could be EAccelerator writing files to it (if you use that, it writes some temporary files there). There are many reasons it could be happening. If you aren't certain, you can simply resize it to be larger. Otherwise, you'll have to watch it closely until you do catch it and then provide the details on what you are seeing.

There are also some scripts that will periodically clear tmp such as tmpwatch, but you need to be careful not to cron clear /tmp as EasyApache writes to /tmp during build, so you cannot simply clear files every 15 minutes or something, since you'll then break EasyApache from recompiling properly.

Thanks.

hello dear

This is the wrong way

for resize /tmp partition you can use :

cp /scripts/securetmp /scripts/securetmp.bck

find this line in the /scripts/securetmp

 $tmpdsksize = int($tmpdsksize);

after find go to five line top you can see similar this line

my $FOUR_GIG_k = ( 1024 * 1024 * 2 );

this mean is 4G byte size of /tmp partition
You can modify the numbers to Favorites , for example i will build /tmp with 16G byte space i change this line to

    my $FOUR_GIG_k = ( 2048 * 2048 * 4 );

2048 * 2048 * 4 = 16G Byte space
after change save and exite from file and run another command on ssh area

service cpanel stop
service httpd stop
service mysql stop
service exim stop
cp -r /tmp /root/tmp.backup
cp -r /usr/tmpDSK /root/tmpDSK.backup
umount -l /tmp
umount -l /var/tmp
rm -fv /usr/tmpDSK
/scripts/securetmp

finish your tmp partition is resized now
run again all service

service cpanel start
service httpd start
service mysql start
service exim start