Server takes too long to reboot

[RockSolid]

Hi Experts,

I have an 8 core server with 16GB RAM that runs cPanel. Since past few months, I noted that the server takes about 45mins to 1 hours to be up after a reboot. My OS is CentOs 6.3. I contacted the data center regarding the issue and they said; The server was at a Quota check and it took too long.

I have another server with same config and it reboots within 20 minutes. what has gone wrong with my old server? is it an issue with Hard disk?

Any suggestion to fix this would be appreciated.

Thank you.
Rock.

 

[PenguinInternet]

It's more likely to be running an fsck on reboot. Have you checked your messages log to see what processes are running on reboot? Even better, try and monitor this with a KVM for example.

To be honest, I'd also be worried about a server taking 20 minutes to reboot. Ours normally reboot in under 5 minutes and normally not much more if they are running an fsck

 

[ThinIce]

In addition to what Penguin said, if the check is happening on each reboot and there is nothing in messages as to why, you might like to check for the forcefsck file

ls -al /forcefsck

How much data does the server have on it / what is your disk subsystem?

 

[Martin.Harper]

I think the best way would be to monitor it through KVM, as mentioned by PenguinInternet. It will give you a fair idea of what is going on there.

 

[RockSolid]

Thank you all for your replies. I am not that familiar with this.

This is my /etc/fstab


#
# /etc/fstab
# Created by anaconda on Tue Apr 16 16:44:59 2013
#
# Accessible filesystems, by reference, are maintained under '/dev/disk'
# See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info
#
UUID=e20f5275-d805-40b4-865e-66ae6dc0fc63 / ext4 noatime,usrjquota=aquota.user,grpjquota=aquota.group,jqfmt=vfsv0 1 1
UUID=2c079929-72d3-4a59-8774-8b0a4218c6ac /boot ext2 defaults 1 2
UUID=5734fb40-ff90-4fb6-aa46-22f474a00736 swap swap defaults 0 0
UUID=ffec3522-7cbe-48c8-845b-c659ec862515 swap swap defaults 0 0
tmpfs /dev/shm tmpfs defaults 0 0
devpts /dev/pts devpts gid=5,mode=620 0 0
sysfs /sys sysfs defaults 0 0
proc /proc proc defaults 0 0
/usr/tmpDSK /tmp ext3 defaults,noauto 0 0



Can I disable fsck on reboot? If so, please let me know how.

Thank you!

 

[PenguinInternet]

The fsck process runs for a reason, therefore I wouldn't recommend at all just disabling these on reboot as you'll eventually run into a mess, I can assure you. You need to find out why this is happening. What is in your messages log when the server reboots - is it even an fsck that is causing the delay?

 

[RockSolid]

The fsck process runs for a reason, therefore I wouldn't recommend at all just disabling these on reboot as you'll eventually run into a mess, I can assure you. You need to find out why this is happening. What is in your messages log when the server reboots - is it even an fsck that is causing the delay?

Thank you for the reply! Please tell me how do I check server reboot log? I am running centos 6 and cPanel.

Thank you!

 

[RockSolid]

The fsck process runs for a reason, therefore I wouldn't recommend at all just disabling these on reboot as you'll eventually run into a mess, I can assure you. You need to find out why this is happening. What is in your messages log when the server reboots - is it even an fsck that is causing the delay?

Please find my boot log below. Is there something wrong ?


^[%G Welcome to CloudLinux Server
Starting udev: ^[%G^[[60G[^[[0;32m OK ^[[0;39m]

Setting hostname xxxxx.xxxxx.xxxxx: ^[[60G[^[[0;32m OK ^[[0;39m]

Checking filesystems
/dev/md0 was not cleanly unmounted, check forced.
^A/dev/md0: |= | 1.1%
^B^A/dev/md0: |====== / 10.9%
^B^A/dev/md0: |============== - 25.2%
^B^A/dev/md0: |===================== \ 38.3%
^B^A/dev/md0: |============================= | 51.4%
^B^A/dev/md0: |===================================== / 66.7%
^B^A/dev/md0: |========================================================| 100.0%
^B^A
^B/dev/md0: 50/131072 files (8.0% non-contiguous), 81648/524276 blocks
^[[60G[^[[0;33mPASSED^[[0;39m]

Remounting root filesystem in read-write mode: ^[[60G[^[[0;32m OK ^[[0;39m]

Mounting local filesystems: ^[[60G[^[[0;32m OK ^[[0;39m]

Checking local filesystem quotas: ^[[60G[^[[0;32m OK ^[[0;39m]

Enabling local filesystem quotas: ^[[60G[^[[0;32m OK ^[[0;39m]

Enabling /etc/fstab swaps: ^[[60G[^[[0;32m OK ^[[0;39m]

Entering non-interactive startup
Calling the system activity data collector (sadc):
Starting securetmp: *** Notice *** No loop module detected
If the loopback block device is built as a module, try running `modprobe loop` as root via ssh and running this script again.
If the loopback block device is built into the kernel itself, you can ignore this message.
Securing /tmp & /var/tmp
Calculating size on /tmp
/tmp calculated to be 4096 M based on available disk space in /usr
Everything looks good with your /tmp. Its the right size and ready to go.
Setting up /tmp... Done
Setting up /var/tmp... Done
Checking fstab for entries ...Done
Logrotate TMPDIR already configured
Process Complete

 

[tipwon]

good job. thank you

 

[phankhue]

I think the best way would be to monitor it through KVM, as mentioned by PenguinInternet.

 

[PenguinInternet]

Ok, it's running an FSCK due to the following:

/dev/md0 was not cleanly unmounted, check forced

Firstly, how are you rebooting the server? Also, can you post the output from dumpe2fs -h /dev/md0

 

[RockSolid]

Firstly, how are you rebooting the server?

I am extremely sorry for the delay in response.

I issue "reboot" and sometimes shutdown -r now to reboot the server.

Also, can you post the output from dumpe2fs -h /dev/md0

dumpe2fs 1.41.12 (17-May-2010)
Filesystem volume name: <none>
Last mounted on: <not available>
Filesystem UUID: 2c079929-72d3-4a59-8774-8b0a4218c6ac
Filesystem magic number: 0xEF53
Filesystem revision #: 1 (dynamic)
Filesystem features: ext_attr resize_inode dir_index filetype sparse_super
Filesystem flags: signed_directory_hash
Default mount options: (none)
Filesystem state: not clean
Errors behavior: Continue
Filesystem OS type: Linux
Inode count: 131072
Block count: 524276
Reserved block count: 26213
Free blocks: 421673
Free inodes: 131016
First block: 1
Block size: 1024
Fragment size: 1024
Reserved GDT blocks: 256
Blocks per group: 8192
Fragments per group: 8192
Inodes per group: 2048
Inode blocks per group: 256
RAID stride: 4
Filesystem created: Tue Apr 16 16:38:34 2013
Last mount time: Wed May 22 04:32:07 2013
Last write time: Wed May 22 04:32:07 2013
Mount count: 1
Maximum mount count: -1
Last checked: Wed May 22 04:32:04 2013
Check interval: 15552000 (6 months)
Next check after: Mon Nov 18 03:32:04 2013
Reserved blocks uid: 0 (user root)
Reserved blocks gid: 0 (group root)
First inode: 11
Inode size: 128
Default directory hash: half_md4
Directory Hash Seed: e30942ca-4bf4-4579-a320-0165a47cbdec

Thank you very much!

 

[PenguinInternet]

Ok, the first thing to note is that your filesystem is not clean:

Filesystem state: not clean

This will be why it is running an fsck on each reboot as the filesystem is unclean. I'd suggest rebooting it with a KVM on it and following the fsck process through - at a guess it's not able to correct something automatically, hence remaining in an unclean state. Your /var/log/messages file should however also give some indication as to the problem as the server reboots.

 

[whr]

Try fastboot

Hello,

If the server boot/reboot take long time always, do a manual fsck. If the file system is clean, add "fastboot" in grub to skip the file system check.