Is CentOS 4.3 , kernel 2.6.9-34 stable ?

fastdns

Member
Jul 17, 2003
12
0
151
India
Hi

I am using :

OS : CentOS release 4.3 (Final)

CPanel Version : WHM 10.8.0 cPanel 10.8.1-R113

Kernel Version : 2.6.9-34.ELsmp

I am observing excessive server load. Usually it runs at 20-30 cpu and spikes up to 65 cpu when I have to stop exim to bring it down.

I have been complaining about high server loads for the past 2 months and everytime I was told that there is nothing wrong with the version of the kernel that I have.

Now, other people using Cent Os 4.3 have reported similar issues with high server load and memory management.

Please read this thread : http://forums.cpanel.net/showthread.php?t=54280&highlight=centos+high+server+load

Read the bug report at : http://bugs.centos.org/view.php?id=1174

have thoroughlly checked my box and there is nothing "unusual" happening on my server that should cause such high server loads, almost to the tune of 70 CPU ??? All that I can see if that the server load goes high while waiting for an i/o read from HD.

My current top shows :

top - 23:50:03 up 8 days, 2:55, 2 users, load average: 65.59, 45.05, 27.05
Tasks: 388 total, 2 running, 374 sleeping, 0 stopped, 12 zombie
Cpu(s): 5.6% us, 3.9% sy, 0.0% ni, 0.0% id, 90.2% wa, 0.3% hi, 0.0% si
Mem: 969596k total, 300172k used, 669424k free, 3312k buffers
Swap: 4192956k total, 333064k used, 3859892k free, 16540k cached

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
12125 mailnull 17 0 8820 3316 2300 S 1.2 0.3 0:00.04 exim
38 root 15 0 0 0 0 D 0.9 0.0 19:15.24 kswapd0
18514 root 15 0 3032 1028 676 S 0.6 0.1 0:16.75 top
4123 root 16 0 3888 1036 680 R 0.6 0.1 0:06.51 top
12161 mailnull 16 0 8816 3336 2324 D 0.6 0.3 0:00.07 exim
12162 cpanel 18 0 9940 3712 1952 D 0.6 0.4 0:00.03 php
12169 root 18 0 11864 3736 1368 D 0.6 0.4 0:00.02 cpsrvd
199 root 15 0 0 0 0 D 0.3 0.0 4:45.77 kjournald
1949 autobulb 15 0 7428 3652 3096 D 0.3 0.4 0:00.04 boxtrapper
8149 nobody 15 0 17796 5548 1400 S 0.3 0.6 0:00.07 httpd
12166 root 16 0 8236 3716 2684 S 0.3 0.4 0:00.03 exim
12182 mailnull 16 0 8808 3124 2168 S 0.3 0.3 0:00.03 exim

Any help on this from anyone in this forum would be highly appreciated.

Regards
 

AndyReed

Well-Known Member
PartnerNOC
May 29, 2004
2,221
4
193
Minneapolis, MN
fastdns said:
Kernel Version : 2.6.9-34.ELsmp

I am observing excessive server load. Usually it runs at 20-30 cpu and spikes up to 65 cpu when I have to stop exim to bring it down.
The kernel is not the only cause of high server load on a CentOS server. Down/upgrading might help, in your case you need to go after the source of the trouble. I suggest you search these forums since this issue has been discussed so many times. Good luck!
 

fastdns

Member
Jul 17, 2003
12
0
151
India
Dear Andy

I have spent so much time in this forum searcing for something that would help me, but all in vain.

Problem is that the same number of accounts were running on Fedora 4 before I shifted my server to this new datacenter and got a CentOS. My new server is running with even lesser accounts now, I have shut down services like spamassassin also, I have moved heavy email users to other servers, but I fail to understand that even then , why is is always running at a high usage. :(

For example, see this top :

top - 02:25:30 up 7 days, 5:30, 2 users, load average: 20.25, 26.61, 23.83
Tasks: 248 total, 2 running, 238 sleeping, 1 stopped, 7 zombie
Cpu(s): 5.0% us, 7.3% sy, 0.0% ni, 0.0% id, 87.4% wa, 0.3% hi, 0.0% si
Mem: 969596k total, 223720k used, 745876k free, 1052k buffers
Swap: 4192956k total, 323824k used, 3869132k free, 15320k cached

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
38 root 15 0 0 0 0 D 6.0 0.0 17:27.63 kswapd0
12697 mailnull 16 0 9284 4372 3280 D 1.0 0.5 0:00.03 exim
12952 root 18 0 10068 3740 2692 S 1.0 0.4 0:00.06 exim
6435 root 16 0 3140 1052 680 S 0.6 0.1 0:06.31 top

The "D" state is the real issue in those processes. In the snippet above, I can see two exim processes, and a kswapd (kernel swap manager) are in D state, meaning they are waiting on feedback from hardware. These processes can get stuck in an infinite read() if the hardware never times out, such as reading from a harddrive.

I'm not sure how much a kernel update will help such a problem, however it's certainly worth a shot. But my datacenter team is refusing to do this as they say that the last stable version is -34 as mentioned on http://mirror.centos.org/centos/4.3/updates/i386/RPMS/

I am stuck....thinking of moving over back to the old company :rolleyes:
 

AndyReed

Well-Known Member
PartnerNOC
May 29, 2004
2,221
4
193
Minneapolis, MN
fastdns said:
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
12697 mailnull 16 0 9284 4372 3280 D 1.0 0.5 0:00.03 exim
12952 root 18 0 10068 3740 2692 S 1.0 0.4 0:00.06 exim

a shot. But my datacenter team is refusing to do this as they say that the last stable version is -34 as mentioned on http://mirror.centos.org/centos/4.3/updates/i386/RPMS/
kernel-2.6.9-40 is the latest and most stable version. We upgraded our clients' kernel to this release and works just fine. If you are interested in learning more about high server load, read this thread: http://forums.cpanel.net/showthread.php?t=50421 Hope this helps!
 

rjf1505

Member
Apr 16, 2005
8
0
151
fastdns said:
Hi

I am using :

OS : CentOS release 4.3 (Final)

CPanel Version : WHM 10.8.0 cPanel 10.8.1-R113

Kernel Version : 2.6.9-34.ELsmp

I am observing excessive server load. Usually it runs at 20-30 cpu and spikes up to 65 cpu when I have to stop exim to bring it down.

I have been complaining about high server loads for the past 2 months and everytime I was told that there is nothing wrong with the version of the kernel that I have.

Now, other people using Cent Os 4.3 have reported similar issues with high server load and memory management.


Regards

I have excactly the same problem !

My host has also swapped my hardware (PERFECT SERVICE !) and the problem stays.
When disabling EXIM i can hold the server under control. I only start exim once in a while.

The server worked fine until several weeks ago, i am going to check if i can see when the kernel was updated.

Is the solution up-down grading the kernel ?

Thanks
 

AndyReed

Well-Known Member
PartnerNOC
May 29, 2004
2,221
4
193
Minneapolis, MN
MMarko said:
It is possible to compile latest kernel 2.6.17.3 from source on production server without any problems.
Yes, it is possible, but it depends on the level of expertise with Linux/FreeBSD OS. If you compile Kernel from source, and just in case it didn't work, you can always switch back to the previous working version you had.
 

chilihost

Well-Known Member
Mar 1, 2005
72
0
156
I have upgraded a couple of my servers to 2.6.9-39.0.2.ELsmp and it is WAY WAY WAY better...see this thread: http://forums.cpanel.net/showthread.php?t=56045

The upgrade is very easy too, you just download the RMPs from the testing area on centOS. I have developed the following set of instructions if you are using grub as your os loader:
Code:
cd /var/spool/up2date

wget http://dev.centos.org/centos/4/testing/i386/RPMS/kernel-2.6.9-39.0.2.EL.i686.rpm
wget http://dev.centos.org/centos/4/testing/i386/RPMS/kernel-devel-2.6.9-39.0.2.EL.i686.rpm
wget http://dev.centos.org/centos/4/testing/i386/RPMS/kernel-hugemem-devel-2.6.9-39.0.2.EL.i686.rpm
wget http://dev.centos.org/centos/4/testing/i386/RPMS/kernel-smp-2.6.9-39.0.2.EL.i686.rpm
wget http://dev.centos.org/centos/4/testing/i386/RPMS/kernel-smp-devel-2.6.9-39.0.2.EL.i686.rpm

rpm -ivh kernel-2.6.9-39.0.2.EL.i686.rpm kernel-devel-2.6.9-39.0.2.EL.i686.rpm kernel-hugemem-devel-2.6.9-39.0.2.EL.i686.rpm kernel-smp-2.6.9-39.0.2.EL.i686.rpm kernel-smp-devel-2.6.9-39.0.2.EL.i686.rpm
Then to be safe, set your server to boot into the new kernel only once:
Code:
vi /boot/grub/grub.conf
in grub, change order of new kernel to second, make the old kernel the first one. Then do the following:

Code:
grub
wait a few seconds for this to come up, then type the following lines
Code:
savedefault --default=1 --once
quit
then reboot your server and check to see if the new kernel was successful:
uname -r

if it is then edit grub.conf so that the new kernel is the first in the list. If this does not work, simply reboot / power cycle and you should come up with your old kernel.

Good luck, use these instructions and this test kernel AT YOUR OWN RISK!!!
 

trhosting.net

Well-Known Member
Mar 7, 2006
153
1
168
Turkey
Redhat released the official kernel-2.6.9-42.
This is the latest stable kernel for RHEL4 (and CentOS 4.3).

We have compiled and installed this kernel on one of our servers.
Now we are testing the performance.
I will notify you about the performance.

If anyone interests on compiling this kernel, see below:

Code:
wget ftp://ftp.redhat.com/pub/redhat/linux/updates/enterprise/4ES/en/os/SRPMS/kernel-2.6.9-42.EL.src.rpm
rpm -ihv kernel-2.6.9-42.EL.src.rpm
The sources of the kernel is now in /usr/src/redhat
Code:
cd /usr/src/redhat
rpmbuild -bb --target=i686 SPECS/kernel-2.6.spec
It takes about 30 minutes to compile the kernel. After a successfull compile, the new kernel rpms are located on "/usr/src/redhat/RPMS/i686" folder.
Code:
-rw-r--r--  1 root root  11327798 Aug 14 05:27 kernel-2.6.9-42.EL.i686.rpm
-rw-r--r--  1 root root 316150714 Aug 14 05:53 kernel-debuginfo-2.6.9-42.EL.i686.rpm
-rw-r--r--  1 root root   3880571 Aug 14 05:27 kernel-devel-2.6.9-42.EL.i686.rpm
-rw-r--r--  1 root root  10818123 Aug 14 05:29 kernel-hugemem-2.6.9-42.EL.i686.rpm
-rw-r--r--  1 root root   3906847 Aug 14 05:29 kernel-hugemem-devel-2.6.9-42.EL.i686.rpm
-rw-r--r--  1 root root  10939279 Aug 14 05:28 kernel-smp-2.6.9-42.EL.i686.rpm
-rw-r--r--  1 root root   3897131 Aug 14 05:28 kernel-smp-devel-2.6.9-42.EL.i686.rpm
You can now install what kernel you want.
 

minzhong

Active Member
Jan 9, 2005
31
0
156
Hi all.

I'm currently running Dual Xeons.

Do dual processor systems require different 'type' of kernel?

I tried other kernels in the past but they don't work.

I'm currently using the default 2.6.9-34.ELsmp from the default CentOS 4.3 installation.

Please advise!

Thank you..
 

chirpy

Well-Known Member
Verifed Vendor
Jun 15, 2002
13,465
30
473
Go on, have a guess
They simply need to smp kernel. You should be able to use the latest CentOSv4 kernel unless you have RAID hardware or the like that the manufacturer hasn't bothered supporting (there are a few). The kernel you are running (2.6.9-34.ELsmp) is now old and vulnerable to root escalation compromises.