2.6.9-42.0.3.ELsmp kernel woes

[chilihost]

I just upgraded cPanel on one of our servers running CentOS 4.4 and it forced a kernel upgrade from 2.6.9-42.0.2.ELsmp to 2.6.9-42.0.3.ELsmp and now the system is running with a high load average (averaging about 2.0-2.5).

Has anyone else noticed this? I am going to reboot back to 42.0.2

And why is cPanel forcing a kernel upgrade now? This is new since 10.9.0 and I for one am NOT happy with this.

 

[chilihost]

after rebooting into 42.0.2, the load average has stabalized in the 0.25-0.75 range and this server is running nicely again.

Has anyone else experienced the same?

 

[lehels]

high load

hello,

We are experiencing the same issue here, but much higher load that you mentioned [about: 20-30 Load average],
How many domains are you have on that server?
Wich is the total size of the mysql databases?

cheers,

 

[matthewdavis]

The problem I had with the 42.0.3 kernel was a kernel panic while I was restoring accounts. Looking at the panic it looked to be related to quota. I haven't gotten around to filing a bug with Red Hat yet.

 

[ispro]

The problem I had with the 42.0.3 kernel was a kernel panic while I was restoring accounts. Looking at the panic it looked to be related to quota. I haven't gotten around to filing a bug with Red Hat yet.

We have it on TWO different servers with SMP 2.6.9-42.0.3 kernel (RHEL4).

Server hangs on "Restoring quota". One of the servers is P4 Dual core and another is high-end Dual Woodcrest...

The pity is that servers are new and we have no older kernels to reboot.

Transferring accounts to them is VERY annoying task!

 

[matthewdavis]

I would really like to see the output of the kernel panic. I'm going to PM you an email address to send them to if that would be ok.

But you can download and install a different kernel and install it manually. Using these commnads it should work:

# up2date --get kernel-2.6.9-42.0.2.EL.i686
and/or
# up2date --get kernel-smp-2.6.9-42.0.2.EL.i686

That would download the kernel to /var/spool/up2date and you can install it from that dir. Using:

# rpm -ivh /var/spool/up2date/kernel-smp-2.6.9-42.0.2.EL.i686.rpm

 

[bmorrice]

we are experiencing the same issues as matthewdavis, its hard to recreate - but when account transfers or account creations occur we get a kernel panic related to quota as well.

this however hasn't been limited to the 2.6.9-42.0.3.EL.i686, we have had the issue happen on 2.6.9-42.0.3.EL.i686

At the moment this is only happening on one of our cpanel boxes, and happens every ~10 or so days....

ben

 

[matthewdavis]

Looks like Red Hat is aware of the problem. This Bugzilla should be helpful:

https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=205001

There is a link to a test kernel. Please, please please do not install the test kernel on a production system. This usually means the the fix will be in the next version of RHEL.

 

[IRCBrasil]

Same problem here. Looking at log when the server crash, I found this:

kernel: kernel BUG at fs/dquot.c:422!

some idea?

 

[gemby]

after rebooting into 42.0.2, the load average has stabalized in the 0.25-0.75 range and this server is running nicely again.

Has anyone else experienced the same?

I have same issue here, currently running 42.0.3 please confirm that high server loads has gone away after booting in 42.0.2. Server is mostly running ok, but after few days, some basic i/o commands a damn slow, eg. mkdir takes 5-6 seconds or so.....

 

[mickalo]

I have same issue here, currently running 42.0.3 please confirm that high server loads has gone away after booting in 42.0.2. Server is mostly running ok, but after few days, some basic i/o commands a damn slow, eg. mkdir takes 5-6 seconds or so.....

what is the best way to reboot to another kernel version as we also run 42.0.3 and have noticed alot of minor problems. does something need to be changed in the grub.conf prior to reboot?

we run RHEL 4 - 2.6.9-42.0.3.ELsmp #1 SMP

thx's
Mickalo

 

[gemby]

what is the best way to reboot to another kernel version as we also run 42.0.3 and have noticed alot of minor problems. does something need to be changed in the grub.conf prior to reboot?

we run RHEL 4 - 2.6.9-42.0.3.ELsmp #1 SMP

thx's
Mickalo

It is simple, pico /boot/grub/grub.conf and edit default= line, first kernel below is 0 second in list is 1 etc....

 

[mickalo]

It is simple, pico /boot/grub/grub.conf and edit default= line, first kernel below is 0 second in list is 1 etc....

that's what I thought, just wanted to double check.

thx's
Mickalo

 

[gorilla]

I just installed the 2.6.9-42.32 kernel and it seems to be working fine

Check below the few steps .
Please be aware that I am using the 64bit version as I am running a dualcore opteron, for intel you should use the 32bit (686) version from the link here
http://people.redhat.com/~jbaron/rhel4/RPMS.kernel/

log into your server,


wget http://people.redhat.com/~jbaron/rhel4/RPMS.kernel/kernel-smp-2.6.9-42.32.EL.x86_64.rpm

and after the download has completed, install the kernel

rpm -ivh kernel-smp-2.6.9-42.32.EL.x86_64.rpm

Check to see if the new kernel is installed by default in grub.conf

cat /boot/grub/grub.conf

Default should be 0
And the kernel you've just installed should be the first one

reboot the server and check what kernel your system is using with uname -a

 

[matthewdavis]

Don't install that kernel unless you know what your doing. Thats an unofficial kernel by a Red Hat engineer ment for testing purposes. Not to be run on a production system.

 

[mike25]

I am seeing this as well with the 2.6.9-42.0.3.ELsmp kernel, Only so far while transferring accounts between servers. Is anyone having this problem with the 2.6.9-42.0.2.ELsmp Kernel? sounds like a downgrade is teh way to go. I would also perfer that cpanel not force a kernel upgrade unless there is a major security hole discovered. Many of us spend a good deal of time hacking our kernels for performance benefits.

 

[IRCBrasil]

With 2.6.9-42.0.2.ELsmp Kernel I never had this problem, but I did upgrade to 2.6.9-42.0.3.ELsmp Kernel becouse there was a security hole on .0.2.

I would also perfer that cpanel not force a kernel upgrade unless there is a major security hole discovered. Many of us spend a good deal of time hacking our kernels for performance benefits.

Is not a bug correction that make the system crash, a good reason to upgrade the kernel?

 

[MN-Robert]

The problem I had with the 42.0.3 kernel was a kernel panic while I was restoring accounts. Looking at the panic it looked to be related to quota. I haven't gotten around to filing a bug with Red Hat yet.

We've had this problem for quite some time probably 4-6 months. We are at softlayer if that makes any difference.

The previous kernel 2.6.9-42.0.2.ELsmp also displayed these problems see the redhat bug report for further info.

 

[IRCBrasil]

We've had this problem for quite some time probably 4-6 months. We are at softlayer if that makes any difference.

I am with this bug only at my 2 servers at softlayer (both servers are dual core 3.0 with SATA HDs). My servers at The Planet (one P4 2.8 with SATA and another dual xeon 2.8 with SATA), running the same 42.0.3 kernel, dont have this problem.
All servers (softlayer and the planet) running RH 4.0.

 

[MN-Robert]

Yes that's why I mentioned it. We have servers elsewhere running the stock standard kernel and it works fine. Softlayer compile their own for their own setup which I believe is causing the issues. But since we have had servers their the issue has existed (March or April 06)