SOLVED 'You must reboot the server to apply kernel updates' persists after multiple reboots

aztopdavid

Well-Known Member
Jan 1, 2016
53
9
58
Arizona
cPanel Access Level
Root Administrator
My VPS is running v86.0.18 (RELEASE) on CENTOS 7.8 and I have a persistent "You must reboot the server to apply kernel updates" warning that doesn't go away after rebooting (several times). The latest "yum -y update" results in "No packages marked for update' and "whmapi1 system_needs_reboot" produces:

kernel:
boot_version: 3.10.0-1062.18.1.el7.x86_64
running_version: 3.10.0-862.14.4.el7.x86_64
needs_reboot: 1
metadata:
command: system_needs_reboot
reason: OK
result: 1
version: 1

"rpm -qa |grep kernel" results:
kernel-3.10.0-1062.18.1.el7.x86_64
kernel-3.10.0-957.27.2.el7.x86_64
kernel-headers-3.10.0-1127.el7.x86_64
kernel-devel-3.10.0-862.14.4.el7.x86_64
kernel-3.10.0-862.14.4.el7.x86_64
kernel-devel-3.10.0-1062.12.1.el7.x86_64
kernel-3.10.0-1062.12.1.el7.x86_64
kernel-devel-3.10.0-1062.18.1.el7.x86_64
kernel-3.10.0-1127.el7.x86_64
kernel-devel-3.10.0-1127.el7.x86_64
kernel-devel-3.10.0-1062.9.1.el7.x86_64
texlive-l3kernel-svn29409.SVN_4469-45.el7.noarch

and finally "uname -r" produces:
3.10.0-862.14.4.el7.x86_64

I'm not sure if this issue will just go away on its own or if I need to do something to make the warning disappear? Thanks!
 

GOT

Get Proactive!
PartnerNOC
Apr 8, 2003
1,755
311
363
Chesapeake, VA
cPanel Access Level
DataCenter Provider
My guess is you have an lxc based vm. Known issue. You can turn this notice off in tweak settings but we haven't found a fix. Seems endemic with lxc.
 

GOT

Get Proactive!
PartnerNOC
Apr 8, 2003
1,755
311
363
Chesapeake, VA
cPanel Access Level
DataCenter Provider
Looking closer at your initial post, your system is not booting into the latest kernel, that is why you are getting the message. Latest is

1062.18.1

Whereas your running is

862.14.4

I would take a look here


To see about setting your system to boot the latest kernel.
 

aztopdavid

Well-Known Member
Jan 1, 2016
53
9
58
Arizona
cPanel Access Level
Root Administrator
Thanks, I saw that in an older thread, but wasn't sure if that would be an advisable solution for me. Perhaps I should just ask the host if/when they plan on doing system maintenance involving rebooting the node?
 

GOT

Get Proactive!
PartnerNOC
Apr 8, 2003
1,755
311
363
Chesapeake, VA
cPanel Access Level
DataCenter Provider
Doesn't really involve rebooting the node. For some reason your system is not flagging new kernels to be utilized, which is normally the default behavior, but this does not require a node reboot to do so.
 

aztopdavid

Well-Known Member
Jan 1, 2016
53
9
58
Arizona
cPanel Access Level
Root Administrator
OK, I'm working my way through the steps in "How to set default boot kernel on Linux" but it's not going all that smoothly. When I ran:

# awk -F\' '$1=="menuentry " {print $2}' /etc/grub2.cfg

I got:

CentOS Linux (3.10.0-1127.el7.x86_64) 7 (Core)
CentOS Linux 7 (Core)

However, when I poked around in the grub2.cfg file, looking at the details of the "menuentry" values in the "/etc/grub.d/10_linux" section, I found these, which include eight different "submenu" choices for that generic "CentOS Linux 7" value from the output above:

Code:
menuentry 'CentOS Linux (3.10.0-1127.el7.x86_64) 7 (Core)'
    linux16 /boot/vmlinuz-3.10.0-1127.el7.x86_64 root=/dev/sda1 ro LANG=en_US.UTF-8
    initrd16 /boot/initramfs-3.10.0-1127.el7.x86_64.img
menuentry 'CentOS Linux 7 (Core)'
    linux16 /boot/vmlinuz-3.10.0-1062.18.1.el7.x86_64 root=UUID=a44fb57f-8c52-7db3-f787-3f47c406178e ro
    initrd16 /boot/initramfs-3.10.0-1062.18.1.el7.x86_64.img
submenu 'Advanced options for CentOS Linux 7 (Core)'
   menuentry 'CentOS Linux (3.10.0-1062.18.1.el7.x86_64) 7 (Core)'
        linux16 /boot/vmlinuz-3.10.0-1062.18.1.el7.x86_64 root=UUID=a44fb57f-8c52-7db3-f787-3f47c406178e ro
        initrd16 /boot/initramfs-3.10.0-1062.18.1.el7.x86_64.img
   menuentry 'CentOS Linux (3.10.0-1062.18.1.el7.x86_64) 7 (Core) (recovery mode)'
        linux16 /boot/vmlinuz-3.10.0-1062.18.1.el7.x86_64 root=UUID=a44fb57f-8c52-7db3-f787-3f47c406178e ro single
        initrd16 /boot/initramfs-3.10.0-1062.18.1.el7.x86_64.img
   menuentry 'CentOS Linux (3.10.0-1062.12.1.el7.x86_64) 7 (Core)'
        linux16 /boot/vmlinuz-3.10.0-1062.12.1.el7.x86_64 root=UUID=a44fb57f-8c52-7db3-f787-3f47c406178e ro
        initrd16 /boot/initramfs-3.10.0-1062.12.1.el7.x86_64.img
   menuentry 'CentOS Linux (3.10.0-1062.12.1.el7.x86_64) 7 (Core) (recovery mode)'
        linux16 /boot/vmlinuz-3.10.0-1062.12.1.el7.x86_64 root=UUID=a44fb57f-8c52-7db3-f787-3f47c406178e ro single
        initrd16 /boot/initramfs-3.10.0-1062.12.1.el7.x86_64.img
   menuentry 'CentOS Linux (3.10.0-957.27.2.el7.x86_64) 7 (Core)'
        linux16 /boot/vmlinuz-3.10.0-957.27.2.el7.x86_64 root=UUID=a44fb57f-8c52-7db3-f787-3f47c406178e ro
        initrd16 /boot/initramfs-3.10.0-957.27.2.el7.x86_64.img
   menuentry 'CentOS Linux (3.10.0-957.27.2.el7.x86_64) 7 (Core) (recovery mode)'
        linux16 /boot/vmlinuz-3.10.0-957.27.2.el7.x86_64 root=UUID=a44fb57f-8c52-7db3-f787-3f47c406178e ro single
        initrd16 /boot/initramfs-3.10.0-957.27.2.el7.x86_64.img
   menuentry 'CentOS Linux (3.10.0-862.14.4.el7.x86_64) 7 (Core)'
        linux16 /boot/vmlinuz-3.10.0-862.14.4.el7.x86_64 root=UUID=a44fb57f-8c52-7db3-f787-3f47c406178e ro
        initrd16 /boot/initrd-3.10.0-862.14.4.el7.x86_64.img
   menuentry 'CentOS Linux (3.10.0-862.14.4.el7.x86_64) 7 (Core) (recovery mode)'
        linux16 /boot/vmlinuz-3.10.0-862.14.4.el7.x86_64 root=UUID=a44fb57f-8c52-7db3-f787-3f47c406178e ro single
        initrd16 /boot/initrd-3.10.0-862.14.4.el7.x86_64.img
So to my untrained eye, it appears that the first entry results in booting 1127, while the second option basically refers to the following three submenu choices:

1062.18.1
957.27.2
862.14.4

and as you mentioned before, there's a mismatch between the boot and the running version that's not currently being resolved by rebooting:

boot_version: 3.10.0-1062.18.1.el7.x86_64
running_version: 3.10.0-862.14.4.el7.x86_64

But if I successfully apply the procedures mentioned in "How to set default boot kernel?" might that result in running 1127 instead of 1062.18.1? Also, "/etc/default/grub" (mentioned in the article) doesn't exist. I'm not sure the article covers a setup with all the different "submenu" choices present on my system -- it appears to deal with a configuration with discrete "menuentry" values to choose from.
 

GOT

Get Proactive!
PartnerNOC
Apr 8, 2003
1,755
311
363
Chesapeake, VA
cPanel Access Level
DataCenter Provider
1127 is the very newest kernel for Cent7 so no suprise there. You've got a lot of kernels installed. It may be less confusing for you if you run

package-cleanup -y --oldkernels --count=3

Which will remove all but the latest kernels.
 
  • Like
Reactions: cPanelLauren

aztopdavid

Well-Known Member
Jan 1, 2016
53
9
58
Arizona
cPanel Access Level
Root Administrator
So, that didn't do it, after rebooting in my host's dashboard and logging back into WHM, I still had the "You must reboot..." warning, so I also did a Graceful Reboot in WHM and it's still there. I suppose I could sign up for KernelCare.
 

GOT

Get Proactive!
PartnerNOC
Apr 8, 2003
1,755
311
363
Chesapeake, VA
cPanel Access Level
DataCenter Provider
Does your host provide a console where you can see the server rebooting as if you were in front of it, if so you could select the kernel it boots into.

Output of this please:

cat /etc/sysconfig/kernel
 

GOT

Get Proactive!
PartnerNOC
Apr 8, 2003
1,755
311
363
Chesapeake, VA
cPanel Access Level
DataCenter Provider
I suspect a few things, but the article should help you. Assuming you have

grep GRUB_DEFAULT /etc/default/grub
GRUB_DEFAULT=saved

Then running

grub2-set-default 0

Then run

grub2-mkconfig -o /boot/grub2/grub.cfg

Then reboot.

If the grub_default setting above is not "saved" then change it before running the other commands.
 
  • Like
Reactions: cPanelLauren

aztopdavid

Well-Known Member
Jan 1, 2016
53
9
58
Arizona
cPanel Access Level
Root Administrator
cat /etc/sysconfig/kernel = "No such file or directory"

I also already mentioned that /etc/default/grub doesn't exist, which is partly why I'm stuck.

There's a "console" button in the host interface and I'm running it, but it appears to give me the same view as my PuTTY session with su root
 

GOT

Get Proactive!
PartnerNOC
Apr 8, 2003
1,755
311
363
Chesapeake, VA
cPanel Access Level
DataCenter Provider
if you have the console open while its booting you might (might) see a centos boot menu that lets you pick your kernel.

If you are missing those files something is wacky with your set up and I could not help you further without risking a non-bootable system.
 

GOT

Get Proactive!
PartnerNOC
Apr 8, 2003
1,755
311
363
Chesapeake, VA
cPanel Access Level
DataCenter Provider
Yeah that makes more sense, the lernels are not getting installed correctly.

I don't really know how to advise you here. You could ask your host it could be something they are doing on deployment for stability reasons.

If you don't want to move servers, get kernelcare to keep the kernel patched and turn off the reboot notices in tweak settings.