You must reboot the server to apply kernel updates warning not dismissing after reboot

[Kent Brockman]

Hello everybody.

I'm seing this issue happenning on several VPS servers. No matter if you do a Graceful Reboot or enter "reboot" command via console, the warning "You must reboot the server to apply kernel updates." won't dissappear.

Common elements to those servers?
cPanel version = 82
Current Tier = RELEASE
OS = CENTOS 7.6 kvm
CloudLinux = not installed

needs-restarting command via console returns nothing, so the warning should not appear, right?

# needs-restarting
#

This behaviour started after cPanel v82
What may be failing to dismiss this warning?

Thanks in advance.

 

[Kent Brockman]

Oh almost forgot: at Tweak Settings, Enable Linux kernel update during nightly maintenance = OFF

 

[quietFinn]

You could try to run:

yum update --assumeno

in command line.

 

[Kent Brockman]

Yep. I did:

# yum update --assumeno
Loaded plugins: fastestmirror, universal-hooks
Loading mirror speeds from cached hostfile
 * EA4: 72.29.88.74
 * cpanel-addons-production-feed: 72.29.88.74
 * cpanel-plugins: 72.29.88.74
 * base: mirror.hackingand.coffee
 * centosplus: mirror.hackingand.coffee
 * extras: mirror.hackingand.coffee
 * updates: mirror.hackingand.coffee
No packages marked for update

 

[cPanelLauren]

Hi @Kent Brockman

Can you provide the output of the following:

whmapi1 system_needs_reboot

rpm -qa |grep kernel

uname -r

 

[Kent Brockman]

Yes, here we are:

[root]# whmapi1 system_needs_reboot
---
data:
  details:
    kernel:
      boot_version: 0-rescue-287a0145183844888c4a7fe5f823fe33
      running_version: 3.10.0-957.27.2.el7.centos.plus.bug16242.x86_64
  needs_reboot: 1
metadata:
  command: system_needs_reboot
  reason: OK
  result: 1
  version: 1

[root]# rpm -qa |grep kernel

kernel-plus-headers-3.10.0-957.27.2.el7.centos.plus.bug16242.x86_64
kernel-plus-3.10.0-514.21.1.el7.centos.plus.x86_64
kernel-plus-3.10.0-327.36.3.el7.centos.plus.x86_64
kernel-plus-3.10.0-514.16.1.el7.centos.plus.x86_64
kernel-plus-devel-3.10.0-957.27.2.el7.centos.plus.bug16242.x86_64
kernel-plus-tools-3.10.0-957.27.2.el7.centos.plus.bug16242.x86_64
kernel-plus-tools-libs-3.10.0-957.27.2.el7.centos.plus.bug16242.x86_64
kernel-plus-3.10.0-862.11.6.el7.centos.plus.x86_64
kernel-plus-3.10.0-957.27.2.el7.centos.plus.bug16242.x86_64

[root]# uname -r
3.10.0-957.27.2.el7.centos.plus.bug16242.x86_64

 

[cPanelLauren]

Thanks for that! It looks like the expected boot version in this instance is the rescue not the kernel version you're running which is the most up to date CentOS Plus kernel.

You should be able to get the boot order by running something like the following:

awk '/menuentry/ && /class/ {count++; print count-1"****"$0 }' /boot/grub2/grub.cfg

You might want to double-check also that grub.cfg is the name of the file in /boot/grub2/ if it returns an error telling you the file doesn't exist.

The boot order starts at 0. So for example this is what I have:

[root@centos7 ~]# awk '/menuentry/ && /class/ {count++; print count-1"****"$0 }' /boot/grub2/grub.cfg
0****menuentry 'CentOS Linux (3.10.0-957.10.1.el7.x86_64) 7 (Core)' --class centos --class gnu-linux --class gnu --class os --unrestricted $menuentry_id_option 'gnulinux-3.10.0-957.10.1.el7.x86_64-advanced-5f1b3a60-8f2a-46fa-9a04-62bb62de9338' {
1****menuentry 'CentOS Linux (0-rescue-d416c1d56b51458db7bdf98185b746dc) 7 (Core)' --class centos --class gnu-linux --class gnu --class os --unrestricted $menuentry_id_option 'gnulinux-0-rescue-d416c1d56b51458db7bdf98185b746dc-advanced-5f1b3a60-8f2a-46fa-9a04-62bb62de9338' {

With that I can see that 0 is the kernel I want to use, so to change this I can run something like:

grub2-set-default 0

Changes like these are highly sensitive and if you are at all unsure of how to make these modifications I would strongly urge you to reach out to your provider or system administrator. If you do not have a qualified system administrator you might find one here: System Administration Services.

Thanks!

 

[Kent Brockman]

Hello, I was unable to take on this until today.

Well, no grub.cfg was found:

# awk '/menuentry/ && /class/ {count++; print count-1"****"$0 }' /boot/grub2/grub.cfg
awk: fatal: cannot open file `/boot/grub2/grub.cfg' for reading (No such file or directory)
# dir /boot/grub2/
device.map  fonts  grub.cfg.bak  grubenv  i386-pc  locale

Why would exist a .bak file?
Anyway I run your suggested command on it:

# awk '/menuentry/ && /class/ {count++; print count-1"****"$0 }' /boot/grub2/grub.cfg.bak

0****menuentry 'CentOS Linux, with Linux 3.10.0-123.el7.x86_64' --class centos --class gnu-linux --class gnu --class os --unrestricted $menuentry_id_option 'gnulinux-3.10.0-123.el7.x86_64-advanced-ac9e46e0-6de5-4e6f-baf2-cca39dffe49b' {
1****menuentry 'CentOS Linux, with Linux 0-rescue-d95a50005420493eb46cd8b69ebd9275' --class centos --class gnu-linux --class gnu --class os --unrestricted $menuentry_id_option 'gnulinux-0-rescue-d95a50005420493eb46cd8b69ebd9275-advanced-ac9e46e0-6de5-4e6f-baf2-cca39dffe49b' {

What would you suggest? Isn't grub.cfg not there for a reason?
Thanks!

 

[cPanelLauren]

As I mentioned previously, your grub.cfg file may be called something else. It could also be in /boot/grub/

The articles here go over this pretty well:

HowTos/Grub2

How to set default boot kernel on Linux ( CentOS / RHEL 7 )

26.5. Customizing the GRUB 2 Configuration File Red Hat Enterprise Linux 7 | Red Hat Customer Portal

The Red Hat Customer Portal delivers the knowledge, expertise, and guidance available through your Red Hat subscription.

access.redhat.com

 

[Kent Brockman]

As I mentioned previously, your grub.cfg file may be called something else. It could also be in /boot/grub/

May the file have an alternative name?

# locate grub.conf
/boot/grub/grub.conf
/boot/grub/grub.conf.kvm
/boot/grub/grub.conf.kvm_virtio
/boot/grub/grub.conf.xen
/boot/grub/grub.conf~
/etc/grub.conf

# locate grub.cfg
/boot/grub2/grub.cfg.bak

Or.. provided that this is a VPS server using kvm, maybe the dc provider would need to reboot the entire node?

 

[aztopdavid]

This thread appears to have ended without resolution, and because it's very similar to my recent thread, I'll give it a bump:

SOLVED - 'You must reboot the server to apply kernel updates' persists after multiple reboots

My VPS is running v86.0.18 (RELEASE) on CENTOS 7.8 and I have a persistent "You must reboot the server to apply kernel updates" warning that doesn't go away after rebooting (several times). The latest "yum -y update" results in "No packages marked for update' and "whmapi1 system_needs_reboot"...

forums.cpanel.net

Something from this thread that didn't come up in mine: @Kent Brockman supplied this bit of info:

Enable Linux kernel update during nightly maintenance = OFF

That's also OFF in my Tweak Settings, but I wonder why it's the default cPanel setting? What would happen if we changed it to "ON"? Might that solve this issue?

 

[Kent Brockman]

Well, hello. I finally ended solving this issue by changing default=0 to default=1 in /etc/grub.conf
Best regards

 

[aztopdavid]

Thanks. That change hasn't worked for me (yet), but I'm still curious about that Tweak Settings default.

 

[Kent Brockman]

Thanks. That change hasn't worked for me (yet), but I'm still curious about that Tweak Settings default.

That default in Tweak Settings just determine if the server must include Kernel updates or not.
Then you must ask yourself: "are Kernel updates a must?" Well, it's a bit long to explain, so I googled it for you :

What is Linux Kernel, and should you upgrade to the latest Kernel?

Don't worry, I'm not going to write another wiki article on Linux Kernel (I know there is Wikipedia for that!), but rather I'm going to explain Linux Kernel for an end-user in simple layman's terms. What is Linux Kernel? Linux Kernel is an important piece of program that forms the…

www.fosslinux.com

 

[cPanelLauren]

The issue you have @aztopdavid is different than what @Kent Brockman had - though similar-sounding your server is not changing to the boot version when restarted.

 

[kamber9]

I can't get the message
You must reboot the server to apply kernel updates warning not dismissing after reboot
to disappear.

data:
details:
kernel:
boot_version: 3.10.0-1160.53.1.el7.x86_64
running_version: 3.10.0-1160.25.1.el7.x86_64
needs_reboot: 1
metadata:
command: system_needs_reboot
reason: OK
result: 1
version: 1

The grub shows 0 (set as my default boot) as 3.10.0-1160.53.1.el7.x86_64 so i'm not sure what's going on?

TIA.

 

[Rawand Sae]

I have the same issue but in my case, the kernel boot_version is newer than running_version!

[root@srv1 ~]# whmapi1 system_needs_reboot
---
data:
details:
kernel:
boot_version: 4.18.0-348.7.1.el8_5.x86_64
running_version: 4.18.0-348.2.1.el8_5.x86_64
needs_reboot: 1
metadata:
command: system_needs_reboot
reason: OK
result: 1
version: 1

[root@srv1 ~]# rpm -qa |grep kernel
kernel-4.18.0-348.7.1.el8_5.x86_64
kernel-4.18.0-348.2.1.el8_5.x86_64
kernel-modules-4.18.0-348.7.1.el8_5.x86_64
kernel-core-4.18.0-348.2.1.el8_5.x86_64
kernel-headers-4.18.0-348.7.1.el8_5.x86_64
kernel-tools-4.18.0-348.7.1.el8_5.x86_64
kernel-core-4.18.0-348.7.1.el8_5.x86_64
kernel-tools-libs-4.18.0-348.7.1.el8_5.x86_64
kernel-modules-4.18.0-348.2.1.el8_5.x86_64

[root@srv1 ~]# uname -r
4.18.0-348.2.1.el8_5.x86_64


any help would be much appreciated.

 

[cPanelAnthony]

Hello! Could either of you open a ticket using the link in my signature and update me with the incident ID?

 

[Rawand Sae]

Hello! Could either of you open a ticket using the link in my signature and update me with the incident ID?

on my server /boot partition was nearly full, after removing some unused img files in /boot now both boot_version and running_version are identical using the newest kernel version
explained in this thread:

DISKWARN blocks Mount Point /boot currently use 85.1%

Hi, I received multiple Disk usage warnings about the Mount Point “/boot” partition being almost full 85.1% Percent Used! I have checked the server to see if the kernel installations are causing this but I can confirm there are only 2 versions of the kernel installed...

forums.cpanel.net

 

[cPRex]

I'm glad to hear things are working well now!