SOLVED Service failures after upgrading system to CentOS version 7.6

marcuszan

Well-Known Member
Apr 19, 2018
63
5
8
Netherlands
cPanel Access Level
Root Administrator
no errors, i do not succesfull boot on hdd from 24hours (im on netboot) and i get this for today :


DBG: 10919: Fri Jan 18 12:12:39 2019: command line: --default-kernel
DBG: Image entry succeeded:
DBG: menuentry 'GNU/Linux with OVH Kernel, OVH kernel 4.9.87-xxxx-std-ipv6-64' {
DBG: load_video
DBG: set gfxpayload=keep
DBG: insmod part_gpt
DBG: insmod part_gpt
DBG: insmod diskfilter
DBG: insmod mdraid09
DBG: insmod ext2
DBG: set root='mduuid/2fbcdcf2f2dde0c4a4d2adc226fd5302'
DBG: if [ x$feature_platform_search_hint = xy ]; then
DBG: search --no-floppy --fs-uuid --set=root --hint='mduuid/2fbcdcf2f2dde0c4a4d2adc226fd5302' 09c4d
bce-1565-42ce-a0b2-2a1179a74bdc
DBG: else
DBG: search --no-floppy --fs-uuid --set=root 09c4dbce-1565-42ce-a0b2-2a1179a74bdc
DBG: fi
DBG: linux /bzImage-4.9.87-xxxx-std-ipv6-64 root=/dev/md3 ro crashkernel=auto rhgb quiet rootde
lay=10
DBG: }
Above is the output from
/var/log/grubby ??
please mention this in your post.
The output shows that you successfully booted an OVH (net)Kernel. This gives you no info on boot issues when using the hdd kernel.

Read here : HowTos/Grub2 - CentOS Wiki
Try suggested steps to check what grub entry is set to default and check if this is correct. If not, change entry as described.
 

dahu

Member
Dec 7, 2001
23
2
303
ok, sorry.

Searching for a solution, after that all seems fine with grub as you said. I found this :
dedoimedo.com/computers/grub2-fedora-command-not-found.html

then i looked at my grub.cf generated (here is a part):
Code:
### BEGIN /etc/grub.d/06_OVHkernel ###
menuentry "GNU/Linux with OVH Kernel, OVH kernel 4.9.87-xxxx-std-ipv6-64" {
        load_video
        set gfxpayload=keep
        insmod part_gpt
        insmod part_gpt
        insmod diskfilter
        insmod mdraid09
        insmod ext2
        set root='mduuid/2fbcdcf2f2dde0c4a4d2adc226fd5302'
        if [ x$feature_platform_search_hint = xy ]; then
          search --no-floppy --fs-uuid --set=root --hint='mduuid/2fbcdcf2f2dde0c4a4d2adc226fd5302'  09c4dbce-15
65-42ce-a0b2-2a1179a74bdc
        else
          search --no-floppy --fs-uuid --set=root 09c4dbce-1565-42ce-a0b2-2a1179a74bdc
        fi
        linux   /bzImage-4.9.87-xxxx-std-ipv6-64 root=/dev/md3 ro crashkernel=auto rhgb quiet rootdelay=10
}
### END /etc/grub.d/06_OVHkernel ###

### BEGIN /etc/grub.d/10_linux ###
menuentry 'CentOS Linux (0-rescue-1e98a4cc1263468b9f9a12e365316b95) 7 (Core)' --class centos --class gnu-linux
--class gnu --class os --unrestricted $menuentry_id_option 'gnulinux-0-rescue-1e98a4cc1263468b9f9a12e365316b95-
advanced-c84d3726-95a0-477e-a4a8-a1fdf71b90c1' {
        load_video
        insmod gzio
        insmod part_gpt
        insmod part_gpt
        insmod diskfilter
        insmod mdraid09
        insmod ext2
        set root='mduuid/2fbcdcf2f2dde0c4a4d2adc226fd5302'
        if [ x$feature_platform_search_hint = xy ]; then
          search --no-floppy --fs-uuid --set=root --hint='mduuid/2fbcdcf2f2dde0c4a4d2adc226fd5302'  09c4dbce-15
65-42ce-a0b2-2a1179a74bdc
        else
          search --no-floppy --fs-uuid --set=root 09c4dbce-1565-42ce-a0b2-2a1179a74bdc
        fi
        linuxefi /vmlinuz-0-rescue-1e98a4cc1263468b9f9a12e365316b95 root=/dev/md3 ro crashkernel=auto rhgb quie
t rootdelay=10
        initrdefi /initramfs-0-rescue-1e98a4cc1263468b9f9a12e365316b95.img
}
then i looked at my installed grub2-module and efi modules seems installed :

Installed Packages
grub2.x86_64 1:2.02-0.76.el7.centos @base
grub2-common.noarch 1:2.02-0.76.el7.centos @base
grub2-efi-x64.x86_64 1:2.02-0.76.el7.centos @base
grub2-efi-x64-modules.noarch 1:2.02-0.76.el7.centos @base
so i looked at /etc/grub.d/06_OVHkernel

Code:
                        if [ "x{$GRUB_GFXPAYLOAD_LINUX}" != xtext ]; then
                                echo -e "\tload_video"
                        fi
                echo -e "\tset gfxpayload=\"${GRUB_GFXPAYLOAD_LINUX}\""
                fi
        fi
        prepare_grub_to_access_device ${GRUB_DEVICE_BOOT} | sed -e "s/^/\t/"
        cat << EOF
        linux   ${rel_dirname}/${basename} root=${linux_root_device_thisversion} ro ${GRUB_CMDLINE_LINUX} ${GRU
B_CMDLINE_LINUX_DEFAULT} ${IXGBE_BOOT_PARAM}
EOF
        initrd=""
        for i in "initrd.img-${version}" "initrd-${version}.img" \
           "initrd-${version}" "initramfs-${version}.img" \
           "initrd.img-${alt_version}" "initrd-${alt_version}.img" \
           "initrd-${alt_version}" "initramfs-${alt_version}.img" \
           "initramfs-genkernel-${version}" \
           "initramfs-genkernel-${alt_version}" \
           "initramfs-linux.img" \
           "initrd-ovh-generic-amd64.img"
        do
                if test -f "${dirname}/${i}" ; then
                        initrd="${i}"
                        break
                fi
        done

        if test -f "${dirname}/${initrd}" ; then
                echo "Found initrd image: ${rel_dirname}/${initrd}" >&2
                cat << EOF
        initrd ${rel_dirname}/${initrd}
EOF
        fi

        cat << EOF
}
EOF
and so, i didn't tried yet but i don't know if it make sens to edit it and replace linux by linuxefi (after "cat << eof" line ) and initrd by initrdefi (after "cat<< eof" line ) and regenerate grub.cfg.?
 
Last edited by a moderator:

dahu

Member
Dec 7, 2001
23
2
303
do you fill in >
/dev/md3
i dont have to select (on reboot ovh dashboard) device for hdd boot, only on netboot where i manually fill /dev/md3 after choosing the kernel i want (its the current state of my server ).
 

marcuszan

Well-Known Member
Apr 19, 2018
63
5
8
Netherlands
cPanel Access Level
Root Administrator
Thanks,

this indicated there is NO kernel installed OR configured, besides the rescue mode stuff. Your server cant boot if there is no installed kernel to choose from.
You will have to check if there are any kernels installed, else install the default centos kernel.
then reconfigure grub
 

dahu

Member
Dec 7, 2001
23
2
303
??? so why is ther a prompt on grub asking to choose between ovhKernel previously mentioned and rescue ?
Doesnt that mean that in fact the /etc/grub.d/06_OVHkernel with :
cat << EOF
linux ${rel_dirname}/${basename} root=${linux_root_device_thisversion} ro ${GRUB_CMDLINE_LINUX} ${GRU
B_CMDLINE_LINUX_DEFAULT} ${IXGBE_BOOT_PARAM}
simply must be linuxefi ?
 

marcuszan

Well-Known Member
Apr 19, 2018
63
5
8
Netherlands
cPanel Access Level
Root Administrator
I hope someone else can jump into this thread and give some ideas ? I am not sure what needs to be done @dafu ...


According to the output of your grub files, where I only see rescue, I assumed you need to install/configure a kernel to boot from.

MY output for example >
Code:
# grep vmlinuz /boot/efi/EFI/centos/grub.cfg
        linuxefi /vmlinuz-4.4.169-1.el7.elrepo.x86_64 root=/dev/md3 ro crashkernel=auto rhgb quiet vga=normal nomodeset rd.auto=1 rd.md.uuid=55fba025:dcfa45b1:a4d2adc2:26fd5302 rd.md.uuid=4f2ff053:53ba30e8:a4d2adc2:26fd5302 rootdelay=10 rootdelay=10 noquiet nosplash net.ifnames=0 biosdevname=0
        linuxefi /vmlinuz-4.4.166-1.el7.elrepo.x86_64 root=/dev/md3 ro crashkernel=auto rhgb quiet vga=normal nomodeset rd.auto=1 rd.md.uuid=55fba025:dcfa45b1:a4d2adc2:26fd5302 rd.md.uuid=4f2ff053:53ba30e8:a4d2adc2:26fd5302 rootdelay=10 rootdelay=10 noquiet nosplash net.ifnames=0 biosdevname=0
        linuxefi /vmlinuz-0-rescue-c1a63cb41f3a079c801badf65b464d62 root=/dev/md3 ro crashkernel=auto rhgb quiet vga=normal nomodeset rd.auto=1 rd.md.uuid=55fba025:dcfa45b1:a4d2adc2:26fd5302 rd.md.uuid=4f2ff053:53ba30e8:a4d2adc2:26fd5302 rootdelay=10 rootdelay=10 noquiet nosplash net.ifnames=0 biosdevname=0

So I have 2 working kernels to choose from and the rescue kernel
 

dahu

Member
Dec 7, 2001
23
2
303
its done. That was it.
in /etc/grub.d/06_OVHkernel

i replaced
Code:
linux   ${rel_dirname}/${basename} root=${linux_root_device_thisversion} ro ${GRUB_CMDLINE_LINUX} ${GRU
B_CMDLINE_LINUX_DEFAULT} ${IXGBE_BOOT_PARAM}
by
Code:
linuxefi   ${rel_dirname}/${basename} root=${linux_root_device_thisversion} ro ${GRUB_CMDLINE_LINUX} ${GRU
B_CMDLINE_LINUX_DEFAULT} ${IXGBE_BOOT_PARAM}
and
Code:
initrd ${rel_dirname}/${initrd}
by
Code:
initrdefi ${rel_dirname}/${initrd}
then
grub2-mkconfig -o /boot/grub2/grub.cfg
grub2-mkconfig -o /boot/efi/EFI/centos/grub.cfg

and the server rebooted on hdd normaly
 
  • Like
Reactions: cPanelMichael

dahu

Member
Dec 7, 2001
23
2
303
Code:
[[email protected] ~]# uname -a
Linux ns3616484.ip-151-80-24.eu 4.9.87-xxxx-std-ipv6-64 #1 SMP Tue Mar 13 18:41:47 CET 2018 x86_64 x86_64 x86_64 GNU/Linux
Code:
[[email protected] ~]# grep vmlinuz /boot/efi/EFI/centos/grub.cfg
        linuxefi /vmlinuz-0-rescue-1e98a4cc1263468b9f9a12e365316b95 root=/dev/md3 ro crashkernel=auto rhgb quiet rootdelay=10
 

marcuszan

Well-Known Member
Apr 19, 2018
63
5
8
Netherlands
cPanel Access Level
Root Administrator
@dahu Thanks for posting. It seems the OVH template that is used to install CentOS with the OVH kernel uses a different setup than stock CentOS regarding grub config files. You are running a kernel that is not listed in
/boot/efi/EFI/centos/grub.cfg
Anyway, glad all works for you again !
 

Benjamin D.

Well-Known Member
Jan 28, 2016
128
17
68
Canada
cPanel Access Level
Root Administrator
It is a centos issue which happened with the centos 7.6 update
Mostly, you need to disable SELinux and/or fix grub from console/KVM and reboot. If you have netboot option reboot via netboot, fix the issue and reboot again from disk.
Thanks, but I cannot afford to lose any more clients, I lost way too many last year when I spent DAYS upgrading to CentOS 7. If I could go back in time and I knew there would be that many issues, I would have stayed the hell away from it and I would have stayed on CentOS 6 like it has been for years before, and which ran PERFECTLY fine. I'm way too afraid to even attempt to reboot the server now. The next time any machine will be rebooted is when I migrate to another server. The last WHM update back in december BRICKED my production server without any warning. It got back up using netboot thanks to a generous OVH employee who investiguated it for 15 mins. Now that it's back in operation, it's gonna stay EXACTLY like it is now for a year, then I'll migrate to another server with an already installed CentOS 7.6. A simple minor WHM update should NEVER, EVER brick a production server, especially when you're on the STABLE "branch"... like WTF?! I'm paying $$ every month for a WHM/cPanel licence and I cannot believe that's what my money is buying lately; Is this dashboard even tested prior to be released under a "production stable" tier? There USED TO BE a difference between LTS, STABLE, RELEASE and CURRENT but over the last year or so, I don't remember seeing much of a difference between the four tiers.
 

rpvw

Well-Known Member
Jul 18, 2013
1,101
458
113
UK
cPanel Access Level
Root Administrator
I am not at all sure how you established cPanel as being responsible for the CentOS update patched and pushed by OVH that broke some CentOS 7 OVH installations, nor how the cPanel licence and fee would somehow be called into question because the supplier of the core operating system had introduced a critical issue for their vps services.

Your rant did, however, make me smile and cheered me up on this rather sombre Sunday evening :-D

I think you might like to do some research on what exactly cPanel are directly responsible for supplying you, and what OS-provided third-party modules (including the Operating Systems) you get. 76 Release Notes - Version 76 Documentation - cPanel Documentation

You might also like to look at how YOU can control what is updated during the daily upcp run (eg you can disable Operating System Packages should you have no faith in what CentOS/OVH are supplying) Update Preferences - Version 76 Documentation - cPanel Documentation

But hey, if you want to blame cPanel for everything (you could add world conflicts and hunger if you want) then please feel free o_O
 
  • Like
Reactions: marcuszan

marcuszan

Well-Known Member
Apr 19, 2018
63
5
8
Netherlands
cPanel Access Level
Root Administrator
@Benjamin D
I have to agree with @rpvw
The issues that you have/had were a result of
Code:
yum update
That simple. This is a CentOS command , ran by WHM, as configured and managed by you.

The issues would have also occurred if you had a server with NO WHM / cPanel and you did a manual
Code:
yum update
So the status of the cPanel dashboard as you mention being STABLE, CURRENT, RELEASE etc has nothing to do with the issues in this thread.
 

SergioIS

Registered
Feb 3, 2019
2
0
1
Spain
cPanel Access Level
Root Administrator
Hello,

After updating cPanel to version v76.0.18 (Centos 7) and restarting the server it does not restart.

Apparently the update corrupts the GRUB, it has happened twice.

On the last occasion the server was installed from 0 two weeks ago, and by restarting it from the WHM panel today it has corrupted the GRUB again.

Any suggestion of the reason and / or solution?

Thank you