SOLVED Kernel conflicts on OVH servers due to missing exclude line in /etc/yum.repos.d/OVH-kernel.repo

[JIKOmetrix]

Hello,

I have 3 servers out of 8 throwing a similar error when cpanel tries to update to current version. cPanel throws the following notice to the admin:

2019-05-31 07:33:01 -0500] E [/usr/local/cpanel/scripts/rpmup] The “/usr/local/cpanel/scripts/rpmup” command (process 21012) reported error number 1 when it ended.

Then I did some research and saw that I some threads asked to run '/usr/local/cpanel/scripts/rpmup' manually to see the output. Below is a sample of that output.

file /usr/include/scsi/..install.cmd conflicts between attempted installs of kernel-4.19-ovh-mod-std-ipv6-64-headers-4.19_ovh_mod_std_ipv6_64-732421.x86_64 and kernel-4.19-ovh-xxxx-std-ipv6-64-headers-4.19_ovh_xxxx_std_ipv6_64-732421.x86_64

file /usr/include/scsi/fc/..install.cmd conflicts between attempted installs of kernel-4.19-ovh-mod-std-ipv6-64-headers-4.19_ovh_mod_std_ipv6_64-732421.x86_64 and kernel-4.19-ovh-xxxx-std-ipv6-64-headers-4.19_ovh_xxxx_std_ipv6_64-732421.x86_64

file /usr/include/sound/..install.cmd conflicts between attempted installs of kernel-4.19-ovh-mod-std-ipv6-64-headers-4.19_ovh_mod_std_ipv6_64-732421.x86_64 and kernel-4.19-ovh-xxxx-std-ipv6-64-headers-4.19_ovh_xxxx_std_ipv6_64-732421.x86_64

file /usr/include/video/..install.cmd conflicts between attempted installs of kernel-4.19-ovh-mod-std-ipv6-64-headers-4.19_ovh_mod_std_ipv6_64-732421.x86_64 and kernel-4.19-ovh-xxxx-std-ipv6-64-headers-4.19_ovh_xxxx_std_ipv6_64-732421.x86_64

file /usr/include/xen/..install.cmd conflicts between attempted installs of kernel-4.19-ovh-mod-std-ipv6-64-headers-4.19_ovh_mod_std_ipv6_64-732421.x86_64 and kernel-4.19-ovh-xxxx-std-ipv6-64-headers-4.19_ovh_xxxx_std_ipv6_64-732421.x86_64

I'm not sure where to start. I have not tried to reboot the servers and run update after reboot. I may try that this weekend.

When I saw the kernel conflicts I checked the servers kernels and found the following.

host86 is booting from internal hard disks to kernel 4.19.18-xxxx-std-ipv6-64

host88 is booting from netboot to kernel 4.14.63-xxxx-std-ipv6-64

host90 is booting from netboot to kernel 4.14.63-xxxx-std-ipv6-64

Any idea on where to start on this?

Thanks,
Mike

 

[jvgn]

I have the same problem.

Can you help me?

Regards.

 

[JIKOmetrix]

Hello,

I'm still trying to fix this. It seems cPanel does not provide Kernel support.

I'm waiting for the weekend to reboot the server with a new kernel installed to see if that fixes the issue.

- Mike

 

[dalem]

that because you are using a non standard kernel "OVH's"

 

[jvgn]

Hello,

I'm still trying to fix this. It seems cPanel does not provide Kernel support.

I'm waiting for the weekend to reboot the server with a new kernel installed to see if that fixes the issue.

- Mike

With the netboot - Network ---> error conflicts
And the netbot - Hard disk ---> error conflicts

that because you are using a non standard kernel "OVH's"

We assume that an OVH kernel is optimized for its machines.

 

[JIKOmetrix]

Hello,

I have 3 out of 8 server with the above kernel issue.

The are all running OVH kernels, since OVH optimizes them with correct drivers and what not run the hardware and system boards without having to load 3 party drivers manually.

It two of the three netboot and have an error, one is hard disk boot with exact same error.

It is not clear why the above example is throwing an error for kernel-4.19-ovh-xxxx-std-ipv6-64 when it is running a netboot kernel of 4.14.63-xxxx-std-ipv6-64.

My other 5 servers that hard drive boot to OVH kernels update cPanel without error.

I'm just looking for some direction on where to start troubleshooting.

- Mike

 

[jvgn]

To have 4 affected servers.

The 4 servers recently installed. I also dont understand the error. It is not understandable.

Surely there are more affected.

 

[JIKOmetrix]

It is only affecting 3 of my 8 servers.

To have 4 affected servers.

The 4 servers recently installed. I also dont understand the error. It is not understandable.

Surely there are more affected.

 

[cPanelMichael]

Hello Everyone,

This looks to be isolated to dedicated servers using outdated OVH kernels. OVH documents how to perform kernel updates on dedicated servers on the link below:

Updating the kernel on a dedicated server

Can you verify if updating the kernel to the latest one provided by OVH using the instructions above is helpful in solving this issue? If not, please contact OVH directly to report the issue and let us know of any specific solutions they share.

Thank you.

 

[JIKOmetrix]

Hello,

Oh, so we just need to manually update the kernel on these servers?

So the cPanel update won't work with kernels lower than 4.19.18-xxxx-std-ipv6-64??

What is the correct kernel to update to, 4.19.45-xxxx-std-ipv6-64?

Thanks,
Mike

Hello Everyone,

This looks to be isolated to dedicated servers using outdated OVH kernels. OVH documents how to perform kernel updates on dedicated servers on the link below:

Updating the kernel on a dedicated server

Can you verify if updating the kernel to the latest one provided by OVH using the instructions above is helpful in solving this issue? If not, please contact OVH directly to report the issue and let us know of any specific solutions they share.

Thank you.

 

[JIKOmetrix]

Hello,

I looked at the servers that are updating without error and they are all running kernel 4.19.18-xxxx-std-ipv6-64.

Our server host86 that is booting from internal hard disks is also running 4.19.18-xxxx-std-ipv6-64.

So I'm sure that is a specific kernel issue, though I can try to update to 4.19.45-xxxx-std-ipv6-64 and see if that makes a difference.

Kernel 4.14.63-xxxx-std-ipv6-64 is from August 2018, is that really considered out of date or old?

- Mike

 

[cPanelMichael]

So the cPanel update won't work with kernels lower than 4.19.18-xxxx-std-ipv6-64??

Hello Mike,

It's not the cPanel update itself, but rather your system's kernel RPMs failing to update upon execution of the command below (it runs as part of the cPanel update process):

/usr/local/cpanel/scripts/rpmup

Thank you.

 

[JIKOmetrix]

Hello,

host86 has "Enable Linux kernel update during nightly maintenance." set to off.

host88 has "Enable Linux kernel update during nightly maintenance." set to off.

host90 has "Enable Linux kernel update during nightly maintenance." set to off.

The above are the three servers throwing the original error during
/usr/local/cpanel/scripts/rpmup execution.

- Mike

 

[cPanelMichael]

Hello Mike,

Do you notice any errors when running the "yum update" command on an affected system?

Thank you.

 

[jvgn]

Hello.

I changed the option on my server with kernel version 4.14 (Enable Linux kernel update during nightly maintenance." to ON) and executed "/usr/local/cpanel/scripts/upcp" but the problem persist.

Servers with lower versions(4.9) of kernel work correctly.

I need update kernel manually to 4.19 ? Or open one ticket to OVH ?

Regards.

 

[cPanelMichael]

I need update kernel manually to 4.19 ? Or open one ticket to OVH ?

Hello @jvgn,

Yes, I recommend reporting this to OVH and seeking their advice on how to proceed. Let us know the outcome of the OVH ticket, and feel free to open a support ticket directly with us if you'd like us to rule out any issues with cPanel & WHM.

Thank you.

 

[JIKOmetrix]

Hello,

I'll run yum update and see what comes of it.

- Mike

Hello Mike,

Do you notice any errors when running the "yum update" command on an affected system?

Thank you.

 

[JIKOmetrix]

Hello,

I have attached a TXT with output from "yum update"

Here is summary info:

    ---> Package kernel-4.19-ovh-mod-std-ipv6-64-headers.x86_64 0:4.19_ovh_mod_std_ipv6_64-732421 will be obsoleting

    ---> Package kernel-4.19-ovh-xxxx-std-ipv6-64-headers.x86_64 0:4.19_ovh_xxxx_std_ipv6_64-732421 will be obsoleting

    ---> Package kernel-4.19-ovh-xxxx-std-ipv6-64-hz1000-headers.x86_64 0:4.19_ovh_xxxx_std_ipv6_64_hz1000-732421 will be obsoleting

    ---> Package kernel-headers.x86_64 0:3.10.0-957.12.2.el7 will be obsoleted

    ---> Package kmod-libs.x86_64 0:20-21.el7 will be updated

    ---> Package kmod-libs.x86_64 0:20-23.el7 will be an update

    ---> Package ovhkernel_4.9_xxxx_std_ipv6.x86_64 0:4.9.178_xxxx_std_ipv6_64-730913 will be updated

    ---> Package ovhkernel_4.9_xxxx_std_ipv6.x86_64 0:4.9.180_xxxx_std_ipv6_64-746430 will be an update

    --> Finished Dependency Resolution



    Dependencies Resolved


    ========================================================================================================================================================================================================

     Package                                                               Arch                         Version                                                       Repository                       Size

    ========================================================================================================================================================================================================

    Installing:

     kernel-4.19-ovh-mod-std-ipv6-64-headers                               x86_64                       4.19_ovh_mod_std_ipv6_64-732421                               ovhkernel                       1.3 M

        replacing  kernel-headers.x86_64 3.10.0-957.12.2.el7

     kernel-4.19-ovh-xxxx-std-ipv6-64-headers                              x86_64                       4.19_ovh_xxxx_std_ipv6_64-732421                              ovhkernel                       1.3 M

        replacing  kernel-headers.x86_64 3.10.0-957.12.2.el7

     kernel-4.19-ovh-xxxx-std-ipv6-64-hz1000-headers                       x86_64                       4.19_ovh_xxxx_std_ipv6_64_hz1000-732421                       ovhkernel                       1.3 M

        replacing  kernel-headers.x86_64 3.10.0-957.12.2.el7



Then a munch of these:

    Total size: 74 M

    Total download size: 50 k

    Is this ok [y/d/N]: y

    Downloading packages:

    No Presto metadata available for base

    kmod-libs-20-23.el7.x86_64.rpm                                                                                                                                                   |  50 kB  00:00:00   

    Running transaction check

    Running transaction test



    Transaction check error:

      file /usr/include/asm-generic/..install.cmd conflicts between attempted installs of kernel-4.19-ovh-xxxx-std-ipv6-64-headers-4.19_ovh_xxxx_std_ipv6_64-732421.x86_64 and kernel-4.19-ovh-xxxx-std-ipv6-64-hz1000-headers-4.19_ovh_xxxx_std_ipv6_64_hz1000-732421.x86_64

      file /usr/include/asm/..install.cmd conflicts between attempted installs of kernel-4.19-ovh-xxxx-std-ipv6-64-headers-4.19_ovh_xxxx_std_ipv6_64-732421.x86_64 and kernel-4.19-ovh-xxxx-std-ipv6-64-hz1000-headers-4.19_ovh_xxxx_std_ipv6_64_hz1000-732421.x86_64

      file /usr/include/drm/..install.cmd conflicts between attempted installs of kernel-4.19-ovh-xxxx-std-ipv6-64-headers-4.19_ovh_xxxx_std_ipv6_64-732421.x86_64 and kernel-4.19-ovh-xxxx-std-ipv6-64-hz1000-headers-4.19_ovh_xxxx_std_ipv6_64_hz1000-732421.x86_64

      file /usr/include/linux/..install.cmd conflicts between attempted installs of kernel-4.19-ovh-xxxx-std-ipv6-64-headers-4.19_ovh_xxxx_std_ipv6_64-732421.x86_64 and kernel-4.19-ovh-xxxx-std-ipv6-64-hz1000-headers-4.19_ovh_xxxx_std_ipv6_64_hz1000-732421.x86_64

 

[jvgn]

I opened a ticket to OVH with this problem. I also made reference to this discussion.

This is the answer:

OVH:"Regarding your query about this case, unfortunately there is no possibility to launch the installation of the operating system from the client area with a custom kernel. For this type of operations, you should use the rescue mode and install the kernel that you consider appropriate from this mode on the operating system installed on the server disks."

I want to cry.

 

[cPanelMichael]

Hello,

Kernel updates are important, however you could try temporarily excluding the kernel and ovhkernel RPMs from the YUM update process until you've received a solution approved by OVH.

To do this, first open the /etc/yum.conf file with your preferred text editor via SSH. EX:

vi /etc/yum.conf

In this file, you'll see an "exclude=" line like this:

exclude=courier* dovecot* exim* filesystem httpd* mod_ssl* mydns* nsd* p0f php* proftpd* pure-ftpd* spamassassin*

Add kernel* and ovhkernel* to this line like this:

exclude=kernel* ovhkernel* courier* dovecot* exim* filesystem httpd* mod_ssl* mydns* nsd* p0f php* proftpd* pure-ftpd* spamassassin*

Then, run the following commands again to confirm the temporary workaround solves the issue:

yum clean all
yum update

If the "yum update" command succeeds this time, attempt another cPanel update (/scripts/upcp) and let us know of any additional errors.

Thank you.