Lack of CL7 kernel updates

[Trane Francks]

I'm putting this here because we have a CL license via our cPanel sub.

I'm concerned about the lack of CloudLinux 7 kernel updates vs what has come down the pipe from CentOS (our server base prior to installing CL). Investigation shows the last CL7 kernel went beta May 23 and production on June 11 (JST, so give or take a day for your locale). Meanwhile, CentOS 7 has had production releases of the kernel on May 23, May 31, June 16, July 7 and July 18.

Of those CentOS releases, 3 are Important CVEs, 1 is Moderate and one is a bug fix.

Anybody have any thoughts or information? I like CL for MySQL Governor, CageFS and mod_lsapi, but the serious lack of speed for kernel updates is a major concern. All those benefits go out the window in the face of an exploited vulnerability.

 

[cPanelLauren]

Hi @Trane Francks

I can confirm that they have, thus far, released fewer kernel updates this year compared to last year (based off their updates on https://www.cloudlinux.com/cloudlinux-os-blog). I am unaware if there is a specific issue that has prevented them from releasing a new kernel since June 11. I do believe they are currently working on a new kernel release that addresses CVE-2018-3665 but further information into when specifically that will be published is not available. Their reasoning behind not releasing more updates we aren't privy to, though I'm sure they're working diligently.

 

[Trane Francks]

I had an exchange with CloudLinux support about the release schedule, but was not, I think, given an adequate explanation. Basically, the reply was "we don't take downstream kernels from CentOS, we do our own fixes." Updates can be found on the blog (which I take as an RSS feed and track daily) or from the command line via the following command:

rpm -q --changelog kernel | grep -i cve

Running that on my up-to-date CloudLinux 7.5 system shows that the latest CVE is from 2017.

I cannot say that I'm satisfied. There have been a number of Important/Critical CVEs released for the Linux kernel in 2018.

 

[cPanelLauren]

Hi @Trane Francks

I spoke to some folks at Cloudlinux about this specific thread today and I was told that they are currently working on a kernel update with CVE-2018 items included based on 3.10.0-862. They noted that it should be ready in a couple of weeks, they also noted because the kernel is different from the stock CentOS kernel it does take some time to push updates. I do believe there may have been unexpected issues with the specific updates they were attempting, this isn't something that is officially released nor do I have any further information. I just know they do rigorously test kernel updates before they're released and it would explain a bit of a delay in releasing.

Also, in the event that you have a kernelcare subscription, the kernel has been patched against several CVE-2018 items fixed in 3.10.0-862.2.3.el7 you can see them all here: https://patches.kernelcare.com/7cb7fa223653275a4b809b362a079e763dda7354/4/kpatch.html

 

[Trane Francks]

Hi, Lauren.

I don't doubt that CL told you that a kernel would be ready in a couple of weeks, but I find it rather doubtful since there are no CloudLinux 7 kernels currently in beta. Perhaps they meant that a beta release would be available in a couple of weeks. If that's true, it's likely to be a month or more before it reaches production. The greater the number of fixes included in a particular kernel release, the longer the beta tends to be because of the greater likelihood of trouble arising.

 

[cPanelLauren]

Hi @Trane Francks

They did confirm it wouldn't be into beta for a couple of weeks and it's unknown how long it would remain in beta. So your timeline on this is pretty accurate. As I mentioned before KernelCare does apply these patches more quickly though it does require a subscription.

Unfortunately, their kernel release scheduling is out of our control, as is information on why it's taking longer than previous releases. I am sorry we couldn't be of more assistance in this respect.


Thanks!

 

[Infopro]

Latest updates RSS feed for CloudLinux can be found here:
CloudLinux - Main | New template

 

[PeteS]

Unless I am mistaken:

The latest version of CL is currently 3.10.0-962.3.2.lve1.5.24.10 (CloudLinux 7 and CloudLinux 6 Hybrid kernel updated), the latest KC version is 3.10.0-962.3.2.lve1.5.24.9, and since I use R1soft backup, the latest KO for CL pushes me back to 3.10.0-962.3.2.lve1.5.24.4

The latest CentOS 7 is 3.10.0-957, released 10/30/18 (Red Hat Enterprise Linux Release Dates - Red Hat Customer Portal).

The latest CL is built on CentOS 3.10.0-862, released 4/10/18 (Red Hat Enterprise Linux Release Dates - Red Hat Customer Portal).

So CL is lagging about a year behind at this point. This is the main reason I halted plans to move to CL, which I still want to do, but not at the cost of being held back this much.

Above in this thread it seems to say that they planned to be out with an update much sooner than this, but are experiencing issues that have held them back. cPanel has an obvious desire to knit the two together, which I appreciate, but how concerned about this should I be, and what is the latest expectation?

----------

On a related note, is the CL integration with cPanel tight enough to notify us of CL kernel updates when they become available, as it does for non-CL servers? (FYI: KC is not an option for me because KC doesn't wait for the R1soft KO to beinstalled before upgrading the kernel, which potentially would break backups.)

-Pete

 

[cPanelMichael]

Hello Pete,

CloudLinux applies patches to their published kernels so you can't always go by the stock kernel version it's based on. For instance, you'll notice entries like this when checking the patches applied to the CloudLinux kernel RPM with the rpm -q --changelog kernel command:

* Wed Mar 20 2019 Vladislav Fomin - 1.5-24.10
- CLKRN-424: IOPS limit support for mq-deadline
- CLKRN-402: create UB cgroup and related mem, blkio cgroups in EXCL mode
- ext4 fixes
- root dentries need RCU-delayed freeing
- mount: Retest MNT_LOCKED in do_umount
- mount: Prevent MNT_DETACH from disconnecting locked mounts
- mm fix reces and clearcache
- fix mntput/mntput race
- ipv6: fix possible use-after-free in ip6_xmit()
- jbd2: don't mark block as modified if the handle is out of credits
- CLKRN-411: avoid too long stalls in iolimits

That said, I do understand your concern about the update frequency. While we're happy to help troubleshoot any issues arising from the use of cPanel & WHM on CloudLinux, general product feedback (e.g. kernel update frequency) is better addressed by CloudLinux directly. I recommend sharing your feedback on the CloudLinux Forums or through a CloudLinux Support Ticket.

On a related note, is the CL integration with cPanel tight enough to notify us of CL kernel updates when they become available, as it does for non-CL servers? (FYI: KC is not an option for me because KC doesn't wait for the R1soft KO to beinstalled before upgrading the kernel, which potentially would break backups.)

Security Advisor will detect if the kernel the system booted with doesn't match the installed kernel when using CloudLinux, but it won't detect when a new CloudLinux kernel is available for installation through YUM.

Thank you.