JIKOmetrix

Well-Known Member
Apr 3, 2007
105
12
168
Running the downgrade allow the DB services to come back online for me.

Code:
yum downgrade MariaDB-server MariaDB-common MariaDB-shared MariaDB-client MariaDB-compat MariaDB-devel
 

cPAusaf

Linux Technical Analyst III
Staff member
Aug 24, 2016
36
9
83
Houston, TX
cPanel Access Level
Root Administrator
We have lost access to our WHM so I cannot open any tickets, any advice? please ASAP
Are you able to access the server via SSH? You can get the cPanel support ID with this command:
Code:
/usr/local/cpanel/cpanel -S
SSH access will be needed to investigate the issue. If you are unable to access SSH, then we recommend reaching out to your provider to see what issues may be occurring.
 

techAMIGO

Registered
Nov 6, 2019
1
0
1
Kerala
cPanel Access Level
Root Administrator
Hi All,

Code:
yum downgrade MariaDB-server MariaDB-common MariaDB-shared MariaDB-client MariaDB-compat MariaDB-devel
it worked in some of my clients servers...
 
Last edited by a moderator:

cPanelMichael

Technical Support Community Manager
Staff member
Apr 11, 2011
47,911
2,233
363
cPanel Access Level
DataCenter Provider
Twitter
Hello Everyone,

MariaDB has fixed the issue in the following unpublished versions:

10.1.43 [ 23703 ]
10.2.29 [ 23911 ]
10.3.20 [ 23909 ]
10.4.10 [ 23907 ]

We don't have a specific time frame to share on the publication of the updated RPMs, but we'll update this thread as soon as they are published. We strongly recommend avoiding workarounds that take the extra step of disabling automatic updates unless you or your system administration team has a process in-place to monitor the issue and enable automatic updates again once the issue is solved.

Thank you.
 

plague

Well-Known Member
Sep 22, 2006
67
14
158
Brasil
cPanel Access Level
Root Administrator
Twitter
@GOT the problem appears on databases using FULLTEXT KEY indexes. Moving/removing those databases from the data directory enables the server to startup.
You can find this databases looking for file names starting with "FTS*" inside the database directory:

user_database/FTS_000000000005cf3b_CONFIG.ibd
 

cPanelLauren

Forums Analyst II
Staff member
Nov 14, 2017
8,052
652
263
Houston
cPanel Access Level
DataCenter Provider
Update: We've found that MariaDB has removed all affected versions from their repository - At this time the instructions to prevent update using yum-versionlock are no longer necessary.

If you did update and were affected, you would still need to downgrade until the issue is resolved and a new build is pushed out by MariaDB but you will not be upgraded to an affected version.
 

quietFinn

Well-Known Member
Feb 4, 2006
1,072
32
178
Finland
cPanel Access Level
Root Administrator
Update: We've found that MariaDB has removed all affected versions from their repository - At this time the instructions to prevent update using yum-versionlock are no longer necessary.

If you did update and were affected, you would still need to downgrade until the issue is resolved and a new build is pushed out by MariaDB but you will not be upgraded to an affected version.
Had to run yum clean all, then it worked.
 
  • Like
Reactions: cPanelLauren

iero

Member
Jul 2, 2013
10
0
1
cPanel Access Level
Root Administrator
I was so angry in the morning, we have a couple of servers down this morning, we are [Moderator Note: Removed Link To Third-Party URL] and we usually have latest version of whm and litespeed, I had no idea what was wrong until i decided to open a ticket, then i found the issue and we finally resolved downgrading, but it was like 2 hours trying ourselves with our customers calling, not good at all
 
Last edited by a moderator:

kennysamuerto

Member
May 20, 2009
6
0
51
Hello,

In my case, one of my clients was affected by this problem.

The solution in our case, to the little information 24 hours ago when it happened, was to restore a backup of the databases, even though we tried all kinds of alternatives to solve it.

At that point, we decided to remove the databases, and replace them with a backup a few hours earlier.

Therefore, in our case, the bug version is maintained, but everything works correctly after recovering the databases from a backup.

So... What can be expected from the behavior of this server? Should it work well? Should we do downgrade even though everything works correctly?

Thanks
 

webtipster

Active Member
Aug 27, 2016
40
6
8
Atlanta, GA
cPanel Access Level
Website Owner
Praying they have a fix for this soon... Downgrading only solved the problem for some of my servers, might have to roll back more on others. cPanel team please post an update when you know more. Thanks!
 

porkchopzz4

Registered
Nov 6, 2019
1
0
1
Cartersville, Georgia
cPanel Access Level
Root Administrator
Update: We've found that MariaDB has removed all affected versions from their repository - At this time the instructions to prevent update using yum-versionlock are no longer necessary.

If you did update and were affected, you would still need to downgrade until the issue is resolved and a new build is pushed out by MariaDB but you will not be upgraded to an affected version.
Is this a guarantee? I'm scared to turn automatic updates back on and go through this again. I was down 16 hours until support found out what was going on.
 

Leonidas Savvides

Active Member
May 10, 2016
32
3
8
Cyprus
cPanel Access Level
Root Administrator
yum downgrade MariaDB-server MariaDB-common MariaDB-shared MariaDB-client MariaDB-compat MariaDB-devel

Note: The above workaround is intended as a temporary workaround only. If your cPanel & WHM server is configured to automatically update your system packages, the issue will re-occur each time the MariaDB RPMs are upgraded until MariaDB publishes a fix for case MDEV-19073.

from this quote
"
If your cPanel & WHM server is configured to automatically update your system packages, the issue will re-occur each time the MariaDB RPMs are upgraded until MariaDB publishes a fix for case MDEV-19073.
"

how to disable server updates from WHM??? is it here:// ??

  • Home »
  • Server Configuration »
  • Update Preferences
if yes choose MANUAL UPDATES ONLY
From
Daily Updates
  1. Automatic
  2. Manual Updates Only
  3. Never Update

????

Or From OS updates

Operating System Package Updates
  1. Automatic
  2. Manual Updates Only
  3. Never Update

disable - choose second option???
this only??

So FINALLY
Set first

Operating System Package Updates

To
Manual Updates Only
And after ssh run downgrade command...?
  • YES CORRECT
  • No
 
Last edited:

Leonidas Savvides

Active Member
May 10, 2016
32
3
8
Cyprus
cPanel Access Level
Root Administrator
I Set first

Operating System Package Updates

To
Manual Updates Only
And after went ssh run the downgrade command...?

I can Not enter SSH I think forgot where is a public key and in what machine (PC)--- I have over 3 years to login since

$ ssh [email protected]

[email protected]: Permission denied (publickey,gssapi-keyex,gssapi-with-mic).


Is it OK - give the command from WHM/Terminal in the browser as shown below????? What other solutions exist???


whm-ssh-terminal.jpg


I mean this cmd here ^^^ >>>
Code:
yum downgrade MariaDB-server MariaDB-common MariaDB-shared MariaDB-client MariaDB-compat MariaDB-devel
 

plague

Well-Known Member
Sep 22, 2006
67
14
158
Brasil
cPanel Access Level
Root Administrator
Twitter
Hello,

In my case, one of my clients was affected by this problem.

The solution in our case, to the little information 24 hours ago when it happened, was to restore a backup of the databases, even though we tried all kinds of alternatives to solve it.

At that point, we decided to remove the databases, and replace them with a backup a few hours earlier.

Therefore, in our case, the bug version is maintained, but everything works correctly after recovering the databases from a backup.

So... What can be expected from the behavior of this server? Should it work well? Should we do downgrade even though everything works correctly?

Thanks
@kennysamuerto downgrade your MariaDB anyway. After the restore if you restart MariaDB, or run a mysqlcheck on databases that use FULLTEXT indexes, it will probably crash again.
I tried the restore before downgrading the installation, and the server crashed in the first restart, after restoring one of those specific databases.
 

cPanelLauren

Forums Analyst II
Staff member
Nov 14, 2017
8,052
652
263
Houston
cPanel Access Level
DataCenter Provider
from this quote
"
If your cPanel & WHM server is configured to automatically update your system packages, the issue will re-occur each time the MariaDB RPMs are upgraded until MariaDB publishes a fix for case MDEV-19073.
"

how to disable server updates from WHM??? is it here:// ??
Again, I'd like to point out the following:

Update: We've found that MariaDB has removed all affected versions from their repository - At this time the instructions to prevent update using yum-versionlock are no longer necessary.

If you did update and were affected, you would still need to downgrade until the issue is resolved and a new build is pushed out by MariaDB but you will not be upgraded to an affected version.

Furthermore, MariaDB have fixed the affected packages and we're just waiting on them to be pushed - we will update here when that is complete, we're hoping for today.
 
  • Like
  • Love
Reactions: Skyla and popeye