Major Issues Lately with MariaDB Upgrades and Updates via WHM

celiac101

Well-Known Member
Dec 19, 2012
139
5
68
cPanel Access Level
Website Owner
After going through 2 days of hell in the last few months, I am sharing my thoughts here:

1) I no longer trust anything to do with WHM's ability to run any type of upgrade or update on MariaDB databases. Within the last few months I've had 2 major crashed database incidents due to:

  • a) a more major upgrade from MariaDB 10.5 to 10.6 using WHM update mysql app;
  • b) a "minor" update to MariaDB 10.6 using the standard yum update command;
I am not sure what is going on since WHM was purchased by a larger company, but to me it's all bad and after more than 10 years I'm now considering switching to an alternative. I've run WHM DB upgrade and updates for over 10 years without such issues, so having such major problems lately breaks my trust in WHM.



2) Why are there not more built in options to automatically repair DBs that have issues, or in an emergency dump a DB and re-create the DB within the WHM interface? Certainly you could offer far more tools to fix database issues than what you currently offer, which is nearly nothing. How hard would it be to offer an emergency tool that would try to perform a mysql dump and extraction in to a new DB with a "-copy" name added to the DB name. After it ran, if successful, it could display a simple message saying:

  • "success-a new DB was created that was tested to have no issues with the name of db-name-copy. To test this with your app be sure to change the DB name in the config file of your site,"
Adding more repair options might at least mitigate the many problems now created when using WHM to upgrade or update MariaDB.
 
Last edited by a moderator:

cPRex

Jurassic Moderator
Staff member
Oct 19, 2014
12,457
1,965
363
cPanel Access Level
Root Administrator
Hey there! If you're referring to the issues yesterday, that was a problem with MariaDB breaking their own repositories, so there wasn't anything we could do on our end for that. MariaDB said that was resolved as of 12:30 yesterday, although some people reported odd behavior for another few hours while the mirrors were getting back in sync.

If you have ideas about adding repair features, please submit a feature request and I can bring that up during our weekly developer meeting. We're always working on and entertaining new features so I'd be happy to discuss that with them.
 

celiac101

Well-Known Member
Dec 19, 2012
139
5
68
cPanel Access Level
Website Owner
No, I'm not referring to that. The auto update that ran did alert me to an issue, but when I ran yup update ~6 hours later the update ran fine, found the repositories, and then broke my databases--all of them on different web sites.
 

cPRex

Jurassic Moderator
Staff member
Oct 19, 2014
12,457
1,965
363
cPanel Access Level
Root Administrator
Any chance you could submit a ticket so we can see that? I'm not saying it's not possible, but a Yum transaction wouldn't touch any data in the database itself, so I'm not sure how the two would be related without us doing some investigation.
 

celiac101

Well-Known Member
Dec 19, 2012
139
5
68
cPanel Access Level
Website Owner
I did, but fixed this myself in the several hours I was waiting for anyone there to respond...oh...did I mention my issue with how long it is now to get any help from you guys?
 

celiac101

Well-Known Member
Dec 19, 2012
139
5
68
cPanel Access Level
Website Owner
and yes, yum update, which updated MariaDB, did crash every database on my server to the point where the only way to repair them was a full export import routine, which took a ton of time.
 

cPRex

Jurassic Moderator
Staff member
Oct 19, 2014
12,457
1,965
363
cPanel Access Level
Root Administrator
How long did you wait for a ticket response? Is your license direct through us so you receive priority support? Could you let me know a ticket number so I could review that? I'm happy to listen to feedback, but I do need some data to act on in order to help.

As far as the Yum issues, I suppose anything is possible but I'm just not sure how a Yum transaction would corrupt database data. If that did happen between Yum and MariaDB, at least cPanel tools wouldn't have been involved.
 

celiac101

Well-Known Member
Dec 19, 2012
139
5
68
cPanel Access Level
Website Owner
#94475125 is the ticket number, and I believe I'm on the misnomer "Priority" support ;)

Apparently the paid upgraded "Premium" support is no longer an option, as it now throws an error when I try to do that.

I can't tell the exact times on the ticket, but it looks like I submitted it at ~4:30PM Pacific time, and the first response on the ticket was ~4 hours later. It was enough time for a non-expert like me to research how, and to rebuild 5 mysql database, 3 of them being over 5 gigs. I'm only glad I was able successfully dump all the data, so that nothing was lost.

But really, I'd love to hear a comment about #2 above. Why the heck, given that essentially all of your customers run mysql databases, doesn't WHM offer more automated recovery options (especially if your software updates/upgrades might be breaking them more often going forward)?

BTW, I posted a reply on that ticket 6 hours ago and nobody has replied. I guess it's a good think my sites are not still down and I was able to recover the DBs myself!
 

cPRex

Jurassic Moderator
Staff member
Oct 19, 2014
12,457
1,965
363
cPanel Access Level
Root Administrator
Thanks for the details. I checked the ticket and confirmed that your license isn't purchased through us, so the ticket is part of the complimentary queue, which has no official timeframe for a reply.

You are correct that there are issues with the support bump in the ticket system, and our developers are working to address that.

I see that Justin created case CPANEL-41340 with our development team to have them explore the possibility of making some type of automatic database copy during the upgrade process on the server, just in case something goes wrong. Traditionally we try and stay hands-off with data, but we'll let the team explore this and see what they come up with.

We also weren't able to access the server in the ticket, so for more details on what may have caused the MySQL problems we'd have to get access to the affected system.
 

celiac101

Well-Known Member
Dec 19, 2012
139
5
68
cPanel Access Level
Website Owner
Thanks for looking into this...regardless of the license type I'm paying your regularly for my license, which should include reasonable support times, but again, your escalation system has been broken for months now, and I have no issues paying $30 or so in an emergency, but I could not even do that the last two emergencies I had (both caused by WHM). I hope you can fix it soon, but I would fire the person who's been working on it. I could have gotten that fixed for you within a few days...
 

cPRex

Jurassic Moderator
Staff member
Oct 19, 2014
12,457
1,965
363
cPanel Access Level
Root Administrator
I've got someone looking into this today, so hopefully that is resolved soon.

Since the license isn't purchased through cPanel, your first contact for support would normally be the hosting provider where the license was purchased.
 

HenrikSCH

Member
Oct 13, 2020
15
2
3
Santo Domingo
cPanel Access Level
Root Administrator
Hi my server gives me an update error .. this is on my MariaDB106.repo


[MariaDB106]

name = MariaDB106

baseurl = http://yum.mariadb.org/10.6/centos8-amd64

gpgkey=https://yum.mariadb.org/RPM-GPG-KEY-MariaDB

gpgcheck=1



But that folder does not exist on the MariaDB server?



I am running Almalinux 8.6 with latest WHM/cPanel and MariaDB 10.6.9


[[email protected] ~]# yum -y update

MariaDB106 862 B/s | 10 B 00:00

Error: Failed to download metadata for repo 'MariaDB106': Cannot download repomd.xml: Cannot download repodata/repomd.xml: All mirrors were tried

Is there a way to fix this? Because i can not even update EA now.
 

ServerHealers

Active Member
Sep 21, 2015
42
18
58
India
cPanel Access Level
Root Administrator
@HenrikSCH
The correct baseurl for AlmaLinux8 would be Index of /yum/10.6/rhel8-amd64/

The repository /etc/yum.repos.d/MariaDB106.repo file should look like:


[MariaDB106]
name = MariaDB106
baseurl = https://yum.mariadb.org/10.6/rhel8-amd64
gpgkey=https://yum.mariadb.org/RPM-GPG-KEY-MariaDB
gpgcheck=1



Replace baseurl correctly in your repository file, and try again after clearing yum cache
 
Last edited: