SOLVED CPANEL-34745 - MariaDB 10.3.26-1 Breaks PHP < 7.2

[matt1206]

Just wanting to post this here, as I've spent the last 2 hours fault finding an issue on my servers this morning.

cPanel servers, all the latest updates. MariaDB automatically updated to 10.3.26-1 over night as part of the cPanel yum update.

As soon as this happened, any XenForo site running on PHP7.2 or below broke!

[04-Nov-2020 08:24:41 UTC] PHP Fatal error:  Uncaught XF\Db\Exception: MySQL query error [1835]: Malformed communication packet in /home/XXX/public_html/community/src/XF/Db/AbstractStatement.php:212
Stack trace:
#0 /home/XXX/public_html/community/src/XF/Db/Mysqli/Statement.php(196): XF\Db\AbstractStatement->getException('MySQL query err...', 1835, 'HY000')
#1 /home/XXX/public_html/community/src/XF/Db/Mysqli/Statement.php(77): XF\Db\Mysqli\Statement->getException('MySQL query err...', 1835, 'HY000')
#2 /home/XXX/public_html/community/src/XF/Db/AbstractAdapter.php(79): XF\Db\Mysqli\Statement->execute()
#3 /home/XXX/public_html/community/src/XF/Db/AbstractAdapter.php(116): XF\Db\AbstractAdapter->query('\n\t\t\tSELECT data...', Array)
#4 /home/XXX/public_html/community/src/XF/DataRegistry.php(137): XF\Db\AbstractAdapter->fetchPairs('\n\t\t\tSELECT data...')
#5 /home/XXX/public_html/community/src/XF/DataRegistry.php(82): XF\DataRegistry->readFromDb(Array, Array)
#6 /home/XXX/public_html/community/src/XF/DataRegistry.php(225): XF\DataRegistry- in /home/XXX/public_html/community/src/XF/Db/AbstractStatement.php on line 212

This also prevents any user accessing MySQL via cPanel:



Swapping to 7.3 or 7.4 works.

I've had to downgrade MariaDB back to the previous version:

# yum history info 264
Loaded plugins: fastestmirror, universal-hooks
Transaction ID : 264
Begin time     : Wed Nov  4 05:35:28 2020
Begin rpmdb    : 1964:269d3e2a5fc59faed15b162386618da46afe3e28
End time       :            05:36:05 2020 (37 seconds)
End rpmdb      : 1964:7aac4ca2ee1e5c81b04babc4f774126e1254d10d
User           : root <root>
Return-Code    : Success
Command Line   : --assumeyes --color=never --config /etc/yum.conf update --disablerepo=epel
Transaction performed with:
    Installed     rpm-4.11.3-43.el7.x86_64                        @base
    Installed     yum-3.4.3-167.el7.centos.noarch                 @base
    Installed     yum-metadata-parser-1.1.4-10.el7.x86_64         @anaconda
    Installed     yum-plugin-fastestmirror-1.1.31-54.el7_8.noarch @updates
Packages Altered:
    Updated MariaDB-client-10.3.25-1.el7.centos.x86_64 @MariaDB103
    Update                 10.3.26-1.el7.centos.x86_64 @MariaDB103
    Updated MariaDB-common-10.3.25-1.el7.centos.x86_64 @MariaDB103
    Update                 10.3.26-1.el7.centos.x86_64 @MariaDB103
    Updated MariaDB-compat-10.3.25-1.el7.centos.x86_64 @MariaDB103
    Update                 10.3.26-1.el7.centos.x86_64 @MariaDB103
    Updated MariaDB-devel-10.3.25-1.el7.centos.x86_64  @MariaDB103
    Update                10.3.26-1.el7.centos.x86_64  @MariaDB103
    Updated MariaDB-server-10.3.25-1.el7.centos.x86_64 @MariaDB103
    Update                 10.3.26-1.el7.centos.x86_64 @MariaDB103
    Updated MariaDB-shared-10.3.25-1.el7.centos.x86_64 @MariaDB103
    Update                 10.3.26-1.el7.centos.x86_64 @MariaDB103
    Updated galera-25.3.30-1.el7.centos.x86_64         @MariaDB103
    Update         25.3.31-1.el7.centos.x86_64         @MariaDB103
history info

And this fixes the issue. So, I don't know if it's a PHP issue, a MariaDB issue, or something combining everything! I've excluded MariaDB and Galera now from automatic updates on my servers.

 

[hseagle2015]

Just wanted to let others know that we're too experiencing the same error on cPanel's "MySQL databases" page. On older Laravel projects (version 5.8) on PHP 7.2 we observed the exact error message (MySQL query error [1835]: Malformed communication packet).

Upgrading those sites from PHP 7.2 to 7.3 solved the problem.

Since the problem affects both cPanel (Perl) and PHP, my guess is that the culprit is definitely MariaDB 10.3.26 update.

 

[JohnBravo]

Confirmed as an issue on 10.2.x as well. I thought upgrading to 10.3 would fix it, but it didn't.

 

[voip]

Same issue here with servers upgraded to MariaDB 10.2.35 tonight.

It seems MariaDB 10.2 and 10.3 tonight upgrades have the same issue with PHP 7.1 and 7.2 al least.

 

[vacancy]

Mariadb's broken updates are now getting annoying.

This is not the first time.

 

[JohnBravo]

Official response to a ticket: cPanel

 

[Bad-Gamer]

Same temprorary soluton applied here while waiting on final solution. We have websites that should be run on PHP 7.2.

 

[cPRex]

As @JohnBravo posted, we have seen this issue is a result of a recent MariaDB update. The article here explains the best way we currently have to resolve the problem:

Updating MariaDB to v10.2.35 or v10.3.26, causes MySQL Databases interface to show MySQL as offline.

Symptoms Updating MariaDB to v10.2.35 or v10.3.26 may hinder select sites to be inaccessible and the MySQL Database interface within cPanel may populate "The MySQL server is currently offline." ...

support.cpanel.net

 

[foolix]

Same here, MariaDB to 10.3.26 tonight

edit: resolved by downgrading to 10.3.25

 

[T1531]

I solved this on my server by upgrading to PHP 7.4 as mentioned.

 

[RK-BZ]

Same issue with 10.2.35-MariaDB and PHP 7.0

 

[cPRex]

It's also important to note that the cPanel interface issue and the PHP version issue are slightly different technical problems. There is now a new article here specifically for the PHP version issues:

After MariaDB update, PHP website has database connection error, 500 error, or white page: MySQL query error: Malformed communication packet

Symptoms Updating MariaDB to v10.3.26 (and 10.2.35) has been reported to generate errors on sites with PHP versions older than 7.3. This update also caused The MySQL Databases interface to show MyS...

support.cpanel.net

while case CPANEL-34745 is related to the cPanel interface problems.

 

[bitpt]

Same problem here...
Solved downgrade to 10.3.25

 

[Lagarto]

In my case a couple of websites were left without working, but when I changed the version of PHP as recommended here, it was restored.
What does not work in all cPanel's of the websites is the access to MySQL® Databases".
"The MySQL server is currently offline."
"adminbin Cpanel/cpmysql/DBCACHE: exit 11"
What I want to mention is that I created a new cPanel for a new client and here if it works well MySQL® Databases.

 

[Kent Brockman]

Fine. Workaround applied. It's a PITA. It's not cool. But well, it worked.
Now, how do we know when a MariaDB release a fix so we can unlock updates?

 

[Kent Brockman]

When cPanel is aware of these issues, it should send a mailing commenting what the hell is happenning, and provide the workaround. That would save both customers and support staff the hundreds of hours/man to debug, think, investigate, and add to that the time we have to lose apologizing with our end customers.

Or if you don't want to spend money sending thousands of emails, add a notification function to WHM so it can show a warning saying "Hey dude! MariaDB did it again, here are the symptoms and the workaround. Thanks for keep paying. Love. cPanel." (Yep, actually I WANT to see such a warning and I assure you other admins will absolutelly love it too.)

And please, don't send me to open a Feature Request. Be proactive guys, we are paying a lot of money for you to make the best panel and this feature shouldn't be that complicated. Don't stagnate please.

 

[cPanelBrianK]

As an update to the discussion for our internal case CPANEL-34745. This was determined to be caused by the MariaDB SHOW GRANTS change to include role information printing a specific column ID. In some older systems the column order has been found to contain three columns('password_last_changed,password_lifetime,account_locked') added by MySQL 5.7 to be present before the MariaDB columns('is_role,default_role,max_statement_time') and this results in the incorrect data to be returned.

We have updated the article with a method to alter this order and resolve the errors with the cPanel Databases interface reporting The MySQL Server is currently offline.

Updating MariaDB to v10.2.35 or v10.3.26, causes MySQL Databases interface to show MySQL as offline.

Symptoms Updating MariaDB to v10.2.35 or v10.3.26 may hinder select sites to be inaccessible and the MySQL Database interface within cPanel may populate "The MySQL server is currently offline." ...

support.cpanel.net

As the recent updates for MariaDB 10.20.25 and 10.3.26 contain security changes, we advise updating these packages again after applying this fix. Please note this does not address the PHP errors seen for certain PDO connections in deprecated releases of PHP. Please continue following the second article that cPrex mentioned for any updates to the secondary issue.

 

[luki743]

Same here, MariaDB to 10.3.26 tonight

edit: resolved by downgrading to 10.3.25

do you have a tutorial to downgrade?

 

[bitpt]

After downgrade yesterday and solve problem, today made changes in cPanelBrianK post

# yum versionlock clear
# /scripts/upcp

And error come back even with
/scripts/restartsrv_mysql

now MARIADB have last Ver.
Lets try

mysql -e "ALTER TABLE mysql.user MODIFY COLUMN password_last_changed timestamp AFTER max_statement_time, MODIFY COLUMN password_lifetime smallint AFTER password_last_changed, MODIFY COLUMN account_locked enum('N','Y') AFTER password_lifetime"
/scripts/restartsrv_mysql

All issues solved, no error in CPANEL MYSQL DATABASES or CPANEL DB Wizard

 

[Thiago Sabaia]

What is the estimate of leaving an official and safe solution?