Confusing update issue, can't seem to resolve.

[zackw]

This is a long issue but I'll try to be as brief as possible. I have a server at InMotionhosting running CentOS and using WHM/cPanel.

A few weeks ago I started getting a report from "updatenow" that updating failed on the server. I got a really long log file in my Email and the last few lines of the log were:

[20130311.220751] Testing RPM transaction
[20130311.220754] file /usr/lib64/libmysqlclient.so.15.0.0 from install of MySQL50-shared-5.0.96-2.cp1136.x86_64 conflicts with file from package Percona-Server-shared-compat-5.5.20-rel24.1.217.rhel5.x86_64
[20130311.220754] file /usr/lib64/libmysqlclient_r.so.15.0.0 from install of MySQL50-shared-5.0.96-2.cp1136.x86_64 conflicts with file from package Percona-Server-shared-compat-5.5.20-rel24.1.217.rhel5.x86_64
[20130311.220754] W Exit Code: 254
[20130311.220754] ***** FATAL: Test install failed: file /usr/lib64/libmysqlclient.so.15.0.0 from install of MySQL50-shared-5.0.96-2.cp1136.x86_64 conflicts with file from package Percona-Server-shared-compat-5.5.20-rel24.1.217.rhel5.x86_64
[20130311.220754] file /usr/lib64/libmysqlclient_r.so.15.0.0 from install of MySQL50-shared-5.0.96-2.cp1136.x86_64 conflicts with file from package Percona-Server-shared-compat-5.5.20-rel24.1.217.rhel5.x86_64
[20130311.220754] The Administrator will be notified to review this output when this script completes
[20130311.220754] ***** FATAL: Error testing if the RPMs will install: Died at /usr/local/cpanel/scripts/updatenow.static line 12521.
[20130311.220754] see How to Fix RPM Installation Failures in WHM for more information
[20130311.220754] The Administrator will be notified to review this output when this script completes
[20130311.220754] E Detected events which require user notification during updatenow. Will send iContact the log

I received one or two of these messages a day, obviously it wasn't being fixed.
I fixed this particular error by simply uninstalling all the Percona stuff, I had only tried to test their backup scripts, no big deal. I no longer got this message and the updates apparently went through the next time.

But then I started getting this message:

Problems were detected with cPanel-provided files which are RPM controlled.
If you did not make these changes intentionally, you can correct them by running:

> /usr/local/cpanel/scripts/check_cpanel_rpms --fix

The following RPMs are found to be altered from their original install state:
MySQL50-server,5.0.96,2.cp1136,/etc/init.d/mysql

I ran the script mentioned, and it did not resolved anything. I have a support ticket with InMotion and they were not able to resolve so far either. At the time they attempted to fix it, I got a slew of Email reports as follows:

FIRST:

fatal,Downgrades to a previous major release are not permitted.

SECOND:

An error was detected which prevented updatenow from completing normally.
Please review the enclosed log for further details

----------------------------------------------------------------------------------------------------

[20130409.100127] Detected version '11.36.0.21' from version file.
[20130409.100127] Target version set to '11.34.1.13'
[20130409.100127] E Blocker found: Downgrades to a previous major release are not permitted.
[20130409.100127] ***** FATAL: An attempt to up/downgrade to 11.34.1.13 was blocked. Please review blockers.
[20130409.100127] The Administrator will be notified to review this output when this script completes
[20130409.100127] E Detected events which require user notification during updatenow. Will send iContact the log

THIRD:

Problems were detected with cPanel-provided files which are RPM controlled.
If you did not make these changes intentionally, you can correct them by running:

> /usr/local/cpanel/scripts/check_cpanel_rpms --fix

The following RPMs are found to be altered from their original install state:
MySQL50-server,5.0.96,2.cp1136,/etc/init.d/mysql

FOURTH:

Maintenance ended, however it did not exit cleanly (3328). Please check the logs for an indication of what happened

Then I guess they just stopped trying, and I have been receiving the same error report daily about the RPM controlled files.

I also noticed that during this my Spamassassin settings were reset. And also when I logged in to WHM I get this message at the top of the screen:




So can anybody tell me what is going on and what needs updated/changed to stop getting these errors?

Thanks!

 

[cPanelKenneth]

It looks like your provider attempted to "solve" the issue by downgrading your installation to a prior version of cPanel & WHM. As the slew of email informed you, that is not possible.

The message you are receiving from check_cpanel_rpms is meant to inform you when files managed by RPMs are modified post installation. Usually running check_cpanel_rpms --fix will resolve the issue. If it doesn't, please open a support ticket with us at https://tickets.cpanel.net

Thank you.

 

[zackw]

Here is the result of running the check_cpanel_rpms. Obviously something is amiss here. Before I open a ticket, maybe the problem will jump out at someone:

**I'm logged in as root user** # /usr/local/cpanel/scripts/check_cpanel_rpms --fix
[20130416.104950]
[20130416.104950]   Problems were detected with cPanel-provided files which are RPM controlled.
[20130416.104950]   If you did not make these changes intentionally, you can correct them by running:
[20130416.104950]
[20130416.104950]   > /usr/local/cpanel/scripts/check_cpanel_rpms --fix
[20130416.104950]
[20130416.104950]   The following files were found to be altered from their original RPM:
[20130416.104950]   MySQL50-server,5.0.96,2.cp1136
[20130416.104950]
[20130416.104950]   Removing 1 broken rpms: MySQL50-server
[20130416.104953]   Downloading [url]http://httpupdate.cpanel.net/RPM/11.36/centos/5/x86_64/rpm.md5[/url]
[20130416.104954]   Downloading [url]http://httpupdate.cpanel.net/RPM/11.36/centos/5/x86_64/MySQL50-server-5.0.96-2.cp1136.x86_64.rpm[/url]
[20130416.104955]   Hooks system enabled
[20130416.104955]   Checking for and running RPM::Versions 'pre' hooks for any RPMs about to be installed warn [check_cpanel_rpms] program "/scripts/premysqlup" is not executable
[20130416.104955]   All required 'pre' hooks have been run
[20130416.104956]   No RPMS need to be uninstalled
[20130416.104956]   Installing new rpms: MySQL50-server-5.0.96-2.cp1136.x86_64.rpm
[20130416.104956]   Preparing packages for installation...
[20130416.104959]   Giving mysqld a couple of seconds to exit nicely
[20130416.105004]   MySQL50-server-5.0.96-2.cp1136
[20130416.105007]   PLEASE REMEMBER TO SET A PASSWORD FOR THE MySQL root USER !
[20130416.105007]   To do so, start the server, then issue the following commands:
[20130416.105007]   /usr/bin/mysqladmin -u root password 'new-password'
[20130416.105007]   /usr/bin/mysqladmin -u root -h elite799.inmotionhosting.com password 'new-password'
[20130416.105007]
[20130416.105007]   Alternatively you can run:
[20130416.105007]   /usr/bin/mysql_secure_installation
[20130416.105007]
[20130416.105007]   which will also give you the option of removing the test
[20130416.105007]   databases and anonymous user created by default.  This is
[20130416.105007]   strongly recommended for production servers.
[20130416.105007]
[20130416.105007]   See the manual for more instructions.
[20130416.105007]
[20130416.105007]   Please report any problems with the /usr/bin/mysqlbug script!
[20130416.105007]
[20130416.105007]   The latest information about MySQL is available on the web at
[20130416.105007]   [url=http://www.mysql.com]MySQL :: The world's most popular open source database[/url]
[20130416.105007]   Support MySQL by buying support/licenses at [url]http://shop.mysql.com[/url]
[20130416.105008]   Starting MySQL.[  OK  ]
[20130416.105010]   Thank you for installing the MySQL Community Server! For Production
[20130416.105010]   systems, we recommend MySQL Enterprise, which contains enterprise-ready
[20130416.105010]   software, intelligent advisory services, and full production support with
[20130416.105010]   scheduled service packs and more.  Visit [url=http://www.mysql.com/enterprise]MySQL :: MySQL Enterprise Edition[/url] for more
[20130416.105010]   information.
[20130416.105016]   This script updates all the mysql privilege tables to be usable bythe current version of MySQLYou can safely ignore all 'Duplicate column' and 'Unknown column' errorsbecause these just mean that your tables are already up to date.This script is safe to run even if your tables are already up to date!doneStarting MySQL[  OK  ]
[20130416.105016]   Prelinking shared libraries and binaries: /usr/sbin/prelink -av -mR 080p LED Monitor .Days $79.99 100W Subwoofer .D/usr/sbin/prelink: /opt/tidy/lib/libtidy-0.99.so.0.0.0 is not present in any config file directories, nor was specified on command line
/usr/sbin/prelink: /opt/mhash/lib/libmhash.so.2.0.1 is not present in any config file directories, nor was specified on command line
/usr/sbin/prelink: /opt/libmcrypt/lib/libmcrypt.so.4.4.8 is not present in any config file directories, nor was specified on command line
/usr/sbin/prelink: /opt/xml2/lib/libxml2.so.2.7.6 is not present in any config file directories, nor was specified on command line
/usr/sbin/prelink: /usr/local/cpanel/3rdparty/lib64/libsrs_alt.so.1.0.0 is not present in any config file directories, nor was specified on command line
/usr/sbin/prelink: /opt/curlssl/lib/libcurl.so.4.2.0 is not present in any config file directories, nor was specified on command line
/usr/sbin/prelink: /usr/local/cpanel/3rdparty/lib64/libpcre.so.1.0.1 is not present in any config file directories, nor was specified on command line
/usr/sbin/prelink: /opt/xslt/lib/libexslt.so.0.8.15 is not present in any config file directories, nor was specified on command line
/usr/sbin/prelink: /opt/pcre/lib/libpcre.so.0.0.1 is not present in any config file directories, nor was specified on command line
/usr/sbin/prelink: /usr/local/cpanel/3rdparty/lib64/libspf2.so.2.1.0 is not present in any config file directories, nor was specified on command line
/usr/sbin/prelink: /opt/xslt/lib/libxslt.so.1.1.26 is not present in any config file directories, nor was specified on command line
/usr/sbin/prelink: /usr/local/cpanel/3rdparty/perl/514/lib64/perl5/5.14.3/x86_64-linux-64int/CORE/libperl.so is not present in any config file directories, nor was specified on command line
/usr/sbin/prelink: Could not prelink /usr/sbin/exim because its dependency /usr/local/cpanel/3rdparty/lib64/libspf2.so.2 could not be prelinked
/usr/sbin/prelink: Could not prelink /usr/bin/php because its dependency /opt/xslt/lib/libexslt.so.0 could not be prelinked
/usr/sbin/prelink: Could not prelink /usr/bin/pk11install because its dependency /usr/lib64/libsoftokn3.so could not be prelinked
[20130416.105047]   Checking for and running RPM::Versions 'post' hooks for any RPMs just installed warn [check_cpanel_rpms] program "/scripts/postmysqlinstall" is not executable
[20130416.105047]   All required 'post' hooks have been run

Thanks!

 

[Infopro]

Here is the result of running the check_cpanel_rpms. Obviously something is amiss here. Before I open a ticket, maybe the problem will jump out at someone:

...

Thanks!

This post from another thread may be useful to you:
Update Causing Problems. - cPanel Forums

 

[zackw]

OK, so are you saying I should empty the contents of /etc/prelink.conf?

Note the line:

[20130416.104955] Checking for and running RPM::Versions 'pre' hooks for any RPMs about to be installed warn [check_cpanel_rpms] program "/scripts/premysqlup" is not executable

I looked and the premysqlup doesn't exist at all.

Also at the end after the prelink lines:

[20130416.105047] Checking for and running RPM::Versions 'post' hooks for any RPMs just installed warn [check_cpanel_rpms] program "/scripts/postmysqlinstall" is not executable

The "postmysqlinstall" is simply not there either.

I tried running the check_cpanel_rpms script again with the --long-list option. It says the exact same thing every time and is not fixing anything.

MySQL50-server,5.0.96,2.cp1136,S.5....T,/etc/init.d/mysql

Note it is saying the RPM has a different file size, MD5 hash, and mTime difference.

Honestly I'm just not sure what's going on. My suspicion is that the system update that took place (refer to my OP at the top), it seems the system was auto-updating MySQL. It originally conflicted with Percona until I removed Percona and I didn't get that error anymore. Is it possible that the system auto-updated MySQL which is why it got out of sync with cPanel?
cPanel does not let me go to previous versions of course, now that it is updated, nor can it seem to fix the RPM and install the correct MySQL. Also remember the error on WHM saying a previous update failed. I'm not really sure if MySQL really updated or not by EITHER system!

Could the reason the RPM fix isn't working because the two scripts mentioned above are not there? It reports they are not executable, but in fact they don't exist. Is that the issue? Is it the prelink errors that are the problem?

Any ideas fellas?

 

[zackw]

Also note that in WHM I am set to auto-update LTS and my current version says I am WHM 11.36.0 (build 21).
Would it help to change my update settings to "stable" and get the build 22? Or change to "release" and get 11.36.1.5?

Thanks

 

[Infopro]

You might want to open a ticket with your questions and get proper answers from cPanel Technical Support.

 

[zackw]

For future reference, this was resolved. Ultimately the problem was a loop of errors. There is a "post-update" script that runs every time MySQL is updated, but that changed an RPM controlled file then caused the error again.

By renaming the /scripts/postmysqlup script so it wouldn't run, the fix_cpanel_rpms --fix command was able to finish.

Very strange series of events, but I haven't got the error reports in Email since doing this.