MySQL error with 'updatenow' script

[AngelNo3]

I received an automated error message from cPanel which begins:

An error was detected which prevented updatenow from completing normally.
Please review the enclosed log for further details

Skipping down, it appears the relevant part of the update log is:

[20140222.025516]   Looking for 'mysql' as: /usr/bin/mysql
[20140222.025516]   Looking for 'mysqlcheck' as: /usr/bin/mysqlcheck
[20140222.025516]   FATAL ERROR: Upgrade failed
[20140222.025516]   /usr/bin/mysqladmin: connect to server at 'localhost' failed
[20140222.025516]   error: 'Can't connect to local MySQL server through socket '/var/lib/mysql/mysql.sock' (2)'
[20140222.025516]   Check that mysqld is running and that the socket: '/var/lib/mysql/mysql.sock' exists!
[20140222.025516]   DBI connect('mysql:localhost','root',...) failed: Can't connect to local MySQL server through socket '/var/lib/mysql/mysql.sock' (2) at /usr/local/cpanel/scripts/restartsrv_mysql line 134
[20140222.025516]   Unable to reset root MySQL password. at /usr/local/cpanel/scripts/securemysql line 151.
[20140222.025517]   Starting MySQL. ERROR! The server quit without updating PID file (/var/lib/mysql/elite2129.inmotionhosting.com.pid).
[20140222.025517]   warning: %post(MySQL55-server-5.5.35-1.cp1136.x86_64) scriptlet failed, exit status 1
[20140222.025517]   MySQL55-test-5.5.35-1.cp1136
[20140222.025519]   MySQL55-devel-5.5.35-1.cp1136
[20140222.025520]   MySQL55-shared-5.5.35-1.cp1136
[20140222.025520] E The following possible errors were detected while installing RPMs:
[20140222.025520] E 'Can't connect to local MySQL server through socket '/var/lib/mysql/mysql.sock' (2)'
[20140222.025520]   The Administrator will be notified to review this output when this script completes
[20140222.025520]   Checking for and running RPM::Versions 'post' hooks for any RPMs just installed
[20140222.025520]   All required 'post' hooks have been run
[20140222.025520]       Re-enabling service monitoring.
[20140222.025520]   All files have been updated.
[20140222.025520] E Detected events which require user notification during updatenow. Will send iContact the log

The main website hosted on this server appears to be working normally, which means MySQL is running (because it's a database-driven website).

I have confirmed that /var/lib/mysql/mysql.sock exists.

I'm not sure what I should do next.

 

[vanessa]

The error indicates that MySQL wasn't running at the time the update occurred. You might want to check /var/lib/mysql/<hostname>.err for errors.

Also, are you on a version of MySQL right now that isn't 5.5? If so, the problem may be caused by invalid directives in /etc/my.cnf. The MySQL error log will tell you if this is the case. Once you have it sorted out, you can try the update manually:

1) In /var/cpanel/cpanel.config, change mysql-version= to mysql-version=5.5 (if it isn't already)

2) Run:

/scripts/check_cpanel_rpms --fix

 

[cPanelMichael]

Hello

Yes, please review the MySQL error log as suggested in the previous post. Search for the date/time that coincides when the updatenow script ran to see if there is any particular output associated with the upgrade failure.

Thank you.

 

[AngelNo3]

I am using MySQL version 5.5.35-cli.

The day before the error, Feb 21, there were 1,264 lines in the MySQL error log like this:

InnoDB: A new raw disk partition was initialized or
InnoDB: innodb_force_recovery is on: we do not allow
InnoDB: database modifications by the user. Shut down
InnoDB: mysqld and edit my.cnf so that newraw is replaced
InnoDB: with raw, and innodb_force_... is removed.

I checked my.cnf, and the options 'newraw'/'raw' and 'innodb_force_…' do not appear to be set. (There is an entry '#innodb_force_recovery=2' but as you can see it is commented out.)

Following that, it appears the MySQL server restarted on its own. Then, starting late on Feb 21 and going into the morning of Feb 22, there are 300-some lines of entries like this, although each refers to a different table (they all refer to tables related to the Horde webmail client):

[ERROR] Cannot find or open table horde/horde_alarms from
the internal data dictionary of InnoDB though the .frm file for the
table exists. Maybe you have deleted and recreated InnoDB data
files but have forgotten to delete the corresponding .frm files
of InnoDB tables, or you have moved .frm files to another database?
or, the table contains indexes that this version of the engine
doesn't support.

Then there is a section of 102 lines with entries like this, again each entry refers to a different Horde table:

InnoDB: Error: table `horde`.`horde_groups_members` does not exist in the InnoDB internal
InnoDB: data dictionary though MySQL is trying to drop it.
InnoDB: Have you copied the .frm file of the table to the
InnoDB: MySQL database directory from another database?

After that, there is an InnoDB shutdown message, and MySQL restart message.

At around 6:10 a.m., +/- a few minutes, each day over Feb 22, 23, 24, and 25, there are entries like this (122 lines total—they are all similar but not identical, but I didn't want to dump a huge log sample here):

140222  6:18:19 [Warning] Unsafe statement written to the binary log using statement format since BINLOG_FORMAT = STATEMENT. INSERT IGNORE... SELECT is unsafe because the order in which rows are retrieved by the SELECT determines which (if any) rows are ignored. This order cannot be predicted and may differ on master and the slave. Statement: INSERT IGNORE INTO p2_urls (url, `hash`) (SELECT DISTINCT s.request, MD5(s.request) FROM `p2log` AS s)
140222  6:18:19 [Warning] Unsafe statement written to the binary log using statement format since BINLOG_FORMAT = STATEMENT. Statements writing to a table with an auto-increment column after selecting from another table are unsafe because the order in which rows are retrieved determines what (if any) rows will be written. This order cannot be predicted and may differ on master and the slave. Statement: INSERT IGNORE INTO p2_urls (url, `hash`) (SELECT DISTINCT s.request, MD5(s.request) FROM `p2log` AS s)

I recently (this afternoon) restarted MySQL due to another issue, apparently related (there was a conflict between a setting in my.cnf and the cPanel script that updates the Horde webmail client, which my hosting company helped identify). There have been no errors since restarting.

- - - Updated - - -

I ran the command:

/scripts/check_cpanel_rpms --fix

…but it didn't output anything.

 

[vanessa]

Sooo....is the cPanel update completing now?