Problem with /usr/local/cpanel/scripts/update_db_cache

ITGabs

Well-Known Member
Jul 30, 2013
81
0
6
cPanel Access Level
Root Administrator
Hi,

I am getting the 5th or 6th email with the following error from cron daemon

Code:
30 */4 * * * /usr/bin/test -x /usr/local/cpanel/scripts/update_db_cache && /usr/local/cpanel/scripts/update_db_cache

Error while connecting to MySQL: Cpanel::DBI::Mysql connect('','root',...) failed: Can't connect to local MySQL server through socket '/tmp/mysql.sock' (2) at /usr/local/cpanel/Cpanel/Mysql.pm line 140
This is a cpanel bug??
I am just a bit intrigued about security implications with root and /tmp/mysql.sock ...the activity of Mysql seems normal

version: CENTOS 6.5 x86_64 vmware
WHM 11.44.0 (build 16)
 

cPanelMichael

Administrator
Staff member
Apr 11, 2011
47,909
2,229
463
Hello :)

Is MySQL running well on your system? If so, try updating the MySQL root password via:

"WHM Home » SQL Services » MySQL Root Password"

Then, check to see if the issue persists by running that command manually:

Code:
/usr/local/cpanel/scripts/update_db_cache
Thank you.
 

ITGabs

Well-Known Member
Jul 30, 2013
81
0
6
cPanel Access Level
Root Administrator
Hi Michael,

MySql is working without any issue. MYSQL version
mysql Ver 14.14 Distrib 5.5.37, for Linux (x86_64) using readline 5.1

I created a new password for root, I tried force an update, reboot the server too.

mysql --host= --user=root --password='rootpass' mysql
connect without problems and any other user connect without problems, cpanel whm phpmyadmin are ok, the problems are just with the /scripts/* that connect with mysql wwwacct killacct etc

I tried to read the line 140 with vim but update_db_cache is a binary, Probably the password it's ok but need the 'localhost' in Cpanel::DBI::Mysql connect('HERE','root',...)

More info:

I did chmod 770 temporally and comment my settings in all '[.]my.cnf' in the server.
I checked the logs and the other files, everything looks ok except that /tmp/mysql.sock not exist in the logs of mysql the sock is on /var/lib/mysql/mysql.sock...

I tested changing the socket to /tmp/mysql.sock in /root/.my.cnf and in /etc/my.cnf and the error change with
socket=/tmp/mysql.sock

#/usr/local/cpanel/scripts/update_db_cache
/usr/bin/mysqladmin: connect to server at 'localhost' failed
error: 'Can't connect to local MySQL server through socket '/var/lib/mysql/mysql.sock' (2)'
Check that mysqld is running and that the socket: '/var/lib/mysql/mysql.sock' exists!
warn [update_db_cache] local mysql server is not running
I checked in the logs and the problem start here with [auto-adjust-mysql-limits] this was exactly after an update
Another thing are the errors of cpses_ the cpses_tool give me time out

[2014-06-19 01:00:02 -0700] info [cpbackup] Started at Thu Jun 19 01:00:02 2014
[2014-06-19 01:53:44 -0700] warn [updatenow] program "/scripts/predovecotup" is not executable at /usr/local/cpanel/Cpanel/Hooks.pm line 213
Cpanel::Hooks::exec_script('main', HASH(0x4323820), HASH(0x42c4e90), undef) called at /usr/local/cpanel/Cpanel/Hooks.pm line 126
Cpanel::Hooks::exec_hook('main', HASH(0x4323820), HASH(0x42c4e90), undef) called at /usr/local/cpanel/Cpanel/Hooks.pm line 60
eval {...} called at /usr/local/cpanel/Cpanel/Hooks.pm line 55
Cpanel::Hooks::hook(HASH(0x42c4e90)) called at /usr/local/cpanel/scripts/updatenow.static-cpanelsync line 28448
eval {...} called at /usr/local/cpanel/scripts/updatenow.static-cpanelsync line 28439
Cpanel::RPM::Versions::File::commit_changes(Cpanel::RPM::Versions::File=HASH(0x2f7ace0)) called at /usr/local/cpanel/scripts/updatenow.static-cpanelsync line 875
eval {...} called at /usr/local/cpanel/scripts/updatenow.static-cpanelsync line 875
Cpanel::Update::Now::install_files(Cpanel::Update::Now=HASH(0x9ff448)) called at /usr/local/cpanel/scripts/updatenow.static-cpanelsync line 405
Cpanel::Update::Now::run(Cpanel::Update::Now=HASH(0x9ff448)) called at /usr/local/cpanel/scripts/updatenow.static-cpanelsync line 39348
eval {...} called at /usr/local/cpanel/scripts/updatenow.static-cpanelsync line 39348
[2014-06-19 01:53:44 -0700] warn [updatenow] program "/scripts/premysqlup" is not executable at /usr/local/cpanel/Cpanel/Hooks.pm line 213
Cpanel::Hooks::exec_script('main', HASH(0x42c39e0), HASH(0x42c4e90), undef) called at /usr/local/cpanel/Cpanel/Hooks.pm line 126
Cpanel::Hooks::exec_hook('main', HASH(0x42c39e0), HASH(0x42c4e90), undef) called at /usr/local/cpanel/Cpanel/Hooks.pm line 60
eval {...} called at /usr/local/cpanel/Cpanel/Hooks.pm line 55
Cpanel::Hooks::hook(HASH(0x42c4e90)) called at /usr/local/cpanel/scripts/updatenow.static-cpanelsync line 28448
eval {...} called at /usr/local/cpanel/scripts/updatenow.static-cpanelsync line 28439
Cpanel::RPM::Versions::File::commit_changes(Cpanel::RPM::Versions::File=HASH(0x2f7ace0)) called at /usr/local/cpanel/scripts/updatenow.static-cpanelsync line 875
eval {...} called at /usr/local/cpanel/scripts/updatenow.static-cpanelsync line 875
Cpanel::Update::Now::install_files(Cpanel::Update::Now=HASH(0x9ff448)) called at /usr/local/cpanel/scripts/updatenow.static-cpanelsync line 405
Cpanel::Update::Now::run(Cpanel::Update::Now=HASH(0x9ff448)) called at /usr/local/cpanel/scripts/updatenow.static-cpanelsync line 39348
eval {...} called at /usr/local/cpanel/scripts/updatenow.static-cpanelsync line 39348
[2014-06-19 01:53:54 -0700] info [smtpmailgidonly] cpanel.config Notification =>*********** via EMAIL [level => 3]
[2014-06-19 01:53:54 -0700] info [smtpmailgidonly] cpanel.config Notification => ************ via PAGER [level => 3]
[06/19/2014:08:53:55 -0000] info [cpsrvd] reloading config based on -HUP signal
[2014-06-19 01:54:33 -0700] warn [auto-adjust-mysql-limits] Cpanel::DBI::Mysql connect('','root',...) failed: Can't connect to local MySQL server through socket '/tmp/mysql.sock' (2) at /usr/local/cpanel/Cpanel/MysqlUtils/Connect.pm line 42
at /usr/local/cpanel/Cpanel/MysqlUtils/MyCnf/Adjust.pm line 61

Cpanel::MysqlUtils::MyCnf::Adjust::catch {...} (__CPANEL_HIDDEN__) called at /usr/local/cpanel/Cpanel/CPAN/Try/Tiny.pm line 162
Cpanel::CPAN::Try::Tiny::try(CODE(0x82b810), Try::Tiny::Catch=REF(0xf40818)) called at /usr/local/cpanel/Cpanel/MysqlUtils/MyCnf/Adjust.pm line 62
Cpanel::MysqlUtils::MyCnf::Adjust::__ANON__() called at /usr/local/cpanel/Cpanel/MysqlUtils/MyCnf/Adjust.pm line 150
Cpanel::MysqlUtils::MyCnf::Adjust::auto_adjust(HASH(0x818fe0)) called at /usr/local/cpanel/scripts/auto-adjust-mysql-limits line 33
scripts::auto_adjust_mysql_limits::run('--no-restart') called at /usr/local/cpanel/scripts/auto-adjust-mysql-limits line 14
[06/19/2014:08:54:57 -0000] info [cpsrvd] reloading config based on -HUP signal
[2014-06-19 01:55:07 -0700] warn [updatenow] program "/scripts/postdovecotup" is not executable at /usr/local/cpanel/Cpanel/Hooks.pm line 213
Cpanel::Hooks::exec_script('main', HASH(0x42d21a0), HASH(0x44483f8), undef) called at /usr/local/cpanel/Cpanel/Hooks.pm line 126
Edit2:
Finally I fixed the problem in a dirty way creating a symlink
# ln -s /var/lib/mysql/mysql.sock /tmp/mysql.sock
# service mysql restart

For what I read the problem should be in some set vars in that /usr/local/cpanel/Cpanel/Mysql.pm (line 140)
The solution should be something like this but I can't see the source code
MYSQL_UNIX_ADDR=/var/lib/mysql/mysql.sock
MYSQL_UNIX_PORT=/var/lib/mysql/mysql.sock
MYSQL_SOCKET=/var/lib/mysql/mysql.sock
 
Last edited:

cPanelMichael

Administrator
Staff member
Apr 11, 2011
47,909
2,229
463
Finally I fixed the problem in a dirty way creating a symlink

# ln -s /var/lib/mysql/mysql.sock /tmp/mysql.sock
# service mysql restart
I am happy to see you were able to resolve the issue. Note we do have an internal case open (#101989) to address this issue throughout cPanel.

Thank you.
 

ITGabs

Well-Known Member
Jul 30, 2013
81
0
6
cPanel Access Level
Root Administrator
I am happy to see you were able to resolve the issue. Note we do have an internal case open (#101989) to address this issue throughout cPanel.

Thank you.
It's possible to add a note in the internal case open (#101989) to update this thread when it's patched or fixed?
 

Venomous21

Well-Known Member
Jun 28, 2012
85
0
6
cPanel Access Level
Root Administrator
Please, forgive me if I should have included this in its own thread but it seemed similar. Would like to know your thoughts.

Received a Cron email from one of my servers this morning:

Cron <[email protected]***> /usr/bin/test -x /usr/local/cpanel/scripts/update_db_cache && /usr/local/cpanel/scripts/update_db_cache
body:
/bin/sh: line 1: 24104 Killed /usr/local/cpanel/scripts/update_db_cache

I checked /var/lib/mysql error log & at the same time, I saw:

140625 08:32:50 mysqld_safe Number of processes running now: 0
140625 08:32:50 mysqld_safe mysqld restarted
140625 8:32:50 [Warning] /usr/sbin/mysqld: Option '--set-variable' is deprecated. Use --variable-name=value instead.
140625 8:32:50 [Warning] '--log_slow_queries' is deprecated and will be removed in a future release. Please use ''--slow_query_log'/'--slow_query_log_file'' instead.
140625 8:32:50 [Warning] '--safe-show-database' is deprecated and will be removed in a future release. Please use 'GRANT SHOW DATABASES' instead.
140625 8:32:50 [Note] Plugin 'FEDERATED' is disabled.
140625 8:32:51 InnoDB: Initializing buffer pool, size = 8.0M
140625 8:32:51 InnoDB: Completed initialization of buffer pool InnoDB: The log sequence number in ibdata files does not match InnoDB: the log sequence number in the ib_logfiles!
140625 8:32:51 InnoDB: Database was not shut down normally!
InnoDB: Starting crash recovery.
InnoDB: Reading tablespace information from the .ibd files...
InnoDB: Restoring possible half-written data pages from the doublewrite
InnoDB: buffer...
140625 8:33:04 InnoDB: Started; log sequence number 0 4072497396
140625 8:33:06 [Note] Event Scheduler: Loaded 0 events
140625 8:33:06 [Note] /usr/sbin/mysqld: ready for connections.
Version: '5.1.73-cll' socket: '/var/lib/mysql/mysql.sock' port: 3306 MySQL Community Server (GPLv2)

A developer was apparently working on a test site around this same time. I will let the developer know and the server has been stable since this incident from this morning. Do you recommend investigating further?

Thank you.
 

ITGabs

Well-Known Member
Jul 30, 2013
81
0
6
cPanel Access Level
Root Administrator
Try to create a new account in WHM, if you get an error, it's the same bug

the work around is this symlink

ln -s /var/lib/mysql/mysql.sock /tmp/mysql.sock
service mysql restart

and again try to create a new account in WHM...
if you still get an error means is another thing, better to revert (delete) the symlink in that case

EDIT:
I got the error again and was my tmpwatch that delete the symlink
just to avoid more complications...

crontab -e
0 * * * * /usr/sbin/tmpwatch --mtime --all 12 /tmp && ln -s /var/lib/mysql/mysql.sock /tmp/mysql.sock 2>/dev/null
 
Last edited:

cPanelMichael

Administrator
Staff member
Apr 11, 2011
47,909
2,229
463
It's possible to add a note in the internal case open (#101989) to update this thread when it's patched or fixed?
Yes, I will update this thread when a resolution has been pushed out.

Thank you.
 

Webuser2014

Member
Jan 10, 2014
9
0
1
cPanel Access Level
Website Owner
Yes, I will update this thread when a resolution has been pushed out.

Thank you.
Ok I am getting this error while running the Cpanel EasyApache as shown below. Any ideas on what to do? Thank you.

-- Begin step 'Updating rule id(s) and directive names in modsec conf file(s)' --
!! Include directives in all configuration files: '/usr/local/apache/conf/httpd.conf /usr/local/apache/conf/extra/proxy-html.conf /usr/local/apache/conf/modsec2.user.conf /usr/local/apache/conf/mod_ruid2.conf /usr/local/apache/conf/modsec2.conf' !!
!! Updated '/usr/local/apache/conf/modsec2.conf' with action id(s): '[1234123456] ' !!
!! Made a backup '/usr/local/apache/conf/modsec2.conf.cpbackup2' before updating '/usr/local/apache/conf/modsec2.conf' !!
-- End step 'Updating rule id(s) and directive names in modsec conf file(s)' --

-- Begin step 'Setting up parser' --
-- End step 'Setting up parser' --

-- Begin step 'Setting up database' --
!! sub-step '1' !!
!! sub-step '2' !!
Cpanel::DBI::Mysql connect('','root',...) failed: Can't connect to local MySQL server through socket '/tmp/mysql.sock' (2) at /usr/local/cpanel/Cpanel/MysqlUtils/Connect.pm line 42
 

cPanelMichael

Administrator
Staff member
Apr 11, 2011
47,909
2,229
463
Have you tried the solution referenced in the previous posts (ln -s /var/lib/mysql/mysql.sock /tmp/mysql.sock)?

Thank you.
 

Webuser2014

Member
Jan 10, 2014
9
0
1
cPanel Access Level
Website Owner
I am confused as to when is the right time to do the sym link. The upgrade seems to be stuck on the sql connection error.

I did ssh in and do that command but it still looks stuck.
 

PPNSteve

Well-Known Member
Mar 13, 2003
413
3
168
Somewhere in Ilex Forest
cPanel Access Level
Root Administrator
Twitter
OK I'm now ALSO having an issue with this.. Went to update MySQL to latest cP version and it stopped as noted above by
Webuser2014
Code:
Cpanel::DBI::Mysql connect('','root',...) failed: Can't connect to local MySQL server through socket '/tmp/mysql.sock' (2) at /usr/local/cpanel/Cpanel/MysqlUtils/Connect.pm line 135
Now MySQL won't restart, build apache is stuck.. and restart the upgrade cause it says an easyapache process is still running (there isn't one running)

NOW WHAT?
 

cPanelMichael

Administrator
Staff member
Apr 11, 2011
47,909
2,229
463
Please ensure you run the following command:

Code:
ln -s /var/lib/mysql/mysql.sock /tmp/mysql.sock
Then, kill off any existing update processes and start it again.

Thank you.
 

cPanelMichael

Administrator
Staff member
Apr 11, 2011
47,909
2,229
463
Note that internal case number 101989 was included with cPanel version 11.44.0.22:

Fixed case 101989: Improve MySQL socket handling for localhost.

Thank you.
 

ITGabs

Well-Known Member
Jul 30, 2013
81
0
6
cPanel Access Level
Root Administrator
Hi Michael,

Thanks for add the info, My servers are in 11.44.1.x but I had a problem again with the socket.

How this was fixed? Apparently still need the symlink in /tmp ?

Please confirm, I am having problems right now.
 

cPanelMichael

Administrator
Staff member
Apr 11, 2011
47,909
2,229
463
Could you elaborate on the specific problem you had with the MySQL socket?

Thank you.
 

ITGabs

Well-Known Member
Jul 30, 2013
81
0
6
cPanel Access Level
Root Administrator
Well everything started with problems with Exim 4 hours ago
"451 Temporary local problem - please try later"

I checked the logs and I found a lot of problems...

Code:
[2014-09-10 07:07:50 -0700] warn [cpanellogd] RRDs Error: /var/cpanel/bandwidth/obrcozat-all-peak.rrd: DBD::mysql::db do failed: Out of resources when opening file './eximstats/smtp.MYD' (Errcode: 24)
Cpanel::Exception::create_raw('Database::Error', 'DBD::mysql::db do failed: Out of resources when opening file ...', 'HASH(0x332c588)') called at /usr/local/cpanel/Cpanel/DBI.pm line 79
Cpanel::DBI::_create_exception('Cpanel::DBI::Mysql::db=HASH(0x308e738)', 'DBD::mysql::db do failed: Out of resources when opening file ...', undef) called at /usr/local/cpanel/Cpanel/DBI.pm line 70
Cpanel::DBI::_error_handler('DBD::mysql::db do failed: Out of resources when opening file ...', 'Cpanel::DBI::Mysql::db=HASH(0x308e738)', undef) called at /usr/local/cpanel/Cpanel/Logd.pm line 1379

Cpanel::DBI::Mysql connect('','leechprotect',...) failed: Can't connect to local MySQL server through socket '/tmp/mysql.sock' (2) at /usr/local/cpanel/Cpanel/MysqlUtils/Connect.pm line 136

...
Ensuring services are online.../usr/bin/mysqladmin: connect to server at 'localhost' failed
error: 'Can't connect to local MySQL server through socket '/var/lib/mysql/mysql.sock' (11)'
Check that mysqld is running and that the socket: '/var/lib/mysql/mysql.sock' exists!
....
a cron fail
/usr/bin/mysqladmin: connect to server at 'localhost' failed
error: 'Lost connection to MySQL server at 'waiting for initial communication packet', system error:
95'
DBD::mysql::st execute failed: Can't read dir of '.' (errno: 24)
at /usr/local/cpanel/Cpanel/Exception.pm line 96
Cpanel::Exception::new('Cpanel::Exception::Database::Error',
'Cpanel::ExceptionMessage::Raw=SCALAR(0x1359180)', 'HASH(0x274ac00)') called at
/usr/local/cpanel/Cpanel/Exception.pm line 47
Cpanel::Exception::create('Database::Error', 'Cpanel::ExceptionMessage::Raw=SCALAR(0x1359180)',
'HASH(0x274ac00)') called at /usr/local/cpanel/Cpanel/Exception.pm line 55
Cpanel::Exception::create_raw('Database::Error', 'DBD::mysql::st execute failed: Can\'t read dir of
\'.\' (errno: ...', 'HASH(0x274ac00)') called at /usr/local/cpanel/Cpanel/DBI.pm line 79
Cpanel::DBI::_create_exception('Cpanel::DBI::Mysql::st=HASH(0x2735a70)', 'DBD::mysql::st execute
failed: Can\'t read dir of \'.\' (errno: ...', undef) called at /usr/local/cpanel/Cpanel/DBI.pm line
70
Cpanel::DBI::_error_handler('DBD::mysql::st execute failed: Can\'t read dir of \'.\' (errno: ...',
'Cpanel::DBI::Mysql::st=HASH(0x2735a70)', undef) called at bin/update_db_cache line 322
Script::Update::DB::Cache::can_get_mysql_usage('HASH(0x1332cb0)', 'HASH(0x13fe430)',
'Cpanel::DB::Map::Collection::Index=HASH(0x2733e60)') called at bin/update_db_cache line 57
Script::Update::DB::Cache::script('Script::Update::DB::Cache') called at bin/update_db_cache line
32
and finally I tried to create a new account (wwwacct) and I got a similar error than before
Code:
Ensuring services are online.../usr/bin/mysqladmin: connect to server at 'localhost' failed
error: 'Can't connect to local MySQL server through socket '/var/lib/mysql/mysql.sock' (11)'
Check that mysqld is running and that the socket: '/var/lib/mysql/mysql.sock' exists!
Mysql was working in all the sites but apparently something happen (36 or 24 hours ago) after I deleted the symlink, few minutes ago I rebooted the server and checked the /tmp folder and now without symlink everything it's working fine.

Probably after I deleted the symlink from the tmpwatch cron I should running it manually first and later restart mysql not 100% sure.

Code:
crontab -e
    0 * * * * /usr/sbin/tmpwatch --mtime --all 24 /tmp && ln -s /var/lib/mysql/mysql.sock /tmp/mysql.sock 2>/dev/null

to 
0 * * * * /usr/sbin/tmpwatch --mtime --all 24 /tmp 2>/dev/null
 
Last edited by a moderator:

cPanelMichael

Administrator
Staff member
Apr 11, 2011
47,909
2,229
463
I am happy to see the issue is now resolved. Could you clarify what issue lead you to manually deleting the symbolic link?

Thank you.