Check out /usr/local/cpanel/logs/ !!!

[H2Hosting.com]

Hi

Check out /usr/local/cpanel/logs/ !!!

File stats_log is 2Gb! 2.147.483.647

ERROR: illegal attempt to update using time 1046782149 when last update time is 1047560796 (minimum one second step)
ERROR: illegal attempt to update using time 1046782235 when last update time is 1047560796 (minimum one second step)
ERROR: illegal attempt to update using time 1046782272 when last update time is 1047560796 (minimum one second step)
ERROR: illegal attempt to update using time 1046782332 when last update time is 1047560796 (minimum one second step)
ERROR: illegal attempt to update using time 1046782490 when last update time is 1047560796 (minimum one second step)
ERROR: illegal attempt to update using time 1046782710 when last update time is 1047560796 (minimum one second step)
ERROR: illegal attempt to update using time 1046782865 when last update time is 1047560796 (minimum one second step)
ERROR: illegal attempt to update using time 1046782886 when last update time is 1047560796 (minimum one second step)
ERROR: illegal attempt to update using time 1046783008 when last update time is 1047560796 (minimum one second step)
ERROR: illegal attempt to update using time 1046783225 when last updat

2gb limit here ;(

On another server ~1.9Gb (1.954.178.437) and the limit will be reached in 1h or less.

... @&?^%%!

[H2Hosting.com]

The same error on second server when I

tail -f stats_log

ERROR: illegal attempt to update using time 1047067617 when last update time is 1047593081 (minimum one second step)
ERROR: illegal attempt to update using time 1047067671 when last update time is 1047593081 (minimum one second step)
ERROR: illegal attempt to update using time 1047067696 when last update time is 1047593081 (minimum one second step)

I killed cpanellogd process on all servers ...... as it create huge log file with errors...

[mitul]

what was the last upgradation done on the server?

[FWC]

Originally posted by mitul
what was the last upgradation done on the server?

The "last upgradation" is what is doing this to all of us...

[Annette]

Do this:

rm /usr/local/cpanel/logs/stats_log
touch /usr/local/cpanel/logs/stats_log
/etc/rc.d/init.d/cpanel3 restart

That will reset the stats_log and the log processing daemon. It's not a permanent solution - the stats_log grows each day and doesn't seem to be rotating on its own any longer, so you'll need to keep an eye on it or just send the results to /dev/null if you don't need the logs for troubleshooting purposes (this is what we're doing).

[FWC]

Originally posted by Annette
Do this:

rm /usr/local/cpanel/logs/stats_log
touch /usr/local/cpanel/logs/stats_log
/etc/rc.d/init.d/cpanel3 restart

That will reset the stats_log and the log processing daemon. It's not a permanent solution - the stats_log grows each day and doesn't seem to be rotating on its own any longer, so you'll need to keep an eye on it or just send the results to /dev/null if you don't need the logs for troubleshooting purposes (this is what we're doing).

I followed all your advice, Annette. Things are working much better now. Thank you.

[EcpHosting]

Yes, this caused all of our servers to put some serious weight on disk space and on server MAX out the main HD. Grr... and this is CPanels STABLE release?

[ozzi4648]

Why dont you add it to logrotate.conf as a daily rotation? No need to delete and touch anything!

[cpanelnick]

Keep Stats Log (/usr/local/cpanel/logs/stats_log) between cpanel restarts (default is off)
Stats Log Level (default is 1) [0...5]:

I've added two options to the EDGE builds which should take care of this.

[cpanelnick]

You might want to check the system time since you may have a clock skew.

rdate -s rdate.darkorb.net

Originally posted by H2Hosting.com
Hi

Check out /usr/local/cpanel/logs/ !!!

File stats_log is 2Gb! 2.147.483.647

ERROR: illegal attempt to update using time 1046782149 when last update time is 1047560796 (minimum one second step)
ERROR: illegal attempt to update using time 1046782235 when last update time is 1047560796 (minimum one second step)
ERROR: illegal attempt to update using time 1046782272 when last update time is 1047560796 (minimum one second step)
ERROR: illegal attempt to update using time 1046782332 when last update time is 1047560796 (minimum one second step)
ERROR: illegal attempt to update using time 1046782490 when last update time is 1047560796 (minimum one second step)
ERROR: illegal attempt to update using time 1046782710 when last update time is 1047560796 (minimum one second step)
ERROR: illegal attempt to update using time 1046782865 when last update time is 1047560796 (minimum one second step)
ERROR: illegal attempt to update using time 1046782886 when last update time is 1047560796 (minimum one second step)
ERROR: illegal attempt to update using time 1046783008 when last update time is 1047560796 (minimum one second step)
ERROR: illegal attempt to update using time 1046783225 when last updat

2gb limit here ;(

On another server ~1.9Gb (1.954.178.437) and the limit will be reached in 1h or less.

... @&?^%%!

[Annette]

Thanks, Nick, for tweaking the logfile size/retention. You're a prince among men.

[dariofg2]

Nick,

I did the

rdate -s rdate.darkorb.net

but it didn't work. cpanellogd dumps a LOT of

...
ERROR: illegal attempt to update using time 1047817297 when last update time is 1047887171 (minimum one second step)
ERROR: illegal attempt to update using time 1047817359 when last update time is 1047887171 (minimum one second step)
...

on /usr/local/cpanel/logs/stats_log

and uses a LOT of CPU at that. I have to restart cpanel3

/etc/init.d/cpanel3 restart

so it can stop using up CPU and the stats_log goes back to show normal stats processing messages. But after a few hours, I see that the CPU usage at my server is high again (2-4%), and it's cpanellogd with those update time messages!!! That did not happen in earlier cpanel versions, what's up?

-Dario

[EcpHosting]

This seems to be a problem for all of us. Any reports on this?

We have done the rdate -s rdate.darkorb.net.. but what exactly does this do?

[dariofg2]

That command synchronizes the server time with darkorb's. It does NOT work, though.

Here's what I did to patch the situation: I created the following script and placed it under crontab running every minute:

#!/bin/bash

str=`tail -1 /usr/local/cpanel/logs/stats_log | cut -c -43`
if [ "$str" == "ERROR: illegal attempt to update using time" ]
then
/etc/init.d/cpanel3 restart
fi

What it does is find out if cpanellogd is "locked up" (again!) and restarts the cpanel services, which resets cpanellogd CPU usage to a normal level.

I sure hope to see a real solution to this problem, though.

-Dario

[dandanfireman]

Date sync is not the problem. We need a resolution to this Nick.