Stats not updating... Logs not rotating... Usr getting full fast

[Legin76]

Hi I have a problem with my stats. None of them are updating on the whole server. . This is causing the logs not to rotate and the /usr drive to get full.

They have not updated since 13 Jan 2011 - 18:17.

I'm not sure but it may have roughly when cpanel last updated.

WHM 11.28.64
REDHAT Enterprise 5.6 i686 standard on srv04

In Statistics Software Configuration it says

The server is having trouble keeping up with your statistics processing schedule. You should increase the time between statistic generation, or upgrade the server. If you have recently decreased the time between statistic generation, you may wish to wait that amount of time to see if the server will catch up before changing back.

and the Schedule Summary is

Web Traffic Statistics every: 12 hours
Bandwidth Statistics every: 2 hours

 

[LinuxTechie]

Hello,

You should go to WHM -> Server Configuration -> Statistics Software Configuration and go to the Schedule Configuration section. By default that is 24 hours. You may want to change this to 48 hours.

 

[Legin76]

Ok.. Thanks.. I've updated the Log Processing Frequency to 24 hours however I don't understand why this would be a problem. Surly the more often it updates the less it has to do in one go and the load is spread out.

 

[Infopro]

I suppose that might depend on what else is running at the time the logs are processed. You wouldn't want this going on at the same time backups or updates are running for example.

AFAIK, if I changed these settings you have at 10AM this morning, 24 hours later at 10AM stats logs would be processed. If I changed these settings and then went and changed my backup settings right afterward to something else, it too might be then set to run 24 hours from now, I just changed the setting for them both.


I'm just curious, do you by chance have Analog stats ticked as "Available to Users"?

 

[Legin76]

I'm with it being set to 12 hours as it was though surly at one time it was processing something else it would do it the next attempt.

I've also tried using Process Statistics for User for one site.. Granted the biggest but after a couple of hours there is no change. The load has been very low during this time.

I'm just curious, do you by chance have Analog stats ticked as "Available to Users"?

Yes I believe do.

 

[Infopro]

Analog stats can take up a lot of space on an account.

Re-reading your original post:

This is causing the logs not to rotate and the /usr drive to get full.

You might want to see what you can do about clearing out some space. Use the forums search tool to search for: /usr full You'll find many pages of threads on this topic that should be helpful there.

 

[Legin76]

Thanks.. thats how I got down to it being the logs.

It just discovered that the manual process button does work but it does not update the "Show which users are behind" info which I guess is only updated once or twice a day.


I've looked round at the other usr folders and other than the domlogs, the biggest folder is /usr/local/cpanel/ and most of that appears to be needed. The largest areas in there by far are /usr/local/cpanel/whostmgr and /usr/local/cpanel/src. I can find little on what is safe to delete from those. There are a few mailman folders using about 50mb each though.

 

[LinuxTechie]

Hello,

Check :

>> /usr/local/cpanel/logs/
>> /usr/local/apache/logs
>> /usr/local/src/
>> /usr/src
>> /usr/local/apache/htdocs

 

[Legin76]

Thanks Varada... The logs in those in all those log folders total about 400mb. But the /usr/local/cpanel/whostmgr and /usr/local/cpanel/src appear use up the most. Knowing what to delete from those is a worry.

My usr directory is always on the full side. It would be nice to take a couple of GB from the / directory which only uses 7% of 7.8G.

I don't think the partition sizes are as balanced as they could be by default. Usr also has 7.8G but uses most of it and logs can grow fast as we all know. I think my / directory would be fine with 2GB unless it needs the space it for some other reason.

 

[cPanelTristan]

Hello Legin76,

Only mailman could be moved in the /usr/local/cpanel directory safely for some of the files. You may want to review this guide on how to do that:

Full /usr partition

In /usr itself, you could move /usr/local/apache/domlogs to another partition as well and symlink it. The steps to do that would be the following:

cd /usr/local/apache
mv /usr/local/apache/domlogs /home/
ln -s /home/domlogs domlogs (you should still be in /usr/local/apache when running this command)

You may want to make a backup of the domlogs before doing the above. The worst that can happen with domlogs, though, is that you lose the files. If you process them every day, that's only a day or less of data.

Thanks.

 

[Legin76]

Thank you.

Does turning on Allow users to update Awstats from cPanel cause cpanel to stop updating them itself? I may have changed that at around the date the issue started.. If it does then its a bad thing.

 

[cPanelTristan]

That setting should not stop Awstats from updating every time the stats run.

 

[Legin76]

cd /usr/local/apache
mv /usr/local/apache/domlogs /home/
ln -s /home/domlogs domlogs (you should still be in /usr/local/apache when running this command)

Could I use this same instructions to move an account to the second disk? The account is only used for backups from another server via ftp.

 

[Legin76]

It appears thats the logs are still not automatically processing. If I manually process them they appear to do what they should but it still shows as not updating since the 13th Jan in the Statistics Software Configuration area on all accounts. Something appears to be stopping them.

Stats Last Processed at:
Thu Jan 13 18:09:46 2011
Bandwidth Last Processed at:
Thu Jan 13 23:12:25 2011

 

[cPanelTristan]

Hello Legin76,

I would suggest opening a ticket with us then at WHM > Support Center > Contact cPanel or using the link in my signature. It's difficult to diagnose why stats aren't processing via the forum, since we'd need to try a run on one or more user accounts manually to see if there are any errors. The manual runs are done with this command in root SSH:

/scripts/runweblogs username

Where username is the cPanel username for the account. This will one time process stats for the user, which then outputs any possible errors.

 

[cPanelTristan]

Could I use this same instructions to move an account to the second disk? The account is only used for backups from another server via ftp.

You could use the mv and symlink commands to move anything, so, yes, you could move an account and then symlink it this way. Otherwise, you could simply move the account if you didn't need to symlink it.

 

[Legin76]

Ok thanks.. Ticket submitted.. 1179282.

 

[rwagener11]

I am having the same issue. Cam out of no where but I can't get the domlogs to process anymore. They just keep filling up.

 

[Legin76]

I believe that in my case there was a file that should have deleted during the last update.

I think restarting cpanel fixed it although they may have deleted the file first.

 

[johnburk]

I have a similar problem. The domlogs get stuck at 512mb and are no longer updating and awstats does not work without domlogs.