2 GIG log file kills web server

[ethank]

I'm having an issue with one of my sites, where the log file for one site goes to 2.0 gigs, and when that happens, it won't log anymore and the webserver starts dying. I have to clear the log for it to start working.

Does anyone know how to fix this?

 

[ntwaddel]

you could tell cpanel to clear the log everytime the stats run, hopefully its not growing to over 2gigs everyday

 

[simplestar]

I can't remember the name of it, but there is an RPM you can install that will watch and dump logs before they hit the 2G mark.

 

[ethank]

you could tell cpanel to clear the log everytime the stats run, hopefully its not growing to over 2gigs everyday

Twice a day it grows, and the stats can't keep up.

 

[aby]

Twice a day it grows, and the stats can't keep up.

You can set up a cron to clear the logs with in a specific time period .

 

[brianoz]

That's the wrong way to fix it, you really need to work out why that log file is growing so rapidly and fix the problem at cause. That sort of log growth doesn't sound like a normal pattern of access.

 

[aby]

That's the wrong way to fix it, you really need to work out why that log file is growing so rapidly and fix the problem at cause. That sort of log growth doesn't sound like a normal pattern of access.

That depends on the site .. i would say... if he is having a very busy site, .. But ... yes .. it is true that it is Normal.. But I have seen many cases ... in which the traffic is so high that you have to go for this.. But I agree, it is worth to investigate more..

 

[lart2150]

i'm new to cpanel but you could try to tell logrote to rote your logs more frequently.

 

[brianoz]

Yeah, sorry for not adding this above, but I did know that 2Gb a day is *possible*, I just think it's likely something is going wrong. We've had MSNbot and Googlebot go into loops or semi-loops on some sites here, so I'm suspecting something like that. It could also be referrer spam, or any one of a number of things. He specifically didn't mention it was a very high traffic site, so if it's not (and he'd know as it usually builds up over time), my bet is there'll be a culprit! Unless it was slashdotted! Anyway ... all theory until the OP responds ...

- b

 

[ethank]

I know its legit traffic, as its a hugely popular gossip blog. I think what I'll end up doing is using google analytics and stop it logging, but I need a solution until I can do that

 

[abubin]

I know its legit traffic, as its a hugely popular gossip blog. I think what I'll end up doing is using google analytics and stop it logging, but I need a solution until I can do that

if the site can generate more than 2gb of log traffics per day then it is really busy and I think you would be looking at around 200,000 hits per day? Over a few million page views per day? If that is true, you would be looking at some really traffic heavy site for a blog? If this is true, how come I don't know about such blog site? hehehe....

Did you set your log level to minimum at WHM?

 

[ethank]

Its pinkisthenewblog.com

I don't run it, but I host it.

 

[PanelGuy]

2 GB Kills Stats

This has been a long running problem. There is a feature request for it, please vote for it:

http://bugzilla.cpanel.net/votes.cgi?action=show_bug&bug_id=3153

 

[65535]

Isn't this a 32bit limitation of Apache rather than anything else?

Rus

 

[myusername]

That's the wrong way to fix it, you really need to work out why that log file is growing so rapidly and fix the problem at cause. That sort of log growth doesn't sound like a normal pattern of access.

I wouldn't say that. We have a site spread across several servers that constantly has this problem. The problem is the servers are so finely tuned to hand a ton of traffic. those logs can indeed be normal for many busy sites.
We set a cron up for ours because the cpanel option to remove them after stats run is not always reliable.

So much for web stats.

Anyways, I voted as well.