Log Rotation

[ecoutez]

Apache Server logs: /etc/httpd/logs
Apache Domain (VirtualHost) logs: /etc/httpd/domlogs/

- Jason

[unite the cows]

Failed loading /usr/local/Zend/lib/ZendOptimizer.so: /usr/local/Zend/lib/ZendOptimizer.so: cannot open shared object file: No Such File or Directory

What is causing this and what can I do to fix this? Sorry to get off topic. Thank you for showing me the correct place for the logs.

[Juanra]

There are several /scripts/installzendopt-php*** scripts to install the Zend Optimizer according to you PHP version.

[unite the cows]

what if I am using php4.2.2 i looks like 4.2.1 is the latest on /scripts/installzendopt-php**** is it ok to use the 421 ?

[ecoutez]

[quote:665b9648ba][i:665b9648ba]Originally posted by bdraco[/i:665b9648ba]

Logs are rotated on the last day of the month, they always have been AFAIK. My guess is that one of the latest patches fixed rotation on the servers that is was broken on.[/quote:665b9648ba]

I'm sure you know, but domlogs aren't rotated... they're just deleted. I can't be the only one getting support requests from clients asking for raw log files which were deleted on the last day of the month.

I'd rather have CPanel not touch the domlogs at all than to throw them away completely. At least then I could setup an entry in logrotate.d and rotate them monthly instead of deleting the data entirely.

Any chance of allowing us to specify a rotation directory? Something like /etc/httpd/domlogs/rotated/ would be great for me. Maybe as part of version 5? *begging*

Thanks,
Jason

[showdoc]

hey folks -

this is my first forray into web hosting and (like you all) just lost our first month's worth of logs... argh!!!! although i did grab one from the 23rd... just playing around with it - trying out some other analyzing software - lucky that i did...

but, really, at THE VERY LEAST move a copy of the log into a designated folder BEFORE deleting it...and let us delete it when we want to... my logs are only like 300k...

and, hehehehe, like electro mentioned - sooo, exactly WHEN are we supposed to copy the &last& log of the &full& month? and what time? and if we copy it too early, we STILL LOSE the last data of the month when it gets deleted again!

it's very critical for us to &SEE& where and specifics of ALL the visitors to our site are.... (and i'm sure there are MANY companies out there that NEED that same information)

i just went round and round today with my hosting co. and read them the riot act about the deletion of the logs... if i was developing software for web hosting/analysis, i know that keeping a RECORD of ALL web activity would be of PARAMOUNT IMPORTANCE to my clients... they NEED that information to survive!!!! if they want to delete it themselves after they receive it, that's fine... but geeeeeeeeezzzz!!! =)

morelater
the show doctor

[dwh]

There's nothing wrong with deleting raw data after a period of time. It's unlikely that a particular user needs a particular line of what someone did at 8:15 am 3 months ago. What everyone seems to be missing is the basics on how this has been handled traditionally.

If data is rolled up properly, the few that need raw data can be given certain options. If the files are bigger then the HD then you need backup on tape. No reason HD has to store this. Pay for it, should be easy for CPanel to give u any option you want.

As for the data rollup, what stats do you need? I think every unique referer is important. If mr.X visited your site 100 times, all you need is 1 line summarizing that info as such:

IP | # of visits
-----------------
Mr. X | 100
Mr. Z | 212
etc..

If url http://y.com/somepage.html refered 1000 people to your site, you don't need every instance of that one month later. You need one line:

referer | # of referals
---------------------------
http://Y.com/somepage | 1000
http://x.com | 212

etc..

I'm oversimplifying it, but you just need to figure out the important unique data you want to keep, define that key, do regular rollups and everyone will be happy.

The problem is, if you wait a full month to handle rollups, especially if you use perl or something like that, the server will croak on all that data. But if you do a daily rollup, then summarize that at the end of the month, you have manageable and useful data that doesn't overwhelm your server.

This is definitely a really big issue that CPanel needs to address.

[Juanra]

We must keep logs for as much as 12 months.

[thomas]

Don't see what the problem is, you chose if the logs should be deleted or not in &Tweak Settings&.

[dwh]

Thomas,
I was confused by your comment because I didn't have any of those options, but it turns out I had a very old version of cpanel on my server. Unfortunately I upgraded. My server loads went through the roof. My domain stopped logging altogether. I get emails every 6 hours that my stats failed. I'm very confused and upset and don't know what to do! The log for one domain is over 2 Gigabyte.

[dwh]

Seems like after 25 days my log hits 2 gigabyte which cpanel doesn't seem to be able to handle...I have to delete the file manually and my logs get messed up...Is there a way to rotate the logs weekly and still get monthly stats in cpanel?

Thank you.

[portman]

That sounds like an old unix/linux file size limit. What OS are you running and version?

There are a number of issues that could arise from a file of that size. The issue may be OS or application (Apache, webilizer, etc.). Certainly I am only talking in generalizations right now.

[Website Rob]

Was it ever resolved as to what WHM/Cpanel does with the logs and at what time of the month?

[dwh]

[quote:314f2d2385][i:314f2d2385]Originally posted by portman[/i:314f2d2385]

That sounds like an old unix/linux file size limit. What OS are you running and version?

There are a number of issues that could arise from a file of that size. The issue may be OS or application (Apache, webilizer, etc.). Certainly I am only talking in generalizations right now.[/quote:314f2d2385]

Red hat 6.2.

Now I'm starting to get those emails about not being able to process the log file within 6 hours after a log file of under a gigabyte.

What are the optimal settings for the log files that you guys use when you have some domains with huge traffic?