Apache error_log "optimization"

[chrismfz]

Not very often but from time to time I check the error_log
of apache in /usr/local/apache/logs.

In about 3-4 months the log is just HUGE.
(Output from a live production server
# cat error_log | wc -l
597663

Ok, 597663 lines log... Let's analyze it a little more:

# grep "robots.txt" error_log | wc -l
39311

# grep "404" error_log | wc -l
250740

# grep "favico" error_log | wc -l
29470

So, 39311+250740+29470=319521 lines of errors in error_log
from 597663 totally.

So there are 278142 error lines needed, or let's say error lines that needs
investigation from 597663 totally.


These error I greped is all for not existent error pages (404), robots.txt and favicon.ico that trey are not exists in every user that doesnt know anything about this. Or just they dont have/care about a robots.txt or a 404 page.


So, I was wondering if there is a "Cpanel" Friendly way or a script to create
those pages or just copy from a "template" to users directories but WITHOUT harming pages of users that already done that.

If I do this with a cp (copy) script in a cronjob I will get a
cp: overwrite message at the 1st user who already has a file of these.
If I do this with a cp -fr, it will overwrite all existing files
Inspecting everones directory is an option but a hardcore one if you have
100/200/300+ users.

Anyone has a script for doing this or a tip how to do it without the risk
of altering/deleting existing 404/robots/favicon files ?

This should reduce the "errors" in apache error_log and we can have a
more reasonable log amount to watch.

Regards,
Chris

[dwh2]

Did you have any luck with this?

[EcoHosting]

I have read a post here on this before. I am trying in vain to locate the right keyword search combo....I know the answer is here!

[Infopro]

There is such a script that I think you might find useful called fix404errors located here. cPlicensing: cPanel Scripts

Never used it myself, but you may be able to work with it and edit for favicons as well.

For this reason, its a good thing to mention (for others who read this) that adding all required error pages (not just a 404), favicon, robots.txt and whatever else you feel is needed to the cpanel3-skel in advance of creating any new accounts. so you can avoid this sort of issue down the road.

I remember dealing with this years ago myself and went thru every account manually and added what was missing, the log entries were driving me nuts.

HTH