Master Apache Access Log

[Mooash]

Hi all,

I'm just wondering if it's possible to create a master apache log file for all accounts on the server? I'm looking to create one then load it into analytics software once it's rotated once a week or once a month. Is this possible to do?

I realise I could possible use the domlogs directory but I believe once this rotates it gets moved to the archived folder in the users home directory, and it archives once it hits a certain size instead of at a certain time. Correct me if I'm wrong.

Anyway, any help would be greatly appreciated!

Cheers,
James

 

[cPanelMichael]

Hello

Have you considered using the "/usr/local/apache/logs/access_log" file? Or, you can turn off the following option under the "Stats and Logs" tab in "WHM Home » Server Configuration » Tweak Settings":

"Delete each domain’s access logs after stats run"

This will ensure the Apache domlogs are not removed.

Thank you.

 

[Mooash]

Hi Michael,

Not a bad idea but not quite what I'm after. The log in /usr/local/logs/access_log only tracks the apache logs for WHM access, whilst handly it isn't what I need.

I need to create a master log for all virtual hosts on the system. I've tried using the method below but I think it breaks cPanel's apache tracking, I've got piped logging enabled so it uses cpanellogd.

Step 1)
Add new log format to apache via pre_virtualhost_2.conf as such
File: /usr/local/apache/conf/includes/pre_virtualhost_2.conf (I did this through the WHM interface)

Include /etc/custom/httpd/*.conf

Step 2)
Place custom format in aforementioned directory like below:
File: /etc/custom/httpd/custom_apache_log.conf

LogFormat "%v %h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" MyCustomFormat

Step 3)
Place file in /usr/local/apache/conf/userdata so it adds it to all virtualhosts as per below
File: /usr/local/apache/conf/userdata/custom_log_tracking.conf

CustomLog logs/custom_access_log MyCustomFormat

Then I followed the instructions here to re-create my apache config which worked fine and everything started logging to that file stored in /usr/local/apache/logs/custom_access_log

Where am I going wrong? It appears to break cPanel's log collecting? I no longer see logs in /usr/local/apache/domlogs/*-log filling with data but now that I've removed it everything seems to be working fine again.

Is there a way I can get the above to work without breaking the inbuilt logging functionality?

 

[santrix]

I just add something like this to the pre_main_global.conf

LogFormat "%a %v %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-agent}i\"" mylog
CustomLog /usr/local/apache/logs/apache_mainlog mylog

This gives me all the info I need to see who's accessing what.

Actually, I do wonder why the access_log configuration cpanel creates does not include the virtual host itself because on all of our servers it seems to log everything for all vhosts anyway, not just requests to the default hostname.

I see in the httpd.conf, <IfModule log_config_module> section it states

LogFormat "%h %l %u %t \"%r\" %>s %b" common
CustomLog "logs/access_log" common

But a few lines later inside a <IfModule mod_log_config.c> test it says

    LogFormat "%v:%p %h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combinedvhost
    LogFormat "%v %{%s}t %I .\n%v %{%s}t %O ." bytesvhost
    LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined
    LogFormat "%h %l %u %t \"%r\" %>s %b" common
    LogFormat "%{Referer}i -> %U" referer
    LogFormat "%{User-agent}i" agent

    CustomLog "|/usr/local/cpanel/bin/splitlogs --main=ourhost.co.uk --suffix=-bytes_log" bytesvhost
    CustomLog "|/usr/local/cpanel/bin/splitlogs --main=ourhost.co.uk --mainout=/usr/local/apache/logs/access_log" combinedvhost

I'm not sure what splitlogs is actually doing and what it is adding to the access_log file specifically, because access_log on our servers seem to follow the format %h %l %u %t \"%r\" %>s %b whereas combinedvhost clearly carries more information than that.

So, I guess I'm still wondering is I really do have to run an extra log for analysis as I stated at the top of this reply, or if there is a more elegant way of altering the apache config to include the canonical servername %v in the default access_log file - this would make abuse hunting much easier in some cases.