The Community Forums

Interact with an entire community of cPanel & WHM users.
  1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Master Log File in WHM

Discussion in 'Workarounds and Optimization' started by WorkinOnIt, Aug 4, 2016.

Tags:
  1. WorkinOnIt

    WorkinOnIt Well-Known Member

    Joined:
    Aug 3, 2016
    Messages:
    49
    Likes Received:
    1
    Trophy Points:
    8
    Location:
    UK
    cPanel Access Level:
    Root Administrator
    Hello

    I am not sure if this is the correct forum for this question, please excuse me if it is not?

    I am running cloudlinux. In WHM (through CSF) I would like to be able to observe (watch) a "master" log of VirtualHost activity.

    For example, in CSF at the moment, I am able to watch error_log. Whilst this is good, it does not supply all info on e.g. IP address, user agent of visitor, http request etc.

    So I'd like to create a custom "master log" to log all VH visitor traffic to a single log file and show e.g. IP address, HTTP request, referer and User Agent.

    Currently I have a number of virtualhosts in Apache 2.4 httpd file - each with their own logging directive. However, it is my understanding from Apache documentation that if a logging directive is placed outside the VH record, it will take precedence.

    I don't really want to overcomplicate this and have to combine logs and then split them etc. and I would still like each virtual host to log its own data. So I'm looking for an easy way to grab the required info and output it to a custom master log ? I presume I would need to enable log rotation for this new master log too.

    I have tried with the following logging directive, placed at the top of httpd file, outside of VH block;

    Code:
    <IfModule log_config_module>
        # info from http://www.techstacks.com/howto/log-client-ip-and-xforwardedfor-ip-in-apache.html
        LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined
        LogFormat "%{X-Forwarded-For}i %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" proxy
        SetEnvIf X-Forwarded-For "^.*\..*\..*\..*" forwarded
        CustomLog "logs/access_log" combined env=!forwarded
        CustomLog "logs/access_log" proxy env=forwarded
    </IfModule>
    Which results in this in the access_log

    Code:
    ::1 - - [05/Aug/2016:02:49:26 +1200] "OPTIONS * HTTP/1.0" 200 - "-" "Apache/2.4.23 (Unix) OpenSSL/1.0.1e-fips mod_bwlimited/1.4 (internal dummy connection)"`
    I also tried various other suggested logging directives found on stackexchange and on here and in google, but so far I am not having any luck.

    Can anyone assist?

    Thank you
     
  2. cPanelMichael

    cPanelMichael Forums Analyst
    Staff Member

    Joined:
    Apr 11, 2011
    Messages:
    30,723
    Likes Received:
    660
    Trophy Points:
    113
    cPanel Access Level:
    Root Administrator
    Hello,

    There's no native feature in cPanel/WHM to facilitate a combined Apache access log, however a user submitted a custom workaround on the following thread:

    apache master/custom/global access log

    Could you review this thread and let us know if this solution is suitable for what you need?

    Thank you.
     
  3. WorkinOnIt

    WorkinOnIt Well-Known Member

    Joined:
    Aug 3, 2016
    Messages:
    49
    Likes Received:
    1
    Trophy Points:
    8
    Location:
    UK
    cPanel Access Level:
    Root Administrator
    Thank you - that is indeed very helpful and I am not sure why I was unable to come across that thread during searches! I am also (like the other poster) surprised this is not a native feature!

    I am a bit of a novice and have some questions that I wonder you can assist with?

    In the thread mentioned here the OP SergeiF suggests the following:

    However, in my cpanel/templates directory - I have the following folders:

    apache2
    apache2_0
    apache2_2
    apache2_4

    Each of these contain the same files:
    - main.default
    - ssl_vhost.default
    - vhost.default

    I am running 2.4 so which folder should I place the log lines in? I also don't seem to have the vhost.local file mentioned - what does cp from vhost.default mean?

    Another question: I presume this will not interfere with the individual log files that are being generated? I still want clients to be able to access e.g. awstats in cpanel.

    Also, once created - where would I set up the log rotate for this file? Should that be done in logrotate.conf - or should I do this through WHM > Service Configuration > cPanel Log Rotation Configuration

    Thanks





     
  4. cPanelMichael

    cPanelMichael Forums Analyst
    Staff Member

    Joined:
    Apr 11, 2011
    Messages:
    30,723
    Likes Received:
    660
    Trophy Points:
    113
    cPanel Access Level:
    Root Administrator
    I suggest using "/etc/apache2/logs/" as the log storage directory if you are using EasyApache 4. Otherwise, you can use "/usr/local/apache/logs" on EasyApache 3. You should utilize the "/var/cpanel/templates/apache2_4" directory if you are using Apache 2.4. Additional information about these template files is available:

    Custom Templates - EasyApache - cPanel Documentation

    The "cp from vhost.default" reference on that thread is an instruction to copy the "vhost.default" file with a Linux command such as "cp -a /var/cpanel/templates/apache2_4/vhost.default /var/cpanel/templates/apache2_4/vhost.local".

    The custom workaround doesn't reference the individual domain access logs. You could always use a custom entry that relies on that workaround and the existing entry of:

    Code:
    [% IF logstyle == 'combined' -%]
        [%- IF !enable_piped_logs || !supported.mod_log_config -%]
      CustomLog [% paths.dir_domlogs %]/[% wildcard_safe(vhost.log_servername) %] combined
    
    You could configure your system's LogRotate utility to rotate this custom log file by creating a new configuration file in the "/etc/logrotate.d/" directory. Instructions on using LogRotate are available at:

    logrotate

    Thank you.
     
  5. WorkinOnIt

    WorkinOnIt Well-Known Member

    Joined:
    Aug 3, 2016
    Messages:
    49
    Likes Received:
    1
    Trophy Points:
    8
    Location:
    UK
    cPanel Access Level:
    Root Administrator
    Thank you for your assistance. A few more questions;

    1) I presume once I've added this entry to vhost.local I should restart Apache? Or?

    2) I am using log_config_module - is this the correct syntax for adding the entry using the following?

    Code:
        <IfModule log_config_module>
            LogFormat "%V %a %l %u %t \"%r\" %s %b \"%{Referer}i\" \"%{User-agent}i\"" vcommon
            CustomLog /var/log/httpd/vhost-access_log vcommon
        </IfModule>
    </VirtualHost>
    

    3) Also does it matter where the log file is placed? At the moment the OP suggested
    /var/log/httpd/vhost-access_log vcommon - I don't have an httpd folder in that location - could I add the log file to /usr/local/apache/logs/ ?

    Thanks
     
  6. WorkinOnIt

    WorkinOnIt Well-Known Member

    Joined:
    Aug 3, 2016
    Messages:
    49
    Likes Received:
    1
    Trophy Points:
    8
    Location:
    UK
    cPanel Access Level:
    Root Administrator
    Further to my above, I've copied the file /var/cpanel/templates/apache2_4/vhost.default and added the log lines to vhost.local and restarted Apache but I am not seeing any log files generated. Do I need to restart the server?

    Here's what I have done:

    1) copied vhost.defaul to vhost.local and at the footer, right after this:
    Code:
    [%- IF !added_include %]
      [%- IF apache_target_version %]
      [%- SET vhost_include_base = paths.dir_conf_userdata _ "/std/$apache_target_version/" %]
      [%- ELSE %]
      [%- SET vhost_include_base = paths.dir_conf_userdata _ '/std/2_4/' %]
      [%- END %]
      [%- domain = legacy_wildcard_safe(vhost.servername) %]
      # To customize this VirtualHost use an include file at the following location
      # Include "[% vhost_include_base %][% vhost.user %]/[% domain %]/*.conf"
    [%- END %]
    
    I have added this:

    Code:
        <IfModule log_config_module>
            LogFormat "%V %a %l %u %t \"%r\" %s %b \"%{Referer}i\" \"%{User-agent}i\"" vcommon
            CustomLog /usr/local/apache/logs/vhost-access_log vcommon
        </IfModule>
    </VirtualHost>
    
    I'm assuming the location of the log is not critical? The OP suggested /var/log/httpd/vhost-access_log vcommon - I don't have an httpd folder in that location.

    My httpd.conf file is located here: /usr/local/apache/conf/

    Should the log entry show in the virtual hosts httpd file?


    Thanks
     
    #6 WorkinOnIt, Aug 8, 2016
    Last edited: Aug 8, 2016
  7. WorkinOnIt

    WorkinOnIt Well-Known Member

    Joined:
    Aug 3, 2016
    Messages:
    49
    Likes Received:
    1
    Trophy Points:
    8
    Location:
    UK
    cPanel Access Level:
    Root Administrator
    @cPanelMichael

    Would the log work if I add directly to httpd virtual host block? Not having any luck with the above
    Thanks
     
  8. cPanelMichael

    cPanelMichael Forums Analyst
    Staff Member

    Joined:
    Apr 11, 2011
    Messages:
    30,723
    Likes Received:
    660
    Trophy Points:
    113
    cPanel Access Level:
    Root Administrator
    Could you verify the specific steps you would use with this method?

    Thank you.
     
Loading...

Share This Page