The Community Forums

Interact with an entire community of cPanel & WHM users!
  1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

apache master/custom/global access log

Discussion in 'EasyApache' started by SergeiF, Jun 29, 2014.

  1. SergeiF

    SergeiF Registered

    Joined:
    Jun 29, 2014
    Messages:
    2
    Likes Received:
    0
    Trophy Points:
    1
    cPanel Access Level:
    Root Administrator
    Recently I had to rebuild a WHM server due to filesystem corruption. I was able to recover most of the functionality/customisation (the cpanel backup is bloody useless btw, will never ever rely on it again).
    The version is WHM 11.44.0 (build 18).

    One problem is I cannot replicate one customisation (even though it works on other WHM machines that we run).
    The customisation in question is combined apache access log.

    Right now it is real pain to monitor abuse on the machine, due to no central/master apache log. I cannot tail domlogs due to there are thousands of accounts on each machine.

    The way I was able to have combined log is by creating/editing /var/cpanel/templates/apache2/main.local

    and adding the following:

    Code:
    LogFormat "%V %a %l %u %t \"%r\" %s %b \"%{Referer}i\" \"%{User-agent}i\"" vcommon
    CustomLog /var/log/httpd/vhost-access_log vcommon
    just after this:
    Code:
        <IfModule [% main.mainifmodulelogconfigmoduleifmodulelogiomodule.item.ifmodule %]>
    [% FOREACH dir IN main.ifmodulelogconfigmodule.ifmodulelogiomodule.logformat.items -%]
            LogFormat [% dir.logformat %]
    [% END -%]
    
        </IfModule>
    
    </IfModule>
    and before this:
    Code:
    <IfModule [% main.mainifmodulealiasmodule.item.ifmodule %]>
    [% FOREACH dir IN main.ifmodulealiasmodule.scriptalias.items -%]
        ScriptAlias [% dir.url %] [% dir.path %]
    [% END -%]
    
    </IfModule>
    Basically I have replicated what we have on other multiple machines (the machine that the combined log works is running WHM 11.38.2 (build 23) ).
    It does not work on new machine. The only entires that are being logged are those that are not in virtual hosts....

    Basically what I am after is a central/master/combined apache access log, so I can tail it and see which domain get what requests from which ip.

    I have searched this forum for this, and documentation, there is a lot of misleading information, most of the forum posts are simply unanswered or answered incorrectly (eg I am not after /usr/local/apache/domlogs/, nor I am after /usr/local/apache/logs/access_log). I don't understand how people get away without having one? Do they not care about abuse? This log is a must when running shared hosting. It should be on by default.

    So the question is: what is the correct way of setting up the combined apache log for all vhosts?
     
  2. cPanelMichael

    cPanelMichael Forums Analyst
    Staff Member

    Joined:
    Apr 11, 2011
    Messages:
    30,678
    Likes Received:
    654
    Trophy Points:
    113
    cPanel Access Level:
    Root Administrator
    Hello :)

    To clarify, is the goal here to have one log file with the data from all of your Apache domlogs combined? If so, you could develop a bash script that combines all of the Apache domlogs into one single log file after the initial logging. Feel free to clarify or let us know if that's not what you are after.

    Thank you.
     
  3. SergeiF

    SergeiF Registered

    Joined:
    Jun 29, 2014
    Messages:
    2
    Likes Received:
    0
    Trophy Points:
    1
    cPanel Access Level:
    Root Administrator
    Hi, The idea is to have a "live" log of all the domains combined. The bash script will not do, as it is much more efficient to tail the log (and perhaps grep for stuff of interest). Sometimes there is no time before server runs out of memory due to DoS.

    In anyway, while my message was being aproved by moderator I figured out the answer:

    The correct place to put the log line is in /var/cpanel/templates/apache2/vhost.local (cp from vhost.default):

    Code:
    ...........
    
    LogFormat "%V %a %l %u %t \"%r\" %s %b \"%{Referer}i\" \"%{User-agent}i\"" vcommon
    CustomLog /var/log/httpd/vhost-access_log vcommon
    
    </VirtualHost> 
    
    It can be placed right before closing </VirtualHost> tag in the file. The ssl_vhost.local should have similar done.
    Don't forget to configure logrotate to rotate the logs.

    Now you can tail the access log for all sites to figure which site or IP is affecting your server.
    Very easy to spot for example wordpress or joomla bruteforces and deal with them (be that botnet or a single machine). This log can also be used as signle watch point for stuff like fail2ban (to deal with bruteforces above) or similar.

    I hope this will be helpfull for someone in the future.
    I wander sometimes why this is not done by default. Logs are very important for security purposes.
    I am not the first one to ask for it:
    http://forums.cpanel.net/f5/master-apache-access-log-352032.html
    http://forums.cpanel.net/f5/global-apache-access-log-164866.html
    http://forums.cpanel.net/f5/apache-access-log-file-all-accounts-150721.html
    http://forums.cpanel.net/f5/there-single-apache-log-all-access-100661.html

    All these were unaswered.
    Another problem I found is that the WHM will delete all the access logs without archival, once the stats were processed by default. We have disabled that anti-feature.
     
  4. cPanelMichael

    cPanelMichael Forums Analyst
    Staff Member

    Joined:
    Apr 11, 2011
    Messages:
    30,678
    Likes Received:
    654
    Trophy Points:
    113
    cPanel Access Level:
    Root Administrator
    Hello :)

    I am happy to see you were able to resolve the issue. Thank you for updating this thread with the solution. Note that you are welcome to submit a feature request to have this behavior utilized by default:

    Submit A Feature Request

    Thank you.
     
  5. acoma

    acoma Registered

    Joined:
    Oct 6, 2014
    Messages:
    1
    Likes Received:
    0
    Trophy Points:
    1
    Location:
    Gold Coast, Queensland, Australia
    cPanel Access Level:
    DataCenter Provider
    You Sir, are a champion!
     
  6. cwdd

    cwdd Registered

    Joined:
    Aug 5, 2013
    Messages:
    1
    Likes Received:
    0
    Trophy Points:
    1
    cPanel Access Level:
    Root Administrator
    isn't /usr/local/apache/logs/access_log already a combined log of requests? I was happy to find these:

    Code:
    	/usr/local/apache/logs/fpcgisock
    	/usr/local/apache/logs/access_log
    	/usr/local/apache/logs/error_log
    	/usr/local/apache/logs/httpd.pid
    	/usr/local/apache/logs/suexec_log
    	/usr/local/apache/logs/ssl_scache.dir
    	/usr/local/apache/logs/ssl_scache.pag
    	/usr/local/apache/logs/suexec_log.offset
    	/usr/local/apache/logs/suphp_log
    
    It may also be worth looking at piped logging and ALSO this site discussing optimization

    Be sure to check out the cPanel config option for piped logging, too

    ed6ec3650e.png
     
Loading...

Share This Page