Please whitelist cPanel in your adblocker so that you’re able to see our version release promotions, thanks!

The Community Forums

Interact with an entire community of cPanel & WHM users!

Unable to find (or modify) log format in Apache configuration.

Discussion in 'General Discussion' started by Sinus Pi, Nov 28, 2018.

  1. Sinus Pi

    Sinus Pi Member

    Joined:
    Apr 19, 2018
    Messages:
    10
    Likes Received:
    2
    Trophy Points:
    3
    Location:
    Poland
    cPanel Access Level:
    Root Administrator
    I'm trying to find - and modify - the "combined" log format Apache uses. And any other log-related settings, really.

    My /etc/apache2/conf/httpd.conf file has the following log format directive:
    Code:
    <IfModule mod_log_config.c>
    ...
        LogFormat "%v:%p %h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined
    </IfModule>
    
    This section seems to be properly produced by the ea4_main.default template, nothing strange about it. But, my /usr/local/apache/domlogs/domain.com log file is formatted differently:
    Code:
    209.299.199.299 - - [25/Nov/2018:03:10:04 -0500] "GET /url/somefile" 200 - "-" "UserAgent"
    
    It clearly isn't using the format which starts with "%v:%p", a hostname and a port.

    Then, my cpanel/userdata/username/domain.com file has the following lines:
    Code:
    customlog:
      -
        format: "\"%{%s}t %I .\\n%{%s}t %O .\""
        target: /usr/local/apache/domlogs/domain.com-bytes_log
      -
        format: combined
        target: /usr/local/apache/domlogs/domain.com
    
    This seems to indicate the "combined" format for the domain.com log, but as shown above, "combined" must've been overridden... somewhere. Also, I find no trace of the domlogs/domain.com file compiled into httpd.conf anywhere that I could find. Not in httpd.conf, not in any virtualhost includes (I have some custom includes, but they're not created with a template).

    So - where does Apache get the log formats, and how do I modify them? Or, more widely: how EXACTLY are the Apache configuration files built? So far I was sure it's cpanel/templates + cpanel/userdata = httpd.conf, but apparently there's more to it.

    My curiosity is also piqued by some httpd.conf.work.QiHpbC20 files in /etc/apache2/conf, there ARE domain.com-related entries in them, but they aren't actively used, now are they?
     
    #1 Sinus Pi, Nov 28, 2018
    Last edited: Nov 28, 2018
  2. cPanelMichael

    cPanelMichael Technical Support Community Manager Staff Member

    Joined:
    Apr 11, 2011
    Messages:
    47,590
    Likes Received:
    2,186
    Trophy Points:
    363
    cPanel Access Level:
    Root Administrator
    Twitter:
    Hello @Sinus Pi,

    You can modify the following options in WHM >> Service Configuration >> Apache Configuration >> Global Configuration to change the default LogFormat configuration:

    LogFormat (combined)
    LogFormat (common)

    The discussions on the following threads offer some more insight into how to make changes to the default Apache LogFormat configuration via the command line:

    SOLVED - Global Log
    Apache override templates installed warning
    mod_cloudflare vs mod_remoteip
    Apache log format issue with EasyApache 4 (apache 2.4), Varnish and mod_remoteip

    The template files in /var/cpanel/templates/apache2_4/ determine how the LogFormat entries are populated when /scripts/rebuildhttpdconf builds the /etc/apache2/conf/httpd.conf file. Here's the default LogFormat configuration in /var/cpanel/templates/apache2_4/main.default in cPanel & WHM version 76:

    Code:
    [% IF enable_piped_logs -%]
    <IfModule mod_log_config.c>
        LogFormat "%v:%p [% IF main.exists('logformat_combined') %][% main.logformat_combined.item.logformat_combined %][% ELSE %]%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"[% END %]" combinedvhost
        <IfModule logio_module>
            LogFormat "%v %{%s}t %I .\n%v %{%s}t %O ." bytesvhost
        </IfModule>
        LogFormat "%v:%p [% IF main.exists('logformat_combined') %][% main.logformat_combined.item.logformat_combined %][% ELSE %]%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"[% END %]" combined
        LogFormat "%v:%p [% IF main.exists('logformat_common') %][% main.logformat_common.item.logformat_common %][% ELSE %]%h %l %u %t \"%r\" %>s %b[% END %]" common
        LogFormat "%{Referer}i -> %U" referer
        LogFormat "%{User-agent}i" agent
    
        [%- SET splitlogsextra = '' -%]
        [%- TRY -%]
            [%- SET splitlogsconf = load_conf('/var/cpanel/conf/splitlogs.conf') -%]
            [%- IF splitlogsconf.maxopen.length %][% splitlogsextra = splitlogsextra _ " --maxopen=" _ splitlogsconf.maxopen %][% END -%]
            [%- IF splitlogsconf.buffer.length %][% splitlogsextra = splitlogsextra _ " --buffer=" _ splitlogsconf.buffer %][% END -%]
            [%- IF splitlogsconf.sslport.length %]
                [% splitlogsextra = splitlogsextra _ " --sslport=" _ splitlogsconf.sslport %]
            [%- ELSIF configured.main_port_ssl.length && configured.main_port_ssl != '443' %]
                [% splitlogsextra = splitlogsextra _ " --sslport=" _ configured.main_port_ssl %]
            [% END -%]
        [%- CATCH -%]
            [%# noop but catch is required or it dies %]
        [%- END -%]
        <IfModule logio_module>
            CustomLog "|/usr/local/cpanel/bin/splitlogs --dir=[% paths.dir_domlogs %] --main=[% wildcard_safe(servername) %] --suffix=-bytes_log[% splitlogsextra %]" bytesvhost
        </IfModule>
        CustomLog "|/usr/local/cpanel/bin/splitlogs --dir=[% paths.dir_domlogs %] --main=[% wildcard_safe(servername) %] --mainout=[% paths.file_access_log %][% splitlogsextra %]" combinedvhost
    </IfModule>
    [% ELSE %]
    <IfModule log_config_module>
    [% IF main.exists('logformat_combined') || main.exists('logformat_common') %]
        LogFormat "[% IF main.exists('logformat_combined') %][% main.logformat_combined.item.logformat_combined %][% ELSE %]%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"[% END %]" combined
        LogFormat "[% IF main.exists('logformat_common') %][% main.logformat_common.item.logformat_common %][% ELSE %]%h %l %u %t \"%r\" %>s %b[% END %]" common
        LogFormat "%{Referer}i -> %U" referer
        LogFormat "%{User-agent}i" agent
    [%  ELSIF main.ifmodulemodlogconfigc.logformat.items.length -%]
    [%      FOREACH dir IN main.ifmodulemodlogconfigc.logformat.items -%]
        LogFormat [% dir.logformat %]
    [%      END -%]
    [%  ELSE -%]
        LogFormat "%{Referer}i -> %U" referer
        LogFormat "%{User-agent}i" agent
        # NOTE: "combined" and "common" are required by WHM
        LogFormat "%h %l %u %t \"%r\" %>s %b" common
        LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined
    [%  END -%]
    
        # access_log format can be set in WHM under 'Basic WebHost Manager® Setup'
    [%  IF main.ifmodulemodlogconfigc.customlog.items.length -%]
    [%      FOREACH dir IN main.ifmodulemodlogconfigc.customlog.items -%]
        CustomLog [% dir.target %] [% dir.format %]
    [%      END -%]
    [%  ELSE -%]
        CustomLog logs/access_log [% logstyle %]
    [%  END -%]
    </IfModule>
    [% END %]
    The /var/cpanel/cpanel.config file is used to determine if pipped logging (WHM >> Apache Configuration >> Piped Log Configuration) is enabled or disabled:

    Code:
    grep piped /var/cpanel/cpanel.config
    enable_piped_logs=1
    That's the correct format and matches what I see on a test system with pipped logging enabled. EX:

    Code:
    1.2.3.4 - - [29/Nov/2018:11:00:43 -0600] "GET /favicon.ico HTTP/1.1" 404 - "-" "$USERAGENT-HERE"
    If you're using mod_remoteip, then you should be able to simply browse to WHM >> Service Configuration >> Apache Configuration >> Global Configuration and change "%h" to "%a" on the following options for the desired effect of logging proxied traffic:

    LogFormat (combined)
    LogFormat (common)

    Let me know if this helps.

    Thank you.
     
    Stop hovering to collapse... Click to collapse... Hover to expand... Click to expand...
  3. Sinus Pi

    Sinus Pi Member

    Joined:
    Apr 19, 2018
    Messages:
    10
    Likes Received:
    2
    Trophy Points:
    3
    Location:
    Poland
    cPanel Access Level:
    Root Administrator
    Thanks for taking the time to respond! Indeed modifying the LogFormat "combined" setting in WHM worked as intended.

    However, in case someone else has the same conundrum, let me state what exactly I learned.

    These settings in Apache's httpd.conf file,

    <IfModule mod_log_config.c> (yes, mod_log_config is loaded, I checked with httpd -M)
    LogFormat "%v:%p %h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combinedvhost
    ...
    LogFormat "%v:%p %h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined
    LogFormat "%v:%p %h %l %u %t \"%r\" %>s %b" common
    ...
    </IfModule>

    ... ARE where the log formats are set, although with "%v:%p " prepended for the splitlogs utility to use. I made the mistake of looking for where my setting would be present literally, "%h %l %u..." - the "%v:%p" misled me into thinking this was NOT the correct place to set it. Instead, should one ever end up modifying the LogFormat's generation via ea4_main.* templates, one must remember to make sure "%v:%p " is present, otherwise splitlogs won't work.

    Cheers!
     
    cPanelMichael likes this.
  4. cPanelMichael

    cPanelMichael Technical Support Community Manager Staff Member

    Joined:
    Apr 11, 2011
    Messages:
    47,590
    Likes Received:
    2,186
    Trophy Points:
    363
    cPanel Access Level:
    Root Administrator
    Twitter:
    Hello @Sinus Pi,

    Thanks for sharing the outcome!

    We recommend using WHM >> Service Configuration >> Apache Configuration >> Global Configuration to change the default LogFormat configuration as opposed to creating a local EA4 template for that purpose.

    I've created an internal request to create a tutorial on how to enable mod_remoteip on cPanel & WHM and made a note to include the caveat noted in your quote.

    Thank you.
     
    Stop hovering to collapse... Click to collapse... Hover to expand... Click to expand...
  5. Sinus Pi

    Sinus Pi Member

    Joined:
    Apr 19, 2018
    Messages:
    10
    Likes Received:
    2
    Trophy Points:
    3
    Location:
    Poland
    cPanel Access Level:
    Root Administrator
    And rightfully you do recommend using WHM's configuration, but should one try to create a specific log format for just one virtualhost, by adding a new variable in cpanel/userdata/<vhostname> and modifying their EA4 templates to use it, they'll need to understand how the %v:%p part is used.
     
    cPanelMichael likes this.
Loading...

Share This Page

  1. This site uses cookies to help personalise content, tailor your experience and to keep you logged in if you register.
    By continuing to use this site, you are consenting to our use of cookies.
    Dismiss Notice