Setting up a custom log - or any other custom setting, really.

[Sinus Pi]

I'm trying to set up an exclusion from the logs, as:

CustomLog /path/to/log combined env=!dontlog

However, I'm completely confused as to how cPanel handles configuration.

If I modify the user domain file in /var/cpanel/userdata/<user>/<domain>, abusing the "format" entry to read "combined env=!dontlog", and run the rebuildhttpconf script, default settings are duplicated next to my entry in the file, and the resulting httpd.conf doesn't change at all.

I even tried to be clever and edit the main.local (from main.default) template file and modify the "CustomLog [% dir.target %] ......." lines to use a new [% dir.env %] entry that I'd placed in the user/domain file, but no go, that didn't do anything, either.

So how exactly does cPanel go about producing httpd.conf, if not by reading the user/domain file and running it through the templates..? Why is it resetting my settings to some defaults, and even trying to read settings back from the httpd.conf, putting the cart before the horse, the way I see it..?

 

[Sinus Pi]

All right, wait a sec. main.local seems to be handling httpd.conf-wide settings, while it's vhost.local that builds each vhost. How silly of me.

But, in the vhost.local template, I see:

CustomLog [% paths.dir_domlogs %]/[% wildcard_safe(vhost.log_servername) %] combined

and not a word of using the settings from the user/domain YAML file, which say:

customlog:
-
format: combined
target: /usr/local/apache/domlogs/zygorguides.com

I'm missing something here.

 

[cPanelMichael]

Hello @Sinus Pi,

First, I'd like to note that we added the following directives to WHM's Global Configuration interface (WHM >> Home >> Service Configuration >> Apache Configuration) in cPanel & WHM version 70:

• LogFormat (combined) — The Apache log file's format.
• LogFormat (common) — The Apache log file's format.

This will allow you to make changes to the default LogFormat entries directly from Web Host manager. That said, as far as customizing the templates manually, you'd also need to modify the ea4_main.default template as noted on the following thread:

SOLVED - Global Log

Thank you.

 

[Sinus Pi]

I may be misunderstanding something, but I don't recall using EA4 - in my WHM panel it says I'd need to switch from EA3 first, and even my EA3 seems like it's ready to "set up Apache" for me, while I have a live, production site here - I wouldn't want something to reset my configuration completely.

However, please clarify this for me, if possible. How do the options in userdata/ correlate to the vhost.local template? I see the template file reference some of the vhost variables, but ignore others. And the userdata file gets oddly regenerated, with options getting duplicated, while I'd naturally expect it to be the "source" set of configuration variables all the templates would obey. What's different here?

In the end, I managed to modify the vhost.local template and add "env=!dontlog" to the CustomLog entry mentioned above, but that added the change to all my vhosts. It's not bad, but had I wanted to change just one vhost, I'd need the template to obey the userdata variables, which it currently ignores. I suppose I could hack the line to read "CustomLog [% vhost.customlog.target %] [% vhost.customlog.format %] [% vhost.customlog.env %]" (if I'm guessing the template format correctly?), but would that do what I want, or am I completely in the dark?

All in all, I searched everywhere for any template or userdata handling documentation, but couldn't find any, aside from instructions on how to operate cPanel from the web interface. Is it not supposed to be modified on the template level, or what?

 

[cPanelMichael]

I may be misunderstanding something, but I don't recall using EA4 - in my WHM panel it says I'd need to switch from EA3 first, and even my EA3 seems like it's ready to "set up Apache" for me, while I have a live, production site here - I wouldn't want something to reset my configuration completely.

Is there anything in-particular that's preventing you from upgrading to EasyApache 4, other than the concerns about the upgrade breaking your websites or resetting your configuration files? The reason I ask is because EasyApache 3 is nearing end-of-life status as explained on the following thread:

EasyApache 3: It's been a long road, but it will be time to say goodbye soon.

Thus, all of our documentation, support, and new feature efforts are focused on EasyApache 4 as opposed to EasyApache 3. We document the migration process at:

The EasyApache 3 to EasyApache 4 Migration Process - EasyApache 4 - cPanel Documentation

In the end, I managed to modify the vhost.local template and add "env=!dontlog" to the CustomLog entry mentioned above, but that added the change to all my vhosts. It's not bad, but had I wanted to change just one vhost, I'd need the template to obey the userdata variables, which it currently ignores. I suppose I could hack the line to read "CustomLog [% vhost.customlog.target %] [% vhost.customlog.format %] [% vhost.customlog.env %]" (if I'm guessing the template format correctly?), but would that do what I want, or am I completely in the dark?

You could setup the custom template with rules that add specific lines for certain domain names. There's an example of how this would work on the following post:

Editing IfModule mod_include.c on virtualhost permanently

However, please clarify this for me, if possible. How do the options in userdata/ correlate to the vhost.local template? I see the template file reference some of the vhost variables, but ignore others. And the userdata file gets oddly regenerated, with options getting duplicated, while I'd naturally expect it to be the "source" set of configuration variables all the templates would obey. What's different here?

The following document provides some insight into how the Apache virtual hosts are built:

Advanced Apache Configuration - EasyApache 4 - cPanel Documentation

How it works
EasyApache 4 builds your global Apache configuration from the following components:

• Global templates — EasyApache 4 uses the global configuration templates in the /var/cpanel/templates/apache2_4/ directory to create your global Apache configuration.
• The modules, options, and extensions that EasyApache 4 installs for Apache.

EasyApache builds each virtualhost tag from the following components:

• Any custom templates in the /var/cpanel/templates/apache2_4/ directory
• Account information for each domain — When you make a configuration change to an account or domain in the cPanel or WHM interfaces, cPanel & WHM changes the content of the appropriate file in the /etc/apache2/conf.d/userdata/ directory.
• Additional templates for each domain — EasyApache includes the files in the appropriate /etc/apache2/conf.d/userdata/ directory for each domain to build the configuration of each virtualhost tag.

Note that this works a bit differently with EasyApache 3:

Advanced Apache Configuration - Easy Apache - cPanel Documentation

Thank you.

 

[Sinus Pi]

All right, that's some interesting reading - but it doesn't answer one thing: why my user datastores were modified when I used the WHM panels to rebuild the Apache configuration. I didn't modify httpd.conf directly, I was hoping it'd get updated according to my datastores, and yet - when I modified my user datastore to read:

customlog:

-

format: combined
target: /usr/local/apache/domlogs/mydomain.com
env: env=!dontlog​

and my template to use a [% customlog.env %] entry (even if that may not have been correct, but never mind that), the datastore was getting modified (by distiller, perhaps?) to:

customlog:

-

format: combined
target: /usr/local/apache/domlogs/mydomain.com​

-

format: combined
target: /usr/local/apache/domlogs/mydomain.com
env: env=!dontlog​

So is there a setting somewhere that would PROHIBIT any and all automated distillery (pun intended) from taking place?

Also: if I switch from EA3 to EA4, what all should I back up in case EA4 decides to be too helpful and resets things to defaults? httpd.conf, obviously, but what else? And, is there a setting that would make EA3/4 render into a DIFFERENT file than httpd.conf, for comparison and safe switching after inspection?

 

[cPanelMichael]

So is there a setting somewhere that would PROHIBIT any and all automated distillery (pun intended) from taking place?

I do see an internal case from the past for an issue where the datastore was updating correctly, but virtual hosts included duplicates for any directives that allow multiples. These types of cases are typically no longer processed due to the impending EOL of EasyApache 3. This is another reason why updating to EasyApache 4 is highly recommended.

Also: if I switch from EA3 to EA4, what all should I back up in case EA4 decides to be too helpful and resets things to defaults? httpd.conf, obviously, but what else? And, is there a setting that would make EA3/4 render into a DIFFERENT file than httpd.conf, for comparison and safe switching after inspection?

The EasyApache 4 conversion backs up the configuration files automatically so you can revert if something goes wrong. This is documented at:

How to Install or Uninstall EasyApache 4 - EasyApache 4 - cPanel Documentation

For example, if you want to go back to EA3 with your previous configuration, you'd run the following command:

/usr/local/cpanel/scripts/migrate_ea3_to_ea4 --revert --run

Thank you.

 

[Sinus Pi]

Having read through the doc and not having found anything scary, I took the plunge and migrated to EA4.

First thing it did was install suPHP for me, which I had to remove immediately, as all my file permissions were wrong. I'd've appreciated a warning during the installation.

Then it moved PHP from an Apache 2.0 module to FastCGI - something I'd've appreciated being warned about, too, as I was relying on certain PHP features of mod_php not so openly present in FastCGI versions.

But the worst thing is that now even localhost HTTP requests take well over 5 seconds. Even a simple "curl http://localhost/index.html" takes 5-6 seconds to fetch, even though that's a cPanel "page not found" error, since localhost isn't mapped to any reasonable vhost and sits at some unused defaults.

Any obvious hints as to what I should check..? It most certainly was running fine before the migration...

 

[cPanelMichael]

But the worst thing is that now even localhost HTTP requests take well over 5 seconds. Even a simple "curl http://localhost/index.html" takes 5-6 seconds to fetch, even though that's a cPanel "page not found" error, since localhost isn't mapped to any reasonable vhost and sits at some unused defaults.

Could you use the link in my signature to open a support ticket so we can take a closer look to see if there's anything obvious that's causing the reduced load time after the update to EasyApache 4? You can post the ticket number here once it's opened.

Thank you.