SOLVED [EA-9540] Centos 8 edge version mod_brotli apache crash

[vacancy]

Hi

While testing with cpanel v91.9999.90 edge version on Centos 8.2 I saw an error in mod_brotli module. When we install the mod_brotli module, there is a conflict with mod_security, resulting in apache stops.

[2020-10-23 00:44:45 +0300] info [xml-api] Initial configuration generation failed with the following message:
[2020-10-23 00:44:45 +0300] info [xml-api] The “/usr/sbin/httpd -DSSL -t -f /etc/apache2/conf/httpd.conf.work.b65c7e39.cfgcheck -C Include "/etc/apache2/conf.modules.d/*.conf"” command (process 12611) reported error number 1 when it ended.
[2020-10-23 00:44:45 +0300] info [xml-api] httpd: Syntax error in -C/-c directive: Syntax error on line 14 of /etc/apache2/conf.modules.d/800-mod_security2.conf: Cannot load modules/mod_security2.so into server: /lib64/libbrotlidec.so.1: undefined symbol: BrotliTransformDictionaryWord
[2020-10-23 00:44:45 +0300] info [xml-api] Rebuilding configuration without any local modifications.
[2020-10-23 00:44:45 +0300] info [xml-api] Failed to generate a syntactically correct Apache configuration.
[2020-10-23 00:44:45 +0300] info [xml-api] Bad configuration file located at /etc/apache2/conf/httpd.conf.work.b65c7e39
[2020-10-23 00:44:45 +0300] info [xml-api] Error:
[2020-10-23 00:44:45 +0300] info [xml-api] The “/usr/sbin/httpd -DSSL -t -f /etc/apache2/conf/httpd.conf.work.b65c7e39.cfgcheck -C Include "/etc/apache2/conf.modules.d/*.conf"” command (process 12613) reported error number 1 when it ended.
[2020-10-23 00:44:45 +0300] info [xml-api] httpd: Syntax error in -C/-c directive: Syntax error on line 14 of /etc/apache2/conf.modules.d/800-mod_security2.conf: Cannot load modules/mod_security2.so into server: /lib64/libbrotlidec.so.1: undefined symbol: BrotliTransformDictionaryWord

 Oct 23 00:44:50 140-82-58-66 systemd[1]: Failed to start Apache web server managed by cPanel EasyApache.
Oct 23 00:44:50 140-82-58-66 restartsrv_httpd[12679]: httpd: Syntax error on line 27 of /etc/apache2/conf/httpd.conf: Syntax error on line 14 of /etc/apache2/conf.modules.d/800-mod_security2.conf: Cannot load modules/mod_security2.so into server: /lib64/libbrotlidec.so.1: undefined symbol: BrotliTransformDictionaryWord

 

[cPanelLauren]

Hi @vacancy

Thank you for the information, the EasyApache team opened the following case for this issue: EA-9379 mod_security2.so undefined symbols /lib64/libbrotlidec.so.1 on CentOS 8

The case was just opened so not too much to report on it besides that it exists at this point. We'll update here when there is more information/any updates on the case.

 

[splaquet]

is there any update on this? i have the same issue. the only method of having httpd work is by uninstalling mod_security2, which obviously is not the most ideal method.

 

[splaquet]

actually... instead of removing mod_security2, i removed ea-apache24-mod_brotli. honestly... not sure why i hadn't thought about that the first time around

 

[cPRex]

I checked EA-9379 just now and this was marked as fixed on December 23. Are you seeing the same error at this time? If so, can you let me know the cPanel version you're using and what version of mod_brotli was installed?

 

[vacancy]

Yes, this problem has been fixed but it has not been updated here.

 

[cPRex]

@vacancy - this was one of the last posts Lauren made, so there was a week or so when I wasn't keeping track of the cases and getting them updated on the forums.

If anyone is still seeing this behavior with the latest edge build and updated packages please let me know!

 

[splaquet]

CLOUDLINUX 8.3
CPANEL v93.9999.114 [TEST BUILD]

Available in EasyApache:
mod_brotli 2.4.46-1.el8.cloudlinux

when installed, this is what happens:

# systemctl status httpd
● httpd.service - Apache web server managed by cPanel EasyApache
   Loaded: loaded (/usr/lib/systemd/system/httpd.service; enabled; vendor preset: disabled)
  Drop-In: /usr/lib/systemd/system/httpd.service.d
           └─alt-php80-fpm.conf
   Active: failed (Result: exit-code) since Wed 2021-01-20 23:34:25 EST; 12min ago
  Process: 596942 ExecStop=/usr/local/cpanel/scripts/restartsrv_httpd stop --no-verbose (code=exited, status=0/SUCCESS)
  Process: 597208 ExecStart=/usr/local/cpanel/scripts/restartsrv_httpd --no-verbose (code=exited, status=1/FAILURE)
Main PID: 119186 (code=exited, status=0/SUCCESS)

Jan 20 23:34:24 server.com systemd[1]: Starting Apache web server managed by cPanel EasyApache...
Jan 20 23:34:25 server.com restartsrv_httpd[597208]: httpd: Syntax error on line 27 of /etc/apache2/conf/httpd.conf: Syntax error on line 14 of /etc/apache2/conf.modules.d/800-mod_security2.conf: Cannot load modules/mod_security2.so into ser>
Jan 20 23:34:25 server.com systemd[1]: httpd.service: Control process exited, code=exited status=1
Jan 20 23:34:25 server.com systemd[1]: httpd.service: Failed with result 'exit-code'.
Jan 20 23:34:25 server.com systemd[1]: Failed to start Apache web server managed by cPanel EasyApache.

 

[cPRex]

Thanks! Let me do some testing with this and I'll get back to you soon.

 

[splaquet]

I just noticed something on my post. With CloudLinux and all, I hadn’t installed any PHP*-FPM mods.

I’ll double check, but by default, I don’t believe that alt-php8-fpm is installed on CL machines.

 

[cPRex]

I created a CloudLinux 8 server and installed cPanel on top of that. It came with the following packages by default:

# rpm -qa | grep fpm
ea-php56-php-fpm-5.6.40-15.el8.cloudlinux.6.x86_64
ea-php55-php-fpm-5.5.38-53.el8.cloudlinux.6.x86_64
ea-php70-php-fpm-7.0.33-16.el8.cloudlinux.6.x86_64

I then installed PHP 8.0 and I see that did install the FPM package as part of that work:

# rpm -qa | grep fpm
ea-php56-php-fpm-5.6.40-15.el8.cloudlinux.6.x86_64
ea-php55-php-fpm-5.5.38-53.el8.cloudlinux.6.x86_64
ea-php80-php-fpm-8.0.0-2.el8.cloudlinux.x86_64
ea-php70-php-fpm-7.0.33-16.el8.cloudlinux.6.x86_64

I also see the following brotli packages present on the machine:

# rpm -qa | grep brotli
brotli-1.0.6-2.el8.x86_64
ea-brotli-1.0.2-4.el8.cloudlinux.2.x86_64

I then installed the mod_brotli package and confirmed Apache wasn't able to start. I removed the mod_security package from the system, and then Apache started normally, so it seems there has been a regression on this case. I've opened EA-9540 for our team to check this again, and I'll keep this page updated as I get more details! Thanks so much for pointing this out!

 

[cPRex]

@splaquet - the plot thickens!

It looks like this isn't a cPanel issue, but CloudLinux being slightly behind on their packages in relation to our Edge tier. I double-checked this on a CentOS 8 system and things are working well, so we've converted this to an upstream tracking case on our end. Once CloudLinux picks up the changes, I would expect this to work, but I'm going to re-mark this one as solved for now.

 

[cPRex]

Another update! CloudLinux is aware and plans to release a fix on January 25.

 

[splaquet]

thank you kindly sir

 

[cPRex]

You're welcome!!!

 

[splaquet]

i was on this page, checking to see if there was any more dialog. since it was after CL's proposed release date for working mod, i tried to install mod_brotli again.

 ea-apache24-mod_brotli.x86_64 1:2.4.46-1.el8.cloudlinux @cl-ea4-testing

i installed it via easyapache... and here's the results. doesn't look like it's working yet

# systemctl status httpd
● httpd.service - Apache web server managed by cPanel EasyApache
   Loaded: loaded (/usr/lib/systemd/system/httpd.service; enabled; vendor preset: disabled)
  Drop-In: /usr/lib/systemd/system/httpd.service.d
           └─alt-php80-fpm.conf
   Active: failed (Result: exit-code) since Sun 2021-02-07 10:03:28 EST; 6s ago
  Process: 1146102 ExecStop=/usr/local/cpanel/scripts/restartsrv_httpd stop --no-verbose (code=exited, status=0/SUCCESS)
  Process: 1147690 ExecStart=/usr/local/cpanel/scripts/restartsrv_httpd --no-verbose (code=exited, status=1/FAILURE)
 Main PID: 1830953 (code=exited, status=0/SUCCESS)

Feb 07 10:03:27 systemd[1]: Starting Apache web server managed by cPanel EasyApache...
Feb 07 10:03:28 restartsrv_httpd[1147690]: httpd: Syntax error on line 27 of /etc/apache2/conf/httpd.conf: Syntax error on line 14 of /etc/apache2/conf.modules.d/800-mod_security2.conf: Cannot l>
Feb 07 10:03:28 systemd[1]: httpd.service: Control process exited, code=exited status=1
Feb 07 10:03:28 systemd[1]: httpd.service: Failed with result 'exit-code'.
Feb 07 10:03:28 systemd[1]: Failed to start Apache web server managed by cPanel EasyApache.

Feb 07 10:03:28 restartsrv_httpd[1147690]: httpd: Syntax error on line 27 of /etc/apache2/conf/httpd.conf: Syntax error on line 14 of /etc/apache2/conf.modules.d/800-mod_security2.conf: Cannot load modules/mod_security2.so into server: /lib64/libbrotlidec.so.1: undefined symbol: BrotliTransformDictionaryWord
Feb 07 10:03:28 systemd[1]: httpd.service: Control process exited, code=exited status=1
Feb 07 10:03:28 systemd[1]: httpd.service: Failed with result 'exit-code'.
Feb 07 10:03:28 systemd[1]: Failed to start Apache web server managed by cPanel EasyApache.

 

[vacancy]

Cloudlinux makes its own rpm.

They are following the updates of cpanel a little behind. Have you tried to post a ticket to cloudlinux?