In Progress UPS-606 - nginx: [emerg] invalid value "0" in "http2" directive, it must be "on" or "off" in /etc/nginx/conf.d/default.conf:9

[danielg1b03RfJv]

Hello,

Aug 9 2023
# server1
nginx failed to start
nginx -t
nginx: [emerg] invalid value "0" in "http2" directive, it must be "on" or "off" in /etc/nginx/conf.d/default.conf:9
nginx: configuration file /etc/nginx/nginx.conf test failed

cpanel 112.0.5 on CloudLinux v8.8.0 STANDARD
ea-nginx-1.25.1-2.el8.cloudlinux.x86_64

# server2
cpanel 112.0.5 on CloudLinux v8.8.0 STANDARD
ea-nginx-1.25.1-1.el8.cloudlinux.1.x86_64
nginx config is ok

# fix
copy /usr/local/cpanel/scripts/ea-nginx from server2 to server1
whm -> NGINX® Manager -> Rebuild Configuration
nginx start ok

# diff
diff ea-nginx-server1 ea-nginx-server2
1012c1012
< my $http2 = _wants_http2();
---
> my $http2 = _wants_http2() ? 'on' : 'off';
1978c1978
< http2 => $global_config_data->{http2},
---
> http2 => $global_config_data->{http2} ? 'on' : 'off',


Kind regards,

 

[cPRex]

Hey there! I created a test server to check that file, and the line just looks like this:

    http2 off;

with no other characters. Is that working for you?

 

[Wil Hatfield]

Hey this seems to be a real issue that just started in like the last 48 hours. Something must have updated somewhere because this came on suddenly. I think cPanel is writing the http2 0; or http2 1; to the /etc/nginx/conf.d/default.conf file and regenerating the user files all with the wrong setting. Then when nginx tries to restart well that kills that apache and nginx both die of course. I can manually reset it to the on or off of course but how do I regenerated all those user files in /etc/nginx/conf.d/users/

 

[cPRex]

Our team did find an issue with the CloudLinux ea-nginx package and we've created case UPS-606 to have our team communicate with CloudLinux about this problem.

 

[Wil Hatfield]

/usr/local/cpanel/scripts/ea-nginx in the source guys. You are turning the 1 or 0 on based on a previous bool and not changing it to "on" or "off".

Previously:

sub _write_global_default {
my $reuseport = Cpanel::Kernel::system_is_at_least('3.9.0') ? 1 : 0;
my $ipv6 = _has_ipv6();
my $http2 = _wants_http2() ? 'on' : 'off';

Currently:

sub _write_global_default {
my $reuseport = Cpanel::Kernel::system_is_at_least('3.9.0') ? 1 : 0;
my $ipv6 = _has_ipv6();
my $http2 = _wants_http2() ;

 

[Wil Hatfield]

Our team did find an issue with the CloudLinux ea-nginx package and we've created case UPS-606 to have our team communicate with CloudLinux about this problem.

So that in sub _write_global_default above came from the CloudLinux version? Wouldn't be a bad idea to include the nginx user files regeneration into rebuildhttpdconf if nginx is enabled. That way in a case like this we could just modify the default.conf and run rebuildhttpdconf and all would be fixed. Eliminates the dependency we have on the cloudlinux distro.

In the meantime I'll try a sed find replace and see if I can get it going temporarily.

 

[cPRex]

Ultimately this will get fixed, but you're welcome to do whatever workaround you like for the time being.

 

[Wil Hatfield]

Of course it has to be done after any kind of upgrade or build, but this is a good bandaid until they fix the distro.

cd /etc/nginx/conf.d
find ./ -type f -exec sed -i 's/http2 0/http2 off/g' {} \;
find ./ -type f -exec sed -i 's/http2 1/http2 on/g' {} \;
/bin/systemctl restart nginx.service

And changing the
my $http2 = _wants_http2();
in /usr/local/cpanel/scripts/ea-nginx to
my $http2 = _wants_http2() ? 'on' : 'off;

 

[Wil Hatfield]

Went ahead and disabled it cause it keep coming back for some reason. I can't babysit it so disabling NGINX is required for now.

 

[MHFraser]

so as a FYI all my websites just went down because of this.
didn't have time to debug but clients requests were getting redirected to the cpanel hostname and failing with a cert error. suspect nginx user confs weren't loaded?

so is uninstall nginx our only option?




Unable to ensure a valid nginx configuration:

nginx: [emerg] invalid value "0" in "http2" directive, it must be "on" or "off" in /etc/nginx/conf.d/default.conf:9
nginx: configuration file /etc/nginx/nginx.conf test failed

 

[MHFraser]

going back through the update history - the update was listed as fully successful despite nginx having an emergency
(nginx error levels: debug, info, notice, warn, error, crit, alert, emerg)

has something slipped through the cracks here?


# dnf history info 130
This system is receiving updates from CloudLinux Network server.
Transaction ID : 130
Begin time : Wed 09 Aug 2023 23:30:23 AEST
Begin rpmdb : 2346:079a175f06666c000b6074aa16aff5f97de86692
End time : Wed 09 Aug 2023 23:30:28 AEST (5 seconds)
End rpmdb : 2346:538fd250288647b0f53115536b54e1875554a6fa
User : root <root>
Return-Code : Success

...

178 Starting nginx …
179 Unable to ensure a valid nginx configuration:
180
181 nginx: [emerg] invalid value "0" in "http2" directive, it must be "on" or "off" in /etc/nginx/conf.d/default.conf:9
182 nginx: configuration file /etc/nginx/nginx.conf test failed
183
184
185 You will need to manually resolve this issue.

 

[cPRex]

Update - CloudLinux has released a fix to their testing repository here:

https://support.cpanel.net/hc/en-us/articles/16716330439575

If you don't have that repository enabled, you can apply the fix with the following command:

yum update --enablerepo=cl-ea4-testing ea-nginx

 

[Luis_Costa]

The above solution wont work if you only have imunify360

 

[cPRex]

@Luis_Costa - I'm not entirely sure how Imunify360 would be related to the ea-nginx tools. Can you provide a bit of context for the compatibility issues you're seeing?

 

[Esquio]

yum update --enablerepo=cl-ea4-testing ea-nginx

This don't work because it sais that is an unknown repository.

How can solve this problem? I have all websites down.

 

[cPRex]

At this point there is not a fix available. The only thing we are recommending to people in this situation is to switch to Apache and wait for CloudLinux to release a fix as there isn't one available from them just yet.

 

[Wil Hatfield]

At this point there is not a fix available. The only thing we are recommending to people in this situation is to switch to Apache and wait for CloudLinux to release a fix as there isn't one available from them just yet.

After going back to Apache we have then seen a lot of 403 Forbidden and http2 errors of various sorts. Is there something the Nginx Http2 setup could have left behind? I checked and the ea-nginx and the nginx-http2 and neither is supposed to be currently installed.

 

[cPRex]

@Wil Hatfield - I wouldn't expect it to leave anything in place, no, since nginx is designed to be a drop-in replacement for Apache on cPanel machines, with no configuration changes required. If you're seeing that behavior, it's likely time for a ticket so we can take a look.

 

[yanirda]

the command

yum update --enablerepo=cl-ea4-testing ea-nginx

fix the websites , they up, what that command do? and why it on testing? there isnt fix yet?

 

[cPRex]

@yanirda - that workaround will work *if* your system uses CloudLinux. For customer that use Imunify360, but do not have CloudLinux, that isn't a recognized repository, as @Esquio found above. CloudLinux has still not issued a global fix that isn't in the testing repository just yet.