SOLVED "File not found." instead of custom 404 file with php-fpm enabled

[PeteS]

I found that badfile.php produces "File not found." instead using the 404 error file designated in .htaccess when php-fpm is enabled. I found the solution here:

SOLVED - Getting file not found vs 404 with PHPFPM

WHM Home » Service Configuration » Apache Configuration » Include Editor
Under the "Pre Main Include" section, add: ProxyErrorOverride on

But that breaks webmail.domain.tld giving this error:

"Access Denied
This server could not verify that you are authorized to access the document requested. Either you supplied the wrong credentials (e.g., bad password), or your browser doesn't understand how to supply the credentials required.

Additionally, a 401 Unauthorized error was encountered while trying to use an ErrorDocument to handle the request."

Anyone get this sorted...?

 

[cPanelMichael]

Hello @PeteS,

I did find one report where a customer successfully used an .htaccess rule like the one below for the individual accounts as an alternative:

RewriteCond %{DOCUMENT_ROOT}%{REQUEST_URI} !-f
RewriteRule ^.+\.php$ /404.shtml

Thank you.

 

[PeteS]

Thanks. It just seems like there should be as more universal solution, or that the above solution shouldn't break webmail. I'll post back if I find anything.

 

[cPanelMichael]

Thanks. It just seems like there should be as more universal solution, or that the above solution shouldn't break webmail. I'll post back if I find anything.

Hello Pete,

One possible solution here is to enable ProxyErrorOverride under the "Pre Main Include" section in "WHM Home » Service Configuration » Apache Configuration » Include Editor", but with exceptions that exclude proxy subdomains. Here's a third-party URL that may help:

How to add exceptions to apache reverse proxy rules

Thank you.

 

[PeteS]

In glancing at that page I don't see the connection. I'll have to give this further consideration after I get back in the office week after next...

 

[Rajeeva Lochana]

Add it to the post virtualhost include

 

[PeteS]

@rajeevacj Thanks, that works. It solves the problem without breaking webmail.domain.tld.

 

[SamuelM]

Hello @PeteS,

I am happy to see that rajeevacj's suggestion worked for you! I will mark this thread as solved.

Best regards

 

[PeteS]

Hello @PeteS,

I am happy to see that rajeevacj's suggestion worked for you! I will mark this thread as solved.

Best regards

It would be helpful to update the solution here: SOLVED - Getting file not found vs 404 with PHPFPM so that this problem is avoided.

-Pete

 

[Rajeeva Lochana]

What do you mean by the above, do you say that the answer should also be answered there, in that thread? I also found an old thread with the same answer by @monkey64 SOLVED - Custom 404 shows “File not found” for php files instead of redirecting

 

[PeteS]

What do you mean by the above, do you say that the answer should also be answered there, in that thread? I also found an old thread with the same answer by @monkey64 SOLVED - Custom 404 shows “File not found” for php files instead of redirecting

I mean that the solution there is not entirely correct (it breaks Webmail), and it can be improved by indicating that the entry should go in post virtualhost include instead.

@cPanelMichael and @cPSamuel It is frustrating that a better answer was not provided here by cPanel immediately, since it is a known problem with a solution, and is documented in other threads. (Yes, I searched for it, but was not able to find them until I knew the solution already...) It seems to me that this is a configuration bug that should be rolled into the next version, right? Why distribute something that requires us to make a special configuration to get the "normal/expected" behavior from Apache when PHPFPM in the first place?

-Pete

 

[Rajeeva Lochana]

I'll answer the same there. Alright. Maybe webmail breaks because it also uses PHP and Apache maybe?

 

[Rajeeva Lochana]

And also, I am not sure about cPanel's way of using php-fpm in files, e.g using the proxypass directive

 

[PeteS]

Note: Today a client called and said webmail.example.com was giving the error again. I checked multiple other domains (all configured the same for FPM, etc.) and they all worked. It was only affecting one domain (that I could find).

I removed the ProxyErrorOverride on directive in the section post virtualhost include of WHM Home » Service Configuration » Apache Configuration » Include Editor and then it worked. I replaced the directive and it still worked.

I suspect the Apache restart was the actual solution.

-Pete

 

[PeteS]

Update: Today most (all?) domains are giving the webmail error. I restarted Apache and can confirm that it is not the solution. I removed the line from the Include Editor and all worked. I replaced it again and some do work (that didn't), others don't.

On further testing I found the sites that worked would then stop working after a while. I found that if I clear the browser history for that site ("forget about this site" in FF history, clear all cache in Chrome) that it again works. ("Either you supplied the wrong credentials (e.g., bad password), or your browser doesn't understand how to supply the credentials required." was my clue.) I'm not sure where to proceed on this aspect of it, because I don't know if it's an issue on the client side or the server side, but it looks like credentials are getting out of sync. Will report back if I find out more...

Both these issues are reported by many. I am torn between which problem to address: the webmail subdomain issues caused by the PHP-FPM fix, or the PHP-FPM 404 issue directly. It really seems like either/both should be addressed by cPanel.

@cPanelMichael Can cPanel replicate these behaviors on a test server?

-Pete

 

[Rajeeva Lochana]

@PeteS I think that the way that Apache connects to PHP-FPM in cPanel is the problem here. I advise cPanel to use SetHandler for PHP-FPM. I am serious here.

ProxyPass.. or so is sometimes not reliable.

I think,

<FilesMatch ...>
    SetHandler  "proxy:fcgi://localhost:9000"
</FilesMatch>

is the best way of connecting to PHP-FPM from Apache.

Sorry for my late reply. I was offline due to some network issues for like 20 days.

 

[PeteS]

Update: After doing a lot of testing I find that enabling ProxyErrorOverride under the "Pre Main Include" *OR* under "Post VirtualHost Include" section in "WHM Home » Service Configuration » Apache Configuration » Include Editor", allows for proper 404 page operation with bad *.php files, *BUT* either one breaks webmail function, and the message (at end of this post) will appear when accessing webmail.domain.tld *IF* it was previously accessed over 9 minutes prior!

If the browser cache is cleared, all is well again... for 9 minutes! It appears that something happens or expires after 9 minutes that invalidates some credential. (This is not at any login time, I tested with accounts where there was no automatic webmail login enabled, though it still may be related.)

Additionally, users have reported that it will work again, but a great while later (like the next day or longer).

So what appears to be the case is that if PHP-FPM is enabled, and ProxyErrorOverride is enabled (to allow custom 404 pages via .htaccess), that info is set and cached in the browser that becomes stale after 9 minutes and then the connection error happens. If the browser cache is cleared, or the credential expires naturally in the browser after a long time, then access works again. Additionally, once a working connection is established, it will work indefinitely as long as the page is refreshed every <9 minutes.

Error message:

"Access Denied
This server could not verify that you are authorized to access the document requested. Either you supplied the wrong credentials (e.g., bad password), or your browser doesn't understand how to supply the credentials required.

Additionally, a 401 Unauthorized error was encountered while trying to use an ErrorDocument to handle the request."

For now I have removed ProxyErrorOverride to that users can access webmail normally, but that prevents proper 404 errors under certain circumstances. I am only partially clear on how all these pieces fit together and I am not able to find a working solution. Can someone with better understanding shed any light on this?

@cPanelMichael Can cPanel replicate these behaviors on a test server?

-Pete

 

[Rajeeva Lochana]

Oh my, this is because of some some authentication between cpanel and webmail I think.

 

[PeteS]

@cPanelMichael can we get some input on this? We have two issues mutually exclusive solutions, it would seem.

-Pete

 

[PeteS]

*edit* removing double post