Major issues with 11.36 and PHP modules

[cpvw]

Hopefully this will help ease someone else's pain after the 11.36 upgrade. This has been an absolute nightmare for us. (To be fair, some of our cpanel installations have been running in excess of 14 years, so it's no wonder some of these issues fell through the QA cracks.)


Issues:

• Client reported multiple sites unreachable and high server load.
• Websites returning blank pages randomly (segmentation fault in error_log).
• PHP complaining about included files that are "missing", when in fact they are not. Reloading the page in the browser will usually fix the problem.

Proximate Cause:

NOTE: For all issues noted above PHP was running in DSO mode.

Changes to the cPanel EasyApache PHP build process left behind multiple stale loadable modules, which caused instability in the mod_php DSO, resulting in the high load and segmentation faults.

Remedial Steps:

1. Run "php -m" as root to show the list of PHP modules in use and save the output. You'll need this later.
2. Create a backup copy of /usr/local/lib/php.ini.
3. Edit /usr/local/lib/php.ini - comment out any line starting with "extension=".
4. Backup the contents of /usr/local/lib/php/extensions/no-debug-non-zts-20060613 to an alternate location and note which .so files were present. You'll need this later.
5. Remove all files from /usr/local/lib/php/extensions/no-debug-non-zts-20060613 before executing the next step.
6. Rename /root/php/ext to /root/php/ext.old, if it exists.
7. Link /root/php/ext to /usr/local/lib/php/extensions/no-debug-non-zts-20060613
   NOTE: This is required on some systems, as the WHM Module Installer installs PHP PECL modules in /root/php/ext, but then configures php.ini to look for them in /usr/local/lib/php/extensions/no-debug-non-zts-20060613
8. Run /scripts/easyapache - select Apache 2.2 if not already selected (resolves known security issues with Apache 2.1) and the latest non-experimental PHP 5.2 version (should be 5.2.17), ASSUMING the server is not already running PHP 5.3, in which case you would pick the latest non-experimental 5.3 version available.
9. In the "exhaustive list" of options for Apache and PHP, make sure the the PDO libraries are selected (i.e. PDO, PDO_MySQL, and PDO_SQLite). The PECL PDO libraries are deprecated and will no longer function properly. Start the EasyApache build process once these selections are made.
10. Remount /tmp and /var/tmp as rw, exec, dev, suid
11. Once the EasyApache build process is complete, log into WHM and navigate to the Software => Module Install => PECL menu.
12. Referencing the list created in step #4, re-install all of the PECL modules listed. Note that some modules (json, homeloader, etc.) have been rolled into the PHP core code base and are no longer required.
13. Remount /tmp and /var/tmp as rw, noexec, nodev, nosuid.
14. Restart httpd.
15. Verify modules in the output of "php -m" match the list from step 1.
16. Watch /etc/httpd/logs/error_log for any sign of segmentation faults.

 

[KelAt]

Thank you for posting and for raising this issue. We've also had a 4 day nightmare with this issue. We tried to apply this fix, and our web host tried to use this fix to help other customers with similar issues. Unfortunately it didn't resolve the issue completely for any of the machines we tried the fix on.

What did work for us was simply:
1. Going into WHM > EasyApache and "Download profile" so we had a backup of our "Previously Saved Config"
2. Then initiating a "Basic" build, following the default options, which built Apache and PHP using our current versions

This basic rebuild disabled mod_ruid2, which we had been using. Apparently, this relies on DSO which was causing the segmentation fault errors. As our web host said,

I have spent some time working with that fix you posted on a couple other servers with the same issue, and unfortunately it was not fully effective in any case I tried. If you want to get around that for now, you can switch your PHP handler to suPHP or FastCGI, which are not affected by the bug (really any configuration or build involving PHP in DSO mode in any form will end up segfaulting). Ideally, cPanel should have a fix out soon (although "soon" can be relative on their terms"), but until then, the cron job that automatically restarts Apache every so often will at least cover the case of everything crashing at the same time.

 

[Iskan]

We have had a number of customers report issues with browsing the client side, including reports of the random pages half loading and other pages failing to load completely with error 324 "Unable to load the webpage because the server sent no data."

having issues with Apache's connections to the server is due to the PHP configuration on our server and the new cPanel version 11.36. Since our server is running DSO as the PHP handler (with mod_php5 and mod_ruid2), we're experiencing this error .

As for information regarding this issue, there is a specific bug present inside of cPanel 11.36 that causes segmentation faults on servers that are running DSO, mod_ruid2 and cPanel 11.36. We've seen this issue many times already and until cPanel produces a fix.