The Community Forums

Interact with an entire community of cPanel & WHM users!
  1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Improving the Internal PHP

Discussion in 'cPanel Announcements' started by cPanelKenneth, Jun 28, 2012.

  1. cPanelKenneth

    cPanelKenneth cPanel Development
    Staff Member

    Joined:
    Apr 7, 2006
    Messages:
    4,460
    Likes Received:
    22
    Trophy Points:
    38
    cPanel Access Level:
    Root Administrator
    The information in this post is about a project in motion. The final delivery may differ from what is discuss here, especially as we consider the feedback you have.

    UPDATE: when this article was written, we were targeting cPanel & WHM 11.34 for delivery of these changes. These changes missed the deadline for 11.34, and will now be available in cPanel & WHM 11.36. The article is updated to reflect that.

    To date we have discussed the overall switching from compile-on-demand to binary packages and how this relates to the perl stack. In this article we’ll focus on changes to the internal PHP stack.

    The Typical cPanel PHP Setup

    Unlike perl, a typical cPanel & WHM server does not have a single installation of PHP. Rather, typically you will find PHP interpreters in the following locations:

    • /usr/local/cpanel/3rdparty/bin/php
    • /usr/bin/php
    • /usr/local/bin/php
    • /var/cpanel/3rdparty/bin/php
    The /usr/local/cpanel location is deployed as part of cPanel & WHM. It is a static implementation generated as part of our build process. This location is often referred to as ‘internal PHP.’ The binaries, and all dependencies, are downloaded and installed using the cpanelsync infrastructure.

    EasyApache handles the PHP installations found in /usr and /usr/local. These are most frequently used by applications served by Apache, however it is possible to use these implementations with applications integrated with the cpanel interface (note: not however with webmail or WHM). These are compiled from source on demand.

    The /var/cpanel/3rdparty instance is also installed as part of cPanel & WHM, however it is built by scripts/makecpphp. Because makecpphp uses EasyApache to build, the /var/cpanel/3rdparty interpreter is customizable. This installation also needs considered when discussing the internal PHP.


    Changes in 11.36

    The internal PHP changes in the following ways with 11.36:

    • It is delivered via pre-built RPMs
    • The version is upgraded to 5.3
    • We added support for using Zend Guard Loader with the internal PHP (you could do this before, but had to perform the changes manually)
    • PEAR dependencies for Horde and the other internal PHP apps are provided by RPM
    • scripts/makecpphp is removed. The preferred way of customizing the internal PHP is by rebuilding our source RPM (note: we’ll be providing documentation on doing this).
    • The loaders are no longer managed by scripts/phpextensionmgr. They are still in Tweak Settings, but are now installed via RPM
    • The Tweak Settings option allows for specifying more than a single loader to use with the internal PHP

    Changes to extensions

    Prior versions of cPanel & WHM built the internal PHP (the one in /usr/local/cpanel) using the following configure options

    --enable-static
    --with-mcrypt
    --with-ldap
    --with-iconv
    --enable-mbstring
    --with-mbstring=all
    --enable-mbstr-enc-trans
    --with-gd
    --with-dom
    --enable-mbregex
    --with-zlib
    --with-jpeg
    --with-png
    --with-imap
    --enable-ftp
    --with-gettext
    --with-xml
    --enable-track-vars
    --enable-sockets
    --with-mysql=/usr
    --with-pgsql
    --enable-cgi


    Some of those configure flags didn’t actually do anything, they were hold-overs from PHP 4. We removed ones that were no longer valid (such as enable-mbstr-enc-trans) and added the following:

    --with-imap-ssl
    --enable-sockets
    --with-pdo-mysql
    --with-pgsql
    --with-pdo-pgsql
    --with-openssl
    --with-kerberos
    --with-curl
    --with-curlwrappers
    --with-tidy
    --with-pcre-regex
    --with-bz2
    --enable-zip

    If you would like other extensions enabled or shipped by default, please let us know.

    For the MySQL extension(s) we are providing a “static” version of the MySQL client library, version 5.0, which installs to /usr/local/cpanel/3rdparty. This client works with all versions of MySQL supported with cPanel & WHM 11.36. Changing your MySQL version will not require rebuilding the PHP mysql extension.

    We also provide a client library for PostgreSQL, and some other dependencies, as RPMs.


    What this means for Upgrades

    During upgrade to 11.36 the older internal implementation is removed. Since it’s possible people have installed custom extensions or modules into /usr/local/cpanel/3rdparty we don’t remove directories that still contain items after we removed what ships with cPanel & WHM.

    It is recommended you examine locations such as /usr/local/cpanel/3rdparty/lib/php after the upgrade. If that location exists it indicates the upgrade found items it was not expecting. Please review any items found there.

    The PHP RPMs install to the prefix /usr/local/cpanel/3rdparty/php/53. To ensure existing applications continue to function after the upgrade we symlink the PHP binary, and several other files to the places in /usr/local/cpanel/3rdparty they existed in 11.32 and older. We expect this to only be the case in 11.36. Successive versions of cPanel & WHM will not guarantee that backwards compatibility.

    Once a system is using cPanel & WHM 11.36 the install in /var/cpanel/3rdparty will no longer be used by the product. If you are not using it in your own applications it is strongly suggested to remove the /var/cpanel/3rdparty directory structure from your server after the upgrade.

    What this means for fresh installs

    Fresh installs will have the following PHP installations:
    • The /usr and /usr/local installations done by EasyApache
    • /usr/local/cpanel/3rdparty/php53

    The /var/cpanel/3rdparty installation will not be done, nor will it be used.

    Environment and Use

    In cPanel & WHM 11.32 and older when using a PHP application within the context of cPanel & WHM, for example Horde Webmail, the PHP interpreter to use was determined by a variety of factors, including the presence of the /var/cpanel/usecpphp flag file. For the most part these applications would use one of:

    • /usr/local/cpanel/3rdparty/bin/php
    • /var/cpanel/3rdparty/bin/php

    Beginning with version 11.36 only the /usr/local/cpanel/3rdparty/php/53/bin/php location will be used to fulfill these requests. This means the /var/cpanel/3rdparty installation of PHP is no longer used. It is strongly recommended, once you are sure nothing in your own applications relies upon it, to remove /var/cpanel/3rdparty after upgrading to 11.36.

    This change also means the /var/cpanel/usecpphp flag file is no longer used. In addition the Cpanel::cpPHP perl module used to determine which php binary would be used by cPanel & WHM is deprecated.

    The cPanel::cpPHP will be shipped with cPanel & WHM 11.36, however calling its functions will result in a message being logged to the cPanel error log (/usr/local/cpanel/logs/error_log). The intent of the log message is so admins can identify applications using the deprecated module, so the application can be updated.

    The default include path, both in the PHP binary and in php.ini is /usr/local/cpanel/3rdparty/php/53/lib/php:/opt/cpanel/php/53/lib/php.

    The install path for the pear and pecl utilities is /opt/cpanel/php/53/lib/php.


    What Comes as RPMs?

    Internal test builds are currently using:

    • PHP 5.3.10
    • PEAR-1.9.4-2
    • ioncube-4.0.14-1
    • sourceguardian-8.2-1
    • zendopt-5.5.0-1

    Before shipping we will update the PHP version to the latest 5.3 available.

    We are also providing the following PEAR and PECL packages as RPMs:
    • Archive-Tar-1.3.10-2
    • Auth-SASL-1.0.6-2
    • Console-Getopt-1.3.1-2
    • Date-1.4.7-2
    • DB-1.7.14-2
    • File-1.4.1-2
    • HTML-Template-IT-1.3.0-2
    • HTTP-1.4.1-2
    • HTTP-Request-1.4.4-2
    • Log-1.12.7-2
    • Mail-1.2.0-2
    • Mail-Mime-1.8.3-2
    • Net-SMTP-1.6.1-2
    • Net-Socket-1.0.10-2
    • Net-URL-1.0.15-2
    • Net-UserAgent-Detect-2.5.2-2
    • PEAR-Command-Packaging-0.3.0-2
    • Structures-Graph-1.0.4-2
    • XML-Parser-1.3.4-2
    • XML-RPC-1.5.5-2
    • XML-Util-1.2.1-2
    The versions in the above lists are not final, expect some of them to change before delivery. If there is something you want added to the list, please let us know.

    Application Needs

    How do I install PEAR or PECL modules for the Internal PHP?

    PEAR modules can be installed using /usr/local/cpanel/3rdparty/php/53/bin/pear. The modules install to /opt/cpanel/php/53/lib.

    To install PECL modules you need to have GCC installed (which it normally is on a cPanel & WHM server). You should be able to install pecl modules using /usr/local/cpanel/3rdparty/php/53/bin/pecl. As with PEAR these modules are installed to /opt/cpanel/php/53/lib.

    What if I need an extension cPanel does not provide in the cPanel-PHP RPM?

    If you have a need to customize the cPanel-PHP RPM, we’d really like to hear from you. Knowing the needs of our customers and users keeps us better informed and guides what we release. It’s quite possible the extensions and changes you want for the PHP RPM are useful to many. Adjusting the RPM we deliver can spread the good, so to speak.

    Customizing the cPanel-PHP RPM deployed to your servers is an option for you as well. I’ll cover this in more detail in a later article. In short going this route will result in replacing the RPM we provide with your own. For changes you need now, rather than later, this will often be your best option.

    More

    This is the third article in our series on migrating to pre-built RPMs. Other articles, written and pending are:

    Discuss this article on our forum.
     
    #1 cPanelKenneth, Jun 28, 2012
    Last edited: Jan 4, 2013
  2. Ark

    Ark Member

    Joined:
    Dec 18, 2012
    Messages:
    7
    Likes Received:
    0
    Trophy Points:
    1
    cPanel Access Level:
    Root Administrator
    Can you guys stop using this option:
    --with-curlwrappers
    It might as well be called --break-fopen-url-wrappers
    It has nothing to do with actually supporting Curl itself, and is in fact, labeled as EXPERIMENTAL in the php ./configure options.
    All this option does is break fopen() with http:// files in several ways that make the streams no longer behave according to the PHP documentation.
     
  3. monarobase

    monarobase Well-Known Member

    Joined:
    Jan 26, 2010
    Messages:
    503
    Likes Received:
    0
    Trophy Points:
    16
    Location:
    France
    cPanel Access Level:
    Root Administrator
    Can we just add pecl extensions ?

    We would like to add uploadprogess PECL so that the PHPMyAdmin upload progress bar works…
     
  4. Kent Brockman

    Kent Brockman Well-Known Member

    Joined:
    Jan 20, 2008
    Messages:
    1,130
    Likes Received:
    3
    Trophy Points:
    38
    Location:
    Buenos Aires, Argentina
    cPanel Access Level:
    Root Administrator
    Twitter:
    Hello Kenneth. I want to ask this: how can I really know for sure that there are no applications using components in /var/cpanel/3rdparty? I found that the versions in /var/cpanel/3rdparty has file dates > than those inside /usr/local/cpanel/3rdparty/, so I should be able to delete them or at least, move them to another location and check if that action breake something.

    BTW, I found that inside /var/cpanel/3rdparty/man there is a folder "man1" dated 01-May-2012, whereas inside /usr/local/cpanel/3rdparty/man there is a folder "man3" dated 11-Nov-2011. Would you say it is safe to remove the /var/cpanel/3rdparty/man/man1 folder?

    Thank you
     
  5. aww

    aww Well-Known Member

    Joined:
    Feb 10, 2005
    Messages:
    152
    Likes Received:
    0
    Trophy Points:
    16
    cPanel Access Level:
    Root Administrator
    Thanks for the writeup Kenneth

    So why exactly does easyapache compile php twice and not just point /usr/bin/php to the physical copy at /usr/local/bin

    The messages from easyapache say "for apache and cli" (/usr/local/bin) and "for system" (/usr/bin)

    Which "system" ? I thought cpanel's own copy was /usr/local/cpanel/3rdparty/php ?
     
    #5 aww, Dec 27, 2012
    Last edited: Dec 27, 2012
  6. cPanelToddR

    cPanelToddR Member
    Staff Member

    Joined:
    Oct 8, 2009
    Messages:
    21
    Likes Received:
    1
    Trophy Points:
    3
    That's actually what it does. EA installs it to /usr/local and only puts symlinks in the /usr prefix.

    The "for system" message is just about putting symlinks in /usr/bin pointing to what was installed in /usr/local/bin
     
  7. aww

    aww Well-Known Member

    Joined:
    Feb 10, 2005
    Messages:
    152
    Likes Received:
    0
    Trophy Points:
    16
    cPanel Access Level:
    Root Administrator
    Sorry, your statement is factually wrong, at least on my 11.34 install, there are two physical copies.

    Looking at the easyapache log, it physically goes through the php build process twice (from the same source).
    Code:
            -- Begin step 'make php for apache and cli' --
    ...
            -- Begin step 'make for system PHP' --
    
    Looking at the inodes of the two sets of files, they are physically different and the L flag is not set so they are neither softlinks nor hardlinks.
    Code:
    ls /usr/local/bin/php* -latri               
    1854523 -rwxr-xr-x 1 root root 21950274 Dec 25 05:14 /usr/local/bin/php
    1854525 -rwxr-xr-x 1 root root     4502 Dec 25 05:15 /usr/local/bin/phpize
    1854489 -rwxr-xr-x 1 root root     3332 Dec 25 05:15 /usr/local/bin/php-config
    
    ls /usr/bin/php* -latri      
    1820509 -rwxr-xr-x 1 root root 21950274 Dec 25 05:18 /usr/bin/php
    5210474 -rwxr-xr-x 1 root root     4490 Dec 25 05:18 /usr/bin/phpize
    1820669 -rwxr-xr-x 1 root root     3314 Dec 25 05:18 /usr/bin/php-config
    1820697 lrwxrwxrwx 1 root root       18 Dec 25 05:19 /usr/bin/php5-cli -> /usr/local/bin/php
    1820696 lrwxrwxrwx 1 root root       12 Dec 25 05:19 /usr/bin/php5-cgi -> /usr/bin/php
    1820698 lrwxrwxrwx 1 root root       18 Dec 25 05:19 /usr/bin/php5 -> /usr/local/bin/php
    1820700 lrwxrwxrwx 1 root root       18 Dec 25 05:19 /usr/bin/php-cli -> /usr/local/bin/php
    1820699 lrwxrwxrwx 1 root root       12 Dec 25 05:19 /usr/bin/php-cgi -> /usr/bin/php
    
    Yes the php5 are links, but the /usr/bin/php is a physical copy, note how phpize is also strangely different in size.
    Based on those timestamps the doubling process adds four minutes to easyapache build time.

    Perhaps though there are circumstance where it only builds twice if it thinks there are special settings that cannot be applied to both? Perhaps certain options trigger a second build where without them it will only do a symlink. However the php cli binaries are identical in size so I have to imagine the build options are identical and easyapache just doesn't realize that for some reason.
     
    #7 aww, Dec 27, 2012
    Last edited: Dec 27, 2012
  8. cPanelKenneth

    cPanelKenneth cPanel Development
    Staff Member

    Joined:
    Apr 7, 2006
    Messages:
    4,460
    Likes Received:
    22
    Trophy Points:
    38
    cPanel Access Level:
    Root Administrator
    What would break if we stopped using --with-curlwrappers ?
     
  9. cPanelKenneth

    cPanelKenneth cPanel Development
    Staff Member

    Joined:
    Apr 7, 2006
    Messages:
    4,460
    Likes Received:
    22
    Trophy Points:
    38
    cPanel Access Level:
    Root Administrator
    Historically PHP required the double compilation in order to get all the handlers in place:
    • cli
    • cgi
    • DSO

    Newer versions of PHP no longer require this double compilation. I believe we changed the behavior for PHP 5.4.

    The "for apache and cli" stage installs the DSO handler (to /usr/local/apache) and the cli interpretor (to /usr/local/bin). The "system" stage installs the CGI handler to /usr/bin.


    The cgi and cli binaries are separate and meant to be used in different contexts, thus the separate locations (as opposed to, say /usr/bin/php and /usr/bin/php-cgi).
     
  10. eva2000

    eva2000 Well-Known Member

    Joined:
    Aug 14, 2001
    Messages:
    322
    Likes Received:
    10
    Trophy Points:
    18
    Location:
    Brisbane, Australia
    cPanel Access Level:
    Root Administrator
    Twitter:
  11. cPanelKenneth

    cPanelKenneth cPanel Development
    Staff Member

    Joined:
    Apr 7, 2006
    Messages:
    4,460
    Likes Received:
    22
    Trophy Points:
    38
    cPanel Access Level:
    Root Administrator
    The internal PHP installation now uses MySQL client libraries provided within /usr/local/cpanel/3rdarty. We have confirmed these libraries are compatible with all versions of MySQL that cPanel & WHM 11.36 supports. Changing your MySQL version will no longer require recompiling the internal PHP as the MySQL client library it uses does not change.

    How this plays out for Percona, Maria and other MySQL variants, I don't know. If the standard MySQL client library is compatible with these alternatives, then no further action will be needed by you. If they are not compatible, then you will need to download the source RPM, modify the RPM spec file to use the correct library, bump the appropriate version number in the spec file, build your new RPM, and install it.
     
  12. Ark

    Ark Member

    Joined:
    Dec 18, 2012
    Messages:
    7
    Likes Received:
    0
    Trophy Points:
    1
    cPanel Access Level:
    Root Administrator
    Really, probably nothing at all. This option only changes how fopen() handles http:// and ftp:// urls. Remember that it is still labeled as EXPERIMENTAL in the php ./configure options.

    I just submitted a patch for curlwrappers to fix the long-standing bug where you can never send outgoing custom headers in your http requests. https://bugs.php.net/bug.php?id=55438
    That bug had no work-around that I knew about.

    There are still many other bugs that only happen when PHP is compiled with curl wrappers, such as fopen() will never return false for URLs like it should, and is documented to do. Instead, you will just get the 404 error text in the stream. You have to work around this by checking stream_get_meta_data() for the 4xx error codes yourself.

    Another bug with curl wrappers is stream_get_meta_data() doesn't work until you read at least 1 byte from the stream. There is no documentation on this, as stream_get_meta_data() should always get you the headers, but it can be worked around by just fread()'ing 1 byte.

    Another bug with curl wrappers is the response headers from stream_get_meta_data() are in wrapper_data->headers instead of just wrapper_data like they should be. Again, people can easily work around this by just testing for the existence of the 'headers' sub-array.

    If you compile without curl wrappers, then you match all of the other copies of PHP that RHEL and CentOS use by default, and you take away all of these bugs by using the native wrappers for URLs with fopen(). Basically, everything works as documented if you don't use this option.
     
  13. cPanelKenneth

    cPanelKenneth cPanel Development
    Staff Member

    Joined:
    Apr 7, 2006
    Messages:
    4,460
    Likes Received:
    22
    Trophy Points:
    38
    cPanel Access Level:
    Root Administrator
    If I am understanding you correctly, use of --with-curlwrappers changes the behavior of calls like fopen(). Applications built around such behavior likely depend upon this changed behavior. Ideally they would be able to detect the difference and handle both --with-curlwrappers and --without-curlwrappers environments. I suspect the reality is different though. Since these are applications built for the cPanel & WHM (as opposed to the Apache) environment, I strongly suspect some, if not too many, are built depending upon the alternate, buggy, behavior.

    The likely course forward is to provide an alternate PHP stack where --with-curlwrappers is not used. This could be made the default, with sufficient communication in advance. In environments where applications are relying upon the buggy behavior, the admin could install the 'legacy' PHP stack, that uses --with-curlwrappers.

    It's too late to do this for 11.36, but thank you for bringing it to our attention, and for providing details.
     
  14. Ark

    Ark Member

    Joined:
    Dec 18, 2012
    Messages:
    7
    Likes Received:
    0
    Trophy Points:
    1
    cPanel Access Level:
    Root Administrator
    To be clear, this only affects fopen() when the file parameter is a url such as http:// or ftp:// -- fopen() for regular file IO is not changed by this option.

    Further, almost all of the bugs caused by this option are situations such that if you worked around the bugs already, your code would still work with a normal PHP install. (such as having to fread() 1 byte to get the meta data... doing the read is harmless and unnecessary when you go back to the normal PHP configuration).
    Anybody who had any intention of working with 4xx status code's text should already be setting the ignore_errors flag for the context options (/http://us3.php.net/manual/en/context.http.php) which will cause fopen() to behave this way intentionally. (Basically, you get forced to use ignore_errors=true when using curl wrappers).

    The only change that I could really see causing trouble is if people assume headers are in wrapper_data->headers instead of just wrapper_data, despite the examples at /http://us3.php.net/manual/en/function.stream-get-meta-data.php all showing them in just wrapper_data.

    I understand what you are saying, however, you never know how many people have just dealt with buggy behavior and not questioned it up to now.
     

Share This Page