Please whitelist cPanel in your adblocker so that you’re able to see our version release promotions, thanks!

The Community Forums

Interact with an entire community of cPanel & WHM users!
  1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

EA4 and Bluehost Apache patch.

Discussion in 'EasyApache' started by Spork Schivago, Nov 25, 2016.

  1. Spork Schivago

    Spork Schivago Well-Known Member

    Joined:
    Jan 21, 2016
    Messages:
    514
    Likes Received:
    54
    Trophy Points:
    28
    Location:
    corning, ny
    cPanel Access Level:
    Root Administrator
    Hi.

    I rented a VPS and purchased a license for cPanel through buycpanel.com. I had a VPS from a previous company and things are a bit different with the new company and the full version of cPanel.

    I run the security advisor and see:
    Code:
    Apache Symlink Protection: the Bluehost provided Apache patch is in effect
    
    It appears that the Bluehost provided Apache patch is being used to provide symlink protection. This is less than optimal. Please review Symlink Race Condition Protection.
    
    But I'm not sure how the Apache source code is getting patched. I run CentOS 7. I run:
    Code:
    yum info ea-apache24
    Loaded plugins: fastestmirror, replace, universal-hooks
    Loading mirror speeds from cached hostfile
     * EA4: 69.72.212.11
     * base: mirrors.linode.com
     * city-fan.org: www.city-fan.org
     * epel: mirror.math.princeton.edu
     * extras: mirrors.linode.com
     * ius: ius.mirror.constant.com
     * updates: mirrors.linode.com
    Installed Packages
    Name        : ea-apache24
    Arch        : x86_64
    Version     : 2.4.23
    Release     : 5.7.1.cpanel
    Size        : 7.7 M
    Repo        : installed
    From repo   : EA4
    Summary     : Apache HTTP Server
    URL         : http://httpd.apache.org/
    License     : ASL 2.0
    Description : The Apache HTTP Server is a powerful, efficient, and extensible
                : web server.
    
    
    Does anyone know how my Apache is getting patched with the bluehost Apache symlink protection patch and how I can prevent it from getting patched? I'm running cPanel 11.60.0.25 Release and EA4.

    Thanks!
     
  2. Vs Nu

    Vs Nu Member

    Joined:
    Jul 17, 2015
    Messages:
    8
    Likes Received:
    0
    Trophy Points:
    1
    Location:
    India
    cPanel Access Level:
    Root Administrator
    Symlink Patch Protection is Recommended to Avoid symlink attack.it won't cause you any issue's.
     
  3. danielpmc

    danielpmc Well-Known Member

    Joined:
    Nov 3, 2016
    Messages:
    64
    Likes Received:
    28
    Trophy Points:
    18
    Location:
    Gainesville, Florida
    cPanel Access Level:
    Reseller Owner
    Hello Spork,

    Take a look at this link, the info i posted is on that page. If you choose to UNINSTALL Bluehost make sure you have something in place to protect from SYMLINK attacks.

    Symlink Race Condition Protection - EasyApache - cPanel Documentation

    Bluehost.com-provided patch (available via EasyApache):
    Links
    See Below

    Upside
    You can install this option via EasyApache.

    Downside
    • Protection from this patch is not as good as a kernel-level or a filesystem-level solution.
    • This patch may slow the performance of high-traffic servers.
    • Incompatible with Mailman.
    • Incompatible with CGI Center apps.
    Hope this helps you out,
    danielpmc
     
    #3 danielpmc, Nov 26, 2016
    Last edited by a moderator: Nov 26, 2016
    Spork Schivago likes this.
  4. Spork Schivago

    Spork Schivago Well-Known Member

    Joined:
    Jan 21, 2016
    Messages:
    514
    Likes Received:
    54
    Trophy Points:
    28
    Location:
    corning, ny
    cPanel Access Level:
    Root Administrator
    Hello. Thank you for responding to my question. I'm sorry if I wasn't clear enough. I understand what the symlink race condition is, I understand the various ways to protect against it. It appears the bluehost symlink race condition patch has been applied to my system and I did not apply it.

    I am interested in knowing how it got applied and I am interested in removing the patch. Thank you.
     
  5. Spork Schivago

    Spork Schivago Well-Known Member

    Joined:
    Jan 21, 2016
    Messages:
    514
    Likes Received:
    54
    Trophy Points:
    28
    Location:
    corning, ny
    cPanel Access Level:
    Root Administrator
    Thank you. I will take a look at the link right now. I understand that I need protection in place to protect against the race condition. The idea is to remove the bluehost patch and see if I'm running a patched kernel. If I am not, I will download the source and run a custom compiled kernel, with the GRSecurity patch. The bluehost patch is good for Apache, but I don't think it provides any protection for symlinks created by PHP (or any other process that isn't Apache). In my case, I think the bluehost patch wouldn't provide very much protection at all.

    Thanks!

    **EDIT: Daniel,
    I looked at the link you posted. I have been there before. I believe that link is for people running EA3, not EA4. This is why I'm having such a hard time trying to figure out how to remove the patch. There is no, "Symlink Race Condition Protection" in EA4. I've searched all the options for keywords:
    Code:
    symlink
    sym
    patch
    bluehost
    
    And it always comes back with nothing. I've contacted buycpanel.com and they said they didn't patch cPanel. I know there are ways to manually implement the bluehost patch. I'd really like to know how it got enabled on my system, who applied it, and how to remove it.

    Any more suggestions?

    Thanks!
     
    #5 Spork Schivago, Nov 26, 2016
    Last edited by a moderator: Nov 26, 2016
  6. danielpmc

    danielpmc Well-Known Member

    Joined:
    Nov 3, 2016
    Messages:
    64
    Likes Received:
    28
    Trophy Points:
    18
    Location:
    Gainesville, Florida
    cPanel Access Level:
    Reseller Owner
    Hello Spork Schivago,

    Your host or buycpanel installed Bluehost as it is a plugin. I suspect they did it because:

    buycpanel: If they installed Bluehost it is probably to support and/or allow something they install with cPanel. Perhaps another plugin.
    Solution: Contact support and ask if, why, how to remove and how to secure after removing Bluehost.

    Your current host: If they installed Bluehost it is probably to support their server setups as some type of workaround.
    Solution: Contact support and ask if, why, how to remove and how to secure after removing Bluehost.

    WHMxtra: If you have this system it may have installed Bluehost.
    Solution: Look in the admin controls and see if you can uninstall and/or secure afterwards.

    This is just my opinion on how to approach the situation starting out. Unfortunately i can not offer much more advice than this. Please post additional info as you progress with the problem solving of this, as a help to others on such a complex issue.

    Hope all works out sooner than later,
    danielpmc
     
  7. Spork Schivago

    Spork Schivago Well-Known Member

    Joined:
    Jan 21, 2016
    Messages:
    514
    Likes Received:
    54
    Trophy Points:
    28
    Location:
    corning, ny
    cPanel Access Level:
    Root Administrator
    Here's the packages that are selected for my EA4 profile:
    Code:
    Current Profile
    The currently installed packages on the server.
    
    Apache 2.4
    ----------
    config
    config-runtime
    mod_bwlimited
    mod_cgi
    mod_deflate
    mod_expires
    mod_headers
    mod_mpm_prefork
    mod_proxy
    mod_proxy_fcgi
    mod_proxy_http
    mod_ruid2
    mod_security2
    mod_speling
    mod_ssl
    mod_unique_id
    tools
    
    PHP 7.0
    -------
    libc-client
    pear
    php-bcmath
    php-bz2
    php-calendar
    php-cli
    php-common
    php-curl
    php-devel
    php-fileinfo
    php-fpm
    php-ftp
    php-gd
    php-imap
    php-mcrypt
    php-mysqlnd
    php-pdo
    php-posix
    php-sockets
    php-xml
    runtime
    
    Others
    ------
    apr
    apr-util
    cpanel-tools
    documentroot
    libmcrypt
    modsec-sdbm-util
    php-cli
    profiles-cpanel
    
    There's a few things that confuse me here. For example, under the Apache 2.4 section, what the heck are the config, config-runtime, and tools package? I don't see them as choices for EA4. Also, what's the Others section all about? I don't remember that before. Could any of these be providing the patch? Maybe I should try reinstalling EA4?
     
  8. danielpmc

    danielpmc Well-Known Member

    Joined:
    Nov 3, 2016
    Messages:
    64
    Likes Received:
    28
    Trophy Points:
    18
    Location:
    Gainesville, Florida
    cPanel Access Level:
    Reseller Owner
    Hello Spork Schivago,

    1. What and where is the Apache 2.4 section from?

    2. Are you in fact running EA4?

    3. What do you mean you dont see them as choices for EA4?

    4. Are you comparing logs from your old server and your new one?

    If you are running 2.4/EA4 and pulling up 2.4/EA3 logs something is not right.

    danielpmc
     
  9. Spork Schivago

    Spork Schivago Well-Known Member

    Joined:
    Jan 21, 2016
    Messages:
    514
    Likes Received:
    54
    Trophy Points:
    28
    Location:
    corning, ny
    cPanel Access Level:
    Root Administrator
    Daniel,

    When I say bluehost, I'm talking about the Apache 2.4 patch that prevents the symlink race condition. I did contact buycpanel and they did not provide this patch, nor did they install it. They said they do not do that stuff, and if they ever had to, they would contact the customer first.

    When EA3 provided Symlink Protection, they were using the patch provided by Bluehost. I simply refer to it as the bluehost patch, because Bluehost was the company who created the patch.

    WHMxtra has been completely removed because I didn't like it, however, a quick google search seems to show it didn't come from them.

    The Bluehost patch modifies the source code to Apache and then when Apache is compiled, it provides protection.

    **EDIT:

    I think I just found the solution to my question:

    Symlink Race Condition Protection - EasyApache 4 - cPanel Documentation

    It seems that perhaps cPanel does provide the Bluehost Apache now, but it isn't listed under EA4, like it was with EA3. This is what the cPanel documentation says:
    Code:
    cPanel & WHM only provides the Bluehost.com-provided patch with some modifications. This is an Apache-level patch and considered a last-resort defense. We provide this patch with EasyApache 4, but the protection is not as reliable as kernel or file system level solutions. This patch can slow the performance of high-traffic servers and is not compatible with Mailman or CGI Center applications. 
    
    About the Bluehost patch
    The Bluehost patch's goal is to improve Apache's ability to detect a race condition. The Bluehost patch modifies Apache and APR so that Apache cannot access certain files. The patch helps to ensure that the system can only access files that the domain's owner owns. However, the Bluehost patch only affects requests for static files such as .html and image files. The Bluehost patch does not affect requests that the system processes with application content handlers. Content handlers include the mod_php, mod_ruid2, mod_cgi, and mod_suphp Apache modules.  
    
    If your system does not handle requests on a per-user basis, then the system serves requests as the nobody user. When an application creates files that the nobody user owns, the file's owner does not match the domain's owner. Because the patch prevents access to static content that does not match the owner, the user cannot view these files.
    
    Enable the symlink race condition patch with EasyApache 4
    To use the symlink protection patch, select On for the Symlink Protection option in the Global Configuration section of WHM's Apache Configuration interface (Home >> Service Configuration >> Apache Configuration). 
    
     
    danielpmc likes this.
  10. danielpmc

    danielpmc Well-Known Member

    Joined:
    Nov 3, 2016
    Messages:
    64
    Likes Received:
    28
    Trophy Points:
    18
    Location:
    Gainesville, Florida
    cPanel Access Level:
    Reseller Owner
    Hey Spork Schivago,

    Yes, success! Now we both know it does come from cPanel. Next question is how to uninstall and protect against Symlinks.

    For that answer i would suggest searching cPanel Forums first, then open a thread:
    How to uninstall the EA4 Bluehost patch and secure afterwards?

    I have learned a lot from this,
    danielpmc
     
    Spork Schivago likes this.
  11. danielpmc

    danielpmc Well-Known Member

    Joined:
    Nov 3, 2016
    Messages:
    64
    Likes Received:
    28
    Trophy Points:
    18
    Location:
    Gainesville, Florida
    cPanel Access Level:
    Reseller Owner
    Hello Spork Schivago,

    Can a person disable this since its only protecting html and images? That and cPanel says it is used as a last resort. What is the first resort?

    danielpmc
     
  12. Spork Schivago

    Spork Schivago Well-Known Member

    Joined:
    Jan 21, 2016
    Messages:
    514
    Likes Received:
    54
    Trophy Points:
    28
    Location:
    corning, ny
    cPanel Access Level:
    Root Administrator
    Hello Daniel,

    It's from EasyApache 4 (EA4). I click on EasyApache 4 (Home >> Software >> EasyApache 4) and then I click on View all packages (from my current profile).

    Yes, from the start. I did not migrate from EA3 to EA4. When I installed cPanel for the first time, I installed the latest release (but not EDGE). I'm running 60.0.25 and that's what was installed the first time around.

    When I provision (or customize) the packages for my current profile, I'm provided with choices as to what I want installed. There's four steps to the provisioning system.
    Code:
    * Apache MPM
    * Apache Modules
    * PHP Versions
    * PHP Extensions
    * Review
    
    During the Apache MPM phase, I pick which Multi-Processing Module I want Apache to use (ie, mod_mpm_prefork). If you look in the list I posted, you'll see how it shows mod_mpm_prefork as one of the installed packages. Notice how there isn't a step called Others. When I go through the Apache Modules section, I'm not provided with any options for config, config-runtime, or tools. I thought perhaps the patch was getting enabled in one of these hidden options that I don't have access to. I now think that perhaps that train of thought was wrong and that every EA4 person has those options

    No. I do have access to the logs somewheres from the old server but I have no compared them. Which logs would I be comparing? The upcp update logs? I don't know where the EA4 logs are stored, but I don't think Apache gets recompiled when we change something with EA4. It's a lot different than EA3. With EA3, every time you changed something, the Apache source would get recompiled.

    I am running Apache 2.4 and EasyApache 4. I am not pulling up any EA3 logs. My cPanel configuration is very similar to the old server (same version of cPanel, etc). I haven't completely hardened it yet though. For example, I didn't change the SSL / TLS cipher suite to support perfect forward secrecy, I haven't tightened down Exim yet, etc.

    Just trying to figure out how to remove this patch.

    It is NOT listed under the Global Configuration Options. I might need to create a ticket here at cPanel and have the developers look into this for me.
     
    #12 Spork Schivago, Nov 26, 2016
    Last edited: Nov 26, 2016
    danielpmc likes this.
  13. Spork Schivago

    Spork Schivago Well-Known Member

    Joined:
    Jan 21, 2016
    Messages:
    514
    Likes Received:
    54
    Trophy Points:
    28
    Location:
    corning, ny
    cPanel Access Level:
    Root Administrator
    According to that document Daniel, I just go to the Global Configuration Options in cPanel for Apache and turn On to Off for Symlink Protection. However, on my server, there is no Symlink Protection under the Global Configuration Options in cPanel for Apache.

    I've grepped through the entire /etc/apache2 directory for SymlinkProtect as well and didn't get any returns
    Code:
    root@franklin:[/etc/apache2]# grep -iR SymlinkProtect *
    root@franklin:[/etc/apache2]# 
    
    I'm wondering where else the SymlinkProtect option might be stored. Maybe someone manually set it. The documentation isn't exactly clear and what file I'd set this directive in.

    As for how to protect myself from this type of attack after removing the bluehost patch, I'm going to go with the kernel solution, if my kernel doesn't currently support it (which I believe it does). There's a kernel GRSecurity patch, but it seems my kernel has some sort of SymLink protection built in by default. I see in one of the sysctl.conf files:
    Code:
    # Enable hard and soft link protection
    fs.protected_hardlinks = 1
    fs.protected_symlinks = 1
    
    Thanks.
     
  14. Spork Schivago

    Spork Schivago Well-Known Member

    Joined:
    Jan 21, 2016
    Messages:
    514
    Likes Received:
    54
    Trophy Points:
    28
    Location:
    corning, ny
    cPanel Access Level:
    Root Administrator
    Back to the drawing board I guess. I took a peek at the EA4 Apache's template, the file that generates the /etc/apache2/conf/httpd.conf file from the various options set in cPanel. This template is:
    Code:
    /var/cpanel/templates/apache2_4/ea4_main.default
    I see the code that enables the symlink protection:
    Code:
    # These are hard-coded values that are required by cPanel & WHM
    PidFile [% paths.dir_run %]/httpd.pid
    User nobody
    Group nobody
    ExtendedStatus [% IF main.exists('extendedstatus') %][% main.extendedstatus.item.extendedstatus %][% ELSE %]Off[% END %]
    LogLevel [% IF main.exists('loglevel') %][% main.loglevel.item.loglevel %][% ELSE %]warn[% END %]
    [%- IF main.exists('symlink_protect') %]
    SymlinkProtect [% main.symlink_protect.item.symlink_protect %]
    SymlinkProtectRoot [% paths.dir_docroot %]
    [% END -%]
    
    I check my /etc/apache2/conf/httpd.conf file and see:
    Code:
    # These are hard-coded values that are required by cPanel & WHM
    PidFile /etc/apache2/run/httpd.pid
    User nobody
    Group nobody
    ExtendedStatus On
    LogLevel warn
    
    As you can clearly see, there is no SymlinkProtect or SymlinkProtectRoot options set.

    I wonder if that sysctl.conf file has anything to do with it....To me though, I think something is very wrong. Why don't I have the option to turn Symlink Protection on and off in WHM like the documentation says to do? A clean install of cPanel....it should be there....the template expects it to be there.
     
  15. Spork Schivago

    Spork Schivago Well-Known Member

    Joined:
    Jan 21, 2016
    Messages:
    514
    Likes Received:
    54
    Trophy Points:
    28
    Location:
    corning, ny
    cPanel Access Level:
    Root Administrator
    I wish @cPanelMichael was here. He's real good with this stuff and really knows the ins and outs of the cPanel source code. I bet he'd know what was going on. He'd ask some questions and have me copy and paste some outputs, maybe have me run a command, and then he'd say could you please create a support ticket so I can take a closer look? And then in the support ticket section, he'd post updates with what was wrong and how he fixed it.

    I bet I'm gonna have to create a ticket.
     
  16. danielpmc

    danielpmc Well-Known Member

    Joined:
    Nov 3, 2016
    Messages:
    64
    Likes Received:
    28
    Trophy Points:
    18
    Location:
    Gainesville, Florida
    cPanel Access Level:
    Reseller Owner
    Hello Spork Schivago,

    Your server is showing modules that you are unable to physically control. Why? And why is the option to disable Symlink not available to you? Something is a little odd. I agree this is probably a good time for you to ask for cPanel Technical assistance.

    danielpmc
     
    Spork Schivago likes this.
  17. Spork Schivago

    Spork Schivago Well-Known Member

    Joined:
    Jan 21, 2016
    Messages:
    514
    Likes Received:
    54
    Trophy Points:
    28
    Location:
    corning, ny
    cPanel Access Level:
    Root Administrator
    I think perhaps no one has access to some of the things that show up in EA4, like the Others section, for example. My original train of thought was the list was customizable, like many things in cPanel. For example, that template I was talking about, it has the .default extension. EA3 used similar templates but you could override them and use a custom template if you gave it a .local extension. EA3 would parse the .local template instead of the .default.

    With EA4, the developers were trying to move away from that style though and where encouraging users to use those pesky Pre Main Include, Pre Virtual Host Include, and Post Virtual Host Include files.

    I'm willing to bet that Others section is supposed to be there and includes stuff users aren't supposed to have control over. I'm willing to bet those things that look like modules in the Apache 2.4 section aren't actual modules and are cPanel specific stuff that we shouldn't have control over. EA4 has to be storing the profile somewhere on disk and I bet when I click the View all packages, when it reads the file in, it's just not hiding the stuff.

    The reason I say this is because when EA4 came out, I remember people asking if cPanel would implement the Bluehost patch into EA4, just like they did with EA3. Eventually, cPanel gave the users the link to the patch and told them that if they wanted to, they could download the source code to Apache, patch and install it manually. If there was a way to implement the patch into EA4, they sure as heck didn't make it clear.

    There's documentations on how to customize various things in cPanel from the shell. For example, I can override the installation of the version of Roundcube that comes with cPanel and have cPanel install any version I want, with my own custom configuration. They have documents telling you how to do this. It's not something the average user would be doing, I wouldn't think. But if they're willing to share that information with users, I couldn't see why they would try to hide the ability to custom configure EA4, you know?
     
  18. cPJacob

    cPJacob cPanel Product Owner
    Staff Member

    Joined:
    May 2, 2014
    Messages:
    605
    Likes Received:
    94
    Trophy Points:
    103
    cPanel Access Level:
    DataCenter Provider
    Twitter:
    Hi,

    We are adding in the SymlinkProtection patch to EA4. It's already been released in the RPMs, but we're adding in the UI support in v62. We're updating SecurityAdvisor to do the proper thing with the new patches.

    Also, the patch is controlled via an 'On/Off' configuration directive in httpd.conf:
    Code:
    SymlinkProtect On
    SymlinkProtectRoot /var/www/html
    
    I hope this helps!
     
    Spork Schivago and danielpmc like this.
  19. Spork Schivago

    Spork Schivago Well-Known Member

    Joined:
    Jan 21, 2016
    Messages:
    514
    Likes Received:
    54
    Trophy Points:
    28
    Location:
    corning, ny
    cPanel Access Level:
    Root Administrator
    cPJacon,

    Thank you for the response. I now know what's going on. I'll post my information on the problem in case anyone else has the same question.

    Currently, at least with cPanel 60, Apache is in fact patched for the Symlink Race Condition, using the Bluehost patch, but the patch is not "active" by default. The way the Security Advisor checks for the patch is by searching the Apache binary, httpd, for the SPT_DOCROOT string. If it finds it, it reports the patch as being active.

    The patch gets activated by the top-level Apache directives, SymlinkProtect On, in the /etc/apache2/conf/httpd.conf configuration file. There's also the SymlinkProtectRoot /var/www/html directive. The SymlinkProtectRoot directive will be ignored if the SymlinkProtect is set to Off or is missing. I believe the proper way to check to see if the Bluehost Symlink Race Condition patch is active is to run the following commands:
    Code:
    grep symlink_protect /var/cpanel/conf/apache/local
    grep Symlink /etc/apache2/conf/httpd.conf
    
    Melanie showed me how to properly check. If grep doesn't report anything, then the patch is inactive.

    Obviously, patching the Security Advisor to just grep the various files looking for symlink_protect and Symlink, respectively, isn't really going to work. If people have commented out the link symlink_protect, grep will still report it has finding it.

    I think what makes it especially confusing is the documentation found here:
    Symlink Race Condition Protection - EasyApache 4 - cPanel Documentation

    It doesn't mention that the Apache Global Configuration option is only available for cPanel 62. It implies so long as you're running EasyApache 4, you should see the option to turn it on or off. Maybe the documentation should be edited to clarify that it's only for 62?

    EA-5670 will address the false positive, if anyone else has noticed the same issues. Just thought everyone would like to know.

    Thanks for taking the time to answer my questions.
     
    cPanelDon likes this.
  20. cPanelMichael

    cPanelMichael Forums Analyst
    Staff Member

    Joined:
    Apr 11, 2011
    Messages:
    38,658
    Likes Received:
    1,425
    Trophy Points:
    363
    cPanel Access Level:
    Root Administrator
    Hello @Spork Schivago,

    That's a good point. I've opened a case with our documentation team to address that concern and add a warning about the feature's availability in cPanel version 62. I'll update this thread with the outcome.

    Thank you.

    Update: The changes to this document are now published and include the following note:

     
    #20 cPanelMichael, Nov 29, 2016
    Last edited: Feb 27, 2017
Loading...

Share This Page