Apache Chroot on cPanel ?!

[mOdY]

Hello,

Does cPanel have any fashionable/automated way to run Apache in a chroot'd environment as well as PHP ! I could go with the old normal hard way for chroot'ing but i wonder if cPanel have anything in the box !

also would like clearance on what exactly does /usr/local/cpanel/bin/chroothttpd do ?

,Thanks.

 

[mOdY]

any body !?

 

[Patiek]

I believe /usr/local/cpanel/bin/chroothttpd was an attempt to do chroot but never worked on.

The script at /usr/local/cpanel/bin/chroothttpd will not work on its own... it is broken in many ways.

I do not believe that I can post the modified working file for the public (copyright issues?), so here are some sections that may need to be updated:

Note addition of /etc/vfilters

@COPIES=("/lib","/usr/lib","/usr/sbin","/usr/share","/usr/bin","/usr/X11R6",
   "/usr/kerberos","/usr/man","/var/lib","/dev","/bin","/etc/vfilters");

Note addition of: /usr/share/ssl/

   @FSS = 
("/usr/local","/var/log","/var/spool","/var/run","/tmp","/usr/share/ssl/","$homedir");

# Note addition of eximmailtrap

@FILES=("exim.pl","antivirus.exim","exim.conf","eximmailtrap","services","ld.so.cache","man.config",
"termcap","inputrc","DIR_COLORS","userdomains","domainips","localdomains",
"passwd","group",
"bashrc","profile","host.conf","ld.so.conf","nsswitch.conf","resolv.conf");

And then for good measure at the end but BEFORE "if ($noexec) { exit; }" (note entire block added in the event we have to include more files from /var/cpanel/ later):

mkdir("/home/virthttpdfs/var/cpanel",0755);

# files to allow access to in cpanel
@FILES=("cpanel.config");

foreach my $file (@FILES) {
   next if (! -e "/var/cpanel/$file");
   system("cp","-fa","/var/cpanel/$file","/home/virthttpdfs/var/cpanel/$file");
}

So make modifications according to changes above. Other stuff may break depending on your system configuration (you may need to add stuff where needed to allow access to certain config files, etc). I believe this breaks formmail and it may break other CGIs (I do not know).

You can try it once by running it:

/usr/local/cpanel/bin/chroothttpd

It could take a while for it to build everything. It will build to /home/virthttpdfs/ <-- make sure you do not mess with this folder.

In order to have this script work everytime apache is started do:

touch /etc/chroothttpd

... and then restarting apache.


Likewise, you can disable it later by doing:

rm /etc/chroothttpd

... and then restarting apache


When you are done making modifications to script

Prevent cPanel from modifying file:

chattr +i /usr/local/cpanel/bin/chroothttpd

And if you need to modify file in future (don't forget to run above again after):

chattr -i /usr/local/cpanel/bin/chroothttpd

You may want to copy the modifications to a safe place just to be sure:

cp /usr/local/cpanel/bin/chroothttpd /root/

Other things that may trip you up that you may not think of:

PHP extensions... are they accessible? Zend? eAccelerator?
These common locations work with modified script:
/usr/lib/php/extensions/.../
/usr/local/lib/Zend/...


Also, some apache modules may need access to places (mod_bandwidth perhaps?).

What are benefits
- More secure.
- No need to run phpsuexec crap or similar (hopefully) and happier customers.
- No need to run safe mode garbage and happier customers.

What are disadvantages
- Higher resource consumption?
- More difficult to track resource usage compared to phpsuexec.
- Not officially support by cPanel
- Possibility of broken CGI scripts.

 

[mOdY]

Hello Patiek,

Thanks so much for such help, I did have now a modified version with your additions.. however i won't be able to test it momentary on a production box and unfortunately I'm not so familiar with perl scripts and really not sure i can understand everything this script do however i script in bash

So, i would like to have some stuff clear in my mind before firing it up...

1. I understand that the chroot'd environment will be under "/home/virthttpdfs/", What if i have accounts under /home2 or /home3 or i have multiple drives for storage ?

2. How will cPanel know that apache is chrooted ?

3. Will there be 2 /etc/passwd & /etc/shadow files ? What will happen when i add a new account from cPanel ?

4. What is this line responsible of ? ( print MTAB "/dev/root $fs type virtfs (rw)\n"; )

5. Will i have to do any changes to where is mysql storage "/var/lib/mysql" is placed and make it under the chroot'd system as well as Exim ?



,Thanks again

 

[Patiek]

Hello Patiek,

Thanks so much for such help, I did have now a modified version with your additions.. however i won't be able to test it momentary on a production box and unfortunately I'm not so familiar with perl scripts and really not sure i can understand everything this script do however i script in bash

So, i would like to have some stuff clear in my mind before firing it up...

1. I understand that the chroot'd environment will be under "/home/virthttpdfs/", What if i have accounts under /home2 or /home3 or i have multiple drives for storage ?

2. How will cPanel know that apache is chrooted ?

3. Will there be 2 /etc/passwd & /etc/shadow files ? What will happen when i add a new account from cPanel ?

4. What is this line responsible of ? ( print MTAB "/dev/root $fs type virtfs (rw)\n"; )

5. Will i have to do any changes to where is mysql storage "/var/lib/mysql" is placed and make it under the chroot'd system as well as Exim ?



,Thanks again

1. Yes, all of your home folders are picked up.

2. cPanel is fairly ignorant of it all. If you take a look at your httpd script located at /etc/init.d/httpd , you will see that it is already programmed to go through the chroothttpd file if the file /etc/chroothttpd exists (which is why I said to create it). Since cPanel utilizes this script for starting / stopping apache, it does not have any problems.

3. Adding a new account in cPanel still goes through apache and restarts apache using the chroothttpd script, copying the newly modified files (remember, cPanel will go through /etc/init.d/httpd which will go through the chroothttpd script so long as /etc/chroothttpd file exists).

4. This is part of the script writing its own entries into the /home/virthttpdfs/etc/mtab file.

5. No, both MySQL and Exim will work as expected. That first list of files in the chroothttpd script will be mounted... so any subfolders in those folders are also accessible. I found that exim broke by default, which is why I said to add /etc/vfilters to that first list (which Exim needs).

 

[Patiek]

Oh, there is one more thing. Sometimes the script does not successfully unmount old folders (for example, when it is restarting / rebuilding). I am not sure if there are any ill-effects as the result of this (I have not noticed any), but multiple entries will appear for the mounted file systems (which you can see via /etc/mtab or command df for example).

If you do not like them you can either:
a) Reboot the system.

b) unmount them yourself AFTER shutting down apache:

umount /home/virthttpdfs/usr/local /home/virthttpdfs/var/log /home/virthttpdfs/var/spool /home/virthttpdfs/var/run /home/virthttpdfs/tmp /home/virthttpdfs/usr/share/ssl /home/virthttpdfs/home /home/virthttpdfs/boot /home/virthttpdfs/home2

(or however many home folders you have) and then proceed by rebuilding index: /usr/local/cpanel/bin/chroothttpd --rebuild

c) Your could edit the chroothttpd script and incorporate some waits in it or do some lazy (umount -l) unmounting.

 

[mOdY]

Hello Patiek,

After applying the following file i totally got what will be needed for the chroot to be cPanel friendly, however i did it in a another way which might be a less processes consuming and i replaced the current chroothttpd file.

The problem now is that the php mailing functionality is totally lost.

Any idea what might be missing for it to function properly?

,Thanks.

 

[Patiek]

Hello Patiek,

After applying the following file i totally got what will be needed for the chroot to be cPanel friendly, however i did it in a another way which might be a less processes consuming and i replaced the current chroothttpd file.

The problem now is that the php mailing functionality is totally lost.

Any idea what might be missing for it to function properly?

,Thanks.

What "following file" did you apply (did you forget to attach)? What did you replace chroothttpd file with?

Whatever script you ended up using, you probably forgot to give exim proper access to files as indicated in my first post. Take a look at your /var/log/exim_* log files to see what is happening.

 

[mOdY]

i miss described, i was meaning by the following file the modified chroothttpd file, and what i replaced with chroothttpd file is a file i stripped almost all of the function of chroothttpd from and left the lines which exec the chroot env and execute the apache process.

the way i did it was by excluding probability of copying files/folder and instead i mounted the needed files/folders needed for the apache process to start smoothly, also i did made sure {/var/log/exim_*, /usr/sbin/exim, /etc/exim*, /var/spool/mail/} all exists in the chroot'd env.


,Thanks.

 

[Patiek]

i miss described, i was meaning by the following file the modified chroothttpd file, and what i replaced with chroothttpd file is a file i stripped almost all of the function of chroothttpd from and left the lines which exec the chroot env and execute the apache process.

the way i did it was by excluding probability of copying files/folder and instead i mounted the needed files/folders needed for the apache process to start smoothly, also i did made sure {/var/log/exim_*, /usr/sbin/exim, /etc/exim*, /var/spool/mail/} all exists in the chroot'd env.


,Thanks.

Wow, you could be missing quite a few libraries, etc. From what you have said, it sounds like you forgot to give access to /etc/vfilters for exim as well as /usr/share/ssl/ for ssl on the system. You are probably also missing access for any php extensions such as Zend or eAccelerator (if installed). Doesn't binding those files remove much of the purpose of chrooting apache? Obviously you can restrict the commands, etc, but ultimately if the attacker is able to access your /etc/passwd (which was binded instead of copied), he could give himself open access to the server.

I agree that copying everything in /usr/bin, etc, is in general not the way to go either (a select group of items would be better).

Perhaps the best way to go here would be to use mod_security's chroot functionality (http://www.modsecurity.org/documentation/modsecurity-apache/1.9.3/modsecurity-manual.html#N108DC) combined with your own custom environment. That way you can take care of apache / php stuff automatically and only have to worry about accessibility for things such as exim. I haven't ever tried this though.

 

[cPanelNick]

Hello,

Does cPanel have any fashionable/automated way to run Apache in a chroot'd environment as well as PHP ! I could go with the old normal hard way for chroot'ing but i wonder if cPanel have anything in the box !

also would like clearance on what exactly does /usr/local/cpanel/bin/chroothttpd do ?

,Thanks.

This isn't meant for general consumption. It was designed to work with some existing internal systems, but its only part of the whole ball of wax.

 

[mOdY]

Well cpanelnick, without offending wether cPanel controled services is security wise or note and without getting into this subject, some believe that chroot'ing publicly used services/daemons is a must and should be a 100% optional in easyapache if not forced to be like that, why would a company risk exposing such important resources through a service reachable by anyone.

But that's only my opinion...

Patiek,

I changed nothing in the way the chroot'd system should be, all files/folders are the same but i just replaced copying with binding to save disk space and the number of opened files on the system, I think i should go with copying files to prevent any attack enforcing it's results on the whole system but only the chroot jail, still mailing through php is a problem although i verified that sendmail/exim/exim*/mail and exim config files are all in same places as the real system but in the chroot.



,Thanks.

 

[budway]

If we managed to get into apache 2.0 with no problems we cold use mod-security chroot feature.

That only works with apache 2.*

 

[Patiek]

If we managed to get into apache 2.0 with no problems we cold use mod-security chroot feature.

That only works with apache 2.*

I don't think so... I think it works with Apache 1.3.x. Otherwise, this statement in the documentation would not make sense:

Certain modules (e.g. mod_fastcgi, mod_fcgi, mod_cgid) fork in the module initialisation phase. If they fork before chroot takes place they create a process that lives outside jail. In this case ModSecurity must be configured to initialise after most modules but before the modules that fork. This is a manual process with Apache 1.3.x. It is an automated process with Apache 2.x since ModSecurity 1.9.3.

 

[budway]

I don't think so... I think it works with Apache 1.3.x. Otherwise, this statement in the documentation would not make sense:

It's been a few month's since I readed the mod-sec doc's lat's time I read only apache 2.* was working in the chroot ev. where mod-security would not let the user go outside his dir (/home/user).

Mayble they added to mod-sec 1.9 I will read mod-sec later this week again to see if there where any changes.

Any way mody way of asking stuff is not the right way...

Any way with apache 2.* coming around the corner for cpanel mod-security chrooted tec. is the way to go if you ask me.

 

[Patiek]

Any way with apache 2.* coming around the corner for cpanel mod-security chrooted tec. is the way to go if you ask me.

Yeah, no kidding. Hopefully when Apache 2.x does arrive as stable for cPanel we can all begin to rest a little easier with a good solid chrooted environment.

 

[mOdY]

Any way mody way of asking stuff is not the right way...

Curious to know one of my lines regarding a "not the right way" of asking about stuff