Limiting Number of PHP processes spawned w/ mod_fcgid ??

[wizzy420]

Does anyone know how to limit the number of PHP processes allowed when using mod_fcgid?

What I would love to be able to do, and would help tremendously with the occasional server crash when a user gets slammed, is to be able to limit this.

Somehow say "once a user has 4 PHP-CGI's running, no more for that user. Sorry."

I'm happy with this being a single, server wide setting. Right now I'm using mod_fcgid. I don't want it to be "total per server" as that means one user could suck up all free slots and no one gets any more. I would like to be able to say "every user in the server gets up to (3,4,5) and if any user hits that number of PHP-CGI processes running, they don't get any more"

The way it is now it's possible a user gets creamed and then all the sudden they end up with 20 PHP's running and sucking up 1 GB or 2 which makes the server unhappy.

Steve

 

[wizzy420]

Gentle Nudge.

 

[Andrew Boring]

Try:

DefaultMaxClassProcessCount 5

I saw your post here while setting up a similar configuration with a test server and searching for the same answer.

I have Apache 2.2 with worker MPM, and configured PHP for fcgi handling. I installed a simple wordpress blog with some youtube videos linked, and then slammed it with ab (apache benchmark) to simulate a ton of concurrent requests.

I ran ab on my laptop with:
ab -c 30 -n 1000 -v 2 http://example.com/

This means 1000 requests with 30 concurrent connections. Initially, Apache/FastCGI would spawn about 20 php5 processes and the system load would shoot up. Limiting to 5 above kept the number of php5 processes to 5.

I was originally running the ab tool while trying to use mod_bw to limit simultaneous connections and return a 503 error, but that didn't seem to work. The DefaultMaxClassProcessCount 5 option is the winner in my testing so far.

 

[rack::SPEED]

I set DefaultMaxClassProcessCount to 10, but when i use ab with same parameters it spawns up to 34 processes. System load shoots up and everything slows down... - Can somebody help?

 

[Andrew Boring]

I set DefaultMaxClassProcessCount to 10, but when i use ab with same parameters it spawns up to 34 processes. System load shoots up and everything slows down... - Can somebody help?

Where did you add DefaultMaxClassProcessCount 10? Did you restart Apache?

I placed the directive (for testing only) in /usr/local/apache/conf/php.conf.
To keep it permanently, you'd need it in pre_virtualhost_global.conf, since that's loaded after php.conf but before the virtual hosts are.

 

[rack::SPEED]

I made my changes in /usr/local/apache/conf/php.conf and restarted apache. It seems that all other settings are working. I checked it multiple times, no typo or other problems.

Funny is, that error_log said 10/10 processes reached, stopping spawning. - Is my server spawning to fast?

Sorry, for my bad english.

 

[Andrew Boring]

If the error_log is showing that it's only spawning 10 processes, then it sounds like it's working properly.

You can check this by running top on the server and counting the number of php processes when running ab from your client machine, or by running something like ps aux | grep php or ps aux | grep php | wc -l when you are running ab. You may need to filter on your test username as well: ps aux | grep php | grep username.

 

[rack::SPEED]

Thank you very much. I've forgotten to update this thread. It's working like expected now. - I had some subdomains on the same account with old starter script, so they continued spawning.

 

[StingRay2k01]

... I don't want it to be "total per server" as that means one user could suck up all free slots and no one gets any more. I would like to be able to say "every user in the server gets up to (3,4,5) and if any user hits that number of PHP-CGI processes running, they don't get any more"
...

Steve

I was initial concerned about this but it just didn't make sense, so I did some testing.
I had 16 (max) processes all being used by one account, then made a request with another account, and 1 or more of the processes went to the new account. So I think it must free them up as necessary.
Now those original proceeses that were owned by the first account, could have been idle and so we able to be free'd up, that i do not know. But it still doesn' tmake sense that one user could hog all the processes. I just don't know enough about the internal workings to say.

Anyone with more experience?

 

[elleryjh]

Hi everyone,

The problem I've been having is that suddenly a certain processor-intensive PHP/MySQL website gets hit hard, either by the same host or several hosts, and spawns a lot of simultaneous (25 or more) PHP processes.

Here's a sample of the full httpd tree:

 root     22108  0.0  0.2 10468 4304 ?        Ss   Jul15   0:03 /usr/local/apache/bin/httpd -k start
 root     22114  0.0  0.1 10388 2968 ?        S    Jul15   0:00  \_ /usr/local/apache/bin/httpd -k start
 nobody   30919  0.0  0.2 10872 4436 ?        S    10:18   0:00  \_ /usr/local/apache/bin/httpd -k start
 user2    31667  0.0  0.3 18720 6236 ?        S    10:20   0:00  |   \_ /usr/bin/php /home/user2/public_html/index.php
 nobody   31742  0.0  0.2 10876 4520 ?        S    10:21   0:00  \_ /usr/local/apache/bin/httpd -k start
 user3 32292  0.0  0.2 19788 6152 ?        S    10:23   0:00  |   \_ /usr/bin/php /home/user3/public_html/index.php
 nobody   31754  0.0  0.2 10872 4464 ?        S    10:21   0:00  \_ /usr/local/apache/bin/httpd -k start
 user3 32242  0.2  0.6 28492 12756 ?       S    10:23   0:00  |   \_ /usr/bin/php /home/user3/public_html/index.php
 nobody   32216  0.0  0.2 10744 4284 ?        S    10:23   0:00  \_ /usr/local/apache/bin/httpd -k start
 someuser   328  2.6  1.0 34400 21676 ?       R    10:24   0:01  |   \_ /usr/bin/php /home/someuser/public_html/index.php
 nobody   32235  0.0  0.1 10600 4012 ?        S    10:23   0:00  \_ /usr/local/apache/bin/httpd -k start
 user3 32248  0.0  0.2 20084 6144 ?        S    10:23   0:00  |   \_ /usr/bin/php /home/user3/public_html/index.php
 nobody   32264  0.0  0.2 10748 4340 ?        S    10:23   0:00  \_ /usr/local/apache/bin/httpd -k start
 nobody   32483  0.0  0.2 10880 4460 ?        S    10:23   0:00  \_ /usr/local/apache/bin/httpd -k start
 nobody   32544  0.0  0.2 10600 4252 ?        S    10:24   0:00  \_ /usr/local/apache/bin/httpd -k start
 nobody   32551  0.0  0.2 10744 4300 ?        S    10:24   0:00  \_ /usr/local/apache/bin/httpd -k start
 someuser   559  2.5  0.8 30092 18012 ?       S    10:25   0:00  |   \_ /usr/bin/php /home/someuser/public_html/index.php
 nobody   32560  0.0  0.2 10736 4268 ?        S    10:24   0:00  \_ /usr/local/apache/bin/httpd -k start
 someuser   516  2.6  1.0 35220 22212 ?       R    10:25   0:00  |   \_ /usr/bin/php /home/someuser/public_html/index.php
 nobody   32566  0.0  0.2 10736 4260 ?        S    10:24   0:00  \_ /usr/local/apache/bin/httpd -k start
 someuser   520  2.8  1.0 34832 21700 ?       R    10:25   0:00  |   \_ /usr/bin/php /home/someuser/public_html/index.php
 nobody   32571  0.0  0.2 10736 4268 ?        S    10:24   0:00  \_ /usr/local/apache/bin/httpd -k start
 nobody   32575  0.0  0.2 10736 4272 ?        S    10:24   0:00  \_ /usr/local/apache/bin/httpd -k start
 someuser   351  2.1  1.0 33792 22368 ?       R    10:24   0:00  |   \_ /usr/bin/php /home/someuser/public_html/index.php
 nobody   32581  0.0  0.2 10736 4264 ?        S    10:24   0:00  \_ /usr/local/apache/bin/httpd -k start
 nobody   32585  0.0  0.2 10736 4280 ?        S    10:24   0:00  \_ /usr/local/apache/bin/httpd -k start
 nobody   32587  0.0  0.2 10732 4348 ?        S    10:24   0:00  \_ /usr/local/apache/bin/httpd -k start
 someuser   531  2.3  1.5 43920 31540 ?       R    10:25   0:00  |   \_ /usr/bin/php /home/someuser/public_html/index.php
 nobody   32719  0.0  0.2 10736 4256 ?        S    10:24   0:00  \_ /usr/local/apache/bin/httpd -k start
 someuser   327  2.7  1.0 34144 22464 ?       S    10:24   0:01  |   \_ /usr/bin/php /home/someuser/public_html/index.php
 nobody   32724  0.0  0.2 10736 4248 ?        S    10:24   0:00  \_ /usr/local/apache/bin/httpd -k start
 someuser   330  2.6  1.0 35104 22724 ?       S    10:24   0:01  |   \_ /usr/bin/php /home/someuser/public_html/index.php
 nobody   32761  0.0  0.2 10736 4252 ?        S    10:24   0:00  \_ /usr/local/apache/bin/httpd -k start
 someuser   346  1.6  1.1 35168 22964 ?       S    10:24   0:00  |   \_ /usr/bin/php /home/someuser/public_html/index.php
 nobody     312  0.0  0.2 10736 4260 ?        S    10:24   0:00  \_ /usr/local/apache/bin/httpd -k start
 someuser   589  0.6  0.3 20708 7124 ?        S    10:25   0:00  |   \_ /usr/bin/php /home/someuser/public_html/index.php
 nobody     314  0.0  0.2 10736 4236 ?        S    10:24   0:00  \_ /usr/local/apache/bin/httpd -k start
 nobody     316  0.0  0.2 10736 4256 ?        S    10:24   0:00  \_ /usr/local/apache/bin/httpd -k start
 nobody     318  0.0  0.2 10736 4220 ?        S    10:24   0:00  \_ /usr/local/apache/bin/httpd -k start
 someuser   319  2.6  1.1 35912 22992 ?       S    10:24   0:01  |   \_ /usr/bin/php /home/someuser/public_html/index.php
 nobody     332  0.0  0.2 10736 4220 ?        S    10:24   0:00  \_ /usr/local/apache/bin/httpd -k start
 someuser   333  2.6  1.1 34648 23036 ?       S    10:24   0:01  |   \_ /usr/bin/php /home/someuser/public_html/index.php
 nobody     334  0.0  0.2 10736 4244 ?        S    10:24   0:00  \_ /usr/local/apache/bin/httpd -k start
 nobody     336  0.0  0.2 10736 4216 ?        S    10:24   0:00  \_ /usr/local/apache/bin/httpd -k start
 someuser   337  2.7  1.0 33976 21956 ?       S    10:24   0:01  |   \_ /usr/bin/php /home/someuser/public_html/index.php
 nobody     338  0.0  0.2 10600 4208 ?        S    10:24   0:00  \_ /usr/local/apache/bin/httpd -k start
 user3   561  2.2  0.5 26164 11496 ?       R    10:25   0:00  |   \_ /usr/bin/php /home/user3/public_html/index.php
 nobody     339  0.0  0.2 10736 4252 ?        S    10:24   0:00  \_ /usr/local/apache/bin/httpd -k start
 nobody     341  0.0  0.2 10736 4244 ?        S    10:24   0:00  \_ /usr/local/apache/bin/httpd -k start
 nobody     343  0.0  0.2 10736 4228 ?        S    10:24   0:00  \_ /usr/local/apache/bin/httpd -k start
 someuser   344  1.6  1.0 34200 22440 ?       S    10:24   0:00  |   \_ /usr/bin/php /home/someuser/public_html/index.php
 nobody     345  0.0  0.2 10736 4216 ?        S    10:24   0:00  \_ /usr/local/apache/bin/httpd -k start
 someuser   394  1.7  1.0 34924 22160 ?       S    10:24   0:00  |   \_ /usr/bin/php /home/someuser/public_html/index.php
 nobody     359  0.0  0.2 10736 4236 ?        S    10:24   0:00  \_ /usr/local/apache/bin/httpd -k start
 nobody     360  0.0  0.2 10736 4216 ?        S    10:24   0:00  \_ /usr/local/apache/bin/httpd -k start
 someuser   361  2.6  1.0 34464 22208 ?       S    10:24   0:01  |   \_ /usr/bin/php /home/someuser/public_html/index.php
 nobody     362  0.0  0.2 10736 4216 ?        S    10:24   0:00  \_ /usr/local/apache/bin/httpd -k start
 someuser   363  2.6  1.0 33896 21952 ?       S    10:24   0:01  |   \_ /usr/bin/php /home/someuser/public_html/index.php
 nobody     364  0.0  0.2 10736 4240 ?        S    10:24   0:00  \_ /usr/local/apache/bin/httpd -k start
 nobody     366  0.0  0.2 10736 4216 ?        S    10:24   0:00  \_ /usr/local/apache/bin/httpd -k start
 someuser   367  2.6  1.0 34144 22272 ?       S    10:24   0:01  |   \_ /usr/bin/php /home/someuser/public_html/index.php
 nobody     368  0.0  0.2 10736 4236 ?        S    10:24   0:00  \_ /usr/local/apache/bin/httpd -k start
 nobody     370  0.0  0.2 10736 4216 ?        S    10:24   0:00  \_ /usr/local/apache/bin/httpd -k start
 someuser   371  2.6  1.1 36056 22988 ?       S    10:24   0:01  |   \_ /usr/bin/php /home/someuser/public_html/index.php
 nobody     372  0.0  0.2 10736 4236 ?        S    10:24   0:00  \_ /usr/local/apache/bin/httpd -k start
 nobody     373  0.0  0.2 10736 4216 ?        S    10:24   0:00  \_ /usr/local/apache/bin/httpd -k start
 someuser   374  2.7  1.0 32872 21696 ?       R    10:24   0:01  |   \_ /usr/bin/php /home/someuser/public_html/index.php
 nobody     375  0.0  0.2 10736 4216 ?        S    10:24   0:00  \_ /usr/local/apache/bin/httpd -k start
 someuser   376  2.7  1.0 34168 22464 ?       S    10:24   0:01  |   \_ /usr/bin/php /home/someuser/public_html/index.php
 nobody     377  0.0  0.2 10608 4200 ?        S    10:24   0:00  \_ /usr/local/apache/bin/httpd -k start
 someuser   525  2.8  0.8 31224 18592 ?       S    10:25   0:00  |   \_ /usr/bin/php /home/someuser/public_html/index.php
 nobody     390  0.0  0.1 10600 4144 ?        S    10:24   0:00  \_ /usr/local/apache/bin/httpd -k start
 nobody     403  0.0  0.2 10736 4216 ?        S    10:24   0:00  \_ /usr/local/apache/bin/httpd -k start
 someuser   404  2.5  1.0 35780 22728 ?       R    10:24   0:01  |   \_ /usr/bin/php /home/someuser/public_html/index.php
 nobody     405  0.0  0.2 10736 4216 ?        S    10:24   0:00  \_ /usr/local/apache/bin/httpd -k start
 someuser   406  2.7  1.0 34188 22172 ?       R    10:24   0:01  |   \_ /usr/bin/php /home/someuser/public_html/index.php
 nobody     407  0.0  0.2 10736 4216 ?        S    10:24   0:00  \_ /usr/local/apache/bin/httpd -k start
 someuser   408  2.8  1.0 34216 22468 ?       S    10:24   0:01  |   \_ /usr/bin/php /home/someuser/public_html/index.php

....

This seems like it shouldn't be allowed by default, as this is quite easy to take down a server... just find a processor-intensive page and repeatedly load it from a handful of hosts

So what I would like to do it limit the number of PHP processes that someuser can run simultaneously. This should not interfere with otheruser's ability to run processes.

I found this thread which says you can limit this with the directive DefaultMaxClassProcessCount using the mod_fcgid module.

I recompiled PHP and apache with Fast CGI / mod_fcgi

Then I put the single line in pre_virtualhost_global.conf:
DefaultMaxClassProcessCount 10

But the directive is not working for me:

Invalid command 'DefaultMaxClassProcessCount', perhaps misspelled or defined by a module not included in the server configuration

Any ideas?

Running Apache 2.2.11, PHP 5.2.10

 

[StingRay2k01]

Here is my setup. I put it in the Pre VirtualHost -- All version section in WHM.


<IfModule mod_fcgid.c>
MaxRequestsPerProcess 500
MaxProcessCount 15
DefaultMaxClassProcessCount 15
DefaultMinClassProcessCount 0
IPCConnectTimeout 60
IPCCommTimeout 60
PHP_Fix_Pathinfo_Enable 1
IdleTimeout 300
IdleScanInterval 30
BusyTimeout 120
BusyScanInterval 90
ErrorScanInterval 3
ZombieScanInterval 3
ProcessLifeTime 3600
</IfModule>


This allows for a maximum of 15 php processes for the whole server.
One user could use them all just like in your example. It doesn't happen often so I'm not concerned.

The only way to limit it per user is to set it up completely differently. Which I tried once, but benchmarking it showed that it was as slow as suPHP, so then there was no point to use it.

I don't know why you get that Invalid command error. Did you use WHM like I did, or edit it directly?

 

[Spiral]

Does anyone know how to limit the number of PHP processes allowed when using mod_fcgid?

Simple! Don't use mod_fcgid! Bad, very bad !!!!!

SuPHP is actually extremely fast and very closely rivals regular
DSO performance only being a very slight marginal bit slower but only
if you DO NOT use mod_fcgid or mod_fastcgi!

Both of these will conflict with SuPHP's own internal script handling
and either will substantially slam down your server's performance!

If you are using regular DSO based PHP, obsolete phpSuExec, or
regular general CGI based PHP (not recommended) then you can
go ahead and use either mod_fastcgi or mod_fcgid to accelerate
the performance of CGI scripts (Perl, Python, and PHP if applicable)
but it will do exactly the opposite for SuPHP which actually uses
it's own optimized DSO based controller to launch PHP in CGI mode
and handles it's own code optimizations very well.

I've seen many a person make the same mistake and think that
they need some sort of accelerator to speed up SuPHP and instead
end up screwing up the performance of their server by making things
drastically slower than faster all along not realizing if they never
installed any of that in the first place, their performance would be fine.

If you are using SuPHP for your PHP, don't install any kind of accelerator.
You DO NOT want CGI scripts operating in "Fast Mode" or any kind of
PHP code optimizer installed.


Sole Exception:
-------------------------------------------------
Zend Optimizer works internally as a loadable PHP module
and is in turn processed by SuPHP's handler and won't
negatively impact SuPHP's performance.

 

[elleryjh]

Simple! Don't use mod_fcgid! Bad, very bad !!!!!

Sounds good - but then how do I limit the number of processes per user?

Yes, I am using suPHP and would like to continue using it.

I've poured through Apache, PHP, suPHP, and linux docs and can't find a way to limit the number of simultaneous PHP processes per user... The only possible solution that I've found is to use mod_fcgi in conjunction...which I also can't get to work yet.

 

[eperdeme]

Hello,

Did you get anywhere with the limiting of suPHP to the number of processes a single user may spawn off ?

 

[Spiral]

The only possible solution that I've found is to use mod_fcgi in conjunction...which I also can't get to work yet.

Sorry, been very busy the past few days and have not been here too much ...

Anyway, your statement above makes no reasonable sense whatsoever ....

You cannot run both FastCGI PHP and SuPHP together as those are competing types of PHP though what you just said is commonly confused with running fastcgi support on Apache while using SuPHP which is entirely different and yes you could do that.

FastCGI / FCGI based PHP is one where you would want to limit the initial process controllers but not the processes themselves.

In SuPHP, the issue isn't so much really an issue of "number of processes" as much as how the processes are spawned and though you could I suppose limit those, it would seem counter productive and possible cause you more significant problem with your site.

Tell you what. I should be on most of the day today on and off, if you would like to try to catch me around, I'd be happy to just take a quick peak at things if you wish and can probably through you a few quick configuration tips on what you should probably do with your system in particular. I am not always here but I do try to answer private messages and email messages as soon as I see them.

 

[anton_latvia]

Simple! Don't use mod_fcgid! Bad, very bad !!!!!

Sorry, but I disagree with your opinion. Yes, fastCGI has several drawbacks, mainly - memory usage, because php processes keep running, but just because of that - it has all libraries loaded into memory and ready for request processing. As a proof I am attaching sheet of my local tests, that I performed on several our servers with different configuration.

I'll explain server setup:
web12 - apache 2.2, php runs in DSO mode, no acceleration, apache mpm-prefork.
web11 - apache 2.2, php runs as suPHP, mpm not defined, I guess prefork?
web3 - apache 2.2, php runs as fastCGI, apache mpm-worker

servers has more or less similar hardware, may be web3 is a bit more powerful and has more memory than other two.

I tested with simple HTML page and simple phpinfo();

As you can see from the document, for HTML page it helps to have mpp-worker.

For PHP - DSO mode is still the fastest, but running script as "nobody" is making troubles from many many points of view. So in general this was fight between suPHP and fastCGI and sorry to say - it was fastCGI which won it by 1100% ! fastCGI has processes 11 times more requests than suPHP just because the fact - that for every request suPHP had to load PHP and all the libraries, process and unload them..

Your comments and suggestions are very welcome, as this is important question for me as well.

Does anyone know how to limit the number of PHP processes allowed when using mod_fcgid?

To make limiting work you have to add fastCGI configuration directives to /usr/local/apache/conf/php.conf _after_ line:

LoadModule fcgid_module modules/mod_fcgid.so

but before:

AddHandler fcgid-script .php5 .php4 .php .php3 .php2 .phtml

Here is my file content:

# Fastcgi configuration for PHP5
LoadModule fcgid_module modules/mod_fcgid.so
MaxRequestsPerProcess 500

<IfModule mod_fcgid.c>
MaxRequestsPerProcess 500
MaxProcessCount 15
DefaultMaxClassProcessCount 15
DefaultMinClassProcessCount 0
IPCConnectTimeout 60
IPCCommTimeout 60
PHP_Fix_Pathinfo_Enable 1
IdleTimeout 120
IdleScanInterval 30
BusyTimeout 120
BusyScanInterval 90
ErrorScanInterval 3
ZombieScanInterval 3
ProcessLifeTime 3600
</IfModule>

AddHandler fcgid-script .php5 .php4 .php .php3 .php2 .phtml
FCGIWrapper /usr/local/cpanel/cgi-sys/php5 .php5
FCGIWrapper /usr/local/cpanel/cgi-sys/php5 .php4
FCGIWrapper /usr/local/cpanel/cgi-sys/php5 .php
FCGIWrapper /usr/local/cpanel/cgi-sys/php5 .php3
FCGIWrapper /usr/local/cpanel/cgi-sys/php5 .php2
FCGIWrapper /usr/local/cpanel/cgi-sys/php5 .phtml

Please remember, that this file will be changed when apache is updated. I tried placing these configuration values in include file "pre-virtualhost" and it did not work from there. I would probably work if I would put all these lines into pre-main include.


Anton.

 

[Spiral]

Anton_latvia, I hate to be the one to point out the blatantly obvious but you quoted a quote that is obsolete and well over a year old ....

Back when this thread first started 2 years ago on up through the following year, there was a period when there was some very serious problems and complications with FastCGI and the security was also lacking as well and that continued through much of the following year.

Both FastCGI and FCGI (two different but similar things incidentally) have evolved greatly since that time and if you actually read the rest of the thread you would have noticed some of the later conversation.

Today, FCGI is quite different than it used to be, many of the problems worked out, and the security roughly identical to that of SuPHP making it an actual viable solution now in certain instances where it would not of been way back when this thread first originally started.

Helpful tip: Read the date stamps on what you reply to!

 

[anton_latvia]

hehe, well, I looked on the last post, which was posted in March this year.. Sorry for bumping the old topic..

 

[Spiral]

Hehe :p Don't feel bad. I do the same myself every once in a while. :D