LinkBack URL
About LinkBacks
apache crashes every 2 hrs
I have been facing probs with apache from the time i upgraded using the new buildapache.sea. After upgrading, every 2 hrs the apache crashes. Well it doesn't actually crash, it stops accepting any new connections, and stays in the memory. Chkservd is not able to detect it and thus apache remains suspended in memory till the time its manually stopped and started back up. Now while this 2hrs approach, the /usr/local/apache/logs/error_logs starts getting filled with
&[warn] [csd] filedescriptor (1032) larger than FD_SETSIZE (1024) found, you probably need to rebuild Apache with a larger FD_SETSIZE&
Now the moment the above messages comes, apache stops accepting any new connections whcih basically means that its exhausted of the file descriptors. Now when i try to look around on the net and on cpanel forums that a possible solution would be to run
&/etc/rc.d/init.d/filelimits start&&
But unfortunately this doesn't work as my machine is running on RH7.2 which doesn't have the &/proc/sys/fs/dquot-max: No such file or directory&. Again i edited the httpd startup script to allocate more FD's and i comment the already ulimit lines on the top which are as follows:
#ulimit -n 1024
#ulimit -n 4096
#ulimit -n 8192
[I don't understand the ulimit being used 3 times inside the startup script, doesn't make sense to me]
Now instead of these i use &ulimit -HSn 8192& as a replacement to assign more FD's.
To check the above working whether this gives more FD's i put in &ulimit -a& before and after the above command to know the previous and the later states. Heres the output
Previous state (when httpd is not running)
core file size (blocks) 1000000
data seg size (kbytes) unlimited
file size (blocks) unlimited
max locked memory (kbytes) unlimited
max memory size (kbytes) unlimited
open files 4096
pipe size (512 bytes) 8
stack size (kbytes) 8192
cpu time (seconds) unlimited
max user processes 14335
virtual memory (kbytes) unlimited
State after the ulimit command is executed and httpd is running
core file size (blocks) 1000000
data seg size (kbytes) unlimited
file size (blocks) unlimited
max locked memory (kbytes) unlimited
max memory size (kbytes) unlimited
open files 8192
pipe size (512 bytes) 8
stack size (kbytes) 8192
cpu time (seconds) unlimited
max user processes 14335
virtual memory (kbytes) unlimited
As you can see the open files increases from 4096 to 8192, but still the problem remains, after 2 hrs, apache again went through the same probs and i had to restart it manually.
Now there is one more thing, after this when httpd is already running, if i issue a stop using /etc/rc.d/init.d/httpd stop, the output of ulimit is as follows:
While httpd is running in memory
core file size (blocks) 1000000
data seg size (kbytes) unlimited
file size (blocks) unlimited
max locked memory (kbytes) unlimited
max memory size (kbytes) unlimited
open files 4096
pipe size (512 bytes) 8
stack size (kbytes) 8192
cpu time (seconds) unlimited
max user processes 14335
virtual memory (kbytes) unlimited
Notice the open files, this means that after the previous startup, number of open files was reset back to 4096. Any ideas on this ?
And any possible solutions on how to increase the FD's ???
Now at present i did a temporary solution for this.
Made a shell script restartapache.sh with the following contents:
killall -9 httpd;/etc/rc.d/init.d/httpd stop;/etc/rc.d/init.d/httpd startssl;/usr/local/apache/bin/apachectl startssl;/etc/rc.d/init.d/httpd****start;/usr/local/apache/bin/apachectl start
Now through cron i run it every hr, so the situation of the FD doesn't arise. But there is one probs which i faced with this also. After every 2 days the apache again has the same probs and then i have to reboot my machine in order to get apache up as even a stop and start apache doesn't work properly.
Any help would really be appreciated.
regards,
Anand
Reply With Quote
