The Community Forums

Interact with an entire community of cPanel & WHM users!
  1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

apache crashes every 2 hrs

Discussion in 'EasyApache' started by anand, Jul 1, 2002.

  1. anand

    anand Well-Known Member

    Joined:
    Nov 11, 2002
    Messages:
    1,435
    Likes Received:
    1
    Trophy Points:
    38
    Location:
    India
    cPanel Access Level:
    DataCenter Provider
    I have been facing probs with apache from the time i upgraded using the new buildapache.sea. After upgrading, every 2 hrs the apache crashes. Well it doesn't actually crash, it stops accepting any new connections, and stays in the memory. Chkservd is not able to detect it and thus apache remains suspended in memory till the time its manually stopped and started back up. Now while this 2hrs approach, the /usr/local/apache/logs/error_logs starts getting filled with
    &[warn] [csd] filedescriptor (1032) larger than FD_SETSIZE (1024) found, you probably need to rebuild Apache with a larger FD_SETSIZE&

    Now the moment the above messages comes, apache stops accepting any new connections whcih basically means that its exhausted of the file descriptors. Now when i try to look around on the net and on cpanel forums that a possible solution would be to run
    &/etc/rc.d/init.d/filelimits start&&

    But unfortunately this doesn't work as my machine is running on RH7.2 which doesn't have the &/proc/sys/fs/dquot-max: No such file or directory&. Again i edited the httpd startup script to allocate more FD's and i comment the already ulimit lines on the top which are as follows:

    #ulimit -n 1024
    #ulimit -n 4096
    #ulimit -n 8192

    [I don't understand the ulimit being used 3 times inside the startup script, doesn't make sense to me]

    Now instead of these i use &ulimit -HSn 8192& as a replacement to assign more FD's.
    To check the above working whether this gives more FD's i put in &ulimit -a& before and after the above command to know the previous and the later states. Heres the output

    Previous state (when httpd is not running)

    core file size (blocks) 1000000
    data seg size (kbytes) unlimited
    file size (blocks) unlimited
    max locked memory (kbytes) unlimited
    max memory size (kbytes) unlimited
    open files 4096
    pipe size (512 bytes) 8
    stack size (kbytes) 8192
    cpu time (seconds) unlimited
    max user processes 14335
    virtual memory (kbytes) unlimited

    State after the ulimit command is executed and httpd is running

    core file size (blocks) 1000000
    data seg size (kbytes) unlimited
    file size (blocks) unlimited
    max locked memory (kbytes) unlimited
    max memory size (kbytes) unlimited
    open files 8192
    pipe size (512 bytes) 8
    stack size (kbytes) 8192
    cpu time (seconds) unlimited
    max user processes 14335
    virtual memory (kbytes) unlimited

    As you can see the open files increases from 4096 to 8192, but still the problem remains, after 2 hrs, apache again went through the same probs and i had to restart it manually.

    Now there is one more thing, after this when httpd is already running, if i issue a stop using /etc/rc.d/init.d/httpd stop, the output of ulimit is as follows:

    While httpd is running in memory

    core file size (blocks) 1000000
    data seg size (kbytes) unlimited
    file size (blocks) unlimited
    max locked memory (kbytes) unlimited
    max memory size (kbytes) unlimited
    open files 4096
    pipe size (512 bytes) 8
    stack size (kbytes) 8192
    cpu time (seconds) unlimited
    max user processes 14335
    virtual memory (kbytes) unlimited

    Notice the open files, this means that after the previous startup, number of open files was reset back to 4096. Any ideas on this ?

    And any possible solutions on how to increase the FD's ???

    Now at present i did a temporary solution for this.

    Made a shell script restartapache.sh with the following contents:

    killall -9 httpd;/etc/rc.d/init.d/httpd stop;/etc/rc.d/init.d/httpd startssl;/usr/local/apache/bin/apachectl startssl;/etc/rc.d/init.d/httpd start;/usr/local/apache/bin/apachectl start

    Now through cron i run it every hr, so the situation of the FD doesn't arise. But there is one probs which i faced with this also. After every 2 days the apache again has the same probs and then i have to reboot my machine in order to get apache up as even a stop and start apache doesn't work properly.

    Any help would really be appreciated.

    regards,

    Anand
     
  2. feanor

    feanor Well-Known Member

    Joined:
    Aug 13, 2001
    Messages:
    836
    Likes Received:
    0
    Trophy Points:
    16
    Did you build apache with the module that raises file descriptors, automatically? I think this works quite well with RH 7.+ machines.....

    I think CPanel's buildapache, in advanced mode, give you the option to add that to the webserver configuration. I think this is probably the least painful solution.
     
  3. anand

    anand Well-Known Member

    Joined:
    Nov 11, 2002
    Messages:
    1,435
    Likes Received:
    1
    Trophy Points:
    38
    Location:
    India
    cPanel Access Level:
    DataCenter Provider
    [quote:e0fb62ca7f][i:e0fb62ca7f]Originally posted by feanor[/i:e0fb62ca7f]


    Did you build apache with the module that raises file descriptors, automatically? I think this works quite well with RH 7.+ machines.....

    I think CPanel's buildapache, in advanced mode, give you the option to add that to the webserver configuration. I think this is probably the least painful solution.[/quote:e0fb62ca7f]

    Inside the cpanel buildapache.sea, in advanced mode following choices are there:

    [ ] Attach Module
    [ ] Mysql Auth Module
    [ ] ~user bandwidth leech protection
    [ ] Bytes Logging Module
    [ ] Dav Module
    [ ] Expires Module
    [*] Frontpage Module
    [*] Raise HARD_SERVER_LIMIT
    [*] Perl Module
    Php Module ---&
    [*] Rewrite Module
    [ ] Roaming Module (for netscape 4.5+)
    [*] SSL Module
    [*] suEXEC Module

    I don't see any file descriptors option. Please correct me if i am wrong.

    regards,

    Anand
     
  4. feanor

    feanor Well-Known Member

    Joined:
    Aug 13, 2001
    Messages:
    836
    Likes Received:
    0
    Trophy Points:
    16
    Make sure that is actually checked:

    [*] Raise HARD_SERVER_LIMIT



    That's what I was referring to.......
    Apologies for the confusion.
     
  5. anand

    anand Well-Known Member

    Joined:
    Nov 11, 2002
    Messages:
    1,435
    Likes Received:
    1
    Trophy Points:
    38
    Location:
    India
    cPanel Access Level:
    DataCenter Provider
    [quote:5c89bf2198][i:5c89bf2198]Originally posted by feanor[/i:5c89bf2198]


    Make sure that is actually checked:

    [*] Raise HARD_SERVER_LIMIT



    That's what I was referring to.......
    Apologies for the confusion.[/quote:5c89bf2198]

    As you see in my post itself, its already checked. The star indicates that.

    Anyone else with any solutions ??

    regards,

    Anand
     
  6. dominet

    dominet Active Member

    Joined:
    Mar 23, 2002
    Messages:
    29
    Likes Received:
    0
    Trophy Points:
    1
    Im having same problem!!! how to upgrade apache what is that .sea thing?
     
  7. anand

    anand Well-Known Member

    Joined:
    Nov 11, 2002
    Messages:
    1,435
    Likes Received:
    1
    Trophy Points:
    38
    Location:
    India
    cPanel Access Level:
    DataCenter Provider
    [quote:66a6ea0805][i:66a6ea0805]Originally posted by dominet[/i:66a6ea0805]

    Im having same problem!!! how to upgrade apache what is that .sea thing?[/quote:66a6ea0805]

    Apache for cpanel is available from layer1.cpanel.net for download.

    Download the .sea file and follow these instructions:
    1. chmod +x buildapache.sea
    2. ./buildapache.sea
    3. Now you can choose between beginner and advanced, beginner retains all your current modules etc, advanced lets u choose between them.

    Now exit and save settings, and the compilation would take over. Your apache is ready...

    regards,

    Anand
     
  8. anand

    anand Well-Known Member

    Joined:
    Nov 11, 2002
    Messages:
    1,435
    Likes Received:
    1
    Trophy Points:
    38
    Location:
    India
    cPanel Access Level:
    DataCenter Provider
    sorry guys, but is there no one who can help on this ??? please ?

    its really got on my nerves...

    atleast someone can point me to the earlier version of buildapache.sea, even with the vulnerabilities whcih existed, i didn't have to hear my customers calls for apache being pulled down every now and then....

    regards,

    Anand
     
  9. feanor

    feanor Well-Known Member

    Joined:
    Aug 13, 2001
    Messages:
    836
    Likes Received:
    0
    Trophy Points:
    16
    What precise kernel version do you have on that box?

    This is a very isolated issue it seems- i can't find this anywhere, not related to RH 7.1 or beyond anyway.

    You might want to consider a kernel upgrade.

    If you roll back to 1.3.24 or sooner for apache, you will completely lose the peace of mind that you're safe from that nasty exploit.....
     
  10. anand

    anand Well-Known Member

    Joined:
    Nov 11, 2002
    Messages:
    1,435
    Likes Received:
    1
    Trophy Points:
    38
    Location:
    India
    cPanel Access Level:
    DataCenter Provider
    [quote:9ac6a8667d][i:9ac6a8667d]Originally posted by feanor[/i:9ac6a8667d]


    What precise kernel version do you have on that box?

    This is a very isolated issue it seems- i can't find this anywhere, not related to RH 7.1 or beyond anyway.

    You might want to consider a kernel upgrade.

    If you roll back to 1.3.24 or sooner for apache, you will completely lose the peace of mind that you're safe from that nasty exploit.....

    [/quote:9ac6a8667d]

    kernel upgrade ?

    I at present have 2.4.18-grsec-1.9.4 with RH 7.2, tell me if i still require one. This is done by burst.net where i host my server now.

    regards,

    Anand
     
  11. anand

    anand Well-Known Member

    Joined:
    Nov 11, 2002
    Messages:
    1,435
    Likes Received:
    1
    Trophy Points:
    38
    Location:
    India
    cPanel Access Level:
    DataCenter Provider
    [quote:04cb1b2d15][i:04cb1b2d15]Originally posted by feanor[/i:04cb1b2d15]

    This is a very isolated issue it seems- i can't find this anywhere, not related to RH 7.1 or beyond anyway.

    [/quote:04cb1b2d15]

    I was searching on the cpanel forums and came up with these:

    http://forums.cpanel.net/read.php?TID=3546

    A search for FD_SETSIZE also reveals that many other people have been facing these probs, but i couldn't find one with RH7.2

    :(

    regards,

    Anand
     
  12. Domenico

    Domenico Well-Known Member

    Joined:
    Aug 14, 2001
    Messages:
    362
    Likes Received:
    0
    Trophy Points:
    16
    Heh, it just happened to me half an hour ago and we have 7.2 running :(
     
  13. md201

    md201 Active Member

    Joined:
    Aug 13, 2001
    Messages:
    28
    Likes Received:
    0
    Trophy Points:
    1
    hi

    any success fixing the apache bug ?
     
  14. portman

    portman Well-Known Member

    Joined:
    May 23, 2002
    Messages:
    70
    Likes Received:
    0
    Trophy Points:
    6
    Here are the instuctions we follwed for each server running RH 7.2 with the the 2.4.9-31 and 2.4.7-10 kernels. We have had no issues and the process was flawless.

    1) cp -r /usr/local/apache/ /usr/local/apache.bak (create a backup)
    2) wget http://layer1.cpanel.net/buildapache.sea
    3) sh buildapache.sea
    - select advanced mode
    - select the following modules:
    Bytes Logging Module
    Dav Module
    Frontpage Module
    Raise HARD_SERVER_LIMIT
    ALL php modules except Curl SSL Support (will break on RH6.2)
    Rewrite module
    SSL Module
    suEXEC Module
    - exit
    - say 'Yes' to the &Do you want to save this apache configuation?&
    - It will now compile

    4) After it is done installing, restart apache.. it dies for somereason..
    5) Go into WebHost Manager and turn off suEXEC
    6) Edit /usr/local/lib/php.ini and add these 2 lines if they are not already there:
    register_globals = on
    safe_mode = off
    7) If there is something in php.ini already you may need to test it by trying to load an XMB forum or some other PHP page. If it is not working and there was other stuff in php.ini, do this:
    mv /usr/local/lib/php.ini /usr/local/lib/php.ini.bak
    create a new /usr/local/lib/php.ini with only the lines:
    register_globals = on
    safe_mode = off
    8) run /etc/rc.d/init.d/httpd restart
    9) If something goes wrong and you cannot fix it, just do:
    /etc/rc.d/init.d/httpd stop
    mv /usr/local/apache/ /usr/local/apache.broken
    mv /usr/local/apache.bak /usr/local/apache
    /etc/rc.d/init.d/httpd startssl
     
  15. moronhead

    moronhead Well-Known Member

    Joined:
    Aug 12, 2001
    Messages:
    706
    Likes Received:
    0
    Trophy Points:
    16
    Portman thanks for your safe upgrade tips. 3 questions:

    * Why would I want to install the suEXEC module if I'll be switching it off later? Will installing it not create problems with mailman etc before I can switch it off?

    * Can we choose not to install any of the PHP modules offered, as we have a customized version running and we don't want to mess up with it.

    * If frontpage (or any of the other extensions such as dav) is already the latest version on the server is there any need to reinstall them?
     
  16. md201

    md201 Active Member

    Joined:
    Aug 13, 2001
    Messages:
    28
    Likes Received:
    0
    Trophy Points:
    1
    Error while running buildapache.sea

    Hi

    i receive following error while i run buildapache.sea and try to run it in advanced mode

    ========
    MANY MANY LINES BEFORE THIS .....

    /usr/include/bits/stdio.h: In function `getc_unlocked':
    /usr/include/bits/stdio.h:52: dereferencing pointer to incomplete type
    /usr/include/bits/stdio.h:52: dereferencing pointer to incomplete type
    /usr/include/bits/stdio.h:52: dereferencing pointer to incomplete type
    /usr/include/bits/stdio.h:53: warning: control reaches end of non-void function
    /usr/include/bits/stdio.h: In function `getchar_unlocked':
    /usr/include/bits/stdio.h:59: dereferencing pointer to incomplete type
    /usr/include/bits/stdio.h:59: dereferencing pointer to incomplete type
    /usr/include/bits/stdio.h:59: dereferencing pointer to incomplete type
    /usr/include/bits/stdio.h:60: warning: control reaches end of non-void function
    /usr/include/bits/stdio.h: In function `fputc_unlocked':
    /usr/include/bits/stdio.h:77: dereferencing pointer to incomplete type
    /usr/include/bits/stdio.h:77: dereferencing pointer to incomplete type
    /usr/include/bits/stdio.h:77: dereferencing pointer to incomplete type
    /usr/include/bits/stdio.h:78: warning: control reaches end of non-void function
    /usr/include/bits/stdio.h: In function `putc_unlocked':
    /usr/include/bits/stdio.h:87: dereferencing pointer to incomplete type
    /usr/include/bits/stdio.h:87: dereferencing pointer to incomplete type
    /usr/include/bits/stdio.h:87: dereferencing pointer to incomplete type
    /usr/include/bits/stdio.h:88: warning: control reaches end of non-void function
    /usr/include/bits/stdio.h: In function `putchar_unlocked':
    /usr/include/bits/stdio.h:94: dereferencing pointer to incomplete type
    /usr/include/bits/stdio.h:94: dereferencing pointer to incomplete type
    /usr/include/bits/stdio.h:94: dereferencing pointer to incomplete type
    /usr/include/bits/stdio.h:95: warning: control reaches end of non-void function
    /usr/include/bits/stdio.h: In function `feof_unlocked':
    /usr/include/bits/stdio.h:114: dereferencing pointer to incomplete type
    /usr/include/bits/stdio.h:115: warning: control reaches end of non-void function
    /usr/include/bits/stdio.h: In function `ferror_unlocked':
    /usr/include/bits/stdio.h:121: dereferencing pointer to incomplete type
    /usr/include/bits/stdio.h:122: warning: control reaches end of non-void function
    make: *** [lxdialog.o] Error 1
    =========

    after this it get stuck.

    can anyone please suggest why i am getting this error.

    i tried this on another server also but it gives me same error.

    i am running
    Red Hat Linux release 7.1 (Seawolf)

    kernal version of server is
    Linux xxxxxxxx.xxxxxxxxx.com 2.4.18-grsec-1.9.4 #2 Wed Apr 10 14:46:24 EDT 2002 i686 unknown

    please help. thank you.
     
  17. md201

    md201 Active Member

    Joined:
    Aug 13, 2001
    Messages:
    28
    Likes Received:
    0
    Trophy Points:
    1
    Nobody :(

    can anyone please help me or atleast give me some hint what is this error about !!!!
     
  18. Djelibeybi

    Djelibeybi Well-Known Member

    Joined:
    Mar 19, 2002
    Messages:
    55
    Likes Received:
    0
    Trophy Points:
    6
    Run /scripts/easyapache or /scripts/fixheaders -- your glibc headers are messed up.
     
  19. md201

    md201 Active Member

    Joined:
    Aug 13, 2001
    Messages:
    28
    Likes Received:
    0
    Trophy Points:
    1
    thank you Djelibeybi, it solved the problem.

    * happy soul * :)
     
Loading...

Share This Page