The Community Forums

Interact with an entire community of cPanel & WHM users!
  1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

apache failing every 3 hours

Discussion in 'EasyApache' started by totalufo, Mar 11, 2002.

  1. totalufo

    totalufo Well-Known Member

    Joined:
    Jan 17, 2002
    Messages:
    160
    Likes Received:
    0
    Trophy Points:
    16
    This is the second server that started doing this. put up 2 new servers and both fail apache about every 3 hours. Any ideas?
     
  2. feanor

    feanor Well-Known Member

    Joined:
    Aug 13, 2001
    Messages:
    836
    Likes Received:
    0
    Trophy Points:
    16
    SYN flood perhaps against your entire network perhaps?
    Or you need to increase max connections on your webservers....

    Check /var/log/messages and netstat -an for hints on whom is assaulting you on port 80- that could definitely be a possibility. Otherwise your connections are maxing out and apache is hanging once it runs out of memory for everybody that you allowed it to listen for
     
  3. totalufo

    totalufo Well-Known Member

    Joined:
    Jan 17, 2002
    Messages:
    160
    Likes Received:
    0
    Trophy Points:
    16
    This guy did actually put about 400 domains on this server


    Here is the netstat -an
    sorry for the huge list

    Active Internet connections (servers and established)
    Proto Recv-Q Send-Q Local Address Foreign Address State
    tcp 0 0 0.0.0.0:2593 0.0.0.0:* LISTEN
    tcp 0 0 0.0.0.0:1 0.0.0.0:* LISTEN
    tcp 0 0 0.0.0.0:2082 0.0.0.0:* LISTEN
    tcp 0 0 0.0.0.0:2083 0.0.0.0:* LISTEN
    tcp 0 0 0.0.0.0:995 0.0.0.0:* LISTEN
    tcp 0 0 0.0.0.0:2084 0.0.0.0:* LISTEN
    tcp 0 0 0.0.0.0:2086 0.0.0.0:* LISTEN
    tcp 0 0 0.0.0.0:2087 0.0.0.0:* LISTEN
    tcp 0 0 0.0.0.0:3306 0.0.0.0:* LISTEN
    tcp 0 0 0.0.0.0:7786 0.0.0.0:* LISTEN
    tcp 0 0 0.0.0.0:6666 0.0.0.0:* LISTEN
    tcp 0 0 0.0.0.0:2092 0.0.0.0:* LISTEN
    tcp 0 0 0.0.0.0:110 0.0.0.0:* LISTEN
    tcp 0 0 0.0.0.0:2095 0.0.0.0:* LISTEN
    tcp 0 0 0.0.0.0:111 0.0.0.0:* LISTEN
    tcp 0 0 0.0.0.0:80 0.0.0.0:* LISTEN
    tcp 0 0 0.0.0.0:2096 0.0.0.0:* LISTEN
    tcp 0 0 0.0.0.0:465 0.0.0.0:* LISTEN
    tcp 0 0 209.51.131.174:53 0.0.0.0:* LISTEN
    tcp 0 0 209.51.131.173:53 0.0.0.0:* LISTEN
    tcp 0 0 209.51.131.172:53 0.0.0.0:* LISTEN
    tcp 0 0 209.51.131.171:53 0.0.0.0:* LISTEN
    tcp 0 0 209.51.131.170:53 0.0.0.0:* LISTEN
    tcp 0 0 127.0.0.1:53 0.0.0.0:* LISTEN
    tcp 0 0 0.0.0.0:21 0.0.0.0:* LISTEN
    tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN
    tcp 0 0 0.0.0.0:25 0.0.0.0:* LISTEN
    tcp 0 0 127.0.0.1:953 0.0.0.0:* LISTEN
    tcp 0 0 209.51.131.170:80 198.81.8.2:62590 TIME_WAIT
    tcp 0 8 209.51.131.170:2593 200.176.88.69:1346 ESTABLISHED
    tcp 0 0 209.51.131.170:80 198.81.8.3:52862 TIME_WAIT
    tcp 0 16 209.51.131.170:2593 200.210.249.102:1056 ESTABLISHED
    tcp 0 0 209.51.131.170:80 198.81.8.148:57364 TIME_WAIT
    tcp 0 0 209.51.131.170:2086 200.168.40.148:1026 ESTABLISHED
    tcp 0 6246 209.51.131.170:80 200.170.214.34:53246 CLOSING
    tcp 0 0 209.51.131.170:2082 200.168.40.148:1025 FIN_WAIT2
    tcp 0 0 209.51.131.170:2086 200.168.40.148:1025 TIME_WAIT
    tcp 0 0 209.51.131.170:21 172.187.33.26:1773 ESTABLISHED
    tcp 0 0 209.51.131.170:25 200.188.191.51:4575 TIME_WAIT
    tcp 0 0 209.51.131.170:2082 200.168.40.148:1032 TIME_WAIT
    tcp 0 0 209.51.131.170:2082 200.168.40.148:1033 TIME_WAIT
    tcp 0 0 209.51.131.170:80 200.168.40.148:1025 ESTABLISHED
    tcp 0 0 209.51.131.170:80 198.81.8.2:62897 TIME_WAIT
    tcp 0 0 209.51.131.170:2593 200.195.174.15:1372 ESTABLISHED
    tcp 0 0 209.51.131.170:80 198.81.8.1:21548 TIME_WAIT
    tcp 0 2882 209.51.131.170:80 200.151.0.235:1716 ESTABLISHED
    tcp 0 0 209.51.131.170:2593 200.216.10.97:1359 ESTABLISHED
    tcp 0 121 209.51.131.170:2082 200.195.123.42:2603 ESTABLISHED
    tcp 0 0 209.51.131.170:80 200.151.0.235:1715 ESTABLISHED
    tcp 0 4593 209.51.131.170:2082 200.195.123.42:2602 ESTABLISHED
    tcp 0 325 209.51.131.170:2082 200.195.123.42:2601 ESTABLISHED
    tcp 0 0 209.51.131.170:80 200.195.123.42:2593 TIME_WAIT
    tcp 0 148 209.51.131.170:2082 200.195.123.42:2600 ESTABLISHED
    tcp 0 0 209.51.131.170:80 198.81.8.1:21674 TIME_WAIT
    tcp 0 0 209.51.131.170:2082 200.195.123.42:2599 TIME_WAIT
    tcp 0 0 209.51.131.170:2082 200.195.123.42:2598 TIME_WAIT
    tcp 0 56 209.51.131.170:25 200.168.71.82:1025 ESTABLISHED
    tcp 0 0 209.51.131.170:2082 200.195.123.42:2597 TIME_WAIT
    tcp 0 0 209.51.131.170:2082 200.216.10.97:1540 FIN_WAIT2
    tcp 0 0 209.51.131.170:2082 200.195.123.42:2596 TIME_WAIT
    tcp 0 0 209.51.131.170:2082 200.195.123.42:2595 TIME_WAIT
    tcp 0 0 209.51.131.170:2082 200.195.123.42:2594 TIME_WAIT
    tcp 0 0 209.51.131.170:2082 200.216.10.97:1539 FIN_WAIT2
    tcp 0 0 209.51.131.170:80 198.81.8.2:62817 TIME_WAIT
    tcp 0 0 209.51.131.170:110 200.242.76.65:1029 TIME_WAIT
    tcp 0 0 209.51.131.170:2082 200.216.10.97:1537 FIN_WAIT2
    tcp 0 0 209.51.131.170:25 200.170.63.188:1043 TIME_WAIT
    tcp 0 0 209.51.131.170:2082 200.226.91.61:1481 TIME_WAIT
    tcp 0 11 209.51.131.170:2593 200.176.85.125:1047 ESTABLISHED
    tcp 0 0 209.51.131.170:2082 200.226.91.61:1483 TIME_WAIT
    tcp 0 0 209.51.131.170:80 198.81.8.1:21592 TIME_WAIT
    tcp 0 0 209.51.131.170:110 200.221.36.197:2243 TIME_WAIT
    tcp 0 0 209.51.131.170:2082 200.226.91.61:1487 TIME_WAIT
    tcp 0 0 209.51.131.170:80 198.81.9.3:28757 TIME_WAIT
    tcp 0 143 209.51.131.170:2593 172.187.88.45:1034 ESTABLISHED
    tcp 0 2 209.51.131.170:2593 200.193.165.42:1037 ESTABLISHED
    tcp 0 0 209.51.131.170:80 198.81.9.3:28499 TIME_WAIT
    tcp 0 2 209.51.131.170:2593 200.193.165.42:1038 ESTABLISHED
    tcp 0 0 209.51.131.170:110 200.220.32.96:2568 TIME_WAIT
    tcp 0 0 209.51.131.170:80 198.81.9.2:53901 TIME_WAIT
    tcp 0 4512 209.51.131.170:22 4.64.25.151:4903 ESTABLISHED
    tcp 0 59 209.51.131.170:2593 200.191.188.171:1101 ESTABLISHED
    tcp 0 0 209.51.131.170:2082 200.226.91.61:1489 ESTABLISHED
    tcp 0 0 209.51.131.170:2082 200.226.91.61:1488 TIME_WAIT
    tcp 0 0 209.51.131.170:2082 200.226.91.61:1491 TIME_WAIT
    tcp 0 0 209.51.131.170:2095 200.178.101.145:6318 ESTABLISHED
    tcp 0 0 209.51.131.170:2082 200.226.91.61:1490 ESTABLISHED
    tcp 0 0 209.51.131.170:25 200.231.206.186:37106 ESTABLISHED
    tcp 0 0 209.51.131.170:80 198.81.8.2:62658 TIME_WAIT
    tcp 0 0 209.51.131.170:80 198.81.9.3:28803 TIME_WAIT
    tcp 0 0 209.51.131.170:110 200.220.32.96:2564 TIME_WAIT
    udp 0 0 0.0.0.0:34339 0.0.0.0:*
    udp 0 0 209.51.131.174:53 0.0.0.0:*
    udp 0 0 209.51.131.173:53 0.0.0.0:*
    udp 0 0 209.51.131.172:53 0.0.0.0:*
    udp 0 0 209.51.131.171:53 0.0.0.0:*
    udp 0 0 209.51.131.170:53 0.0.0.0:*
    udp 0 0 127.0.0.1:53 0.0.0.0:*
    Active UNIX domain sockets (servers and established)
    Proto RefCnt Flags Type State I-Node Path
    unix 2 [ ACC ] STREAM LISTENING 72713 /usr/local/cpanel/3rdparty/interchange/etc/socket
    unix 13 [ ] DGRAM 1195 /dev/log
    unix 2 [ ACC ] STREAM LISTENING 1676769 /var/lib/mysql/mysql.sock
    unix 2 [ ACC ] STREAM LISTENING 72711 /usr/local/cpanel/3rdparty/interchange/etc/socket.ipc
    unix 2 [ ] DGRAM 1682232
    unix 2 [ ] DGRAM 1665283
    unix 2 [ ] DGRAM 72684
    unix 2 [ ] DGRAM 72682
    unix 2 [ ] DGRAM 72677
    unix 2 [ ] DGRAM 72658
    unix 2 [ ] DGRAM 72655
    unix 2 [ ] DGRAM 1947
    unix 2 [ ] DGRAM 1420
    unix 2 [ ] DGRAM 1364
    unix 2 [ ] DGRAM 1204
    unix 2 [ ] STREAM CONNECTED 633
     
  4. feanor

    feanor Well-Known Member

    Joined:
    Aug 13, 2001
    Messages:
    836
    Likes Received:
    0
    Trophy Points:
    16
    Not helpful really.
    You need to do some more of that combined with tailing apache logs, and analyzing bandwidth to determine who is actually doing damage, and against what specifically.

    Or you definitely need to increase MaxClients within your httpd.conf(s), if that is part or all of the issue.

    Or....... ?
    Just need to dig up some more details, it appears.
     
  5. totalufo

    totalufo Well-Known Member

    Joined:
    Jan 17, 2002
    Messages:
    160
    Likes Received:
    0
    Trophy Points:
    16
    ok, I stuck a monitor on this server and tailed the /var/log/messages when apache failed

    Here's what I got. I think you might be right

    Mar 12 03:10:45 localhost stunnel[8543]: stunnel 3.22 on i686-pc-linux-gnu PTHREAD+LIBWRAP with OpenSSL 0.9.5a 1 Apr 2000
    Mar 12 03:10:45 localhost stunnel[8545]: Using '127.0.0.1.2095' as tcpwrapper service name
    Mar 12 03:10:45 localhost stunnel[8544]: FD_SETSIZE=16384, file ulimit=1024 -& 500 clients allowed
    Mar 12 03:10:45 localhost stunnel[8542]: FD_SETSIZE=16384, file ulimit=1024 -& 500 clients allowed
    Mar 12 03:10:45 localhost stunnel[8545]: stunnel 3.22 on i686-pc-linux-gnu PTHREAD+LIBWRAP with OpenSSL 0.9.5a 1 Apr 2000
    Mar 12 03:10:45 localhost stunnel[8546]: FD_SETSIZE=16384, file ulimit=1024 -& 500 clients allowed
     
  6. feanor

    feanor Well-Known Member

    Joined:
    Aug 13, 2001
    Messages:
    836
    Likes Received:
    0
    Trophy Points:
    16
    /etc/rc.d/init.d/filelimits start &
    Do that a couple times actually....

    And then increase your MaxClients and StartServers in your httpd.conf to something reasonable.

    Should be golden then.
    ;)
     
  7. totalufo

    totalufo Well-Known Member

    Joined:
    Jan 17, 2002
    Messages:
    160
    Likes Received:
    0
    Trophy Points:
    16
    Here's what I get.

    Increasing file system limits [ OK ]
    /etc/rc.d/init.d/filelimits: /proc/sys/fs/dquot-max: No such file or directory
     
  8. feanor

    feanor Well-Known Member

    Joined:
    Aug 13, 2001
    Messages:
    836
    Likes Received:
    0
    Trophy Points:
    16
    Ah you must have RH 7.2
    Since we've had the ability for that OS version on cpanel machines..... filelimits seem to be controlled elsewhere.....

    I am hoping someone else on the forum may have some input on this, as this is something I have always taken for granted- having that init script I always relied upon it until now.

    Anyone else have some insight on the most efficient ways to increase file descriptors within RH 7.2 ?


    Thanks... :)
     
  9. totalufo

    totalufo Well-Known Member

    Joined:
    Jan 17, 2002
    Messages:
    160
    Likes Received:
    0
    Trophy Points:
    16
    After increasing the maxx connections to the highest/safest levels, it still fails. It has to be something else as well
     
  10. bdraco

    bdraco Guest

    [quote:03ff85730e][i:03ff85730e]Originally posted by totalufo[/i:03ff85730e]

    After increasing the maxx connections to the highest/safest levels, it still fails. It has to be something else as well[/quote:03ff85730e]

    Check the /usr/local/apache/logs/error_log
     
  11. totalufo

    totalufo Well-Known Member

    Joined:
    Jan 17, 2002
    Messages:
    160
    Likes Received:
    0
    Trophy Points:
    16
    Here's what I get during a fail

    [Sat Mar 2 12:55:36 2002] [warn] Loaded DSO libexec/mod_env.so uses plain Apache 1.3 API, this module might c$
    [Sat Mar 2 12:55:36 2002] [warn] Loaded DSO libexec/mod_log_config.so uses plain Apache 1.3 API, this module $
    [Sat Mar 2 12:55:36 2002] [warn] Loaded DSO libexec/mod_mime.so uses plain Apache 1.3 API, this module might $
    [Sat Mar 2 12:55:36 2002] [warn] Loaded DSO libexec/mod_negotiation.so uses plain Apache 1.3 API, this module$
    [Sat Mar 2 12:55:36 2002] [warn] Loaded DSO libexec/mod_status.so uses plain Apache 1.3 API, this module migh$
    [Sat Mar 2 12:55:36 2002] [warn] Loaded DSO libexec/mod_include.so uses plain Apache 1.3 API, this module mig$
    [Sat Mar 2 12:55:36 2002] [warn] Loaded DSO libexec/mod_autoindex.so uses plain Apache 1.3 API, this module m$
    [Sat Mar 2 12:55:36 2002] [warn] Loaded DSO libexec/mod_dir.so uses plain Apache 1.3 API, this module might c$
    [Sat Mar 2 12:55:36 2002] [warn] Loaded DSO libexec/mod_cgi.so uses plain Apache 1.3 API, this module might c$
    [Sat Mar 2 12:55:36 2002] [warn] Loaded DSO libexec/mod_asis.so uses plain Apache 1.3 API, this module might $
    [Sat Mar 2 12:55:36 2002] [warn] Loaded DSO libexec/mod_imap.so uses plain Apache 1.3 API, this module might $
    [Sat Mar 2 12:55:36 2002] [warn] Loaded DSO libexec/mod_actions.so uses plain Apache 1.3 API, this module mig$
    [Sat Mar 2 12:55:36 2002] [warn] Loaded DSO libexec/mod_userdir.so uses plain Apache 1.3 API, this module mig$
    [Sat Mar 2 12:55:36 2002] [warn] Loaded DSO libexec/mod_alias.so uses plain Apache 1.3 API, this module might$
    [Sat Mar 2 12:55:36 2002] [warn] Loaded DSO libexec/mod_access.so uses plain Apache 1.3 API, this module migh$
     
  12. totalufo

    totalufo Well-Known Member

    Joined:
    Jan 17, 2002
    Messages:
    160
    Likes Received:
    0
    Trophy Points:
    16
    Ah HAH! Found the problem so it seams. Now just a matter of fixing it.

    [Wed Mar 13 01:00:05 2002] [warn] Loaded DSO libexec/mod_userdir.so uses plain Apache 1.3 API, this module might crash under EAPI! (please recompile it with -DEAPI)

    Here's the entire tail of the error log

    [Wed Mar 13 01:00:05 2002] [warn] Loaded DSO libexec/mod_autoindex.so uses plain Apache 1.3 API, this module might crash under EAPI! (please recompile it with -DEAPI)
    [Wed Mar 13 01:00:05 2002] [warn] Loaded DSO libexec/mod_dir.so uses plain Apache 1.3 API, this module might crash under EAPI! (please recompile it with -DEAPI)
    [Wed Mar 13 01:00:05 2002] [warn] Loaded DSO libexec/mod_cgi.so uses plain Apache 1.3 API, this module might crash under EAPI! (please recompile it with -DEAPI)
    [Wed Mar 13 01:00:05 2002] [warn] Loaded DSO libexec/mod_asis.so uses plain Apache 1.3 API, this module might crash under EAPI! (please recompile it with -DEAPI)
    [Wed Mar 13 01:00:05 2002] [warn] Loaded DSO libexec/mod_imap.so uses plain Apache 1.3 API, this module might crash under EAPI! (please recompile it with -DEAPI)
    [Wed Mar 13 01:00:05 2002] [warn] Loaded DSO libexec/mod_actions.so uses plain Apache 1.3 API, this module might crash under EAPI! (please recompile it with -DEAPI)
    [Wed Mar 13 01:00:05 2002] [warn] Loaded DSO libexec/mod_userdir.so uses plain Apache 1.3 API, this module might crash under EAPI! (please recompile it with -DEAPI)
    [Wed Mar 13 01:00:05 2002] [warn] Loaded DSO libexec/mod_alias.so uses plain Apache 1.3 API, this module might crash under EAPI! (please recompile it with -DEAPI)
    httpd: module &mod_alias.c& could not be loaded, because the dynamic
    module limit was reached. Please increase DYNAMIC_MODULE_LIMIT and recompile.
     
  13. bdraco

    bdraco Guest

    [quote:0789a1ea2f][i:0789a1ea2f]Originally posted by totalufo[/i:0789a1ea2f]

    Ah HAH! Found the problem so it seams. Now just a matter of fixing it.

    [Wed Mar 13 01:00:05 2002] [warn] Loaded DSO libexec/mod_userdir.so uses plain Apache 1.3 API, this module might crash under EAPI! (please recompile it with -DEAPI)

    Here's the entire tail of the error log

    [Wed Mar 13 01:00:05 2002] [warn] Loaded DSO libexec/mod_autoindex.so uses plain Apache 1.3 API, this module might crash under EAPI! (please recompile it with -DEAPI)
    [Wed Mar 13 01:00:05 2002] [warn] Loaded DSO libexec/mod_dir.so uses plain Apache 1.3 API, this module might crash under EAPI! (please recompile it with -DEAPI)
    [Wed Mar 13 01:00:05 2002] [warn] Loaded DSO libexec/mod_cgi.so uses plain Apache 1.3 API, this module might crash under EAPI! (please recompile it with -DEAPI)
    [Wed Mar 13 01:00:05 2002] [warn] Loaded DSO libexec/mod_asis.so uses plain Apache 1.3 API, this module might crash under EAPI! (please recompile it with -DEAPI)
    [Wed Mar 13 01:00:05 2002] [warn] Loaded DSO libexec/mod_imap.so uses plain Apache 1.3 API, this module might crash under EAPI! (please recompile it with -DEAPI)
    [Wed Mar 13 01:00:05 2002] [warn] Loaded DSO libexec/mod_actions.so uses plain Apache 1.3 API, this module might crash under EAPI! (please recompile it with -DEAPI)
    [Wed Mar 13 01:00:05 2002] [warn] Loaded DSO libexec/mod_userdir.so uses plain Apache 1.3 API, this module might crash under EAPI! (please recompile it with -DEAPI)
    [Wed Mar 13 01:00:05 2002] [warn] Loaded DSO libexec/mod_alias.so uses plain Apache 1.3 API, this module might crash under EAPI! (please recompile it with -DEAPI)
    httpd: module &mod_alias.c& could not be loaded, because the dynamic
    module limit was reached. Please increase DYNAMIC_MODULE_LIMIT and recompile.[/quote:0789a1ea2f]

    copy all the vhosts to a diffrent file.. move the httpd.conf to somewhere else reinstall buildapache.sea. Put the vhosts on the bottom of the new httpd.conf it creates. restart httpd.
     
Loading...

Share This Page