The Community Forums

Interact with an entire community of cPanel & WHM users!
  1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Apache tuning help: Server freezing during high load

Discussion in 'Workarounds and Optimization' started by houseoftech, Nov 10, 2010.

  1. houseoftech

    houseoftech Member

    Joined:
    Aug 30, 2007
    Messages:
    13
    Likes Received:
    0
    Trophy Points:
    1
    Hi,

    I'm hoping someone can help me with tuning Apache. My server always freezes up under heavy load when a single website makes a post somewhere linking to the site. It sends about 3,000 visitors in a 3-4 hour window. I have already checked the site for errors and tried to optimize it some.

    My server specs:

    Intel Xeon 3210 @ 2.13 GHz (4 CPUs)
    2 GB RAM

    I ran a script to watch the Apache connections during the high traffic time and this is the last thing it updated to before the server was rebooted.

    procs -----------memory---------- ---swap-- -----io---- --system-- -----cpu------
    r b swpd free buff cache si so bi bo in cs us sy id wa st
    1 12 1796692 9352 5572 21568 0 1 31 43 1 11 2 1 96 1 0
    Load
    21:11:58 up 7 days, 39 min, 1 user, load average: 60.07, 30.67, 12.54
    CTRL
    Apache Processes
    1
    Active Apache Conections
    64
    Apache Conections
    253
    SYN Conections
    11
    IPCS
    2

    Current Config settings in WHM:

    StartServers: 5
    MinSpareServers: 5
    MaxSpareServers: 10
    MaxClients: 150
    MaxRequestsPerChild: 5000

    PHP Config:

    memory_limit 256M
    max_execution_time 30

    -Corie
     
  2. RCraft

    RCraft Well-Known Member

    Joined:
    Nov 7, 2010
    Messages:
    52
    Likes Received:
    0
    Trophy Points:
    6
    Hello,

    I'm guessing this is a shared server. Add these to your apache config:

    Keepalive on
    Keepalivetimeout 1

    Also, you may wish to impose limits on cpu time, memory and maximum processes per user via the RLimit commands.

    I'm on my cell at the moment so I'll come back to this when I get home. Can you give us the output of the access log around the time this happens along with /var/log/messages and top (or ps)

    The more info we have the better we can assist.

    Edit: Here's an excellent resource I've found as well. If you aren't running prefork (static content) or worker (dynamic content) I'd recommend one of them depending on what you're running on your system.

    http://www.devside.net/articles/apache-performance-tuning
     
    #2 RCraft, Nov 10, 2010
    Last edited: Nov 10, 2010
  3. houseoftech

    houseoftech Member

    Joined:
    Aug 30, 2007
    Messages:
    13
    Likes Received:
    0
    Trophy Points:
    1
    The server is dedicated to me, and running 81 sites, but most get only up to 50 visits a day. Here are some clippings from various logs. I wasn't logging top, but you can see above the last unix load reported.

    I looked around in the httpd.conf file and it looks like cPanel is generating most of it. I added the KeepAlive directive to /usr/local/apache/conf/includes/post_virtualhost_global.conf

    /usr/local/apache/logs/access_log

    Code:
    127.0.0.1 - - [09/Nov/2010:21:07:54 -0500] "OPTIONS * HTTP/1.0" 200 -
    127.0.0.1 - - [09/Nov/2010:21:07:55 -0500] "OPTIONS * HTTP/1.0" 200 -
    127.0.0.1 - - [09/Nov/2010:21:07:56 -0500] "OPTIONS * HTTP/1.0" 200 -
    127.0.0.1 - - [09/Nov/2010:21:07:59 -0500] "OPTIONS * HTTP/1.0" 200 -
    127.0.0.1 - - [09/Nov/2010:21:08:05 -0500] "OPTIONS * HTTP/1.0" 200 -
    127.0.0.1 - - [09/Nov/2010:21:08:06 -0500] "OPTIONS * HTTP/1.0" 200 -
    127.0.0.1 - - [09/Nov/2010:21:08:09 -0500] "OPTIONS * HTTP/1.0" 200 -
    127.0.0.1 - - [09/Nov/2010:21:08:10 -0500] "OPTIONS * HTTP/1.0" 200 -
    127.0.0.1 - - [09/Nov/2010:21:08:11 -0500] "OPTIONS * HTTP/1.0" 200 -
    127.0.0.1 - - [09/Nov/2010:21:08:12 -0500] "OPTIONS * HTTP/1.0" 200 -
    127.0.0.1 - - [09/Nov/2010:21:08:13 -0500] "OPTIONS * HTTP/1.0" 200 -
    127.0.0.1 - - [09/Nov/2010:21:35:02 -0500] "GET /whm-server-status HTTP/1.0" 200 3878
    127.0.0.1 - - [09/Nov/2010:21:35:11 -0500] "GET / HTTP/1.0" 200 111  
    127.0.0.1 - - [09/Nov/2010:21:35:25 -0500] "OPTIONS * HTTP/1.0" 200 -
    127.0.0.1 - - [09/Nov/2010:21:35:26 -0500] "OPTIONS * HTTP/1.0" 200 -
    127.0.0.1 - - [09/Nov/2010:21:35:39 -0500] "OPTIONS * HTTP/1.0" 200 -
    domlog for the specific site

    Code:
    64.234.100.194 - - [09/Nov/2010:21:08:14 -0500] "GET base_mini_head_bg.gif HTTP/1.1" 200 96 "boxes.css" "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.19) Gecko/2010031422 Firefox/3.0.19 (.NET CLR 3.5.30729)"
    64.234.100.194 - - [09/Nov/2010:21:08:14 -0500] "GET footer-logo.jpg HTTP/1.1" 200 2145 "fleece.html?ref=nf" "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.19) Gecko/2010031422 Firefox/3.0.19 (.NET CLR 3.5.30729)"
    64.234.100.194 - - [09/Nov/2010:21:08:14 -0500] "GET icon_basket.gif HTTP/1.1" 200 604 "boxes.css" "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.19) Gecko/2010031422 Firefox/3.0.19 (.NET CLR 3.5.30729)"
    64.234.100.194 - - [09/Nov/2010:21:08:14 -0500] "GET bottom.png HTTP/1.1" 200 724 "style.css" "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.19) Gecko/2010031422 Firefox/3.0.19 (.NET CLR 3.5.30729)"
    xx.xx.xx.x - - [09/Nov/2010:21:08:02 -0500] "GET /index.php?page=header HTTP/1.1" 200 4947 "-" "PHP/5.2.9"
    166.205.15.223 - - [09/Nov/2010:21:08:07 -0500] "GET /shop/js/index.php?c=auto&f=,prototype/prototype.js,prototype/validation.js,scriptaculous/builder.js,scriptaculous/effects.js,scriptaculous/dragdrop.js,scriptaculous/controls.js,scriptaculous/slider.js,varien/js.js,varien/form.js,varien/menu.js,mage/translate.js,mage/cookies.js,varien/product.js,calendar/calendar.js,calendar/calendar-setup.js HTTP/1.1" 200 343304 "shirt-sale.html" "Mozilla/5.0 (iPhone; U; CPU iPhone OS 3_1_3 like Mac OS X; en-us) AppleWebKit/528.18 (KHTML, like Gecko) Version/4.0 Mobile/7E18 Safari/528.16"
    xx.xx.xx.x - - [09/Nov/2010:21:08:04 -0500] "GET /index.php?page=header HTTP/1.1" 200 4947 "-" "PHP/5.2.9"
    xx.xx.xx.x - - [09/Nov/2010:21:08:05 -0500] "GET /index.php?page=header HTTP/1.1" 200 4947 "-" "PHP/5.2.9"
    166.137.142.213 - - [09/Nov/2010:21:08:00 -0500] "GET /shop/index.php/fleece.html HTTP/1.1" 200 19783 "-" "Mozilla/5.0 (iPhone; U; CPU iPhone OS 4_1 like Mac OS X; en-us) AppleWebKit/532.9 (KHTML, like Gecko) Mobile/8B117"
    66.175.172.239 - - [09/Nov/2010:21:08:50 -0500] "GET /favicon.ico HTTP/1.1" 200 1150 "fleece.html" "Mozilla/5.0 (Linux; U; Android 2.1-update1; en-us; HTC Bravo_C Build/ERE27) AppleWebKit/530.17 (KHTML, like Gecko) Version/4.0 Mobile Safari/530.17"
    70.171.25.250 - - [09/Nov/2010:21:08:50 -0500] "GET /favicon.ico HTTP/1.1" 200 1150 "-" "Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/534.7 (KHTML, like Gecko) Chrome/7.0.517.44 Safari/534.7"
    xx.xx.xx.x - - [09/Nov/2010:21:08:18 -0500] "GET /index.php?page=shop_footer HTTP/1.1" 200 642 "-" "PHP/5.2.9"
    xx.xx.xx.x - - [09/Nov/2010:21:08:06 -0500] "GET /index.php?page=header HTTP/1.1" 200 4947 "-" "PHP/5.2.9"
    65.82.99.253 - - [09/Nov/2010:21:08:10 -0500] "GET /shop/ HTTP/1.1" 200 6559 "fleece.html" "Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_4; en-US) AppleWebKit/534.7 (KHTML, like Gecko) Chrome/7.0.517.44 Safari/534.7"
    xx.xx.xx.x - - [09/Nov/2010:21:08:18 -0500] "GET /index.php?page=header HTTP/1.1" 200 4948 "-" "PHP/5.2.9"
    xx.xx.xx.x - - [09/Nov/2010:21:08:21 -0500] "GET /index.php?page=header HTTP/1.1" 200 4948 "-" "PHP/5.2.9"
    xx.xx.xx.x - - [09/Nov/2010:21:35:09 -0500] "GET /index.php?page=header HTTP/1.1" 200 4946 "-" "PHP/5.2.9"
    xx.xx.xx.x - - [09/Nov/2010:21:35:13 -0500] "GET /index.php?page=header HTTP/1.1" 200 4946 "-" "PHP/5.2.9"

    /var/log/messages

    Code:
    Nov  9 19:36:29 s1 kernel: closeBackup: backup finished device major=8 minor=8
    Nov  9 21:31:51 s1 pure-ftpd: (?@217.79.182.213) [INFO] New connection from 217.79.182.213
    Nov  9 21:31:53 s1 pure-ftpd: (?@217.79.182.213) [INFO] Logout.
    Nov  9 21:46:05 s1 pure-ftpd: (?@141.28.122.101) [INFO] New connection from 141.28.122.101  
    Nov  9 21:46:05 s1 pure-ftpd: (?@141.28.122.101) [INFO] Logout.
    Nov  9 22:18:36 s1 pure-ftpd: (?@174.132.242.154) [INFO] New connection from 174.132.242.154
    Nov  9 22:18:42 s1 pure-ftpd: (?@174.132.242.154) [INFO] Logout.
    Nov  9 21:20:39 s1 named[2484]: client 68.171.232.24#47161: view external: error sending response: host unreachable
    Nov  9 21:20:57 s1 named[2484]: client 68.171.232.24#59241: view external: error sending response: host unreachable
    Nov  9 21:20:59 s1 named[2484]: client 205.152.37.39#55634: view external: error sending response: host unreachable
    Nov  9 21:21:08 s1 named[2484]: client 68.171.232.24#21083: view external: error sending response: host unreachable
    Nov  9 21:34:20 s1 syslogd 1.4.1: restart.
    -Corie
     
  4. houseoftech

    houseoftech Member

    Joined:
    Aug 30, 2007
    Messages:
    13
    Likes Received:
    0
    Trophy Points:
    1
    Here's the latest from last night's crash. I used sys-snap to log details. The post was made at 9:00 PM and by 9:01 the server was crawling.

    Code:
    Wed Nov 10 21:01:56 EST 2010
    37.19 12.74 4.66 1/526 1111
    MemTotal:      2059560 kB
    MemFree:          8144 kB
    Buffers:          5844 kB
    Cached:          20204 kB
    SwapCached:     197540 kB
    Active:        1223956 kB
    Inactive:	637744 kB
    HighTotal:           0 kB
    HighFree:            0 kB
    LowTotal:      2059560 kB
    LowFree:          8144 kB
    SwapTotal:     2040212 kB
    SwapFree:	542424 kB
    Dirty:            1792 kB
    Writeback:	 26684 kB
    AnonPages:     1684040 kB
    Mapped:          11368 kB
    Slab:            67844 kB
    PageTables:	 76520 kB
    NFS_Unstable:        0 kB
    Bounce:              0 kB
    CommitLimit:   3069992 kB
    Committed_AS:  5491868 kB
    VmallocTotal: 34359738367 kB
    VmallocUsed:    264340 kB
    VmallocChunk: 34359472975 kB
    HugePages_Total:     0
    HugePages_Free:      0
    HugePages_Rsvd:      0
    Hugepagesize:     2048 kB
    procs -----------memory---------- ---swap-- -----io---- --system-- -----cpu------
     r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa st
     0 45 1497788   8012   5844  20204    1    5    42    52   16   65  3  2 94  1  0
     2 46 1500404   8156   5836  19632 10244 11936 10284 12000 3423 6394  6  4  0 90  0
     1 81 1517996   8000   5480  19696 2172 18376  2232 18428 1313 2126  6 18  0 76  0
     1 64 1524468   9956   5476  19168 4160 8220  4200  8632 1424 2436  1  6  0 93  0
     1 62 1525748  11804   5484  19448 4980 4604  5096  4604 1436 2518  3  5  0 93  0
     2 56 1523484   9104   5452  19640 5520  252  5524   252 1455 2572  1  1  0 98  0
     1 62 1530908   8048   5248  19220 3888 9768  3932  9820 1364 2444  0 10  0 90  0
     3 74 1574628   9060   5140  18836 13612 49160 14024 49228 5432 9814  3  9  0 89  0
     2 77 1582900   8784   5164  18800 3088 9512  3148  9584 1301 2432  7  5  0 88  0
     3 64 1638904   8672   5324  18468 61816 82900 63432 83384 20791 37349  2  5  0 93  0
    
    It looks like the memory is being used up and dumping into the swap file.

    -Corie
     
  5. syslint

    syslint Well-Known Member

    Joined:
    Oct 9, 2006
    Messages:
    249
    Likes Received:
    6
    Trophy Points:
    18
    Location:
    India
    cPanel Access Level:
    Root Administrator
    Twitter:
    Can you give the result of
    netstat -pant | grep :80 | awk '{ print $5}' | cut -d: -f1 | sort | uniq -c | sort -n
     
  6. houseoftech

    houseoftech Member

    Joined:
    Aug 30, 2007
    Messages:
    13
    Likes Received:
    0
    Trophy Points:
    1
    1 0.0.0.0
    1 127.0.0.1
    10 75.216.175.57
     
    #6 houseoftech, Nov 11, 2010
    Last edited: Nov 11, 2010
  7. RCraft

    RCraft Well-Known Member

    Joined:
    Nov 7, 2010
    Messages:
    52
    Likes Received:
    0
    Trophy Points:
    6
    There's definitely something chewing up all the memory on your system. Run this command:

    ps -eo pmem,pcpu,pid,user,rss,vsize,args | { head -1 ; sort -k 1 -r -n ; } | head -10

    It'll show you the top 10 processes sorted. This may help narrow down what is causing it.

    Based on your low traffic it's definitely something specific to your system, maybe a particular script, that is being executed and memory leaking.

    If you can get into the system during this run the following command to free up the memory (this is non-destructive and no data will be lost): echo 3 > /proc/sys/vm/drop_caches

    It may save your system from a crash/reboot
     
  8. houseoftech

    houseoftech Member

    Joined:
    Aug 30, 2007
    Messages:
    13
    Likes Received:
    0
    Trophy Points:
    1
    %MEM %CPU PID USER RSS VSZ COMMAND
    6.5 1.9 12754 root 135120 205760 /usr/bin/perl /usr/local/assp/assp.pl /usr/local/assp
    2.5 0.2 11961 nobody 53144 222964 /usr/local/apache/bin/httpd -k start -DSSL
    1.9 0.4 8782 nobody 39224 208728 /usr/local/apache/bin/httpd -k start -DSSL
    1.8 0.2 10955 nobody 37876 207724 /usr/local/apache/bin/httpd -k start -DSSL
    1.4 0.0 11681 nobody 30396 199920 /usr/local/apache/bin/httpd -k start -DSSL
    1.3 0.4 13548 nobody 27352 196700 /usr/local/apache/bin/httpd -k start -DSSL
    1.1 0.1 13470 nobody 24112 193432 /usr/local/apache/bin/httpd -k start -DSSL
    0.9 0.0 11552 nobody 19172 188840 /usr/local/apache/bin/httpd -k start -DSSL
    0.8 0.0 5896 root 17836 184868 /usr/local/apache/bin/httpd -k start -DSSL

    System memory hovers around 50% all the time, until a heavy traffic spike comes in.
     
  9. RCraft

    RCraft Well-Known Member

    Joined:
    Nov 7, 2010
    Messages:
    52
    Likes Received:
    0
    Trophy Points:
    6
    Nothing stands out. Do you have KVM? If not, then the next time it crashes contact your datacenter and have them hook a console up to your server and see what's happening on the screen.
     
  10. houseoftech

    houseoftech Member

    Joined:
    Aug 30, 2007
    Messages:
    13
    Likes Received:
    0
    Trophy Points:
    1
    Am I just maybe allowing more connections than the server can handle?

    StartServers: 5
    MinSpareServers: 5
    MaxSpareServers: 10
    MaxClients: 150
    MaxRequestsPerChild: 5000

    Does MaxClients 150 mean serving pages to 150 different IPs that request them?
     
  11. RCraft

    RCraft Well-Known Member

    Joined:
    Nov 7, 2010
    Messages:
    52
    Likes Received:
    0
    Trophy Points:
    6
    No way. You should be able to handle much more traffic on that server.

    Decrease the maxrequestsperchild to 500. 5000 is a bit high and considering your memory leaking issues, 500 would be more suitable for the time being.

    Are you running fastcgi by chance? If so, I can give some more apache config recommendations based on that.

    Also, maxclients is the number of people that can access the server at any given time. Anyone over the first 150 people will just get stuck at a loading screen until a spot becomes available, so if anything it's a bit low. We use a maxclients of 2000 on our shared servers with hundreds of sites.
     
  12. houseoftech

    houseoftech Member

    Joined:
    Aug 30, 2007
    Messages:
    13
    Likes Received:
    0
    Trophy Points:
    1
    Thanks for all the help. I have lowered MaxRequestsPerChild to 500. What exactly is that?

    I'm trying to figure out how much memory each Apache process is using. This is from the log at time of crash.

    USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
    nobody 29312 0.4 0.4 224920 8672 ? D 20:56 0:01 \_ /usr/local/apache/bin/httpd -k start -DSSL
    nobody 29314 1.0 0.4 208696 10244 ? D 20:56 0:03 \_ /usr/local/apache/bin/httpd -k start -DSSL
    nobody 29315 0.6 0.8 213596 16616 ? S 20:56 0:02 \_ /usr/local/apache/bin/httpd -k start -DSSL
    nobody 29817 1.2 0.7 212792 16052 ? D 20:57 0:03 \_ /usr/local/apache/bin/httpd -k start -DSSL
    nobody 31735 1.6 0.6 213588 12948 ? D 20:59 0:02 \_ /usr/local/apache/bin/httpd -k start -DSSL
    nobody 32025 1.6 0.7 213584 15188 ? D 20:59 0:02 \_ /usr/local/apache/bin/httpd -k start -DSSL
    nobody 32041 2.1 0.8 213844 17300 ? S 20:59 0:03 \_ /usr/local/apache/bin/httpd -k start -DSSL
    nobody 32042 2.0 0.4 213844 8412 ? D 20:59 0:03 \_ /usr/local/apache/bin/httpd -k start -DSSL
    nobody 32242 1.0 0.6 212564 14300 ? S 21:00 0:01 \_ /usr/local/apache/bin/httpd -k start -DSSL
    nobody 32648 0.3 0.9 201244 18800 ? S 21:00 0:00 \_ /usr/local/apache/bin/httpd -k start -DSSL
    nobody 32649 0.5 0.8 199460 16704 ? D 21:00 0:00 \_ /usr/local/apache/bin/httpd -k start -DSSL

    0.6% of 2,048MB means each process is using 122MB? And some are as high as 1.2%. Does 244MB per process sound right?
     
  13. GaryT

    GaryT Well-Known Member

    Joined:
    May 19, 2010
    Messages:
    321
    Likes Received:
    3
    Trophy Points:
    16
    You should enable:

    This will show the account with the high load.

    See without this your going to have to dig extremely deep to find the exact cause, By finding the user is a very fast way to move forward on things.
     
  14. houseoftech

    houseoftech Member

    Joined:
    Aug 30, 2007
    Messages:
    13
    Likes Received:
    0
    Trophy Points:
    1
    Hi Gary,

    Apache suExec actually is on. I know which account the high traffic is coming from anyways.
     
  15. RCraft

    RCraft Well-Known Member

    Joined:
    Nov 7, 2010
    Messages:
    52
    Likes Received:
    0
    Trophy Points:
    6
    You're using far too much memory with apache. On average each apache process should be using maybe 15 -50 mb of RAM.

    As for maxrequestsperchild - that tells the web server how many requests can be made per each child process before it's killed off. Generally, a high setting is preferable, but when you have memory leaks it's best to set it to a lower value so that the process dies off before it can hog too much memory.

    Run easyapache and add mpm-prefork - this will significantly reduce your apache memory footprint.

    Set your apache conf to this:

    ======================

    <IfModule prefork.c>
    StartServers 2
    MinSpareServers 2
    MaxSpareServers 5
    MaxClients 150
    MaxRequestsPerChild 500
    </IfModule>

    LoadModule deflate_module modules/mod_deflate.so
    <Location />
    AddOutputFilterByType DEFLATE text/html text/plain text/css text/xml application/x-javascript
    </Location>

    KeepAlive On
    KeepAliveTimeout 2
    MaxKeepAliveRequests 80

    <Directory />
    Options FollowSymLinks
    </Directory>

    <Directory />
    AllowOverride None
    </Directory>

    ExtendedStatus Off

    Timeout 45

    ======================

    Also, I would recommend looking into nginx or litehttpd. I've been using nginx lately and highly recommend it.
     
  16. syslint

    syslint Well-Known Member

    Joined:
    Oct 9, 2006
    Messages:
    249
    Likes Received:
    6
    Trophy Points:
    18
    Location:
    India
    cPanel Access Level:
    Root Administrator
    Twitter:
    Do you have installed complex modsecurity ruels. If so try to disable those rules and then restart apache. I had same issue with mod security plugin and rules from gotroot
     
  17. houseoftech

    houseoftech Member

    Joined:
    Aug 30, 2007
    Messages:
    13
    Likes Received:
    0
    Trophy Points:
    1
    Sorry, that's a miscalculation. 0.6% is 0.006 * 2048, so the processes are using 12 to 45 MB of memory.

    I ran EasyApache and recompiled with MPM-Prefork and Mod Deflate as you suggested. This has significantly reduced the memory footprint of apache, and switched some of the load over to the CPU.

    I used J Meter to test a high load. What would a reasonable test be? I first tested 200 users * 2 pages within 60 seconds and the server handled it. Unix load went up to 8 (4 CPUs), but dropped quickly. I then tested 500 * 2 pages within 60 seconds and the server locked up. Here's some of the sys-snap log:

    Code:
    MemTotal:      2059560 kB
    MemFree:          8188 kB
    Buffers:          9492 kB
    Cached:          35288 kB
    SwapCached:     309484 kB
    Active:        1514732 kB
    Inactive:       337588 kB
    HighTotal:           0 kB
    HighFree:            0 kB
    LowTotal:      2059560 kB
    LowFree:          8188 kB
    SwapTotal:     2040212 kB
    SwapFree:        31240 kB
    Dirty:            4472 kB
    Writeback:        2252 kB
    AnonPages:     1592788 kB
    Mapped:          18004 kB
    Slab:            66420 kB
    PageTables:      77976 kB
    NFS_Unstable:        0 kB
    Bounce:              0 kB
    CommitLimit:   3069992 kB
    Committed_AS:  4956736 kB
    VmallocTotal: 34359738367 kB
    VmallocUsed:    264356 kB
    VmallocChunk: 34359472943 kB
    HugePages_Total:     0
    HugePages_Free:      0
    HugePages_Rsvd:      0
    Hugepagesize:     2048 kB
    procs -----------memory---------- ---swap-- -----io---- --system-- -----cpu------
     r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa st
     9 105 2008972   8304   9492  35288    7    6    41    44   20   44  2  1 96  1  0
    10 104 2009660  10128   9704  35272 3024 1792  3232  1852 1422  467 33 67  0  0  0
    11 104 2008736  14964   9908  35484 2952   20  3192    20 1319  460 33 67  0  0  0
    10 104 2007912  10208  10212  35728 3940  340  4456   340 1401  622 35 65  0  0  0
    10 106 2007796   8020  10264  35724 2648  684  2812   684 1379  439 33 67  0  0  0
    12 106 2009200   8216  10268  35720  988 1640  1100  1640 1414  241 35 65  0  0  0
    11 134 2021772   8620  10384  36752 11200 16156 12696 16640 10408 2840 33 67  0  0  0
    10 134 2022316   8060  10376  36608 2388 1136  2404  1136 1323  336 33 67  0  0  0
    10 133 2023708   8368  10380  36548 1480 1752  1484  1752 1446  293 32 68  0  0  0
    12 133 2025324   8040  10384  36724 2008 2080  2020  2080 1446  322 33 67  0  0  0
    USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND 
    nobody   26905  0.5  0.4 150616  8976 ?        D    08:21   0:34  \_ /usr/local/apache/bin/httpd -k start -DSSL
    nobody   30934  0.3  1.0 165356 22268 ?        D    08:27   0:21  \_ /usr/local/apache/bin/httpd -k start -DSSL
    nobody   10317  0.4  0.4 151184  9944 ?        D    08:43   0:20  \_ /usr/local/apache/bin/httpd -k start -DSSL
    nobody   25794  0.3  0.6 151372 13200 ?        D    09:03   0:13  \_ /usr/local/apache/bin/httpd -k start -DSSL
    nobody   25819  0.7  0.4 164964  9952 ?        D    09:03   0:27  \_ /usr/local/apache/bin/httpd -k start -DSSL
    nobody   30228  0.6  1.0 150632 21836 ?        D    09:10   0:19  \_ /usr/local/apache/bin/httpd -k start -DSSL
    nobody    7541 66.3  0.3 151060  6708 ?        R    09:23  26:10  \_ /usr/local/apache/bin/httpd -k start -DSSL
    nobody    7564  0.5  0.4 151116  9348 ?        D    09:23   0:12  \_ /usr/local/apache/bin/httpd -k start -DSSL
    nobody    8353  0.3  0.4 151132  9136 ?        D    09:24   0:07  \_ /usr/local/apache/bin/httpd -k start -DSSL
    nobody   14034 77.2  0.3 150808  6784 ?        R    09:31  24:32  \_ /usr/local/apache/bin/httpd -k start -DSSL
    nobody   14035  0.1  0.9 150772 20540 ?        S    09:31   0:01  \_ /usr/local/apache/bin/httpd -k start -DSSL
    nobody   14036  0.0  0.9 150776 18796 ?        D    09:31   0:01  \_ /usr/local/apache/bin/httpd -k start -DSSL
    nobody   14089  0.0  0.4 150772 10112 ?        D    09:31   0:01  \_ /usr/local/apache/bin/httpd -k start -DSSL
    nobody   14100  0.0  0.4 150772  9908 ?        D    09:31   0:01  \_ /usr/local/apache/bin/httpd -k start -DSSL
    nobody   14101  0.0  0.4 150808  8924 ?        D    09:31   0:01  \_ /usr/local/apache/bin/httpd -k start -DSSL
    nobody   14104 76.9  0.3 150772  6884 ?        R    09:31  24:24  \_ /usr/local/apache/bin/httpd -k start -DSSL
    nobody   14106  0.0  0.4 150804  9828 ?        D    09:31   0:01  \_ /usr/local/apache/bin/httpd -k start -DSSL
    nobody   14108  0.0  0.4 150772  9940 ?        D    09:31   0:01  \_ /usr/local/apache/bin/httpd -k start -DSSL
    nobody   14126 81.4  0.3 150772  6884 ?        R    09:31  25:48  \_ /usr/local/apache/bin/httpd -k start -DSSL
    nobody   14127  0.0  0.4 150772  9776 ?        D    09:31   0:01  \_ /usr/local/apache/bin/httpd -k start -DSSL
    nobody   14128  0.0  0.9 150772 19992 ?        D    09:31   0:01  \_ /usr/local/apache/bin/httpd -k start -DSSL
    nobody   14129  0.0  0.4 150772  9748 ?        D    09:31   0:01  \_ /usr/local/apache/bin/httpd -k start -DSSL
    nobody   14218  1.4  1.2 150808 26424 ?        D    09:31   0:27  \_ /usr/local/apache/bin/httpd -k start -DSSL
    nobody   14221  0.0  0.7 150772 16000 ?        S    09:31   0:01  \_ /usr/local/apache/bin/httpd -k start -DSSL
    nobody   14222  0.0  0.8 138768 17628 ?        D    09:31   0:00  \_ /usr/local/apache/bin/httpd -k start -DSSL
    nobody   14223 53.5  0.3 150508  6884 ?        R    09:31  16:56  \_ /usr/local/apache/bin/httpd -k start -DSSL
    nobody   14225  0.0  0.4 140048  9444 ?        D    09:31   0:00  \_ /usr/local/apache/bin/httpd -k start -DSSL
    
    What exactly is supposed to happen when MaxClients is reached? Should an error page be served saying "Server at capacity, try again later?"
     
    #17 houseoftech, Nov 14, 2010
    Last edited: Nov 14, 2010
  18. RCraft

    RCraft Well-Known Member

    Joined:
    Nov 7, 2010
    Messages:
    52
    Likes Received:
    0
    Trophy Points:
    6
    What happens when MaxClients is reached is that any further requests will be pending until a slot frees up. So, what they'll see is a white page that's "loading" while they're waiting.

    Also, to note the difference between prefork and worker - prefork is best for static content, and worker is for your dynamic content - especially php.

    I'm glad the overall memory usage has dropped. Keep tweaking it and you'll eventually find the best setup for your server.

    It's obvious that you're running out of memory. Just remember, and this is something I tell my customers all the time, optimization can do a lot, but it's no substitute for hardware. If your server can't handle what you're throwing at it, and you've tweaked it as much as you can, then the only thing you can do is add more horsepower.

    Your next step from here to further reduce the load would be to look into something like nginx. The more of the load that you can take off apache, the better off you'll be. Nginx or litespeed, or lighttpd are all some different options you may wish to look into. Whether you want to replace apache or continue to run apache and setup something like nginx to handle your static content is up to you.

    I'd also recommend eaccelerator and fastcgi for apache - it caches php files so that they aren't constantly compiled and further reduces the resources that apache will take up.
     
Loading...

Share This Page