The Community Forums

Interact with an entire community of cPanel & WHM users!
  1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

The curious case of high 5 min load average

Discussion in 'Workarounds and Optimization' started by jclyde, Dec 17, 2015.

  1. jclyde

    jclyde Registered

    Joined:
    Dec 17, 2015
    Messages:
    1
    Likes Received:
    0
    Trophy Points:
    1
    Location:
    singapore
    cPanel Access Level:
    Website Owner
    Hi guys

    Looking for some expert advice here. I'm a first time sys admin on my own server and I can't figure the bottle neck in my server.

    I've been receiving tons of high 5 min load average alert ranging between 8 - 80 from CSF

    So I went ahead and installed sqltuner on my server and let it run for 3 days

    The results don't show anything out of the ordinary but I'm still getting high 5 min load average daily. I'm trying to find the bottle neck (CPU, load caused by out of memory issues or I/O-bound load)

    Would be stoked if someone can share any insights...

    (I've included sqltuner's report and the high load email output below)

    Code:
    -------- Security Recommendations  -------------------------------------------
    [OK] There are no anonymous accounts for any database users
    [OK] All database users have passwords assigned
    [!!] There is no basic password file list!
    
    -------- Performance Metrics -------------------------------------------------
    [--] Up for: 120d 18h 27m 20s (227M q [21.795 qps], 51M conn, TX: 907B, RX: 26B)
    [--] Reads / Writes: 38% / 62%
    [--] Binary logging is disabled
    [--] Total buffers: 15.4G global + 4.1M per thread (600 max threads)
    [OK] Maximum reached memory usage: 16.2G (51.75% of installed RAM)
    [OK] Maximum possible memory usage: 17.8G (56.91% of installed RAM)
    [OK] Slow queries: 0% (11/227M)
    [OK] Highest usage of available connections: 33% (199/600)
    [OK] Aborted connections: 0.56%  (284327/51183230)
    [OK] Query cache efficiency: 83.0% (78M cached / 94M selects)
    [!!] Query cache prunes per day: 10288
    [OK] Sorts requiring temporary tables: 0% (392 temp sorts / 1M sorts)
    [OK] Temporary tables created on disk: 4% (65K on disk / 1M total)
    [OK] Thread cache hit rate: 99% (199 created / 51M connections)
    [OK] Table cache hit rate: 22% (425 open / 1K opened)
    [OK] Open file limit used: 0% (433/50K)
    [OK] Table locks acquired immediately: 99% (41M immediate / 41M locks)
    
    -------- MyISAM Metrics -----------------------------------------------------
    [!!] Key buffer used: 20.2% (108M used / 536M cache)
    [OK] Key buffer size / total MyISAM indexes: 512.0M/14.7M
    [OK] Read Key buffer hit rate: 99.8% (51M cached / 121K reads)
    [!!] Write Key buffer hit rate: 40.8% (4M cached / 2M writes)
    
    -------- InnoDB Metrics -----------------------------------------------------
    [--] InnoDB is enabled.
    [OK] InnoDB buffer pool / data size: 14.6G/140.9M
    [!!] InnoDB buffer pool instances: 1
    [!!] InnoDB Used buffer: 3.39% (32546 used/ 959999 total)
    [OK] InnoDB Read buffer efficiency: 100.00% (5437258684 hits/ 5437259670 total)
    [!!] InnoDB Write buffer efficiency: 0.00% (0 hits/ 1 total)
    [OK] InnoDB log waits: 0.00% (0 waits / 24069213 writes)
    
    -------- AriaDB Metrics -----------------------------------------------------
    [--] AriaDB is disabled.
    
    -------- Replication Metrics -------------------------------------------------
    [--] No replication slave(s) for this server.
    [--] This is a standalone server..
    
    -------- Recommendations -----------------------------------------------------
    General recommendations:
        Run OPTIMIZE TABLE to defragment tables for better performance
        Increasing the query_cache size over 128M may reduce performance
    Variables to adjust:
        query_cache_size (> 128M) [see warning above]
        innodb_buffer_pool_instances(=14)
    
    
    ----------------------
    (The only change I've ade is to reduce InnoDB size and add multiple pool instances)
    
    
    Code:
    Time:                    Sun Dec  6 05:43:53 2015 -0500
    1 Min Load Avg:          80.26
    5 Min Load Avg:          21.19
    15 Min Load Avg:         7.46
    Running/Total Processes: 221/875
    
    
    ps.txt
    
    Output from ps:
    
    [Removed - Data Not Required]
    
    vmstat.txt
    
    Output from vmstat:
    procs -----------memory---------- ---swap-- -----io---- --system-- -----cpu-----
      r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa st
    225  0  57272 165216 910924 23311604    0    0    44    47    1    0  5  2 93  0  0   
    
    
    [Removed - Included Actual Domain Names]
    
    This is my TOP output:
    
    root@ns513521 [~]# top
    top - 08:21:31 up 153 days,  3:51,  1 user,  load average: 0.15, 0.27, 0.51
    Tasks: 230 total,  2 running, 227 sleeping,  0 stopped,  1 zombie
    Cpu(s):  0.4%us,  0.3%sy,  0.0%ni, 99.3%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
    Mem:  32855908k total, 25102984k used,  7752924k free,  886004k buffers
    Swap:  1569780k total,  63984k used,  1505796k free, 21254784k cached
    
    This is my ioStat output:
    
    
    Device:  tps  Blk_read/s  Blk_wrtn/s  Blk_read  Blk_wrtn
    sda  21.99  1476.57  549.50 19540299168 7271856568
    sdb  19.53  982.85  549.49 13006647390 7271718616
    sdc  19.46  978.26  549.49 12945853934 7271718616
    md2  9.78  492.22  399.94 6513868322 5292584264
    md1  20.05  27.80  140.15  367920858 1854711136
     
  2. cPanelMichael

    cPanelMichael Forums Analyst
    Staff Member

    Joined:
    Apr 11, 2011
    Messages:
    30,678
    Likes Received:
    651
    Trophy Points:
    113
    cPanel Access Level:
    Root Administrator

Share This Page