The Community Forums

Interact with an entire community of cPanel & WHM users!
  1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

I need MySQL Optimizations

Discussion in 'Workarounds and Optimization' started by ledux, Jun 3, 2014.

  1. ledux

    ledux Registered

    Joined:
    Jun 3, 2014
    Messages:
    2
    Likes Received:
    0
    Trophy Points:
    1
    cPanel Access Level:
    Root Administrator
    Hello! I have a dedicated server Intel i7 CPU 950@3.07GHz with 24GB RAM on CentOS 5.5 x86_64 and I need advice for tuning MySQL database (myisam and innodb) (ver. 5.0.95). I have a Joomla portal (myisam db) and database size is around 200 MB. CMS has 20-30000 inputs per day, and the problem happens when we let the links on Facebook page (70,000 fans about). Today I'm on FB published an interesting story about that server literally froze and I had to restart the database. Please give me some advise in the form of a template for a new and better configuration database (myisam and innodb) .

    Thank you very much, p.s: I'm sorry for my english :)

    MySQLTuner report:

    Code:
    [OK] Currently running supported MySQL version 5.0.95-log
    [OK] Operating on 64-bit architecture
    
    -------- Storage Engine Statistics -------------------------------------------
    [--] Status: -Archive -BDB -Federated +InnoDB -ISAM -NDBCluster
    [--] Data in MyISAM tables: 163M (Tables: 768)
    [--] Data in InnoDB tables: 14M (Tables: 383)
    [!!] Total fragmented tables: 21
    
    -------- Security Recommendations -------------------------------------------
    [OK] All database users have passwords assigned
    
    -------- Performance Metrics -------------------------------------------------
    [--] Up for: 30m 39s (448K q [244.105 qps], 7K conn, TX: 5B, RX: 75M)
    [--] Reads / Writes: 90% / 10%
    [--] Total buffers: 3.3G global + 4.2M per thread (800 max threads)
    [OK] Maximum possible memory usage: 6.6G (28% of installed RAM)
    [OK] Slow queries: 1% (5K/448K)
    [!!] Highest connection usage: 100% (801/800)
    [OK] Key buffer size / total MyISAM indexes: 2.9G/45.6M
    [OK] Key buffer hit rate: 100.0% (71M cached / 8K reads)
    [OK] Query cache efficiency: 51.2% (202K cached / 394K selects)
    [!!] Query cache prunes per day: 45713
    [OK] Sorts requiring temporary tables: 0% (83 temp sorts / 15K sorts)
    [!!] Temporary tables created on disk: 45% (8K on disk / 18K total)
    [OK] Thread cache hit rate: 89% (801 created / 7K connections)
    [OK] Table cache hit rate: 99% (2K open / 2K opened)
    [OK] Open file limit used: 25% (3K/12K)
    [OK] Table locks acquired immediately: 98% (212K immediate / 216K locks)
    [OK] InnoDB data size / buffer pool: 14.6M/32.0M
    
    -------- Recommendations -----------------------------------------------------
    General recommendations:
    Run OPTIMIZE TABLE to defragment tables for better performance
    MySQL started within last 24 hours - recommendations may be inaccurate
    Reduce or eliminate persistent connections to reduce connection usage
    Temporary table size is already large - reduce result set size
    Reduce your SELECT DISTINCT queries without LIMIT clauses
    Variables to adjust:
    max_connections (> 800)
    wait_timeout (< 28800)
    interactive_timeout (< 28800)
    query_cache_size (> 8M)
    

    MySQL Performance Tuning primer

    Code:
    SLOW QUERIES
    The slow query log is enabled.
    Current long_query_time = 1 sec.
    You have 5656 out of 475784 that take longer than 1 sec. to complete
    Your long_query_time seems to be fine
    
    BINARY UPDATE LOG
    The binary update log is NOT enabled.
    You will not be able to do point in time recovery
    See /http://dev.mysql.com/doc/refman/5.0/...-recovery.html
    
    WORKER THREADS
    Current thread_cache_size = 200
    Current threads_cached = 14
    Current threads_per_sec = 0
    Historic threads_per_sec = 0
    Your thread_cache_size is fine
    
    MAX CONNECTIONS
    Current max_connections = 800
    Current threads_connected = 400
    Historic max_used_connections = 801
    The number of used connections is 100% of the configured maximum.
    You should raise max_connections
    
    INNODB STATUS
    Current InnoDB index space = 8 M
    Current InnoDB data space = 14 M
    Current InnoDB buffer pool free = 45 %
    Current innodb_buffer_pool_size = 32 M
    Depending on how much space your innodb indexes take up it may be safe
    to increase this value to up to 2 / 3 of total system memory
    
    MEMORY USAGE
    Max Memory Ever Allocated : 6.35 G
    Configured Max Per-thread Buffers : 3.32 G
    Configured Max Global Buffers : 3.03 G
    Configured Max Memory Limit : 6.35 G
    Physical Memory : 23.53 G
    Max memory limit seem to be within acceptable norms
    
    KEY BUFFER
    Current MyISAM index space = 45 M
    Current key_buffer_size = 2.92 G
    Key cache miss rate is 1 : 9081
    Key buffer free ratio = 81 %
    Your key_buffer_size seems to be fine
    
    QUERY CACHE
    Query cache is enabled
    Current query_cache_size = 8 M
    Current query_cache_used = 3 M
    Current query_cache_limit = 1 M
    Current Query cache Memory fill ratio = 39.80 %
    Current query_cache_min_res_unit = 4 K
    MySQL won't cache query results that are larger than query_cache_limit in size
    
    SORT OPERATIONS
    Current sort_buffer_size = 1 M
    Current read_rnd_buffer_size = 1 M
    Sort buffer seems to be fine
    
    JOINS
    Current join_buffer_size = 1.00 M
    You have had 0 queries where a join could not use an index properly
    Your joins seem to be using indexes properly
    
    OPEN FILES LIMIT
    Current open_files_limit = 12288 files
    The open_files_limit should typically be set to at least 2x-3x
    that of table_cache if you have heavy MyISAM usage.
    Your open_files_limit value seems to be fine
    
    TABLE CACHE
    Current table_cache value = 4096 tables
    You have a total of 1169 tables
    You have 2757 open tables.
    The table_cache value seems to be fine
    
    TEMP TABLES
    Current max_heap_table_size = 256 M
    Current tmp_table_size = 256 M
    Of 11184 temp tables, 45% were created on disk
    Perhaps you should increase your tmp_table_size and/or max_heap_table_size
    to reduce the number of disk-based temporary tables
    Note! BLOB and TEXT columns are not allow in memory tables.
    If you are using these columns raising these values might not impact your
    ratio of on disk temp tables.
    
    TABLE SCANS
    Current read_buffer_size = 1 M
    Current table scan ratio = 518 : 1
    read_buffer_size seems to be fine
    
    TABLE LOCKING
    Current Lock Wait ratio = 1 : 42
    You may benefit from selective use of InnoDB.

    my.cnf

    Code:
    [mysqld]
    local-infile=0
    datadir=/var/lib/mysql
    #skip-locking
    skip-bdb
    #skip-networking
    safe-show-database
    #query_cache_limit=4M
    query_cache_size=8M
    query_cache_type=1
    max_connections=800
    #interactive_timeout=10
    #wait_timeout=20
    #connect_timeout=20
    thread_cache_size=200
    key_buffer=3000M 
    log-queries-not-using-indexes
    join_buffer_size=1M
    max_connect_errors=20
    max_allowed_packet=32M
    table_cache = 4096
    tmp_table_size=256M
    max_heap_table_size=256M
    #bulk_insert_buffer_size=512M
    thread_stack = 256K
    sort_buffer_size = 1M
    read_buffer_size = 1M
    read_rnd_buffer_size = 1M
    thread_concurrency=16 
    myisam_sort_buffer_size=128M
    myisam_max_sort_file_size=256M
    myisam_repair_threads=4
    server-id=1
    log-slow-queries = /var/lib/mysql/mysql-slow.log
    expire_logs_days=7
    long_query_time = 1
    low_priority_updates=1
    concurrent_insert=2
    open_files_limit=12288
    
    innodb_buffer_pool_size=32M
    innodb_additional_mem_pool_size=50M
    innodb_file_io_threads=16
    innodb_lock_wait_timeout=50
    innodb_log_buffer_size=16M
    innodb_flush_log_at_trx_commit = 2
    
    [mysql.server]
    user=mysql
    
    [safe_mysqld]
    err-log=/var/log/mysqld.log
    pid-file=/var/lib/mysql/mysql.pid
    open_files_limit=12288
    
    [mysqldump]
    quick
    max_allowed_packet=32M
    
    [mysql]
    no-auto-rehash
    #safe-updates
    
    [isamchk]
    key_buffer=64M
    sort_buffer=64M
    read_buffer=16M
    write_buffer=16M
    
    [myisamchk]
    key_buffer=64M
    sort_buffer=64M
    read_buffer=16M
    write_buffer=16M
    
    [mysqlhotcopy]
    interactive-timeout
     
  2. cPanelMichael

    cPanelMichael Forums Analyst
    Staff Member

    Joined:
    Apr 11, 2011
    Messages:
    30,854
    Likes Received:
    675
    Trophy Points:
    113
    cPanel Access Level:
    Root Administrator
  3. ledux

    ledux Registered

    Joined:
    Jun 3, 2014
    Messages:
    2
    Likes Received:
    0
    Trophy Points:
    1
    cPanel Access Level:
    Root Administrator
    Hello, here's the new statistics and configuration.


    Code:
    -- MYSQL PERFORMANCE TUNING PRIMER --
                 - By: Matthew Montgomery -
    
    MySQL Version 5.0.95 x86_64
    
    Uptime = 1 days 5 hrs 17 min 46 sec
    Avg. qps = 184
    Total Questions = 19467163
    Threads Connected = 8
    
    Warning: Server has not been running for at least 48hrs.
    It may not be safe to use these recommendations
    
    To find out more information on how each of these
    runtime variables effects performance visit:
    [url=http://dev.mysql.com/doc/refman/5.0/en/server-system-variables.html]MySQL :: MySQL 5.0 Reference Manual :: 5.1.4 Server System Variables[/url]
    Visit [url]http://www.mysql.com/products/enterprise/advisors.html[/url]
    for info about MySQL's Enterprise Monitoring and Advisory Service
    
    SLOW QUERIES
    The slow query log is NOT enabled.
    Current long_query_time = 10 sec.
    You have 126334 out of 19467186 that take longer than 10 sec. to complete
    Your long_query_time seems to be fine
    
    BINARY UPDATE LOG
    The binary update log is NOT enabled.
    You will not be able to do point in time recovery
    See [url=http://dev.mysql.com/doc/refman/5.0/en/point-in-time-recovery.html]MySQL :: MySQL 5.0 Reference Manual :: 7.5 Point-in-Time (Incremental) Recovery Using the Binary Log[/url]
    
    WORKER THREADS
    Current thread_cache_size = 150
    Current threads_cached = 70
    Current threads_per_sec = 0
    Historic threads_per_sec = 0
    Your thread_cache_size is fine
    
    MAX CONNECTIONS
    Current max_connections = 800
    Current threads_connected = 8
    Historic max_used_connections = 78
    The number of used connections is 9% of the configured maximum.
    You are using less than 10% of your configured max_connections.
    Lowering max_connections could help to avoid an over-allocation of memory
    See "MEMORY USAGE" section to make sure you are not over-allocating
    
    INNODB STATUS
    Current InnoDB index space = 8 M
    Current InnoDB data space = 14 M
    Current InnoDB buffer pool free = 43 %
    Current innodb_buffer_pool_size = 32 M
    Depending on how much space your innodb indexes take up it may be safe
    to increase this value to up to 2 / 3 of total system memory
    
    MEMORY USAGE
    Max Memory Ever Allocated : 5.42 G
    Configured Max Per-thread Buffers : 3.32 G
    Configured Max Global Buffers : 5.10 G
    Configured Max Memory Limit : 8.42 G
    Physical Memory : 23.53 G
    Max memory limit seem to be within acceptable norms
    
    KEY BUFFER
    Current MyISAM index space = 45 M
    Current key_buffer_size = 4.88 G
    Key cache miss rate is 1 : 288697
    Key buffer free ratio = 81 %
    Your key_buffer_size seems to be too high.
    Perhaps you can use these resources elsewhere
    
    QUERY CACHE
    Query cache is enabled
    Current query_cache_size = 128 M
    Current query_cache_used = 34 M
    Current query_cache_limit = 128 M
    Current Query cache Memory fill ratio = 27.12 %
    Current query_cache_min_res_unit = 4 K
    MySQL won't cache query results that are larger than query_cache_limit in size
    
    SORT OPERATIONS
    Current sort_buffer_size = 1 M
    Current read_rnd_buffer_size = 1 M
    Sort buffer seems to be fine
    
    JOINS
    Current join_buffer_size = 1.00 M
    You have had 50 queries where a join could not use an index properly
    You should enable "log-queries-not-using-indexes"
    Then look for non indexed joins in the slow query log.
    If you are unable to optimize your queries you may want to increase your
    join_buffer_size to accommodate larger joins in one pass.
    
    Note! This script will still suggest raising the join_buffer_size when
    ANY joins not using indexes are found.
    
    OPEN FILES LIMIT
    Current open_files_limit = 12000 files
    The open_files_limit should typically be set to at least 2x-3x
    that of table_cache if you have heavy MyISAM usage.
    Your open_files_limit value seems to be fine
    
    TABLE CACHE
    Current table_cache value = 4096 tables
    You have a total of 1197 tables
    You have 1885 open tables.
    The table_cache value seems to be fine
    
    TEMP TABLES
    Current max_heap_table_size = 1000 M
    Current tmp_table_size = 1000 M
    Of 403153 temp tables, 49% were created on disk
    Perhaps you should increase your tmp_table_size and/or max_heap_table_size
    to reduce the number of disk-based temporary tables
    Note! BLOB and TEXT columns are not allow in memory tables.
    If you are using these columns raising these values might not impact your
    ratio of on disk temp tables.
    
    TABLE SCANS
    Current read_buffer_size = 1 M
    Current table scan ratio = 549 : 1
    read_buffer_size seems to be fine
    
    TABLE LOCKING
    Current Lock Wait ratio = 1 : 261
    You may benefit from selective use of InnoDB.

    Code:
    -------- General Statistics --------------------------------------------------
    [--] Skipped version check for MySQLTuner script
    [OK] Currently running supported MySQL version 5.0.95
    [OK] Operating on 64-bit architecture
    
    -------- Storage Engine Statistics -------------------------------------------
    [--] Status: -Archive -BDB -Federated +InnoDB -ISAM -NDBCluster
    [--] Data in MyISAM tables: 164M (Tables: 782)
    [--] Data in InnoDB tables: 14M (Tables: 397)
    [!!] Total fragmented tables: 26
    
    -------- Security Recommendations  -------------------------------------------
    [OK] All database users have passwords assigned
    
    -------- Performance Metrics -------------------------------------------------
    [--] Up for: 1d 5h 20m 52s (19M q [184.731 qps], 264K conn, TX: 263B, RX: 3B)
    [--] Reads / Writes: 91% / 9%
    [--] Total buffers: 6.1G global + 4.2M per thread (800 max threads)
    [OK] Maximum possible memory usage: 9.4G (39% of installed RAM)
    [OK] Slow queries: 0% (126K/19M)
    [OK] Highest usage of available connections: 9% (78/800)
    [OK] Key buffer size / total MyISAM indexes: 4.9G/45.7M
    [OK] Key buffer hit rate: 100.0% (3B cached / 11K reads)
    [OK] Query cache efficiency: 45.8% (7M cached / 17M selects)
    [!!] Query cache prunes per day: 24121
    [OK] Sorts requiring temporary tables: 0% (1K temp sorts / 690K sorts)
    [!!] Temporary tables created on disk: 49% (401K on disk / 806K total)
    [OK] Thread cache hit rate: 99% (78 created / 264K connections)
    [OK] Table cache hit rate: 99% (1K open / 1K opened)
    [OK] Open file limit used: 16% (1K/12K)
    [OK] Table locks acquired immediately: 99% (10M immediate / 10M locks)
    [OK] InnoDB data size / buffer pool: 14.9M/32.0M
    
    -------- Recommendations -----------------------------------------------------
    General recommendations:
        Run OPTIMIZE TABLE to defragment tables for better performance
        Enable the slow query log to troubleshoot bad queries
        Temporary table size is already large - reduce result set size
        Reduce your SELECT DISTINCT queries without LIMIT clauses
    Variables to adjust:
        query_cache_size (> 128M)
    
    [/quote]
    
    [quote]
    [mysqld]
    skip-bdb
    local-infile=0
    max_connections = 800
    key_buffer_size = 5000M
    long_query_time = 10
    open_files_limit= 12000
    max_allowed_packet = 32M
    table_cache = 4096
    sort_buffer_size = 1M
    read_buffer_size = 1M
    read_rnd_buffer_size = 1M
    myisam_sort_buffer_size = 128M
    thread_cache_size = 150
    query_cache_type = 1
    query_cache_size = 128M
    query_cache_limit = 128M
    join_buffer_size = 1M
    max_heap_table_size = 1000M
    tmp_table_size = 1000M
    log-queries-not-using-indexes
    low_priority_updates=1
    concurrent_insert=2
    
    # Uncomment the following if you are using InnoDB tables
    # You can set .._buffer_pool_size up to 50 - 80 %
    # of RAM but beware of setting memory usage too high
    #innodb_buffer_pool_size = 16M
    #innodb_additional_mem_pool_size = 2M
    # Set .._log_file_size to 25 % of buffer pool size
    #innodb_log_file_size = 5M
    #innodb_log_buffer_size = 64M
    #innodb_flush_log_at_trx_commit = 1
    #innodb_lock_wait_timeout = 50
    
    innodb_buffer_pool_size=32M
    innodb_additional_mem_pool_size=50M
    innodb_file_io_threads=16
    innodb_lock_wait_timeout=50
    innodb_log_buffer_size=16M
    innodb_flush_log_at_trx_commit = 2
    
    
    # Disable Federated by default
    skip-federated
    #log-bin=mysql-bin
    #sync-binlog = 1
    expire_logs_days=3
    server-id = 1
    
    [mysqldump]
    max_allowed_packet = 32M
    
    [mysql]
    no-auto-rehash
    
    [isamchk]
    key_buffer=64M
    sort_buffer=64M
    read_buffer=16M
    write_buffer=16M
    
    [myisamchk]
    key_buffer=64M
    sort_buffer=64M
    read_buffer=16M
    write_buffer=16M
    
    [mysqlhotcopy]
    interactive-timeout
    [/quote]
    
    httpd.conf
    
    [quote]
    
    ## Server-Pool Size Regulation (MPM specific)
    ##
    
    # prefork MPM
    # StartServers: number of server processes to start
    # MinSpareServers: minimum number of server processes which are kept spare
    # MaxSpareServers: maximum number of server processes which are kept spare
    # ServerLimit: maximum value for MaxClients for the lifetime of the server
    # MaxClients: maximum number of server processes allowed to start
    # MaxRequestsPerChild: maximum number of requests a server process serves
    <IfModule mpm_prefork_module>
    StartServers 8
    MinSpareServers 5
    MaxSpareServers 20
    MaxClients 150
    MaxRequestsPerChild 100
    </IfModule>
    
    # worker MPM
    # StartServers: initial number of server processes to start
    # MaxClients: maximum number of simultaneous client connections
    # MinSpareThreads: minimum number of worker threads which are kept spare
    # MaxSpareThreads: maximum number of worker threads which are kept spare
    # ThreadsPerChild: constant number of worker threads in each server process
    # MaxRequestsPerChild: maximum number of requests a server process serves
    <IfModule mpm_worker_module>
    StartServers 2
    MaxClients 150
    MinSpareThreads 25
    MaxSpareThreads 75
    ThreadsPerChild 25
    MaxRequestsPerChild 0
    </IfModule>
    
     
    #3 ledux, Jun 5, 2014
    Last edited by a moderator: Jun 5, 2014
  4. cPanelJared

    cPanelJared Technical Analyst
    Staff Member

    Joined:
    Feb 25, 2010
    Messages:
    1,842
    Likes Received:
    18
    Trophy Points:
    38
    Location:
    Houston, TX
    cPanel Access Level:
    Root Administrator
    Friendly Moderator Note

    I changed your QUOTE tags to CODE tags. CODE is preferred when pasting shell output, because it uses a fixed-width font. A shortcut button for CODE is available if you click Advanced Editor; it is the little pound sign (#) on the second row of icons in the editor.
     
Loading...

Share This Page