The Community Forums

Interact with an entire community of cPanel & WHM users!
  1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Memcached + Apache 2.2 crashes site every 5-10 minutes

Discussion in 'Workarounds and Optimization' started by 63bus, Nov 2, 2014.

  1. 63bus

    63bus Active Member

    Joined:
    Mar 31, 2007
    Messages:
    34
    Likes Received:
    0
    Trophy Points:
    6
    Apache 2.2/2.4 (was running 2.4 but currently back to 2.2 to try to fix below issues)
    Server Version: Apache/2.2.29 (Unix) mod_ssl/2.2.29 OpenSSL/0.9.8e-fips-rhel5 mod_bwlimited/1.4 Apache mod_fcgid/2.3.9
    PHP 5.4
    Memcached 1.4.15, running locally on the same server.

    I recently had some database slowness on my site related to queries on large table (120M rows) and noticed that phpMyAdmin -> Status -> Advisor said I should no longer use the mySQL Query cache. My site was 200 requests/second and the trigger was anything over 100/second.

    I installed memcached per this guide I found Easy way to install Memcached and memcache in a cPanel server | CryBit - /http://crybit.com/easy-way-to-install-memcached-and-memcache-in-a-cpanel-server/
    Everything was fine, it was installed, I could see it was live in phpinfo and via this command:
    memcached-tool 127.0.0.1:11211 stats

    After installation, I began switching my PHP code to use memcache.
    Example:
    PHP:
    $memcache = new Memcache;
    $memcache->connect('127.0.0.1''11211');
    $sql "SELECT ids from id_table WHERE id_type=5";
    $ids $memcache->get(md5($sql));
    if (!
    $ids)
    {
        
    $ids get_ids_from_database($sql);
        if (
    count($ids))
        {
            
    $memcache->set(md5($sql), $idsMEMCACHE_COMPRESSED60);
        }
    }
    I verified data was being loaded from the cache in my PHP page, and I also could see data being stored using the memcached-tool command noted above .

    Soon after I began using memcached, Apache would lock up and go to 256 connections (my configured max through WHM) every 30 minutes or so. This time would vary, it was seemingly random. I could also see database queries massively backing up when viewed via:
    mytop -d mysql

    I thought it was database issues so I began caching more and more stuff in my code, converted some high update/select tables to INNODB, etc. This actually seemed to help as my site was faster but as I cached even more, my site became unusable as Apache would overload at shorter and shorter increments, down to only working from 5-10 minutes today. I finally decided to disable all memcaching just as a test and boom, site came back up everything is ok, barring the occasional high load/high # of Apache connections that I think is normal for my site, especially when I still have the mySQL cache off.

    I would like to use memcached as I have a bunch of queries that are long-running, often taking 3-30 seconds, and occasionally my site will still run slow due to these queries.

    What can I check and/or change to get memcached working reliably?

    Edit #1:
    After looking at a lot of PHP code samples, I see very few people doing this in examples:
    PHP:
    $ids $memcache->get(md5($sql));
    This example is more common, where some text is appended:
    PHP:
    $ids $memcache->get('sql:' md5($sql));
    Is there an issue with just using md5 alone to generate a key?

    Edit #2:
    My site is a medium sized site - it has 800-1000 people online most of the time. I wonder if calling md5() hundreds/times/second is an issue? The major pages that I initially optimized had 5-10 memcached and md5 calls each.

    Edit #3:
    I took out some md5() calls as a test and just used a simple key like 'users' and my site still began overloading fairly quickly.
    As another test, I also removed MEMCACHE_COMPRESSED and replaced it with a 0 to disable any compression in case that was a speed issue. The data I am caching is an array and it was NOT cached. Possible bug?

    Edit #4:
    Is mod_fcgid an issue? I mean overall but here are my settings from Pre VirtualHost Include, All Versions, in case there is an issue with the settings:
     
    #1 63bus, Nov 2, 2014
    Last edited: Nov 2, 2014
  2. 63bus

    63bus Active Member

    Joined:
    Mar 31, 2007
    Messages:
    34
    Likes Received:
    0
    Trophy Points:
    6
    For reference, in EasyApache I have these selected, among other things:
    MPM Prefork
    eAccelerator 0.9.6.1
    Mod FastCGI v2.3.9
    Mod Security 2.8.0

    Just in case any of those are causing some conflict or should be altered to have memcached work.
     
  3. 63bus

    63bus Active Member

    Joined:
    Mar 31, 2007
    Messages:
    34
    Likes Received:
    0
    Trophy Points:
    6
    Resolution to this issue:
    There was no issue with memcached.

    Originally my site was overloading because one of my tables had grown so large (120M rows) that the mySQL queries against it could not be processed in a timely fashion.

    Even though I had used memcached to cache a lot of my "busy" queries, Apache and my site were still overloading because the mySQL Query Cache was off.

    A combination of redesigning the large table design and my queries against it, turning the Query cache back on, some mySQL database parameter tuning, and using memcached for my high-volume queries has my site running smooth now with a low load.

    Notes:
    My server had auto-updated Memcached to version 1.4.21 while I was working on these issues but I don't believe this had any effect.
    I am also back on Apache 2.4.
     
    #3 63bus, Jan 13, 2015
    Last edited: Jan 13, 2015
  4. cPanelMichael

    cPanelMichael Forums Analyst
    Staff Member

    Joined:
    Apr 11, 2011
    Messages:
    30,738
    Likes Received:
    661
    Trophy Points:
    113
    cPanel Access Level:
    Root Administrator
  5. James.Simpson

    James.Simpson Member

    Joined:
    Feb 18, 2013
    Messages:
    15
    Likes Received:
    0
    Trophy Points:
    1
    Location:
    United Kingdom
    cPanel Access Level:
    Root Administrator
    Just out of interest, how did you install memcached on Apache 2.4, as the official guide states Memcached is not compatible with this version of Apache?
     
  6. 63bus

    63bus Active Member

    Joined:
    Mar 31, 2007
    Messages:
    34
    Likes Received:
    0
    Trophy Points:
    6
    Memcached site: memcached - a distributed memory object caching system

    Where does it says that?
    It's working fine for me since installing it above.
     
  7. James.Simpson

    James.Simpson Member

    Joined:
    Feb 18, 2013
    Messages:
    15
    Likes Received:
    0
    Trophy Points:
    1
    Location:
    United Kingdom
    cPanel Access Level:
    Root Administrator
  8. 63bus

    63bus Active Member

    Joined:
    Mar 31, 2007
    Messages:
    34
    Likes Received:
    0
    Trophy Points:
    6
    The Apache module mod_cache is not the same as memcached.
     
Loading...

Share This Page