The Community Forums

Interact with an entire community of cPanel & WHM users!
  1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

High RAM usage

Discussion in 'Workarounds and Optimization' started by GoWilkes, Apr 28, 2017.

  1. GoWilkes

    GoWilkes Well-Known Member

    Joined:
    Sep 26, 2006
    Messages:
    395
    Likes Received:
    5
    Trophy Points:
    168
    cPanel Access Level:
    Root Administrator
    I recently discovered in my Munin report that my RAM usage has been very high since around April 1. I'm not sure of the exact date, but it looks to be the week of the 1st, anyway.

    On my end, the only thing that's changed is that I updated PHP to version 5.6.23 on April 9. I also had to modify several scripts to use cURL instead of file_get_contents(), since file_get_contents() is (for some reason) deprecated in 5.6.23.

    My server has 4G of RAM, but during peak hours it's trying to use close to 6G, so this is a concern!

    I spoke to my server management company, and their only suggestion was that they were seeing a lot of TIME_WAIT connections, with the last IP showing 168 connections:

    Code:
    # The last 5 in the list
    xxx.xxx.xxx.xxx is connected 66 in state: TIME_WAIT
    xxx.xxx.xxx.xxx is connected 69 in state: TIME_WAIT
    xxx.xxx.xxx.xxx is connected 71 in state: TIME_WAIT
    xxx.xxx.xxx.xxx is connected 103 in state: TIME_WAIT
    xxx.xxx.xxx.xxx is connected 168 in state: TIME_WAIT
    The IPs all looked local, though, so it's not an issue of an attack or anything; it's apparently a setting, somewhere. But why it only started to have a negative impact on 4/1/17, I don't know.

    After doing some research, I modified /etc/sysctl.conf to:

    Code:
    # Added these
    net.ipv4.tcp_tw_recycle = 0
    net.ipv4.tcp_tw_reuse = 0
    net.ipv4.tcp_orphan_retries = 1
    net.ipv4.tcp_fin_timeout = 25
    net.ipv4.tcp_max_orphans = 8192
    
    # Changed from 65536
    net.netfilter.nf_conntrack_max = 32768
    
    # Changed from 432000
    net.netfilter.nf_conntrack_tcp_timeout_established = 86400
    Then in WHM's Apache Configuration I lowered Server Limit from 400 to 256 (Munin showed the highest in the last week was about 120), and Max Clients from 300 to 150.

    Then in /etc/my.cnf, I changed max_connections from 300 to 100.

    Can you guys suggest anything else I should check... possibly related to the PHP update?

    TIA!
     
  2. cPanelMichael

    cPanelMichael Forums Analyst
    Staff Member

    Joined:
    Apr 11, 2011
    Messages:
    36,995
    Likes Received:
    1,275
    Trophy Points:
    363
    cPanel Access Level:
    Root Administrator
  3. GoWilkes

    GoWilkes Well-Known Member

    Joined:
    Sep 26, 2006
    Messages:
    395
    Likes Received:
    5
    Trophy Points:
    168
    cPanel Access Level:
    Root Administrator
    Thanks, Michael :) I went through the thread, but most of it was geared toward a high CPU load, not RAM. I installed the monitor.sh script (for which I had to use dos2unix, fwiw), but the results didn't tell me a whole lot:

    Code:
    1 minute load avg: 0
    Swap used: 193180 kB
    Mem used: 3728488 kB
    packets received (2 secs): 459
    packets sent (2 secs): 235
    So you can see that, at almost 11pm, I'm using 3.7G of my 4.0G of RAM, but that doesn't get me any closer to finding out why.

    Using top c, then sorting by M (that's a capital, not a lowercase... took me a hot minute to figure THAT out), ClamAV was the highest user at 12.0%. MySQL was next in line at 7.9%, then at 3.2%:

    Code:
    /usr/lib/jvm/jre-1.8.0/bin/java -server -Xms512m -Xmx512m -XX:NewRatio=3 -XX:SurvivorRatio=4 -XX:TargetSurvivorRatio
    2
    "named" and "cpsrvd" are next, then I have a lot of httpd processes showing around 19m under RES.

    I turned off ClamAV for testing purposes (before running monitor.sh) and RAM went down from around 4.5G to just under 4G, but that's all. I can't turn off MySQL since all of my sites use it, and I have no clue what XX:SurvivorRatio is.

    The tuning I did to sysctl.conf and my.cnf last night seemed to have no real impact, as today's load was just as high as the day before.

    I just now changed Server Limit from 400 to 150, Max Clients from 300 to 100, Max Requests Per Child from 0 (unlimited?) to 10000, and Timeout from 120 to 60. This seemed to have no impact at all, though; my RAM usage is still at 3.70G, even with ClamAV turned off.

    I'm not sure whether I need to be tweaking MySQL, or if maybe the number of httpd processes is the problem? The monitor.sh email showed 100 of those processes, and at 19m of RES each that's about 2G of RAM, isn't it?
     
  4. cPanelMichael

    cPanelMichael Forums Analyst
    Staff Member

    Joined:
    Apr 11, 2011
    Messages:
    36,995
    Likes Received:
    1,275
    Trophy Points:
    363
    cPanel Access Level:
    Root Administrator
  5. GoWilkes

    GoWilkes Well-Known Member

    Joined:
    Sep 26, 2006
    Messages:
    395
    Likes Received:
    5
    Trophy Points:
    168
    cPanel Access Level:
    Root Administrator
    Softlayer mentioned that Java has a very large Virtual Size on the server, close to 3GB.

    None of my sites intentionally use Java; I built them all, and I know that I don't code in Java. Is there any reason for it to be running, then? If not, how do I turn it off? Or at least modify the settings to minimize it?

    It's also notable that ClamAV seems to be taking up a lot of RAM for no reason. I had turned off the option to scan anything, but had left it installed. I just now uninstalled it, and RAM usage dropped from 5G to 3.5G.
     
    #5 GoWilkes, May 11, 2017
    Last edited: May 11, 2017
  6. Anoop P Alias

    Anoop P Alias Well-Known Member

    Joined:
    Mar 31, 2015
    Messages:
    65
    Likes Received:
    6
    Trophy Points:
    8
    Location:
    Kochi,Kerala,India
    cPanel Access Level:
    Root Administrator
    the Java process should be Solr used for full-text search in IMAP. it came with new CPanel version which might explain why you see an increase in ram recently and yes java is notorious for RAM requirement
     
    GoWilkes likes this.
  7. GoWilkes

    GoWilkes Well-Known Member

    Joined:
    Sep 26, 2006
    Messages:
    395
    Likes Received:
    5
    Trophy Points:
    168
    cPanel Access Level:
    Root Administrator
    Thanks, Anoop, that was definitely the culprit!!! I uninstalled it, and immediately my RAM load dropped by 1.5G!
     
  8. jonnyydeoliveira

    Joined:
    Feb 15, 2016
    Messages:
    19
    Likes Received:
    1
    Trophy Points:
    3
    Location:
    Brazil
    cPanel Access Level:
    Root Administrator
    How you uninstalled solr ?
     
  9. Infopro

    Infopro cPanel Sr. Product Evangelist
    Staff Member

    Joined:
    May 20, 2003
    Messages:
    15,617
    Likes Received:
    296
    Trophy Points:
    433
    Location:
    Pennsylvania
    cPanel Access Level:
    Root Administrator
    Twitter:
Loading...

Share This Page