The Community Forums

Interact with an entire community of cPanel & WHM users!
  1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Exim Memory Usage [cPanel 11]

Discussion in 'General Discussion' started by wswd, Jul 26, 2007.

  1. wswd

    wswd Well-Known Member

    Joined:
    Aug 9, 2005
    Messages:
    112
    Likes Received:
    0
    Trophy Points:
    16
    cPanel Access Level:
    Root Administrator
    Since nobody in the EXIM forums seems to have an answer, I thought I would try here.

    Since installing cPanel 11, I have noticed a huge increase in memory usage, possibly a memory leak, since the memory usage starts off low and ends up ridiculously high.

    We usually sit around 43% usage. After about 3-4 days, it might be in the low 70's, and after a week or more, in the 90's.

    A restart of EXIM takes it right back to 41-43% and the cycle starts all over again.

    Anybody else notice this? Aside from having a CRON restart EXIM every few days, is there a fix for this, or anyway around this. If it's helps, I'm using the release build 11.6.0-RELEASE_15076, though this started happening with the first 11 install I put on.

    It happens on VPS accounts, ded. servers, and other people we have talked with have also had this issue. Any idea how to fix it?

    Thanks in advance!

    Bryan
     
  2. cPanelKenneth

    cPanelKenneth cPanel Development
    Staff Member

    Joined:
    Apr 7, 2006
    Messages:
    4,458
    Likes Received:
    22
    Trophy Points:
    38
    cPanel Access Level:
    Root Administrator
    Are you certain it's not spamassassin (spamd)?
     
  3. bsasninja

    bsasninja Well-Known Member

    Joined:
    Sep 2, 2004
    Messages:
    528
    Likes Received:
    0
    Trophy Points:
    16
    cpanelkenneth why you dont replace spamassassin for ASSP X in Cpanel 11? Is 100% better and not a resource hog.
    Also it will not cause load hedeaches to users.

    I suggest you set it up at cpanel plugins. Or gives the user the posibility to choose between spamassassin or assp x (as you can do with ftp servers)

    Thanks!
     
  4. wswd

    wswd Well-Known Member

    Joined:
    Aug 9, 2005
    Messages:
    112
    Likes Received:
    0
    Trophy Points:
    16
    cPanel Access Level:
    Root Administrator
    Not 100% sure...no. That was actually my original assumption.
     
  5. sehh

    sehh Well-Known Member

    Joined:
    Feb 11, 2006
    Messages:
    579
    Likes Received:
    5
    Trophy Points:
    18
    Location:
    Europe
    I'm having the same problem. After lots of searching and testing i found out that:

    1) SA is not the problem. It does use a lot of memory and if i limit it to max-children=1 then it uses considerable less memory. Based on my tests it never "leaks", just uses lots of constant memory.

    2) Exim has an issue of consuming lots of memory if you are sending thousands of emails. The solution here is to limit the number of emails per domain from WHM.

    3) ClamAV is the worst. It leaks like hell and at least once a week i have to "service restart exim" in order to restart the clamd process. It uses huge amounts of memory for no apparent reason.
     
  6. wswd

    wswd Well-Known Member

    Joined:
    Aug 9, 2005
    Messages:
    112
    Likes Received:
    0
    Trophy Points:
    16
    cPanel Access Level:
    Root Administrator
    1) Interesting.

    2) The majority of our servers have hardly any e-mail usage.

    3) We don't run clam AV

    Would be great to figure out what the heck is going on with this. :mad:

    Thanks for the info!!
     
  7. nyjimbo

    nyjimbo Well-Known Member

    Joined:
    Jan 25, 2003
    Messages:
    1,125
    Likes Received:
    0
    Trophy Points:
    36
    Location:
    New York
    Once they went to 0.90.x it went crazy. We have a system here just for preprocessing mail for some high volume domains and we had to remove Clam as it was going nuts eating all ram and then everything swapping to disk. They say 0.91 is better but I am going to wait a while before I jump back into it.
     
  8. freedman

    freedman Well-Known Member

    Joined:
    Feb 13, 2005
    Messages:
    312
    Likes Received:
    1
    Trophy Points:
    18
    I dont experience any of these problems.

    occasionally clamd will crash or something, and often the freshclam process gets weird, but a simple restart fixes that.

    if the worst thing is to put a cron job to restart clamav, then you're not so bad off :P

    but, our servers have minimal cpu and not unreasonable memory utilization with this configuration.
     
  9. cPanelKenneth

    cPanelKenneth cPanel Development
    Staff Member

    Joined:
    Apr 7, 2006
    Messages:
    4,458
    Likes Received:
    22
    Trophy Points:
    38
    cPanel Access Level:
    Root Administrator
    wswd, would you mind opening a support ticket so we can determine what is causing this?
     
  10. wswd

    wswd Well-Known Member

    Joined:
    Aug 9, 2005
    Messages:
    112
    Likes Received:
    0
    Trophy Points:
    16
    cPanel Access Level:
    Root Administrator
    Hi Kenneth. Thanks for the reply, will do!
     
  11. ANewDay

    ANewDay Well-Known Member

    Joined:
    Sep 11, 2006
    Messages:
    86
    Likes Received:
    0
    Trophy Points:
    6
    Location:
    USA
    clamAV is pretty much useless, most viruses won't affect the linux. It's the person reading the mail that is affected and they usually run antivirus already. ClamAV causes a lot of problems, highly not recommended.
     
  12. freedman

    freedman Well-Known Member

    Joined:
    Feb 13, 2005
    Messages:
    312
    Likes Received:
    1
    Trophy Points:
    18
    This is an obsurd statement.

    First, as email service providers (and, if you run a hosting business that allows people to use email, then you're an email service provider) you should take whatever steps necessary to protect people from viruses.

    I've had NO problems with clamav on any of my servers. The load isn't incredibly high and it's kept a lot of people from getting viruses.
    This in turn helps keep my server load down, since, if one of my clients email users were to get a virus that would then resend a bunch of spam through my server, it would get mine blacklisted... So, if you're not doing it for them, do it for yourself.. you need to make sure that nothing coming out of your server is going to get your server blacklisted at the very least.
     
  13. cPanelKenneth

    cPanelKenneth cPanel Development
    Staff Member

    Joined:
    Apr 7, 2006
    Messages:
    4,458
    Likes Received:
    22
    Trophy Points:
    38
    cPanel Access Level:
    Root Administrator
    Thanks for that reply, freedman.
     
  14. mambovince

    mambovince Well-Known Member

    Joined:
    Jan 15, 2005
    Messages:
    192
    Likes Received:
    0
    Trophy Points:
    16
    Location:
    London, UK
    I find ClamAV very usefull for tracking down trojan files when I suspect an account.
    Is there also a way to make it run via a cronjob on all public_html folders for all the hosted accounts?

    Thanks,

    - Vince
     
  15. freedman

    freedman Well-Known Member

    Joined:
    Feb 13, 2005
    Messages:
    312
    Likes Received:
    1
    Trophy Points:
    18
    clamscan

    check the manpage for clamscan

    the most basic way to run it is:
    # clamscan -i -r /home

    there are some options you might consider.. --no-pe and --noole2 will ignore MS Windows executables and documents.

    NOTE: if you're already running clamd, then it should be a tad faster to use clamdscan

    If someone wants to get really sophistocated, I'd love to see a script which:
    - on first use, does a full scan, and writes the scan start date to a file.
    - on subsequent manual use, finds all files modified since then and only scans those files.
    -* utilizes "inotify" to actively scan new/modified files


    It would be portentially resource intensive to scan all your homedirectories on a regular basis, and it's not necessary, unless you just want to make sure your hard drive is put to good use.
     
  16. mambovince

    mambovince Well-Known Member

    Joined:
    Jan 15, 2005
    Messages:
    192
    Likes Received:
    0
    Trophy Points:
    16
    Location:
    London, UK
    What you describe there is exactly what I was hoping would be in existance already. :cool:

    Thanks,

    - Vince
     
  17. mambovince

    mambovince Well-Known Member

    Joined:
    Jan 15, 2005
    Messages:
    192
    Likes Received:
    0
    Trophy Points:
    16
    Location:
    London, UK
    OK, an update on my Exim high resource usage:

    It sems to happen whenever the mail queue has more than 100 (approx, but below 100 is fine) messages in there, especially if there are many that are either bounced or frozen.

    Once I cleare these, load goes down almost immediately.
    The annoying thing is, in the Exim conf. rules I have the following

    timeout_frozen_after = 1d
    ignore_bounce_errors_after = 2h

    But it doesn't seem to take any notice :confused:

    Am I missing something, or is my Exim broken?

    Many thanks,

    - Vince
     
  18. verdon

    verdon Well-Known Member

    Joined:
    Nov 1, 2003
    Messages:
    836
    Likes Received:
    2
    Trophy Points:
    18
    Location:
    Northern Ontario, Canada
    cPanel Access Level:
    Root Administrator
    I'm not sure those settings have ever worked reliably for me. Instead I've learned to keep my queue tidy. using a dictionary attack acl/script and a mailscanner/spamassassin solution has greatly reduced the amount of noise like this in my queue.
     
    #18 verdon, Aug 28, 2007
    Last edited: Aug 28, 2007
  19. sehh

    sehh Well-Known Member

    Joined:
    Feb 11, 2006
    Messages:
    579
    Likes Received:
    5
    Trophy Points:
    18
    Location:
    Europe
    aah yes the famous exim queue problem. I've already solved the problem and posted the solution in another thread, but i'll post it here in case its of help to someone.

    edit your /etc/exim.conf and change/add these values:

    Code:
    ignore_bounce_errors_after = 12h
    timeout_frozen_after = 1d
    auto_thaw = 2d
    

    edit your cron jobs with "crontab -e" and add:

    Code:
    0 0 * * * /usr/sbin/exiqgrep -i -o 259200 | xargs /usr/sbin/exim -Mrm >/dev/null 2>/dev/null
    

    let me explain what this does:

    1) the exim config options will make sure emails are not kept in the queue unless absolutely necessary: ignores errors after 12 hours, frozen emails are timed out after 1 day and all emails are re-tried in 2 days.

    2) all other emails left in the queue for 4 days are deleted via the cron job.

    it may look a bit aggressive but that was the only way to stop the queue from growing on a busy server. Maybe of our users send emails to bad emails and those would stay in the queue for months!!!
     
  20. mambovince

    mambovince Well-Known Member

    Joined:
    Jan 15, 2005
    Messages:
    192
    Likes Received:
    0
    Trophy Points:
    16
    Location:
    London, UK
    Hi sehh,
    Appreciate your suggestion, but why doesn't Exim abide by my eisting rules- is it down to Exim broken, or a cPanel bug?

    Would keeping my rules, and just adding your suggested cron work for me anyway?

    Also a bit confused as to why you have auto_thaw set for 2 days, even though the timeout is set for just 1 day.

    Is my ignor_bounce_errors_after 2 hours too aggressive?

    I just viewed my queue and it only had 136 messages. cpu load was at 1.5 and 36 of these were frozen/bnounced (didn't check how many of each). As soon as I cleared these 36, the load came down to 0.35

    Something strange about having more than 100 in the mail queue??


    Many thanks,

    - Vince
     
Loading...

Share This Page