Exim causes high server load

anoetic

Active Member
Feb 5, 2006
25
0
151
I've spent the past several hours searching for solutions to my problem, and I've seen some threads that approach an accurate description of the problem, but I wasn't able to find a solution. So, I hope I can describe this problem adequately to receive some attention on this forum.

In brief, the problem is that over time, exim processes increase and drive my server load higher and higher. (e.g., I woke up this morning and found the load to be over 80 and I was barely able to get in to reboot the server.) Usually, if I catch this before it gets too high, I can simply restart exim and the load drops back down to under 1 or maybe 2 -- certainly an acceptible level.

The load will usually remain low for around an hour, at which time, I assume exim starts it queue run (it's set to 60 minutes in WHM currently).

One other factor which leads me to the conclusion that processing the queue is a major contributing factor is that this problem gets worse as the size of the queue grows, whic it does at a faster rate once the load begins to rise.

I'm basically asking what can be done to solve this problem. Are there settings I can implement in the exim configuration which will encourage exim to process mail when it is delivered rather than queueing it? When it is necessary to queue mail, is it possible to keep queue runs from driving the load too high without letting the size of the queue grow to an unmanageable size?

In case it matters, I'm running Fedora 3 with the latest kernel updates on a Dual Pentium D 3.0GHz with 1GB RAM. If the mail stats in WHM can be believed, I process around 1000 messages an hour during peak times. Let me know if there's any other info you need.
 

anoetic

Active Member
Feb 5, 2006
25
0
151
Here's an example:

1 Min Load Avg: 33.40
5 Min Load Avg: 33.78
15 Min Load Avg: 31.55
Running/Total Processes: 34/186

Output from ps:
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
mailnull 17626 0.0 0.0 0 0 ? X+ 17:16 0:00 [exim]
mailnull 4511 0.0 0.0 7292 864 ? Ss 10:56 0:00 /usr/sbin/exim -bd -oX 26
mailnull 4517 0.0 0.0 7296 884 ? Ss 10:56 0:03 /usr/sbin/exim -bd
mailnull 17620 0.0 0.1 7328 1556 ? S 17:16 0:00 \_ /usr/sbin/exim -bd
mailnull 17642 0.0 0.1 7332 1640 ? S 17:16 0:00 \_ /usr/sbin/exim -bd
mailnull 17655 0.6 0.3 8120 3416 ? S 17:16 0:00 \_ /usr/sbin/exim -bd
mailnull 17667 0.0 0.0 7304 476 ? S 17:16 0:00 \_ /usr/sbin/exim -bd
mailnull 17668 0.0 0.0 7304 476 ? S 17:16 0:00 \_ /usr/sbin/exim -bd
mailnull 4527 0.0 0.0 7296 844 ? Ss 10:56 0:00 /usr/sbin/exim -C /etc/exim_outgoing.conf -q60m
mailnull 4545 0.0 0.0 7288 844 ? Ss 10:56 0:00 /usr/sbin/exim -tls-on-connect -bd -oX 465
root 2315 0.0 0.1 7284 1624 ? S 14:48 0:00 /usr/sbin/exim -C /etc/exim_outgoing.conf -Mc 1HD4Oe-0000bC-6Q 1HD4Oe-0000bC-B4 1HD4Oe-0000bC-Ci
root 3095 13.4 0.3 11596 3600 ? R 14:56 18:51 \_ /usr/sbin/exim -C /etc/exim_outgoing.conf -Mc 1HD4Oe-0000bC-6Q 1HD4Oe-0000bC-B4 1HD4Oe-0000bC-Ci
root 2994 1.9 0.4 11348 4012 ? R 14:55 2:41 /usr/sbin/exim -C /etc/exim_outgoing.conf -Mc 1HD4VW-0000mA-UQ 1HD4VX-0000mA-0B
root 4394 0.0 0.1 7292 1628 ? S 15:07 0:00 /usr/sbin/exim -C /etc/exim_outgoing.conf -Mc 1HD4hK-00018d-EV 1HD4hK-00018d-Cj
root 4395 11.3 0.3 11464 3540 ? S 15:07 14:37 \_ /usr/sbin/exim -C /etc/exim_outgoing.conf -Mc 1HD4hK-00018d-EV 1HD4hK-00018d-Cj
mailnull 17225 0.0 0.2 11464 2264 ? S 17:12 0:00 \_ /usr/sbin/exim -C /etc/exim_outgoing.conf -Mc 1HD4hK-00018d-EV 1HD4hK-00018d-Cj
mailnull 17588 0.0 0.2 11464 2264 ? S 17:15 0:00 \_ /usr/sbin/exim -C /etc/exim_outgoing.conf -Mc 1HD4hK-00018d-EV 1HD4hK-00018d-Cj
root 4398 11.3 0.4 11460 4164 ? S 15:07 14:37 /usr/sbin/exim -C /etc/exim_outgoing.conf -Mc 1HD4hK-00018d-9e
mailnull 17541 0.0 0.2 11572 2292 ? S 17:15 0:00 \_ /usr/sbin/exim -C /etc/exim_outgoing.conf -Mc 1HD4hK-00018d-9e
mailnull 17601 0.0 0.2 11580 2332 ? S 17:15 0:00 \_ /usr/sbin/exim -C /etc/exim_outgoing.conf -Mc 1HD4hK-00018d-9e
root 4899 0.0 0.1 7292 1628 ? S 15:13 0:00 /usr/sbin/exim -C /etc/exim_outgoing.conf -Mc 1HD4mn-0001Gx-F7 1HD4mn-0001Gx-DM 1HD4mn-0001Gx-8x
root 4900 10.6 0.3 11476 3484 ? R 15:13 13:08 \_ /usr/sbin/exim -C /etc/exim_outgoing.conf -Mc 1HD4mn-0001Gx-F7 1HD4mn-0001Gx-DM 1HD4mn-0001Gx-8x
root 4906 10.5 0.4 11468 4104 ? R 15:13 12:57 /usr/sbin/exim -C /etc/exim_outgoing.conf -Mc 1HD4mn-0001Gx-Az
root 5551 0.0 0.1 7292 1624 ? S 15:22 0:00 /usr/sbin/exim -C /etc/exim_outgoing.conf -Mc 1HD4ux-0001RR-Q2 1HD4ux-0001RR-Uw 1HD4uy-0001RR-0L
root 8893 8.7 0.3 11476 3464 ? R 15:50 7:34 \_ /usr/sbin/exim -C /etc/exim_outgoing.conf -Mc 1HD4ux-0001RR-Q2 1HD4ux-0001RR-Uw 1HD4uy-0001RR-0L
root 5555 9.8 0.4 11464 4092 ? R 15:22 11:16 /usr/sbin/exim -C /etc/exim_outgoing.conf -Mc 1HD4ux-0001RR-SX
root 5731 0.0 0.1 7280 1620 ? S 15:23 0:00 /usr/sbin/exim -C /etc/exim_outgoing.conf -Mc 1HD4wM-0001UI-RY 1HD4wM-0001UI-LE 1HD4wM-0001UI-Pt
root 5732 10.2 0.3 11456 3460 ? R 15:23 11:36 \_ /usr/sbin/exim -C /etc/exim_outgoing.conf -Mc 1HD4wM-0001UI-RY 1HD4wM-0001UI-LE 1HD4wM-0001UI-Pt
root 5736 10.1 0.4 11472 4096 ? R 15:23 11:26 /usr/sbin/exim -C /etc/exim_outgoing.conf -Mc 1HD4wM-0001UI-NU
root 7807 4.5 0.4 11464 4148 ? S 15:40 4:23 /usr/sbin/exim -C /etc/exim_outgoing.conf -Mc 1HD5Cs-00021c-FL 1HD5Cs-00021c-IB 1HD5Cs-00021c-IX
1HD5Cs-00021c-H4
mailnull 17657 0.0 0.2 11592 2344 ? S 17:16 0:00 \_ /usr/sbin/exim -C /etc/exim_outgoing.conf -Mc 1HD5Cs-00021c-FL 1HD5Cs-00021c-IB 1HD5Cs-00021c-IX
1HD5Cs-00021c-H4
root 8323 0.0 0.1 7284 1624 ? S 15:44 0:00 /usr/sbin/exim -C /etc/exim_outgoing.conf -Mc 1HD5H0-00029t-PJ 1HD5H0-00029t-Q6 1HD5H0-00029t-F3
root 17569 6.2 0.3 11336 3104 ? R 17:15 0:03 \_ /usr/sbin/exim -C /etc/exim_outgoing.conf -Mc 1HD5H0-00029t-PJ 1HD5H0-00029t-Q6 1HD5H0-00029t-F3
mailnull 8996 0.0 1.7 23932 17936 ? S 15:51 0:00 MailWatch SQL
root 9398 1.0 0.3 11332 3900 ? R 15:55 0:52 /usr/sbin/exim -C /etc/exim_outgoing.conf -Mc 1HD5Ra-0002RX-0H 1HD5RZ-0002RX-TK 1HD5RZ-0002RX-Vk
root 10239 0.0 0.1 7280 1620 ? S 16:04 0:00 /usr/sbin/exim -C /etc/exim_outgoing.conf -Mc 1HD5aS-0002ev-Uu 1HD5aT-0002ev-0O 1HD5aS-0002ev-P1
root 10246 8.2 0.3 11464 3416 ? R 16:04 5:55 \_ /usr/sbin/exim -C /etc/exim_outgoing.conf -Mc 1HD5aS-0002ev-Uu 1HD5aT-0002ev-0O 1HD5aS-0002ev-P1
root 10245 8.1 0.3 11332 3992 ? R 16:04 5:51 /usr/sbin/exim -C /etc/exim_outgoing.conf -Mc 1HD5aS-0002ev-Sa
root 11944 0.0 0.1 7280 1620 ? S 16:25 0:00 /usr/sbin/exim -C /etc/exim_outgoing.conf -Mc 1HD5u9-00036U-3S 1HD5u8-00036U-Va
root 11945 6.9 0.3 11332 3316 ? R 16:25 3:35 \_ /usr/sbin/exim -C /etc/exim_outgoing.conf -Mc 1HD5u9-00036U-3S 1HD5u8-00036U-Va
root 11949 0.0 0.1 7284 1624 ? S 16:25 0:00 /usr/sbin/exim -C /etc/exim_outgoing.conf -Mc 1HD5u9-00036U-2X 1HD5u9-00036U-0m
root 11950 6.9 0.3 11348 3396 ? R 16:25 3:32 \_ /usr/sbin/exim -C /etc/exim_outgoing.conf -Mc 1HD5u9-00036U-2X 1HD5u9-00036U-0m
 

anoetic

Active Member
Feb 5, 2006
25
0
151
Here is the remainder of the report:

root 12468 0.0 0.1 7288 1624 ? S 16:30 0:00 /usr/sbin/exim -C /etc/exim_outgoing.conf -Mc 1HD5z0-0003Ev-Ml 1HD5z0-0003Ev-LC 1HD5z0-0003Ev-Gu
root 12469 6.2 0.3 11340 3320 ? R 16:30 2:53 \_ /usr/sbin/exim -C /etc/exim_outgoing.conf -Mc 1HD5z0-0003Ev-Ml 1HD5z0-0003Ev-LC 1HD5z0-0003Ev-Gu
root 12473 6.5 0.3 11336 3940 ? R 16:30 3:02 /usr/sbin/exim -C /etc/exim_outgoing.conf -Mc 1HD5z0-0003Ev-Iw
root 13077 0.0 0.1 7280 1620 ? S 16:35 0:00 /usr/sbin/exim -C /etc/exim_outgoing.conf -Mc 1HD63h-0003OZ-Sj 1HD63h-0003OZ-Ue 1HD63h-0003OZ-Of
root 13078 6.7 0.3 11336 3352 ? R 16:35 2:46 \_ /usr/sbin/exim -C /etc/exim_outgoing.conf -Mc 1HD63h-0003OZ-Sj 1HD63h-0003OZ-Ue 1HD63h-0003OZ-Of
root 13081 6.1 0.3 11336 3940 ? R 16:35 2:32 /usr/sbin/exim -C /etc/exim_outgoing.conf -Mc 1HD63h-0003OZ-QR
root 13808 0.0 0.1 7284 1628 ? S 16:41 0:00 /usr/sbin/exim -C /etc/exim_outgoing.conf -Mc 1HD6A6-0003aX-Cv 1HD6A6-0003aX-J9 1HD6A6-0003aX-Jv
root 16049 5.1 0.3 11336 3280 ? R 17:02 0:42 \_ /usr/sbin/exim -C /etc/exim_outgoing.conf -Mc 1HD6A6-0003aX-Cv 1HD6A6-0003aX-J9 1HD6A6-0003aX-Jv
root 13813 5.7 0.3 11344 3940 ? R 16:41 1:59 /usr/sbin/exim -C /etc/exim_outgoing.conf -Mc 1HD6A6-0003aX-IB
root 14368 0.0 0.1 7288 1624 ? S 16:45 0:00 /usr/sbin/exim -C /etc/exim_outgoing.conf -Mc 1HD6Ds-0003jO-4j 1HD6Ds-0003jO-5u 1HD6Ds-0003jR-Bk
1HD6Ds-0003jO-2V 1HD6Ds-0003jO-6J
root 14369 8.0 0.3 11464 3448 ? R 16:45 2:29 \_ /usr/sbin/exim -C /etc/exim_outgoing.conf -Mc 1HD6Ds-0003jO-4j 1HD6Ds-0003jO-5u 1HD6Ds-0003jR-Bk
1HD6Ds-0003jO-2V 1HD6Ds-0003jO-6J
root 14616 0.0 0.1 7280 1620 ? S 16:48 0:00 /usr/sbin/exim -C /etc/exim_outgoing.conf -Mc 1HD6Gd-0003nK-C2 1HD6Gd-0003nK-Ik 1HD6Gd-0003nK-Hq
root 16977 5.5 0.3 11332 3256 ? R 17:10 0:19 \_ /usr/sbin/exim -C /etc/exim_outgoing.conf -Mc 1HD6Gd-0003nK-C2 1HD6Gd-0003nK-Ik 1HD6Gd-0003nK-Hq
root 14645 5.9 0.3 11344 3944 ? R 16:48 1:39 /usr/sbin/exim -C /etc/exim_outgoing.conf -Mc 1HD6Gd-0003nK-DR
root 14771 0.0 0.1 7292 1628 ? S 16:50 0:00 /usr/sbin/exim -C /etc/exim_outgoing.conf -Mc 1HD6I1-0003pp-CQ 1HD6I0-0003pm-4Q 1HD6I0-0003po-Ru
1HD6I1-0003pp-DX 1HD6I1-0003pp-7y 1HD6Hx-0003pl-RX
root 14772 8.1 0.3 11468 3432 ? R 16:50 2:09 \_ /usr/sbin/exim -C /etc/exim_outgoing.conf -Mc 1HD6I1-0003pp-CQ 1HD6I0-0003pm-4Q 1HD6I0-0003po-Ru
1HD6I1-0003pp-DX 1HD6I1-0003pp-7y 1HD6Hx-0003pl-RX
root 14914 0.0 0.1 7288 1624 ? S 16:52 0:00 /usr/sbin/exim -C /etc/exim_outgoing.conf -Mc 1HD6KL-0003sQ-Jq 1HD6KL-0003sQ-R2 1HD6KL-0003sQ-Sc
root 14915 5.5 0.3 11340 3272 ? R 16:52 1:20 \_ /usr/sbin/exim -C /etc/exim_outgoing.conf -Mc 1HD6KL-0003sQ-Jq 1HD6KL-0003sQ-R2 1HD6KL-0003sQ-Sc
root 14924 5.3 0.3 11332 3924 ? R 16:52 1:16 /usr/sbin/exim -C /etc/exim_outgoing.conf -Mc 1HD6KK-0003sL-64 1HD6KL-0003sQ-Og
root 16085 0.0 0.1 7284 1624 ? S 17:03 0:00 /usr/sbin/exim -C /etc/exim_outgoing.conf -Mc 1HD6Ut-0004B7-Qd 1HD6Ut-0004B7-V3
root 16086 4.9 0.3 11336 3256 ? R 17:03 0:39 \_ /usr/sbin/exim -C /etc/exim_outgoing.conf -Mc 1HD6Ut-0004B7-Qd 1HD6Ut-0004B7-V3
root 16184 0.0 0.1 7280 1624 ? S 17:03 0:00 /usr/sbin/exim -C /etc/exim_outgoing.conf -Mc 1HD6Ut-0004B7-S5 1HD6Ur-0004B1-24 1HD6Uv-0004B6-Ex
1HD6Ut-0004B7-Tt
root 16185 4.7 0.3 11332 3272 ? R 17:03 0:36 \_ /usr/sbin/exim -C /etc/exim_outgoing.conf -Mc 1HD6Ut-0004B7-S5 1HD6Ur-0004B1-24 1HD6Uv-0004B6-Ex
1HD6Ut-0004B7-Tt
root 16718 0.0 0.1 7280 1620 ? S 17:08 0:00 /usr/sbin/exim -C /etc/exim_outgoing.conf -Mc 1HD6a1-0004L4-Fg 1HD6a1-0004L4-FI 1HD6a1-0004L4-6n
1HD6a1-0004L4-Do
root 17297 9.7 0.3 11328 3292 ? R 17:13 0:18 \_ /usr/sbin/exim -C /etc/exim_outgoing.conf -Mc 1HD6a1-0004L4-Fg 1HD6a1-0004L4-FI 1HD6a1-0004L4-6n
1HD6a1-0004L4-Do
root 17015 0.0 0.1 7280 1620 ? S 17:11 0:00 /usr/sbin/exim -C /etc/exim_outgoing.conf -Mc 1HD6cc-0004Q7-Km 1HD6cc-0004Q7-Iv 1HD6cc-0004Q7-KP
1HD6cc-0004Q7-BZ
root 17640 10.7 0.3 11328 3136 ? R 17:16 0:01 \_ /usr/sbin/exim -C /etc/exim_outgoing.conf -Mc 1HD6cc-0004Q7-Km 1HD6cc-0004Q7-Iv 1HD6cc-0004Q7-KP
1HD6cc-0004Q7-BZ
root 17117 4.8 0.3 11332 3880 ? R 17:12 0:12 /usr/sbin/exim -C /etc/exim_outgoing.conf -MCS -MCP -MC remote_smtp it1.ptt.yu 212.62.32.7
2 1HCyCT-0005tE-JJ

Output from vmstat:
procs -----------memory---------- ---swap-- -----io---- --system-- ----cpu----
r b swpd free buff cache si so bi bo in cs us sy id wa
32 0 576 89368 52072 372656 0 0 18 147 175 81 91 2 7 0
 

rgripoll

Active Member
Mar 19, 2003
27
0
151
Did you find any solution to this? I'm having the same problem.
 

anoetic

Active Member
Feb 5, 2006
25
0
151
Well, it's been a while, but ultimately, I totally removed MailScanner and eventually exim caught up with the mail queue and the load went down. Apparently, just disabling MailScanner isn't enough. I had to totally remove it to solve the problem.
 

anoetic

Active Member
Feb 5, 2006
25
0
151
In my case, it was MailScanner installed by ConfigServer. They could not explain why MailScanner caused this problem -- only that MailScanner apparently could not handle the mail traffic passing through my server. I have quite a few clients running Mailman mailing lists and it was the list mail that was most effected.
 

abubin

Well-Known Member
Dec 7, 2004
401
3
168
hmm...I thought you got this problem after upgrading to cpanel 11. But apparently you did not mention about any upgrades. Sorry, my mind is filled up with this upgrade thing.

Anyway, reading your first post again, I realized that your load is really high. 1000 mails in an hour is too much for a server with only 1gb ram.

If I were you, I would seriously consider adding more RAM. 4GB would be sweet but 2GB would also be better than 1GB. Personally, I can't live without mailscanner. But for your case, maybe you need to save on the cost. It's all up to you.
 

anoetic

Active Member
Feb 5, 2006
25
0
151
Perhaps you got the idea that this problem occurred after upgrading to cPanel 11 from another thread. I was (and still am) having trouble with Spam Assassin after upgrading to cPanel 11, but my problem with MailScanner occurred much before then. As I said, I long ago removed MailScanner and everything has been running smoothly (load less than 1 most of the time) since then.

Even now, SA works, but cPanel doesn't seem to know it. If I tick the checkbox to have cPanel notify me when SA fails (in the Service Manager) then cPanel sends me constant notifications that SA is failing even though it is not failing. It is running. I've never found a solution to this problem other than unchecking that option in the Service Manager.