SOLVED Email stuck in queue due to server load

[pixelhub]

Hi,

I am having some trouble with our server, that i hope someone can help me with,

all mail sent and received gets put in to the queue and stays there until it is manually pushed through.

I sent a test email and the exim_mainlog showed this as the error

2017-10-14 10:10:01 1e3ISG-0000e2-U6 H=mail-lf0-f66.google.com [209.85.215.66]:53079 Warning: "SpamAssassin as user detected message as NOT spam (0.4)"
2017-10-14 10:10:01 1e3ISG-0000e2-U6 <= [email protected] H=mail-lf0-f66.google.com [209.85.215.66]:53079 P=esmtps X=TLSv1:AES128-SHA:128 CV=no S=4251 [email protected]l.com T="test incoming mail" for [email protected]
2017-10-14 10:10:01 1e3ISG-0000e2-U6 no immediate delivery: load average 146.31
2017-10-14 10:10:01 SMTP connection from mail-lf0-f66.google.com [209.85.215.66]:53079 closed by QUIT

there is also a lot of spam bounce back emails that I believe is email spoofing.

with the load average being 146.31 i checked what was being used on the server and it showed MYSQL hogging the CPU.

I checked MYSQL and it goes offline a short time after restarting the service or re booting the server.

[email protected] [~]# top c

top - 10:50:41 up 13:15, 1 user, load average: 143.92, 143.97, 143.37
Tasks: 312 total, 1 running, 309 sleeping, 2 stopped, 0 zombie
Cpu(s): 18.6%us, 29.8%sy, 0.0%ni, 51.6%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Mem: 4148308k total, 3914864k used, 233444k free, 223532k buffers
Swap: 2096472k total, 60k used, 2096412k free, 2614496k cached

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND

2949 mysql 18 0 811m 465m 8236 S 192.1 11.5 1507:46 /usr/sbin/mysqld --basedir=/usr --datadir=/var/lib/mysql --plugin-dir=/usr/lib/mysql/plugin --user=mysql -
6028 root 15 0 2712 1104 748 R 0.7 0.0 0:00.20 top c
1755 root 16 0 2280 516 400 S 0.3 0.0 0:09.87 mcstransd

[email protected] [~]# mysqladmin processlist

| Id | User | Host | db | Command | Time | State | Info
| 3 | eximstats | localhost | eximstats | Sleep | 1658 | |
| 43924 | leechprotect | localhost | leechprotect | Sleep | 6710 | |
| 46639 | cphulkd | localhost | cphulkd | Query | 1141 | statistics | SELECT USER,SERVICE,TYPE,TIMESTAMPDIFF(SECOND, '1970-01-01', LOGINTIME) as LOGINTIME,TIMESTAMPDIFF(S
| 47119 | root | localhost | | Query | 0 | init | show processlist

cphulkd is repeated 148 times

At this point I'm stuck on what to do. I think that MYSQL is hogging the resources and the result is the server cant preform other tasks such as send and receive emails. Is that right?

Any help would be greatly appreciated

Thanks
Craig

 

[pixelhub]

I have been looking at the /ect/my.cnf file and installed MySQLTuner form

https://github.com/major/MySQLTuner-perl

I made the recommended changes to the variables in the my.cnf file, restarted MYSQL and all seems to be working.

there as some general recommendations i don't understand

    Control warning line(s) into /var/lib/mysql/server.domain.co.uk.err file
    Control error line(s) into /var/lib/mysql/server.domain.co.uk.err file
    Configure your accounts with ip or subnets only, then update your configuration with skip-name-resolve=1

But after changing the my.cnf file, all is good.
hope this helps someone else (if it makes any sense).

Cheers
Craig

 

[Muhammed Fasal]

Do you have CSF firewall installed on your server? If yes, you can disable cpHulkd service on your server safely. Because they both do the same job, CSF will do more than what cpHulkd can do.

cPHulk is only Brute Force detection/failed login blocking, whereas a Firewall or a security solution (CSF) includes a lot more.
If you need advanced features for your server security like to avoid Apache DDOS attack then only you need to think about CSF otherwise cPHulk will almost do all the other features provided by CSF like auto-blocking of IP address on failed login attempts.

Actually, CSF works on top of iptables. The rules you add in CSF will be added to iptables on the back end. While cPHulk uses MySQL database rather than iptables.

As I have mentioned cPHulk uses a database, it may consume more resource while on a BruteForce attack.

You can check the cPHulkd log entries at:

tail -f /usr/local/cpanel/logs/cphulkd.log​

You can follow this step on Commandline to disable cPHulkd service:

/usr/local/cpanel/bin/cphulk_pam_ctl --disable​

OR

/usr/local/cpanel/etc/init/stopcphulkd

​

 

[cPWilliamL]

Hi @pixelhub,

I am glad to hear you were able to optimize MySQL. I thought I'd also provide more information on exim and excessive load. To reduce system stress, the exim service will not deliver mail once the load is over 36(default). This is set via 'deliver_queue_load_max' at WHM > Service Configuration > Exim Configuration Manager > Advanced. With that said, addressing the load issue first was certainly the best way to go about fixing this.

I'll go ahead and mark this thread solved for you. Thanks for sharing your solution!