refused: too many connections

[Jcats]

We've been getting a large amount of reports from random customers reporting connectivity issues specifically with exim, when checking their IP's, all have the same issue with hitting the 100 SMTP limit.

refused: too many connections

Wanted to see if others are experiencing this as well in case there is an issue with a recent cPanel update.

Severs running: 11.90.0.5

 

[cPanelLauren]

I've had one other forums user indicating that this limit is being reached. I'm curious, are all the users checking from the same location when this occurs? What's the log output from /var/log/exim_mainlog if you're able to grab it for me as well.

Thanks!

 

[Jcats]

We have a few cases, one is from an office so there are several devices on the same IP which initially I was leaning towards too many devices but they've had service with us for quite some time and then not to far after that report we a few others reporting the same issue with just 1 - 2 devices like a desktop mail client and a mobile client and still reaching these limits.

 

[Jcats]

Getting more and more reports its like everyone is starting off with 90+ attempts, example:

# grep 123.123.123.123  /var/log/exim_mainlog |grep count
2020-09-06 22:23:50 SMTP connection from [123.123.123.123]:55095 (TCP/IP connection count = 100)
2020-09-07 01:14:18 SMTP connection from [123.123.123.123]:55627 (TCP/IP connection count = 100)
2020-09-07 01:34:19 SMTP connection from [123.123.123.123]:55646 (TCP/IP connection count = 99)
2020-09-07 09:34:38 SMTP connection from [123.123.123.123]:50438 (TCP/IP connection count = 100)
2020-09-07 09:41:05 SMTP connection from [123.123.123.123]:50516 (TCP/IP connection count = 99)
2020-09-07 09:56:56 SMTP connection from [123.123.123.123]:50597 (TCP/IP connection count = 100)
2020-09-07 10:19:14 SMTP connection from [123.123.123.123]:50774 (TCP/IP connection count = 95)
2020-09-07 22:11:56 SMTP connection from [123.123.123.123]:50056 (TCP/IP connection count = 100)
2020-09-08 01:19:57 SMTP connection from [123.123.123.123]:50657 (TCP/IP connection count = 100)
2020-09-08 01:53:21 SMTP connection from [123.123.123.123]:51723 (TCP/IP connection count = 100)
2020-09-08 01:56:24 SMTP connection from [123.123.123.123]:51743 (TCP/IP connection count = 100)
2020-09-08 09:07:03 SMTP connection from [123.123.123.123]:51986 (TCP/IP connection count = 100)
2020-09-08 09:25:51 SMTP connection from [123.123.123.123]:52037 (TCP/IP connection count = 100)
2020-09-08 10:25:52 SMTP connection from [123.123.123.123]:52312 (TCP/IP connection count = 99)
2020-09-08 10:49:41 SMTP connection from [123.123.123.123]:52348 (TCP/IP connection count = 100)
2020-09-08 12:30:59 SMTP connection from [123.123.123.123]:52525 (TCP/IP connection count = 98)
2020-09-08 12:51:09 SMTP connection from [123.123.123.123]:53179 (TCP/IP connection count = 97)
2020-09-08 13:05:52 SMTP connection from [123.123.123.123]:53198 (TCP/IP connection count = 98)
2020-09-08 13:10:02 SMTP connection from [123.123.123.123]:53209 (TCP/IP connection count = 99)
2020-09-08 13:29:54 SMTP connection from [123.123.123.123]:53236 (TCP/IP connection count = 98)
2020-09-08 13:31:46 SMTP connection from [123.123.123.123]:53238 (TCP/IP connection count = 99)
2020-09-08 13:44:28 SMTP connection from [123.123.123.123]:53255 (TCP/IP connection count = 100)
2020-09-08 13:46:54 SMTP connection from [123.123.123.123]:53256 (TCP/IP connection count = 100)
2020-09-08 13:49:53 SMTP connection from [123.123.123.123]:53259 (TCP/IP connection count = 100)
2020-09-08 14:26:47 SMTP connection from [123.123.123.123]:53313 (TCP/IP connection count = 97)

 

[keat63]

This won't particularly help your case but may assist others in the future.
Here is a typical snapshot of the connection counts from my server

2020-09-09 08:13:21 SMTP connection from [x.x.x.x]:36164 (TCP/IP connection count = 2)
2020-09-09 08:13:49 SMTP connection from [x.x.x.x]:43996 (TCP/IP connection count = 3)
2020-09-09 08:14:53 SMTP connection from [x.x.x.x]:45864 (TCP/IP connection count = 1)
2020-09-09 08:16:19 SMTP connection from [x.x.x.x]:15601 (TCP/IP connection count = 1)
2020-09-09 08:16:19 SMTP connection from [x.x.x.x]:28856 (TCP/IP connection count = 2)
2020-09-09 08:16:51 SMTP connection from [x.x.x.x]:60624 (TCP/IP connection count = 3)

 

[cPanelLauren]

@Jcats is the IP 123.123.123.123 the clients IP address or is it an unrecognized IP?

 

[Jcats]

That is a client IP but I think the problem was actually from some kind of an attack, I ended up blocking a few ranges that had a lot of failed counts throughout our network and the issues seems to of subsided. I didn't get a chance to check exim doc but I am assuming apart from exim rate limiting a single IP, there is also most likely an option that starts to limit all connections if a 'global' threshold is met so the attack was effectively causing limitations for the entire mail server, just an assumption as again I haven't had a chance to really dig further into it as reports have stopped since blocking those ranges.

 

[keat63]

I saw 123.123.123.123 and automatically assumed that you had obfuscated the real IP to protect the identity

 

[Jcats]

Yeah I did, but it was the same IP and it was the clients IP. That IP wasn't related to the attacks we were seeing, I just thought it was odd the connection count started reporting after 90+ hits when usually you will actually see it increment from 1

 

[cPanelLauren]

@Jcats

That's exactly what I was going for, the only instances I've seen this occur is when there actually is an attack. I'm glad you were able to find the issue though.

 

[Jcats]

Ahh gotcha, sorry misunderstood but thank you :D