Strange timeout problems

[a.post]

Over the last few weeks I'm experiencing strange behaviour on a small cPanel server. Connections to SSH, SMTP, IMAP and HTTP (incl. WHM web services) do not respond for several minutes after which everything works again for a few minutes. However icmp/ping does give me instant replies all the time and active SSH sessions are not dropped but stay alive.

This behaviour seems to be related to a single host (the one I use to manage the machine). Connections from other IP's at the same time do not show any trouble.

There is nothing in the logs which give me any clue. Also the appearance of the problem (timing) do not give me any clues on lockouttimers of some sort.

I have:
- checked all logs for any clues: apache, cpanel, dovecot, lfd, maillog, exim, messages, audit/cphulk, modsec, etc.
- disabled LFD/CSF > no effect
- disabled selinux > no effect
- disabled cPHulk > no effect
- checked system monitoring (graphs), no cpu, network, memory, io related issues seem apparent
- checked all whitelists to see if the troubled host is still on it (yes it is)
- checked if the IP was listed in iptables, no it is not
- checked if there is a connection limit issue (not more then a handful of established connections)
- updated all yum packages
- rebooted several times
- made a tcpdump on the external interface of the cPanel machine, it confirms what I see. Incoming regular request and after that TCP retransmits for all TCP related traffic from the troubled host, in combination with good traffic from the same services to other IP's. And also working icmp requests and the active ssh session from the troubled host. And after a few minutes everything turns to normal and works again.

Somehow it seems some application is blocking the troubled IP from making new TCP connections. However I'm out of clues... does anyone have any leads for me to look at?

P.S. It's an oversized AWS Amazon Linux machine which has been working without any (noticeable) problems over de last year or so.

 

[cPanelMichael]

This behaviour seems to be related to a single host (the one I use to manage the machine). Connections from other IP's at the same time do not show any trouble.

Hello,

It's possible the issue relates to the router or network of the IP address making the connection to the cPanel server. Have you checked to verify if there are any issues with the Internet provider of the connecting user?

Thank you.

 

[weeming21]

some of my customers also facing this problems
if more than 3 or 4 users in same network connect to webmail, then sometimes will be connection timeout

when the problem occur, they still able to ping to the webmail ip, but cannot telnet, stuck at connecting to 1.1.1.1 .....
and they can telnet to another ip of the same server without any problem

 

[a.post]

In my case I did not manage to get a definitive answer. However we've made a guess it was due to some IMAP or SMTP connection overflow. Mostly clients with Postbox (OSX Mail client) which seem to generate the issue. Switching back to native OSX Mail seemed to have cooled down the servers.

I think we could have pinpointed the issue if we would have switched to full debugging mode on exim/dovecot. However the problem solved itself in a few days, so we didn't invest any extra time in it.

 

[cPanelMichael]

some of my customers also facing this problems
if more than 3 or 4 users in same network connect to webmail, then sometimes will be connection timeout

when the problem occur, they still able to ping to the webmail ip, but cannot telnet, stuck at connecting to 1.1.1.1 .....
and they can telnet to another ip of the same server without any problem

Feel free to open a support ticket using the link in my signature if you'd like us to take a closer look and rule out any issues with the cPanel software.

Thank you.