Intermittent prompt for email username/password on mail clients

[zaher]

Hello,

I have an unusual problem with the email service. All of a sudden the mail clients of all accounts would prompt users to enter their username/password and of course would keep prompting even if user puts correct credentials. Then shortly after, everything is back to normal without changing anything.
During this time, load of the server is good, memory usage is good, and the mail service would still be running.

Background Info

1. CENTOS 6.5 x86_64
2. WHM 11.44.0 (build 19)
3. Running Exim
4. Running Courier
5. Maximum IMAP Connections: 50
6. Maximum POP3 Connections: 50

The Number of Authentication Daemons was 5, but I have increased it to 6 just now to see if this is going to solve the problem. Yet, what could the other reason for such a behavior?

Many thanks

 

[cPanelMichael]

Hello

Are you using Courier or Dovecot? Also, did you check to ensure the accounts were not locked out by cPhulk brute force detection?

Thank you.

 

[zaher]

Hello

Are you using Courier or Dovecot? Also, did you check to ensure the accounts were not locked out by cPhulk brute force detection?

Thank you.

Hello Micheal,
Thanks for the reply. As mentioned earlier, I am running Courier.
CPhulk is disabled; and in all cases the service resumes is shorter interval that if CPhulk was to automatically unblock the IP address later.
I am using however, ConfigServer Security & Firewall (CSF), but this is not the cause of this issue and I don't'see any blocks and our IP addresses are all white-listed.

 

[cPanelMichael]

Look for instances of "authentication error: Input/output error" in /var/log/maillog. EX:

# grep "Input/output error" /var/log/maillog

You will typically see several occurrences of this error when there are not enough authentication daemons.

However, it could also indicate there are IP addresses with several failed login attempts which should be blocked. For example, you can run this command:

# grep 'LOGIN FAILED' /var/log/maillog|awk '{print $9}'|sort|uniq -c | sort -n

Blocking the IP addresses that have several failed connection attempts (this is typically a brute force attack) in your firewall can be useful in these types of cases.

Thank you.

 

[zaher]

Look for instances of "authentication error: Input/output error" in /var/log/maillog. EX:

# grep "Input/output error" /var/log/maillog

You will typically see several occurrences of this error when there are not enough authentication daemons.

I have executed the suggested line and I have errors such

Jul  7 00:13:31 server pop3d-ssl: authentication error: Input/output error
Jul  7 04:43:33 server pop3d: authentication error: Input/output error

These errors are however mostly occurring outside the peak hours of our server; so they can't be legitimate log-in attempts - check the following section of this thread.

However, it could also indicate there are IP addresses with several failed login attempts which should be blocked. For example, you can run this command:

# grep 'LOGIN FAILED' /var/log/maillog|awk '{print $9}'|sort|uniq -c | sort -n

Blocking the IP addresses that have several failed connection attempts (this is typically a brute force attack) in your firewall can be useful in these types of cases.

Thank you.

I have CSF automatically dealing with this after 5 failed login attempts; and actually i have a lot of these cases, just yesterday I had 190x5 unique failed login attempts mainly to smtpauth.


I have already increased the number of mail deamons from 5 to 6 before creating this thread; shall I increase it more despite the above?

 

[JaredR.]

You can try increasing the number of mail daemons. On a busy server, finding the right combination of settings for the POP/IMAP server can take a lot of experimentation, and it may need occasional changes as the traffic on the server changes.

Also, while we do not support CSF or LFD, we have occasionally seen LFD kill legitimate processes, including Courier or Dovecot processes. I recommend that you check the LFD log to see if it is perhaps killing the processes, because that could cause the behavior you have observed.

 

[zaher]

You can try increasing the number of mail daemons. On a busy server, finding the right combination of settings for the POP/IMAP server can take a lot of experimentation, and it may need occasional changes as the traffic on the server changes.

I am suspecting it is not due to legitimate high traffic, it is most likely due to DOS (due to the facts mentioned upthere).
It would be nice to have a feature in cPanel to adjust daemons based on schedule (hours of the day/days of the week).

Also, while we do not support CSF or LFD, we have occasionally seen LFD kill legitimate processes, including Courier or Dovecot processes. I recommend that you check the LFD log to see if it is perhaps killing the processes, because that could cause the behavior you have observed.

I have checked LFD log for any occurrence related to that but it doesn't seem to be killing these processes.

 

[JaredR.]

You are welcome to submit a feature request here:

cPanel Feature Requests

Our developers use feature requests to gauge the demand for a requested feature, and the more votes and comments it receives, the more likely it is to be considered for a future version of cPanel.

 

[zaher]

What I noticed is that when this thing happens, whostmgr2 - top ./top would top the CPU usage. Is that of any sort related?

 

[JaredR.]

whostmgr2 - top ./top is Home » System Health » Process Manager. top is the process that gets and reports the data on the processes, and top is itself a process, so it reports on itself. top does cause some load, and it is completely normal to see it in the process list while you are running it.

Home » System Health » Process Manager is the same as running top from the Linux command prompt. If top is near the top of the process list in order of either memory or CPU usage, then the overall load on your server usually is not very high.

 

[zaher]

No worries then.

I have another question, why does the server consider them as failed login attempts when it is actually running out of daemons?
Ex. Jul 16 12:43:06 server pop3d-ssl: LOGIN FAILED

This is causing CSF to block the IPs of legitimate users.

Thanks

 

[cPanelMichael]

I have another question, why does the server consider them as failed login attempts when it is actually running out of daemons?
Ex. Jul 16 12:43:06 server pop3d-ssl: LOGIN FAILED

It's accurate that the login attempt itself failed. The reason why it failed, assuming you correlated the failure to an entry in /var/log/maillog, is due to the lack of an authentication daemon.

Thank you.