ADVISORY: Exim / Courier-IMAP Authentication Issue: Feb 22 2011

[cPanelJared]

We have become aware of a problem that is affecting multiple servers for multiple customers and is preventing sending and receiving mail. So far, the problem is only affecting Courier-IMAP, that we have seen. The symptom is a message like the following in /var/log/maillog:

2011-02-28 08:46:13 courier_login authenticator failed for 201008195189.[redacted] ([redacted]) [[redacted]]: 435 Unable to authenticate at present (set_id=[redacted]): socket read timed out inside "and{...}" condition
2011-02-28 08:46:13 courier_login authenticator failed for ([redacted]) [[redacted]]: 435 Unable to authenticate at present (set_id=[redacted]): socket read timed out inside "and{...}" condition

The common theme that we are seeing is the following:

socket read timed out inside "and{...}" condition

Increasing the number of authentication daemons in Main >> Service Configuration >> Mailserver Configuration seems to help in some cases, but it is not a permanent fix.

As a temporary work-around, if you are using Courier-IMAP and you encounter this problem, you can change to Dovecot using Main >> Service Configuration >> Mailserver Selection.

Our developers are aware of this issue and are investigating the cause now. This is being tracked using internal case number 47563.

[cPanelJared]

As far as we have been able to observe in our investigation, the cause of this issue was a brute-force attack against Brazilian top-level domains (domains ending with .br). In one example, there were over 140,000 attempts to authenticate via SMTP using a user @ a Brazilian domain on a single server. We have not yet observed this behavior happening on a server that does not host Brazilian domains.

If you have noticed large-scale brute-force attempts on mailboxes @ Brazilian domains on your server, please provide the IP addresses that you have noticed making the attempts.

A defense against this, if your server has been affected, is to change your POP/IMAP server to Dovecot, using Main >> Service Configuration >> Mailserver Selection, and to enable cPHulkd brute force protection, in Main >> Security Center >> cPHulk Brute Force Protection.

[sulnet]

I changed to dovecot, but the problem still.


dovecot_login authenticator failed

[cPanelJared]

If you have changed to Dovecot and mail authentication is still failing, please submit a ticket so that we may log into your server and work with you individually to achieve a timely resolution.

[silas_i]

If you change to Dovecot and still doesn´t work, after changing Mail Server Type, use the /scripts/mailperm/ to fix a mailbox.

In same cases will need change password to back work a account!

[sulnet]

Where can I to observe the solution for Case 47563?

[cPanelJared]

Internal cases are not publicly viewable.

In every case we have seen with the symptoms described in this thread, the problem was caused by a large brute-force attack against e-mail addresses with domains ending in .br. Every server we observed had thousands, or tens of thousands, of log-in failures to e-mail addresses ending in .br, coming from different source IP addresses. It is likely that a botnet was involved, because the symptoms were so similar on so many servers.

Based on these observations, we concluded that the problem was not caused by cPanel or by Courier-IMAP. Switching to Dovecot only masks the problem. The problem is still that the server is under a brute-force attack against .br domains, and this is a problem no matter what service is used for POP and IMAP.

The best advice that we can give is to observe /var/log/maillog and identify which source IP addresses are attempting the failed log-ins, and block them in the server's firewall. In severe cases, I would also recommend contacting your data center and asking if the offending IP addresses can be blocked upstream, before they ever get to your server.

This is an unusual situation, but our conclusion is that it is the result of a very targeted attack against .br e-mail addresses, and not the result of a problem in cPanel or Courier-IMAP. The issue can be mitigated using basic system administration, and you should contact your data center for assistance with blocking the IP addresses that are causing the problem on your server.

[asciigirl]

I've fixed a similar issue using fail2ban.

[cPanelDavidG]

I've fixed a similar issue using fail2ban.

Can you, and everyone else that experiences this issue please submit a bug report? You can do this by clicking "bugs" at the top of the page. If you experience issues submitting a bug, send me an email to sales@cpanel.net so we can route you into the bug report system via email. This will let us see if the issue is consistent with the observations Jared mentioned or if something else is going on that needs addressing. The more bug reports we receive, the more complete picture we receive and the faster a resolution can appear in the product. This will also give us the ability to email you when the product defect is resolved.

Since this internal case has been closed as a result of the observations Jared mentioned, I am removing it from this thread's title.

Anyone else experiencing this issue, please submit a bug report. I am now closing this thread.