Every morning at 4:44am I get failed pop3 logins reported. But they are not password problems.

[jasgot]

I think it happens during a software upgrade. LFD reports auth failed for those users trying to get pop3 mail at that time. It is within a few seconds of the same time every day. It only happens to the people checking mail during that minute or two at 4:44am.

Here is the error report:
Subject: lfd: blocked x.x.x.x
Time: Tue Oct 1 04:44:50 2013 -0400
IP: x.x.x.x
Failures: 5 (pop3d)
Interval: 300 seconds
Blocked: Yes

Log entries:

Oct 1 04:44:34 sabrina dovecot: pop3-login: Disconnected: Inactivity (auth failed, 1 attempts): user=<[email protected]>, method=PLAIN, rip=client.ip.add.ress, lip=host.ip.add.ress
Oct 1 04:44:34 sabrina dovecot: pop3-login: Disconnected: Inactivity (auth failed, 1 attempts): user=<[email protected]>, method=PLAIN, rip=client.ip.address, lip=host.ip.add.ress
Oct 1 04:44:34 sabrina dovecot: pop3-login: Disconnected: Inactivity (auth failed, 1 attempts): user=<[email protected]>, method=PLAIN, rip=client.ip.address, lip=host.ip.add.ress
Oct 1 04:44:34 sabrina dovecot: pop3-login: Disconnected: Inactivity (auth failed, 1 attempts): user=<[email protected]>, method=PLAIN, rip=client.ip.address, lip=host.ip.add.ress
Oct 1 04:44:34 sabrina dovecot: pop3-login: Disconnected: Inactivity (auth failed, 1 attempts): user=<[email protected]>, method=PLAIN, rip=client.ip.address, lip=host.ip.add.ress

This is what I find in cron near that time:
0 4 * * * /usr/sbin/cxs --upgrade --quiet
0 4 * * * /usr/local/cpanel/scripts/cpbackup
37 4 * * * /usr/local/cpanel/scripts/upcp --cron


is upcp causing LFD to give erroneous reports? These same accounts check their mail 24 hours a day with no issues, but at this time every day, they are tagged as trying to log in with a bad password. so every morning I have to remove them from iptables and then they are fine until 4:44am the next day.

I also see this just before the above error, everyday.
Time: Wed Oct 9 04:42:01 2013 -0400
Error: Failed to detect code [dVt8TcB7JrCPoQTRBZihM6qjtxoXu] in SYSLOG_LOG [/var/log/messages]

SYSLOG may not be running correctly on sabrina.syo.com


Thanks so much for your insight.
Jason

 

[cPanelMichael]

Hello

It's important to note that LFD is a third-party application. You may want to report this issue directly on their support forums so they are aware of this potential issue with their product. As for the POP3 errors, are they all occurring under the same IP address? If so, you may want to try increasing the following setting under "WHM Home » Service Configuration » Mailserver Configuration":

"Maximum POP3 Connections per IP"

Thank you.

 

[jasgot]

Michael,
Thanks for your interest. I think you miss-construed the problem. LFD only reports the failed log in attempts, it does not cause them. LFD only reads the cpanel error logs. Also, ConfigServer has already been approached with this problem and they have demonstrated that it is not an LFd issue.

The problem is not the number of logins, it is an authentication failure.
Changing the number of pop3 logins per IP will not solve the authentication problem.

The problem is that if a user tries to check mail during the upcp update, the pop3 auth fails even with the correct account credentials.

Hello

It's important to note that LFD is a third-party application. You may want to report this issue directly on their support forums so they are aware of this potential issue with their product. As for the POP3 errors, are they all occurring under the same IP address? If so, you may want to try increasing the following setting under "WHM Home » Service Configuration » Mailserver Configuration":

"Maximum POP3 Connections per IP"

Thank you.

 

[cPanelMichael]

The problem is that if a user tries to check mail during the upcp update, the pop3 auth fails even with the correct account credentials.

I recommend opening a support ticket so we can attempt to reproduce this issue on your system. You can open a support ticket via:

Submit A Ticket

You can post the ticket number here so we can update this thread with the outcome.

Thank you.

 

[jasgot]

Your Request id is: 4379639.

 

[cPanelMichael]

Your Request id is: 4379639.

Per the ticket, it looks like this may have been an isolated issue. We were unable to reproduce the issue at the referenced time.

 

[sneader]

I apologize in advance for jumping on a thread that is over a year old, but I am having the EXACT same problem on multiple servers. I have a ticket opened on this, Ticket #7094033.

@jasgot, did you ever resolve this?

- Scott

 

[sneader]

I received a very enlightening reply to my ticket and I will share it here, in case it helps someone else with this same problem (auth failed problems during upcp)

I have a strong suspicion that prelink is the cause of this because I have seen it cause this exact issue before, and it is a utility that we do not recommend be installed on servers because it can cause a lot of weird and unexpected problems. By default it isn't part of a CentOS 6 minimal installation so I'm not sure how it gets installed on many servers.

Basically, what prelink does is scan various system binaries and if any of the libraries they are dynamically linked with have been updated, it will perform a read/move/write of that binary file which in turn causes a security check in a Dovecot authentication wrapper to fail, which prevents authentication from working again until Dovecot is restarted.

When I looked at /var/log/maillog around the time of your example, I noticed that auth began working again immediately after Dovecot was restarted, so this reinforces my suspicion that prelink was the cause.

I also found that according to /var/log/maillog the auth failures started here:

Sep 2 00:05:49 somehost dovecot: pop3-login: Disconnected (auth failed, 1 attempts in 2 secs): user=<[email protected]>, method=PLAIN, rip=1.1.1.1, lip=2.2.2.2, session=<iAennrwe4ABKiqBz>

Just a few seconds prior to that, several packages had been updated by yum (which was run by upcp), one of which was this package:

Sep 2 00:05:38 somehost yum[434322]: Updated: nss-softokn-freebl-3.14.3-23.el6_7.i686

The dovecot-auth wrapper happens to be linked against this library:

[[email protected] ~]# ldd /usr/local/cpanel/bin/dovecot-auth | grep libfreebl
libfreebl3.so => /lib64/libfreebl3.so (0x0000003243800000)
[[email protected] ~]# rpm -qf /lib64/libfreebl3.so
nss-softokn-freebl-3.14.3-23.el6_7.x86_64

So, prelink most likely did a scan and determined that /usr/local/cpanel/bin/dovecot-auth needed to be re-prelinked and did so, which broke authentication until the service restart a few minutes later.

I highly recommend reversing what prelink has done and then completely uninstall it:

1) Set PRELINKING=no in /etc/sysconfig/prelink
2) Run "/etc/cron.daily/prelink"
3) Run "rpm -e --nodeps prelink"
4) /scripts/restartsrv_dovecot

I hope this helps resolve the situation for good, please let me know if you have any questions.

I did some reading on prelink and I do see how it might have some advantages... but if cPanel does not recommend that it be used, and it's causing customers problems... my gut says to remove it.

- Scott

 

[cPanelMichael]

I am happy to see you were able to receive an answer to your question. Thank you for taking the time to update this thread with the response.