How to identify which device is failing?

[keat63]

I've a user problem which is driving me up the wall.

(pop3d) Failed POP3 login from xx.xx.xx.xx, this eventually results in a CSF block, locking her out entirely.

The user has 3 devices, Laptop, Tablet, Iphone.
I can't resolve this at her premise, as the resulting lock out, will also lock me out of WHM, meaning I can't clear her IP from CSF.

So yesterday I had all three devices in the office, reconfigured them all, and sent test emails to, and then from each device, then handed them back.
I whitelisted her IP in CSF.

Came in to the office this morning and found a ton of these over night in the logs again.

(pop3d) Failed POP3 login from xx.xx.xx.xx.

At the moment her IP is whitelisted, but she uses dynamic IP so will eventually get locked out.

Is there any way in the logs at my end, that I can determine what device is causing this, and why did all three work in the office, but not at her premise. ?


Feb 14 15:18:22 xxxxx dovecot: pop3-login: Disconnected: Inactivity (auth failed, 1 attempts in 180 secs): user=<[email protected]>, method=PLAIN, rip=xxx.xxx.xxx.xxx, lip=yyy.yyy.yyy.yyy, session=<iR6UINyBDbGPn2TE>

 

[sparek-3]

Unfortunately, traffic like this is essentially NAT'd at the customer's end. Devices connect to the user's router or modem and then that router or modem disperses the connection out to the rest of the Internet.

As far as the rest of the Internet is concerned - that user's connection is one connection. They may have 50 devices hidden behind their internal LAN connected to that router. But as far as the Internet is concerned, that is all one connection. That's all it can see.

MAC address tracking is only done point-to-point. Meaning that individual MAC addresses are only sent from one connection to the next connection in the process (the user's device to the user's router) so there is no way for you to track this based on MAC addresses or any other individual piece of device identification.

Having said all of that... generally, if a device is coming up with an error message saying they are using the wrong username and password... that's usually the device that is generating this issue.

 

[keat63]

I've asked her to turn two devices off leaving only one switched on over night.
I'll check the logs in the morning.

I cant really monitor this during the day as theres no exact pattern to when it happens, but at least over night I'll have a bunch of errors logged.

I guess three evenings will reveal what's causing it.

I'm confused why each device could send and receive when in our white listed office though.
Surely I wouldn't be able to send or receive with an authority fail.
That's what's driving me mad.

 

[cPanelMichael]

Hello @keat63,

Were you able to narrow down the device leading to the lockouts?

Thank you.

 

[keat63]

By a process of elimination, ensuring only one device connected at a time, I've narrowed this down to an Android tablet.

What is strange is that I sent emails to and from each device to prove they could authenticate before I handed them back, so I've no idea why it's failing to authenticate at her home address.

There is something I don't like though, the tablet had no native mail app, only GMail.
Her pop3 account is being handled as an additional account on the GMail app.
Maybe i'ts related to this somehow.

I need to do a little more digging when I can get my hands on it.

 

[sparek-3]

If the account is being checked by Gmail, then the user's IP address won't be involved in the check - so the server won't be blocking the user's IP address because of failed login attempts from Gmail.

So, either that's not the right device or something else is going on.

 

[keat63]

Not being checked by gmail (the service), but the email client software, the Gmail app.
It's definately her IP that's getting blocked, and definately something to do with the tablet.

Just confused as to what and why it worked OK in the office.

I even mentioned in my test emails, words along the lines "test outbound from tablet" and replied with similar wording 'reply inbound to tablet' etc etc.

 

[sparek-3]

I wasn't aware that the Gmail client app could check other email accounts directly.

... TIL

 

[keat63]

A further update on this and it's getting stranger.
It seems that my issue may not be related to a specific device afterall, and is something to do with CPHULK mis reporting IP location.

Login via wifi - failed to authenticate
Mar 15 12:10:11 leeds dovecot: pop3-login: Aborted login (auth failed, 1 attempts in 2 secs): user=<[email protected]>, method=PLAIN, rip=zzz.zzz.100.196, lip=xxx.xxx.221.31, session=<OeuN6yCErIqPn2TE>

Same user, same device via mobil 4G
Mar 15 12:10:30 leeds dovecot: pop3-login: Login: user=<[email protected]>, method=PLAIN, rip=yyy.yyy.242.132, lip=xxx.xxx.221.31, mpid=24139, session=<WjHU7CCENEjVzfKE>
Mar 15 12:10:33 leeds dovecot: pop3([email protected]): Disconnected: Logged out top=2/6212, retr=2/173565, del=0/2182, size=199631385, bytes=113/179966

If I have all counties blacklisted in CPHULK, (except the UK), the user fails to authenticate, and I'm guessing which eventually triggers CSF.

Whats strange is the IP is definately a UK IP.
Further to this, CSF is configured to only allow UK Ip's to login via UK IP's so why does CSF allow the IP, but CPHULK doesn't. ??


When I see a failed login, logs will shows me the town which triggered the error.
Quite clearly, her location.

 

[rpvw]

Since that device username/password combination clearly works over 4G, check your CPHulk for the IP ending 100.196 and see if it is in Blocked IP Addresses or One-Day Blocks - some other device that is connected to the same network may have got that IP blocked.

If you have it installed, you might also want to look in CSF/LFD perm and temp blocks to see if the IP has been stopped there, and indeed, look in any other login or mail related software that can ban or block IPs

 

[keat63]

I disabled CSF fully for a few minutes, and the device was still failing, so I ruled out CSF.
It was only when I set all country zones to 'not specified' in CPHULK, that the emails started working again.
And it seems for both users.

Mar 15 12:57:32 leeds lfd[31551]: (pop3d) Failed POP3 login from xxx.xxx.xxx.xxx (GB/United Kingdom/Wakefield/Pontefract/xxx.xxx.xxx.xxx.dyn.plus.net): 1 in the last 3600 secs - *Blocked in csf* [LF_POP3D]

Whilst the UK is whitelisted in CPHULK and CSF.
I'm guessing, CPHULK seems to not like this IP, fails to to allow authentication which subsequently triggers a block in CSF.

 

[rpvw]

This demonstrates the potential problems with IP country lists. We may know that the user is based in the UK, and whitelist that country, but it is entirely possible that the IP country list thinks the IP is somewhere else.

I regularly see different countries reported between blocks made in CPHulk and the same IP blocked by CSF - and neither are guaranteed to be correct

 

[keat63]

Authentication failed is a bit of a red herring, and then considering this could have a knock on effect with the next process makes it even more confusing.

I'm guessing CPHULK and CSF use different IP country lists, as I have CSF limited to just a few countries and the emails are getting through this hurdle.
But not the CPHULK one.

 

[cPanelMichael]

Hello @keat63,

cPHulk uses the Geo::IPfree Perl module to determine the originating country associated with the IP address. The database utilized by this Perl module comes from IP to Country Database (IPV4 and IPV6) and we update it with each new major cPanel & WHM version. Can you visit that link and enter the IP address in-question to see if it returns the correct country code?

Thank you.

 

[keat63]

I could see in CPHULK that CPHULK seems to think it's a US IP.
Following the above link also reports it as being in the US.

BUT CSF is seeing the correct location.
(GB/United Kingdom/Wakefield/Pontefract/xxx.xxx.xxx.xxx.dyn.plus.net)

The user is within 9 miles of these two towns, and uses PlusNet

PlusNet possibly utilises B Class:
143.159.0.0 - 143.159.255.255

I've checked various random IP's within this range, all of which were showing PlusNet and all of which showing in the US.

 

[cPanelMichael]

Following the above link also reports it as being in the US.

Here's a quote from GEO IP Database FAQ that explains how this can happen:

Q. I looked up my IP and it said it was from one country but I am somewhere else. What gives?

We use the registry assignments provided by the registrars. However discrepancies creep in especially in cases of large multinational companies who have their base of operation on one country and satellite offices in other countries. Typically what happens is that a company based in say, the United States, also has a branch in Africa or Asia.

In many cases, when the company is assigned a new block of IP addresses these will reflect the correct country (Africa or Asia for example). However, sometimes the location of the parent company is used - USA for example. This leads to an anomaly, where looking up the IP you know to be in one country appears to be somewhere else.

This is especially true of companies like AOL, Verizon, Sprint, Telefonica and others who do not operate only in one country. We have found IPs for both AOL and Verizon for example which are outside the US but show as originating in the US which is obviously not correct.

Although this only represents a tiny fraction of a percent, this is something that should be kept in mind.

With that in-mind, I recommend opening a feature request to seek the implementation of a more accurate IP address database. Can you open the request and post the link here's once it's opened?

Thank you.

 

[keat63]

https://features.cpanel.net/topic/implementation-of-a-more-accurate-ip-address-database