Spam campaign being sent through dozens of email accounts/multiple servers

[MediaServe]

It's not unheard of for our clients to have email accounts hacked and spam to be sent--we try to stay on top of it and suspend sites when this happens until the client is made aware of what's going on and can resolve the issue. Some of the new mail restrictions (max defers etc.) are helping keep the outgoing spam volumes from creating problems until we discover them too.

However, today spam reports came flooding in through our feedback loop, and we discovered the same spam campaign (pushing Raspberry Ultra Drops) was being sent through dozens of compromised accounts on 13 of our servers (that we know of so far).

I have never seen such a coordinated compromise like this. Anyone else had this happen? We run a pretty tight security ship, so until I find indications otherwise I suspect this is simply a case of compromised email accounts, but to have it happen across so many servers and used to send a single spam campaign is disconcerting.

 

[raventec]

Re: Spam campaign being sent through dozens of email accounts/multiple serv

Same here. Exactly as you described.

 

[MediaServe]

Re: Spam campaign being sent through dozens of email accounts/multiple serv

Same here. Exactly as you described.

Interesting! So far it looks like a simple matter of cracked email passwords, nothing more invasive (but I'm still investigating). I'm just a little surprised at how many mailboxes on multiple servers were being used for the same spam campaign--makes me wonder if they cracked the passwords over time and then just let loose a big campaign using all of them, or what.

The IPs involved in sending the spam were all within 89.36.141.0/24 (actually just a handful of IPs in that range, but I blocked the entire /24 at our edge routers.) Are the same IPs involved on your end?

 

[raventec]

Re: Spam campaign being sent through dozens of email accounts/multiple serv

The IPs involved in sending the spam were all within 89.36.141.0/24 (actually just a handful of IPs in that range, but I blocked the entire /24 at our edge routers.) Are the same IPs involved on your end?

Nice find! That's the same IP's on one of two affected machines. After changing the passwords on the affected accounts and blocking 89.36.141.0/24 so far no more problems. The other machine I can't tell as I had to dump the blocks.
I saw some scanning the other day where they were only trying 3 or 4 passwords before another ip tried. Thinking their method slips under our brute force detection methods possibly?

 

[sneader]

Re: Spam campaign being sent through dozens of email accounts/multiple serv

I just wanted to add my two cents. We had the exact same problem, across all of our cPanel servers. Every single one of our servers had at least a couple email accounts (some as many as 10 email accounts), being used to relay spam through our servers, all coming from the network of 89.36.141.0/24.

In reviewing our logs, they did not log in via POP or IMAP, only SMTP. The messages were trickled out, so as to not trip our outgoing mail rate 'red flags'. It actually started Nov 13th, but came to a head yesterday ( Nov 15th)

Unlike raventec, I do not see ANY attempted password brute force attempts. To restate, not a single password failure. These guys knew the email address and email password, and logged in to send mail, effortlessly.

To resolve, I changed the email passwords for all affected customers, then blocked 89.36.141.0/24 in Exim (Access Lists > Blacklisted SMTP IP Addresses)

I have worked with a number of these customers, trying to find something common among them, but have not been successful.

As much as I absolutely HATE AOL... I have to tip my hat at AOL for their "SCOMP Reports" -- had I not started to receive those spam reports via their Feedback Loop, I would not have caught the problem so quickly.

- Scott

 

[raventec]

Re: Spam campaign being sent through dozens of email accounts/multiple serv

One server is suffering multiple types of email attacks including this one making it really hard to see exactly what is going on on that one. The first server I ran into this on however was exactly as MediaServe described.
Sneader - Access Lists > Blacklisted SMTP IP Addresses Ha! great idea!

 

[sneader]

Re: Spam campaign being sent through dozens of email accounts/multiple serv

Hey guys, take a look at your logs for 89.44.0.2 -- do you see what I see? A really slow, once per hour brute force password cracker going, I'd say. I've blocked that IP on all of my boxes.

I bring it up here, since both IPs trace back to Romania (this IP, and the IP block that hit us yesterday)

- Scott

 

[mtindor]

Re: Spam campaign being sent through dozens of email accounts/multiple serv

The traffic from 89.44.0.2 (SMTP Auth) and 89.44.0.12 (POP3 brute forcing) as well as the 89.36.141.0/24 traffic do indeed all appear to be related. Accounts affected so far did indeed have fairly weak or blatantly weak passwords].

Incidentally, the traffic is not specifically directed to cPanel [or *nix even] servers. I've seen the same traffic and account hijacking on Smartermail / Merak / MDaemon as well. So I wouldn't suspect this is anything cPanel-specific.

The gold ole 89.36.141.0/24 IP addresses seem to be somewhat rate-limiting. I haven't seen 10s of thousands of spam generated from the hijacked accounts [unless I was just lucky to catch it quick]. It was usually one or two very brief spam runs on each hijacked account.

M

 

[MediaServe]

Re: Spam campaign being sent through dozens of email accounts/multiple serv

Had a fresh wave of this pushing printer ink start again yesterday, using (mostly) different cracked email accounts. The IP block used this time was 89.35.57.0/24, so in my case the source IPs to block were:

89.35.57.0/24
89.36.141.0/24

Hopefully others can get these ranges blocked before they fall victim also.

 

[Infopro]

Re: Spam campaign being sent through dozens of email accounts/multiple serv

Not sure if this comment is helpful or not.

Oldest example I can find if I search my logs (from LFD alert emails) specifically for this:
"dovecot_login authenticator failed for"

Time:     Tue Mar  8 09:30:19 2011 -0500
IP:       202.109.143.50 (CN/China/-)
Failures: 5 (smtpauth)
Interval: 300 seconds
Blocked:  Permanent Block

Log entries:

2011-03-08 09:30:10 dovecot_login authenticator failed for (JJIDC-K) [202.109.143.50]: 535 Incorrect authentication data (set_id=webmaster)
2011-03-08 09:30:10 dovecot_login authenticator failed for (JJIDC-K) [202.109.143.50]: 535 Incorrect authentication data (set_id=webmaster)
2011-03-08 09:30:13 dovecot_login authenticator failed for (JJIDC-K) [202.109.143.50]: 535 Incorrect authentication data (set_id=webmaster)
2011-03-08 09:30:13 dovecot_login authenticator failed for (JJIDC-K) [202.109.143.50]: 535 Incorrect authentication data (set_id=webmaster)
2011-03-08 09:30:15 dovecot_login authenticator failed for (JJIDC-K) [202.109.143.50]: 535 Incorrect authentication data (set_id=webmaster)

Newest example:

Time:     Sun Nov 18 03:51:52 2012 -0500
IP:       123.85.197.135 (CN/China/-)
Failures: 3 (smtpauth)
Interval: 300 seconds
Blocked:  Permanent Block

Log entries:

2012-11-18 03:51:36 dovecot_login authenticator failed for (ylmf-pc) [123.85.197.135]:1271: 535 Incorrect authentication data (set_id=customerpartialdomainname)
2012-11-18 03:51:40 dovecot_login authenticator failed for (ylmf-pc) [123.85.197.135]:1303: 535 Incorrect authentication data (set_id=customerpartialdomainname)
2012-11-18 03:51:43 dovecot_login authenticator failed for (ylmf-pc) [123.85.197.135]:1344: 535 Incorrect authentication data (set_id=customerpartialdomainname)

What do they have in common? They both got blocked permanently, by LFD, automatically. I updated my settings a while back when I started getting these emails more often (some days as many as 20 emails), and saw this thread:
have slipped through the cPHulk Brute Force Protection? - cPanel Forums

To add here, my DENY_IP_LIMIT is set to 400 on all servers.

Is this related to your accounts getting poked and compromised? I'm not sure about that. Are you auto blocking the attempts? If not, maybe doing so would be helpful to slow them down a bit.

 

[mtindor]

Re: Spam campaign being sent through dozens of email accounts/multiple serv

Had a fresh wave of this pushing printer ink start again yesterday, using (mostly) different cracked email accounts. The IP block used this time was 89.35.57.0/24, so in my case the source IPs to block were:

89.35.57.0/24
89.36.141.0/24

Hopefully others can get these ranges blocked before they fall victim also.

Thank you. That is helpful. I'm also seeing traffic from the same new block today, and I see that [in my case] they are actually trying to make use of an account that they previously hijacked [for which the password has already been changed and they are failing auth].

Time to block em.

Mike

 

[sneader]

Re: Spam campaign being sent through dozens of email accounts/multiple serv

Hi InfoPro. Note here:

Failures: 3 (smtpauth)
Interval: 300 seconds
Blocked: Permanent Block

The problem is a bunch of these guys are not-so-brute forcing passwords... I am seeing the same IP test passwords on an account in as little as once per hour. This doesn't come even close to hitting the threshold you have set (3 failures in 5 minutes)

(other stuff deleted here -- more to come)

- Scott

 

[nettigritty]

Re: Spam campaign being sent through dozens of email accounts/multiple serv

Seeing the exact same thing on Linux as well as Windows servers. New IP range that just started attempting access

188.214.30.0/24

 

[dalem]

Re: Spam campaign being sent through dozens of email accounts/multiple serv

ditto here caught all of the ranges sending low volume spam from all servers from a few hacked accounts
I have to say ingenious brute force method

 

[nospa]

Re: Spam campaign being sent through dozens of email accounts/multiple serv

If you will cut IP range, you will never know which email accounts are hacked. Currently they are sending low volume of spam, but if there will be some crazy spammer - he will try to send as many as possible emails, and this could hurt. My opinion is to allow them to send spam, and monitor exim logs for such subnets, and then if this subnet will send email with any account - change this account password.

 

[mtindor]

Re: Spam campaign being sent through dozens of email accounts/multiple serv

If you will cut IP range, you will never know which email accounts are hacked. Currently they are sending low volume of spam, but if there will be some crazy spammer - he will try to send as many as possible emails, and this could hurt. My opinion is to allow them to send spam, and monitor exim logs for such subnets, and then if this subnet will send email with any account - change this account password.

I agree. I removed the firewall blocks yesterday. As long as it's low volume, it's better to find out what is hijacked.

Another IP block: 93.114.140.0/24

I would also suggest keeping an eye on any of the address space listed in this query:

whois "RTA-ERENDI-CONSULT-SRL"@whois.ripe.net

or

whois "RTA-ERENDI-CONSULT-SRL"@whois.ripe.net|grep inetnum

The spam relaying traffic has thus far been reported from four blocks within the list.

m

 

[nettigritty]

Re: Spam campaign being sent through dozens of email accounts/multiple serv

This is what I have now

89.36.141.0/24 #Romanian Spammers
89.35.57.0/24 #Romanian Spammers
188.214.30.0/24 #Romanian Spammers
93.114.140.0/24 #Romanian Spammers
89.44.0.0/24 #Romanian Spammers

 

[MediaServe]

Re: Spam campaign being sent through dozens of email accounts/multiple serv

89.34.236.0/22 can be added to this list--caught them doing it again today, this time without any auth info to trace it back to a client unfortunately.

The netname for this network is INTERPRETIVE-NETWORKING-SRL, and I've blocked all of their netblocks (we don't have any Romanian clients). Here's the info in case anyone wants it:

# netname INTERPRETIVE-NETWORKING-SRL
85.204.120.0/24;
86.104.245.0/24;
86.105.195.0/24;
86.106.11.0/24;
86.106.12.0/24;
86.106.169.0/24;
89.34.16.0/22;
89.34.236.0/22;
89.35.32.0/24;
89.36.34.0/24;
89.43.182.0/23;
93.114.133.0/24;
93.115.253.0/24;

And another

# netname SC-XILES-SRL
31.14.32.0/19;

 

[MediaServe]

Re: Spam campaign being sent through dozens of email accounts/multiple serv

Well, this problem is larger than I thought. It seems these guys must have compromised a number of different Romanian networks and are using them to carry out this spam campaign. I started looking for patterns across multiple servers again (had only received spam reports from one server so far) and they are using multiple servers yet again--and this time it's difficult to figure out which accounts they compromised, as the entries all seem to have "check_mail_permissions could not determine the sender domain" in the entries.

I've manually blocked half a dozen large Romanian networks, and finally decided I'm tired of it, so for now I'm locking out all Romanian IPs -- not sure what else to do until I can see if there is anything I can use to trace this to compromised email accounts.

If anyone wants the list of Romanian IP blocks, reply and I'll see if I can post a file in CIDR notation.

 

[keithl]

Re: Spam campaign being sent through dozens of email accounts/multiple serv

We saw the same thing this last weekend with a single user account, though not just Romanian IP's. The three IP's I've spotted are 115.79.115.100 (vietnam), 109.228.118.145 (montenegro) and 109.197.82.92 (romania). The one upside I guess is with them sending almost 40k messages within 25 hours from a single email address makes it easy to find them in the logs, especially when done over the weekend!