help tracking spammer

[BigLebowski]

I have tends of thousands of these entries in my /var/log/exim_mainlog
smtp tweak is enabled.
Set SMTP Sender: headers is enabled
log_selector = +all -ident_timeout -pid is added to exim.conf

But still there is nothing to identify where this spam is originating. Any ideas please?

2011-07-19 16:15:17 1QjC0z-0005QR-3p <= [email protected] H=localhost (User) [127.0.0.1]:41781 I=[127.0.0.1]:25 P=smtp S=1525 T="Access To Your Online Banking Has Been Suspended." from <[email protected]> for [email protected] [email protected] [email protected] [email protected] [email protected] [email protected] [email protected] [email protected] [email protected] [email protected] [email protected] [email protected] [email protected] [email protected] [email protected] [email protected] [email protected] [email protected] [email protected] [email protected] [email protected] [email protected] [email protected] [email protected] [email protected] [email protected] [email protected] [email protected] [email protected] [email protected] [email protected] [email protected] [email protected] [email protected] [email protected] [email protected] [email protected] [email protected] [email protected] [email protected] [email protected] [email protected] [email protected] [email protected] [email protected] [email protected] [email protected] [email protected] [email protected] [email protected]
2011-07-19 16:15:17 1QjC0z-0005jH-ED <= <> R=1QjBzz-0003vb-0n U=mailnull P=local S=8076 T="Mail delivery failed: returning message to sender" from <> for [email protected]

Best
Dude

 

[Sash]

Same problem. Did you find a solution?

 

[sararn]

me too. there has got to be a quick/easy fix

 

[acenetgeorge]

This is not really the forum to look into tracking down spam, but given the loopback address in the log snippet you posted, I would run a 'ps -ax | less' and look for any system binaries that are running under a username. That may be a mass mailer script doing the spamming.

Try running this in SSH, and see if there are any results. These would be anything that is connecting via port 25 that should not be. We've had good luck tracking down the Dark Mailer script using this:

netstat -placen | grep 127.0.0.1:25 | grep ESTABLISHED | grep -v exim | grep -v tailwatchd

or more broadly:

netstat -plan | grep ':25'| grep ESTAB

Good Luck!

 

[mtindor]

Also,

Look for entries in /var/log/exim_mainlog that have "cwd" in them, such as:

2011-10-11 14:45:18 cwd=/home/accountname/public_html 3 args: /usr/sbin/sendmail -t -i

You could do something like:

cat /var/log/exim_mainlog|grep -v '/var/spool/exim'|grep -v 'exim -bpc'

That should help you out. It isn't optimum, but off the top of my head that's the quickest thing I can tell you to do in order to look for what you are seeking.

M

 

[Sash]

The spammer is connecting directly to the SMTP port via sockets. Typical "ps aux" and scanning the mail log won't show how the spammer is sending the messages.

 

[mtindor]

Yes, it will reflect in the mail log if they are sending to 127.0.0.1:25.

If you are logging +all in exim, you're going to see the 'cwd' entries I mentioned above to indicate where the script is being run from that is generating the connections to Exim.

M

 

[Sash]

This type of attack doesn't show the "cwd" entries. This type of attack is different than typical PHP script spam that uses the mail function.

The log entry in post 1, that's exactly what shows in the mail log for each message being sent.

 

[mtindor]

Yes, it will reflect in the mail log if they are sending to 127.0.0.1:25.

If you are logging +all in exim, you're going to see the 'cwd' entries I mentioned above to indicate where the script is being run from that is generating the connections to Exim.

M

Or maybe not. but i would think if you have a script on the server that is connecting to exim to send mail, there is going to be a corresponding 'cwd' entry. Did you check?

Mike

 

[mtindor]

Ok got it. Thanks for educating me I understand now.

 

[mtindor]

Perhaps I'm not understanding correctly. Do you suspect it is some PHP script doing this?

If so, do you have Mailheaders compiled into your PHP? If you do, and it is using PHP to send, it should generate an X-Header in the outgoing mail such as:

X-PHP-Script: www.example.com/~user/testapp/send-mail.php for 10.0.0.1

Mike

 

[Sash]

They're connecting directly to the SMTP server. Mailheaders only work if they're using the PHP mail function.

 

[dalem]

run

netstat -cen | grep 127.0.0.1:25

& wait for their spam run

and if you don't want to wait

netstat -cen | grep 127.0.0.1:25 > /root/spammer.txt &

and you will see their uid connecting

 

[mtindor]

run

& wait for their spam run

and if you don't want to wait


and you will see their uid connecting

Nice job. Works great. That's a keeper. Thanks

 

[Sash]

Here's the output of netstat -cen | grep 127.0.0.1:25

tcp 0 0 127.0.0.1:25 127.0.0.1:42378 TIME_WAIT 0 0
tcp 0 0 127.0.0.1:25 127.0.0.1:42378 TIME_WAIT 0 0
tcp 0 0 127.0.0.1:25 127.0.0.1:42378 TIME_WAIT 0 0
tcp 0 0 127.0.0.1:25 127.0.0.1:42378 TIME_WAIT 0 0
tcp 0 0 127.0.0.1:25 127.0.0.1:42378 TIME_WAIT 0 0
tcp 0 0 127.0.0.1:25 127.0.0.1:42378 TIME_WAIT 0 0
tcp 0 0 127.0.0.1:25 127.0.0.1:42378 TIME_WAIT 0 0
tcp 0 0 127.0.0.1:25 127.0.0.1:42378 TIME_WAIT 0 0
tcp 0 0 127.0.0.1:25 127.0.0.1:42378 TIME_WAIT 0 0
tcp 0 0 127.0.0.1:25 127.0.0.1:42378 TIME_WAIT 0 0
tcp 0 0 127.0.0.1:25 127.0.0.1:42378 TIME_WAIT 0 0
tcp 0 0 127.0.0.1:25 127.0.0.1:42378 TIME_WAIT 0 0
tcp 0 0 127.0.0.1:25 127.0.0.1:42378 TIME_WAIT 0 0
tcp 0 0 127.0.0.1:25 127.0.0.1:42378 TIME_WAIT 0 0
tcp 0 0 127.0.0.1:25 127.0.0.1:42378 TIME_WAIT 0 0
tcp 0 0 127.0.0.1:25 127.0.0.1:42378 TIME_WAIT 0 0
tcp 0 0 127.0.0.1:25 127.0.0.1:42378 TIME_WAIT 0 0
tcp 0 0 127.0.0.1:25 127.0.0.1:42378 TIME_WAIT 0 0
tcp 0 0 127.0.0.1:25 127.0.0.1:42378 TIME_WAIT 0 0

Where's the UID?

 

[mtindor]

tcp 0 0 127.0.0.1:25 127.0.0.1:45991 ESTABLISHED 47 48385048
tcp 0 0 127.0.0.1:45991 127.0.0.1:25 ESTABLISHED 32008 48385047

That is a matched pair for an active connection between a user account process and exim on localhost:25.

47 = UID of mailnull
32008 = UID of a specific user

You want to examine active connections, not ones from the past.

Mike

 

[victomeyezr]

I found the spam connection on mine, but when I suspend the user, it doesn't have any effect on the mail

 

[mtindor]

You've got to find the script / process generating the spam. If you simply suspended the user but the process is still running, it'll remain running til you stop it.

If you're running suphp / suexec, and if it's simply coming from a breached user account, you should be able to find the running process by doing something like:

lsof -n|grep username

There are better flags to use with that, but I'm tired and not thinking straight.

Bottom line is if it's still spamming after the account is suspended, either that account was not the responsible account or a process ran as that user is continuing to run in the background generating spam. Or, there could be a cron entry for said script to be run even if the account is suspended. I'm not sure if suspending a user account will suspend user cron jobs or not.

Mike

 

[acenetryan]

We've encountered this issue as well. In our case, we've noticed that the user we believe is sending spam has an active SSH connection and is listed as having /usr/local/cpanel/bin/noshell as their shell. Since this doesn't actually prevent users from authenticating to SSH, they can still access the server over port 22. noshell will simply dump a "Shell access is not enabled on your account" message and kill the connection after a minute or two. If you have "AllowTcpForwarding" enabled in your SSH config, it's possible for the spammer to forward port 25 from their local machine to the remote server over port 22. If you encounter this type of spam, ensure "AllowTcpForwarding" is set to "no" in /etc/ssh/sshd_config. That should at least prevent this type of obfuscated spam from originating from your server. Of course, this manner of spamming requires the spammer have the username and password to authenticate. You should still take normal actions for dealing with compromised accounts (change passwords, update scripts, etc).

 

[Sash]

We've encountered this issue as well. In our case, we've noticed that the user we believe is sending spam has an active SSH connection and is listed as having /usr/local/cpanel/bin/noshell as their shell. Since this doesn't actually prevent users from authenticating to SSH, they can still access the server over port 22. noshell will simply dump a "Shell access is not enabled on your account" message and kill the connection after a minute or two. If you have "AllowTcpForwarding" enabled in your SSH config, it's possible for the spammer to forward port 25 from their local machine to the remote server over port 22. If you encounter this type of spam, ensure "AllowTcpForwarding" is set to "no" in /etc/ssh/sshd_config. That should at least prevent this type of obfuscated spam from originating from your server. Of course, this manner of spamming requires the spammer have the username and password to authenticate. You should still take normal actions for dealing with compromised accounts (change passwords, update scripts, etc).

Thanks, that fixed a problem we had.

Is there a way to see what user is logged in when the SPAM is being sent?