BigLebowski

Well-Known Member
Dec 24, 2007
75
0
56
I have tends of thousands of these entries in my /var/log/exim_mainlog
smtp tweak is enabled.
Set SMTP Sender: headers is enabled
log_selector = +all -ident_timeout -pid is added to exim.conf

But still there is nothing to identify where this spam is originating. Any ideas please?

Best
Dude
 

acenetgeorge

Well-Known Member
PartnerNOC
Mar 6, 2008
68
4
58
Southfield, MI
cPanel Access Level
DataCenter Provider
This is not really the forum to look into tracking down spam, but given the loopback address in the log snippet you posted, I would run a 'ps -ax | less' and look for any system binaries that are running under a username. That may be a mass mailer script doing the spamming.

Try running this in SSH, and see if there are any results. These would be anything that is connecting via port 25 that should not be. We've had good luck tracking down the Dark Mailer script using this:

netstat -placen | grep 127.0.0.1:25 | grep ESTABLISHED | grep -v exim | grep -v tailwatchd

or more broadly:

netstat -plan | grep ':25'| grep ESTAB

Good Luck!
 

mtindor

Well-Known Member
Sep 14, 2004
1,363
65
178
inside a catfish
cPanel Access Level
Root Administrator
Also,

Look for entries in /var/log/exim_mainlog that have "cwd" in them, such as:

2011-10-11 14:45:18 cwd=/home/accountname/public_html 3 args: /usr/sbin/sendmail -t -i

You could do something like:

Code:
cat /var/log/exim_mainlog|grep -v '/var/spool/exim'|grep -v 'exim -bpc'
That should help you out. It isn't optimum, but off the top of my head that's the quickest thing I can tell you to do in order to look for what you are seeking.

M
 

Sash

Well-Known Member
Feb 18, 2003
252
0
166
The spammer is connecting directly to the SMTP port via sockets. Typical "ps aux" and scanning the mail log won't show how the spammer is sending the messages.
 

mtindor

Well-Known Member
Sep 14, 2004
1,363
65
178
inside a catfish
cPanel Access Level
Root Administrator
Yes, it will reflect in the mail log if they are sending to 127.0.0.1:25.

If you are logging +all in exim, you're going to see the 'cwd' entries I mentioned above to indicate where the script is being run from that is generating the connections to Exim.

M
 

Sash

Well-Known Member
Feb 18, 2003
252
0
166
This type of attack doesn't show the "cwd" entries. This type of attack is different than typical PHP script spam that uses the mail function.

The log entry in post 1, that's exactly what shows in the mail log for each message being sent.
 

mtindor

Well-Known Member
Sep 14, 2004
1,363
65
178
inside a catfish
cPanel Access Level
Root Administrator
Yes, it will reflect in the mail log if they are sending to 127.0.0.1:25.

If you are logging +all in exim, you're going to see the 'cwd' entries I mentioned above to indicate where the script is being run from that is generating the connections to Exim.

M
Or maybe not. but i would think if you have a script on the server that is connecting to exim to send mail, there is going to be a corresponding 'cwd' entry. Did you check?

Mike
 

mtindor

Well-Known Member
Sep 14, 2004
1,363
65
178
inside a catfish
cPanel Access Level
Root Administrator
Perhaps I'm not understanding correctly. Do you suspect it is some PHP script doing this?

If so, do you have Mailheaders compiled into your PHP? If you do, and it is using PHP to send, it should generate an X-Header in the outgoing mail such as:

X-PHP-Script: www.example.com/~user/testapp/send-mail.php for 10.0.0.1

Mike
 

Sash

Well-Known Member
Feb 18, 2003
252
0
166
They're connecting directly to the SMTP server. Mailheaders only work if they're using the PHP mail function.
 

dalem

Well-Known Member
PartnerNOC
Oct 24, 2003
2,977
152
368
SLC
cPanel Access Level
DataCenter Provider
run
netstat -cen | grep 127.0.0.1:25
& wait for their spam run

and if you don't want to wait
netstat -cen | grep 127.0.0.1:25 > /root/spammer.txt &
and you will see their uid connecting
 

Sash

Well-Known Member
Feb 18, 2003
252
0
166
Here's the output of netstat -cen | grep 127.0.0.1:25

tcp 0 0 127.0.0.1:25 127.0.0.1:42378 TIME_WAIT 0 0
tcp 0 0 127.0.0.1:25 127.0.0.1:42378 TIME_WAIT 0 0
tcp 0 0 127.0.0.1:25 127.0.0.1:42378 TIME_WAIT 0 0
tcp 0 0 127.0.0.1:25 127.0.0.1:42378 TIME_WAIT 0 0
tcp 0 0 127.0.0.1:25 127.0.0.1:42378 TIME_WAIT 0 0
tcp 0 0 127.0.0.1:25 127.0.0.1:42378 TIME_WAIT 0 0
tcp 0 0 127.0.0.1:25 127.0.0.1:42378 TIME_WAIT 0 0
tcp 0 0 127.0.0.1:25 127.0.0.1:42378 TIME_WAIT 0 0
tcp 0 0 127.0.0.1:25 127.0.0.1:42378 TIME_WAIT 0 0
tcp 0 0 127.0.0.1:25 127.0.0.1:42378 TIME_WAIT 0 0
tcp 0 0 127.0.0.1:25 127.0.0.1:42378 TIME_WAIT 0 0
tcp 0 0 127.0.0.1:25 127.0.0.1:42378 TIME_WAIT 0 0
tcp 0 0 127.0.0.1:25 127.0.0.1:42378 TIME_WAIT 0 0
tcp 0 0 127.0.0.1:25 127.0.0.1:42378 TIME_WAIT 0 0
tcp 0 0 127.0.0.1:25 127.0.0.1:42378 TIME_WAIT 0 0
tcp 0 0 127.0.0.1:25 127.0.0.1:42378 TIME_WAIT 0 0
tcp 0 0 127.0.0.1:25 127.0.0.1:42378 TIME_WAIT 0 0
tcp 0 0 127.0.0.1:25 127.0.0.1:42378 TIME_WAIT 0 0
tcp 0 0 127.0.0.1:25 127.0.0.1:42378 TIME_WAIT 0 0
Where's the UID?
 

mtindor

Well-Known Member
Sep 14, 2004
1,363
65
178
inside a catfish
cPanel Access Level
Root Administrator
tcp 0 0 127.0.0.1:25 127.0.0.1:45991 ESTABLISHED 47 48385048
tcp 0 0 127.0.0.1:45991 127.0.0.1:25 ESTABLISHED 32008 48385047
That is a matched pair for an active connection between a user account process and exim on localhost:25.

47 = UID of mailnull
32008 = UID of a specific user

You want to examine active connections, not ones from the past.

Mike
 

victomeyezr

Well-Known Member
Sep 25, 2008
54
0
56
I found the spam connection on mine, but when I suspend the user, it doesn't have any effect on the mail
 

mtindor

Well-Known Member
Sep 14, 2004
1,363
65
178
inside a catfish
cPanel Access Level
Root Administrator
You've got to find the script / process generating the spam. If you simply suspended the user but the process is still running, it'll remain running til you stop it.

If you're running suphp / suexec, and if it's simply coming from a breached user account, you should be able to find the running process by doing something like:

lsof -n|grep username

There are better flags to use with that, but I'm tired and not thinking straight.

Bottom line is if it's still spamming after the account is suspended, either that account was not the responsible account or a process ran as that user is continuing to run in the background generating spam. Or, there could be a cron entry for said script to be run even if the account is suspended. I'm not sure if suspending a user account will suspend user cron jobs or not.

Mike
 

acenetryan

Well-Known Member
PartnerNOC
Aug 21, 2005
197
1
168
We've encountered this issue as well. In our case, we've noticed that the user we believe is sending spam has an active SSH connection and is listed as having /usr/local/cpanel/bin/noshell as their shell. Since this doesn't actually prevent users from authenticating to SSH, they can still access the server over port 22. noshell will simply dump a "Shell access is not enabled on your account" message and kill the connection after a minute or two. If you have "AllowTcpForwarding" enabled in your SSH config, it's possible for the spammer to forward port 25 from their local machine to the remote server over port 22. If you encounter this type of spam, ensure "AllowTcpForwarding" is set to "no" in /etc/ssh/sshd_config. That should at least prevent this type of obfuscated spam from originating from your server. Of course, this manner of spamming requires the spammer have the username and password to authenticate. You should still take normal actions for dealing with compromised accounts (change passwords, update scripts, etc).
 

Sash

Well-Known Member
Feb 18, 2003
252
0
166
We've encountered this issue as well. In our case, we've noticed that the user we believe is sending spam has an active SSH connection and is listed as having /usr/local/cpanel/bin/noshell as their shell. Since this doesn't actually prevent users from authenticating to SSH, they can still access the server over port 22. noshell will simply dump a "Shell access is not enabled on your account" message and kill the connection after a minute or two. If you have "AllowTcpForwarding" enabled in your SSH config, it's possible for the spammer to forward port 25 from their local machine to the remote server over port 22. If you encounter this type of spam, ensure "AllowTcpForwarding" is set to "no" in /etc/ssh/sshd_config. That should at least prevent this type of obfuscated spam from originating from your server. Of course, this manner of spamming requires the spammer have the username and password to authenticate. You should still take normal actions for dealing with compromised accounts (change passwords, update scripts, etc).
Thanks, that fixed a problem we had.

Is there a way to see what user is logged in when the SPAM is being sent?