Mysterious Spam originating from our server?

[Deleted member 3231]

I have been recieving almost 2000 AOL notifications of spam emails each day since I signed up to their feedback loop service but I am at a loss as to where they are originating from.

The server is very well secured with apf,bdf,mod_sec etc and is running the latest release version of Cpanel. An example header is included below. There are a number of strange things about the header. First of all it seems to be missing all Anti-abuse headers and secondly it's got qmail style message id's which is obviously impossible for a Cpanel / Exim server. All the messages are basically the same content but the domain name in the message body seems to change every few days and the subject line also changes slightly.

I have run numerous scans of the server with rkhunter, chkrootkit etc and they came up with nothing. I can't see any strange processes running and mod_sec appears to successfully blocking numerous BCC form injection attacks which were causing most of our spam issues lately. I just cannot figure out where this is originating from so any ideas would be appreciated as it's getting annoying now.

-------- Original Message --------
X-AOL-UID: 193.1787634993
X-AOL-DATE: Thu, 23 Feb 2006 1:07:55 AM Eastern Standard Time
Return-Path: <[email protected]>
Received: from rly-xm04.mx.aol.com (rly-xm04.mail.aol.com [172.20.83.105]) by air-xm01.mail.aol.com (vx) with ESMTP id MAILINXM11-60643fd5136385; Thu, 23 Feb 2006 01:07:55 -0500
Received: from xeon.OURSERVERHOSTNAME (xeon.OURSERVERHOSTNAME [OURSERVERIP]) by rly-xm04.mx.aol.com (vx) with ESMTP id MAILRELAYINXM42-60643fd5136385; Thu, 23 Feb 2006 01:07:50 -0500
Received: (qmail 23207 invoked by uid 13339); Fri, 24 Feb 2006 05:28:58 +0200 (CEST)
Message-Id: <[email protected]>
From: "Lashonna Hanes" <[email protected]>
To: <Undisclosed Recipients>
Date: Fri, 24 Feb 2006 05:28:58 +0200 (CEST)
Subject: Bringingx another feelings to daughter
Mime-Version: 1.0
Content-Type: text/plain
X-AOL-IP: OUR SERVER IP

That day I invitedd my co-workers and together we used my daughter in allw her entersa inb http://wisenen.net/?kqlm at once

 

[mctDarren]

Check to make sure no one has their email automatically being forwarded to AOL. Low scoring spam that slips through can be a pain if the user then flags it as spam. I have the same problem, and AOL just went to my DC and complained - even after they told me it would not be a problem over the phone.

 

[Deleted member 3231]

I checked this and none of my users are forwarding to AOL addresses.

Check to make sure no one has their email automatically being forwarded to AOL. Low scoring spam that slips through can be a pain if the user then flags it as spam. I have the same problem, and AOL just went to my DC and complained - even after they told me it would not be a problem over the phone.

 

[sparek-3]

What about any netscape forwarders (I can't remember if its netscape.com or netscape.net). I vaguely recall seeing some of the AOL messages we received and they would mention a netscape address and sure enough, the user would have an e-mail address forwarding to a netscape address.

I'm really not sure, but it might give you something else to look for.

 

[Deleted member 3231]

Just checked this and there are no netscape forwarders either. It's very strange. Also the number of notifications increased today which would indicate more spam was being sent. I still cannot find anything in the exim logs. Very strange.

What about any netscape forwarders (I can't remember if its netscape.com or netscape.net). I vaguely recall seeing some of the AOL messages we received and they would mention a netscape address and sure enough, the user would have an e-mail address forwarding to a netscape address.

I'm really not sure, but it might give you something else to look for.

 

[mike25]

Check your mail queue for large queues of failed messages. If you cat and grep through /var/exim_mainlog for the queue sometimes the log will show the directory of orgin on the server. If you see the exim processes running live you can also then check the /proc directory for the PID of the exim process that you belive is sending spam. If you are able to discover which account the spam is coming from I would then go to /usr/local/apache/domlogs and the cat that domains access log and grep for POST. My guess would be that you will locate a problem PHP script. Even with BCC rules in place some spammers are able to get through incsecure PHP mailing scripts. I wish I knew how so I could beef up my mod_sec rules.

 

[AndyReed]

My guess would be that you will locate a problem PHP script. Even with BCC rules in place some spammers are able to get through incsecure PHP mailing scripts.

You're right, mike25. Now, finding that bad/insecure php/cgi script(s) is the best possible solution to stop SPAM originating from your server. One of our clients was hit hard with a spammer who used bad/insecure Php and cgi scripts to deliver thousands of messages a day. Our client tried every possible way to stop that SPAM, but without much success. We installed a script that helped pinpoint the bad/insecure scripts. The spammer was prevented from using our client's server any further.

 

[escv123]

Can you post your solution here ?

Could you post your solution here ?

You're right, mike25. Now, finding that bad/insecure php/cgi script(s) is the best possible solution to stop SPAM originating from your server. One of our clients was hit hard with a spammer who used bad/insecure Php and cgi scripts to deliver thousands of messages a day. Our client tried every possible way to stop that SPAM, but without much success. We installed a script that helped pinpoint the bad/insecure scripts. The spammer was prevented from using our client's server any further.

 

[iperhosting.com]

Could you post your solution here ?

Sorry for my english, the solutions exists?

Than'ks
Alex

 

[richy]

Under Tweak Settings enable:"Track the origin of messages sent though the mail server by adding the X-Source headers" and in the "Exim Configuration Editor" add "log_selector = +arguments +subject" in the first entry box in the "Advanced Settings".

This should provide more information for you to trace the spam. Turning on phpSuExec (and suExec) is a good idea as well.

 

[iperhosting.com]

Under Tweak Settings enable:"Track the origin of messages sent though the mail server by adding the X-Source headers" and in the "Exim Configuration Editor" add "log_selector = +arguments +subject" in the first entry box in the "Advanced Settings".

This should provide more information for you to trace the spam. Turning on phpSuExec (and suExec) is a good idea as well.

With phpSuExec script running? Example phpbbforum, my script ecc.. ecc...

Thank's

 

[Solokron]

Many of you are missing the key element of the spam. "qmail". You cannot tail log files or enable exim features to track down mail that is utilizing qmail. This is a script that is utilizing its own qmail to send email. We have seen this before and tracking down the script is necessary.

 

[chirpy]

You're quite correct. It's also not qmail, but a spoofed header line. The spam script simply connects out on port 25 and sends the spam directly, bypassing exim completely. The only way to track it down directly is to watch port 25 connections and traffic or enable the SMTP Tweak/firewall blocking to disallow sending via port 25 to force users and script so use the sendmail/exim binary as they really ought to.