The Community Forums

Interact with an entire community of cPanel & WHM users!
  1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Spam from localhost

Discussion in 'General Discussion' started by astopy, Sep 6, 2005.

  1. astopy

    astopy Well-Known Member

    Joined:
    Apr 3, 2003
    Messages:
    165
    Likes Received:
    0
    Trophy Points:
    16
    cPanel Access Level:
    Root Administrator
    In my mail queue I'm seeing a lot of bounce messages from spam which is originating from my server - I can only assume there were also messages which were delivered. From looking at the headers it looks like something running on the server is connecting to localhost:25 and relaying through it, though I see nothing to suggest which user it could be (according to my configuration it should not be possible to relay without authenticating). Example mail headers are below:

    replyxp.com is not hosted by us, and different emails use a different domain - e.g. replypad.com, turboreply.com, replyswiftly.com and others. one.valcatohosting.com is my server. Obviously I am anxious to fix this as soon as possible. For the moment I have put a stop to it by adding helo_verify_hosts=127.0.0.1 to exim.conf, though this has broken mail from legitimate sources (most importantly WHM Autopilot).

    My suspicion is that there is a process running on the server which is sending these emails, so I guess the easiest thing would be if I could log all connections to port 25 from localhost along with the pid/uid.

    Suggestions greatly appreciated.
     
  2. chirpy

    chirpy Well-Known Member

    Joined:
    Jun 15, 2002
    Messages:
    13,475
    Likes Received:
    20
    Trophy Points:
    38
    Location:
    Go on, have a guess
    You most likely have a compromised php script on the server which the spammers are exploiting. A few things:

    1. Make sure all phpBB and phpNuke installations are upgraded to the very latest release

    2. Enable extended exim logging to try and determine the culprit script. Add the following to the first textbox of the Advanced Mode Exim Configuration Editor in WHM:

    log_selector = +all

    3. Check your /tmp /var/tmp /dev/shm and various other likely places where exploits are uploaded to and try and trace their creation back using the domlogs. Then clean them up

    4. Install mod_security with a good set of SecFilter rules
     
  3. astopy

    astopy Well-Known Member

    Joined:
    Apr 3, 2003
    Messages:
    165
    Likes Received:
    0
    Trophy Points:
    16
    cPanel Access Level:
    Root Administrator
    Easier said than done - running 'locate viewtopic.php' turns up almost 200 phpBB installations. Checking that many installations, then getting those users to upgrade would probably take weeks (most users are lazy when it comes to this kind of thing).

    This didn't turn up anything of use, for example:
    Code:
    2005-09-07 16:24:13 rejected "EHLO coolreply.com" from localhost (coolreply.com) [127.0.0.1]:48988 I=[127.0.0.1]:25
    2005-09-07 16:24:13 SMTP connection from localhost (coolreply.com) [127.0.0.1]:48988 I=[127.0.0.1]:25 lost
    2005-09-07 18:48:16 rejected "EHLO coolreply.com" from localhost (coolreply.com) [127.0.0.1]:59111 I=[127.0.0.1]:25
    2005-09-07 18:48:16 SMTP connection from localhost (coolreply.com) [127.0.0.1]:59111 I=[127.0.0.1]:25 lost
    2005-09-07 18:48:17 rejected "EHLO coolreply.com" from localhost (coolreply.com) [127.0.0.1]:59113 I=[127.0.0.1]:25
    2005-09-07 18:48:17 SMTP connection from localhost (coolreply.com) [127.0.0.1]:59113 I=[127.0.0.1]:25 lost
    Though this does show that it is definitely something running on the server sending it.

    I do this every couple of days or so anyway; I haven't found anything which sends emails (attempts at DDoS attacks, and other such annoying scripts are common, though). While we're on the subject, this kind of thing is starting to get really difficult to control. It seems as though I have no way to prevent things being uploaded/run on the server - is there some kind of long-term solution to this security problem that I'm not aware of?

    This is something I'll have to look into.


    In the meantime, I'm still curious as to how this thing is able to send mail at all - how is it able to authenticate with exim?
     
  4. astopy

    astopy Well-Known Member

    Joined:
    Apr 3, 2003
    Messages:
    165
    Likes Received:
    0
    Trophy Points:
    16
    cPanel Access Level:
    Root Administrator
    I've been playing with netstat to see if I can track down the user sending the messages. I've got "netstat -cen | grep 127.0.0.1:25" running now, so hopefully I can find it soon :)
     
  5. webignition

    webignition Well-Known Member

    Joined:
    Jan 22, 2005
    Messages:
    1,880
    Likes Received:
    0
    Trophy Points:
    36
    As I understand it, most PHP-based exploits work more or less as follows:

    1. Malicious user adds a certain query string on to the end of a known-to-be-vulnerable URL of a known-to-be-vulnerable application (phpBB, for example)

    2. Said vulerable application (I shall use vApp from now on to shorten things), through the values in the query string, reads in data from an external file (most commonly a text file with shell commands) and, due to the vulnerability, executes the shell commands through one of the PHP functions that allows such things

    3. Said shell commands executed by vApp download a file to /tmp, be it an executable binary, perl script, compressed or uncompressed, uncompress the file (if needed), and either execute the relevant binary or perl script from /tmp or copy it elsewhere and execute it from there

    The malicous executable may then be an IRC server (a common way of telling other malicious users of the successful exploit), a sneaky mail wrapper or something of that nature.

    In short, you need to make sure you secure /tmp at the very least and install mod_security and add relevant rules to catch malicious URLs and so nip them in the bud.

    A good selection of rules can be found at http://modsecrules.monkeydev.org/index.php
     
  6. astopy

    astopy Well-Known Member

    Joined:
    Apr 3, 2003
    Messages:
    165
    Likes Received:
    0
    Trophy Points:
    16
    cPanel Access Level:
    Root Administrator
    Thanks for that, I'll definitely set up mod_security :)
     
  7. astopy

    astopy Well-Known Member

    Joined:
    Apr 3, 2003
    Messages:
    165
    Likes Received:
    0
    Trophy Points:
    16
    cPanel Access Level:
    Root Administrator
    If anyone's interested I believe I've tracked down the offending account. It was set up about a month ago, seemingly by a "marketing" company - the only thing in the account is an installation of YaBB which hasn't been configured, which I'm guessing they were using to send their messages. I've suspended the account, and so far there haven't been any more attempts at sending these spam messages. :)
     
  8. panayot

    panayot Well-Known Member

    Joined:
    Nov 18, 2004
    Messages:
    125
    Likes Received:
    0
    Trophy Points:
    16
    How did you find the offender? Through exim logs or some other method?

    Thanks in advance :)
     
  9. astopy

    astopy Well-Known Member

    Joined:
    Apr 3, 2003
    Messages:
    165
    Likes Received:
    0
    Trophy Points:
    16
    cPanel Access Level:
    Root Administrator
    I found the offending account by leaving the netstat command I mentioned earlier running over night - it output the UID of the account whenever it accessed localhost:25 :) When I looked at the output, there was only one account accessing it, and upon inspection of the contact info it was obviously a spammer.
     
  10. n3x1s

    n3x1s Registered

    Joined:
    Sep 11, 2005
    Messages:
    4
    Likes Received:
    0
    Trophy Points:
    1
    I had this same problem on Friday night. The server started spamming like crazy at around 8:00pm and I caught it at around 10PM. I switched to runlevel 1 until I could get to the NOC and look at things at the console. I was unable to find anything so I just killed exim so that the website could continue to run while I looked into the problem.

    After reading this post, I searched thru the /tmp directory and found a file named 'r0nin' which was listed as executable. I chmod'd it to 660. I am about to start exim again to see what happens.

    BTW: I looked thru all of the logs and found that a customer was uploading forum files for a phpBB forum at about the same time all of the spamming started. The spamming started about 2 hours after the customer began uploading files. I'll look thru these to find out what version of phpBB he has running.

    Other than this, what else can I do to avoid this in the future? I'll be looking into mod_security shortly.
     
  11. astopy

    astopy Well-Known Member

    Joined:
    Apr 3, 2003
    Messages:
    165
    Likes Received:
    0
    Trophy Points:
    16
    cPanel Access Level:
    Root Administrator
    I've seen that file before, phpBB was the problem then. You might want to take a look at any processes running as the web server's user - these things seem to be able to list themselves as 'httpd' in top, even though it's often perl running the script. Though since you switched to single user mode this shouldn't be a problem.

    Don't bother looking through the files, phpBB stores it's version number in the DB somewhere (I don't remember exactly where).

    Search for one of the threads about securing your server, I've found them to be quite useful :) It's probably worth mounting /tmp with the noexec flag, though I've found that doesn't stop scripts being run from there (the user can just call perl directly, with the path to the script as an argument). It also seems to be possible to run binaries from a noexec partition under Linux, though the trick I read about for doing that doesn't seem to work on other operating systems (FreeBSD specifically).
     
  12. n3x1s

    n3x1s Registered

    Joined:
    Sep 11, 2005
    Messages:
    4
    Likes Received:
    0
    Trophy Points:
    1
    Thanks for you help. I'll most certainly look for those related posts. I'm VERY new to Linux so please forgive me.

    Right now the server is in runlevel 3 with only exim disabled. I'm guessing Apache runs as user 'nobody'. Below is a list of currently running process by that user:

    851 root Sep10 /usr/local/apache/bin/httpd -DSSL
    877 nobody Sep10 /usr/local/apache/bin/httpd -DSSL
    878 nobody Sep10 /usr/local/apache/bin/httpd -DSSL
    879 nobody Sep10 /usr/local/apache/bin/httpd -DSSL
    880 nobody Sep10 /usr/local/apache/bin/httpd -DSSL
    881 nobody Sep10 /usr/local/apache/bin/httpd -DSSL
    1000 nobody Sep10 /usr/local/apache/bin/httpd -DSSL
    1023 nobody Sep10 /usr/local/apache/bin/httpd -DSSL
    1024 nobody Sep10 /usr/local/apache/bin/httpd -DSSL
    2412 nobody Sep10 /usr/local/apache/bin/httpd -DSSL
    4584 nobody Sep10 /usr/local/apache/bin/httpd -DSSL
    23178 nobody 14:58 /usr/local/apache/bin/httpd -DSSL
    23194 nobody 14:59 /usr/local/apache/bin/httpd -DSSL
    23203 nobody 14:59 /usr/local/apache/bin/httpd -DSSL
    23204 nobody 14:59 /usr/local/apache/bin/httpd -DSSL
    23205 nobody 14:59 /usr/local/apache/bin/httpd -DSSL
    23206 nobody 14:59 /usr/local/apache/bin/httpd -DSSL
    23207 nobody 14:59 /usr/local/apache/bin/httpd -DSSL
    23208 nobody 14:59 /usr/local/apache/bin/httpd -DSSL
    23209 nobody 14:59 /usr/local/apache/bin/httpd -DSSL
    23210 nobody 14:59 /usr/local/apache/bin/httpd -DSSL
    23211 nobody 14:59 /usr/local/apache/bin/httpd -DSSL
    23212 nobody 14:59 /usr/local/apache/bin/httpd -DSSL
    23213 nobody 14:59 /usr/local/apache/bin/httpd -DSSL

    Do these seem valid?
     
  13. astopy

    astopy Well-Known Member

    Joined:
    Apr 3, 2003
    Messages:
    165
    Likes Received:
    0
    Trophy Points:
    16
    cPanel Access Level:
    Root Administrator
    It's difficult to tell - I use phpSuExec on my server, so when these things get created and run it's as a user other than 'nobody' so it's obvious that they're fake. Stop apache then look to see if there are any processes still running.
     
  14. n3x1s

    n3x1s Registered

    Joined:
    Sep 11, 2005
    Messages:
    4
    Likes Received:
    0
    Trophy Points:
    1
    I stopped Apache and all of the previous instances of /usr/local/apache/bin/httpd -DSSL are dead, including all from 'nobody' and 'root.

    EDIT:

    Can you give me a little insite into phpsuexec? I understand the use/need/purpose of it. I'm just trying to figure out how to go about implementing it.
     
    #14 n3x1s, Sep 11, 2005
    Last edited: Sep 11, 2005
  15. astopy

    astopy Well-Known Member

    Joined:
    Apr 3, 2003
    Messages:
    165
    Likes Received:
    0
    Trophy Points:
    16
    cPanel Access Level:
    Root Administrator
    Enable it by recompiling apache with /scripts/easyapache in SSH, then make sure suExec is enabled in WHM.
     
  16. MDurai

    MDurai Member

    Joined:
    May 15, 2005
    Messages:
    9
    Likes Received:
    0
    Trophy Points:
    1
    Hey Odhinn,
    With that command you used did you get alot of these?:
    warning, got duplicate tcp line.
     
  17. astopy

    astopy Well-Known Member

    Joined:
    Apr 3, 2003
    Messages:
    165
    Likes Received:
    0
    Trophy Points:
    16
    cPanel Access Level:
    Root Administrator
    Actually, yes I did now that you mention it. In the end I modified the command to this to filter them out:
    Code:
    netstat -cen 2>/dev/null | grep 127.0.0.1:25
    This redirects stderr to /dev/null, so it won't flood your terminal.
     
  18. MDurai

    MDurai Member

    Joined:
    May 15, 2005
    Messages:
    9
    Likes Received:
    0
    Trophy Points:
    1
    Hey, sorry to bother ya again but I'm gettin this:

    tcp 0 0 127.0.0.1:25 127.0.0.1:36414 ESTA
    BLISHED 47 79780818
    tcp 0 0 127.0.0.1:36414 127.0.0.1:25 ESTA
    BLISHED 32047 79780815

    How can I translate this into human so to speak? Still learning each day :)
     
  19. astopy

    astopy Well-Known Member

    Joined:
    Apr 3, 2003
    Messages:
    165
    Likes Received:
    0
    Trophy Points:
    16
    cPanel Access Level:
    Root Administrator
    32047 is the user-id of the account in question. Run "grep 32047 /etc/passwd" to find out the corresponding user name.
     
  20. MDurai

    MDurai Member

    Joined:
    May 15, 2005
    Messages:
    9
    Likes Received:
    0
    Trophy Points:
    1
    Freakin sweet. Dude you are great. Do you have any contact details for future info? :D
     
Loading...

Share This Page