The Community Forums

Interact with an entire community of cPanel & WHM users!
  1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Last Ditch Effort... Potential Ownage?

Discussion in 'General Discussion' started by Mindlash, Nov 5, 2003.

  1. Mindlash

    Mindlash Well-Known Member

    Joined:
    Jul 7, 2003
    Messages:
    78
    Likes Received:
    0
    Trophy Points:
    6
    I've had a lengthy chat with gpan (cheetaweb) about this particular instance and will say I'm leaning towards a format/restore (on 3 servers)... but would like to extend the invitation to anyone that may have a clue as to what I am experiencing to lend a hand and possibly save me many many hours of backing up/restoring/downtime/client complaints/etc.

    What starts this voyage is the fact that I have some servers at another DC, that makes available in their control panel stats based on information gathered via SNMP (port 161)... such as running processes, logged in users... etc. I've been noticing that sometimes it works, sometimes it doesn't. It wasn't until today that I had a problem accessing this SNMP information on all three servers at the same time.

    Right about this time I got a message from the DC admin saying that my server was sending (or as they like to put it 'spewing') forged/spoofed packets outbound. Luckily their routers filter this, as this is a classic case of DoS attacks (my server(s) being a potential zombie attacker). Similarly a few days ago, the DC mentioned something similar about spewage, and outbound traffic reaching 10mbit.

    At any rate, back to SNMP... in doing a netstat I was able to find that snmpd was now listening on port 199 as opposed to 161 (default port)... Which would explain why I had problems pulling up the statistical information via the control panel, but would not explain why it was intermittently pulling it up (sometimes works, sometimes doesn't). ... Which brought my DC admin to think that possibly some kernel-level trojan/process was monitoring legit requests for port 161 and redirecting them to 199 as needed (but not efficiently enough to run undetected by my very brilliant DC admin).

    That being said, I recently (after the first spewage incident) found there was an exploit for 'ls' which I immediately updated on all 3 servers. Though I highly doubt this... there may be a chance that the upgrade from RedHat modified the snmp port (though the details of the RPM say those files haven't changed since Feb 11 2003. My MD5 Checksums on the RPM installed packages for snmp verify properly, so I don't think that is the problem.

    I may be way off here, but port 199 is for an SNMP multiplexor... possibly used for a DDoS... which would makes sense seeing as 3 of my servers on the same subnet have now had their default SNMP port changed...

    The init script for snmp is in tact and verified against a non-affected machine, so that's out of the question.

    I'm really at my wits-end with this problem, and dreadfully need to find if in deed the last resort is the only resort (format/restore), or if someone has some glimpse into a potential answer that may save the day.

    Here’s more information than you ever wanted ::::::::::::::::::::::::::::

    Server Specs:

    SERVER1
    Dell P4 2.6 1GB Ram
    RedHat 9
    WHM 8.5.4 / cPanel 8.5.4-R7 / WHM X v2.1.1
    Kernel: 2.4.20-20.9
    OpenSSH_3.5p1, SSH protocols 1.5/2.0, OpenSSL 0x0090701f
    Apache Core 1.3.28
    Exim 4.24
    PHP 4.3.2 4.3.3
    FrontPage 5.0.2.2634
    mod_ssl 2.8.15
    OpenSSL 0.9.7a
    PHP Suexec 0.1b

    SERVER2
    Dell P4 2.6 1GB Ram
    RedHat 9
    WHM 8.5.4 / cPanel 8.5.4-R7 / WHM X v2.1.1
    Kernel: 2.4.20-8
    OpenSSH_3.5p1, SSH protocols 1.5/2.0, OpenSSL 0x0090701f
    Apache Core 1.3.28
    Exim 4.24
    PHP 4.3.2 4.3.3
    FrontPage 5.0.2.2634
    mod_ssl 2.8.15
    OpenSSL 0.9.7a

    SERVER3
    Dell P4 2.6 1GB Ram
    RedHat 9
    DirectAdmin v1.195
    Kernel: 2.4.20-8
    OpenSSH_3.5p1, SSH protocols 1.5/2.0, OpenSSL 0x0090701f
    Apache 1.3.28
    Exim 4.24
    MySQL 4.0.14
    Named 9.2.1
    ProFTPd 1.2.8
    sshd Running
    vm-Pop3d 1.1.6
    OpenSSL 0.9.7a

    All with similar netstat output:
    # netstat -natp|grep snmp
    tcp 0 0 0.0.0.0:199 0.0.0.0:* LISTEN 2189/snmpd
    tcp 0 0 127.0.0.1:199 127.0.0.1:32773 ESTABLISHED 2189/snmpd
    tcp 0 0 127.0.0.1:32773 127.0.0.1:199 ESTABLISHED 2165/dcsnmp32d
    (note the dcsnmp32d is the SNMP app by the DC to collect info for the control panel)

    Thank you for your time, I know this was a long post. I appreciate any and all replies to this thread.

    Take care,
    .:. Mindlash
     
  2. Website Rob

    Website Rob Well-Known Member

    Joined:
    Mar 23, 2002
    Messages:
    1,506
    Likes Received:
    0
    Trophy Points:
    36
    Location:
    Alberta, Canada
    cPanel Access Level:
    Root Administrator
    Although I cannot help with your current situation, this free script might:

    http://www.rfxnetworks.com/pmon.php

    You would need to get things working again on the proper port, in order for the script to do it's job -- it will ignore services that are currently holding sockets open -but- will alert you when a socket change is made.
     
Loading...

Share This Page