Please whitelist cPanel in your adblocker so that you’re able to see our version release promotions, thanks!

The Community Forums

Interact with an entire community of cPanel & WHM users!

Apache Processes

Discussion in 'Security' started by reggaebkk, Nov 4, 2014.

  1. reggaebkk

    reggaebkk Registered

    Nov 4, 2014
    Likes Received:
    Trophy Points:
    cPanel Access Level:
    Root Administrator
    I re-open this ticket because this issue has been tormenting me for quite some time.

    My host tells me that they are normal apache processes, and everywhere I read that they are normal.

    I have been looking at this issue very closely and in my case I could notice that:

    - When upgrading from centos 5.x to 6.x, these processes stopped for a few hours, then started to return 1 by 1, and now I have a dozen always open

    - if I kill them all and restart http service, then my load goes down for a while, up to a couple of hours, then returns.

    - they seem to be responsible for high load, but they usually show 0% usage of memory and CPU (or is it Maldet that uses so much resources to keep up with what they are doing)

    - from time to time I see one that is run by root, but most of the time they are run by nobody which is I think the normal way, so why the root one?

    but more concretely this is what bothers me:

    When I track these processes, some show this kind of result:

    "lstat("/home/XXXXXX/public_html/culture_reggae_afro", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
    open("/home/XXXXXX/public_html/culture_reggae_afro/.htaccess", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
    lstat("/home/XXXXXX/public_html/culture_reggae_afro/artists", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
    open("/home/XXXXXX/public_html/culture_reggae_afro/artists/.htaccess", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
    lstat("/home/XXXXXX/public_html/culture_reggae_afro/artists/fj", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
    open("/home/XXXXXX/public_html/culture_reggae_afro/artists/fj/.htaccess", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
    lstat("/home/XXXXXX/public_html/culture_reggae_afro/artists/fj/him.jpg", {st_mode=S_IFREG|0644, st_size=36495, ...}) = 0
    Going on a website that has no php script and look for .htaccess files... fine
    "/home/hotelcom/public_html/crans-montana_directory/submit_useful_link.html", {st_mode=S_IFREG|0644, st_size=14714, ...}) = 0
    open("/.htaccess", O_RDONLY|O_CLOEXEC)  = -1 ENOENT (No such file or directory)
    lstat("/home", {st_mode=S_IFDIR|0711, st_size=4096, ...}) = 0
    open("/home/.htaccess", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
    lstat("/home/YYYYYYY", {st_mode=S_IFDIR|0711, st_size=4096, ...}) = 0
    open("/home/YYYYYYY/.htaccess", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
    lstat("/home/YYYYYYY/public_html", {st_mode=S_IFDIR|0750, st_size=4096, ...}) = 0
    open("/home/YYYYYYY/public_html/.htaccess", O_RDONLY|O_CLOEXEC) = 88
    going to another website that is not directly linked to that 1st one

    "stat("/home/ZZZZZZ/public_html/components", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
    open("/home/ZZZZZZZ/public_html/components/.htaccess", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
    lstat("/home/ZZZZZZZ/public_html/components/com_jvotesystem", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
    open("/home/ZZZZZZZ/public_html/components/com_jvotesystem/.htaccess", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
    lstat("/home/ZZZZZZZ/public_html/components/com_jvotesystem/ajax.php", {st_mode=S_IFREG|0644, st_size=40315, ...}) = 0
    In the process mentioned above, it seems to be looking for .htaccess files everywhere
    I have attached a truncated process tracking output, of course I don't understand a word in it.
    it just goes on and on and on for hours or days.

    In other such processes it's looking for files that don't exist, here index.xhtml and index.shtml... extensions I never use:

    lstat("/home/UUUUUUUU/public_html/country_info/asia/east_asia/china/lhasa/index.shtml", 0x7fff6c9da440) = -1 ENOENT (No such file or directory)
    access("/var/cpanel/bwlimited/", F_OK) = -1 ENOENT (No such file or directory)
    stat("/home/UUUUUUUUo/public_html/country_info/asia/east_asia/china/lhasa/index.xhtml", 0x7fff6c9da440) = -1 ENOENT (No such file or directory)
    lstat("/home", {st_mode=S_IFDIR|0711, st_size=4096, ...}) = 0
    lstat("/home/UUUUUUUU", {st_mode=S_IFDIR|0711, st_size=4096, ...}) = 0
    lstat("/home/UUUUUUUUU/public_html", {st_mode=S_IFDIR|0750, st_size=4096, ...}) = 0
    lstat("/home/UUUUUUUU/public_html/country_info", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
    lstat("/home/UUUUUUUUU/public_html/country_info/asia", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
    lstat("/home/UUUUUUUU/public_html/country_info/asia/east_asia", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
    lstat("/home/UUUUUUUU/public_html/country_info/asia/east_asia/china", {st_mode=S_IFDIR|0750, st_size=4096, ...}) = 0
    lstat("/home/UUUUUUUU/public_html/country_info/asia/east_asia/china/lhasa", {st_mode=S_IFDIR|0750, st_size=4096, ...}) = 0
    When I kill them, restart apache, they return instantly.

    So here are my questions:

    1/ What are these processes actually doing apart from looking for .htaccess or vulnerable files everywhere?

    2/ why doesn't anything on the system controls processes that last so long and work their way from account to account? I have CSF setup pretty tight but it doesn't seem to matter.

    3/ is there a way to track what IP is actually using this process and block it?

    4/ why are these processes increase my load but don't seem to consume cpu and ram?

    5/ how can I setup cpanel to recognise these dodgy processes and kill them fast?

    6/ Is there a way to track eventual script that is exploited or exploiting the system this way? (all my scripts are up to date, but maybe I am carrying a parasite for some time and I don't know about it. I have used freelancers in the past and it's not impossible that one of them left a small gizmo in a corner.

    ... That's a long post for a 1st post... but this issue has been increasingly bothering me for several years now and I'd really like to hear what other people think about it.

    Attached Files:

  2. cPanelMichael

    cPanelMichael Technical Support Community Manager Staff Member

    Apr 11, 2011
    Likes Received:
    Trophy Points:
    cPanel Access Level:
    Root Administrator
    Hello :)

    I have moved this post to our "Security" forum, as those questions are more appropriate here.

    Thank you.
    Stop hovering to collapse... Click to collapse... Hover to expand... Click to expand...
  3. quizknows

    quizknows Well-Known Member

    Oct 20, 2009
    Likes Received:
    Trophy Points:
    cPanel Access Level:
    DataCenter Provider
    A couple things here.

    For gathering data and seeing what IP(s) are in question, I'd have a look at the output of "httpd fullstatus" as well as "lsof -i :80". These should help you track things a little better and help you associate IP addresses to requests and PIDs. Also review the domlogs (domain access logs) for the domain(s) in question.

    It's normal for apache to look for .htaccess anywhere under the user account. If you open a file that's X subdirectories deep, Apache will likely look for any .htaccess files in X subdirectories. For example if I make a web request for /home/UUUUUUUUU/public_html/country_info/asia/east_asia/china/lhasa/index.xhtml, Apache is probably going to look for:

    (and so on).

    Also It is normal to have one httpd process as root and all the child processes as the Apache user ("nobody" on a cPanel system). You may also see /usr/local/cpanel/bin/splitlogs run as root.

    I hope this sheds a little light.

Share This Page

  1. This site uses cookies to help personalise content, tailor your experience and to keep you logged in if you register.
    By continuing to use this site, you are consenting to our use of cookies.
    Dismiss Notice