The Community Forums

Interact with an entire community of cPanel & WHM users!
  1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

how do I find the script that cause high load?

Discussion in 'General Discussion' started by yousef, Jan 12, 2008.

  1. yousef

    yousef Member

    Joined:
    Jul 20, 2004
    Messages:
    5
    Likes Received:
    0
    Trophy Points:
    1
    Hello,
    the server load goes high in less than one minuets to reach 200 :eek: , when I run "top" command I noticed that the apache use 40% of the CPU resources and the sleeping tasks is around 300 tasks.

    Also in "Main >> Server Status >> CPU/Memory/MySQL Usage" I found that user nobody use a lot of resources.

    Also in the service status I found that cpsrvd and ftpd services are failed.

    My System info is:
    WHM 11.11.0 cPanel 11.16.0-S18450
    CENTOS Enterprise 4.6 i686 on standard - WHM X v3.1.0
     
  2. claudio

    claudio Well-Known Member

    Joined:
    Jul 31, 2004
    Messages:
    201
    Likes Received:
    0
    Trophy Points:
    16
    hi

    you can try during this overload see apache status at WHM to see which web site is using this resources...

    also you can in tweak settings (at whm either) enable pop flood protection and limit number of times that users can acess their pop

    check in your /tmp if there is suspicious files do not run or open then just delete then if you are not sure how check this files

    install PRM (process resource monitor) then adjust http, mysql and general configuration rules this program will stop any process that is over a threshold you had determine and will email you about the issue

    good luck
    claudio
     
  3. driverC

    driverC Well-Known Member

    Joined:
    Jul 23, 2004
    Messages:
    112
    Likes Received:
    0
    Trophy Points:
    16
    I had the same problem and it turned out that hackers were filling up my customers' guestbooks with spam like crazy maxing out MySQL. Now MySQL always appears on top as sleeping even when it is running which is kind of confusing.

    Try this:

    1. Log in via SSH.
    2. Enter "mysql"
    3. Enter "show processlist;"
    4. Check if there is any process in the "Query mode" and look how long it has been running. I.e. a process in query mode that was running for like 30 seconds is almost certainly a hacker filling up one of your guestbooks. Look at the username and find the guestbook or phpBB in the user's account and change the directory permissions to CHMOD 000.

    Another reason why my servers were overloaded recently was a large amount of spam being sent to my server. I could greatly improve the performance by enabling anti spam protection on MTA level: WHM -> Exim configuration and enable Spamcop blacklist.

    Also look at the Apache status page in WHM and look at the Apache processes that are running at the moment. Maybe someone is using a script that is causing a lot of load. Or if you see a lot of ? signs on the status page someone may be doing a DDoS against your server.
     
    #3 driverC, Jan 13, 2008
    Last edited: Jan 13, 2008
  4. yousef

    yousef Member

    Joined:
    Jul 20, 2004
    Messages:
    5
    Likes Received:
    0
    Trophy Points:
    1
    thanks for help guys
    Love you
    :)
    :)
     
  5. erick_paper

    erick_paper Well-Known Member

    Joined:
    Apr 19, 2005
    Messages:
    245
    Likes Received:
    0
    Trophy Points:
    16
    I decided not to use spamcop because it really has a lot of false positives.


    This sounds useful. How should I know which IP is calling what? Or which script? The output of "Apache status" seems very geeky, with lots of "......" (dots).
     
  6. driverC

    driverC Well-Known Member

    Joined:
    Jul 23, 2004
    Messages:
    112
    Likes Received:
    0
    Trophy Points:
    16
    When logged in via shell enter "top". Then press the "i" key to look at running processes only. You may see some Apache processes running for a long amount of time. Now quickly reload the Apache status page and try to find the process ID in the list...this way you will know which site/domain causes the high load. Usually proxy or chat scripts can cause issues. If you see only dots and question marks it is most likely a DDOS attack. In this case edit your httpd.conf and reduce the timeout:

    pico /usr/local/apache/conf/httpd.conf

    Change:
    Timeout 300

    to:
    Timeout 15

    This usually solves DDOS attacks against Apache but in some cases you will need to do more than this. Also if you like paste a copy of these commands here and I'll let you know what I can see:

    top -c -b -n 1

    mysql
    show processlist;

    service httpd fullstatus

    (the fullstatus command may not work...in this case copy the Apache server status page from WHM)
     
  7. erick_paper

    erick_paper Well-Known Member

    Joined:
    Apr 19, 2005
    Messages:
    245
    Likes Received:
    0
    Trophy Points:
    16
    Thanks so much for this really useful and helpful post. I have installed Apache 2.2.6, which seems to be working faster.

    Firstly, I have installed dos_evasive and tuned the timeout and MaxClients stuff. Here are my settings from httpd.conf:

    Secondly, I have also installed the "(D)Dos Deflate" thing from medialayer: http://deflate.medialayer.com/

    Thirdly, I have installed Chirpy's CSF/LFD and enabled people who connect more than 150 times per minute to be blocked, plus I have enabled the new "Syn Flood" feature with a "5/s" timing thing. Is there a better tool for blocking Syn Floods?

    Fourthly, I have also installed "atop" instead of top, and I can tell now which processes are taking the most amount of memory and resources. PLus there is that netstat command to see how many TCP/Syn connections are active.

    Fifthly, I have installed the new mod_sec2, and installed many rules from gotroot.com -- including blacklisting of common compromised boxes, common rootkit attacks, common spambots etc.

    The server seems to be much better now. Apart from these precautions, anything else you can think of that I can use to *prevent* DDoS attacks?

    So many thanks for the info!
     
    #7 erick_paper, Jan 20, 2008
    Last edited: Jan 20, 2008
  8. Kent Brockman

    Kent Brockman Well-Known Member

    Joined:
    Jan 20, 2008
    Messages:
    1,130
    Likes Received:
    3
    Trophy Points:
    38
    Location:
    Buenos Aires, Argentina
    cPanel Access Level:
    Root Administrator
    Twitter:
    Hi! where can i get that "atop"?
    Cheerz
     
  9. troxalias

    troxalias Well-Known Member

    Joined:
    Nov 21, 2001
    Messages:
    96
    Likes Received:
    0
    Trophy Points:
    6
    Location:
    Athens - Greece
    MySQL Processes

    You can find here (http://www.cphelp.gr/kb/?View=entry&EntryID=14) a perl script that automatically monitors mysql and kills users which exceed a specific number of connections or Select Queries that run for a long time. It may be helpful in case you have many dynamic sites on your server.
     
  10. Kent Brockman

    Kent Brockman Well-Known Member

    Joined:
    Jan 20, 2008
    Messages:
    1,130
    Likes Received:
    3
    Trophy Points:
    38
    Location:
    Buenos Aires, Argentina
    cPanel Access Level:
    Root Administrator
    Twitter:
    Cool. I see the code and this script can autoinstall itself to run every 60 seconds. Before starting it: how much resources it consumes? and how to stop it in case I don't finally need it? can I simply and easily kill the pid and exclude its entry from the crontab?
     
  11. troxalias

    troxalias Well-Known Member

    Joined:
    Nov 21, 2001
    Messages:
    96
    Likes Received:
    0
    Trophy Points:
    6
    Location:
    Athens - Greece
    Well in my system it uses around 4MB of RAM in idle mode. While running (every 60 seconds by default) it depends on how large is the output of "show processlist" command of mysql and the connection by itself. I could say that in no way it would require more than 20 MB of RAM. To stop it, just kill the processes pid. There are no entries in the crontab for this to work, but if you want to start in on server reboot you could add it to your rc.sysinit (depending on your distro).
     
  12. Kent Brockman

    Kent Brockman Well-Known Member

    Joined:
    Jan 20, 2008
    Messages:
    1,130
    Likes Received:
    3
    Trophy Points:
    38
    Location:
    Buenos Aires, Argentina
    cPanel Access Level:
    Root Administrator
    Twitter:
    ok, i'll try to give it a try when posible.
    best regards!
     
  13. Kent Brockman

    Kent Brockman Well-Known Member

    Joined:
    Jan 20, 2008
    Messages:
    1,130
    Likes Received:
    3
    Trophy Points:
    38
    Location:
    Buenos Aires, Argentina
    cPanel Access Level:
    Root Administrator
    Twitter:
    hi, in case I find those ? signs on the status page, how can be stopped that DDoS attack?
     
Loading...

Share This Page