The Community Forums

Interact with an entire community of cPanel & WHM users!
  1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Server Load and Runaway processes

Discussion in 'General Discussion' started by pguy, Jul 1, 2003.

  1. pguy

    pguy Well-Known Member

    Joined:
    Feb 21, 2003
    Messages:
    73
    Likes Received:
    0
    Trophy Points:
    156
    Hello Folks,

    Although we have had this issue before, this time it has had me stumped. The server load seems to spike to 150 or so and renders all unusable. Sites are not served and all goes down until a reboot or waiting for 10 - 15 minutes tends to bring the server load back to some usable level.


    I've gone through all the logs and have found absolutely no clue. No formmails, no spam etc

    At times I have noticed that httpd tends to start growing in size. Via the top process it can be seen taking up sizes above 200MB or so. I have checked access_logs , CPU usage from WHM and there is nothing weird at all !

    Even the httpd, if restarted when the size starts growing, tends to function normally and brings down loads to around 1 or so.

    Where else can i see ? I seriously am stumped and need some clues as to how I get about isolating the problem.

    Warmest Regards,

    cPanel.net Support Ticket Number:
     
    #1 pguy, Jul 1, 2003
  2. pguy

    pguy Well-Known Member

    Joined:
    Feb 21, 2003
    Messages:
    73
    Likes Received:
    0
    Trophy Points:
    156
    Found the issue and boy was it tough :D

    One of our clients had put in a Rewrite rule in .htacess which stated a certain page to load if a 404 occured. Here the page which was to be loaded was not found and this pushed apache into a loop. If you just hit any 404 page, it could take the server down ! <sheesh>


    I really had thought apache would have been able to handle such an issue.. :rolleyes:

    cPanel.net Support Ticket Number:
     
    #2 pguy, Jul 2, 2003
  3. hostultra

    hostultra Well-Known Member

    Joined:
    Aug 21, 2002
    Messages:
    167
    Likes Received:
    0
    Trophy Points:
    166
    How did you find it?

    cPanel.net Support Ticket Number:
     
    #3 hostultra, Jul 2, 2003
  4. Website Rob

    Website Rob Well-Known Member

    Joined:
    Mar 23, 2002
    Messages:
    1,505
    Likes Received:
    0
    Trophy Points:
    316
    Location:
    Alberta, Canada
    cPanel Access Level:
    Root Administrator
    Nice catch pguy, and hat's off to someone's troubleshooting ability. I too, would be interested in knowing how it was done as runaway scripts can be a real 'bug-a-boo' for any ServerAdmin.

    cPanel.net Support Ticket Number:
     
    #4 Website Rob, Jul 2, 2003
  5. sexy_guy

    sexy_guy Well-Known Member

    Joined:
    Mar 19, 2003
    Messages:
    848
    Likes Received:
    0
    Trophy Points:
    166
    Well im sure he doesnt have 400 sites on his box so finding something like that is doeable. Try finding something like that 417 sites on the box.

    cPanel.net Support Ticket Number:
     
    #5 sexy_guy, Jul 2, 2003
  6. ciphervendor

    ciphervendor Well-Known Member

    Joined:
    Aug 26, 2002
    Messages:
    1,052
    Likes Received:
    0
    Trophy Points:
    166
    What kernel / OS are you running?

    cPanel.net Support Ticket Number:
     
    #6 ciphervendor, Jul 2, 2003
  7. pguy

    pguy Well-Known Member

    Joined:
    Feb 21, 2003
    Messages:
    73
    Likes Received:
    0
    Trophy Points:
    156
    I actually have 800 sites on that specific box. So that made the job that more tough.

    I am sure the way I did it would help someone out there. TOP had to be constantly running. I for one simply had no clue as to howto lay the one on this issue.

    I started to see httpd's size grow upto 400MB and increasing at a rate of around a 100 MB every minute. I saw the PID and then in WHM clicked on "Current Processes" , then clicked on the right PID and there was a clue to the website name.

    To make it all the worse, it was a robots.txt ! Damn... one seek by a search engine and the server would go down ! Now thats news isn't it !

    I tried to browse to that robots.txt , and found that it wasn't coming up. I was here thinking that may be it was a spider trapped in a loop or something. But that wasn't meant to be.

    Since robots.txt didn't come up, it meant that .htaccess was setup with some values. So checked that and voila a Rewrite rule was put in stating that if a 404 was found, it should load a specific php file. Now this php file was missing and this caused apache to go into tizzy everytime someone hit a 404 on that page !

    I could call it an offending piece of code, but the client was actually using the same piece of code on other websites rather successfully. The only difference - the referenced php file was available on those sites and here it wasn't.


    Funny, it wasn't a runaway script was it ? That's where I was looking all this while ! Just a simple issue. Even after setting up WHM to ensure Apache's size not to exceed 196 MB or so, that seems to have no effect on issues like these.

    But yes, one 404 and the server down ;) .. that sums this case up.

    Cheers
    :D

    cPanel.net Support Ticket Number:
     
    #7 pguy, Jul 2, 2003
  8. pguy

    pguy Well-Known Member

    Joined:
    Feb 21, 2003
    Messages:
    73
    Likes Received:
    0
    Trophy Points:
    156
    We're running RH 7.3 with 2.4.9-34enterprise Kernel.

    cPanel.net Support Ticket Number:
     
    #8 pguy, Jul 2, 2003
  9. ciphervendor

    ciphervendor Well-Known Member

    Joined:
    Aug 26, 2002
    Messages:
    1,052
    Likes Received:
    0
    Trophy Points:
    166
    Any particular reason why you're using the enterprise kernel? Have you considered manually compiling 2.4.21 on your box?

    cPanel.net Support Ticket Number:
     
    #9 ciphervendor, Jul 2, 2003
  10. pguy

    pguy Well-Known Member

    Joined:
    Feb 21, 2003
    Messages:
    73
    Likes Received:
    0
    Trophy Points:
    156
    No particular reasons. The kernel was updated quite a while ago by the DC guys, when the previous kernel had some RAM usage issues. Haven't tried an updat e to the latest version yet.

    Any specific reason I need to look into for a kernel upgrade ?

    cPanel.net Support Ticket Number:
     
    #10 pguy, Jul 2, 2003
  11. ciphervendor

    ciphervendor Well-Known Member

    Joined:
    Aug 26, 2002
    Messages:
    1,052
    Likes Received:
    0
    Trophy Points:
    166
    Well the new kernel certainly has a lot of vuln plugs; one of the most important is ptrace. The most recent version also includes IDE speed improvements. Memory management isn't an issue anymore and 2.4.21 has been incredibly stable for me on SCSI/RAID & IDE machines.

    You'll notice reliability & speed increases in your system after this kernel upgrade.
     
    #11 ciphervendor, Jul 2, 2003
  12. ciphervendor

    ciphervendor Well-Known Member

    Joined:
    Aug 26, 2002
    Messages:
    1,052
    Likes Received:
    0
    Trophy Points:
    166
    Just kind of further to the above....are you still running ext2? If so, you would also benefit from an upgrade to the ext3 file system.

    cPanel.net Support Ticket Number:
     
    #12 ciphervendor, Jul 2, 2003
  13. pguy

    pguy Well-Known Member

    Joined:
    Feb 21, 2003
    Messages:
    73
    Likes Received:
    0
    Trophy Points:
    156
    With 800 onboard sites, I am a definitely queasy about trying a kernel update for now. I've read quite a bit of "how-to"s on it, but never tried to update one before.

    I might have the DC staff take a look at it. Mighty thanks for the tip and the heads up on the vulnerability issue.

    Anything to get the machine go faster and remain stable over periods of time :D

    cPanel.net Support Ticket Number:
     
    #13 pguy, Jul 2, 2003
  14. pguy

    pguy Well-Known Member

    Joined:
    Feb 21, 2003
    Messages:
    73
    Likes Received:
    0
    Trophy Points:
    156
    Didn't they have an issue with ext3 and quotas working only on RH 7.2 ?

    cPanel.net Support Ticket Number:
     
    #14 pguy, Jul 2, 2003
  15. ciphervendor

    ciphervendor Well-Known Member

    Joined:
    Aug 26, 2002
    Messages:
    1,052
    Likes Received:
    0
    Trophy Points:
    166
    That was fixed in kernel release 2.4.17

    cPanel.net Support Ticket Number:
     
    #15 ciphervendor, Jul 2, 2003
  16. pguy

    pguy Well-Known Member

    Joined:
    Feb 21, 2003
    Messages:
    73
    Likes Received:
    0
    Trophy Points:
    156
    Thanks mate. You have been very helpful. I guess its time to look at a kernel update for the box. Anything to keep em secure and fast ;)

    Thanks for your time and concern :)

    Cheers,

    cPanel.net Support Ticket Number:
     
    #16 pguy, Jul 2, 2003
  17. ciphervendor

    ciphervendor Well-Known Member

    Joined:
    Aug 26, 2002
    Messages:
    1,052
    Likes Received:
    0
    Trophy Points:
    166
    If you recompile your kernel you may also want to consider patching 2.4.21 with grsecurity (grsecurity.net). That will keep your box tighter than a frogs ass under water.

    cPanel.net Support Ticket Number:
     
    #17 ciphervendor, Jul 2, 2003
  18. pguy

    pguy Well-Known Member

    Joined:
    Feb 21, 2003
    Messages:
    73
    Likes Received:
    0
    Trophy Points:
    156
    Point Noted and Taken. You're good about server security. I haven't really looked into all the other areas of security, but the kernel and GRsecurity seem to be the way to go.

    Just got APF installed and it seems to be working pretty well. I've got lots more to learn and I am sure with folks like you around, I know I have support and hope :D

    cPanel.net Support Ticket Number:
     
    #18 pguy, Jul 2, 2003
(You must log in or sign up to post here.)
Loading...
Similar Threads - Server Load Runaway
  1. Robertigno

    Server overloaded

    Robertigno, Jul 13, 2017, in forum: General Discussion
    Replies:
    7
    Views:
    100
    cPanelMichael
    Jul 24, 2017 at 3:31 PM
  2. crismarques silva

    Dedicated server overloading because of mysql

    crismarques silva, Jun 21, 2017, in forum: Database Discussions
    Replies:
    3
    Views:
    132
    cPanelMichael
    Jun 21, 2017
  3. Italo Duarte

    Load on Server Issue

    Italo Duarte, Apr 28, 2017, in forum: General Discussion
    Replies:
    4
    Views:
    223
    cPanelMichael
    Apr 28, 2017
  4. Mosaab

    Problem with server load

    Mosaab, Feb 26, 2017, in forum: General Discussion
    Replies:
    5
    Views:
    321
    Mosaab
    Mar 2, 2017
  5. lambov

    server load very high suddenly

    lambov, Feb 16, 2017, in forum: EasyApache
    Replies:
    2
    Views:
    665
    cPanelMichael
    Feb 17, 2017

Share This Page