Script to automatically e-mail output of "top" command

[casey]

Does anyone have or know of a script that will automatically send the output of "top" when the load reaches a certain level? I am having a load spike issue that I'm finding impossible to track.

I get load spikes of up to around 25 for about 4 or 5 minutes at a time. The problem is that it does not occur on a regular schedule (and not even every day), so it's not a cron job. After 4 or 5 minutes the load returns to normal. I happened to be looking at the hotsanic graphs today and caught it as it was happening, but the load was so high that I could not connect to the server via ssh. Of course, by the time I contacted my datacenter to have a look via console the load was already back to normal.

I have checked
/usr/local/apache/logs/error_log
/usr/local/apache/logs/suexec_log
/var/log/exim_mainlog
/var/log/messages

but none of those show any suspicious activity at all during the load spikes. I'm thinking it's someone uploading or untarring files, probably not even realizing they're using so much of the available resources.

 

[kernow]

No script, but you could install PRM. This is a script that monitors cpanel, apache, mysql etc and emails you when a service has exceeded the defined resource limits. The email also tells you which user was causing the heavy load. More info here:
http://www.webhostgear.com/85.html

 

[casey]

Thanks. I actually have prm installed, but it's not catching anything. Maybe I should look into lowering some of the limits...

 

[kernow]

Yep, lower them right down for a day or two and see who's causing the problem.

 

[PWSowner]

I thought you knew about my WHM/cPanel scripts. I have a server monitor script that does just what you need.
(see my sig)

 

[Danny_T]

The post of PWSowner above: To access the scripts you must join The Club and must pay for it. Just to let the readers know that the Monitoring script is not a free script.

Danny.

 

[casey]

I thought you knew about my WHM/cPanel scripts. I have a server monitor script that does just what you need.
(see my sig)

As a long time member, yes, I do know about those scripts. I knew I had read something about such a script somewhere, I just couldn't remember where. Thank you. :D

 

[casey]

The post of PWSowner above: To access the scripts you must join The Club and must pay for it. Just to let the readers know that the Monitoring script is not a free script.

Danny.

Actually, the script is free. You can get "added features" by joining, but the monitoring script itself is free.

 

[PWSowner]

The post of PWSowner above: To access the scripts you must join The Club and must pay for it. Just to let the readers know that the Monitoring script is not a free script.

Danny.

The free version of the server monitor script works just fine for most people. The member version just has some extra features.

WHM addon scripts are the only ones that require membership to get access to and a one time $10 fee for all scripts isn't much either.

 

[PWSowner]

As a long time member, yes, I do know about those scripts. I knew I had read something about such a script somewhere, I just couldn't remember where. Thank you. :D

No problem. I figured you needed a reminder. You were one of the first scripts club members.

 

[shaun]

you could take a look at /var/log/dcpumon also, depending on how long ago your load spiked up a log might be there. Also if the server locked up from excessive load this is a good place to take a look...

 

[casey]

you could take a look at /var/log/dcpumon also, depending on how long ago your load spiked up a log might be there. Also if the server locked up from excessive load this is a good place to take a look...

Thanks for the info. I'll take a look there next time it spikes. The server isn't locking up exactly, it just becomes slow for about 4 minutes or so.

 

[PWSowner]

Thanks for the info. I'll take a look there next time it spikes. The server isn't locking up exactly, it just becomes slow for about 4 minutes or so.

That's what was happening to me a while back too. That was when I wrote the server monitor script. Like you, I couldn't catch the problem, but the script did. It turned out someone had a homemade perl script to crawl Yahoo and Dmoz that they ran manually now and then.

 

[PWSowner]

you could take a look at /var/log/dcpumon also, depending on how long ago your load spiked up a log might be there. Also if the server locked up from excessive load this is a good place to take a look...

I need to add that to my WHM View Log Files script.
It currently doesn't include those files.

 

[Manuel_accu]

This may be useful:

http://forums.cpanel.net/showthread.php?t=56995&

 

[casey]

Well, thanks to Mike's script I have been able to determine that it is Apache that is causing the load spikes. When they spike I have several httpd processes that each use up to 50% CPU. Since these spikes are unpredictable I am having a terrible time pinpointing whether it is an attack or a user's script. Is there something else I can do? The loads drop back to normal after 5 or 10 minutes, so I am inclined to think that it is a user doing something rather than an attack.

 

[panayot]

You can see what is apache doing from WHM -> Apache Status, or you can put this line in your monitoring script:

GET http://127.0.0.1/whm-server-status