Apache/PHP CPU issue

[n000b]

One of my servers has randomly started crashing occasionally (once every few days). The server is staying online but the load is spiking up to 400+ and the server is just grinding to a halt. I managed to get some logs while it happened, and I noticed that Apache/PHP is absolutely flogging the server. Here is my Apache's status at a normal time on the server:

Apache Server Status for localhost

Server Version: Apache/2.2.14 (Unix) mod_ssl/2.2.14
OpenSSL/0.9.8e-fips-rhel5 mod_auth_passthrough/2.1 mod_bwlimited/1.4
FrontPage/5.0.2.2635

Server Built: Mar 4 2010 18:15:04
_________________________________________________________________

Current Time: Monday, 12-Jul-2010 12:24:01 EST
Restart Time: Monday, 12-Jul-2010 12:23:02 EST
Parent Server Generation: 51
Server uptime: 59 seconds
Total accesses: 366 - Total Traffic: 33.7 MB
CPU Usage: u.44 s.17 cu7 cs0 - 12.9% CPU load
6.2 requests/sec - 0.6 MB/second - 94.4 kB/request
8 requests currently being processed, 17 idle workers

WW__.__..K.C.W__._...W_W___W__.._.._...__.......................
................................................................
................................................................
................................................................

Scoreboard Key:
"_" Waiting for Connection, "S" Starting up, "R" Reading Request,
"W" Sending Reply, "K" Keepalive (read), "D" DNS Lookup,
"C" Closing connection, "L" Logging, "G" Gracefully finishing,
"I" Idle cleanup of worker, "." Open slot with no current process

And here it is during the problem:

Apache Server Status for localhost

Server Version: Apache/2.2.14 (Unix) mod_ssl/2.2.14
OpenSSL/0.9.8e-fips-rhel5 mod_auth_passthrough/2.1 mod_bwlimited/1.4
FrontPage/5.0.2.2635

Server Built: Mar 4 2010 18:15:04
_________________________________________________________________

Current Time: Monday, 12-Jul-2010 12:21:03 EST
Restart Time: Saturday, 10-Jul-2010 08:41:16 EST
Parent Server Generation: 48
Server uptime: 2 days 3 hours 39 minutes 47 seconds
Total accesses: 542650 - Total Traffic: 4.9 GB
CPU Usage: u1.43 s.87 cu65.11 cs0 - .0362% CPU load
2.92 requests/sec - 27.9 kB/second - 9.6 kB/request
181 requests currently being processed, 36 idle workers

WCWCCWCWCWWCCCCCWCLWCCWCWWCW_WWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWW
WWWWWWWWWWWWWWWWWWWWWWWWWWWWWW_WWWWWWWWWWWWWWWWW_WWWWWWWW.WWWW.W
WWWWW_W.WWWWWWWW___.WWWWWWWWW..W__KWW._WWW_____WW_W.W.WWW__.W_W_
_W.WW_._.WW__._W__.__W_WWWWW_W.W.._W.W__.WW._W..................

Scoreboard Key:
"_" Waiting for Connection, "S" Starting up, "R" Reading Request,
"W" Sending Reply, "K" Keepalive (read), "D" DNS Lookup,
"C" Closing connection, "L" Logging, "G" Gracefully finishing,
"I" Idle cleanup of worker, "." Open slot with no current process

During the problem there are hundreds of PHP processes running and 50+ Apache processes running.

The reason the load is spiking is because the server is running out of memory (the server usually sits at around 1GB free memory, but this problem uses it all up). Does anyone know what could be causing Apache or PHP to do this? I'm running PHP5, Apache 2.2 with suPHP.

[Miraenda]

Are you using suPHP and can see which users are running these processes? The WHM > Daily Process Logs area should be showing which specific users are high usage. Given it's hundreds of PHP processes, it sounds more like one or more accounts causing it than anything else. If it were a denial of service attack, those normally result in high number of Apache processes but not a high number of PHP processes. Some high traffic or high usage script is the most likely culprit.

[n000b]

Thank you - I have studied the logs and it appears as though a certain website is getting hit hard by something.