Apache process cause huge load

[Sergiu Tot]

Hello !

I have a problem with an apache process that is causing huge load. It starts from time to time - I'm not sure what is making it start beacause there's nothing in cron, but it appears every few minutes - and when it starts is uses a lot of RAM (up to 1.3GB) and create a huge load on server, up to 70 (from 0.5-1.5 which is the average). If I don't kill the processes immediatly it blocks the server. Another interesting thing that I observed is that it starts 2 or 3 paralell processes.

On strace I get the following result:

Process 14808 attached - interrupt to quit
write(3, "ffb\r\n\n&\n&\n&=\n&=\n&\n&=\n&\n&\n&=\n&\n&="..., 4098) = 4098
write(3, "ffb\r\n=\n&\n&=\n&=\n&\n&\n&=\n&=\n&\n&=\n&\n"..., 4098) = 4098
write(3, "ffb\r\n\n&\n&=\n&=\n&\n&=\n&\n&\n&=\n&=\n&\n&"..., 4098) = 4098
write(3, "ffb\r\n&=\n&\n&\n&=\n&=\n&\n&\n&=\n&\n&=\n&="..., 4098) = 4098
write(3, "ffb\r\n=\n&\n&\n&=\n&\n&=\n&=\n&\n&=\n&\n&\n&"..., 4098) = 4098
write(3, "ffb\r\n\n&\n&\n&=\n&\n&=\n&=\n&\n&=\n&\n&\n&="..., 4098) = 4098
write(3, "ffb\r\n&\n&\n&=\n&\n&=\n&=\n&\n&\n&=\n&=\n&\n"..., 4098) = 4098
write(3, "ffb\r\n\n&\n&=\n&=\n&\n&\n&=\n&\n&=\n&=\n&\n&"..., 4098) = 4098
write(3, "ffb\r\n&=\n&\n&=\n&\n&\n&=\n&=\n&\n&\n&=\n&\n"..., 4098) = 4098
write(3, "ffb\r\n=\n&\n&=\n&\n&\n&=\n&\n&=\n&=\n&\n&\n&"..., 4098) = 4098
write(3

On lsof it shows me that it opens a lot (all, I guess) of files from /usr/local/apache/domlogs.

Does anyone has any idee what the cause may be ?

[Qsave]

Try rebuilding apache.

[nyjimbo]

Does anyone has any idee what the cause may be ?

Well first thing, DO NOT try rebuilding apache at this time.

Second, you are going to need more info. This could be an attack, if you are right and all the domlogs are getting sudden writing to them there could be an attack to flood apache and cause a buffer overrun. The fact it eats that much ram makes me wonder if its something running on the machine itself, because a normal outside http attack should just keep spawning children and not make one http task grow that big.

Can you do an apache status in WHM at that time to get a quick snapshot of any weird stuff, look for that pid that is eating all the ram?. Also can you see if some of the apache logs get similiar lines of info, that is see if several of them at the same moment (or within seconds) get huge requests of junk?.

Do you see pattern to it, that is can you time it within a few seconds or is it fairly random.