Re: Apache displaying massive Reading request, making the webpage down. att
Fellows,
Recently we got a DoS attack to one of our servers and it just keeping overloading Apache through tons of Reading processes but it didn’t indicate the domain attacked or the IP causing the attack.
We fixed the situation applying the following procedure:
First of all we started suspending several accounts through WHM (the accounts with more traffic) and analyzing the error_log file with:
tail -f /etc/httpd/logs/error_log
The idea was to detect repetitive abnormal errors. In parallel, We had to stop and start apache several times and cleaning any process related to apache during our analysis since the DoS attack was happening at the same time.
We did that with:
service httpd stop
fuser -k 80/tcp
service httpd start
After suspend several accounts we detected an abnormal repetitive error showing:
[error] [client XXX.XXX.XXX.XXX] request failed: error reading the headers
We proceed to block such IP using CSF/BFD and the attack was totally stopped. Then we unsuspended the accounts and all came back to the normal behavior.
Finally, please note that there are several ways to detect the IP attacking the server. They are:
To see what IPs are being connected to the server and how many connections are for each IP:
netstat -anp |grep 'tcp\|udp' | awk '{print $5}' | cut -d: -f1 | sort | uniq -c | sort -n
To see how many connections are being received by each server IP:
netstat -plan |grep :80 | awk '{print $4}' | cut -d: -f1 | sort | uniq -c | sort -n
To see the total amount of active apache connections:
netstat -apn | grep :80 | wc -l
To get the total update on the apache status to see what domain is receiving the bigger amount of hits:
lynx
Apache Status
Other helpful command:
/usr/sbin/httpd fullstatus
We hope it could help if you are having an attack. God Luck …
