apache ddos attack anyone help?

[katmai]

i got a ton of those requests, and i really have no idea what to do right now. mod_security will not help because this is not a valid httpd request. any clues?

[Tue Apr 10 00:01:06 2007] [error] [client 82.81.92.220] request failed: erroneous characters after protocol string: $MyNick (adsl)Yw|$Lock EXTENDEDPROTOCOLABCABCABCABCABCABC Pk=DCPLUSPLUS0.690ABCABC|
[Tue Apr 10 00:01:06 2007] [error] [client 82.81.92.220] request failed: erroneous characters after protocol string: $MyNick (adsl)Yw|$Lock EXTENDEDPROTOCOLABCABCABCABCABCABC Pk=DCPLUSPLUS0.690ABCABC|
[Tue Apr 10 00:01:06 2007] [error] [client 82.81.92.220] request failed: erroneous characters after protocol string: $MyNick (adsl)Yw|$Lock EXTENDEDPROTOCOLABCABCABCABCABCABC Pk=DCPLUSPLUS0.690ABCABC|
[Tue Apr 10 00:01:06 2007] [error] [client 82.81.92.220] request failed: erroneous characters after protocol string: $MyNick (adsl)Yw|$Lock EXTENDEDPROTOCOLABCABCABCABCABCABC Pk=DCPLUSPLUS0.690ABCABC|
[Tue Apr 10 00:01:06 2007] [error] [client 82.81.92.220] request failed: erroneous characters after protocol string: $MyNick (adsl)Yw|$Lock EXTENDEDPROTOCOLABCABCABCABCABCABC Pk=DCPLUSPLUS0.690ABCABC|
[Tue Apr 10 00:01:06 2007] [error] [client 82.81.92.220] request failed: erroneous characters after protocol string: $MyNick (adsl)Yw|$Lock EXTENDEDPROTOCOLABCABCABCABCABCABC Pk=DCPLUSPLUS0.690ABCABC|

[katmai]

nobody came close to this issue ?

[brendanrtg]

Setup a firewall and ban that ip address?

[chae]

Hi Yah,

Do a search on here for $MyNick you'll see two post from me regarding this. The only option I had was to null route the IP until the attack was over.

[Parahosting]

Banning the IP will not fix the problem, because there might be a chance that the IP address WILL change and render it useless. Possibilities are, you're dealing with an actual hacker.

[jayh38]

try CSF, it will do a nice job keeping up with the IP changes as needed. set the connection limit to 200 which is default I believe. There are many other tools in there that may help deal with a situation like this.

[brianoz]

I second the comments on csf - it will detect the flood and ban that IP, and if the IP changes it will also ban the new IP. It won't cope well with a true DDOS - nothing does, but it will help a lot.

www.configserver.com/cp/csf.html

[ramprage]

I wrote a custom DDoS script just for these type of attacks. It runs on cron and detects IPs which have to many connections and can block them using APF or CSF or just iptables.

Works well, I used it with a client who was getting a major attack. Some ips had over 4,000 connections. In less than half hour it blocked them all and saved his server. He tried everything for days and nothing worked. Half hour after my script was on it saved his server.

So yeah it works well

[pirrup]

could you post the script? would be very helpfull for others

[katmai]

guys . for 1 thing.

the attack is run by admins of big dc++ hubs, they make a redirect on their users to the apache, and overload it with requests.

mod_security - no use because it's not a valid http request.

one thing if you run iptables is to setup:

iptables -I INPUT -p tcp --dport 80 -i eth0 -m state --state NEW -m recent --set
iptables -I INPUT -p tcp --dport 80 -i eth0 -m state --state NEW -m recent --update --seconds 10 --hitcount 5 -j DROP


this would help in dropping part of the illegal connections, doesn't solve the issue, but at least the apache doesn't crash and still is accessible.

another solution would be to recompile the kernel, which i don't quite like it right now screwing my 140 days uptime i know sounds silly, but i still would love some help on this.

btw regarding csf.

pretty useless, imagine having to drop like 1000 ip addresses, but what do you do if the hub ddosing you is 5000 users or 8000 users ? you make 16000 rules in csf? your server will overload even without any ddos going on at that point. unless you have some quad processors and hell of ram to do it.

second of all. the connections don't stay established. for example, 1 ip gets like 2 3 connections, it takes only 350 ips to fill up your apache (i run myself 1000 maxclients) so csf will not pick up anything.

[ramprage]

could you post the script? would be very helpfull for others

My script isn't free, this one is $25 if interested.

[Cristi4n]

Also remember, when using:
iptables -I INPUT -p tcp --dport 80 -i eth0 -m state --state NEW -m recent --set
iptables -I INPUT -p tcp --dport 80 -i eth0 -m state --state NEW -m recent --update --seconds 10 --hitcount 5 -j DROP

enable KeepAlive in apache, set Timeout to something small like 10 set keepalive timeout more than 10 seconds, and you will block a lot of repetead connections and allow legit clients to connect.
I personally used seconds 4 and hitcount 2. (DO NOT drop hitcount below 2).
One other problem remains, the flood will still continue, at lower rate. If you can keep up it's ok. Remember not to end up in swap during that attack, increase apache limit until everything works ok.
If you want to block the ip's permanently (I blocked like 20.000 ips, a small number) you can use snort with snortsam and add a rule to block everuthing containing $MyNick, or simply use snort-inline and block everything containing $MyNick or DCPLUSPLUS. Snort Inline will work better as packets will not get to apache.
Are you sure you only see lines with $MyNick or you see also a lot more lines with a simple "-" ?

Remember also, there is no way to block a ddos, you can only mitigate the attack.
CSF and APF in my case were no use. I had a lot of IP's (DDOS not DOS). Also consider that hardware firewalls will not help in such attack. DO NOT THROW your money on hardware firewalls. You can instead buy 1 or 2 GB of ram and increase apache limit even more and at lower price.

I managed to mitigate the attack at my server (was not easy) but I did it and after 48 hours it was gone.

[AndyReed]

Also consider that hardware firewalls will not help in such attack. DO NOT THROW your money on hardware firewalls.

I'll have to disagree with you on this. If your host doesn't have a good hardware firewall, then you are out of luck. One of our clients hosting his dedicated server with us was under heavy DDoS attack. Since we use the latest hardware firewall, Cisco ASA 5520, although was a little slow from time to time during that nasty attack, his server never went down and was/is accessible all the time.

[Spiral]

the attack is run by admins of big dc++ hubs, they make a redirect on their users to the apache, and overload it with requests.

Just a tip for everyone ....

As one hosting server generally doesn't need to talk to another other
than for DNS and email traffic, I will often block entire data centers
from my servers for inbound web access.

This is very easy to do with IPTABLES and basically would allow any visitor
coming from a home cable or DSL connection to reach your web server
but block web access from any server machine out there.

iptables -A INPUT -s 0.0.0.0/0 -d 1.1.1.1/1 -p tcp --dport 80:2097 -j REJECT

*** 0.0.0.0/0 would be replaced by the CIDR range to be blocked ****

*** 1.1.1.1/1 would be replaced by your server's IP or CIDR range ****

(I put the destination into the iptables block so that outbound connections
from your server to some other server would still work while incoming
connections from the banned IP range would be blocked)

This does a nice job of blocking proxies too since nearly all proxies
run off of server machines hosted at data centers.

You can find out the IP ranges assigned to data centers with simple
WHOIS inquiries against sample IPs at http://www.arin.net

[Cristi4n]

Ok, MABY there are some good hardware firewalls. Still most of them do nothing in case of a ddos.
You do not need a hardware firewall that blocks spyware in this case, so for anyone that even thinks of buying one please ask someone with high experience.
IT DEPENDS IF IT IS A DDOS OR DOS. Many people talk about dos and ddos as they are the same thing.
In the case presented above, I see only one IP, and this can be consider a DOS. Still I believe the guy pasted only a few lines and he is dealing with the same problem I had. A ddos not a dos.
In this case:
One ip connects 3~4 times and then waits for 5~10 minutes and connects again.
SO:
CSF, APF and others DO NOT HELP
Scripts like the above that costs 25$ DOES NOT HELP AT ALL. CSF and APFdoes the same thing and maby even better (Is good for DOS, not DDOS)
mod_evasive a.s.o does not help either.
The rules for iptables presented above are a good start. After that, you are on your own.
You can do a simple cat in the error log and grep for all the ips that flood untill now and ban them. That is also good as they will try to reconnect.


About blocking a range of IPs, can be a solution but was not a good one in my case. My server has clients from country X for example and the ddos came from 3 major networks in that country. Blocking range of ips is a bad ideea as you will block almost all your legit visitors.
It can be a good ideea in some cases.
Also I saw in error log lines with $MyNick and DCPLUSPLUS but I do not think they are real DC clients. Real DC clients do not try to reconnect that fast and do not cause a ddos for 24 hours.
In my case, a client sent a line with $MyNick then started to send "-" and only that. They are not real DC clients, they are just computers that got a virus. I just unblocked a client of mine that was flooding my server without knowing. He doesn't even have a DC client installed.