DDoS, what to do and how to prevent it

[thecoalman]

Briefly a DDoS attack was started on one of my domains last Monday and continued until today about 4AM, almost an entire week. I probably could have resolved this in about a day with proper support but it ended up being 4 day fiasco because of the abysmal support from my host.

I don't know what the definition of massive is but at it's peak it was about 11.5 million requests per hour, roughly 3000 per second. Here is graph provided by Cloudflare which is service I'll explain in a bit. The reason the bulk of the attack is in the AM is these were mostly overseas computers and that's when the sun is shining in Asia and Europe.




Cloudflare is service that provides many benefits including filtering out a DDoS attack, as a humorous side note this attack has actually increased the performance of my site. All traffic is routed through their network , they have a free plan which surprisingly they let me use through this entire attack. What is surprising abou that is this could not have been cheap, I'm sure the hardware is very expensive to handle this. I'll be upgrading to paid plan, they saved my ass and will do so again if necessary. I'm a cheap kind of guy but I'll pay for value when I see it.

Their is a lot of benefits to this service, just for example they will serve cached pages if your site is down, they block bad bots, scrapers, options for listing fail over servers etc. The DDoS is only one feature.

If your domain is under attack the first thing your host is probably going to do is null route the IP dropping the traffic into a blackhole so any traffic flowing to other IP's on the server can continue.

If this happens to you the first thing to do is obtain a new IP. This is critically important because it's unlikely this happened at random and if someone is determined they can go after the IP directly. If you are doing this as preventive measure realize that IP may be archived and if the person that wants to attack your domain is determined they can look it up. There is two distinct reasons for the new IP.

1. You are going to need to change your DNS entries so traffic is routed through the Cloudflare network. This attack will continue on the old IP until the DNS propagates. In my case requests from a single IP was bringing the server to crawl so if you have hundreds of them it may be quite awhile before it stops. By abandoning the old IP only traffic being routed through Cloudflare will reach the server.
2. If you use Cloudflares DNS service since all traffic is routed through their network the shiny new IP is not exposed. Their service will hide your IP thus the attacker cannot directly attack the IP. I will make one caveat, I don't know if that is 100% effective.

You are also going to want to install mod_evasive or something similar. As I mentioned above one IP was making it through the Cloudflare service, how or why I don't at this point. Mod evasive will be able to clean up anything making it through cloudflare.

One other thing to keep in mind is if your server acts as mail server for that domain the IP can be exposed in an email.

Site has been up for 3 days, I don't know if I'm out of the woods yet or this last week was just round 1. I've tried to cover my bases as much as possible and I'll continue to work at tightening things up even more, any suggestions or critical comments I would love to hear them. I'm no expert and my ear is always open to someone that wants to pass on some knowledge. That is why I posted this, hopefully my very painful experience can make yours less painful if this happens to you.

[thecoalman]

I need to add one more comment, Cloudflare can be enabled in the Plesk panel and other panels. I do not know how it is implemented in the other panels but you do not want to implement it through the Plesk panel. While this is an easy one click implementation there is some glaring issues.

It does not protect the non WWW domain, if you are using the non WWW domain skip it altogether because it's useless. The second problem is the IP remains exposed. Lastly you will need to wait for DNS propagation before it can effectively stop an attack against the www domain.

[thecoalman]

I was hit again a few weeks ago, I had overlooked the IP spitting out the domain. In any event there is ways to prevent a domain from being spit out by the IP through WHM, Plesk or whatever but that is not effective either. If you alter Windows hosts file so example.com points to XXX.XXX.XXX.XXX you can determine that XXX.XXX.XXX.XXX is hosting example.com . After doing some research there is actually tools for this that will go through a net block. If the person attacking your website knows the IP range they can find the IP if you have not protected it.

If you are using Cloudlflare or another proxy you have a giant hammer to fix this issue and some added benfits. All legitimate http(s) traffic should be coming from their IP's thus you can block everyone else. This will also add another layer of DDoS protection for your domain, in the event the IP is found the firewall can drop the connection which is going to be the most efficient way to handle it if it makes it to the server. Your software firewall is not going to stop a large scale attack but it's better than nothing.

Quote:

One other thing to keep in mind is if your server acts as mail server for that domain the IP can be exposed in an email.

There is other vectors for this too, for example if you have a forum that allows remote file uploads from a URL. Some of these things that are exploitable may not be that obvious, phpBB for example has a feature that server side will try and determine the size of an image and scale it down if it exceeds X*X. Features like this will expose the IP.