Lately, a lot of our customers' websites has been crawled by a lot of bots. Yesterday, a single website was crawled by 4 different bots at the same time. All of the bots were bad bots.
We want to block these bots but I'm wondering which method is the best performance wise or if it really doesn't matter.
So, does anyone have any recommendations for blocking bad bots?
We want to block these bots but I'm wondering which method is the best performance wise or if it really doesn't matter.
So, does anyone have any recommendations for blocking bad bots?