I'm running WHM on my development server, and Google has indexed several sites there. Can I do something server-wide to block access from the search engine spider user agents like googlebot, msnbot, etc.?
|Thread starter||Similar threads||Forum||Replies||Date|
|N||Blocking countries : The big Google Ads issue||Security||3|
|W||After latest WHM Update, CSF is blocking all ICMP Requests||Security||1|
|V||Blocking Bad Bots in Apache configuration||Security||1|
|I||CSF is blocking googlebot||Security||1|
|A||Mod Security Rules Incorrectly Blocking Googlebot||Security||14|