GoogleBot won't leave domain alone

keat63

Well-Known Member
Nov 20, 2014
1,904
254
113
cPanel Access Level
Root Administrator
I have a test domain, that i'd prefer to not have listed in Google etc.
I don't know how, as i've never submitted it to Google, but it's listed, and I can't seem to get it de-listed.
About a month ago, i renamed the public_html directory (it's only a development site, i don't need it just yet), but many times per day, GoogleBot is here looking for the domain.

It was ignoring robots.txt, hence the reason i took the whole site down, thinking that Google would get the message and take a hike.

I don't want to firewall it out, as this may affect the other valid domains.

To be perfectly honest, it doesn't really matter too much, but it's annoying me that they won't leave me alone.
Any other thoughts on how i might achieve this, or will it eventually give up of it's own accord ?
 

keat63

Well-Known Member
Nov 20, 2014
1,904
254
113
cPanel Access Level
Root Administrator
Unfortunately, i don't think it's going to be that easy, as the web site is a PHP site.

I did see something where you can request to remove using Google web master tools, but this particular site isn't listed in Web Master Tools in the first place, as i didn't want it indexing.
 

cPanelMichael

Administrator
Staff member
Apr 11, 2011
47,910
2,216
463
Hello,

I'm not aware of any other methods you can use to block Google from indexing the website, other than by blocking their IP addresses in a firewall.

Thank you.
 

nwd

Member
Feb 28, 2014
15
0
51
cPanel Access Level
Root Administrator
If the site is accessible on the web at all, the "bots" will find it. Surprise you only talking about Google as Yahoo, bing, Ask and other bots will find it as well and continue to "bother" it. Since its a "test" site, you can modify the htaccess file for that account/site to Deny all other IP's, and allow just your IP (or set of IP's / ranges).
Code:
order deny,allow
deny from all
allow from 111.222.333.444
This way you don't mess with your firewall which of course affects all sites on the server, and this method will only affect the one in question.
If you are not sure how to modify your htaccess file, heres the info you need. http://stackoverflow.com/questions/4400154/htaccess-deny-all-allow-only-one-ip
thanks,
Jas