The Community Forums

Interact with an entire community of cPanel & WHM users!
  1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

GoogleBot won't leave domain alone

Discussion in 'General Discussion' started by keat63, Apr 15, 2015.

  1. keat63

    keat63 Well-Known Member

    Joined:
    Nov 20, 2014
    Messages:
    765
    Likes Received:
    20
    Trophy Points:
    18
    cPanel Access Level:
    Root Administrator
    I have a test domain, that i'd prefer to not have listed in Google etc.
    I don't know how, as i've never submitted it to Google, but it's listed, and I can't seem to get it de-listed.
    About a month ago, i renamed the public_html directory (it's only a development site, i don't need it just yet), but many times per day, GoogleBot is here looking for the domain.

    It was ignoring robots.txt, hence the reason i took the whole site down, thinking that Google would get the message and take a hike.

    I don't want to firewall it out, as this may affect the other valid domains.

    To be perfectly honest, it doesn't really matter too much, but it's annoying me that they won't leave me alone.
    Any other thoughts on how i might achieve this, or will it eventually give up of it's own accord ?
     
  2. cPanelMichael

    cPanelMichael Forums Analyst
    Staff Member

    Joined:
    Apr 11, 2011
    Messages:
    30,811
    Likes Received:
    671
    Trophy Points:
    113
    cPanel Access Level:
    Root Administrator
  3. keat63

    keat63 Well-Known Member

    Joined:
    Nov 20, 2014
    Messages:
    765
    Likes Received:
    20
    Trophy Points:
    18
    cPanel Access Level:
    Root Administrator
    Unfortunately, i don't think it's going to be that easy, as the web site is a PHP site.

    I did see something where you can request to remove using Google web master tools, but this particular site isn't listed in Web Master Tools in the first place, as i didn't want it indexing.
     
  4. cPanelMichael

    cPanelMichael Forums Analyst
    Staff Member

    Joined:
    Apr 11, 2011
    Messages:
    30,811
    Likes Received:
    671
    Trophy Points:
    113
    cPanel Access Level:
    Root Administrator
    Hello,

    I'm not aware of any other methods you can use to block Google from indexing the website, other than by blocking their IP addresses in a firewall.

    Thank you.
     
  5. nwd

    nwd Member

    Joined:
    Feb 28, 2014
    Messages:
    13
    Likes Received:
    0
    Trophy Points:
    1
    cPanel Access Level:
    Root Administrator
    If the site is accessible on the web at all, the "bots" will find it. Surprise you only talking about Google as Yahoo, bing, Ask and other bots will find it as well and continue to "bother" it. Since its a "test" site, you can modify the htaccess file for that account/site to Deny all other IP's, and allow just your IP (or set of IP's / ranges).
    Code:
    order deny,allow
    deny from all
    allow from 111.222.333.444
    This way you don't mess with your firewall which of course affects all sites on the server, and this method will only affect the one in question.
    If you are not sure how to modify your htaccess file, heres the info you need. http://stackoverflow.com/questions/4400154/htaccess-deny-all-allow-only-one-ip
    thanks,
    Jas
     
Loading...

Share This Page