Please whitelist cPanel in your adblocker so that you’re able to see our version release promotions, thanks!

The Community Forums

Interact with an entire community of cPanel & WHM users!

How to stop google indexing of /~

Discussion in 'General Discussion' started by trec-r, Aug 13, 2016.

  1. trec-r

    trec-r Active Member

    Joined:
    Aug 10, 2008
    Messages:
    41
    Likes Received:
    2
    Trophy Points:
    58
    cPanel Access Level:
    Root Administrator
    Some of my client's websites are now showing up in google as mydomain.com/~theirsite. How can I prevent Google from indexing anything in mydomain.com/~. I tried to change the robots.txt file but the google search console still shows the URLs as allowed.
     
  2. 24x7server

    24x7server Well-Known Member

    Joined:
    Apr 17, 2013
    Messages:
    1,878
    Likes Received:
    89
    Trophy Points:
    78
    Location:
    India
    cPanel Access Level:
    Root Administrator
    Stop hovering to collapse... Click to collapse... Hover to expand... Click to expand...
  3. cPanelMichael

    cPanelMichael Technical Support Community Manager
    Staff Member

    Joined:
    Apr 11, 2011
    Messages:
    44,803
    Likes Received:
    1,898
    Trophy Points:
    363
    cPanel Access Level:
    Root Administrator
    Twitter:
    Hello,

    The previous post is accurate. You can disable Apache Mod_Userdir via "WHM Home >> Security Center >> Apache mod_userdir Tweak" to ensure that URL is not accessible.

    Thank you.
     
    Stop hovering to collapse... Click to collapse... Hover to expand... Click to expand...
  4. trec-r

    trec-r Active Member

    Joined:
    Aug 10, 2008
    Messages:
    41
    Likes Received:
    2
    Trophy Points:
    58
    cPanel Access Level:
    Root Administrator
    Thank you for your replies. However, I do not want to turn off the userdir tweak.

    Got it working now. Updated robots.txt with the following:

    Disallow: /~*/

    Tested in Google Search Console. Looks like it requires the trailing / to work properly.
     
  5. cPanelMichael

    cPanelMichael Technical Support Community Manager
    Staff Member

    Joined:
    Apr 11, 2011
    Messages:
    44,803
    Likes Received:
    1,898
    Trophy Points:
    363
    cPanel Access Level:
    Root Administrator
    Twitter:
    I'm happy to see you were able to find a suitable entry for your robots.txt file. Thank you for updating this thread with the outcome.
     
    Stop hovering to collapse... Click to collapse... Hover to expand... Click to expand...
Loading...

Share This Page

  1. This site uses cookies to help personalise content, tailor your experience and to keep you logged in if you register.
    By continuing to use this site, you are consenting to our use of cookies.
    Dismiss Notice