The Community Forums

Interact with an entire community of cPanel & WHM users!
  1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

How to disallow some crawler globally in httpd conf?

Discussion in 'General Discussion' started by postcd, May 27, 2014.

  1. postcd

    postcd Well-Known Member

    Joined:
    Oct 22, 2010
    Messages:
    621
    Likes Received:
    6
    Trophy Points:
    18
    Hello,

    if i want to disable some web crawler globally on all existing and future accounts, how to do it persistently?

    example this can be added to .htaccess but i need persistent global file of httpd ?

    I found Home » Service Configuration » Apache Configuration » Include Editor
    there are 3 include files, not sure if i can use this and add mentioned thing? persistently?

    Also what if i want to add some robots.txt rule globally?
    like:
     
    #1 postcd, May 27, 2014
    Last edited: May 27, 2014
  2. cPanelMichael

    cPanelMichael Forums Analyst
    Staff Member

    Joined:
    Apr 11, 2011
    Messages:
    30,808
    Likes Received:
    667
    Trophy Points:
    113
    cPanel Access Level:
    Root Administrator
Loading...
Similar Threads - disallow crawler globally
  1. akust0m
    Replies:
    1
    Views:
    282

Share This Page