Please whitelist cPanel in your adblocker so that you’re able to see our version release promotions, thanks!

The Community Forums

Interact with an entire community of cPanel & WHM users!
  1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Globally stop bad bots in Apache

Discussion in 'EasyApache' started by speckados, Jan 25, 2018.

Tags:
  1. speckados

    speckados Well-Known Member

    Joined:
    May 21, 2003
    Messages:
    311
    Likes Received:
    0
    Trophy Points:
    166
    Location:
    Pastrana :: Guadalajara :: España
    cPanel Access Level:
    DataCenter Provider
    Twitter:
    Usually use a tip on htaccess for stop badbots for some sites.

    But in last times, too many sites has same problem: too many bad boots with high traffic.

    What is best method for put on httpd.conf or for global sites ?

    Code:
    # Block Bad Bots & Scrapers
    SetEnvIfNoCase User-Agent "MJ12bot" bad_bot
    SetEnvIfNoCase User-Agent "pingdom" bad_bot
    SetEnvIfNoCase User-Agent "Aboundex" bad_bot
    SetEnvIfNoCase User-Agent "80legs" bad_bot
    SetEnvIfNoCase User-Agent "360Spider" bad_bot
    SetEnvIfNoCase User-Agent "^Java" bad_bot
    SetEnvIfNoCase User-Agent "^Cogentbot" bad_bot
    SetEnvIfNoCase User-Agent "^Alexibot" bad_bot
    SetEnvIfNoCase User-Agent "^asterias" bad_bot
    SetEnvIfNoCase User-Agent "^attach" bad_bot
    SetEnvIfNoCase User-Agent "^BackDoorBot" bad_bot
    SetEnvIfNoCase User-Agent "^BackWeb" bad_bot
    SetEnvIfNoCase User-Agent "Bandit" bad_bot
    SetEnvIfNoCase User-Agent "^BatchFTP" bad_bot
    SetEnvIfNoCase User-Agent "^Bigfoot" bad_bot
    SetEnvIfNoCase User-Agent "^Black.Hole" bad_bot
    SetEnvIfNoCase User-Agent "^BlackWidow" bad_bot
    SetEnvIfNoCase User-Agent "^BlowFish" bad_bot
    SetEnvIfNoCase User-Agent "^BotALot" bad_bot
    SetEnvIfNoCase User-Agent "Buddy" bad_bot
    SetEnvIfNoCase User-Agent "^BuiltBotTough" bad_bot
    SetEnvIfNoCase User-Agent "^Bullseye" bad_bot
    SetEnvIfNoCase User-Agent "^BunnySlippers" bad_bot
    SetEnvIfNoCase User-Agent "^Cegbfeieh" bad_bot
    SetEnvIfNoCase User-Agent "^CheeseBot" bad_bot
    SetEnvIfNoCase User-Agent "^CherryPicker" bad_bot
    SetEnvIfNoCase User-Agent "^ChinaClaw" bad_bot
    SetEnvIfNoCase User-Agent "Collector" bad_bot
    SetEnvIfNoCase User-Agent "Copier" bad_bot
    SetEnvIfNoCase User-Agent "^CopyRightCheck" bad_bot
    SetEnvIfNoCase User-Agent "^cosmos" bad_bot
    SetEnvIfNoCase User-Agent "^Crescent" bad_bot
    SetEnvIfNoCase User-Agent "^Custo" bad_bot
    SetEnvIfNoCase User-Agent "^AIBOT" bad_bot
    SetEnvIfNoCase User-Agent "^DISCo" bad_bot
    SetEnvIfNoCase User-Agent "^DIIbot" bad_bot
    SetEnvIfNoCase User-Agent "^DittoSpyder" bad_bot
    SetEnvIfNoCase User-Agent "^Download\ Demon" bad_bot
    SetEnvIfNoCase User-Agent "^Download\ Devil" bad_bot
    SetEnvIfNoCase User-Agent "^Download\ Wonder" bad_bot
    SetEnvIfNoCase User-Agent "^dragonfly" bad_bot
    SetEnvIfNoCase User-Agent "^Drip" bad_bot
    SetEnvIfNoCase User-Agent "^eCatch" bad_bot
    SetEnvIfNoCase User-Agent "^EasyDL" bad_bot
    SetEnvIfNoCase User-Agent "^ebingbong" bad_bot
    SetEnvIfNoCase User-Agent "^EirGrabber" bad_bot
    SetEnvIfNoCase User-Agent "^EmailCollector" bad_bot
    SetEnvIfNoCase User-Agent "^EmailSiphon" bad_bot
    SetEnvIfNoCase User-Agent "^EmailWolf" bad_bot
    SetEnvIfNoCase User-Agent "^EroCrawler" bad_bot
    SetEnvIfNoCase User-Agent "^Exabot" bad_bot
    SetEnvIfNoCase User-Agent "^Express\ WebPictures" bad_bot
    SetEnvIfNoCase User-Agent "Extractor" bad_bot
    SetEnvIfNoCase User-Agent "^EyeNetIE" bad_bot
    SetEnvIfNoCase User-Agent "^Foobot" bad_bot
    SetEnvIfNoCase User-Agent "^flunky" bad_bot
    SetEnvIfNoCase User-Agent "^FrontPage" bad_bot
    SetEnvIfNoCase User-Agent "^Go-Ahead-Got-It" bad_bot
    SetEnvIfNoCase User-Agent "^gotit" bad_bot
    SetEnvIfNoCase User-Agent "^GrabNet" bad_bot
    SetEnvIfNoCase User-Agent "^Grafula" bad_bot
    SetEnvIfNoCase User-Agent "^Harvest" bad_bot
    SetEnvIfNoCase User-Agent "^hloader" bad_bot
    SetEnvIfNoCase User-Agent "^HMView" bad_bot
    SetEnvIfNoCase User-Agent "^HTTrack" bad_bot
    SetEnvIfNoCase User-Agent "^humanlinks" bad_bot
    SetEnvIfNoCase User-Agent "^IlseBot" bad_bot
    SetEnvIfNoCase User-Agent "^Image\ Stripper" bad_bot
    SetEnvIfNoCase User-Agent "^Image\ Sucker" bad_bot
    SetEnvIfNoCase User-Agent "Indy\ Library" bad_bot
    SetEnvIfNoCase User-Agent "^InfoNaviRobot" bad_bot
    SetEnvIfNoCase User-Agent "^InfoTekies" bad_bot
    SetEnvIfNoCase User-Agent "^Intelliseek" bad_bot
    SetEnvIfNoCase User-Agent "^InterGET" bad_bot
    SetEnvIfNoCase User-Agent "^Internet\ Ninja" bad_bot
    SetEnvIfNoCase User-Agent "^Iria" bad_bot
    SetEnvIfNoCase User-Agent "^Jakarta" bad_bot
    SetEnvIfNoCase User-Agent "^JennyBot" bad_bot
    SetEnvIfNoCase User-Agent "^JetCar" bad_bot
    SetEnvIfNoCase User-Agent "^JOC" bad_bot
    SetEnvIfNoCase User-Agent "^JustView" bad_bot
    SetEnvIfNoCase User-Agent "^Jyxobot" bad_bot
    SetEnvIfNoCase User-Agent "^Kenjin.Spider" bad_bot
    SetEnvIfNoCase User-Agent "^Keyword.Density" bad_bot
    SetEnvIfNoCase User-Agent "^larbin" bad_bot
    SetEnvIfNoCase User-Agent "^LexiBot" bad_bot
    SetEnvIfNoCase User-Agent "^lftp" bad_bot
    SetEnvIfNoCase User-Agent "^libWeb/clsHTTP" bad_bot
    SetEnvIfNoCase User-Agent "^likse" bad_bot
    SetEnvIfNoCase User-Agent "^LinkextractorPro" bad_bot
    SetEnvIfNoCase User-Agent "^LinkScan/8.1a.Unix" bad_bot
    SetEnvIfNoCase User-Agent "^LNSpiderguy" bad_bot
    SetEnvIfNoCase User-Agent "^LinkWalker" bad_bot
    SetEnvIfNoCase User-Agent "^lwp-trivial" bad_bot
    SetEnvIfNoCase User-Agent "^LWP::Simple" bad_bot
    SetEnvIfNoCase User-Agent "^Magnet" bad_bot
    SetEnvIfNoCase User-Agent "^Mag-Net" bad_bot
    SetEnvIfNoCase User-Agent "^MarkWatch" bad_bot
    SetEnvIfNoCase User-Agent "^Mass\ Downloader" bad_bot
    SetEnvIfNoCase User-Agent "^Mata.Hari" bad_bot
    SetEnvIfNoCase User-Agent "^Memo" bad_bot
    SetEnvIfNoCase User-Agent "^Microsoft.URL" bad_bot
    SetEnvIfNoCase User-Agent "^Microsoft\ URL\ Control" bad_bot
    SetEnvIfNoCase User-Agent "^MIDown\ tool" bad_bot
    SetEnvIfNoCase User-Agent "^MIIxpc" bad_bot
    SetEnvIfNoCase User-Agent "^Mirror" bad_bot
    SetEnvIfNoCase User-Agent "^Missigua\ Locator" bad_bot
    SetEnvIfNoCase User-Agent "^Mister\ PiX" bad_bot
    SetEnvIfNoCase User-Agent "^moget" bad_bot
    SetEnvIfNoCase User-Agent "^Mozilla/3.Mozilla/2.01" bad_bot
    SetEnvIfNoCase User-Agent "^Mozilla.*NEWT" bad_bot
    SetEnvIfNoCase User-Agent "^NAMEPROTECT" bad_bot
    SetEnvIfNoCase User-Agent "^Navroad" bad_bot
    SetEnvIfNoCase User-Agent "^NearSite" bad_bot
    SetEnvIfNoCase User-Agent "^NetAnts" bad_bot
    SetEnvIfNoCase User-Agent "^Netcraft" bad_bot
    SetEnvIfNoCase User-Agent "^NetMechanic" bad_bot
    SetEnvIfNoCase User-Agent "^NetSpider" bad_bot
    SetEnvIfNoCase User-Agent "^Net\ Vampire" bad_bot
    SetEnvIfNoCase User-Agent "^NetZIP" bad_bot
    SetEnvIfNoCase User-Agent "^NextGenSearchBot" bad_bot
    SetEnvIfNoCase User-Agent "^NG" bad_bot
    SetEnvIfNoCase User-Agent "^NICErsPRO" bad_bot
    SetEnvIfNoCase User-Agent "^niki-bot" bad_bot
    SetEnvIfNoCase User-Agent "^NimbleCrawler" bad_bot
    SetEnvIfNoCase User-Agent "^Ninja" bad_bot
    SetEnvIfNoCase User-Agent "^NPbot" bad_bot
    SetEnvIfNoCase User-Agent "^Octopus" bad_bot
    SetEnvIfNoCase User-Agent "^Offline\ Explorer" bad_bot
    SetEnvIfNoCase User-Agent "^Offline\ Navigator" bad_bot
    SetEnvIfNoCase User-Agent "^Openfind" bad_bot
    SetEnvIfNoCase User-Agent "^OutfoxBot" bad_bot
    SetEnvIfNoCase User-Agent "^PageGrabber" bad_bot
    SetEnvIfNoCase User-Agent "^Papa\ Foto" bad_bot
    SetEnvIfNoCase User-Agent "^pavuk" bad_bot
    SetEnvIfNoCase User-Agent "^pcBrowser" bad_bot
    SetEnvIfNoCase User-Agent "^PHP\ version\ tracker" bad_bot
    SetEnvIfNoCase User-Agent "^Pockey" bad_bot
    SetEnvIfNoCase User-Agent "^ProPowerBot/2.14" bad_bot
    SetEnvIfNoCase User-Agent "^ProWebWalker" bad_bot
    SetEnvIfNoCase User-Agent "^psbot" bad_bot
    SetEnvIfNoCase User-Agent "^Pump" bad_bot
    SetEnvIfNoCase User-Agent "^QueryN.Metasearch" bad_bot
    SetEnvIfNoCase User-Agent "^RealDownload" bad_bot
    SetEnvIfNoCase User-Agent "Reaper" bad_bot
    SetEnvIfNoCase User-Agent "Recorder" bad_bot
    SetEnvIfNoCase User-Agent "^ReGet" bad_bot
    SetEnvIfNoCase User-Agent "^RepoMonkey" bad_bot
    SetEnvIfNoCase User-Agent "^RMA" bad_bot
    SetEnvIfNoCase User-Agent "Siphon" bad_bot
    SetEnvIfNoCase User-Agent "^SiteSnagger" bad_bot
    SetEnvIfNoCase User-Agent "^SlySearch" bad_bot
    SetEnvIfNoCase User-Agent "^SmartDownload" bad_bot
    SetEnvIfNoCase User-Agent "^Snake" bad_bot
    SetEnvIfNoCase User-Agent "^Snapbot" bad_bot
    SetEnvIfNoCase User-Agent "^Snoopy" bad_bot
    SetEnvIfNoCase User-Agent "^sogou" bad_bot
    SetEnvIfNoCase User-Agent "^SpaceBison" bad_bot
    SetEnvIfNoCase User-Agent "^SpankBot" bad_bot
    SetEnvIfNoCase User-Agent "^spanner" bad_bot
    SetEnvIfNoCase User-Agent "^Sqworm" bad_bot
    SetEnvIfNoCase User-Agent "Stripper" bad_bot
    SetEnvIfNoCase User-Agent "Sucker" bad_bot
    SetEnvIfNoCase User-Agent "^SuperBot" bad_bot
    SetEnvIfNoCase User-Agent "^SuperHTTP" bad_bot
    SetEnvIfNoCase User-Agent "^Surfbot" bad_bot
    SetEnvIfNoCase User-Agent "^suzuran" bad_bot
    SetEnvIfNoCase User-Agent "^Szukacz/1.4" bad_bot
    SetEnvIfNoCase User-Agent "^tAkeOut" bad_bot
    SetEnvIfNoCase User-Agent "^Teleport" bad_bot
    SetEnvIfNoCase User-Agent "^Telesoft" bad_bot
    SetEnvIfNoCase User-Agent "^TurnitinBot/1.5" bad_bot
    SetEnvIfNoCase User-Agent "^The.Intraformant" bad_bot
    SetEnvIfNoCase User-Agent "^TheNomad" bad_bot
    SetEnvIfNoCase User-Agent "^TightTwatBot" bad_bot
    SetEnvIfNoCase User-Agent "^Titan" bad_bot
    SetEnvIfNoCase User-Agent "^True_Robot" bad_bot
    SetEnvIfNoCase User-Agent "^turingos" bad_bot
    SetEnvIfNoCase User-Agent "^TurnitinBot" bad_bot
    SetEnvIfNoCase User-Agent "^URLy.Warning" bad_bot
    SetEnvIfNoCase User-Agent "^Vacuum" bad_bot
    SetEnvIfNoCase User-Agent "^VCI" bad_bot
    SetEnvIfNoCase User-Agent "^VoidEYE" bad_bot
    SetEnvIfNoCase User-Agent "^Web\ Image\ Collector" bad_bot
    SetEnvIfNoCase User-Agent "^Web\ Sucker" bad_bot
    SetEnvIfNoCase User-Agent "^WebAuto" bad_bot
    SetEnvIfNoCase User-Agent "^WebBandit" bad_bot
    SetEnvIfNoCase User-Agent "^Webclipping.com" bad_bot
    SetEnvIfNoCase User-Agent "^WebCopier" bad_bot
    SetEnvIfNoCase User-Agent "^WebEMailExtrac.*" bad_bot
    SetEnvIfNoCase User-Agent "^WebEnhancer" bad_bot
    SetEnvIfNoCase User-Agent "^WebFetch" bad_bot
    SetEnvIfNoCase User-Agent "^WebGo\ IS" bad_bot
    SetEnvIfNoCase User-Agent "^Web.Image.Collector" bad_bot
    SetEnvIfNoCase User-Agent "^WebLeacher" bad_bot
    SetEnvIfNoCase User-Agent "^WebmasterWorldForumBot" bad_bot
    SetEnvIfNoCase User-Agent "^WebReaper" bad_bot
    SetEnvIfNoCase User-Agent "^WebSauger" bad_bot
    SetEnvIfNoCase User-Agent "^Website\ eXtractor" bad_bot
    SetEnvIfNoCase User-Agent "^Website\ Quester" bad_bot
    SetEnvIfNoCase User-Agent "^Webster" bad_bot
    SetEnvIfNoCase User-Agent "^WebStripper" bad_bot
    SetEnvIfNoCase User-Agent "^WebWhacker" bad_bot
    SetEnvIfNoCase User-Agent "^WebZIP" bad_bot
    SetEnvIfNoCase User-Agent "Whacker" bad_bot
    SetEnvIfNoCase User-Agent "^Widow" bad_bot
    SetEnvIfNoCase User-Agent "^WISENutbot" bad_bot
    SetEnvIfNoCase User-Agent "^WWWOFFLE" bad_bot
    SetEnvIfNoCase User-Agent "^WWW-Collector-E" bad_bot
    SetEnvIfNoCase User-Agent "^Xaldon" bad_bot
    SetEnvIfNoCase User-Agent "^Xenu" bad_bot
    SetEnvIfNoCase User-Agent "^Zeus" bad_bot
    SetEnvIfNoCase User-Agent "ZmEu" bad_bot
    SetEnvIfNoCase User-Agent "^Zyborg" bad_bot
    # Vulnerability Scanners
    SetEnvIfNoCase User-Agent "Acunetix" bad_bot
    SetEnvIfNoCase User-Agent "FHscan" bad_bot
    # Aggressive Chinese Search Engine
    SetEnvIfNoCase User-Agent "Baiduspider" bad_bot
    # Aggressive Russian Search Engine
    SetEnvIfNoCase User-Agent "Yandex" bad_bot
    <Limit GET POST HEAD>
    Order Allow,Deny
    Allow from all
    # Cyveillance
    deny from 38.100.19.8/29
    deny from 38.100.21.0/24
    deny from 38.100.41.64/26
    deny from 38.105.71.0/25
    deny from 38.105.83.0/27
    deny from 38.112.21.140/30
    deny from 38.118.42.32/29
    deny from 65.213.208.128/27
    deny from 65.222.176.96/27
    deny from 65.222.185.72/29
    Deny from env=bad_bot
    </Limit>
     
  2. speckados

    speckados Well-Known Member

    Joined:
    May 21, 2003
    Messages:
    311
    Likes Received:
    0
    Trophy Points:
    166
    Location:
    Pastrana :: Guadalajara :: España
    cPanel Access Level:
    DataCenter Provider
    Twitter:
    Hi.

    According manual How to Create a ModSecurity Vendor

    Code:
    # mkdir tamainut001
    echo 'SecRule REQUEST_HEADERS:User-Agent "@pmFromFile /etc/httpd/conf.d/badbots.txt" \
    "id:444444,rev:1,severity:2,log,msg:'BAD BOT - Detected and Blocked. '"' > tamainut001/44_tamainut.conf
    
    # zip -r tamainut001.zip tamainut001
    # md5sum tamainut001.zip
    02e20c3e46431cff58b84137d801d4f0 tamainut001.zip

    put file on my server example.com/tamainut001.zip
    put file badbots.txt on Cpanel server with content

    Code:
    60Spider
    80legs
    AIBOT
    Aboundex
    Acunetix
    AhrefsBot
    Alexibot
    Anonymizer
    Attributor
    ...
    
    Edit meta_tamainut001.yaml
    Code:
    ---
    1.0.0:
      MD5: 7ce5b898da4e038cb70b38fe96a9e427
      distribution: tamainut-0
      url: https://example.com/tamainut001.zip
    attributes:
      description: 'Anti Bad Bots'
      name: 'Anti Bad Bots'
      vendor_url: https://example.com/
      report_url: https://example.com/intranet/submitticket.php
    Go to my WHM / ModSecuroty Vendors / Add Vendor
    Vendor Configuration URL

    https://example.com/meta_example.yaml

    Click on Load button.

    See information about Vendor... and click on save

    Code:
    Error:API failure: The vendor metadata does not contain an entry for your version of ModSecurity, “2.9.2”. The only version of ModSecurity this rule set supports is “1.0.0”.

    If use rule, for example putting an other vendor, such configserver, rule work (not block bad Apache restart, and log every accesos from bad bot)


    Some help please..
     
    #2 speckados, Jan 25, 2018
    Last edited by a moderator: Jan 25, 2018
  3. Tearabite

    Tearabite Well-Known Member

    Joined:
    Nov 28, 2010
    Messages:
    70
    Likes Received:
    11
    Trophy Points:
    58
    Location:
    Southern California
    cPanel Access Level:
    Root Administrator
    We use ModSecurity to block bad bots similar to what speckados posted. In addition to that we also use CSF to block the IP of the bad bot if it continues hitting a site.. In addition to that, if we continue to get bad bots from nearby IP's, CSF will then block the entire range (usually a nefarious hosting company).
     
  4. speckados

    speckados Well-Known Member

    Joined:
    May 21, 2003
    Messages:
    311
    Likes Received:
    0
    Trophy Points:
    166
    Location:
    Pastrana :: Guadalajara :: España
    cPanel Access Level:
    DataCenter Provider
    Twitter:
  5. Tearabite

    Tearabite Well-Known Member

    Joined:
    Nov 28, 2010
    Messages:
    70
    Likes Received:
    11
    Trophy Points:
    58
    Location:
    Southern California
    cPanel Access Level:
    Root Administrator
    Our rule is virtually the same - just different filenames and slightly different list of bots and we have set it up as a custom rule instead of a vendor configuration, so basically the same thing.
    We also block by known paths/vulnerability probes.
     
  6. cPanelMichael

    cPanelMichael Forums Analyst
    Staff Member

    Joined:
    Apr 11, 2011
    Messages:
    42,734
    Likes Received:
    1,706
    Trophy Points:
    363
    cPanel Access Level:
    Root Administrator
    Hello,

    You used the following entry in your metadata file:

    Code:
    1.0.0:
    This needs to be the version of Mod_Security installed on the server. EX:

    Code:
    2.9.2:
    Thank you.
     
  7. speckados

    speckados Well-Known Member

    Joined:
    May 21, 2003
    Messages:
    311
    Likes Received:
    0
    Trophy Points:
    166
    Location:
    Pastrana :: Guadalajara :: España
    cPanel Access Level:
    DataCenter Provider
    Twitter:
    Well, but not work now by MD5

    Code:
    API failure: The downloaded vendor archive does not match the expected digest (MD5 or SHA512).
    On meta_tamainut.yaml same MD5 sum
    Code:
    md5sum tamainut001.zip
    9a1896a0191******a515f3de39982550  tamainut001.zip
     
  8. speckados

    speckados Well-Known Member

    Joined:
    May 21, 2003
    Messages:
    311
    Likes Received:
    0
    Trophy Points:
    166
    Location:
    Pastrana :: Guadalajara :: España
    cPanel Access Level:
    DataCenter Provider
    Twitter:
    Wel..

    After try with SHA512 hash, work fine.


    Tahnks.
     
  9. cPanelMichael

    cPanelMichael Forums Analyst
    Staff Member

    Joined:
    Apr 11, 2011
    Messages:
    42,734
    Likes Received:
    1,706
    Trophy Points:
    363
    cPanel Access Level:
    Root Administrator
    I'm glad to see it's now working. This is noted in our documentation:

    Thank you.
     
  10. Nirjonadda

    Nirjonadda Well-Known Member

    Joined:
    May 8, 2013
    Messages:
    557
    Likes Received:
    14
    Trophy Points:
    18
    cPanel Access Level:
    Root Administrator
    Please Do post here the tutorial. How to block bad bot?
     
  11. Nirjonadda

    Nirjonadda Well-Known Member

    Joined:
    May 8, 2013
    Messages:
    557
    Likes Received:
    14
    Trophy Points:
    18
    cPanel Access Level:
    Root Administrator
    Does this correct the location of the place "Post VirtualHost Include" adding to block some bad bots and hosts?

    Code:
    # Start Block Bad Bots and Scrapers
    
    SetEnvIfNoCase User-Agent "MJ12bot" bad_bot
    SetEnvIfNoCase User-Agent "pingdom" bad_bot
    SetEnvIfNoCase User-Agent "Aboundex" bad_bot
    SetEnvIfNoCase User-Agent "80legs" bad_bot
    SetEnvIfNoCase User-Agent "360Spider" bad_bot
    SetEnvIfNoCase User-Agent "^Java" bad_bot
    SetEnvIfNoCase User-Agent "^Cogentbot" bad_bot
    SetEnvIfNoCase User-Agent "^Alexibot" bad_bot
    SetEnvIfNoCase User-Agent "^asterias" bad_bot
    SetEnvIfNoCase User-Agent "^attach" bad_bot
    SetEnvIfNoCase User-Agent "^BackDoorBot" bad_bot
    SetEnvIfNoCase User-Agent "^BackWeb" bad_bot
    SetEnvIfNoCase User-Agent "Bandit" bad_bot
    SetEnvIfNoCase User-Agent "^BatchFTP" bad_bot
    SetEnvIfNoCase User-Agent "^Bigfoot" bad_bot
    SetEnvIfNoCase User-Agent "^Black.Hole" bad_bot
    SetEnvIfNoCase User-Agent "^BlackWidow" bad_bot
    SetEnvIfNoCase User-Agent "^BlowFish" bad_bot
    SetEnvIfNoCase User-Agent "^BotALot" bad_bot
    SetEnvIfNoCase User-Agent "Buddy" bad_bot
    SetEnvIfNoCase User-Agent "^BuiltBotTough" bad_bot
    SetEnvIfNoCase User-Agent "^Bullseye" bad_bot
    SetEnvIfNoCase User-Agent "^BunnySlippers" bad_bot
    SetEnvIfNoCase User-Agent "^Cegbfeieh" bad_bot
    SetEnvIfNoCase User-Agent "^CheeseBot" bad_bot
    SetEnvIfNoCase User-Agent "^CherryPicker" bad_bot
    SetEnvIfNoCase User-Agent "^ChinaClaw" bad_bot
    SetEnvIfNoCase User-Agent "Collector" bad_bot
    SetEnvIfNoCase User-Agent "Copier" bad_bot
    SetEnvIfNoCase User-Agent "^CopyRightCheck" bad_bot
    SetEnvIfNoCase User-Agent "^cosmos" bad_bot
    SetEnvIfNoCase User-Agent "^Crescent" bad_bot
    SetEnvIfNoCase User-Agent "^Custo" bad_bot
    SetEnvIfNoCase User-Agent "^AIBOT" bad_bot
    SetEnvIfNoCase User-Agent "^DISCo" bad_bot
    SetEnvIfNoCase User-Agent "^DIIbot" bad_bot
    SetEnvIfNoCase User-Agent "^DittoSpyder" bad_bot
    SetEnvIfNoCase User-Agent "^Download\ Demon" bad_bot
    SetEnvIfNoCase User-Agent "^Download\ Devil" bad_bot
    SetEnvIfNoCase User-Agent "^Download\ Wonder" bad_bot
    SetEnvIfNoCase User-Agent "^dragonfly" bad_bot
    SetEnvIfNoCase User-Agent "^Drip" bad_bot
    SetEnvIfNoCase User-Agent "^eCatch" bad_bot
    SetEnvIfNoCase User-Agent "^EasyDL" bad_bot
    SetEnvIfNoCase User-Agent "^ebingbong" bad_bot
    SetEnvIfNoCase User-Agent "^EirGrabber" bad_bot
    SetEnvIfNoCase User-Agent "^EmailCollector" bad_bot
    SetEnvIfNoCase User-Agent "^EmailSiphon" bad_bot
    SetEnvIfNoCase User-Agent "^EmailWolf" bad_bot
    SetEnvIfNoCase User-Agent "^EroCrawler" bad_bot
    SetEnvIfNoCase User-Agent "^Exabot" bad_bot
    SetEnvIfNoCase User-Agent "^Express\ WebPictures" bad_bot
    SetEnvIfNoCase User-Agent "Extractor" bad_bot
    SetEnvIfNoCase User-Agent "^EyeNetIE" bad_bot
    SetEnvIfNoCase User-Agent "^Foobot" bad_bot
    SetEnvIfNoCase User-Agent "^flunky" bad_bot
    SetEnvIfNoCase User-Agent "^FrontPage" bad_bot
    SetEnvIfNoCase User-Agent "^Go-Ahead-Got-It" bad_bot
    SetEnvIfNoCase User-Agent "^gotit" bad_bot
    SetEnvIfNoCase User-Agent "^GrabNet" bad_bot
    SetEnvIfNoCase User-Agent "^Grafula" bad_bot
    SetEnvIfNoCase User-Agent "^Harvest" bad_bot
    SetEnvIfNoCase User-Agent "^hloader" bad_bot
    SetEnvIfNoCase User-Agent "^HMView" bad_bot
    SetEnvIfNoCase User-Agent "^HTTrack" bad_bot
    SetEnvIfNoCase User-Agent "^humanlinks" bad_bot
    SetEnvIfNoCase User-Agent "^IlseBot" bad_bot
    SetEnvIfNoCase User-Agent "^Image\ Stripper" bad_bot
    SetEnvIfNoCase User-Agent "^Image\ Sucker" bad_bot
    SetEnvIfNoCase User-Agent "Indy\ Library" bad_bot
    SetEnvIfNoCase User-Agent "^InfoNaviRobot" bad_bot
    SetEnvIfNoCase User-Agent "^InfoTekies" bad_bot
    SetEnvIfNoCase User-Agent "^Intelliseek" bad_bot
    SetEnvIfNoCase User-Agent "^InterGET" bad_bot
    SetEnvIfNoCase User-Agent "^Internet\ Ninja" bad_bot
    SetEnvIfNoCase User-Agent "^Iria" bad_bot
    SetEnvIfNoCase User-Agent "^Jakarta" bad_bot
    SetEnvIfNoCase User-Agent "^JennyBot" bad_bot
    SetEnvIfNoCase User-Agent "^JetCar" bad_bot
    SetEnvIfNoCase User-Agent "^JOC" bad_bot
    SetEnvIfNoCase User-Agent "^JustView" bad_bot
    SetEnvIfNoCase User-Agent "^Jyxobot" bad_bot
    SetEnvIfNoCase User-Agent "^Kenjin.Spider" bad_bot
    SetEnvIfNoCase User-Agent "^Keyword.Density" bad_bot
    SetEnvIfNoCase User-Agent "^larbin" bad_bot
    SetEnvIfNoCase User-Agent "^LexiBot" bad_bot
    SetEnvIfNoCase User-Agent "^lftp" bad_bot
    SetEnvIfNoCase User-Agent "^libWeb/clsHTTP" bad_bot
    SetEnvIfNoCase User-Agent "^likse" bad_bot
    SetEnvIfNoCase User-Agent "^LinkextractorPro" bad_bot
    SetEnvIfNoCase User-Agent "^LinkScan/8.1a.Unix" bad_bot
    SetEnvIfNoCase User-Agent "^LNSpiderguy" bad_bot
    SetEnvIfNoCase User-Agent "^LinkWalker" bad_bot
    SetEnvIfNoCase User-Agent "^lwp-trivial" bad_bot
    SetEnvIfNoCase User-Agent "^LWP::Simple" bad_bot
    SetEnvIfNoCase User-Agent "^Magnet" bad_bot
    SetEnvIfNoCase User-Agent "^Mag-Net" bad_bot
    SetEnvIfNoCase User-Agent "^MarkWatch" bad_bot
    SetEnvIfNoCase User-Agent "^Mass\ Downloader" bad_bot
    SetEnvIfNoCase User-Agent "^Mata.Hari" bad_bot
    SetEnvIfNoCase User-Agent "^Memo" bad_bot
    SetEnvIfNoCase User-Agent "^Microsoft.URL" bad_bot
    SetEnvIfNoCase User-Agent "^Microsoft\ URL\ Control" bad_bot
    SetEnvIfNoCase User-Agent "^MIDown\ tool" bad_bot
    SetEnvIfNoCase User-Agent "^MIIxpc" bad_bot
    SetEnvIfNoCase User-Agent "^Mirror" bad_bot
    SetEnvIfNoCase User-Agent "^Missigua\ Locator" bad_bot
    SetEnvIfNoCase User-Agent "^Mister\ PiX" bad_bot
    SetEnvIfNoCase User-Agent "^moget" bad_bot
    SetEnvIfNoCase User-Agent "^Mozilla/3.Mozilla/2.01" bad_bot
    SetEnvIfNoCase User-Agent "^Mozilla.*NEWT" bad_bot
    SetEnvIfNoCase User-Agent "^NAMEPROTECT" bad_bot
    SetEnvIfNoCase User-Agent "^Navroad" bad_bot
    SetEnvIfNoCase User-Agent "^NearSite" bad_bot
    SetEnvIfNoCase User-Agent "^NetAnts" bad_bot
    SetEnvIfNoCase User-Agent "^Netcraft" bad_bot
    SetEnvIfNoCase User-Agent "^NetMechanic" bad_bot
    SetEnvIfNoCase User-Agent "^NetSpider" bad_bot
    SetEnvIfNoCase User-Agent "^Net\ Vampire" bad_bot
    SetEnvIfNoCase User-Agent "^NetZIP" bad_bot
    SetEnvIfNoCase User-Agent "^NextGenSearchBot" bad_bot
    SetEnvIfNoCase User-Agent "^NG" bad_bot
    SetEnvIfNoCase User-Agent "^NICErsPRO" bad_bot
    SetEnvIfNoCase User-Agent "^niki-bot" bad_bot
    SetEnvIfNoCase User-Agent "^NimbleCrawler" bad_bot
    SetEnvIfNoCase User-Agent "^Ninja" bad_bot
    SetEnvIfNoCase User-Agent "^NPbot" bad_bot
    SetEnvIfNoCase User-Agent "^Octopus" bad_bot
    SetEnvIfNoCase User-Agent "^Offline\ Explorer" bad_bot
    SetEnvIfNoCase User-Agent "^Offline\ Navigator" bad_bot
    SetEnvIfNoCase User-Agent "^Openfind" bad_bot
    SetEnvIfNoCase User-Agent "^OutfoxBot" bad_bot
    SetEnvIfNoCase User-Agent "^PageGrabber" bad_bot
    SetEnvIfNoCase User-Agent "^Papa\ Foto" bad_bot
    SetEnvIfNoCase User-Agent "^pavuk" bad_bot
    SetEnvIfNoCase User-Agent "^pcBrowser" bad_bot
    SetEnvIfNoCase User-Agent "^PHP\ version\ tracker" bad_bot
    SetEnvIfNoCase User-Agent "^Pockey" bad_bot
    SetEnvIfNoCase User-Agent "^ProPowerBot/2.14" bad_bot
    SetEnvIfNoCase User-Agent "^ProWebWalker" bad_bot
    SetEnvIfNoCase User-Agent "^psbot" bad_bot
    SetEnvIfNoCase User-Agent "^Pump" bad_bot
    SetEnvIfNoCase User-Agent "^QueryN.Metasearch" bad_bot
    SetEnvIfNoCase User-Agent "^RealDownload" bad_bot
    SetEnvIfNoCase User-Agent "Reaper" bad_bot
    SetEnvIfNoCase User-Agent "Recorder" bad_bot
    SetEnvIfNoCase User-Agent "^ReGet" bad_bot
    SetEnvIfNoCase User-Agent "^RepoMonkey" bad_bot
    SetEnvIfNoCase User-Agent "^RMA" bad_bot
    SetEnvIfNoCase User-Agent "Siphon" bad_bot
    SetEnvIfNoCase User-Agent "^SiteSnagger" bad_bot
    SetEnvIfNoCase User-Agent "^SlySearch" bad_bot
    SetEnvIfNoCase User-Agent "^SmartDownload" bad_bot
    SetEnvIfNoCase User-Agent "^Snake" bad_bot
    SetEnvIfNoCase User-Agent "^Snapbot" bad_bot
    SetEnvIfNoCase User-Agent "^Snoopy" bad_bot
    SetEnvIfNoCase User-Agent "^sogou" bad_bot
    SetEnvIfNoCase User-Agent "^SpaceBison" bad_bot
    SetEnvIfNoCase User-Agent "^SpankBot" bad_bot
    SetEnvIfNoCase User-Agent "^spanner" bad_bot
    SetEnvIfNoCase User-Agent "^Sqworm" bad_bot
    SetEnvIfNoCase User-Agent "Stripper" bad_bot
    SetEnvIfNoCase User-Agent "Sucker" bad_bot
    SetEnvIfNoCase User-Agent "^SuperBot" bad_bot
    SetEnvIfNoCase User-Agent "^SuperHTTP" bad_bot
    SetEnvIfNoCase User-Agent "^Surfbot" bad_bot
    SetEnvIfNoCase User-Agent "^suzuran" bad_bot
    SetEnvIfNoCase User-Agent "^Szukacz/1.4" bad_bot
    SetEnvIfNoCase User-Agent "^tAkeOut" bad_bot
    SetEnvIfNoCase User-Agent "^Teleport" bad_bot
    SetEnvIfNoCase User-Agent "^Telesoft" bad_bot
    SetEnvIfNoCase User-Agent "^TurnitinBot/1.5" bad_bot
    SetEnvIfNoCase User-Agent "^The.Intraformant" bad_bot
    SetEnvIfNoCase User-Agent "^TheNomad" bad_bot
    SetEnvIfNoCase User-Agent "^TightTwatBot" bad_bot
    SetEnvIfNoCase User-Agent "^Titan" bad_bot
    SetEnvIfNoCase User-Agent "^True_Robot" bad_bot
    SetEnvIfNoCase User-Agent "^turingos" bad_bot
    SetEnvIfNoCase User-Agent "^TurnitinBot" bad_bot
    SetEnvIfNoCase User-Agent "^URLy.Warning" bad_bot
    SetEnvIfNoCase User-Agent "^Vacuum" bad_bot
    SetEnvIfNoCase User-Agent "^VCI" bad_bot
    SetEnvIfNoCase User-Agent "^VoidEYE" bad_bot
    SetEnvIfNoCase User-Agent "^Web\ Image\ Collector" bad_bot
    SetEnvIfNoCase User-Agent "^Web\ Sucker" bad_bot
    SetEnvIfNoCase User-Agent "^WebAuto" bad_bot
    SetEnvIfNoCase User-Agent "^WebBandit" bad_bot
    SetEnvIfNoCase User-Agent "^Webclipping.com" bad_bot
    SetEnvIfNoCase User-Agent "^WebCopier" bad_bot
    SetEnvIfNoCase User-Agent "^WebEMailExtrac.*" bad_bot
    SetEnvIfNoCase User-Agent "^WebEnhancer" bad_bot
    SetEnvIfNoCase User-Agent "^WebFetch" bad_bot
    SetEnvIfNoCase User-Agent "^WebGo\ IS" bad_bot
    SetEnvIfNoCase User-Agent "^Web.Image.Collector" bad_bot
    SetEnvIfNoCase User-Agent "^WebLeacher" bad_bot
    SetEnvIfNoCase User-Agent "^WebmasterWorldForumBot" bad_bot
    SetEnvIfNoCase User-Agent "^WebReaper" bad_bot
    SetEnvIfNoCase User-Agent "^WebSauger" bad_bot
    SetEnvIfNoCase User-Agent "^Website\ eXtractor" bad_bot
    SetEnvIfNoCase User-Agent "^Website\ Quester" bad_bot
    SetEnvIfNoCase User-Agent "^Webster" bad_bot
    SetEnvIfNoCase User-Agent "^WebStripper" bad_bot
    SetEnvIfNoCase User-Agent "^WebWhacker" bad_bot
    SetEnvIfNoCase User-Agent "^WebZIP" bad_bot
    SetEnvIfNoCase User-Agent "Whacker" bad_bot
    SetEnvIfNoCase User-Agent "^Widow" bad_bot
    SetEnvIfNoCase User-Agent "^WISENutbot" bad_bot
    SetEnvIfNoCase User-Agent "^WWWOFFLE" bad_bot
    SetEnvIfNoCase User-Agent "^WWW-Collector-E" bad_bot
    SetEnvIfNoCase User-Agent "^Xaldon" bad_bot
    SetEnvIfNoCase User-Agent "^Xenu" bad_bot
    SetEnvIfNoCase User-Agent "^Zeus" bad_bot
    SetEnvIfNoCase User-Agent "ZmEu" bad_bot
    SetEnvIfNoCase User-Agent "^Zyborg" bad_bot
    
    # Vulnerability Scanners
    SetEnvIfNoCase User-Agent "Acunetix" bad_bot
    SetEnvIfNoCase User-Agent "FHscan" bad_bot
    
    # Aggressive Chinese Search Engine
    SetEnvIfNoCase User-Agent "Baiduspider" bad_bot
    
    # Aggressive Russian Search Engine
    SetEnvIfNoCase User-Agent "Yandex" bad_bot
    
    # End Block Bad Bots and Scrapers
     
    #11 Nirjonadda, Jan 27, 2018
    Last edited: Jan 27, 2018
  12. cPanelMichael

    cPanelMichael Forums Analyst
    Staff Member

    Joined:
    Apr 11, 2011
    Messages:
    42,734
    Likes Received:
    1,706
    Trophy Points:
    363
    cPanel Access Level:
    Root Administrator
Loading...

Share This Page