Patrickintheforest

Registered
Aug 3, 2018
2
0
1
Essex
cPanel Access Level
Website Owner
In my robots.txt file I have denied Yandexbot
Under robots spider visitors, hits, I see for example
Yandexbot 0/2
Does this mean that the bot tried twice but was blocked.
If so I know my Yandexbot deny has worked
Patrick
 

cPanelMichael

Administrator
Staff member
Apr 11, 2011
47,913
2,205
363
Hello @Patrickintheforest,

The access attempt itself from the bot/crawler is still logged, and that's what Awstats is showing you. Note the robots.txt file won't technically prevent a crawler/bot from accessing your website files. There's some useful information on this topic at:

Block bots by specific urls with robots.txt in php apache environment

If you have root access to the server, it's sometimes a better idea to block the bot in your firewall rules. There's a thread on this topic at:

SOLVED - Blocking bad bots

Thank you.