Logesh K

Member
Jun 25, 2015
11
0
1
Namakkal, TamilNadu, India
cPanel Access Level
Root Administrator
My server was on High load and high outgoing bandwidth. Upon checking with the iftop tool I found lot of IP's with hostname of the Bots - semrush, ahreafs, baidu were crawling the websites on my server. But I am unable to find out which account/domain was crawled by these bots.

Can someone help me in this?

Thanks
 

cPanelLauren

Forums Analyst II
Staff member
Nov 14, 2017
6,777
536
263
Houston
cPanel Access Level
DataCenter Provider
Hi @Logesh K

The domain access logs should give you this data, they can be found at /etc/apache2/domlogs/ for each domain.
 

Logesh K

Member
Jun 25, 2015
11
0
1
Namakkal, TamilNadu, India
cPanel Access Level
Root Administrator
Hi @Logesh K

The domain access logs should give you this data, they can be found at /etc/apache2/domlogs/ for each domain.
Thanks for the reply. But I don't find it easy to check the numerous files manually to find out the domain. Is there any way to find realtime data on this? Because once my server freezed due to ahrefs and semrush bots, while I was watching the bandwidth data using iftop.
 

Henry Carter

Member
Jul 31, 2019
18
0
1
India
cPanel Access Level
Website Owner
Hello Logesh,

You can hit the following command on terminal as a root, it will show up the top utilizing domain access logs. cd /var/log/apache2/domlogs && ll -S | head You can view Bandwidth utilization through WHM >> Home >> Account Information >> View Bandwidth Usage, hit "Xfer(Best Fit)" on the top to sort the usage list. Use "iotop" command, it shows a table of current I/O usage by processes or threads.