Strange Bandwidth Issues

[Legin76]

On of my customers sites has had its bandwidth usage go up by six fold with no notable increase of bandwidth usage, but has not seen any increase in traffic or sales in the time since it went up. There are no notable large files and even awstats shows roughly the same bandwidth usage during the period.

Early last year it was using roughly 50gbs of http usage on average each month. Then between May and November last year it suddenly increased to about 80gbs. Half way though November it jumped again with a total of 140gbs for that month. Since December it has been around 260gbs but this month it looks like its going to get to 300gbs bandwidth. I know that its normal for sites to fluctuate a small amount, but the visitor numbers usually go up with it. I've looked though the site and there are no new large files added near those dates and awstats shows the monthly bandwidth at around 35gbs almost every month, the only exceptions being August 12 and April 13 where it has it just over 40gb. I know that awstats may not be 100% accurate but I would expect it's figures to go up too. Google Analytics also shows roughly consistent visitor numbers.

See the attached for screen shots.

 

[cPanelMichael]

Hello

The following document contains information that should be useful to you:

Bandwidth Discrepancies

Please review this and let us know if you have additional questions.

Thank you.

 

[Legin76]

Thank you but to be honest that did not help me much. I think I have discovered partially what is happening though.. It's something you may want to add to that document.

In awstats if you go to the individual month stats then it says "Not viewed traffic *" "* Not viewed traffic includes traffic generated by robots, worms, or replies with special HTTP status codes." This is excluded from the normal bandwidth data.

For June 2013 it shows 257.83 GB. In the Robots / Spiders section Unknown robot (identified by 'bot*') 6,359,741+991 hits 239.97 GB bandwidth. I found the following post relating to it http://forums.cpanel.net/f5/bandwidth-reporting-323082.html

I've still not worked out exactly what it is I need to do to find it and stop it but I'm sure I'll get it shortly.

 

[Legin76]

It appears that it may be bingbot. It appears to be loading about 30 pages at any one time.