Redirect referrer spam, without awstats counting it as visit

walt

Member
Oct 30, 2015
14
0
1
Houston, Tx
cPanel Access Level
Website Owner
Hello, I am trying to limit referrer spam from urls that look like the first line below. The referrals I am interested in are mostly like the second line:
Code:
http://referralspam.ru
http://www.domain.com/viewtopic.php
My site is on a shared hosting account, so my tools are limited to mod_rewrite in the htaccess file. I tried adding the following mod-rewrite rule to catch any referrer url with
http//(anything except '/').(1-4 alphabetic chars, may be followed by '/'):
Code:
RewriteCond %{HTTP_REFERER} ^http://[^/]*\.[a-zA-Z]{1,4}/?$ [NC]
RewriteRule .* /system/services/referrer/filter.php?ref=%{HTTP_REFERER} [L]
The filter.php file records the referrer information in a log file, so I can check if any legitimate request might have been caught. I can then modify the htaccess rules to make the necessary corrections. The file also displays a page with an apology, and a button that the visitor can press to enter the site.

I have a couple of issues though, which I was hoping to get some advice on.

Is there a way to redirect referrer spam, and log it, in such a way that Awstats will not count it as a visitor?

Also, the mod_rewrite rule does not seem to be catching anything. I still see the same types of referrals in my Awstats, and my log file stays empty. However after testing my rule from a Google link, it worked fine, including logging the referrer (I changed my rule to detect 'https', then changed it back):
Code:
2016-01-06 21:36:29 Referrer redirected by htaccess: https://www.google.com
Here is the code for my filter.php:
Code:
<?php
// file: filter.php
// prmissions: 644
$no_error = @ require($_SERVER['DOCUMENT_ROOT']."/../system/services/referrer/filter.inc");
?>
Code:
<?php
// File: filter.inc
define ('HOME_FOLDER', $_SERVER['DOCUMENT_ROOT']."/../system/services/referrer/");
define ('LOGFILE', HOME_FOLDER.'referrer.log');

$backgrImg = "/system/services/referrer/g3sf15.gif";
$textColorNormal = "#ffd700";
$link = "http://www.mywebsite.com";

if(isset($_GET['ref'])) {
  $referrer =  "Referrer redirected by htaccess: ".strip_tags(trim($_GET['ref']));
}
else if(isset($_SERVER['HTTP_REFERER'])){
  $referrer = "Referrer not redirected by htaccess: ".strip_tags(trim($_SERVER['HTTP_REFERER']));
}
else{
  $referrer = "Direct access from: ".strip_tags(trim($_SERVER['REMOTE_ADDR']));
}
$time = date('Y-m-d H:i:s', $_SERVER['REQUEST_TIME']);
$str = $time." ".$referrer.PHP_EOL;
error_log($str, 3, LOGFILE); // append to log
?>
<!DOCTYPE html>

<html>
<head>
<meta charset="UTF-8">
<meta name=viewport content="width=device-width, initial-scale=1">
<title>Referral filter</title>
<style>
body {
  color: <?php echo $textColorNormal ?>;
  font-size: 100%;
  font-weight: bold;
}
ul{
  list-style-type: none;
}
.button_container a {
  border-bottom: 1px solid #777777;
  border-left: 1px solid #000000;
  border-right: 1px solid #333333;
  border-top: 1px solid #000000;
  background-color: gray;
  color: <?php echo $textColorNormal ?>;
  height: 1em;
  padding: 0.2em;
  width: 2em;
  text-decoration: none;
}
</style>
</head>

<body background="<?php echo $backgrImg ?>" >

<div id="google_translate_element"></div>

<h2>
<ul>
<li>Apologies, the site that directed you here may have</li>
<li>been accidentally flagged as a bad site.</li>
<li><p></p></li>
<li>Please click on the button to enter.<li>
<li><p></p></li>
<li><div class="button_container"><a href="<?php echo $link ?>">Enter</a></div></li>
</ul>
</h2>

<script type="text/javascript">
  function googleTranslateElementInit() {
   new google.translate.TranslateElement({pageLanguage: 'en', layout:
   google.translate.TranslateElement.InlineLayout.SIMPLE}, 'google_translate_element');
  }
</script>
<script type="text/javascript"
   src="//translate.google.com/translate_a/element.js?cb=googleTranslateElementInit"></script>

</body>
</html>
UPDATE:
I cought a little bugger!
Code:
2016-01-06 21:51:06 Referrer redirected by htaccess: http://vesta-lada.net/
2016-01-06 21:51:06 Referrer redirected by htaccess: http://vesta-lada.net/
2016-01-06 21:51:07 Referrer redirected by htaccess: http://vesta-lada.net/
Not quite sure when it started to work, but I remember the last thing that was added to the mod_rewrite rule was the ending "/?".

However, these visits are still recorded in Awstats 'Links from an external page'. Are they counted in the stats for unique visitor? If so, I guess the original question still remains. Is there anything I can tweak in the Awstats configuration file? I have an awstats.mydomain.conf file in folder home/tmp/awstats.

It crossed my mind to change the rewrite rule to 'Fail, but redirect to this page first':
Code:
RewriteRule .* /system/services/referrer/filter.php?ref=%{HTTP_REFERER} [F, L]
It seemed like too dumb of an idea to even try.

UPDATE:
Apologies I was not sure if this should be posted as a different question.

The variable %{HTTP_REFERER} can return two different strings for the same domain:
Code:
http://www.domain_name.com
http://domain_name.com
Why the difference?

Thanks!
 
Last edited:

cPanelMichael

Administrator
Staff member
Apr 11, 2011
47,910
2,212
363
Hello :)

You can browse to "WHM Home >> Server Configuration >> Statistics Software Configuration" and enable "Allow Awstats configuration Include", as documented at:

Generators Configuration - Documentation - cPanel Documentation

Include a custom AWStats configuration file
If you wish to customize a user's configuration of AWStats:

  • Select Allow Awstats configuration Include file.
  • Add the configuration file to ~/tmp/awstats/awstats.conf.include
You may also want to review the Awstats documentation if you do not receive additional user-feedback on the custom configuration options:

AWStats - Free log file analyzer for advanced statistics (GNU GPL).

Thank you.