Sick of the AOL email forwarding problem? Try this add-on.

[electric]

The Problem

As you all know, if any user on your server has set up email forwarding to an AOL account... AOL will blacklist your server if the AOL user then tags any of the forwarded emails as spam. (AOL considers the email as spam, because it originated from your server.)



There is currently a feature request in bugzilla which would see cpanel implement the ability to NOT allow @aol forwarders to be created... (vote for it!).. but in the meantime, we need a way to easily find these nasty forwarders so we can deal with them.

The Solution

Attached to this post is a very simple WHM add-on called "AOL Forwarder Finder". This add-on will give you a nice list of any forwarders on your server that are sending email to AOL, the responsible domain, and also the reseller's domain.



** Special thanks to sparek-3, who wrote the code to actually find the @aol addresses, etc. All I did was change a few things, and then make it accessible as a WHM add-on.

The idea is that the list of forwarders can be used as a source into some other system which will send an email/warning to the appropriate users. Or, you can deal with each forwarder manually. Or, if you want to get tricky, you could modify the script to automatically notify the appropriate user and then remove the forwarder, etc...

If you make any changes to this script, please let me know and I'll update the copy here, so that it's in one easy-to-find place.

##----------------------------------------------------
## Usage:
##
## 1) Copy to "/usr/local/cpanel/whostmgr/docroot/cgi/" directory.
## 2) Rename to "addon_aolforwarderfinder.cgi".
## 3) Chown to root:root.
## 4) Chmod to 755.
## 5) Log into WHM as user with root privs and click the link
## called "AOL Forwarder Finder" under the Add-ons section.
##----------------------------------------------------

Please let me know if you find any bugs or have any suggestions/improvements/comments.

 

[celliott]

Sounds promising, will give this a go later on.

Thanks

 

[noox]

Thanks! I installed it and it worked perfectly.

Does anybody know how long it takes till AOL allows mails from this server again? Do I have to ask them to remove my server from their deny list?

About two years ago I noticed that AOL rejects my mails. I signed in for this AOL service where they send you the mails they consider as spam from your server. Thus I found out that it's a forwarder which got spammed.

When I tried this again the last months I did not get any response from AOL.

Thanks,
Hannes

 

[celliott]

I now have this installed and it has been useful already

 

[lloyd_tennison]

Most of the problem is because the server is forwarding SPAM. Stop the SPAM from reaching your server and it solves the problem. The SURBL list solves most of those problems. FYI - I always had far more trouble with Yahoo that AOL.

 

[noox]

I think it's not a good practice to filter the spam from your customers.

There have been law suits in Germany because a employee did not get a mail a admin has filtered out. Imagine somebody misses a mail of big business importance.

I don't think that anybody is able to catch exactly that spam that AOL or Yahoo considers as spam while not filtering any non-spam.

In Europe we have the advantage that allmost nobody has an AOL (or Yahoo) account. I'm here in Austria and I don't know anybody with an AOL account. On my discussion forums only a handful users from germany have an AOL account.

So not allowing forwarders to AOL (or Yahoo) would be best practice for me.

 

[lloyd_tennison]

I would agree that not filtering SPAM is an ideal solution - but this is the real world. I block over 2 million messages a day one one server. Imagine if I processed all that garbage!

A "real" business email would have received a proper bounce message to a live person, so either the server in the case you mentioned was not properly configured - or just a bogus lawsuit (there are certainly are more than a few of them :D .)

 

[sparek-3]

It should be noted that this is really not so much of an issue with AOL as it is with end users and forwarding their e-mail. The anti-spam tactics of the large e-mail providers is changing to where they are blocking hosts that are sending spam. This is unfortunate, but there's really nothing that the e-mail provider can do in regards to this. As much as I despise AOL's method of blocking hosts, I cannot put all of the blame on them. The issue itself is not just bound to AOL, it is tied to all remote mail services that users may be forwarding mail to (i.e. hotmail.com, yahoo.com, verizon.net, comcast.net, etc). It just so happens that AOL seems to have a much larger user base and their tactics point more to just blocking an IP, whereas some of the other providers simply direct messages from a "known" spam source into their junk box.

It is becoming more and more of an issue and really it is the end users who are going to need to change. If an end user is wanting to receive all of their e-mail at their aol.com address or their yahoo.com address, then they should post that e-mail address on their website or business cards. They do not need to be forwarding mail from their domain name to these addresses, because of the issue with spam. If the users want to use an @theirdomain.com e-mail address then they should set it up as a POP or IMAP account and use a POP or IMAP client to check these messages. Otherwise, spammers will continue to send spam to those addresses, which will be forwarded off of your server and to another service, where that service will detect the spam message as coming from your server. Just because Yahoo or Hotmail has not blocked your server yet, does not mean that they won't.

If a user wishes to forward their mail off of the server, then they need to insure that the host they are forwarding those messages to will always accept those messages. Otherwise, you run the risk of clogging the mail queue on your server. This is something that I don't think most mail server administrators will agree to do, and I cannot blame them. If end users want to receive e-mail at their domain name, the only way they can be assured of receiving all of their messages (assuming you do not have any RBLs or other anti-spam measures in place on your server, something that you would control) then they would be better served to use a POP or IMAP account.

I would like to see an enhancement (and I know I should submit this as an enhancement request, but I'm not sure how well it would be adopted) where you can not only block users from forwarding mail to AOL, but block users from forwarding their mail to any outside e-mail address. Instead just making the e-mail forwarders an internal option (forward mail between accounts on the user's account). Perhaps have it to where server administrators could set up forwarders for specific users who are assured that the server they are forwarding the messages to, will accept those messages. This may seem like a harsh treatment, but its the only way I can see where you can keep your server free of any blocking from other services because of spam forwarded e-mails.

 

[jcsolutions]

Excellent script. Thank you very much!

 

[blweb]

Yahoo Modification.

For what it's worth I've changed the code a bit to reflect yahoo in case anybody has a need for it. Thanks again for the great script!

#!/usr/local/bin/perl

#WHMADDON:yahooforwardfinder:Yahoo Forwarder Finder

##----------------------------------------------------
## Version 1.0
##
## Usage:
##
##   1) Copy to "/usr/local/cpanel/whostmgr/docroot/cgi/" directory.
##   2) Rename to "addon_yahooforwarderfinder.cgi".
##   3) Chown to root:root.
##   4) Chmod to 755.
##   5) Log into WHM as user with root privs and click the link 
##      called "AOL Forwarder Finder" under the Add-ons section.
##----------------------------------------------------

BEGIN {
   push(@INC,'/usr/local/cpanel','/usr/local/cpanel/Cpanel');
   push(@INC,"/usr/local/cpanel/whostmgr/docroot/cgi");
}

use whmlib;

#--------------------------------
my $delimiter = "::";
#--------------------------------

print "Content-type: text/html\r\n\r\n";
defheader("Yahoo Finder");

#-------------------------------------
if (!$ACL{all}) {
        print "<p>Sorry, but this function may only be run by the server administrator.</p>";
        exit();
}

#-------------------------------------
# Start main work.

&printscriptinfo;

my @yahooforwarders;
my @print;

chdir("/etc/valiases");
@yahooforwarders = `grep -i \@yahoo. *`;

if (! @yahooforwarders) { 
	print "<p>There were no \@yahoo forwarders found. Lucky!</p>";
	exit();
}

foreach my $line (@yahooforwarders) {
	my $owner;
	my $realdomain;
	my $reseller;
	
	my @yahoos;
	my @yahooaddresses;
	
	my ($domain, $forwarder, $addresses) = split(/:/, $line);
	$owner = `/scripts/whoowns $domain`;
	$realdomain = &domainname($owner);
	$reseller = &findreseller($owner);
	
	@yahooaddresses = split(/,/, $addresses);
	my @yahoos;
	foreach (@yahooaddresses) {
		$_ = &trim($_);
		if ($_ =~ /yahoo.com/i) {
			push(@yahoos, $_);
		}
	}

	if ($forwarder eq "*") {
		$forwarder = "default address";
	}
	foreach(@yahoos) {
		push(@print, $reseller . $delimiter . $realdomain . $delimiter . $domain . $delimiter . $forwarder . $delimiter . $_ . "\n<br />");
	}
}

# Print results.
@print = sort { $a cmp $b } @print;
foreach(@print) {
	print($_);
}
	

## Subroutines
sub trim {
	my $string = shift(@_);
	$string =~ s/\s//g;
	return $string;
}

sub domainname {
	my $user;
	my $domain;

	$user = $_[0]; 

	open(CPU,"/var/cpanel/users/" . $user);
	while(<CPU>) {
	        if (/^DNS=(\S+)/) {
	                $domain = $1;
	        }
	}

	close(CPU);
	
	return $domain;
}

sub findreseller {
	my $user;
	my $reseller;

	$user = $_[0];
	open(CPU,"/var/cpanel/users/" . $user);
	while(<CPU>) {
		if (/^OWNER=(\S+)/) {
			$reseller = $1;
		}
	}
	close(CPU);
	
	#if ($reseller ne "root") {
		return &domainname($reseller); 
	#}
	#else {
	#	return "NOT RESOLD";
	#}
}

sub printscriptinfo {
	print "<p>This script shows a list of all email forwarders on the server that are set up to forward email to Yahoo.</p><p><b>[reseller's domain]</b> = The reseller who owns the [cpanel domain].<br /><b>[cpanel domain]</b> = The main cpanel domain where the forwarder is located.<br /><b>[forwarder domain]</b> = The actual domain for the forwarder. (ie: It might be an add-on or parked domain.)<br /><b>[forwarder address]</b> = The actual email address that is doing the forwarding.<br /><b>[yahoo address]</b> = The destination/target email address.</p>";
	print "<p>[reseller's domain] " . $delimiter . " [cpanel domain] " . $delimiter . " [forwarder domain] " . $delimiter . " [forwarder address] " . $delimiter . " [yahoo address]</p>";
}

 

[cmb]

This is awesome, thank you. I also copied and made one for comcast.net, since they seem to be my biggest problem.

It'd be even better if there were a way to remove them in the WHM with this script OR better yet, a cron that would run once a day and automatically remove any forwarders... anyone?

 

[wkdwich]

Not to be argumentative on this, but what is the point here?? If all this ultility is doing is bringing to light the users that have forwarders to AOL accounts.. whats the end purpose?? These people use AOL because they like AOL or are very computer illiterate, some travel and find AOL easy to access in almost any place they go -- that doesn't make them bad people

Now this is not to say I *like* AOL.. on the contrary I think they should be drawn and quartered or worse.. but the fact is more than half of the users on my server only use AOL for their mail.. hence they have forwarders. I have made every attmpt to educate them not to mark mails as spam in their AOL accounts. I also get the notices when they do so I can wag my finger and remind them not to do that

My issue is that AOL sees my server as the origination of some /all of these mails, "the last pass through" the AOL postmaster told me, was going to get blamed for the offending mails.

So lets say Mary is a user on my server.. and [email protected] sends her a spam that does not get caught by my spamassassin and through normal course of events it now gets passed to Mary's AOL account. One of 2 things will now happen..

1. AOL will deem it as spam, refuse it.. causing it to hang around in my queue
2. AOL does NOT deem it as spam and delivers it to Mary

If #1 happens too often my queue gets loaded up fast
If #2 happens and Mary now marks it as spam, AOL gives me a bad mark

Now also there are MANY systems refusing mails on the incoming end.. 99.9% of these mails in my experience are indeed spam and should rightly be refused..

I'd like to see a tool that would see these refusals hanging around in the queue for lets say.. 6 attempts at delivery then trashed.. Maybe using the refused/bounce reason as the trigger..

For instamce in my queue right now is a spam sent from off my server to a user who has a forwarder to a yahoo account. This mail is 110% spam.. the new server isn't reading my custom SA filters yet.

SMTP<< 451 VS2-MF Excessive unknown recipients - possible Open Relay http://help.yahoo.com/help/us/mail/spam/spam-18.html (#4.4.5) 66.249.19.51

Why couldn't there be a rule set to read the attempt at delivery and mark it for X times then trash it automagically. The rule set should be completely editable because everything changes so often..

I belive it is rare that a valid mail would end up in this scenerio.. I would never want to see any host server dictating what ISP and/or email program their people can or should be using. I do encourage everyone to use Eudora and I also encourage all to stray from the AOL-Hell

I hope that made sense.. it is 6am and I have pulled an all-nighter setting a new dedicated live..

 

[Lyttek]

The point is that an entire server can be blacklisted because of 1 user. And the fact that lots of people use AOL is also part of the point... in fact, the more people use AOL, the more it becomes the point.

If person A has their email service degraded because of the actions of person B, my explaining to personA WHY that's happened doesn't matter... they still aren't getting the service they want or contracted and will possibly leave for another host.


Deleting the email automagically doesn't solve the problem... it just hides the problem from you, the system administrator. The user still has the problem of email not working right.

 

[PWSowner]

Other problem is, the user doesn't have to report the spam. Some of the spam is automatically detected by the mail provider, and since it appears to come from our server, we become labelled as spammers.

We still haven't made a final decision as to what to do because we have users who use their Rogers (ISP) mail account, and Rogers is now Yahoo, so due to the spam being forwarded, and NOT reported by the user, every piece of mail from our server to Yahoo/Rogers goes right in the bulk folder.

We've lost 2 customers because they didn't check their bulk folders, and therefore didn't get our welcome email. Now every time I email someone with a Yahoo or Rogers email, I have to go into my Yahoo account and email them from there to tell them they have an email from me in their bulk folder. Their whole "auto bulking" system is really getting annoying, but since those idiots are big companies, we are forced to come up with a way to work around the problem.

 

[rs-freddo]

I would like to see an enhancement (and I know I should submit this as an enhancement request, but I'm not sure how well it would be adopted) where you can not only block users from forwarding mail to AOL, but block users from forwarding their mail to any outside e-mail address. Instead just making the e-mail forwarders an internal option (forward mail between accounts on the user's account). Perhaps have it to where server administrators could set up forwarders for specific users who are assured that the server they are forwarding the messages to, will accept those messages. This may seem like a harsh treatment, but its the only way I can see where you can keep your server free of any blocking from other services because of spam forwarded e-mails.

I think this is a great idea! I agree that if people want to receive email at an outside address they should give out their outside address, and if they give out their hosted domain address they should be required to pick it up off the server. I use internal forwards a lot so internal forwards need to be kept.

 

[Codeguy]

Not Just AOL

Comcast does the exact same thing.

 

[chirpy]

This may seem like a harsh treatment, but its the only way I can see where you can keep your server free of any blocking from other services because of spam forwarded e-mails.

That seems like a good idea to me. If you also add a little education of end-users in how to POP more than one account would make it less harsh a policy.

 

[graham_w]

Thanks for the script - I modified it from AOL to another ISP we are having similar trouble with and it works perfectly. Thanks!

 

[GTFO]

Not to be rude, but i've achieved the same results in shell by

cd /etc/valiases/
grep aol * >> /root/aol-forwarders

Domain names are listed in the aol-forwarders file, etc. Does this script do any more than that specifically?

 

[perlchild]

I think this is a great idea! I agree that if people want to receive email at an outside address they should give out their outside address, and if they give out their hosted domain address they should be required to pick it up off the server. I use internal forwards a lot so internal forwards need to be kept.

Why can't you just use mail routing(aka ldap) instead? Or better yet, why not upgrade smtp(you'd have to rename it to something else I think) to a level where this isn't such a problem? It's getting to the point where 80% of the email traffic unrelated and orthogonal, that is, unrelated, to the design goals of the protocol. Can we finally ask the IETF for a successor protocol which doesn't place such a burden on the isps?