Rejecting mail when account is over quota

[sparek-3]

I am wondering if the "Discard emails for users who have exceeded their quota instead of keeping them in the queue." option in the WHM really works. Below is part of a message that I noticed that was in one of our server's mail queue. I've excluded some of the content, but left the message IDs so that they can be referenced later on:

1FLcIR-0001rz-GY-D
A message that you sent could not be delivered to one or more of its
recipients. This is a permanent error. The following address(es) failed:

  username@server.tld
    (generated from emailaccount@domain.tld)
    retry timeout exceeded

------ This is a copy of the message, including all the headers. ------
.
.
.
Received: id 1FLcIN-0006Dv-9N
From: <invalidemail@anotherdomain.tld>

The ID of the message that is in the queue is 1FLcIR-0001rz-GY. The 1FLcIN-0006Dv-9N ID refers to a message that was originally sent to one of our users on our server. When the message was sent, this account was over its quota. When I look through the mail logs, first for 1FLcIN-0006Dv-9N, I see:

1FLcIN-0006Dv-9N <= invalidemail@anotherdomain.tld H=(some.irrelevent.stuff) [xx.xx.xx.xx] P=esmtp S=1854
1FLcIN-0006Dv-9N == username@server.tld <emailaccount@domain.tld> R=localuser T=local_delivery defer (122): Disk quota exceeded: mailbox is full
1FLcIN-0006Dv-9N ** username@server.tld <emailaccount@domain.tld>: retry timeout exceeded
1FLcIR-0001rz-GY <= <> R=1FLcIN-0006Dv-9N U=mailnull P=local S=2707
1FLcIN-0006Dv-9N Completed

This seems to show the message coming into our server, but it is rejected because username is over their quota.

When I grep the mail logs for the 1FLcIR-0001rz-GY ID, I see:

1FLcIR-0001rz-GY <= <> R=1FLcIN-0006Dv-9N U=mailnull P=local S=2707
1FLcIR-0001rz-GY ** invalidemail@anotherdomain.tld R=lookuphost T=remote_smtp: SMTP error from remote mail server after end of data: host host.anotherdomain.tld [yy.yy.yy.yy]: 550 Message was not accepted -- invalid mailbox.  Local mailbox invalidemail@anotherdomain.tld is unavailable: account is disabled
1FLcIR-0001rz-GY Frozen (delivery error message)
1FLcIR-0001rz-GY Message is frozen

This seems to show that the 1FLcIN-0006Dv-9N message was rejected by the server that hosts anotherdomain.tld (the message probably did not originate from there), but remains on our server's queue.

I'm not doubting that this message is a spam message, and it was likely sent from an invalid e-mail account which is causing the return message to be rejected. My question is, why is our server sending the return message? Shouldn't the message just be rejected once it is determined that the username account is over its quota?

Maybe this is just the way this feature is suppose to work. I'm not really complaining, I'm just interested in understanding what's going on. I suppose the option is working, because the message to emailaccount@domain.tld is not staying in the queue, its the return message that is staying in the queue. I just really don't know why the return message is even being generated. I guess I thought this feature would work more like the ":fail:" option, in that if an account is over its quota, then our SMTP server would send a DENY back to the sending server, forcing the sending server to handle the return e-mail. I suppose the message has to be accepted before it can be determined that the account is over its quota.

Like I said, I'm not really complaining, just looking to see if there is some type of explanation for this or if there is anything else that can be done.

Thanks

[chirpy]

It can't send a deny because it needs to receive the message before it can tell how big it really is, at which stage (i.e. after DATA) it's too late to reject it in the SMTP protocol. That's why you get the overquota bounce in the queue for emails that cannot be returned. The reason they're in the queue is that the SMTP protocol persists in trying to find someone to deliver an error to. Unfortunately, this is almost always spam, since genuine senders would accept the return.

[sparek-3]

That's sort of what I figured. Would it be possible for exim to look up the quota of the account after the RCPT TO stage and if the account is at or over its quota, then deny the message? Even if it is possible, it would probably need to be something that is changed within the default main exim configuration. I'm just wondering if something like this would be feasible and if it had any drawbacks. It looks to me like it would prevent atleast some queue build up on the server.

Are there any other suggestions for dealing with these queued bounce messages? Would lowering the timeout_frozen_after setting affect this?

Again, not really complaining about any of this, just wondering if anyone else had noticed a build of these types of messages and if there was anything that could be done to help with this.

[chirpy]

Just looked through the exim docs and it doesn't seem possible using inbuilt exim commands because all quota related commands are part of the appendfile transport which is one of the last things to happen in local mail delivery. It would almost certainly be possible by checking using a perl script in the ACL section, though.

timeout_frozen_after should help if you reduce it to, say, 2d (days)

[chirpy]

I've just spent a couple of hours investigating this. I've tried implementing an ACL at the RCPT stage for quota checking but it won't work - at this stage, the exim process doesn't have the required privileges to ascertain the size of the users mailbox. So, even though I can establish the correct mailbox directory and the quota, I cannot establish the current size of the mailbox. Only during the appendfile transport can exim change its context so that it can establish the size of the users mailbox.

Shame really, as the perl script I wrote to do this works a treat, except, of course, it doesn't because it cannot compare the mailbox size to the quota

[sparek-3]

Hey chirpy, I really appreciate your efforts on this, I really have no idea when it comes to configuring exim and writing ACLs and I know that you seem to have a really good grasp of all of this.

I'm wondering if there is some confusion in terms of the quota that I was referring to. I was referring to the overall account's quota and not the individual mailbox quota. If an account's overall username is at or over its quota (whether it be e-mail, hosting files, stats, etc). Then would it be possible to reject incoming e-mail before the DATA command in the e-mail transaction?

This may be what you are referring to in your reponse, but you mention that you are unable to determine the size of the mailbox at this stage, so I'm not sure if you are referring to the overall quota or just the mailbox quota.

Again, I appreciate you looking into this. And it should be noted that there really isn't any urgency on this. I am content to leave the configuration the way it is now. I just really didn't know if this was an issue that other people had noticed and whether or not there was enough interest to either look for a solution or log an enhancement request with CPanel (again, if it is something that could be fixed).

Again, thanks for your help.

[brianoz]

Chirpy, I can write something to find the size of a mailbox from exim if that's useful? (yes, taking into account exim's lack of permission - a small well-tied-down setuid C program should do the job nicely). Let me know if you want to take this further!

[chirpy]

It's still something I'm working on and will be sure to get a hold of you should I need that, thanks

[forlinuxsupport]

Hi

If the mailbox is over quota, will the email get bounced back to the sender so they see the error or will the email be accepted by the server and just get nuked ?

I hope the sender gets a bounce to say the recipients mailbox is over quota, so he can notify the receipient of that email. (by phone etc)

Cheers
Andy

[chirpy]

The latter (the sender will get a bounce). However, that is the cause of the problem being described abouve where that very bounce has nowhere to go, usually because it is spam.

[forlinuxsupport]

you mean the former ??

Sender will get the bounce message to say over quota

[sparek-3]

I've been a bit busy working on a few things and I haven't been able to keep up with this thread that much (not that it would really matter that much, not sure if I can really contribute anything as writing Exim ACLs and Exim configuration is a bit above me). But I did just want to say that I appreciate everyone's input on this. I don't want anyone to feel like this is something that they have to work on or have to resolve. I do think, that if it can be resolved, then it would be beneficial for everyone, if everyone could use it.

Again, I appreciate the effort that everyone is taking on this. I know sometimes, you get so involved with a particular project that you just have to stay with it to see it through.

[nxds]

I have hacked together something to reject mail to over or at quota accounts at RCPT time. It's very simple and inelegant. A shell script, run from cron every 5 minutes, puts a list of domains that are at quota in the file /etc/exim_deny_quotalimit. Exim then reads that file in an ACL and rejects messages to those domains. No doubt there are much better ways of doing this.

Here's the cronjob:

*/5 * * * * /usr/local/bin/quotaatlimit > /etc/exim_deny_quotalimit

Here's the shell script:

#!/bin/sh
# list domains that have reached their quota limit

sed 's/=.*$//' /etc/quota.conf | while read user; do
        quota -q -s -u $user;
done 2>/dev/null | awk '/user/ { print $5 }' |
while read name; do
        awk -F: '/'$name'/ { print $1 }' /etc/userdomains
done

Here's the bit that goes into exim.conf:

  #if it gets here it isn't mailman

  deny domains = /etc/exim_deny_quotalimit
         message = mail to $domain has been prohibited - account size limit reached

The whole point of doing this is that I don't want NDRs containing spam being sent from my servers to innocent victims. Now the NDRs are the responsibility of the sending server, not mine.

Only tested on WHM 10.8.0 cPanel 10.8.2-R119/CentOS 4.3 i686 - WHM X v3.1.0.

[brianoz]

while read name; do
awk -F: '/'$name'/ { print $1 }' /etc/userdomains
done[/CODE]

Actually, simple IS elegant in coding and design!

In this case however, I have a small improvement - you need to anchor the match with $name or you'll match accounts with the same prefix, ie: southc and southcor would both be denied email if southc was full. Likewise any domain name on the server with "southc" in it will also be denied email.

You need to use: awk -F: "/: $name\$/"' { print $1 }' /etc/userdomains

[nxds]

Thanks for spotting that Brian!

Another improvement/bug fix is to change the cron job so that the exim_deny_quotalimit file is not empty while the quotaatlimit script is running. Use instead something like:

*/5 * * * * /usr/local/bin/quotaatlimit > /tmp/q.$$ && mv /tmp/q.$$ /etc/exim_deny_quotalimit