php script sending mail... how to avoid limits?

[danimal]

Ok,

This may be a strange request as most of my searching turned out people's desire to limit mail sending from scripts...

I have a client who has a mysql table of email subscribers. They have a custom-built php script that takes values from a web form and then loops through the email addresses from the table and sends each the same email using the mail() function in php.

It's functional and it's protected via login (it's an admin-type tool), but I've discovered a new wrinkle...

they have about 2200 subscribers and the last time (a few days ago) that they tried to send an email using the php script/form it only went to about 500 of them.

So I'm racking my brain trying to figure out why it would have stopped at about 500.

• I've added a line to increase the php execution time limit.
• I've set Server Configuration -> Tweak Settings -> Maximum each domain... to 0 (unlimited)
• I've put in a "slowdown" line in the script so that every 25 emails, it sleeps 5 seconds

None of these seem to make any difference. It still only gets through about 500-800 emails before it just stops... no errors... just stops.

Can anyone help me understand what else might be causing this? I want to understand how to allow this script to send to all 2200 subscribers and what I need to set/adjust to make that happen.

P.S. FWIW: it's for a band and the 2200 is their opted-in fan base, so it gets used regularly to send schedules and whatnot.

Thanks!

-Danimal

 

[philpem]

Why not use cPanel's internal mailing list management system?
Open cPanel (port 2083, https), select Mail, then Mailing Lists. Create the mailing list, then click the Edit button on the right when you get the list of mailing lists back. Configure the list to only allow sending from designated email addresses, then mass-subscribe all your subscribers. That should be a fair bit more stable than a PHP-based mailing list manager.

 

[danimal]

I've suggested this as an option to my client.

The problem, though, is that the web site in question already has a custom php admin tool that lets the band members do all sorts of stuff to the site (like modify the calendar, for example). So they are used to the mail tool as-is.

To be honest, I don't have a problem telling my client "you must change it", but I hate not knowing why the script failed. I'd really like to know what was causing the timeout/failures.

Is there a default setting in exim that might cause this?

So it's become more of a crusade for knowledge than anything else.

-Danimal

 

[glansing]

danimal,

If an e-mail fails to send are you terminating the script or just ignoring failures? It would be interesting to see whether it fails in the exact same place every time - if it does one thing I might do is make sure all the e-mail addresses in the database are of a valid format - perhaps write a script that validates their format and spits out ones that don't. I can provide you with a pcre pattern if you need one.

Also - are you communicating with the SMTP server directly, or are you using PHP's mail()?

 

[danimal]

Glansing,

The while loop keeps processing. the mail() function call's return value is checked, but nothing really happens. If it's true, it proceeds. If it's false, it flags the email address in a variable and proceeds.

I'm using the PHP mail() script, not direct SMTP/sendmail calls.

Also, as best as I can determine, the script doesn't crash. It just stops. That is to say, there is no error message of any kind. It acts just like a script would if you hit the stop button on your browser while it was running.

But of course, I'm not hitting the stop button. I'm just trying to figure out what might be causing this script to act this way.

I wonder: does PHP support a way to call the mail() function but not actually have the email sent? I'd love to troubleshoot this some more but without sending test emails to 2200 subscribers (who might not like that).

Hmmm... I still welcome any thoughts/suggestions on what might cause this kind of php script to just stop.

Oh, and yes, I'd like that pcre pattern. Anything that can help. I agree that it would be useful to crank through the email addresses and see if any are "bad". But assuming that is the case, what would you expect it to do to the php script? I would imagine it would either barf in mail() but not cause the script to crash, or it would crash. Either way, it shouldn't exhibit the behavior I'm seeing.

Anyway... thanks for the input, folks!

-Danimal

 

[danimal]

*bump*

Any other thoughts anyone? I don't mean to be a pest, but it's tearing me up that I can't figure this one out.

-Danimal

 

[andywest]

We have a php application that also uses PHP's mail() function. And we also hit a limit when sending a lot of emails, in a procedural loop, to opt-in customers. What we had to do was break up the sending of mail into separate user-directed send packages (< X emails each send) within our code (it worked since we kept track of emails sent in a db, but not elegant). At the time our site was on a shared server.

In looking at our php code we figured we were hitting a processing time limit for the php function we wrote, as it took a long time to sequentially send so many emails and return from the function, and php indicated an error message. We did not have access to the php.ini file.

We now have a dedicated server.

Suggest setting increased processor time limits within php.ini. We'll probably revisit this issue with our new server.

Would like to hear if revising php.ini works for you.

Andy

 

[danimal]

Andy,

Thank you for the reply!

As it turns out, I don't think this is the issue. I am on a dedicated box, but what I discovered is that I can change the PHP timeout limit within a php scirpt:

set_time_limit()

I created a little test script that printed an "X" every 60 seconds and the rest of the time, just cranked through a while(true) loop. At first, it went about a couple of minutes. But when I put in the set_time_limit() function and extended it to say 20 minutes, the script ran for the full 20 minutes. So I didn't need to adjust php.ini to do this.

But when I put this time limit into the mailer script, it didn't help.

The other reason I noticed this was that when the test script failed because of a timeout, it actually printed a php timeout error message (I forget the exact wording). But with the mailer script, it just... stopped. No warning, nothing.

I'm still trying to get to the bottom of this.

-Danimal