Piping mail to php scripts - howto and checklist

webignition

Well-Known Member
Jan 22, 2005
1,876
0
166
Introduction
Over time I've encountered various problems when trying to pipe mail to php scripts, and from time to time people ask how this can be done. I thought I would then post a quick howto and list of things to check if you encounter errors. You should find that this covers all problems and errors you might encounter.​

Step one: creating your script
Your script can, of course, do whatever it likes. However there are two things it will certainly have to do: specify what should be used to interpret it and read the contents of the email piped to it. The the email can be read from STDIN and, maybe surprisingly, an error will be bounced back to the sender if this isn't done.

First things first: the shebang line
It should be the very first line in the script and must specify the path to the php binary.

Suitable values may be:
Code:
#!/usr/bin/php -q
#!/usr/local/bin/php -q
#!/usr/local/php5/bin/php5 -q
On my system (CentOS), the first is the public version of php installed by cPanel. This is generally recommended. The second is, from what I can tell, the OS installation of php (the file modification time on the binary appears to match the OS install date). The third is my separate php5 installation. I've included this to point out that you can pick and choose which version of php if the options is indeed there.

Note the '-q' parameter. This suppresses header output as sometimes php just has to output something. If anything is output by php, the sender will receive a bounceback message containing, among other things, whatever php chose to output. We don't really want this.

Reading the email from STDIN
Below is a rather simple function to do the job for you. This is the same as the function I previously posted (http://forums.cpanel.net/showthread.php?t=39083), with a few more comments to explain what it does.

PHP:
	function mailRead($iKlimit = "")
	{
		// Purpose:
		//   Reads piped mail from STDIN
		//
		// Arguements:
		//   $iKlimit (integer, optional): specifies after how many kilobytes reading of mail should stop
		//   Defaults to 1024k if no value is specified
		//	 A value of -1 will cause reading to continue until the entire message has been read
		//
		// Return value:
		//   A string containing the entire email, headers, body and all.
		
		// Variable perparation		
			// Set default limit of 1024k if no limit has been specified
			if ($iKlimit == "") {
				$iKlimit = 1024;
			}
			
			// Error strings
			$sErrorSTDINFail = "Error - failed to read mail from STDIN!";
		
		// Attempt to connect to STDIN
		$fp = fopen("php://stdin", "r");
		
		// Failed to connect to STDIN? (shouldn't really happen)
		if (!$fp) {
			echo $sErrorSTDINFail;
			exit();
		}
		
		// Create empty string for storing message
		$sEmail = "";
		
		// Read message up until limit (if any)
		if ($iKlimit == -1) {
			while (!feof($fp)) {
			    $sEmail .= fread($fp, 1024);
			}					
		} else {
			while (!feof($fp) && $i_limit < $iKlimit) {
			    $sEmail .= fread($fp, 1024);
				$i_limit++;
			}		
		}
		
		// Close connection to STDIN
		fclose($fp);
		
		// Return message
		return $sEmail;
	}
Note: if a connection to STDIN can't be established, an error message is output. Any output, as I mention above, will cause a bounceback to the sender. In this case if the email can't be read from STDIN, we'll get a whole load of errors anyway, so we might as well try and give some meaning to the errors.

Be careful when setting the size limit ($iKlimit). If this is smaller than the full size of the message, not all of the message will be read from STDIN and a bounceback message may be returned to the sender. I say 'may' as I've seen this behaviour vary from server to server.

If you're only expecting 'normal' emails, you should be safe with setting the limit as -1 so that the entire message is read in. The limit only really comes in useful if you want to stop too much of a message being read in and if you don't care too much about bouncebacks.

For example, if someone sends an email with a 100MB attachment and your script tries to read in the entire thing it will most likely fail due to the memory limit imposed on php. Set the limit to roughtly 1MB less than the php memory limit if in doubt.

That covers the essentials of what your script should contain. Next let's look at permissions.​

Step two: getting the permissions correct
When uploaded, php scripts commonly have permissions of 0644 and directories have permissions of 0755. This is not quite right for what we want to do. For mail to be piped to scripts, both the script and the directory it is in should have permissions of 0755.​

Step three: getting the piping working
This is the dead easy bit as you just need to specify the correct destination for an email filter within cPanel.

Go to cPanel > Mail > E-mail Filtering > Add Filter. Enter whatever you like for the filter and then specify the destination as '|/home/username/path/to/script.php' (without the quotes).

Note: the first character is the pipe character. For me with my UK keyboard, this is typed by pressing shift and the key to the left of z.

You can pop your php script anywhere you like within your webspace and it doesn't need to be in your public_html directory. In fact, it would generally be better if it is not in your public_html directory to prevent people from directly browsing to it.​

Step four: extraneous carriage returns
Sometimes php is picky and doesn't favour carriage returns in scripts. I found that one server of mine doesn't care, whereas another does. I still can't figure out why. To be on the safe side, you shouldn't have any carriage return characters in your script.

These will generally be introduced if you are using a Windows-based editor. Unix-based editors use a single line feed (LF) character to denote a new line, whereas Windows-based editors often use a carriage return and line feed (CRLF). These extraneous CR characters aren't appreciated. A good editor will let you specify the line feed character to use - pick the unix/linux type if you can.

To be on the safe side, the dos2unix command can fix things for you. This can be run from a shell window as follows:
Code:
dos2unix /path/to/script.php

Step five: troubleshooting
If you've written your script correctly, set the permissions on it and it's directory fine, and created your email filter correctly everything will work. Chances are it won't work your first attempt.

Checklist
If in doubt, quickly check that all of the following are correct:
1. You have the correct shebang line
2. Your script reads data from STDIN
3. Your script has permissions of 0755
4. The directory in which your script lives has permissions of 0755
5. Your filter is set up properly. Use the tester in cPanel to ensure that mail is being picked up by the filter. Double check that the path to your script is correct.
6. You have no extraneous carriage returns

If any of the above are not correct, you'll encounter errors. Even if they are all correct, you may still encounter errors. So what should you do if you come across an error?

Troubleshooting
You'll encounter two different types of error: errors with anything mentioned on the checklist or errors with the coding of the php script.

In either case, the sender will get an email back titled 'Mail delivery failed: returning message to sender'.

Scroll to the bottom of the email. If there are php-based error messages, there will be a syntactical or logical error with your code. If not, go through the above checklist again as it will be due to one of these things.

That's about it for troubleshooting. Go through the checklist to ensure that everything is set up correctly and study php-based errors in bounceback emails and correct coding errors.​

Step six: your script's environment and testing
The way things are
Your script won't be running in the same enviroment as if it were being browsed to. This will affect the superglobal arrays you might often use.

In fact the only superglobal array that contains anything useful is $_SERVER. And most of this is generally of no use. One value that is of use is $_SERVER['argv'].

For scripts executed from the command line, $_SERVER['argv'] will contain the arguements passed to the script in the form of an array. In the case of mail piping, this (at least for me) contains only one element and that is the full path to the script i.e. $_SERVER['argv'][0].

If find this useful if I want to include common library files that are used elsewhere within the same account. I normally deal with common library files by setting the include path, but that won't work here - your script is on it's own and has no idea where everything else is. But if you know the path to the script, you can navigate round to where your library files are without having to hardcode the exact path.

Testing
The process of testing your scripts is not really much different when emails are being piped to them, with the exception that you can't just output test values to the screen. Ok, you can output test values and then sift through the bounceback message to find them.

I find an easier way is to simply email test values to myself. This way your script can execute smoothly and not generate a bounceback message and you can still peek at what the case is at certain points along the way.

And I think that generally covers all you need to know about piping email to php scripts.
 

webignition

Well-Known Member
Jan 22, 2005
1,876
0
166
Update
webignition said:
In fact the only superglobal array that contains anything useful is $_SERVER. And most of this is generally of no use. One value that is of use is $_SERVER['argv'].
$_SERVER['argv'] won't be populated by default under PHP 5. If you're finding it's empty, you'll need to enable an option in your php.ini:
Code:
register_argc_argv = On
The parameter defaults of 'Off' for performance reasons. I haven't encountered any performance issues with it enabled.
 

nazoreen

Active Member
May 18, 2003
34
0
156
Hello,

Excuse me, but I don't speak english very well !

What is a "pipe email" ? what is the goal of the script ?

Thank you...
 

heymon

Registered
Mar 30, 2007
1
0
151
Thanks

Thanks, it's just what I was looking for. Script works as expected.

I have a website hosted by HostGator using cPanel.:D
 

arabbetts

Registered
Aug 28, 2007
1
0
51
Email Piping now working

Thank you for the article, I've been hashbanging my head for nearly a day and I didn't realise as you explained that you could pipe via a filter, I did have it working through a forwarder but it stopped working when cp11 was installed. I did it via the filter and it worked first time!

Andy
 

subjectx

Member
Jun 11, 2008
5
0
51
i'm getting this when i send mail to set receiptant:

Code:
Mail Delivery System <[email protected]> 		to me
	show details	 2:12 pm (0 minutes ago) 
This message was created automatically by mail delivery software.

A message that you sent could not be delivered to one or more of its
recipients. This is a permanent error. The following address(es) failed:

 pipe to |/home/xxxx/scripts/receive.php
   generated by [email protected]

The following text was generated during the delivery attempt:

------ pipe to |/home/xxxx/scripts/receive.php
      generated by [email protected] ------



------ This is a copy of the message, including all the headers. ------

Return-path: <[email protected]>
Received: from wf-out-1314.google.com ([209.85.200.168])
       by alpha.racunalnistvo.net with esmtp (Exim 4.68)
       (envelope-from <[email protected]>)
       id 1K7r5V-0003oA-Kk
       for [email protected]; Sun, 15 Jun 2008 14:12:01 +0200
Received: by wf-out-1314.google.com with SMTP id 24so4568261wfg.7
       for <[email protected]>; Sun, 15 Jun 2008 05:12:10 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
       d=gmail.com; s=gamma;
       h=domainkey-signature:received:received:message-id:date:from:to
        :subject:mime-version:content-type;
       bh=hnGeIXTnfp228NCK10fFnL+cw95up8cu9HpEjpulTuk=;
       b=U69GCd8qAEeG22z9ZhKky/Si0A3ckDG2Ee35cdvqG8zOE7qS0s8siR7JTY67u59QnM
        PadGJ0eaPaDK36gHvlSY73fdWFxwuzwAf4lUQhwoiHgTBrx3rRkx5IEb637aViPJTUam
        1bLr3D/R4ivdfp1NQSfkASrm/d6ijOuhWno7Q=
DomainKey-Signature: a=rsa-sha1; c=nofws;
       d=gmail.com; s=gamma;
       h=message-id:date:from:to:subject:mime-version:content-type;
       b=CDwJarbo4soOl9gKw/ANMWeSEzxMrLO0gLTHLv8k4BkbgKaMxtr20JHyGZajs0DEuW
        7uG0YRChBP3WE4x3jO+76pJ/BvmLXxFk7jWfH4V8dyDQ6+Z+dehgN5Cv/+z7YHis7ktV
        CtJvRqw9mPgkTP67ql6cqsEat0OztN0neXjVU=
Received: by 10.142.53.9 with SMTP id b9mr472659wfa.343.1213531930302;
       Sun, 15 Jun 2008 05:12:10 -0700 (PDT)
Received: by 10.142.140.18 with HTTP; Sun, 15 Jun 2008 05:12:10 -0700 (PDT)
Message-ID: <[email protected]>
Date: Sun, 15 Jun 2008 14:12:10 +0200
From: "Anxxxxer" <[email protected]>
To: [email protected]
Subject: s
MIME-Version: 1.0
Content-Type: multipart/alternative;
       boundary="----=_Part_23889_29023720.1213531930296"

------=_Part_23889_29023720.1213531930296
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

sdddd

--


------=_Part_23889_29023720.1213531930296
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

sdddd<br clear="all"><br>-- <br>Lp, Anze Pratnemer

------=_Part_23889_29023720.1213531930296--
Any solution?

I'm getting same error if i use forwarder and pipe to program, but there, script doesnt execute, while if i set filter for this mail and pipe it to script it works, but in both casses i get this error-reply..
 

subjectx

Member
Jun 11, 2008
5
0
51
after some lucky incident, i deleted empty new line feed between hashbang and <? start of my script and it started working without any strange error mails.

I deleted all of whitespaces after ?> too..
 

zabran

Registered
Jun 6, 2008
4
0
51
after some lucky incident, i deleted empty new line feed between hashbang and <? start of my script and it started working without any strange error mails.

I deleted all of whitespaces after ?> too..
Is the following the problem?

Step four: extraneous carriage returns
Sometimes php is picky and doesn't favour carriage returns in scripts. I found that one server of mine doesn't care, whereas another does. I still can't figure out why. To be on the safe side, you shouldn't have any carriage return characters in your script.

These will generally be introduced if you are using a Windows-based editor. Unix-based editors use a single line feed (LF) character to denote a new line, whereas Windows-based editors often use a carriage return and line feed (CRLF). These extraneous CR characters aren't appreciated. A good editor will let you specify the line feed character to use - pick the unix/linux type if you can.
Right now, my script executed correctly but there is a bounceback to sender despite deleting empty new line feeds and carriage returns.

The bounceback states that:

Code:
PHP Warning:  Module 'ffmpeg' already loaded in Unknown on line 0
Can someone please help?
 
Last edited:

zabran

Registered
Jun 6, 2008
4
0
51
I tried using webignition's method

cPanel > Mail > E-mail Filtering > Add Filter

instead of

cPanel > Mail > E-mail Forwarders > Add Forwarder

and my script failed to execute.

The bounceback was more explicit though

Code:
The following text was generated during the delivery attempt:

------ pipe to |/home/username/public_html/domain/script.php 2
       generated by [email protected] ------

PHP Warning:  Module 'ffmpeg' already loaded in Unknown on line 0
Error in argument 1, char 3: option not found 
Usage: php [-q] [-h] [-s] [-v] [-i] [-f <file>]
       php <file> [args...]
  -a               Run interactively
  -b <address:port>|<port> Bind Path for external FASTCGI Server mode
  -C               Do not chdir to the script's directory
  -c <path>|<file> Look for php.ini file in this directory
  -n               No php.ini file will be used
  -d foo[=bar]     Define INI entry foo with value 'bar'
  -e               Generate extended information for debugger/profiler
  -f <file>        Parse <file>.  Implies `-q'
  -h               This help
  -i               PHP information
  -l               Syntax check only (lint)
  -m               Show compiled in modules
  -q               Quiet-mode.  Suppress HTTP Header output.
  -s               Display colour syntax highlighted source.
  -v               Version number
  -w               Display source with stripped comments and whitespace.
  -z <file>        Load Zend extension <file>.
  -T <count>       Measure execution time of script repeated <count> times.
 

rolinger

Active Member
Feb 13, 2017
33
1
8
Tampa
cPanel Access Level
Root Administrator
I know this is old but hopefully someone will see it and respond. Because I am having trouble getting bounced emails in a timely fashion I am forced to save my bounced messages to a file and then pipe the file to my script in order to test.

Test email:
From: [email protected]
To: [email protected]
Subject: Registration Link

For MS Outlook a bounced email has the base message and then three attachments in it:

The base message is the actual bounce, in it the
From: [email protected]
To: [email protected]

The three attachments:
1. details.txt (has Reporting MTA info in it)
2. Untitled Attachment 0s511.txt (original email header info)
3. "Register Link" .msg -( the original email)

In MS Outlook, when I save the bounced email to an external file, it looks like its taking all 4 components, the base email and the three attachments, and flattening them into a single file. When piping and parsing that single file through my script, there are now two TO/FROM addresses. The original From/To and then the base bounce From (mailer-daemon)/To: [email protected]

When the pipe happens from the cPanel mail filter - what actually gets piped to the script? Because the base bounce file doesn't contain the original TO address, it only contains the original FROM address as the new TO address.