Ok, but how does the secondary mailserver know which mail goes where?
The secondary server does nothing with routing other than store the messages for later delivery.
For example:
domain.com set up with 2 MX records.
mx1.domain.com
mx2.domain.com
mx1 hosts domain.com and mx2 is a backup (priorities are 0 & 10 respectively).
So assume mx1 goes down...
Any sending mail servers attempt a connection with mx1 (based on the DNS MX record and priority - in this example "0"), when there is no response, the sending mail server looks for the next highest mx priority, in this case, mx2 (priority "10"). mx2's "/etc/secondarymx" file tells mx2 that even though the domain is not hosted locally, it should queue the message for later delivery to mx1. The message will not be routed to any inboxes locally on mx2 (i.e, you won't be able to log into mx2 and check the e-mail for any domains in the "/etc/secondarymx" file). mx2 will poll mx1 and once mx1 responds, mx2 will deliver the messages intended for mx1's local domains.
All a secondary mail server does is prevent bounced messages after the 5 days or so. Of course it also prevents you from loosing important messages should you suffer an extended outage. Although, if your primary server is down for more than 5 days you'll probably have bigger issues by then - heh.
You might be looking for a mail forwarding service (something that delivers a copy of email to a whole separate server for local delivery. This is a whole different ball game. Check out some of the third party DNS services out there for mail forwarding.
From
http://www.zoneedit.com/:
Backup Mail Service new! - Want to run your own mail server, but you're afraid of missing important e-mails? Our optional "Store and Forward" service will accept e-mails for you while your mail server is down or unavailable, and redeliver them when you're back up!
Of course such a great service is not free.