GoWilkes

Well-Known Member
Sep 26, 2006
692
33
178
cPanel Access Level
Root Administrator
I need to set up 17 cron jobs as root, and I need them to run as often as possible (I'm moving an active site to a new server, and I'm copying over user-uploaded files from the old server to the new while waiting for the DNS to propagate).

AFAIK, this should work for one if I put it in /etc/crontabs:

Code:
*/5  *  *  *  *  root rsync -aupog -e 'ssh -p 1234' [email protected]:/home/old/public_html/dir/ /home/new/public_html/dir/
This would make it run every 5 minutes. But is there a way for me to make it run one, wait 1 minute, then run the next? This way, I wouldn't be running two at once, and it would give it 1 minute for the CPU load to go down before moving on to the next.

FWIW, this would run for two weeks (max), at which time the old server goes offline permanently.

- - - Updated - - -

I don't have a lot of SH experience, but would this work?:

Code:
#!/bin/sh

rsync -aupog -e 'ssh -p 1234' [email protected]:/home/old/public_html/dir1/ /home/new/public_html/dir1/
sleep 30

rsync -aupog -e 'ssh -p 1234' [email protected]:/home/old/public_html/dir2/ /home/new/public_html/dir2/
sleep 30

rsync -aupog -e 'ssh -p 1234' [email protected]:/home/old/public_html/dir3/ /home/new/public_html/dir3/
sleep 30

# and so on
Then, since there would be 17 of these with a 30 second pause between (meaning a minimum of 8 1/2 minutes) set the cron to run the shell script every 10 minutes.

Is there an easier / better option?
 

vanessa

Well-Known Member
PartnerNOC
Sep 26, 2006
833
28
178
Virginia Beach, VA
cPanel Access Level
DataCenter Provider
First of all, don't put it in /etc/crontab - use /var/spool/cron/root or dump it into /etc/cron.d/ folder.

You probably don't want to manage the parallelism issue with crond itself - you should put these commands into a script and have the script create a lock file to prevent it from running the same command when another is in progress. This way the script may be executed while another is running, but won't actually do anything. Here's an example, not saying it's the prettiest one out there:

Code:
#!/bin/bash

LOCKFILE=/var/run/myrsync.lock

[ -e $LOCKFILE ] && exit 1  # If lock exists, exit

touch $LOCKFILE

rsync -aupog -e 'ssh -p 1234' [email protected]:/home/old/public_html/dir1/ /home/new/public_html/dir1/

rm -f $LOCKFILE
 

GoWilkes

Well-Known Member
Sep 26, 2006
692
33
178
cPanel Access Level
Root Administrator
Good plan, thanks Vanessa. Should I still use sleep 30 after each rm command? I'm concerned that sleep might still tie up the CPU (which is exactly what I'm trying to avoid), but I also need to give a break for a few seconds between each rsync to let the CPU load go down.
 

GoWilkes

Well-Known Member
Sep 26, 2006
692
33
178
cPanel Access Level
Root Administrator
While we're on the subject... any suggestion on putting an end date on the cron, so that it doesn't run past the end of the month? I know that I can remove it manually, I'm just wondering if there's a way to make it stop automatically.
 

cPanelMichael

Administrator
Staff member
Apr 11, 2011
47,880
2,260
463
While we're on the subject... any suggestion on putting an end date on the cron, so that it doesn't run past the end of the month? I know that I can remove it manually, I'm just wondering if there's a way to make it stop automatically.
You could create an additional cron job that removes the existing script or cron job on a certain day of the month.

Thank you.