The Community Forums

Interact with an entire community of cPanel & WHM users!
  1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Stagger cron jobs as root

Discussion in 'General Discussion' started by GoWilkes, Mar 16, 2014.

  1. GoWilkes

    GoWilkes Well-Known Member

    Joined:
    Sep 26, 2006
    Messages:
    367
    Likes Received:
    1
    Trophy Points:
    18
    cPanel Access Level:
    Root Administrator
    I need to set up 17 cron jobs as root, and I need them to run as often as possible (I'm moving an active site to a new server, and I'm copying over user-uploaded files from the old server to the new while waiting for the DNS to propagate).

    AFAIK, this should work for one if I put it in /etc/crontabs:

    Code:
    */5  *  *  *  *  root rsync -aupog -e 'ssh -p 1234' root@123.45.67.89:/home/old/public_html/dir/ /home/new/public_html/dir/
    
    This would make it run every 5 minutes. But is there a way for me to make it run one, wait 1 minute, then run the next? This way, I wouldn't be running two at once, and it would give it 1 minute for the CPU load to go down before moving on to the next.

    FWIW, this would run for two weeks (max), at which time the old server goes offline permanently.

    - - - Updated - - -

    I don't have a lot of SH experience, but would this work?:

    Code:
    #!/bin/sh
    
    rsync -aupog -e 'ssh -p 1234' root@123.45.67.89:/home/old/public_html/dir1/ /home/new/public_html/dir1/
    sleep 30
    
    rsync -aupog -e 'ssh -p 1234' root@123.45.67.89:/home/old/public_html/dir2/ /home/new/public_html/dir2/
    sleep 30
    
    rsync -aupog -e 'ssh -p 1234' root@123.45.67.89:/home/old/public_html/dir3/ /home/new/public_html/dir3/
    sleep 30
    
    # and so on
    
    Then, since there would be 17 of these with a 30 second pause between (meaning a minimum of 8 1/2 minutes) set the cron to run the shell script every 10 minutes.

    Is there an easier / better option?
     
  2. vanessa

    vanessa Well-Known Member
    PartnerNOC

    Joined:
    Sep 26, 2006
    Messages:
    817
    Likes Received:
    22
    Trophy Points:
    18
    Location:
    Virginia Beach, VA
    cPanel Access Level:
    DataCenter Provider
    First of all, don't put it in /etc/crontab - use /var/spool/cron/root or dump it into /etc/cron.d/ folder.

    You probably don't want to manage the parallelism issue with crond itself - you should put these commands into a script and have the script create a lock file to prevent it from running the same command when another is in progress. This way the script may be executed while another is running, but won't actually do anything. Here's an example, not saying it's the prettiest one out there:

    Code:
    #!/bin/bash
    
    LOCKFILE=/var/run/myrsync.lock
    
    [ -e $LOCKFILE ] && exit 1  # If lock exists, exit
    
    touch $LOCKFILE
    
    rsync -aupog -e 'ssh -p 1234' root@123.45.67.89:/home/old/public_html/dir1/ /home/new/public_html/dir1/
    
    rm -f $LOCKFILE
    
     
  3. GoWilkes

    GoWilkes Well-Known Member

    Joined:
    Sep 26, 2006
    Messages:
    367
    Likes Received:
    1
    Trophy Points:
    18
    cPanel Access Level:
    Root Administrator
    Good plan, thanks Vanessa. Should I still use sleep 30 after each rm command? I'm concerned that sleep might still tie up the CPU (which is exactly what I'm trying to avoid), but I also need to give a break for a few seconds between each rsync to let the CPU load go down.
     
  4. GoWilkes

    GoWilkes Well-Known Member

    Joined:
    Sep 26, 2006
    Messages:
    367
    Likes Received:
    1
    Trophy Points:
    18
    cPanel Access Level:
    Root Administrator
    While we're on the subject... any suggestion on putting an end date on the cron, so that it doesn't run past the end of the month? I know that I can remove it manually, I'm just wondering if there's a way to make it stop automatically.
     
  5. cPanelMichael

    cPanelMichael Forums Analyst
    Staff Member

    Joined:
    Apr 11, 2011
    Messages:
    30,678
    Likes Received:
    648
    Trophy Points:
    113
    cPanel Access Level:
    Root Administrator
    You could create an additional cron job that removes the existing script or cron job on a certain day of the month.

    Thank you.
     
Loading...

Share This Page