The Community Forums

Interact with an entire community of cPanel & WHM users!
  1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Two questions about WGET

Discussion in 'General Discussion' started by SuperBaby, Mar 9, 2004.

  1. SuperBaby

    SuperBaby Well-Known Member

    Joined:
    Nov 27, 2003
    Messages:
    331
    Likes Received:
    0
    Trophy Points:
    16
    Location:
    Thailand
    cPanel Access Level:
    Website Owner
    Twitter:
    I have this line in my crontab to download a text file from a remote server every hour:

    1 * * * * wget --tries=3 --user-agent=monduna.com/2.5 http://www.remotedomain.com/filename.txt

    The problems are:

    (i) A backup copy is created on each execution giving filename.txt AND filename.txt.1, filename.txt.2, filename.txt.3, filename.txt.4 ....
    - How do I overwrite the old copy and not creating old backup copies?

    (ii) How do I save the filename.txt into a specific folder. I tried "wget http://.....filename.txt /home/username/folder" but it didn't work.
     
  2. jsteel

    jsteel Well-Known Member

    Joined:
    Jul 4, 2002
    Messages:
    646
    Likes Received:
    0
    Trophy Points:
    16
    Location:
    Atlanta, GA
    Create a script that purges the old ile first and then gets it again. Call this script in your cron job.
     
  3. SuperBaby

    SuperBaby Well-Known Member

    Joined:
    Nov 27, 2003
    Messages:
    331
    Likes Received:
    0
    Trophy Points:
    16
    Location:
    Thailand
    cPanel Access Level:
    Website Owner
    Twitter:
    Yes, this is exactly what I did:

    0 * * * * wget --tries=3 --user-agent=monduna.com/2.5 http://www.remotesite.com/filename.txt
    0 * * * * rm /home/username/filename.txt.*

    Any idea about the solution for my question (ii) above. I need filename.txt to be saved under /home/username/folder/ instead of /home/username/
     
  4. PWSowner

    PWSowner Well-Known Member

    Joined:
    Nov 10, 2001
    Messages:
    2,948
    Likes Received:
    3
    Trophy Points:
    38
    Location:
    ON, Canada
    You want to use the mv command to move it to a new location, but rather than 3 cron jobs, you're better off writing a simple script to do it all and calling that script.
     
  5. SarcNBit

    SarcNBit Well-Known Member

    Joined:
    Oct 14, 2003
    Messages:
    1,010
    Likes Received:
    3
    Trophy Points:
    38
    If you run that then you will delete what you are trying to save. The more recent downloads will have the higher incremented numbers.

    If you are only trying to keep the most recent copy of the file then you need to run wget with the -r (for recursive) option.

    [EDIT]Looking at the wget man page (man is your friend and should be consulted before posting here ;) ), it looks like -o will allow you to save the file where you want it and it will overwrite it if it already exists (the as -r). That should allow you to do it in one command.

    wget http://example.com/file -o /newdirectory/file [/EDIT]
     
    #5 SarcNBit, Mar 9, 2004
    Last edited: Mar 9, 2004
  6. SuperBaby

    SuperBaby Well-Known Member

    Joined:
    Nov 27, 2003
    Messages:
    331
    Likes Received:
    0
    Trophy Points:
    16
    Location:
    Thailand
    cPanel Access Level:
    Website Owner
    Twitter:
  7. SarcNBit

    SarcNBit Well-Known Member

    Joined:
    Oct 14, 2003
    Messages:
    1,010
    Likes Received:
    3
    Trophy Points:
    38
    When you use the -o option it automatically sets the tries value to 1. That is why I did not bother putting the tries=3 option in my reply.
     
  8. SuperBaby

    SuperBaby Well-Known Member

    Joined:
    Nov 27, 2003
    Messages:
    331
    Likes Received:
    0
    Trophy Points:
    16
    Location:
    Thailand
    cPanel Access Level:
    Website Owner
    Twitter:
    OH, NO! It is still not working. The -o will only write a log to the file specified and not grab and save the file there. Using man wget, it seems like it should be -0 (dash zero) instead. But when I tried that, it said the option is invalid.
     
  9. SuperBaby

    SuperBaby Well-Known Member

    Joined:
    Nov 27, 2003
    Messages:
    331
    Likes Received:
    0
    Trophy Points:
    16
    Location:
    Thailand
    cPanel Access Level:
    Website Owner
    Twitter:
  10. dgbaker

    dgbaker Well-Known Member
    PartnerNOC

    Joined:
    Sep 20, 2002
    Messages:
    2,578
    Likes Received:
    3
    Trophy Points:
    38
    Location:
    Toronto, Ontario Canada
    cPanel Access Level:
    DataCenter Provider
Loading...

Share This Page