The Community Forums

Interact with an entire community of cPanel & WHM users!
  1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Backup restoration from Amazon S3 Backup

Discussion in 'Data Protection' started by mobboss, Apr 1, 2014.

Tags:
  1. mobboss

    mobboss Member

    Joined:
    Jan 29, 2014
    Messages:
    8
    Likes Received:
    0
    Trophy Points:
    1
    cPanel Access Level:
    DataCenter Provider
    I have the backups configured correctly and they are backing up to S3 but I am trying to restore an account that got deleted by whmcs.

    I goto Backup Restoration and none of my accounts are listed and the date selectors are not displaying any valid date range choices.

    I also am having issues logging in ssh I get the following error "Permission denied (publickey,gssapi-keyex,gssapi-with-mic)."

    So as a temporary solustion I have logged into the clients newly re-created cpanel account used the backup restore wizard and uploaded the .tar.gz file from the downloaded version on S3.

    I guess its a long way to work around this but will this single upload restor mysql, email and directory contents or will I have to upload 3 versions of the file because I had three option on the restore.

    Thanks in advance for any help just trying to get this handled before the client wakes up in the morning and sees his ecommerce site is down.

    - - - Updated - - -

    Files just finished uploading and of course I get
    [an error occurred while processing this directive]
     
  2. cPanelMichael

    cPanelMichael Forums Analyst
    Staff Member

    Joined:
    Apr 11, 2011
    Messages:
    30,678
    Likes Received:
    648
    Trophy Points:
    113
    cPanel Access Level:
    Root Administrator
    Hello :)

    Please see the following document for instructions on how to restore a backup that is stored on a remote destination:

    cPanel - Remote Restoration

    Let us know if you experience the same issue when using this method.

    Thank you.
     
  3. benito

    benito Well-Known Member

    Joined:
    Jan 8, 2004
    Messages:
    296
    Likes Received:
    1
    Trophy Points:
    18
    Location:
    Mar del Plata - Argentina
    Hello!

    Anyone knows a tool to retrieve all backup files from Amazon S3 to a new server using the console? Its not easy to download one by one and then re-upload to the server.

    Thanks!
     
  4. cPanelMichael

    cPanelMichael Forums Analyst
    Staff Member

    Joined:
    Apr 11, 2011
    Messages:
    30,678
    Likes Received:
    648
    Trophy Points:
    113
    cPanel Access Level:
    Root Administrator
    You must manually obtain the backups from the Amazon S3 server using a method of retrieval that Amazon supports. You may want to review their documentation or consult with their support team to see what exact methods are supported or recommended.

    Thank you.
     
  5. DanH42

    DanH42 Active Member

    Joined:
    Sep 11, 2011
    Messages:
    35
    Likes Received:
    0
    Trophy Points:
    6
    Location:
    Bloomington, IL
    cPanel Access Level:
    Root Administrator
    Does this mean that there's no way to restore a backup (of any type) from S3 through cPanel? That seems like such an obvious bit of functionality that I've been operating under the assumption that it was possible, but I just actually tried and it doesn't seem to be.

    EDIT: I just did a little more research, and it looks like S3 support was actually just added to cPanel recently, and is still in its infancy. There's a feature request that sums things up fairly well.
     
    #5 DanH42, Jun 11, 2014
    Last edited: Jun 11, 2014
  6. cPanelMichael

    cPanelMichael Forums Analyst
    Staff Member

    Joined:
    Apr 11, 2011
    Messages:
    30,678
    Likes Received:
    648
    Trophy Points:
    113
    cPanel Access Level:
    Root Administrator
  7. DanH42

    DanH42 Active Member

    Joined:
    Sep 11, 2011
    Messages:
    35
    Likes Received:
    0
    Trophy Points:
    6
    Location:
    Bloomington, IL
    cPanel Access Level:
    Root Administrator
    Thanks for the link; it brings up some useful points I may not have considered.

    The whole "restore from remote destination" process screams "automatable", but that's starting to get out-of-scope for this thread. I don't see a generic feature request (for all remote types, not just S3), so I may create one.
     
  8. DanH42

    DanH42 Active Member

    Joined:
    Sep 11, 2011
    Messages:
    35
    Likes Received:
    0
    Trophy Points:
    6
    Location:
    Bloomington, IL
    cPanel Access Level:
    Root Administrator
    Take a look at s3cmd. It's pre-packaged for most distributions. For example, on CentOS 6:
    Code:
    sudo wget -O /etc/yum.repos.d/s3tools.repo http://s3tools.org/repo/RHEL_6/s3tools.repo
    sudo yum install s3cmd
    s3cmd --configure
    It's a great tool for working with Amazon S3, and I've used it in a number of cases where I needed to automate the handling of a bunch of files.
     
    #8 DanH42, Jun 11, 2014
    Last edited: Jun 11, 2014
  9. DanH42

    DanH42 Active Member

    Joined:
    Sep 11, 2011
    Messages:
    35
    Likes Received:
    0
    Trophy Points:
    6
    Location:
    Bloomington, IL
    cPanel Access Level:
    Root Administrator
    As a quick workaround, I wrote the following script (which uses s3cmd mentioned above) to do restores from S3:

    Code:
    #! /bin/bash
    
    backup_dir="/var/backups" # The directory cPanel puts backups in, without trailing slash
    s3_bucket_name="mybucket" # Set this to the anme of your S3 bucket
    s3_prefix="/backups" # Leave blank ("") if you don't use a prefix
    
    if [ -z "$1" ] || [ -z "$2" ]; then
    	echo "Usage: remote-restore YYYY-MM-DD <user>"
    	exit
    fi
    
    cd "$backup_dir/$1/accounts" && sudo s3cmd get "s3://$s3_bucket_name$s3_prefix/$1/$2.tar.gz"
    
    read -p "Press the [Enter] key once you are done restoring the backup..."
    
    sudo rm *.tar.gz
    Just change the two variables at the top according to your needs, and place the script somewhere in your path (and mark it executable). Note: Since this runs s3cmd as root, you'll need to run s3cmd --configure as root as well while setting things up.

    To use the script, just supply the date you'd like to restore from and the name of the user you'd like to restore. For example, if I want to restore files from user "joe" to yesterday's backup:

    Code:
    remote-restore 2014-06-10 joe
    It'll ask you for your password if you haven't authenticated to sudo recently, download the appropriate backup, and then hang. Once it pauses, go into WHM and do whatever restorations you wanted. Once you're done, go back to the console and hit [Enter], and it will remove the archive from the local disk.

    This obviously isn't a perfect solution, but I hope it's at least enough to help other users in similar situations get a start in the right direction.

    It also demonstrates just how trivial it would be to automate the process. With a little more error-handling, this is essentially all cPanel needs to do as well, and pulling in from other remote sources such as FTP is similarly simple.
     
  10. cPanelMichael

    cPanelMichael Forums Analyst
    Staff Member

    Joined:
    Apr 11, 2011
    Messages:
    30,678
    Likes Received:
    648
    Trophy Points:
    113
    cPanel Access Level:
    Root Administrator
  11. benito

    benito Well-Known Member

    Joined:
    Jan 8, 2004
    Messages:
    296
    Likes Received:
    1
    Trophy Points:
    18
    Location:
    Mar del Plata - Argentina
    Thank you very much Dan for this valuable information.
     
  12. DanH42

    DanH42 Active Member

    Joined:
    Sep 11, 2011
    Messages:
    35
    Likes Received:
    0
    Trophy Points:
    6
    Location:
    Bloomington, IL
    cPanel Access Level:
    Root Administrator
  13. cPanelMichael

    cPanelMichael Forums Analyst
    Staff Member

    Joined:
    Apr 11, 2011
    Messages:
    30,678
    Likes Received:
    648
    Trophy Points:
    113
    cPanel Access Level:
    Root Administrator
  14. DanH42

    DanH42 Active Member

    Joined:
    Sep 11, 2011
    Messages:
    35
    Likes Received:
    0
    Trophy Points:
    6
    Location:
    Bloomington, IL
    cPanel Access Level:
    Root Administrator
    The feature request I submitted still shows as "On Moderation". I probably should have waited to link to it until it was approved; sorry about that.
     
  15. instantaccess

    instantaccess Registered

    Joined:
    Jun 24, 2016
    Messages:
    1
    Likes Received:
    0
    Trophy Points:
    0
    Location:
    Virginia - USA
    cPanel Access Level:
    Root Administrator
    I know this is a very old thread. It's still an issue. Here's my current workflow if it helps anyone. To restore an account from S3:

    I log into S3 and locate the .tar file I want to restore from. I right mouse click and set the property of the file to public. A download link is displayed.

    I SSH into my server and navigate to backup/date to backup from/accounts (If need-be, make a folder for the date "2016-06-17") and use "wget http://amazon.url" (where amazon.url is the url of the .tar file as displayed on Amazon (noted in the step above). This will download the file directly off S3 to your server and place it into the backups folder.

    Lastly, log into the WHM, select the file from the restoration menu and restore it.

    Don't forget to go back to Amazon and remove the public permission on the .tar file for security reasons.
     
  16. benito

    benito Well-Known Member

    Joined:
    Jan 8, 2004
    Messages:
    296
    Likes Received:
    1
    Trophy Points:
    18
    Location:
    Mar del Plata - Argentina
    I share my method for multiple amazon S3 files download.

    1. Make public all your backup files. You can select all.
    2. Copy and paste the public links one by one into a txt file.
    3. Recreate this file into the folder where you want the backups.
    4. Go to that folder
    5. Run this command "wget -i filelist.txt"
    6. Done
     
  17. RWH Tech

    RWH Tech Well-Known Member

    Joined:
    Oct 1, 2015
    Messages:
    73
    Likes Received:
    10
    Trophy Points:
    8
    Location:
    Brazil
    cPanel Access Level:
    Root Administrator
    You can use s3fs to mount your bucket locally and backup to it. It does add time to the backup process, though.
     
Loading...

Share This Page