Backup restoration from Amazon S3 Backup

[mobboss]

I have the backups configured correctly and they are backing up to S3 but I am trying to restore an account that got deleted by whmcs.

I goto Backup Restoration and none of my accounts are listed and the date selectors are not displaying any valid date range choices.

I also am having issues logging in ssh I get the following error "Permission denied (publickey,gssapi-keyex,gssapi-with-mic)."

So as a temporary solustion I have logged into the clients newly re-created cpanel account used the backup restore wizard and uploaded the .tar.gz file from the downloaded version on S3.

I guess its a long way to work around this but will this single upload restor mysql, email and directory contents or will I have to upload 3 versions of the file because I had three option on the restore.

Thanks in advance for any help just trying to get this handled before the client wakes up in the morning and sees his ecommerce site is down.

- - - Updated - - -

Files just finished uploading and of course I get
[an error occurred while processing this directive]

 

[cPanelMichael]

Hello

Please see the following document for instructions on how to restore a backup that is stored on a remote destination:

cPanel - Remote Restoration

Let us know if you experience the same issue when using this method.

Thank you.

 

[benito]

Hello!

Anyone knows a tool to retrieve all backup files from Amazon S3 to a new server using the console? Its not easy to download one by one and then re-upload to the server.

Thanks!

 

[cPanelMichael]

You must manually obtain the backups from the Amazon S3 server using a method of retrieval that Amazon supports. You may want to review their documentation or consult with their support team to see what exact methods are supported or recommended.

Thank you.

 

[DanH42]

Does this mean that there's no way to restore a backup (of any type) from S3 through cPanel? That seems like such an obvious bit of functionality that I've been operating under the assumption that it was possible, but I just actually tried and it doesn't seem to be.

EDIT: I just did a little more research, and it looks like S3 support was actually just added to cPanel recently, and is still in its infancy. There's a feature request that sums things up fairly well.

 

[cPanelMichael]

Right, currently backups stored on any type of remote storage device must be restored manually using the instructions here:

Remote Restoration

Thank you.

 

[DanH42]

Thanks for the link; it brings up some useful points I may not have considered.

The whole "restore from remote destination" process screams "automatable", but that's starting to get out-of-scope for this thread. I don't see a generic feature request (for all remote types, not just S3), so I may create one.

 

[DanH42]

Anyone knows a tool to retrieve all backup files from Amazon S3 to a new server using the console?

Take a look at s3cmd. It's pre-packaged for most distributions. For example, on CentOS 6:

sudo wget -O /etc/yum.repos.d/s3tools.repo http://s3tools.org/repo/RHEL_6/s3tools.repo
sudo yum install s3cmd
s3cmd --configure

It's a great tool for working with Amazon S3, and I've used it in a number of cases where I needed to automate the handling of a bunch of files.

 

[DanH42]

As a quick workaround, I wrote the following script (which uses s3cmd mentioned above) to do restores from S3:

#! /bin/bash

backup_dir="/var/backups" # The directory cPanel puts backups in, without trailing slash
s3_bucket_name="mybucket" # Set this to the anme of your S3 bucket
s3_prefix="/backups" # Leave blank ("") if you don't use a prefix

if [ -z "$1" ] || [ -z "$2" ]; then
	echo "Usage: remote-restore YYYY-MM-DD <user>"
	exit
fi

cd "$backup_dir/$1/accounts" && sudo s3cmd get "s3://$s3_bucket_name$s3_prefix/$1/$2.tar.gz"

read -p "Press the [Enter] key once you are done restoring the backup..."

sudo rm *.tar.gz

Just change the two variables at the top according to your needs, and place the script somewhere in your path (and mark it executable). Note: Since this runs s3cmd as root, you'll need to run s3cmd --configure as root as well while setting things up.

To use the script, just supply the date you'd like to restore from and the name of the user you'd like to restore. For example, if I want to restore files from user "joe" to yesterday's backup:

remote-restore 2014-06-10 joe

It'll ask you for your password if you haven't authenticated to sudo recently, download the appropriate backup, and then hang. Once it pauses, go into WHM and do whatever restorations you wanted. Once you're done, go back to the console and hit [Enter], and it will remove the archive from the local disk.

This obviously isn't a perfect solution, but I hope it's at least enough to help other users in similar situations get a start in the right direction.

It also demonstrates just how trivial it would be to automate the process. With a little more error-handling, this is essentially all cPanel needs to do as well, and pulling in from other remote sources such as FTP is similarly simple.

 

[cPanelMichael]

Thank you for taking the time to provide that workaround to this thread. I'm sure others will find it useful.

 

[benito]

Thank you very much Dan for this valuable information.

 

[DanH42]

Apparently I can't edit an old post, but for anyone else interested, here's a link to the feature request: Login Form | cPanel Feature Requests

 

[cPanelMichael]

The link you referenced in your last post is not valid. I believe you may be referring to the following feature request:

Complete Amazon S3 Backup Solution | cPanel Feature Requests

Thank you.

 

[DanH42]

The feature request I submitted still shows as "On Moderation". I probably should have waited to link to it until it was approved; sorry about that.

 

[instantaccess]

I know this is a very old thread. It's still an issue. Here's my current workflow if it helps anyone. To restore an account from S3:

I log into S3 and locate the .tar file I want to restore from. I right mouse click and set the property of the file to public. A download link is displayed.

I SSH into my server and navigate to backup/date to backup from/accounts (If need-be, make a folder for the date "2016-06-17") and use "wget http://amazon.url" (where amazon.url is the url of the .tar file as displayed on Amazon (noted in the step above). This will download the file directly off S3 to your server and place it into the backups folder.

Lastly, log into the WHM, select the file from the restoration menu and restore it.

Don't forget to go back to Amazon and remove the public permission on the .tar file for security reasons.

 

[benito]

I share my method for multiple amazon S3 files download.

1. Make public all your backup files. You can select all.
2. Copy and paste the public links one by one into a txt file.
3. Recreate this file into the folder where you want the backups.
4. Go to that folder
5. Run this command "wget -i filelist.txt"
6. Done

 

[RWH Tech]

You can use s3fs to mount your bucket locally and backup to it. It does add time to the backup process, though.