How to configure Amazon S3 backups in WHM

ChrisI

Technical Analyst II
Mar 24, 2014
67
12
83
cPanel Access Level
Root Administrator
The purpose of this post is to provide a guide for setting up Amazon S3 backups in the WHM interface, as well as the steps to getting it set up in Amazon's interface. Amazon S3 is a service that is offered by Amazon that can be used for data storage, which also works well for storing cPanel backups. There, you are able to store your backups remotely on Amazon's servers as well as locally on the server. That will ensure that your backups are safe from being lost due to issues like hard drive failures. These are the steps that can be used to set that up.

1. Sign up for an Amazon account at https://aws.amazon.com/

2. Log into the Amazon account AWS Console at AWS Management Console

3. Click on S3 under Storage & Content Delivery.

Screen Shot 2014-09-28 at 12.27.13 PM.png

4. Click Create Bucket to create a new bucket.

Screen Shot 2014-09-28 at 12.29.16 PM.png

5. Create a new directory using the 'Create Folder' button at the top left if you would like to store your backups in their own folder. I went ahead and created a folder named 'Backups' to store the backups in.

Screen Shot 2014-09-28 at 12.31.20 PM.png Screen Shot 2014-09-28 at 12.31.48 PM.png

6. At the top right, click on your name, then 'Security Credentials' and select 'Continue to Security Credentials' when the warning pops up.

Screen Shot 2014-09-28 at 12.33.04 PM.png Screen Shot 2014-09-28 at 12.33.43 PM.png

7. Click the + sign to the left of 'Access Keys (Access Key ID and Secret Access Key)', then click 'Create New Access Key' to create a new key pair. You will likely also want to download the file when prompted so that you can save the credentials, since you are no longer able to view them once the initial box with that information is closed. You can also view the access key from the popup that shows by clicking the 'Show Access Key' link, which is no longer able to be viewed once that box is closed.

Screen Shot 2014-09-28 at 12.34.31 PM.png Screen Shot 2014-09-28 at 12.35.17 PM.png

8. Log into WHM and go to Backup >> Backup Configuration and make sure that the backups are enabled for all of the users.

Screen Shot 2014-09-28 at 12.36.10 PM.png Screen Shot 2014-09-28 at 12.36.45 PM.png Screen Shot 2014-09-28 at 12.38.56 PM.png

9. At the very bottom of the backups page under 'Additional Destinations', ensure that the destination type in the box is 'Amazon S3', then click 'Create new destination'.

Screen Shot 2014-09-28 at 12.39.35 PM.png

10. Use this information in the boxes once that is selected:

Destination name: Whatever you would like for you backup destination to be named in the WHM. This is your choice and does not affect the connection to the Amazon S3 server.

Folder: Name of the folder that was created to store the backups if any. If that is left blank, the root of the bucket will be used.

Bucket: Name of the bucket that was created that will store the backups. Please keep in mind that we currently have case 92993 open, which prevents Buckets with .'s to be unable to validate in the WHM backup configuration, so those must not contain any .'s in order to work correctly at this time and until that case is resolved.

Access Key ID and Secret Access Key: Those are both provided under the 'Show Access Key' section on your amazon security credentials page and in the file that is downloaded when going to the Security Credentials page. You can copy/paste that information from what is provided by Amazon.

Timeout: The time, in seconds, that the server will wait for a response from the remote Amazon server. The 30 second default will be sufficient for most backups, however that can be raised if you experience timeouts when uploading the backups.

Screen Shot 2014-09-28 at 12.41.40 PM.png

11. Once those are filled out, click 'Save and Validate Destination'

Screen Shot 2014-09-28 at 12.42.20 PM.png

12. If all of the information is added correctly, the backups page should show a message that says 'Success: Destination Saved'.

Screen Shot 2014-09-28 at 12.43.14 PM.png

13. You should now be able to allow the backups to run and they should upload to your Amazon S3 account. You can use this command as root via ssh to force the backups to run.

Code:
/usr/local/cpanel/bin/backup --force
When that is run, it will provide the log that it is using for the backups.

Code:
-bash-4.1# /usr/local/cpanel/bin/backup --force
info [backup] Started at Sun Sep 28 12:00:38 2014
info [backup] Process started in background.
info [backup] Log file: /usr/local/cpanel/logs/cpbackup/1411923638.log
You can use a command like this to verify that the backup is running correctly.

Code:
tail -f /usr/local/cpanel/logs/cpbackup/1411920835.log
You can also use this command to verify that the backups are uploaded correctly.

Code:
-bash-4.1# tail -f /usr/local/cpanel/logs/cpbackup_transporter.log
That should show output that shows the uploads completing successfully.

Code:
-bash-4.1# tail -f /usr/local/cpanel/logs/cpbackup_transporter.log
[2014-09-28 12:01:43 -0500] info [cpbackup_transporter] cPanel Backup Transporter Queue Daemon started.
[2014-09-28 12:01:43 -0500] info [cpbackup_transporter] cpbackup_transporter - started
[2014-09-28 12:01:43 -0500] info [cpbackup_transporter] cpbackup_transporter - Checking queue for tasks
[2014-09-28 12:01:43 -0500] info [cpbackup_transporter] cpbackup_transporter - Processing next task
[2014-09-28 12:01:43 -0500] info [cpbackup_transporter] Instantiating Object
[2014-09-28 12:01:43 -0500] info [cpbackup_transporter] Starting a "copy" operation on the "Amazon Backup" destination ID "rxpoY3vkuP1gI9ZYHemw8abH".
[2014-09-28 12:01:43 -0500] info [cpbackup_transporter] Validating destination path /Backups/2014-09-28
[2014-09-28 12:01:44 -0500] info [cpbackup_transporter] Path exists
[2014-09-28 12:01:44 -0500] info [cpbackup_transporter] Uploading /backup/2014-09-28/accounts/cptest2.tar.gz to /Backups/2014-09-28/cptest2.tar.gz
[2014-09-28 12:01:44 -0500] info [cpbackup_transporter] Attempting to upload /backup/2014-09-28/accounts/cptest2.tar.gz to /Backups/2014-09-28/cptest2.tar.gz for destination: Amazon Backup
[2014-09-28 12:01:44 -0500] info [cpbackup_transporter] Upload attempt #1 starting for /backup/2014-09-28/accounts/cptest2.tar.gz to /Backups/2014-09-28/cptest2.tar.gz for destination: Amazon Backup
[2014-09-28 12:01:45 -0500] info [cpbackup_transporter] Successful transfer of /backup/2014-09-28/accounts/cptest2.tar.gz to /Backups/2014-09-28/cptest2.tar.gz for destination Amazon Backup
14. Log into your Amazon S3 account and verify that the backups were uploaded correctly by clicking the 'Services' button at the top left of your Amazon account, then selecting S3. That will take you to your file tree. From there, you click on the folder that you chose to store the backups in, which should contain the backups that were uploaded from the server.

Screen Shot 2014-09-28 at 12.50.00 PM.png Screen Shot 2014-09-28 at 12.51.26 PM.png



Amazon also provides a list of the possible error codes here, which will display in the errors if there are any problems when uploading the backups.

Error Responses - Amazon Simple Storage Service
 

sytrus

Registered
Nov 18, 2014
1
0
1
cPanel Access Level
Root Administrator
If you try to configure but getting this error:
Could not upload test file: InvalidRequest: The authorization mechanism you have provided is not supported. Please use AWS4-HMAC-SHA256.
Change Amazon S3 server from Frankfurt, new S3 regions deployed after January, 2014 will not support V2 authorisation!
 

jecro77

Member
Mar 14, 2008
6
0
51
Switzerland
Can it be, that your tool "cpbackup_transporter" upload the backups unsecured over http? The amazon log shows this here:

[23/Nov/2014:02:00:43 +0000] IP arn:aws:iam::000:user/USER 000 REST.PUT.PART backups/2014-11-23/ACCOUNT.tar.gz "PUT /backups%2F2014-11-23%2FACCOUNT%2Etar%2Egz?partNumber=00&uploadId=00 HTTP/1.1" 200 - - 00 "-" "libwww-perl/6.05" -
 

RDI

Registered
Feb 20, 2014
1
0
1
cPanel Access Level
Website Owner
So are you saying that this is NOT a secure method backup ?

Going into Amazon S3 ??

if so any other suggestions ??

- - -
jecro77

Can it be, that your tool "cpbackup_transporter" upload the backups unsecured over http? The amazon log shows this here:

[23/Nov/2014:02:00:43 +0000] IP arn:aws:iam::000:user/USER 000 REST.PUT.PART backups/2014-11-23/ACCOUNT.tar.gz "PUT /backups%2F2014-11-23%2FACCOUNT%2Etar%2Egz?partNumber=00&uploadId=00 HTTP/1.1" 200 - - 00 "-" "libwww-perl/6.05" -
 

vdlwebdesign

Registered
Jan 15, 2016
3
0
1
south wales
cPanel Access Level
Reseller Owner
i login via my reseller dashboard or direct link how do i login as root?

Im trying to backup all the websites and email accounts i have set up to AWS S3 i don't want to do this VIA a website plugin as i will only copy the website and database and would have to go into every site and set it up, if i set it up in WHM will is backup the full account?

is there a way i can set the backup from the individual c-pannels?

sion
 

cPanelMichael

Administrator
Staff member
Apr 11, 2011
47,908
2,216
463
Hello :)

The information you provided suggests you have reseller access only. You won't have access to the backup configuration options offered through Web Host Manager if you do not have "root" access to the server. You can backup an account manually using the "Backup Wizard" option in cPanel to a remote destination (FTP/SCP) if your web hosting provider has provided you with that feature.

Thank you.
 

billmcollam

Member
Jan 12, 2015
22
2
53
Ontario Canada
cPanel Access Level
Reseller Owner
Hi,
I used this tutorial a few months ago to successfully set up my backup to S3. It works fine - S3 essentially has a copy of all my WHM backups. But what if I wanted to change the retention periods so that S3 (the cheaper space) retains the data longer?
So for example I might want WHM to keep 3 most recent instances of daily backups but keep my weeklys and monthlies only on S3?
would this involve fiddling with the transporter scripts?
 

cPanelMichael

Administrator
Staff member
Apr 11, 2011
47,908
2,216
463
if I wanted to change the retention periods so that S3 (the cheaper space) retains the data longer?
So for example I might want WHM to keep 3 most recent instances of daily backups but keep my weeklys and monthlies only on S3?
would this involve fiddling with the transporter scripts?
The following thread should help:

Retain backups on Amazon S3 only

Thank you.