The purpose of this post is to provide a guide for setting up Amazon S3 backups in the WHM interface, as well as the steps to getting it set up in Amazon's interface. Amazon S3 is a service that is offered by Amazon that can be used for data storage, which also works well for storing cPanel backups. There, you are able to store your backups remotely on Amazon's servers as well as locally on the server. That will ensure that your backups are safe from being lost due to issues like hard drive failures. These are the steps that can be used to set that up.
1. Sign up for an Amazon account at https://aws.amazon.com/
2. Log into the Amazon account AWS Console at AWS Management Console
3. Click on S3 under Storage & Content Delivery.
4. Click Create Bucket to create a new bucket.
5. Create a new directory using the 'Create Folder' button at the top left if you would like to store your backups in their own folder. I went ahead and created a folder named 'Backups' to store the backups in.
6. At the top right, click on your name, then 'Security Credentials' and select 'Continue to Security Credentials' when the warning pops up.
7. Click the + sign to the left of 'Access Keys (Access Key ID and Secret Access Key)', then click 'Create New Access Key' to create a new key pair. You will likely also want to download the file when prompted so that you can save the credentials, since you are no longer able to view them once the initial box with that information is closed. You can also view the access key from the popup that shows by clicking the 'Show Access Key' link, which is no longer able to be viewed once that box is closed.
8. Log into WHM and go to Backup >> Backup Configuration and make sure that the backups are enabled for all of the users.
9. At the very bottom of the backups page under 'Additional Destinations', ensure that the destination type in the box is 'Amazon S3', then click 'Create new destination'.
10. Use this information in the boxes once that is selected:
Destination name: Whatever you would like for you backup destination to be named in the WHM. This is your choice and does not affect the connection to the Amazon S3 server.
Folder: Name of the folder that was created to store the backups if any. If that is left blank, the root of the bucket will be used.
Bucket: Name of the bucket that was created that will store the backups. Please keep in mind that we currently have case 92993 open, which prevents Buckets with .'s to be unable to validate in the WHM backup configuration, so those must not contain any .'s in order to work correctly at this time and until that case is resolved.
Access Key ID and Secret Access Key: Those are both provided under the 'Show Access Key' section on your amazon security credentials page and in the file that is downloaded when going to the Security Credentials page. You can copy/paste that information from what is provided by Amazon.
Timeout: The time, in seconds, that the server will wait for a response from the remote Amazon server. The 30 second default will be sufficient for most backups, however that can be raised if you experience timeouts when uploading the backups.
11. Once those are filled out, click 'Save and Validate Destination'
12. If all of the information is added correctly, the backups page should show a message that says 'Success: Destination Saved'.
13. You should now be able to allow the backups to run and they should upload to your Amazon S3 account. You can use this command as root via ssh to force the backups to run.
When that is run, it will provide the log that it is using for the backups.
You can use a command like this to verify that the backup is running correctly.
You can also use this command to verify that the backups are uploaded correctly.
That should show output that shows the uploads completing successfully.
14. Log into your Amazon S3 account and verify that the backups were uploaded correctly by clicking the 'Services' button at the top left of your Amazon account, then selecting S3. That will take you to your file tree. From there, you click on the folder that you chose to store the backups in, which should contain the backups that were uploaded from the server.
Amazon also provides a list of the possible error codes here, which will display in the errors if there are any problems when uploading the backups.
Error Responses - Amazon Simple Storage Service
1. Sign up for an Amazon account at https://aws.amazon.com/
2. Log into the Amazon account AWS Console at AWS Management Console
3. Click on S3 under Storage & Content Delivery.
4. Click Create Bucket to create a new bucket.
5. Create a new directory using the 'Create Folder' button at the top left if you would like to store your backups in their own folder. I went ahead and created a folder named 'Backups' to store the backups in.
6. At the top right, click on your name, then 'Security Credentials' and select 'Continue to Security Credentials' when the warning pops up.
7. Click the + sign to the left of 'Access Keys (Access Key ID and Secret Access Key)', then click 'Create New Access Key' to create a new key pair. You will likely also want to download the file when prompted so that you can save the credentials, since you are no longer able to view them once the initial box with that information is closed. You can also view the access key from the popup that shows by clicking the 'Show Access Key' link, which is no longer able to be viewed once that box is closed.
8. Log into WHM and go to Backup >> Backup Configuration and make sure that the backups are enabled for all of the users.
9. At the very bottom of the backups page under 'Additional Destinations', ensure that the destination type in the box is 'Amazon S3', then click 'Create new destination'.
10. Use this information in the boxes once that is selected:
Destination name: Whatever you would like for you backup destination to be named in the WHM. This is your choice and does not affect the connection to the Amazon S3 server.
Folder: Name of the folder that was created to store the backups if any. If that is left blank, the root of the bucket will be used.
Bucket: Name of the bucket that was created that will store the backups. Please keep in mind that we currently have case 92993 open, which prevents Buckets with .'s to be unable to validate in the WHM backup configuration, so those must not contain any .'s in order to work correctly at this time and until that case is resolved.
Access Key ID and Secret Access Key: Those are both provided under the 'Show Access Key' section on your amazon security credentials page and in the file that is downloaded when going to the Security Credentials page. You can copy/paste that information from what is provided by Amazon.
Timeout: The time, in seconds, that the server will wait for a response from the remote Amazon server. The 30 second default will be sufficient for most backups, however that can be raised if you experience timeouts when uploading the backups.
11. Once those are filled out, click 'Save and Validate Destination'
12. If all of the information is added correctly, the backups page should show a message that says 'Success: Destination Saved'.
13. You should now be able to allow the backups to run and they should upload to your Amazon S3 account. You can use this command as root via ssh to force the backups to run.
Code:
/usr/local/cpanel/bin/backup --force
Code:
-bash-4.1# /usr/local/cpanel/bin/backup --force
info [backup] Started at Sun Sep 28 12:00:38 2014
info [backup] Process started in background.
info [backup] Log file: /usr/local/cpanel/logs/cpbackup/1411923638.log
Code:
tail -f /usr/local/cpanel/logs/cpbackup/1411920835.log
Code:
-bash-4.1# tail -f /usr/local/cpanel/logs/cpbackup_transporter.log
Code:
-bash-4.1# tail -f /usr/local/cpanel/logs/cpbackup_transporter.log
[2014-09-28 12:01:43 -0500] info [cpbackup_transporter] cPanel Backup Transporter Queue Daemon started.
[2014-09-28 12:01:43 -0500] info [cpbackup_transporter] cpbackup_transporter - started
[2014-09-28 12:01:43 -0500] info [cpbackup_transporter] cpbackup_transporter - Checking queue for tasks
[2014-09-28 12:01:43 -0500] info [cpbackup_transporter] cpbackup_transporter - Processing next task
[2014-09-28 12:01:43 -0500] info [cpbackup_transporter] Instantiating Object
[2014-09-28 12:01:43 -0500] info [cpbackup_transporter] Starting a "copy" operation on the "Amazon Backup" destination ID "rxpoY3vkuP1gI9ZYHemw8abH".
[2014-09-28 12:01:43 -0500] info [cpbackup_transporter] Validating destination path /Backups/2014-09-28
[2014-09-28 12:01:44 -0500] info [cpbackup_transporter] Path exists
[2014-09-28 12:01:44 -0500] info [cpbackup_transporter] Uploading /backup/2014-09-28/accounts/cptest2.tar.gz to /Backups/2014-09-28/cptest2.tar.gz
[2014-09-28 12:01:44 -0500] info [cpbackup_transporter] Attempting to upload /backup/2014-09-28/accounts/cptest2.tar.gz to /Backups/2014-09-28/cptest2.tar.gz for destination: Amazon Backup
[2014-09-28 12:01:44 -0500] info [cpbackup_transporter] Upload attempt #1 starting for /backup/2014-09-28/accounts/cptest2.tar.gz to /Backups/2014-09-28/cptest2.tar.gz for destination: Amazon Backup
[2014-09-28 12:01:45 -0500] info [cpbackup_transporter] Successful transfer of /backup/2014-09-28/accounts/cptest2.tar.gz to /Backups/2014-09-28/cptest2.tar.gz for destination Amazon BackupAmazon also provides a list of the possible error codes here, which will display in the errors if there are any problems when uploading the backups.
Error Responses - Amazon Simple Storage Service
