The Community Forums

Interact with an entire community of cPanel & WHM users!
  1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

How to configure Amazon S3 backups in WHM

Discussion in 'Workarounds and Optimization' started by cPanelChrisI, Oct 15, 2014.

Tags:
  1. cPanelChrisI

    cPanelChrisI Technical Analyst
    Staff Member

    Joined:
    Mar 24, 2014
    Messages:
    2
    Likes Received:
    3
    Trophy Points:
    3
    cPanel Access Level:
    Root Administrator
    The purpose of this post is to provide a guide for setting up Amazon S3 backups in the WHM interface, as well as the steps to getting it set up in Amazon's interface. Amazon S3 is a service that is offered by Amazon that can be used for data storage, which also works well for storing cPanel backups. There, you are able to store your backups remotely on Amazon's servers as well as locally on the server. That will ensure that your backups are safe from being lost due to issues like hard drive failures. These are the steps that can be used to set that up.

    1. Sign up for an Amazon account at https://aws.amazon.com/

    2. Log into the Amazon account AWS Console at AWS Management Console

    3. Click on S3 under Storage & Content Delivery.

    Screen Shot 2014-09-28 at 12.27.13 PM.png

    4. Click Create Bucket to create a new bucket.

    Screen Shot 2014-09-28 at 12.29.16 PM.png

    5. Create a new directory using the 'Create Folder' button at the top left if you would like to store your backups in their own folder. I went ahead and created a folder named 'Backups' to store the backups in.

    Screen Shot 2014-09-28 at 12.31.20 PM.png Screen Shot 2014-09-28 at 12.31.48 PM.png

    6. At the top right, click on your name, then 'Security Credentials' and select 'Continue to Security Credentials' when the warning pops up.

    Screen Shot 2014-09-28 at 12.33.04 PM.png Screen Shot 2014-09-28 at 12.33.43 PM.png

    7. Click the + sign to the left of 'Access Keys (Access Key ID and Secret Access Key)', then click 'Create New Access Key' to create a new key pair. You will likely also want to download the file when prompted so that you can save the credentials, since you are no longer able to view them once the initial box with that information is closed. You can also view the access key from the popup that shows by clicking the 'Show Access Key' link, which is no longer able to be viewed once that box is closed.

    Screen Shot 2014-09-28 at 12.34.31 PM.png Screen Shot 2014-09-28 at 12.35.17 PM.png

    8. Log into WHM and go to Backup >> Backup Configuration and make sure that the backups are enabled for all of the users.

    Screen Shot 2014-09-28 at 12.36.10 PM.png Screen Shot 2014-09-28 at 12.36.45 PM.png Screen Shot 2014-09-28 at 12.38.56 PM.png

    9. At the very bottom of the backups page under 'Additional Destinations', ensure that the destination type in the box is 'Amazon S3', then click 'Create new destination'.

    Screen Shot 2014-09-28 at 12.39.35 PM.png

    10. Use this information in the boxes once that is selected:

    Destination name: Whatever you would like for you backup destination to be named in the WHM. This is your choice and does not affect the connection to the Amazon S3 server.

    Folder: Name of the folder that was created to store the backups if any. If that is left blank, the root of the bucket will be used.

    Bucket: Name of the bucket that was created that will store the backups. Please keep in mind that we currently have case 92993 open, which prevents Buckets with .'s to be unable to validate in the WHM backup configuration, so those must not contain any .'s in order to work correctly at this time and until that case is resolved.

    Access Key ID and Secret Access Key: Those are both provided under the 'Show Access Key' section on your amazon security credentials page and in the file that is downloaded when going to the Security Credentials page. You can copy/paste that information from what is provided by Amazon.

    Timeout: The time, in seconds, that the server will wait for a response from the remote Amazon server. The 30 second default will be sufficient for most backups, however that can be raised if you experience timeouts when uploading the backups.

    Screen Shot 2014-09-28 at 12.41.40 PM.png

    11. Once those are filled out, click 'Save and Validate Destination'

    Screen Shot 2014-09-28 at 12.42.20 PM.png

    12. If all of the information is added correctly, the backups page should show a message that says 'Success: Destination Saved'.

    Screen Shot 2014-09-28 at 12.43.14 PM.png

    13. You should now be able to allow the backups to run and they should upload to your Amazon S3 account. You can use this command as root via ssh to force the backups to run.

    Code:
    /usr/local/cpanel/bin/backup --force
    When that is run, it will provide the log that it is using for the backups.

    Code:
    -bash-4.1# /usr/local/cpanel/bin/backup --force
    info [backup] Started at Sun Sep 28 12:00:38 2014
    info [backup] Process started in background.
    info [backup] Log file: /usr/local/cpanel/logs/cpbackup/1411923638.log
    You can use a command like this to verify that the backup is running correctly.

    Code:
    tail -f /usr/local/cpanel/logs/cpbackup/1411920835.log
    You can also use this command to verify that the backups are uploaded correctly.

    Code:
    -bash-4.1# tail -f /usr/local/cpanel/logs/cpbackup_transporter.log
    That should show output that shows the uploads completing successfully.

    Code:
    -bash-4.1# tail -f /usr/local/cpanel/logs/cpbackup_transporter.log
    [2014-09-28 12:01:43 -0500] info [cpbackup_transporter] cPanel Backup Transporter Queue Daemon started.
    [2014-09-28 12:01:43 -0500] info [cpbackup_transporter] cpbackup_transporter - started
    [2014-09-28 12:01:43 -0500] info [cpbackup_transporter] cpbackup_transporter - Checking queue for tasks
    [2014-09-28 12:01:43 -0500] info [cpbackup_transporter] cpbackup_transporter - Processing next task
    [2014-09-28 12:01:43 -0500] info [cpbackup_transporter] Instantiating Object
    [2014-09-28 12:01:43 -0500] info [cpbackup_transporter] Starting a "copy" operation on the "Amazon Backup" destination ID "rxpoY3vkuP1gI9ZYHemw8abH".
    [2014-09-28 12:01:43 -0500] info [cpbackup_transporter] Validating destination path /Backups/2014-09-28
    [2014-09-28 12:01:44 -0500] info [cpbackup_transporter] Path exists
    [2014-09-28 12:01:44 -0500] info [cpbackup_transporter] Uploading /backup/2014-09-28/accounts/cptest2.tar.gz to /Backups/2014-09-28/cptest2.tar.gz
    [2014-09-28 12:01:44 -0500] info [cpbackup_transporter] Attempting to upload /backup/2014-09-28/accounts/cptest2.tar.gz to /Backups/2014-09-28/cptest2.tar.gz for destination: Amazon Backup
    [2014-09-28 12:01:44 -0500] info [cpbackup_transporter] Upload attempt #1 starting for /backup/2014-09-28/accounts/cptest2.tar.gz to /Backups/2014-09-28/cptest2.tar.gz for destination: Amazon Backup
    [2014-09-28 12:01:45 -0500] info [cpbackup_transporter] Successful transfer of /backup/2014-09-28/accounts/cptest2.tar.gz to /Backups/2014-09-28/cptest2.tar.gz for destination Amazon Backup
    14. Log into your Amazon S3 account and verify that the backups were uploaded correctly by clicking the 'Services' button at the top left of your Amazon account, then selecting S3. That will take you to your file tree. From there, you click on the folder that you chose to store the backups in, which should contain the backups that were uploaded from the server.

    Screen Shot 2014-09-28 at 12.50.00 PM.png Screen Shot 2014-09-28 at 12.51.26 PM.png



    Amazon also provides a list of the possible error codes here, which will display in the errors if there are any problems when uploading the backups.

    Error Responses - Amazon Simple Storage Service
     
    WireNine and huypham like this.
  2. sytrus

    sytrus Registered

    Joined:
    Nov 18, 2014
    Messages:
    1
    Likes Received:
    0
    Trophy Points:
    1
    cPanel Access Level:
    Root Administrator
    If you try to configure but getting this error:
    Change Amazon S3 server from Frankfurt, new S3 regions deployed after January, 2014 will not support V2 authorisation!
     
  3. jecro77

    jecro77 Member

    Joined:
    Mar 14, 2008
    Messages:
    6
    Likes Received:
    0
    Trophy Points:
    1
    Location:
    Switzerland
    Can it be, that your tool "cpbackup_transporter" upload the backups unsecured over http? The amazon log shows this here:

    [23/Nov/2014:02:00:43 +0000] IP arn:aws:iam::000:user/USER 000 REST.PUT.PART backups/2014-11-23/ACCOUNT.tar.gz "PUT /backups%2F2014-11-23%2FACCOUNT%2Etar%2Egz?partNumber=00&uploadId=00 HTTP/1.1" 200 - - 00 "-" "libwww-perl/6.05" -
     
  4. elias9

    elias9 Member

    Joined:
    Oct 30, 2014
    Messages:
    14
    Likes Received:
    0
    Trophy Points:
    1
    cPanel Access Level:
    Root Administrator
    Thank you so much.
     
  5. RDI

    RDI Registered

    Joined:
    Feb 20, 2014
    Messages:
    1
    Likes Received:
    0
    Trophy Points:
    1
    cPanel Access Level:
    Website Owner
    So are you saying that this is NOT a secure method backup ?

    Going into Amazon S3 ??

    if so any other suggestions ??

    - - -
     
  6. vdlwebdesign

    vdlwebdesign Registered

    Joined:
    Jan 15, 2016
    Messages:
    3
    Likes Received:
    0
    Trophy Points:
    1
    Location:
    south wales
    cPanel Access Level:
    Reseller Owner
    Hi I'm trying to link my AWS S3 into WHM to backup all my websites on a daily basis however, iv not got the option on my WHM for backup

    can you advise what i need to do?

    i have attached screenshots of my options on WHM
     

    Attached Files:

  7. Infopro

    Infopro cPanel Sr. Product Evangelist
    Staff Member

    Joined:
    May 20, 2003
    Messages:
    14,478
    Likes Received:
    203
    Trophy Points:
    63
    Location:
    Pennsylvania
    cPanel Access Level:
    Root Administrator
    Twitter:
    It appears you're logged in to WHM as a Reseller account with limited access to several things. If you've got root access also, you can modify what your Reseller has access to, in Reseller center in WHM.
     
  8. vdlwebdesign

    vdlwebdesign Registered

    Joined:
    Jan 15, 2016
    Messages:
    3
    Likes Received:
    0
    Trophy Points:
    1
    Location:
    south wales
    cPanel Access Level:
    Reseller Owner
    I only got access to this WHM Account where would i login to the full version?

    i can't find anywhere i can change the settings,

    thanks
    sion
     
  9. Infopro

    Infopro cPanel Sr. Product Evangelist
    Staff Member

    Joined:
    May 20, 2003
    Messages:
    14,478
    Likes Received:
    203
    Trophy Points:
    63
    Location:
    Pennsylvania
    cPanel Access Level:
    Root Administrator
    Twitter:
    You don't. If you have root access, and login as root, all settings on the menu should be there for backups (and Reseller Center for changing your Reseller account's available options).
     
  10. vdlwebdesign

    vdlwebdesign Registered

    Joined:
    Jan 15, 2016
    Messages:
    3
    Likes Received:
    0
    Trophy Points:
    1
    Location:
    south wales
    cPanel Access Level:
    Reseller Owner
    i login via my reseller dashboard or direct link how do i login as root?

    Im trying to backup all the websites and email accounts i have set up to AWS S3 i don't want to do this VIA a website plugin as i will only copy the website and database and would have to go into every site and set it up, if i set it up in WHM will is backup the full account?

    is there a way i can set the backup from the individual c-pannels?

    sion
     
  11. cPanelMichael

    cPanelMichael Forums Analyst
    Staff Member

    Joined:
    Apr 11, 2011
    Messages:
    30,854
    Likes Received:
    676
    Trophy Points:
    113
    cPanel Access Level:
    Root Administrator
    Hello :)

    The information you provided suggests you have reseller access only. You won't have access to the backup configuration options offered through Web Host Manager if you do not have "root" access to the server. You can backup an account manually using the "Backup Wizard" option in cPanel to a remote destination (FTP/SCP) if your web hosting provider has provided you with that feature.

    Thank you.
     
  12. billmcollam

    billmcollam Registered

    Joined:
    Jan 12, 2015
    Messages:
    1
    Likes Received:
    0
    Trophy Points:
    1
    cPanel Access Level:
    Reseller Owner
    Hi,
    I used this tutorial a few months ago to successfully set up my backup to S3. It works fine - S3 essentially has a copy of all my WHM backups. But what if I wanted to change the retention periods so that S3 (the cheaper space) retains the data longer?
    So for example I might want WHM to keep 3 most recent instances of daily backups but keep my weeklys and monthlies only on S3?
    would this involve fiddling with the transporter scripts?
     
  13. cPanelMichael

    cPanelMichael Forums Analyst
    Staff Member

    Joined:
    Apr 11, 2011
    Messages:
    30,854
    Likes Received:
    676
    Trophy Points:
    113
    cPanel Access Level:
    Root Administrator
    The following thread should help:

    Retain backups on Amazon S3 only

    Thank you.
     
  14. Henry Cobbllah

    Henry Cobbllah Registered

    Joined:
    Oct 16, 2016
    Messages:
    1
    Likes Received:
    0
    Trophy Points:
    1
    Location:
    Ghana
    cPanel Access Level:
    Root Administrator
    How much do the backups cost on s3 aws
     
  15. UHLHosting

    UHLHosting Well-Known Member

    Joined:
    Sep 26, 2014
    Messages:
    53
    Likes Received:
    4
    Trophy Points:
    8
    Location:
    Bratislava
    cPanel Access Level:
    Root Administrator
    Twitter:
    cPanelMichael likes this.
Loading...
Similar Threads - configure Amazon backups
  1. kernow
    Replies:
    1
    Views:
    1,323

Share This Page