The Community Forums

Interact with an entire community of cPanel & WHM users!
  1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

What is a good Remote Backup solution?

Discussion in 'Data Protection' started by bgarrant, Aug 12, 2016.

  1. bgarrant

    bgarrant Active Member

    Joined:
    Jun 27, 2012
    Messages:
    26
    Likes Received:
    8
    Trophy Points:
    3
    cPanel Access Level:
    Root Administrator
    I want to offer remote backups as a service to my hosting clients. It would be great if it could be automated with WHMCS, but the most important part is that I can pick and choose what clients get the service.

    I see WHM has a backup remote option to S3 or SFTP. Is the built in backup solution the best way to do this or is there a plugin or service that is better?

    Also, what is a good storage provider? I need to backup about 80GB of data.

    I see many services online like BackupAgent and WHMEasyBackup. Any recommendations?
     
  2. Anthony Parsons

    Anthony Parsons Active Member

    Joined:
    Aug 4, 2016
    Messages:
    25
    Likes Received:
    1
    Trophy Points:
    3
    Location:
    Australia
    cPanel Access Level:
    Root Administrator
    I use S3 for mine as an off-site solution. I have daily backups kept for a week on my primary disk, purely for any local account issues, then all get sent to S3 each day too. The files in total are about 10Gb compressed, 40Gb if uncompressed. I needed an additional two cores head room for the backup to run effectively without creating a bottleneck for traffic... works effectively. Without those additional two cores for backup, original four core load was spiking to 15.00 and 20.00 respectively during the backup, with the additional two cores (6 core), now no more than 5.00 with global traffic as usual for my off-peak period.
     
  3. bgarrant

    bgarrant Active Member

    Joined:
    Jun 27, 2012
    Messages:
    26
    Likes Received:
    8
    Trophy Points:
    3
    cPanel Access Level:
    Root Administrator
    That sounds like a nice solution. So you use the native backup solution in WHM to generate your backup and send to S3? I am only looking to use this for clients that "opt-in" to this service as we already have 7 days local backups kept at our data center provider. Is it possible to just select the clients that get this service? Also, is the connection to S3 encrypted and secure for transfer? Thanks.
     
  4. Anthony Parsons

    Anthony Parsons Active Member

    Joined:
    Aug 4, 2016
    Messages:
    25
    Likes Received:
    1
    Trophy Points:
    3
    Location:
    Australia
    cPanel Access Level:
    Root Administrator
    Yes, S3 is a secured connection. See: Backup Configuration - Documentation - cPanel Documentation

    To my knowledge, no, you can't select accounts that it sends offsite. If the account is set to backup, and you set offsite backups, then it sends it.

    Saying that, looking at the S3 pricing, you don't get charged for sending data to S3, you only get charged for storing it and retrieving it. I haven't looked at whether a script could simply wipe out established backups automatically, so you aren't paying for the storage for those who aren't paying you.
     
  5. bgarrant

    bgarrant Active Member

    Joined:
    Jun 27, 2012
    Messages:
    26
    Likes Received:
    8
    Trophy Points:
    3
    cPanel Access Level:
    Root Administrator
    It looks like you can select what accounts get backed up so you can omit some that way. We do a bare metal complete backup and keep for 7 days already. I just want to offer the offsite backup option to clients that want it and charge them a small fee for it. S3 looks like about $0.03 per GB according to Cloud Storage Pricing – Amazon Simple Storage Service (S3) – AWS.
     
    Anthony Parsons likes this.
  6. bgarrant

    bgarrant Active Member

    Joined:
    Jun 27, 2012
    Messages:
    26
    Likes Received:
    8
    Trophy Points:
    3
    cPanel Access Level:
    Root Administrator
    Since we back up all accounts all ready server wide we just want to add the offsite backup for a few selected accounts. They would be the only ones that need it. Does this look possible with S3?

    We would need a way to delete backups over 30 days old on S3. Can cPanel do this natively?

    There are some programs like cpremote.net that may help with the deletion of old backups it seems if cPanel can't handle it.
     
    #6 bgarrant, Aug 13, 2016
    Last edited by a moderator: Aug 13, 2016
    Anthony Parsons likes this.
  7. Anthony Parsons

    Anthony Parsons Active Member

    Joined:
    Aug 4, 2016
    Messages:
    25
    Likes Received:
    1
    Trophy Points:
    3
    Location:
    Australia
    cPanel Access Level:
    Root Administrator
    Yes, if you're only using the WHM backup tool to do this additional service, then yes, that would do the job. They want that service, you just activate their account using the method you already outlined on a per user basis.

    I only use the WHM backup, so not running other backup solutions.
     
  8. bgarrant

    bgarrant Active Member

    Joined:
    Jun 27, 2012
    Messages:
    26
    Likes Received:
    8
    Trophy Points:
    3
    cPanel Access Level:
    Root Administrator
    Thanks Anthony. My hosting provider has had backups not work and the full server backup is pretty much useless as you have to restore whole server. Can you restore individual files from the WHM backup?

    I have a lot of drive space at my office and thinking about doing SFTP backups to here maybe. Is S3 very expensive? I have about 100 clients sites and 40 GB of data. Small sites mostly
     
    #8 bgarrant, Aug 13, 2016
    Last edited by a moderator: Aug 13, 2016
    Anthony Parsons likes this.
  9. bgarrant

    bgarrant Active Member

    Joined:
    Jun 27, 2012
    Messages:
    26
    Likes Received:
    8
    Trophy Points:
    3
    cPanel Access Level:
    Root Administrator
    How do you handle removing old copies on S3? I assume you have like a week and then purge old copies somehow?
     
    Anthony Parsons likes this.
  10. Anthony Parsons

    Anthony Parsons Active Member

    Joined:
    Aug 4, 2016
    Messages:
    25
    Likes Received:
    1
    Trophy Points:
    3
    Location:
    Australia
    cPanel Access Level:
    Root Administrator
    At present, I haven't, as I'm waiting to see whether WHM backup removes them when it removes files in the backup directory. Otherwise, I guess just manually in the S3 container or I will investigate an automated solution. I only just shifted from ServInt to Linode, managed to self managed... so still testing some solutions for myself.

    Yer, I had the same issue with ServInt in the past with needed backups. When i needed them, they had failed at their end.

    I'm with Linode now, so needing the entire servers files really isn't an issue, as you can deploy a linux version and WHM within 30 15 minutes, then you just have to restore the server settings backup, then each account. But you can do all that in one sitting. So within an hour or two, have the entire server restored if a catastrophic SSD failure occurred.

    If you really want, you can just install everything in a clean environment, all your additions and modules, everything but the accounts, clone the entire thing and let it sit on the smallest deployment version, which is $10 a month. I have thought about doing that, cloning my current setup, removing all the accounts, and just let it sit with an inactive WHM as an entire server copy. The more I think about that, writing this, I think I'm going to do just that...
    No, your options are everything, OR, just the files as you can uncheck the DB, mail and sub-domains reinstalling... so you don't lose new db content.

    I just did this with a nothing account to verify... and it worked as above. It restored the mysql user and permissions, but didn't touch the db itself. I made a change in the db to verify after restore, and it remained. So if your files needed to be restored, easy done.
     
  11. Anthony Parsons

    Anthony Parsons Active Member

    Joined:
    Aug 4, 2016
    Messages:
    25
    Likes Received:
    1
    Trophy Points:
    3
    Location:
    Australia
    cPanel Access Level:
    Root Administrator
  12. bgarrant

    bgarrant Active Member

    Joined:
    Jun 27, 2012
    Messages:
    26
    Likes Received:
    8
    Trophy Points:
    3
    cPanel Access Level:
    Root Administrator
    Thanks for the advice. I really appreciate it.
     
    Anthony Parsons likes this.
  13. cPanelMichael

    cPanelMichael Forums Analyst
    Staff Member

    Joined:
    Apr 11, 2011
    Messages:
    30,678
    Likes Received:
    653
    Trophy Points:
    113
    cPanel Access Level:
    Root Administrator
    Hello,

    I encourage you to add your input and vote for the existing feature request for this functionality at:

    Ability to control retention for S3 Backup Destination

    There's also a post you may find helpful on the above feature request about Amazon's lifecycle configuration:

    Lifecycle Configuration for a Bucket without Versioning - Amazon Simple Storage Service

    Thank you.
     
    Anthony Parsons likes this.
  14. bgarrant

    bgarrant Active Member

    Joined:
    Jun 27, 2012
    Messages:
    26
    Likes Received:
    8
    Trophy Points:
    3
    cPanel Access Level:
    Root Administrator
    So even though I have WHM set to just save 7 days, it won't delete the older versions at the moment on S3? Just want to understand the issue. I have 5 days now so have not got to day 7 yet.
     
    Anthony Parsons likes this.
  15. Anthony Parsons

    Anthony Parsons Active Member

    Joined:
    Aug 4, 2016
    Messages:
    25
    Likes Received:
    1
    Trophy Points:
    3
    Location:
    Australia
    cPanel Access Level:
    Root Administrator
    Ha... I reached 8 yesterday, and it deleted the first folder containing my first backup from my S3. It is only keeping 7 days backup folders in S3.
     
  16. bgarrant

    bgarrant Active Member

    Joined:
    Jun 27, 2012
    Messages:
    26
    Likes Received:
    8
    Trophy Points:
    3
    cPanel Access Level:
    Root Administrator
    That is fantastic! I really like this S3 integration in WHM.
     
  17. Anthony Parsons

    Anthony Parsons Active Member

    Joined:
    Aug 4, 2016
    Messages:
    25
    Likes Received:
    1
    Trophy Points:
    3
    Location:
    Australia
    cPanel Access Level:
    Root Administrator
    Me too, the moment I found out it worked with WHM's auto delete older backups. :)

    There was no way I wanted backups building up costing me $$$ for useless space.
     
  18. bgarrant

    bgarrant Active Member

    Joined:
    Jun 27, 2012
    Messages:
    26
    Likes Received:
    8
    Trophy Points:
    3
    cPanel Access Level:
    Root Administrator
    Do you know what the minimum permissions for IAM user is in Amazon AWS for WHM to do the backups? Read, write, delete on the bucket but any others? Just curious if you know. Thanks Anthony.
     
  19. Anthony Parsons

    Anthony Parsons Active Member

    Joined:
    Aug 4, 2016
    Messages:
    25
    Likes Received:
    1
    Trophy Points:
    3
    Location:
    Australia
    cPanel Access Level:
    Root Administrator
    No idea... I don't remember having to play with any permissions, I just use my main account, setup S3 bucket, connected it and away it works.
     
  20. bgarrant

    bgarrant Active Member

    Joined:
    Jun 27, 2012
    Messages:
    26
    Likes Received:
    8
    Trophy Points:
    3
    cPanel Access Level:
    Root Administrator
    If you use AWS for anything else you should set up IAM user and access codes. That way root can't be compromised. I just setup users with PowerUser permission on the individual bucket in use. Just an extra security step as the data is not encrypted.

    I was wondering if it would work if we enable encryption on the bucket in S3? Anyone know about that?
     
Loading...

Share This Page