Backups not going through to Amazon

[ozfiddler]

I have set up an Amazon S3 account, got the parameters and entered them in WHM. When I click Validate it says that validation was successful.

But nothing is going through to Amazon. The folder there remains empty. There are backups in the backup folder on the server though. Is there some step I'm missing? I have all users set to "on" for Backup User Selection.

 

[cPanelMichael]

Hello

Please check the backup logs in "/usr/local/cpanel/logs/cpbackup" and the "/usr/local/cpanel/logs/cpbackup_transporter.log" file. Do you notice any particular error messages or entries related to the Amazon S3 backup destination?

Thank you.

 

[ozfiddler]

Thanks for the reply. The backups logs are the same as the ones that are emailed to me:

/bin/gtar: Removing leading `/' from member names
/bin/gtar: /var/lib/mysql/mysql.sock: socket ignored
/bin/gtar: Removing leading `/' from member names
/bin/gtar: Removing leading `/' from member names
/bin/gtar: Removing leading `/' from member names
/bin/gtar: Removing leading `/' from member names
/bin/gtar: Removing leading `/' from member names
/bin/gtar: Removing leading `/' from member names
/bin/gtar: /var/cpanel/dnsadmin/sock: socket ignored
/bin/gtar: Removing leading `/' from hard link targets
/bin/gtar: Removing leading `/' from member names
/bin/gtar: Removing leading `/' from member names
/bin/gtar: Removing leading `/' from member names
/bin/gtar: Removing leading `/' from member names
/bin/gtar: Removing leading `/' from member names
/bin/gtar: Removing leading `/' from member names
[backup] System backups will not be uploaded to any remote destinations
[backup] BACKUPACCTS disabled

What is BACKUPACCTS?

The other log file just shows a number of entries, all like this:

[2014-04-03 02:01:51 +1100] info [cpbackup_transporter] cPanel Backup Transporter Queue Daemon started.

[2014-04-03 02:01:54 +1100] info [cpbackup_transporter] cPanel Backup Transporter Queue Daemon is being stopped.

No mention of Amazon anywhere.

 

[ozfiddler]

OK - I tried setting up an ftp backup destination. It validated OK and it all looks good. I did a forced backup and nothing went through to my ftp location. I get a similar looking log file, including this line again:

[backup] System backups will not be uploaded to any remote destinations

 

[ozfiddler]

Ah.. I think I have it now. Am I right in thinking that you can't backup the system files at all to ANY remote location...? I tried the forced backup again to my ftp location only this time I added one account. The account came through OK but no system files.

Yet in the docs it says "If system backup is enabled, the system backup files are saved locally into the default backup directory (/backup). You can also upload system backup files to additional destinations."

 

[Jonathan More]

I am also interested in this, asked this in different string but hopefully will get answer here.
I get only accounts backed up to Amazon, not system files.

 

[ozfiddler]

Hooray! Adding that "upload_system_backup: 1" line to the backup configuration file does the trick, as explained in the other thread on this same issue.

It's all working now... thanks for your time.

 

[cPanelMichael]

Yes, system backups must be manually enabled for remote destinations. More information on this is available at:

System Backups

Thank you.

 

[Jonathan More]

Just to confirm, I need to add this upload_system_backup: 1 to amazon_cloud_UID... file that can be found in var/cpanel/backups directory?

 

[cPanelMichael]

Just to confirm, I need to add this upload_system_backup: 1 to amazon_cloud_UID... file that can be found in var/cpanel/backups directory?

Yes, that is correct.

Thanks.

 

[Jonathan More]

All is working now fine, except sometimes there comes error with one specific account. It gives backup transfer error "Unable to send account to destination Amazon Cloud". This account is big, about 8 Gb so maybe that has something to do with it?

 

[cPanelMichael]

All is working now fine, except sometimes there comes error with one specific account. It gives backup transfer error "Unable to send account to destination Amazon Cloud". This account is big, about 8 Gb so maybe that has something to do with it?

Are there any entries in /usr/local/cpanel/logs/cpbackup_transporter.log when this happens? Have you tried increasing the "Timeout" value for this backup destination?

Thank you.

 

[Jonathan More]

Thanks, I checked the log and there is error:
warn [cpbackup_transporter] Upload attempt failed: Amazon::S3: Amazon responded with 400 Bad Request.
There seems to be 3 attempts, all with 400 Bad Requests, until it skips to next account.

Where could I increase the timeout value?

 

[cPanelPeter]

Hello,

Please open a support ticket using the link in my signature. One of our analysts will investigate that for you. Please post the ticket number here when you have one so we can update this thread accordingly.

 

[Jonathan More]

Thanks, here is the ticket:
4984561

 

[cPanelMichael]

To update, it appears this issue was related to the Amazon server itself. The error code is referenced on their FAQ:

Amazon S3 Technical FAQs

The user was advised to contact their support to see if more information about the error is available.

Thank you.

 

[spaceman]

We have exactly the same here.... a larger account (10Gb) refusing to backup to S3, smaller accounts OK. Timing out we assume. cPanel have taken a look at our server, and believe that the problem lies at Amazon's end, not ours.

cPanel's advise to us is:

"I would recommend trying to set the timeout on the Amazon S3 backup destination much higher to see if that helps resolve the issue. You might try a setting like 3000 or higher to see if that helps."

The challenge I have right now is that, after initial Googling, it's far from obvious where in the S3 management console is the option to adjust the timeout value. Will keep looking...

 

[cPanelMichael]

Have you tried contacting their support team for assistance?

Thank you.

 

[spaceman]

Have you tried contacting their support team for assistance?

Thank you.

Good news, in the sense that I think we've found the problem:

Uploading Objects - Amazon Simple Storage Service
S3 Backup Destination - 5GB file size limit | cPanel Feature Requests

"Upload objects in a single operation—With a single PUT operation you can upload objects up to 5 GB in size."

So it's not a timeout issue per se. It's file size issue.