Problem creating Amazon S3 backup destination in 11.42

[keddie]

I'm experimenting with setting up cpanel backups to an s3 bucket using the new functionality in 11.42

I have setup a new bucket called cpanelbackups.mydomain.com

In WHM backup configuration, I have:

- Created a new destination
- Named it "S3 Backups"
- Added cpanelbackups.mydomain.com to "Bucket Name"
- Added access and secret keys

When I click "Save & validate" the response is: "Error: Connection to server timed out"

Do I need to specify the full bucket endpoint in "Bucket name"?

My bucket is located in Ireland / EU West.

I have a feeling I'm missing something very obvious here

 

[cPanelMichael]

Hello

Could you try using the full bucket URL and letting us know the outcome? Also, have you tried pinging the URL from the server's command line and making sure it's not blocked?

Thank you.

 

[jeffcan]

Hello,
I have been experiencing same problem as well

- I have created new bucket
- I have created new user and gave him a permission to control s3
- I added my details to cPanel but I can't validate it

any advice?
thank you

 

[slim]

IM having the same issue - Would love to know why.

 

[cPanelMichael]

Feel free to open a support ticket so we can take a closer look and attempt to reproduce the issue. You can post the ticket number here so we can update this thread with the outcome.

Thank you.

 

[jeffcan]

thank you Michael. I have created ticket regarding this issue
Support Request Id 4633909

 

[slim]

I was able to get it to work - I think it was something to do with the way I was copying and pasting the Access Key ID and Secret Access Key's.

However now that its setup and validating ok, the backup still fails. It successfully transfers 1 of 2 backups. The second backup, which is much bigger than the first - 3.9 gig fails with this error:

[2014-03-07 02:41:18 +1100] warn [cpbackup_transporter] Upload attempt failed: read failed: Connection reset by peer

I have a ticket open, but so far, cPanel are of little help.

Support told me to check my firewall and to add the IP of the 'ftp' server into it. Well, its not an FTP server, its S3, and why would it only fail after successfully transferring the first backup file!

#4634071

 

[slim]

I discovered that the bucket I created was across the other side of the world, and it appears the transfer of the larger backup was failing. I have now created a bucket in the same state as my server and am trying again.

I really cant see why the distance should make a difference - It should be fault tolerant and able to recover from disconnects/timeouts etc and still be able to complete the backup!

- - - Updated - - -

If it helps anyone - You should create an IAM user and apply the permissions to that user (S3 full access).. then use the credentials of that IAM user in WHM.

 

[jeffcan]

Hello Slim,
I did same thing.

Created IAM user and applied s3 full access permission and used users credentials in WHM but still receiving time-out message.

 

[slim]

I also think it takes a while after creating the credentials for them to 'work'..

 

[jeffcan]

Hello All,
I was able to validate my settings. cPanel support was really helpful and they were able to address the problem.
If you use . (dot) in your bucket name, you can't connect to amazon servers due SSL issue...

So guys have to make sure that there is no DOT in your bucket name.


@slim,
now, it's time to see if my backup will fail like yours. I have server on gigabit network and heavy duty backup files!

 

[cPanelMichael]

I am happy to see the issue has been resolved. Thank you for updating us with the outcome.

 

[cPanelPeter]

Hello,

Regarding ticket 4634071 (for Slim), I see that Melanie was able to assist you and it would appear that the issue might indeed have something to do with distance from your location to the Amazon S3 Bucket. Amazon does mention on their page that you should choose a bucket location close to your vicinity.

 

[slim]

Ah yes - The dot in the bucket name would explain why it didnt work for me the first time.. That should be front and center in the settings page in WHM - 'Warning - dont use bucket names with dots in them!'..

After setting the bucket name without a dot and making the bucket in Sydney (close to my servers) the problems went away.

Now the only problem I have is the complete lack of settings WHM gives us - It is currently unworkable. There needs to be more flexible options regarding what goes to S3, how often etc.

 

[cPanelMichael]

Now the only problem I have is the complete lack of settings WHM gives us - It is currently unworkable. There needs to be more flexible options regarding what goes to S3, how often etc.

I encourage you to open a feature request for specific options you feel should be available:

Submit A Feature Request

Thank you.

 

[cPanelKenneth]

Case 92993 was opened regarding the username validation issue.

 

[briansol]

Amazon does mention on their page that you should choose a bucket location close to your vicinity.

for disaster recovery purposes, I do the exact opposite. I store from my server in wash. DC to Oregon. If dc gets nuked, I can still pull a backup from the other shore.
Or if the tsunami wipes out Oregon, I still have local backups in DC.

Never let a region be your point of failure, even if it takes longer to move the data there.

 

[rajuru]

Hello All,
I was able to validate my settings. cPanel support was really helpful and they were able to address the problem.
If you use . (dot) in your bucket name, you can't connect to amazon servers due SSL issue...

So guys have to make sure that there is no DOT in your bucket name.


@slim,
now, it's time to see if my backup will fail like yours. I have server on gigabit network and heavy duty backup files!

You don't know how much you helped by mentioning this. Thanks a lot. It also means how bad cpanel ui/ux is. This hard limitation should be mentioned just right of the field.