New Backup Destination Timeout

[mhofer]

I'm Trying to create a custom destination. It works so far, one thing i got problems is the Timeout we can specify in the webinterface.

According to the documentation the maximum timeout is 300 seconds, however it can take more to transfer some users to the other server which will result in additional put-calls and eventually an error...

is there some way to increase this or can the custom script report something back like 'i'm still running, please wait...'?

regards,
Marcel

 

[cPanelMichael]

Hello

The timeout value should not be an issue if an active upload is occurring. Are you sure the timeout is not coming from the FTP configuration on the remote server? What specific error message do you see?

Thank you.

 

[mhofer]

hi,

the destination is actually a custom backup utility called 'duplicity' wrapped around a bash-script which runs a sync-job for the current file. when this is running for more than 300 seconds the backup calls my custom script again, and after another 300 seconds again, etc. until it disables my custom destination due to too many errors, however the sync was still running.

each time the script gets called another sync-job starts and messes up the destination.

here is the log of calls from a test-run (with 30s timeout to demonstrate):

20:26:56: /backup/test.sh put / /var/cpanel/backups/tmp.test.8167-1370370416 test/validate.tmp-8167-1370370416.txt
20:27:00: /backup/test.sh get / test/validate.tmp-8167-1370370416.txt /var/cpanel/backups/tmp.test.8167-1370370416.copy
20:27:02: /backup/test.sh delete / test/validate.tmp-8167-1370370416.txt
20:29:16: /backup/test.sh ls / test/2013-06-04
20:29:17: /backup/test.sh mkdir / test/2013-06-04
20:29:17: /backup/test.sh put / /backup/2013-06-04/accounts/rsternig.tar test/2013-06-04/rsternig.tar
20:29:49: /backup/test.sh put / /backup/2013-06-04/accounts/rsternig.tar test/2013-06-04/rsternig.tar
20:31:35: /backup/test.sh ls / test/2013-06-04
20:31:36: /backup/test.sh mkdir / test/2013-06-04
20:31:36: /backup/test.sh put / /backup/2013-06-04/accounts/kah.tar test/2013-06-04/kah.tar
20:32:09: /backup/test.sh put / /backup/2013-06-04/accounts/kah.tar test/2013-06-04/kah.tar
20:32:57: /backup/test.sh put / /backup/2013-06-04/accounts/kah.tar test/2013-06-04/kah.tar

the backup continues but will not use the custom-script anymore. this is what i recieve per e-mail after backup completion:

+===================================+
| Transport Disabled |
+===================================+
| Name: duplicity test
| Typ: Custom
| Remote Host:
|
| Reason: Transport surpassed error threshhold with “4” errors
+===================================+

do i understand something wrong about the custom backup destination?

 

[cPanelMichael]

Could you open a bug report for this issue so we can have our developers determine if this behavior should be modified, or if an additional option should be made available to disable the timeout? You can open a bug report via:

Submit A Bug Report

Please post the ticket number you receive here so we can keep this thread updated.

Thank you.

 

[mhofer]

Hi Michael

I just tried to submit a bug-report however i seem to be unable in chrome and firefox :/

When i click 'Continue' after filling out everything nothing happens (firebug shows a javascript error: ReferenceError: display_next_step is not defined).
When i click Enter on the IP-Field i receive the following error message:
The provided IP and support access id do not match.

However i cant specify a support-id and i was able to submit a ticket for the same server ip under the same account a few weeks ago... seems like the bug-report system is bugged?

What should i do?

 

[cPanelMichael]

Could you open up the bug report as a support ticket, and include the information you provided about not being able to submit it to the bug report queue? This will allow us to address both issues.

Thank you.

 

[mhofer]

I have done so, ticket #4192481

looking forward on a solution

thanks you

 

[cPanelMichael]

To update, there is an internal case open with our developers to increase the backup timeout value in seconds up to 50000. For reference, the case number is 69433.

Thank you.

 

[schoeps]

I am getting the same error "Reason: Transport surpassed error threshhold with “4” errors", will increasing the timout value help?

 

[cPanelMichael]

I am getting the same error "Reason: Transport surpassed error threshhold with “4” errors", will increasing the timout value help?

Are your backups large enough that it's legitimately taking an extended time frame to send them to the remote destination? The ability to adjust the timeout value is implemented with case 69433 in cPanel 11.38.0.17. This version is currently not yet available in the "Release" build tier.

Thank you.

 

[mhofer]

Hi Michael

Just saw the note about case 69433 in the changelog, however after upgrading I can still only set timeouts between 30-300...?

@schoeps: increasing the timeout might help, but not solve your problem unless you have a maximum account size which is always transferred within that timeout... also your destination might not work properly, so no timeout will help, only fixing the destination

 

[cPanelMichael]

Just saw the note about case 69433 in the changelog, however after upgrading I can still only set timeouts between 30-300...?

What version of cPanel are you using? The change implemented with case 69433 is currently only available in cPanel 11.38.0.17 (Edge and Current build tiers). In "WHM Home » Backup » Backup Configuration", you should see an option called "Advanced Settings" that you can select to see the "Maximum destination timeout" value.

Thank you.

 

[mhofer]

ah found it, thank you!
i was looking at Backup -> Backup Configuration -> Additional Destination
There was a timeout too... how do those 2 timeouts effect each other?

 

[schoeps]

Are your backups large enough that it's legitimately taking an extended time frame to send them to the remote destination? The ability to adjust the timeout value is implemented with case 69433 in cPanel 11.38.0.17. This version is currently not yet available in the "Release" build tier.

Thank you.

Thanks for responding. Yes, the backups exceed 8GB on some accounts, legacy had no problem here.

 

[cPanelMichael]

Case 69433 deals with the backup destination time out value. If a separate timeout value is still causing you problems, please open a new ticket dealing with that specific time out.

Thank you.

 

[schoeps]

Case 69433 deals with the backup destination time out value. If a separate timeout value is still causing you problems, please open a new ticket dealing with that specific time out.

Thank you.

Build 17 does not support numbers higher than 300.

 

[cPanelMichael]

Build 17 does not support numbers higher than 300.

What specific value are you attempting to modify?

Thank you.

 

[schoeps]

What specific value are you attempting to modify?

Thank you.

The "Timeout" value for remote backup/ additional destination.

 

[cPanelMichael]

The "Timeout" value for remote backup/ additional destination.

Please paste the output of the following command:

# cat /usr/local/cpanel/version

Also, to confirm, you are not able to set the "Maximum destination timeout" under "Advanced Settings" in "WHM Home » Backup » Backup Configuration" to a value higher than 300?

Thank you.

 

[schoeps]

Please paste the output of the following command:

# cat /usr/local/cpanel/version

Also, to confirm, you are not able to set the "Maximum destination timeout" under "Advanced Settings" in "WHM Home » Backup » Backup Configuration" to a value higher than 300?

Thank you.

I misread the thread. I thought you had said build 17 would allow for the "50000" fix, but apparently, not. All is correct, but those big transfers are not working. Any ETA on the patch?