New Backup Destination Timeout

mhofer

Member
May 10, 2013
16
0
1
cPanel Access Level
Root Administrator
I'm Trying to create a custom destination. It works so far, one thing i got problems is the Timeout we can specify in the webinterface.

According to the documentation the maximum timeout is 300 seconds, however it can take more to transfer some users to the other server which will result in additional put-calls and eventually an error...

is there some way to increase this or can the custom script report something back like 'i'm still running, please wait...'?

regards,
Marcel
 

cPanelMichael

Administrator
Staff member
Apr 11, 2011
47,913
2,201
363
Hello :)

The timeout value should not be an issue if an active upload is occurring. Are you sure the timeout is not coming from the FTP configuration on the remote server? What specific error message do you see?

Thank you.
 

mhofer

Member
May 10, 2013
16
0
1
cPanel Access Level
Root Administrator
hi,

the destination is actually a custom backup utility called 'duplicity' wrapped around a bash-script which runs a sync-job for the current file. when this is running for more than 300 seconds the backup calls my custom script again, and after another 300 seconds again, etc. until it disables my custom destination due to too many errors, however the sync was still running.

each time the script gets called another sync-job starts and messes up the destination.

here is the log of calls from a test-run (with 30s timeout to demonstrate):
20:26:56: /backup/test.sh put / /var/cpanel/backups/tmp.test.8167-1370370416 test/validate.tmp-8167-1370370416.txt
20:27:00: /backup/test.sh get / test/validate.tmp-8167-1370370416.txt /var/cpanel/backups/tmp.test.8167-1370370416.copy
20:27:02: /backup/test.sh delete / test/validate.tmp-8167-1370370416.txt
20:29:16: /backup/test.sh ls / test/2013-06-04
20:29:17: /backup/test.sh mkdir / test/2013-06-04
20:29:17: /backup/test.sh put / /backup/2013-06-04/accounts/rsternig.tar test/2013-06-04/rsternig.tar
20:29:49: /backup/test.sh put / /backup/2013-06-04/accounts/rsternig.tar test/2013-06-04/rsternig.tar
20:31:35: /backup/test.sh ls / test/2013-06-04
20:31:36: /backup/test.sh mkdir / test/2013-06-04
20:31:36: /backup/test.sh put / /backup/2013-06-04/accounts/kah.tar test/2013-06-04/kah.tar
20:32:09: /backup/test.sh put / /backup/2013-06-04/accounts/kah.tar test/2013-06-04/kah.tar
20:32:57: /backup/test.sh put / /backup/2013-06-04/accounts/kah.tar test/2013-06-04/kah.tar
the backup continues but will not use the custom-script anymore. this is what i recieve per e-mail after backup completion:
+===================================+
| Transport Disabled |
+===================================+
| Name: duplicity test
| Typ: Custom
| Remote Host:
|
| Reason: Transport surpassed error threshhold with “4” errors
+===================================+
do i understand something wrong about the custom backup destination?
 

cPanelMichael

Administrator
Staff member
Apr 11, 2011
47,913
2,201
363
Could you open a bug report for this issue so we can have our developers determine if this behavior should be modified, or if an additional option should be made available to disable the timeout? You can open a bug report via:

Submit A Bug Report

Please post the ticket number you receive here so we can keep this thread updated.

Thank you.
 

mhofer

Member
May 10, 2013
16
0
1
cPanel Access Level
Root Administrator
Hi Michael

I just tried to submit a bug-report however i seem to be unable in chrome and firefox :/

When i click 'Continue' after filling out everything nothing happens (firebug shows a javascript error: ReferenceError: display_next_step is not defined).
When i click Enter on the IP-Field i receive the following error message:
The provided IP and support access id do not match.

However i cant specify a support-id and i was able to submit a ticket for the same server ip under the same account a few weeks ago... seems like the bug-report system is bugged? ;)

What should i do?
 

cPanelMichael

Administrator
Staff member
Apr 11, 2011
47,913
2,201
363
Could you open up the bug report as a support ticket, and include the information you provided about not being able to submit it to the bug report queue? This will allow us to address both issues.

Thank you.
 

cPanelMichael

Administrator
Staff member
Apr 11, 2011
47,913
2,201
363
To update, there is an internal case open with our developers to increase the backup timeout value in seconds up to 50000. For reference, the case number is 69433.

Thank you.
 

schoeps

Well-Known Member
Sep 22, 2004
81
7
158
I am getting the same error "Reason: Transport surpassed error threshhold with “4” errors", will increasing the timout value help?
 

cPanelMichael

Administrator
Staff member
Apr 11, 2011
47,913
2,201
363
I am getting the same error "Reason: Transport surpassed error threshhold with “4” errors", will increasing the timout value help?
Are your backups large enough that it's legitimately taking an extended time frame to send them to the remote destination? The ability to adjust the timeout value is implemented with case 69433 in cPanel 11.38.0.17. This version is currently not yet available in the "Release" build tier.

Thank you.
 

mhofer

Member
May 10, 2013
16
0
1
cPanel Access Level
Root Administrator
Hi Michael

Just saw the note about case 69433 in the changelog, however after upgrading I can still only set timeouts between 30-300...?

@schoeps: increasing the timeout might help, but not solve your problem unless you have a maximum account size which is always transferred within that timeout... also your destination might not work properly, so no timeout will help, only fixing the destination
 

cPanelMichael

Administrator
Staff member
Apr 11, 2011
47,913
2,201
363
Just saw the note about case 69433 in the changelog, however after upgrading I can still only set timeouts between 30-300...?
What version of cPanel are you using? The change implemented with case 69433 is currently only available in cPanel 11.38.0.17 (Edge and Current build tiers). In "WHM Home » Backup » Backup Configuration", you should see an option called "Advanced Settings" that you can select to see the "Maximum destination timeout" value.

Thank you.
 

mhofer

Member
May 10, 2013
16
0
1
cPanel Access Level
Root Administrator
ah found it, thank you!
i was looking at Backup -> Backup Configuration -> Additional Destination
There was a timeout too... how do those 2 timeouts effect each other?
 

schoeps

Well-Known Member
Sep 22, 2004
81
7
158
Are your backups large enough that it's legitimately taking an extended time frame to send them to the remote destination? The ability to adjust the timeout value is implemented with case 69433 in cPanel 11.38.0.17. This version is currently not yet available in the "Release" build tier.

Thank you.

Thanks for responding. Yes, the backups exceed 8GB on some accounts, legacy had no problem here.
 

cPanelMichael

Administrator
Staff member
Apr 11, 2011
47,913
2,201
363
Case 69433 deals with the backup destination time out value. If a separate timeout value is still causing you problems, please open a new ticket dealing with that specific time out.

Thank you.
 

schoeps

Well-Known Member
Sep 22, 2004
81
7
158
Case 69433 deals with the backup destination time out value. If a separate timeout value is still causing you problems, please open a new ticket dealing with that specific time out.

Thank you.

Build 17 does not support numbers higher than 300.
 

cPanelMichael

Administrator
Staff member
Apr 11, 2011
47,913
2,201
363
The "Timeout" value for remote backup/ additional destination.
Please paste the output of the following command:

Code:
# cat /usr/local/cpanel/version
Also, to confirm, you are not able to set the "Maximum destination timeout" under "Advanced Settings" in "WHM Home » Backup » Backup Configuration" to a value higher than 300?

Thank you.
 

schoeps

Well-Known Member
Sep 22, 2004
81
7
158
Please paste the output of the following command:

Code:
# cat /usr/local/cpanel/version
Also, to confirm, you are not able to set the "Maximum destination timeout" under "Advanced Settings" in "WHM Home » Backup » Backup Configuration" to a value higher than 300?

Thank you.
I misread the thread. I thought you had said build 17 would allow for the "50000" fix, but apparently, not. All is correct, but those big transfers are not working. Any ETA on the patch?