In Progress CPANEL-40457 - Restarting cpbackup_transporter skips the current backup file

AGY

Member
Sep 15, 2019
10
0
1
Virginia
cPanel Access Level
Root Administrator
I occasionally see an issue where uploads to Backblaze slow or perhaps hang. I then restart the process with cpbackup_transporter --stop and then --start. It would appear that the shutdown process marks the file in progress done, because when it starts back up it grabs the next file.

[2022-04-11 15:29:56 -0400] info [cpbackup_transporter] Uploading account backup /backup/2022-04-11/accounts/account1.tar.gz to 2022-04-11/accounts/account1.tar.gz (from 2022-04-11/accounts/account1.tar.gz)
[2022-04-11 15:29:56 -0400] info [cpbackup_transporter] Attempting to upload /backup/2022-04-11/accounts/account1.tar.gz to 2022-04-11/accounts/account1.tar.gz for destination: Backblaze-CP
[2022-04-11 15:29:56 -0400] info [cpbackup_transporter] Upload attempt #1 starting for /backup/2022-04-11/accounts/account1.tar.gz to 2022-04-11/accounts/account1.tar.gz for destination: Backblaze-CP

# /usr/local/cpanel/bin/cpbackup_transporter --stop
[2022-04-11 16:44:25 -0400] info [cpbackup_transporter] Initializing log file
Graceful shutdown of cPanel Backup Transporter Queue Daemon requested.
Graceful shutdown due to SIGHUP 34064
[2022-04-11 16:44:25 -0400] info [cpbackup_transporter] Graceful shutdown due to SIGHUP
Shutdown complete
[2022-04-11 16:44:25 -0400] info [cpbackup_transporter] cpbackup_transporter - Checking queue for tasks
[2022-04-11 16:44:25 -0400] info [cpbackup_transporter] cPanel Backup Transporter Queue Daemon is being stopped.
[2022-04-11 16:45:38 -0400] info [cpbackup_transporter] cpbackup_transporter - started
[2022-04-11 16:45:38 -0400] info [cpbackup_transporter] cpbackup_transporter - Checking queue for tasks
[2022-04-11 16:45:38 -0400] info [cpbackup_transporter] cpbackup_transporter - Processing next task
[2022-04-11 16:45:38 -0400] info [cpbackup_transporter] Instantiating Object
[2022-04-11 16:45:38 -0400] info [cpbackup_transporter] Starting a "copy" operation on the "Backblaze-CP" destination ID "<snip>".
[2022-04-11 16:45:38 -0400] info [cpbackup_transporter] Validating destination path 2022-04-11/accounts/
[2022-04-11 16:45:40 -0400] info [cpbackup_transporter] Path exists
[2022-04-11 16:45:40 -0400] info [cpbackup_transporter] Uploading account backup /backup/2022-04-11/accounts/account2.tar.gz to 2022-04-11/accounts/account2.tar.gz (from 2022-04-11/accounts/account2.tar.gz)

When the transport finishes the skipped files will still exist in the backup directory and are not present in the Backblaze bucket.

cpanel_and_whm: 11.102.0
operating_system_name: centos
operating_system_version: 7
 

AGY

Member
Sep 15, 2019
10
0
1
Virginia
cPanel Access Level
Root Administrator
I made the switch last night and the troubles continue. I'm investigating the throughput issues with my provider (I can get 30 MB/s for a while and then it throttles back to a couple hundred KB/s), but the skipping backup bit is still the case. A backup being transported gets "lost" when cpbackup_transporter is stopped and restarted.
 

phil99

Well-Known Member
Jun 10, 2018
57
22
58
UK
cPanel Access Level
Root Administrator
We have also seen this behaviour of losing the backup that was in progress when you stop and restart the cpbackup_transporter process. Again this was with Backblaze B2 using the B2 protocol. (It is not easy for us to swap protocols, but the bucket is not S3 enabled.)