Google Drive - Backup transport errors

[Art0311]

Hi,

I've been using Google Drive to backup WHM for more than a year.

It started having "Backup transport errors" from 7 Dec. Upon checking the Google Drive, WHM managed to upload some accounts but not everything as per settings.
Based on cpbackup_transporter.log, it started with error 429 (about 4-10 lines) then followed by error 502 and 503 for each backup file.

Also, there's below error message for each backup:
Upload attempt failed: malformed JSON string, neither tag, array, object, number, string or atom, at character offset 0 (before "(end of string)") at /usr/local/cpanel/3rdparty/perl/532/lib/perl5/cpanel_lib/JSON.pm line 190.

 

[cPRex]

Hey there! Can you post the "429" error message you're seeing so we can have more details on that?

 

[Art0311]

Sure, below is the complete cpbackup_transporter.log



[2022-12-19 02:54:31 +0800] info [cpbackup_transporter] Initializing log file
[2022-12-19 02:54:31 +0800] info [cpbackup_transporter] cpbackup_transporter - parent starting
[2022-12-19 02:54:31 +0800] info [cpbackup_transporter] cpbackup_transporter - child starting
[2022-12-19 02:54:31 +0800] info [cpbackup_transporter] cPanel Backup Transporter Queue Daemon started.
[2022-12-19 02:54:31 +0800] info [cpbackup_transporter] cpbackup_transporter - started
[2022-12-19 02:54:31 +0800] info [cpbackup_transporter] cpbackup_transporter - Checking queue for tasks
[2022-12-19 02:54:31 +0800] info [cpbackup_transporter] cpbackup_transporter - Processing next task
[2022-12-19 02:54:31 +0800] info [cpbackup_transporter] Instantiating Object
[2022-12-19 02:54:32 +0800] info [cpbackup_transporter] Starting a "copy" operation on the "Destination Name" destination ID "DestinationID".
[2022-12-19 02:54:32 +0800] info [cpbackup_transporter] Validating destination path /GoogleDrivePath/2022-12-19/accounts/
[2022-12-19 02:54:36 +0800] info [cpbackup_transporter] Path exists
[2022-12-19 02:54:36 +0800] info [cpbackup_transporter] Uploading account backup /backup/2022-12-19/accounts/account1.tar.gz to /GoogleDrivePath/2022-12-19/accounts/account1.tar.gz (from 2022-12-19/accounts/account1.tar.gz)
[2022-12-19 02:54:36 +0800] info [cpbackup_transporter] Attempting to upload /backup/2022-12-19/accounts/account1.tar.gz to /GoogleDrivePath/2022-12-19/accounts/account1.tar.gz for destination: Destination Name
[2022-12-19 02:54:36 +0800] info [cpbackup_transporter] Upload attempt #1 starting for /backup/2022-12-19/accounts/account1.tar.gz to /GoogleDrivePath/2022-12-19/accounts/account1.tar.gz for destination: Destination Name
[2022-12-19 02:54:47 +0800] info [cpbackup_transporter] Failed with 429: Too Many Requests
[2022-12-19 02:55:06 +0800] info [cpbackup_transporter] Failed with 429: Too Many Requests
[2022-12-19 03:09:16 +0800] warn [cpbackup_transporter] Upload attempt failed: malformed JSON string, neither tag, array, object, number, string or atom, at character offset 0 (before "(end of string)") at /usr/local/cpanel/3rdparty/perl/532/lib/perl5/cpanel_lib/JSON.pm line 190.

[2022-12-19 03:09:16 +0800] info [cpbackup_transporter] Upload attempt #2 starting for /backup/2022-12-19/accounts/account1.tar.gz to /GoogleDrivePath/2022-12-19/accounts/account1.tar.gz for destination: Destination Name
[2022-12-19 03:09:30 +0800] info [cpbackup_transporter] Failed with 429: Too Many Requests
[2022-12-19 03:09:45 +0800] info [cpbackup_transporter] Failed with 429: Too Many Requests
[2022-12-19 03:10:01 +0800] info [cpbackup_transporter] Failed with 429: Too Many Requests
[2022-12-19 03:10:20 +0800] info [cpbackup_transporter] Failed with 429: Too Many Requests
[2022-12-19 03:10:20 +0800] warn [cpbackup_transporter] Upload attempt failed: upload_file_content_single() missing upload_uri at /usr/local/cpanel/3rdparty/perl/532/lib/perl5/cpanel_lib/Net/Google/Drive/Simple/V3.pm line 996

[2022-12-19 03:10:20 +0800] info [cpbackup_transporter] Upload attempt #3 starting for /backup/2022-12-19/accounts/account1.tar.gz to /GoogleDrivePath/2022-12-19/accounts/account1.tar.gz for destination: Destination Name
[2022-12-19 03:10:33 +0800] info [cpbackup_transporter] Failed with 429: Too Many Requests
[2022-12-19 03:10:52 +0800] info [cpbackup_transporter] Failed with 429: Too Many Requests
[2022-12-19 03:11:10 +0800] info [cpbackup_transporter] Failed with 429: Too Many Requests
[2022-12-19 03:11:29 +0800] info [cpbackup_transporter] Failed with 429: Too Many Requests
[2022-12-19 03:11:29 +0800] warn [cpbackup_transporter] Upload attempt failed: upload_file_content_single() missing upload_uri at /usr/local/cpanel/3rdparty/perl/532/lib/perl5/cpanel_lib/Net/Google/Drive/Simple/V3.pm line 996

[2022-12-19 03:11:29 +0800] info [cpbackup_transporter] cpbackup_transporter - Processing next task
[2022-12-19 03:11:29 +0800] info [cpbackup_transporter] Instantiating Object
[2022-12-19 03:11:29 +0800] info [cpbackup_transporter] Starting a "copy" operation on the "Destination Name" destination ID "DestinationID".
[2022-12-19 03:11:29 +0800] info [cpbackup_transporter] Validating destination path /GoogleDrivePath/2022-12-19/accounts/
[2022-12-19 03:11:32 +0800] info [cpbackup_transporter] Path exists
[2022-12-19 03:11:32 +0800] info [cpbackup_transporter] Uploading account backup /backup/2022-12-19/accounts/account2.tar.gz to /GoogleDrivePath/2022-12-19/accounts/account2.tar.gz (from 2022-12-19/accounts/account2.tar.gz)
[2022-12-19 03:11:32 +0800] info [cpbackup_transporter] Attempting to upload /backup/2022-12-19/accounts/account2.tar.gz to /GoogleDrivePath/2022-12-19/accounts/account2.tar.gz for destination: Destination Name
[2022-12-19 03:11:32 +0800] info [cpbackup_transporter] Upload attempt #1 starting for /backup/2022-12-19/accounts/account2.tar.gz to /GoogleDrivePath/2022-12-19/accounts/account2.tar.gz for destination: Destination Name
[2022-12-19 03:11:44 +0800] info [cpbackup_transporter] Failed with 429: Too Many Requests
[2022-12-19 03:11:55 +0800] info [cpbackup_transporter] Failed with 502: Bad Gateway
[2022-12-19 03:12:14 +0800] info [cpbackup_transporter] Failed with 429: Too Many Requests
[2022-12-19 03:12:32 +0800] info [cpbackup_transporter] Failed with 429: Too Many Requests
[2022-12-19 03:12:32 +0800] warn [cpbackup_transporter] Upload attempt failed: upload_file_content_single() missing upload_uri at /usr/local/cpanel/3rdparty/perl/532/lib/perl5/cpanel_lib/Net/Google/Drive/Simple/V3.pm line 996

[2022-12-19 03:12:32 +0800] info [cpbackup_transporter] Upload attempt #2 starting for /backup/2022-12-19/accounts/account2.tar.gz to /GoogleDrivePath/2022-12-19/accounts/account2.tar.gz for destination: Destination Name
[2022-12-19 03:12:47 +0800] info [cpbackup_transporter] Failed with 429: Too Many Requests
[2022-12-19 03:13:03 +0800] info [cpbackup_transporter] Failed with 429: Too Many Requests
[2022-12-19 03:13:14 +0800] info [cpbackup_transporter] Failed with 502: Bad Gateway
[2022-12-19 03:13:30 +0800] info [cpbackup_transporter] Failed with 429: Too Many Requests
[2022-12-19 03:13:30 +0800] warn [cpbackup_transporter] Upload attempt failed: upload_file_content_single() missing upload_uri at /usr/local/cpanel/3rdparty/perl/532/lib/perl5/cpanel_lib/Net/Google/Drive/Simple/V3.pm line 996

[2022-12-19 03:13:30 +0800] info [cpbackup_transporter] Upload attempt #3 starting for /backup/2022-12-19/accounts/account2.tar.gz to /GoogleDrivePath/2022-12-19/accounts/account2.tar.gz for destination: Destination Name
[2022-12-19 03:13:44 +0800] info [cpbackup_transporter] Failed with 429: Too Many Requests
[2022-12-19 03:14:04 +0800] info [cpbackup_transporter] Failed with 429: Too Many Requests
[2022-12-19 03:14:20 +0800] info [cpbackup_transporter] Failed with 429: Too Many Requests
[2022-12-19 03:14:30 +0800] info [cpbackup_transporter] Failed with 503: Service Unavailable
[2022-12-19 03:14:54 +0800] warn [cpbackup_transporter] Upload attempt failed: malformed JSON string, neither tag, array, object, number, string or atom, at character offset 0 (before "(end of string)") at /usr/local/cpanel/3rdparty/perl/532/lib/perl5/cpanel_lib/JSON.pm line 190.

[2022-12-19 03:14:54 +0800] info [cpbackup_transporter] cpbackup_transporter - Processing next task
[2022-12-19 03:14:54 +0800] info [cpbackup_transporter] Instantiating Object
[2022-12-19 03:14:54 +0800] info [cpbackup_transporter] Starting a "copy" operation on the "Destination Name" destination ID "DestinationID".
[2022-12-19 03:14:54 +0800] info [cpbackup_transporter] Validating destination path /GoogleDrivePath/2022-12-19/accounts/
[2022-12-19 03:14:57 +0800] info [cpbackup_transporter] Path exists
[2022-12-19 03:14:57 +0800] info [cpbackup_transporter] Uploading account backup /backup/2022-12-19/accounts/account3.tar.gz to /GoogleDrivePath/2022-12-19/accounts/account3.tar.gz (from 2022-12-19/accounts/account3.tar.gz)
[2022-12-19 03:14:57 +0800] info [cpbackup_transporter] Attempting to upload /backup/2022-12-19/accounts/account3.tar.gz to /GoogleDrivePath/2022-12-19/accounts/account3.tar.gz for destination: Destination Name
[2022-12-19 03:14:57 +0800] info [cpbackup_transporter] Upload attempt #1 starting for /backup/2022-12-19/accounts/account3.tar.gz to /GoogleDrivePath/2022-12-19/accounts/account3.tar.gz for destination: Destination Name
[2022-12-19 03:15:04 +0800] info [cpbackup_transporter] Successful transfer of /backup/2022-12-19/accounts/account3.tar.gz to /GoogleDrivePath/2022-12-19/accounts/account3.tar.gz for destination Destination Name
[2022-12-19 03:15:04 +0800] info [cpbackup_transporter] The backup has been successfully uploaded at least once, now we will delete the local copy (/backup/2022-12-19/accounts/account3.tar.gz) since keeplocal (0) is disabled.
[2022-12-19 03:15:04 +0800] info [cpbackup_transporter] cpbackup_transporter - Processing next task
[2022-12-19 03:15:04 +0800] info [cpbackup_transporter] Instantiating Object
[2022-12-19 03:15:04 +0800] info [cpbackup_transporter] Starting a "copy" operation on the "Destination Name" destination ID "DestinationID".
[2022-12-19 03:15:04 +0800] info [cpbackup_transporter] Validating destination path /GoogleDrivePath/2022-12-19/accounts/
[2022-12-19 03:15:07 +0800] info [cpbackup_transporter] Path exists
[2022-12-19 03:15:07 +0800] info [cpbackup_transporter] Uploading account backup /backup/2022-12-19/accounts/account4.tar.gz to /GoogleDrivePath/2022-12-19/accounts/account4.tar.gz (from 2022-12-19/accounts/account4.tar.gz)
[2022-12-19 03:15:07 +0800] info [cpbackup_transporter] Attempting to upload /backup/2022-12-19/accounts/account4.tar.gz to /GoogleDrivePath/2022-12-19/accounts/account4.tar.gz for destination: Destination Name
[2022-12-19 03:15:07 +0800] info [cpbackup_transporter] Upload attempt #1 starting for /backup/2022-12-19/accounts/account4.tar.gz to /GoogleDrivePath/2022-12-19/accounts/account4.tar.gz for destination: Destination Name
[2022-12-19 03:15:14 +0800] info [cpbackup_transporter] Successful transfer of /backup/2022-12-19/accounts/account4.tar.gz to /GoogleDrivePath/2022-12-19/accounts/account4.tar.gz for destination Destination Name
[2022-12-19 03:15:14 +0800] info [cpbackup_transporter] The backup has been successfully uploaded at least once, now we will delete the local copy (/backup/2022-12-19/accounts/account4.tar.gz) since keeplocal (0) is disabled.
[2022-12-19 03:15:14 +0800] info [cpbackup_transporter] cpbackup_transporter - Processing next task
[2022-12-19 03:15:14 +0800] info [cpbackup_transporter] Instantiating Object
[2022-12-19 03:15:14 +0800] info [cpbackup_transporter] Starting a "copy" operation on the "Destination Name" destination ID "DestinationID".
[2022-12-19 03:15:14 +0800] info [cpbackup_transporter] Validating destination path /GoogleDrivePath/2022-12-19/accounts/
[2022-12-19 03:15:17 +0800] info [cpbackup_transporter] Path exists
[2022-12-19 03:15:17 +0800] info [cpbackup_transporter] Uploading account backup /backup/2022-12-19/accounts/account5.tar.gz to /GoogleDrivePath/2022-12-19/accounts/account5.tar.gz (from 2022-12-19/accounts/account5.tar.gz)
[2022-12-19 03:15:17 +0800] info [cpbackup_transporter] Attempting to upload /backup/2022-12-19/accounts/account5.tar.gz to /GoogleDrivePath/2022-12-19/accounts/account5.tar.gz for destination: Destination Name
[2022-12-19 03:15:17 +0800] info [cpbackup_transporter] Upload attempt #1 starting for /backup/2022-12-19/accounts/account5.tar.gz to /GoogleDrivePath/2022-12-19/accounts/account5.tar.gz for destination: Destination Name
[2022-12-19 03:15:24 +0800] info [cpbackup_transporter] Successful transfer of /backup/2022-12-19/accounts/account5.tar.gz to /GoogleDrivePath/2022-12-19/accounts/account5.tar.gz for destination Destination Name
[2022-12-19 03:15:24 +0800] info [cpbackup_transporter] The backup has been successfully uploaded at least once, now we will delete the local copy (/backup/2022-12-19/accounts/account5.tar.gz) since keeplocal (0) is disabled.
[2022-12-19 03:15:24 +0800] info [cpbackup_transporter] cpbackup_transporter - Processing next task
[2022-12-19 03:15:24 +0800] info [cpbackup_transporter] Instantiating Object
[2022-12-19 03:15:24 +0800] info [cpbackup_transporter] Starting a "copy" operation on the "Destination Name" destination ID "DestinationID".
[2022-12-19 03:15:24 +0800] info [cpbackup_transporter] Validating destination path /GoogleDrivePath/2022-12-19/accounts/
[2022-12-19 03:15:27 +0800] info [cpbackup_transporter] Path exists
[2022-12-19 03:15:27 +0800] info [cpbackup_transporter] Uploading account backup /backup/2022-12-19/accounts/account6.tar.gz to /GoogleDrivePath/2022-12-19/accounts/account6.tar.gz (from 2022-12-19/accounts/account6.tar.gz)
[2022-12-19 03:15:27 +0800] info [cpbackup_transporter] Attempting to upload /backup/2022-12-19/accounts/account6.tar.gz to /GoogleDrivePath/2022-12-19/accounts/account6.tar.gz for destination: Destination Name
[2022-12-19 03:15:27 +0800] info [cpbackup_transporter] Upload attempt #1 starting for /backup/2022-12-19/accounts/account6.tar.gz to /GoogleDrivePath/2022-12-19/accounts/account6.tar.gz for destination: Destination Name
[2022-12-19 03:15:36 +0800] info [cpbackup_transporter] Successful transfer of /backup/2022-12-19/accounts/account6.tar.gz to /GoogleDrivePath/2022-12-19/accounts/account6.tar.gz for destination Destination Name
[2022-12-19 03:15:36 +0800] info [cpbackup_transporter] The backup has been successfully uploaded at least once, now we will delete the local copy (/backup/2022-12-19/accounts/account6.tar.gz) since keeplocal (0) is disabled.
[2022-12-19 03:15:36 +0800] info [cpbackup_transporter] cpbackup_transporter - Processing next task
[2022-12-19 03:15:36 +0800] info [cpbackup_transporter] Instantiating Object
[2022-12-19 03:15:36 +0800] info [cpbackup_transporter] Starting a "copy" operation on the "Destination Name" destination ID "DestinationID".
[2022-12-19 03:15:36 +0800] info [cpbackup_transporter] Validating destination path /GoogleDrivePath/2022-12-19/accounts/
[2022-12-19 03:15:39 +0800] info [cpbackup_transporter] Path exists
[2022-12-19 03:15:39 +0800] info [cpbackup_transporter] Uploading account backup /backup/2022-12-19/accounts/account7.tar.gz to /GoogleDrivePath/2022-12-19/accounts/account7.tar.gz (from 2022-12-19/accounts/account7.tar.gz)
[2022-12-19 03:15:39 +0800] info [cpbackup_transporter] Attempting to upload /backup/2022-12-19/accounts/account7.tar.gz to /GoogleDrivePath/2022-12-19/accounts/account7.tar.gz for destination: Destination Name
[2022-12-19 03:15:39 +0800] info [cpbackup_transporter] Upload attempt #1 starting for /backup/2022-12-19/accounts/account7.tar.gz to /GoogleDrivePath/2022-12-19/accounts/account7.tar.gz for destination: Destination Name
[2022-12-19 03:15:49 +0800] info [cpbackup_transporter] Successful transfer of /backup/2022-12-19/accounts/account7.tar.gz to /GoogleDrivePath/2022-12-19/accounts/account7.tar.gz for destination Destination Name
[2022-12-19 03:15:49 +0800] info [cpbackup_transporter] The backup has been successfully uploaded at least once, now we will delete the local copy (/backup/2022-12-19/accounts/account7.tar.gz) since keeplocal (0) is disabled.
[2022-12-19 03:15:49 +0800] info [cpbackup_transporter] cpbackup_transporter - Processing next task
[2022-12-19 03:15:49 +0800] info [cpbackup_transporter] Instantiating Object
[2022-12-19 03:15:49 +0800] info [cpbackup_transporter] Starting a "copy" operation on the "Destination Name" destination ID "DestinationID".
[2022-12-19 03:15:49 +0800] info [cpbackup_transporter] Validating destination path /GoogleDrivePath/2022-12-19/accounts/
[2022-12-19 03:15:53 +0800] info [cpbackup_transporter] Path exists
[2022-12-19 03:15:53 +0800] info [cpbackup_transporter] Uploading account backup /backup/2022-12-19/accounts/account8.tar.gz to /GoogleDrivePath/2022-12-19/accounts/account8.tar.gz (from 2022-12-19/accounts/account8.tar.gz)
[2022-12-19 03:15:53 +0800] info [cpbackup_transporter] Attempting to upload /backup/2022-12-19/accounts/account8.tar.gz to /GoogleDrivePath/2022-12-19/accounts/account8.tar.gz for destination: Destination Name
[2022-12-19 03:15:53 +0800] info [cpbackup_transporter] Upload attempt #1 starting for /backup/2022-12-19/accounts/account8.tar.gz to /GoogleDrivePath/2022-12-19/accounts/account8.tar.gz for destination: Destination Name
[2022-12-19 03:16:00 +0800] info [cpbackup_transporter] Successful transfer of /backup/2022-12-19/accounts/account8.tar.gz to /GoogleDrivePath/2022-12-19/accounts/account8.tar.gz for destination Destination Name
[2022-12-19 03:16:00 +0800] info [cpbackup_transporter] The backup has been successfully uploaded at least once, now we will delete the local copy (/backup/2022-12-19/accounts/account8.tar.gz) since keeplocal (0) is disabled.
[2022-12-19 03:16:00 +0800] info [cpbackup_transporter] cpbackup_transporter - Processing next task
[2022-12-19 03:16:00 +0800] info [cpbackup_transporter] Instantiating Object
[2022-12-19 03:16:00 +0800] info [cpbackup_transporter] Starting a "copy" operation on the "Destination Name" destination ID "DestinationID".
[2022-12-19 03:16:00 +0800] info [cpbackup_transporter] Validating destination path /GoogleDrivePath/2022-12-19/accounts/
[2022-12-19 03:16:03 +0800] info [cpbackup_transporter] Path exists
[2022-12-19 03:16:03 +0800] info [cpbackup_transporter] Uploading account backup /backup/2022-12-19/accounts/account9.tar.gz to /GoogleDrivePath/2022-12-19/accounts/account9.tar.gz (from 2022-12-19/accounts/account9.tar.gz)
[2022-12-19 03:16:03 +0800] info [cpbackup_transporter] Attempting to upload /backup/2022-12-19/accounts/account9.tar.gz to /GoogleDrivePath/2022-12-19/accounts/account9.tar.gz for destination: Destination Name
[2022-12-19 03:16:03 +0800] info [cpbackup_transporter] Upload attempt #1 starting for /backup/2022-12-19/accounts/account9.tar.gz to /GoogleDrivePath/2022-12-19/accounts/account9.tar.gz for destination: Destination Name
[2022-12-19 03:16:10 +0800] info [cpbackup_transporter] Successful transfer of /backup/2022-12-19/accounts/account9.tar.gz to /GoogleDrivePath/2022-12-19/accounts/account9.tar.gz for destination Destination Name
[2022-12-19 03:16:10 +0800] info [cpbackup_transporter] The backup has been successfully uploaded at least once, now we will delete the local copy (/backup/2022-12-19/accounts/account9.tar.gz) since keeplocal (0) is disabled.
[2022-12-19 03:16:10 +0800] info [cpbackup_transporter] cpbackup_transporter - Processing next task
[2022-12-19 03:16:10 +0800] info [cpbackup_transporter] Instantiating Object
[2022-12-19 03:16:10 +0800] info [cpbackup_transporter] Starting a "copy" operation on the "Destination Name" destination ID "DestinationID".
[2022-12-19 03:16:10 +0800] info [cpbackup_transporter] Validating destination path /GoogleDrivePath/2022-12-19/accounts/
[2022-12-19 03:16:13 +0800] info [cpbackup_transporter] Path exists
[2022-12-19 03:16:13 +0800] info [cpbackup_transporter] Uploading account backup /backup/2022-12-19/accounts/account10.tar.gz to /GoogleDrivePath/2022-12-19/accounts/account10.tar.gz (from 2022-12-19/accounts/account10.tar.gz)
[2022-12-19 03:16:13 +0800] info [cpbackup_transporter] Attempting to upload /backup/2022-12-19/accounts/account10.tar.gz to /GoogleDrivePath/2022-12-19/accounts/account10.tar.gz for destination: Destination Name
[2022-12-19 03:16:13 +0800] info [cpbackup_transporter] Upload attempt #1 starting for /backup/2022-12-19/accounts/account10.tar.gz to /GoogleDrivePath/2022-12-19/accounts/account10.tar.gz for destination: Destination Name
[2022-12-19 03:16:59 +0800] info [cpbackup_transporter] Successful transfer of /backup/2022-12-19/accounts/account10.tar.gz to /GoogleDrivePath/2022-12-19/accounts/account10.tar.gz for destination Destination Name
[2022-12-19 03:16:59 +0800] info [cpbackup_transporter] The backup has been successfully uploaded at least once, now we will delete the local copy (/backup/2022-12-19/accounts/account10.tar.gz) since keeplocal (0) is disabled.
[2022-12-19 03:16:59 +0800] info [cpbackup_transporter] cpbackup_transporter - Processing next task
[2022-12-19 03:16:59 +0800] info [cpbackup_transporter] Instantiating Object
[2022-12-19 03:16:59 +0800] info [cpbackup_transporter] Starting a "copy" operation on the "Destination Name" destination ID "DestinationID".
[2022-12-19 03:16:59 +0800] info [cpbackup_transporter] Validating destination path /GoogleDrivePath/2022-12-19/accounts/
[2022-12-19 03:17:02 +0800] info [cpbackup_transporter] Path exists
[2022-12-19 03:17:02 +0800] info [cpbackup_transporter] Uploading account backup /backup/2022-12-19/accounts/account11.tar.gz to /GoogleDrivePath/2022-12-19/accounts/account11.tar.gz (from 2022-12-19/accounts/account11.tar.gz)
[2022-12-19 03:17:02 +0800] info [cpbackup_transporter] Attempting to upload /backup/2022-12-19/accounts/account11.tar.gz to /GoogleDrivePath/2022-12-19/accounts/account11.tar.gz for destination: Destination Name
[2022-12-19 03:17:02 +0800] info [cpbackup_transporter] Upload attempt #1 starting for /backup/2022-12-19/accounts/account11.tar.gz to /GoogleDrivePath/2022-12-19/accounts/account11.tar.gz for destination: Destination Name
[2022-12-19 03:17:16 +0800] info [cpbackup_transporter] Failed with 429: Too Many Requests
[2022-12-19 03:17:36 +0800] info [cpbackup_transporter] Failed with 429: Too Many Requests
[2022-12-19 03:17:46 +0800] info [cpbackup_transporter] Failed with 502: Bad Gateway
[2022-12-19 03:18:03 +0800] info [cpbackup_transporter] Failed with 429: Too Many Requests
[2022-12-19 03:18:03 +0800] warn [cpbackup_transporter] Upload attempt failed: upload_file_content_single() missing upload_uri at /usr/local/cpanel/3rdparty/perl/532/lib/perl5/cpanel_lib/Net/Google/Drive/Simple/V3.pm line 996

[2022-12-19 03:18:03 +0800] info [cpbackup_transporter] Upload attempt #2 starting for /backup/2022-12-19/accounts/account11.tar.gz to /GoogleDrivePath/2022-12-19/accounts/account11.tar.gz for destination: Destination Name
[2022-12-19 03:18:08 +0800] info [cpbackup_transporter] Failed with 502: Bad Gateway
[2022-12-19 03:18:25 +0800] info [cpbackup_transporter] Failed with 429: Too Many Requests
[2022-12-19 03:18:39 +0800] info [cpbackup_transporter] Successful transfer of /backup/2022-12-19/accounts/account11.tar.gz to /GoogleDrivePath/2022-12-19/accounts/account11.tar.gz for destination Destination Name
[2022-12-19 03:18:39 +0800] info [cpbackup_transporter] The backup has been successfully uploaded at least once, now we will delete the local copy (/backup/2022-12-19/accounts/account11.tar.gz) since keeplocal (0) is disabled.
[2022-12-19 03:18:39 +0800] info [cpbackup_transporter] cpbackup_transporter - Processing next task
[2022-12-19 03:18:39 +0800] info [cpbackup_transporter] Instantiating Object
[2022-12-19 03:18:39 +0800] info [cpbackup_transporter] Starting a "copy" operation on the "Destination Name" destination ID "DestinationID".
[2022-12-19 03:18:39 +0800] info [cpbackup_transporter] Validating destination path /GoogleDrivePath/2022-12-19/accounts/
[2022-12-19 03:18:42 +0800] info [cpbackup_transporter] Path exists
[2022-12-19 03:18:42 +0800] info [cpbackup_transporter] Uploading account backup /backup/2022-12-19/accounts/account12.tar.gz to /GoogleDrivePath/2022-12-19/accounts/account12.tar.gz (from 2022-12-19/accounts/account12.tar.gz)
[2022-12-19 03:18:42 +0800] info [cpbackup_transporter] Attempting to upload /backup/2022-12-19/accounts/account12.tar.gz to /GoogleDrivePath/2022-12-19/accounts/account12.tar.gz for destination: Destination Name
[2022-12-19 03:18:42 +0800] info [cpbackup_transporter] Upload attempt #1 starting for /backup/2022-12-19/accounts/account12.tar.gz to /GoogleDrivePath/2022-12-19/accounts/account12.tar.gz for destination: Destination Name
[2022-12-19 03:18:47 +0800] info [cpbackup_transporter] Failed with 502: Bad Gateway
[2022-12-19 03:18:57 +0800] info [cpbackup_transporter] Failed with 503: Service Unavailable
[2022-12-19 03:19:16 +0800] info [cpbackup_transporter] Failed with 429: Too Many Requests
[2022-12-19 03:19:35 +0800] info [cpbackup_transporter] Failed with 429: Too Many Requests
[2022-12-19 03:19:53 +0800] info [cpbackup_transporter] Failed with 429: Too Many Requests
[2022-12-19 03:19:53 +0800] warn [cpbackup_transporter] Upload attempt failed: Error uploading /backup/2022-12-19/accounts/account12.tar.gz: Too Many Requests
[2022-12-19 03:19:53 +0800] info [cpbackup_transporter] Upload attempt #2 starting for /backup/2022-12-19/accounts/account12.tar.gz to /GoogleDrivePath/2022-12-19/accounts/account12.tar.gz for destination: Destination Name
[2022-12-19 03:20:06 +0800] info [cpbackup_transporter] Failed with 429: Too Many Requests
[2022-12-19 03:20:17 +0800] info [cpbackup_transporter] Failed with 503: Service Unavailable
[2022-12-19 03:20:48 +0800] warn [cpbackup_transporter] Upload attempt failed: malformed JSON string, neither tag, array, object, number, string or atom, at character offset 0 (before "(end of string)") at /usr/local/cpanel/3rdparty/perl/532/lib/perl5/cpanel_lib/JSON.pm line 190.

[2022-12-19 03:20:48 +0800] info [cpbackup_transporter] Upload attempt #3 starting for /backup/2022-12-19/accounts/account12.tar.gz to /GoogleDrivePath/2022-12-19/accounts/account12.tar.gz for destination: Destination Name
[2022-12-19 03:21:02 +0800] info [cpbackup_transporter] Failed with 429: Too Many Requests
[2022-12-19 03:21:18 +0800] info [cpbackup_transporter] Failed with 429: Too Many Requests
[2022-12-19 03:21:37 +0800] info [cpbackup_transporter] Failed with 429: Too Many Requests
[2022-12-19 03:21:53 +0800] info [cpbackup_transporter] Failed with 429: Too Many Requests
[2022-12-19 03:21:53 +0800] warn [cpbackup_transporter] Upload attempt failed: upload_file_content_single() missing upload_uri at /usr/local/cpanel/3rdparty/perl/532/lib/perl5/cpanel_lib/Net/Google/Drive/Simple/V3.pm line 996

[2022-12-19 03:21:53 +0800] info [cpbackup_transporter] cpbackup_transporter - Processing next task
[2022-12-19 03:21:53 +0800] info [cpbackup_transporter] Instantiating Object
[2022-12-19 03:21:53 +0800] info [cpbackup_transporter] Starting a "copy" operation on the "Destination Name" destination ID "DestinationID".
[2022-12-19 03:21:53 +0800] info [cpbackup_transporter] Validating destination path /GoogleDrivePath/2022-12-19/accounts/
[2022-12-19 03:21:56 +0800] info [cpbackup_transporter] Path exists
[2022-12-19 03:21:56 +0800] info [cpbackup_transporter] Uploading account backup /backup/2022-12-19/accounts/account13.tar.gz to /GoogleDrivePath/2022-12-19/accounts/account13.tar.gz (from 2022-12-19/accounts/account13.tar.gz)
[2022-12-19 03:21:56 +0800] info [cpbackup_transporter] Attempting to upload /backup/2022-12-19/accounts/account13.tar.gz to /GoogleDrivePath/2022-12-19/accounts/account13.tar.gz for destination: Destination Name
[2022-12-19 03:21:56 +0800] info [cpbackup_transporter] Upload attempt #1 starting for /backup/2022-12-19/accounts/account13.tar.gz to /GoogleDrivePath/2022-12-19/accounts/account13.tar.gz for destination: Destination Name
[2022-12-19 03:22:01 +0800] info [cpbackup_transporter] Failed with 502: Bad Gateway
[2022-12-19 03:22:19 +0800] info [cpbackup_transporter] Failed with 429: Too Many Requests
[2022-12-19 03:22:54 +0800] info [cpbackup_transporter] Successful transfer of /backup/2022-12-19/accounts/account13.tar.gz to /GoogleDrivePath/2022-12-19/accounts/account13.tar.gz for destination Destination Name
[2022-12-19 03:22:54 +0800] info [cpbackup_transporter] The backup has been successfully uploaded at least once, now we will delete the local copy (/backup/2022-12-19/accounts/account13.tar.gz) since keeplocal (0) is disabled.
[2022-12-19 03:22:55 +0800] info [cpbackup_transporter] cpbackup_transporter - Processing next task
[2022-12-19 03:22:55 +0800] info [cpbackup_transporter] Instantiating Object
[2022-12-19 03:22:55 +0800] info [cpbackup_transporter] Starting a "copy" operation on the "Destination Name" destination ID "DestinationID".
[2022-12-19 03:22:55 +0800] info [cpbackup_transporter] Validating destination path /GoogleDrivePath/2022-12-19/accounts/
[2022-12-19 03:22:57 +0800] info [cpbackup_transporter] Path exists
[2022-12-19 03:22:57 +0800] info [cpbackup_transporter] Uploading account backup /backup/2022-12-19/accounts/account14.tar.gz to /GoogleDrivePath/2022-12-19/accounts/account14.tar.gz (from 2022-12-19/accounts/account14.tar.gz)
[2022-12-19 03:22:57 +0800] info [cpbackup_transporter] Attempting to upload /backup/2022-12-19/accounts/account14.tar.gz to /GoogleDrivePath/2022-12-19/accounts/account14.tar.gz for destination: Destination Name
[2022-12-19 03:22:57 +0800] info [cpbackup_transporter] Upload attempt #1 starting for /backup/2022-12-19/accounts/account14.tar.gz to /GoogleDrivePath/2022-12-19/accounts/account14.tar.gz for destination: Destination Name
[2022-12-19 03:23:02 +0800] info [cpbackup_transporter] Failed with 503: Service Unavailable
[2022-12-19 03:23:12 +0800] info [cpbackup_transporter] Failed with 503: Service Unavailable
[2022-12-19 03:23:23 +0800] warn [cpbackup_transporter] Upload attempt failed: malformed JSON string, neither tag, array, object, number, string or atom, at character offset 0 (before "(end of string)") at /usr/local/cpanel/3rdparty/perl/532/lib/perl5/cpanel_lib/JSON.pm line 190.

[2022-12-19 03:23:23 +0800] info [cpbackup_transporter] Upload attempt #2 starting for /backup/2022-12-19/accounts/account14.tar.gz to /GoogleDrivePath/2022-12-19/accounts/account14.tar.gz for destination: Destination Name
[2022-12-19 03:23:31 +0800] info [cpbackup_transporter] Successful transfer of /backup/2022-12-19/accounts/account14.tar.gz to /GoogleDrivePath/2022-12-19/accounts/account14.tar.gz for destination Destination Name
[2022-12-19 03:23:31 +0800] info [cpbackup_transporter] The backup has been successfully uploaded at least once, now we will delete the local copy (/backup/2022-12-19/accounts/account14.tar.gz) since keeplocal (0) is disabled.
[2022-12-19 03:23:31 +0800] info [cpbackup_transporter] cpbackup_transporter - Processing next task
[2022-12-19 03:23:31 +0800] info [cpbackup_transporter] Instantiating Object
[2022-12-19 03:23:31 +0800] info [cpbackup_transporter] Starting a "copy" operation on the "Destination Name" destination ID "DestinationID".
[2022-12-19 03:23:31 +0800] info [cpbackup_transporter] Validating destination path /GoogleDrivePath/2022-12-19/accounts/
[2022-12-19 03:23:34 +0800] info [cpbackup_transporter] Path exists
[2022-12-19 03:23:34 +0800] info [cpbackup_transporter] Uploading account backup /backup/2022-12-19/accounts/account15.tar.gz to /GoogleDrivePath/2022-12-19/accounts/account15.tar.gz (from 2022-12-19/accounts/account15.tar.gz)
[2022-12-19 03:23:34 +0800] info [cpbackup_transporter] Attempting to upload /backup/2022-12-19/accounts/account15.tar.gz to /GoogleDrivePath/2022-12-19/accounts/account15.tar.gz for destination: Destination Name
[2022-12-19 03:23:34 +0800] info [cpbackup_transporter] Upload attempt #1 starting for /backup/2022-12-19/accounts/account15.tar.gz to /GoogleDrivePath/2022-12-19/accounts/account15.tar.gz for destination: Destination Name
[2022-12-19 03:23:41 +0800] info [cpbackup_transporter] Successful transfer of /backup/2022-12-19/accounts/account15.tar.gz to /GoogleDrivePath/2022-12-19/accounts/account15.tar.gz for destination Destination Name
[2022-12-19 03:23:41 +0800] info [cpbackup_transporter] The backup has been successfully uploaded at least once, now we will delete the local copy (/backup/2022-12-19/accounts/account15.tar.gz) since keeplocal (0) is disabled.
[2022-12-19 03:23:41 +0800] info [cpbackup_transporter] cpbackup_transporter - Processing next task
[2022-12-19 03:23:41 +0800] info [cpbackup_transporter] Instantiating Object
[2022-12-19 03:23:41 +0800] info [cpbackup_transporter] Starting a "copy_backup_metadata" operation on the "Destination Name" destination ID "DestinationID".
[2022-12-19 03:23:41 +0800] info [cpbackup_transporter] Uploading backup metadata file /backup/2022-12-19/accounts/.master.meta to Destination Name
[2022-12-19 03:23:41 +0800] info [cpbackup_transporter] Validating destination path /GoogleDrivePath/2022-12-19/accounts/
[2022-12-19 03:23:44 +0800] info [cpbackup_transporter] Path exists
[2022-12-19 03:23:44 +0800] info [cpbackup_transporter] Uploading backup metadata /backup/2022-12-19/accounts/.master.meta to /GoogleDrivePath/2022-12-19/accounts/.master.meta (from 2022-12-19/accounts/.master.meta)
[2022-12-19 03:23:44 +0800] info [cpbackup_transporter] Attempting to upload /backup/2022-12-19/accounts/.master.meta to /GoogleDrivePath/2022-12-19/accounts/.master.meta for destination: Destination Name
[2022-12-19 03:23:44 +0800] info [cpbackup_transporter] Upload attempt #1 starting for /backup/2022-12-19/accounts/.master.meta to /GoogleDrivePath/2022-12-19/accounts/.master.meta for destination: Destination Name
[2022-12-19 03:23:48 +0800] info [cpbackup_transporter] Failed with 502: Bad Gateway
[2022-12-19 03:23:59 +0800] info [cpbackup_transporter] Failed with 502: Bad Gateway
[2022-12-19 03:24:10 +0800] info [cpbackup_transporter] Failed with 502: Bad Gateway
[2022-12-19 03:24:29 +0800] info [cpbackup_transporter] Failed with 429: Too Many Requests
[2022-12-19 03:24:40 +0800] info [cpbackup_transporter] Successful transfer of /backup/2022-12-19/accounts/.master.meta to /GoogleDrivePath/2022-12-19/accounts/.master.meta for destination Destination Name
[2022-12-19 03:24:40 +0800] info [cpbackup_transporter] The backup has been successfully uploaded at least once, now we will delete the local copy (/backup/2022-12-19/accounts/.master.meta) since keeplocal (0) is disabled.
[2022-12-19 03:24:40 +0800] info [cpbackup_transporter] cpbackup_transporter - Processing next task
[2022-12-19 03:24:40 +0800] info [cpbackup_transporter] Instantiating Object
[2022-12-19 03:24:40 +0800] info [cpbackup_transporter] Starting a "prune" operation on the "Destination Name" destination ID "DestinationID".
[2022-12-19 03:24:40 +0800] info [cpbackup_transporter] Performing prune operation, retaining 7 items on: Destination Name
[2022-12-19 03:24:44 +0800] info [cpbackup_transporter] Pruning backup directory: /GoogleDrivePath/2022-12-08, from Destination Name
[2022-12-19 03:24:46 +0800] info [cpbackup_transporter] cpbackup_transporter - Processing next task
[2022-12-19 03:24:46 +0800] info [cpbackup_transporter] Removing backup staging directory: /backup/2022-12-19
[2022-12-19 03:24:48 +0800] info [cpbackup_transporter] cpbackup_transporter - Processing next task
[2022-12-19 03:25:16 +0800] info [cpbackup_transporter] cpbackup_transporter - Waiting up to 270s for new tasks
[2022-12-19 03:25:46 +0800] info [cpbackup_transporter] cpbackup_transporter - Waiting up to 240s for new tasks
[2022-12-19 03:26:16 +0800] info [cpbackup_transporter] cpbackup_transporter - Waiting up to 210s for new tasks
[2022-12-19 03:26:46 +0800] info [cpbackup_transporter] cpbackup_transporter - Waiting up to 180s for new tasks
[2022-12-19 03:27:16 +0800] info [cpbackup_transporter] cpbackup_transporter - Waiting up to 150s for new tasks
[2022-12-19 03:27:46 +0800] info [cpbackup_transporter] cpbackup_transporter - Waiting up to 120s for new tasks
[2022-12-19 03:28:16 +0800] info [cpbackup_transporter] cpbackup_transporter - Waiting up to 90s for new tasks
[2022-12-19 03:28:46 +0800] info [cpbackup_transporter] cpbackup_transporter - Waiting up to 60s for new tasks
[2022-12-19 03:29:16 +0800] info [cpbackup_transporter] cpbackup_transporter - Waiting up to 30s for new tasks
[2022-12-19 03:29:46 +0800] info [cpbackup_transporter] cpbackup_transporter - Waiting up to 0s for new tasks
[2022-12-19 03:29:47 +0800] info [cpbackup_transporter] cpbackup_transporter - Exiting - the queue has been emptied; no more work to do after waiting for 300s
[2022-12-19 03:29:47 +0800] info [cpbackup_transporter] cPanel Backup Transporter Queue Daemon is being stopped.

 

[cPRex]

Thanks for that - that's exactly what I needed to see.

This error indicates that you are reaching some limit on the Google side of things, and that cPanel isn't able to move the data once that limit is reached. Can you contact Google directly to see if they can provide more information on this?

 

[Art0311]

Thanks, I've contacted Google and there's actually an API setting in Google Workspace Admin that needs to be updated.
Sign in - Google Accounts > Security > API controls > App access control > Filter 'CPanelBackup' > Change access to 'Trusted'

Will update again if this solve the problem.

 

[Art0311]

Here's the reply from google:

I would like to share the fact that on the project metrics A the error rate is 0% for the method “drive.files.create”. Thus, the issue likely resides in a configuration on the third party tool that they are using “CPanel”, for which you will need to contact the tool maker.

However, the errors returned, “502 bad gateway” and “429 too many requests” refer to a rateLimitExceeded error that occurs when the user has sent too many requests in a given amount of time. This is explained in this documentation[1], for which the workaround is available in this documentation[2].

For inquiries regarding upload quota limits, you can refer to this documentation[3].

[1] https://developers.google.com/drive/api/guides/handle-errors#resolve_a_429_error_too_many_requests
[2] https://developers.google.com/drive/api/guides/limits#exponential
[3] https://support.google.com/a/answer/10445916

The issue is not resolved at the moment. Does anyone know if cPanel implements exponential backoff algorithm?

 

[cPRex]

We don't implement any type of exponential backoff algorithm as part of our backup process. We do disable a remote transport that fails after three attempts.

Is this a shared Google drive? If so, there have been some known ratelimiting issues in the past with that specific type of drive:

https://support.cpanel.net/hc/en-us/articles/1500008502762-BackUp-Transport-To-Google-Drive-Destination-Fails-Halfway-With-A-403-Error-If-the-Google-Drive-Account-Is-A-Shared-Drive

but the errors do confirm the ratelimiting is happening on the Google side of things.

 

[Art0311]

No, it's not a Shared Drive.

Suddenly the backup is working again for the past week without changing CPanel settings.

 

[cPRex]

Since nothing changed on the cPanel side of things, that's even further indication that the issue wouldn't be on the server.