S3 Compatible backup configuration

Operating System & Version
cpanel_and_whm: 11.102.0
cPanel & WHM Version
operating_system_name: centos operating_system_version: 7

nixon7606

Member
Aug 27, 2020
11
1
3
Centennial, CO
cPanel Access Level
Root Administrator
Attempting to configure an S3Compatible backup destination in WHM. S3 server is running on MinIO, have tested the S3 server using the "mc" MinIO S3 client from the WHM server, this works fine. When attempting to configure the destination in cPanel, "Validation" always fails. I am pretty sure we're not falling into the trap of bucketname.domain.com where bucketname needs to be specified separately. Regardless of how we've attempted to configure this, the validation step fails. Logs are scarce as far as I can tell. The typical error we get is "Error: Validation for transport “cpbackup” failed: Could not upload test file: Timeout"; (cpbackup is the name we've given this destination). Has anyone succeeded in getting cPanel to use MinIO as a backup destination? What logs can I be checking? Are there any OS dependencies that should be present for S3 clients to work? Furthermore, tcpdump on the MinIO server while doing validation shows no packets actually hitting the S3 server from the cPanel server.

Any help would be appreciated.

[2022-04-18 11:31:23 -0600] info [xml-api] Validation for transport “cpbackup” failed: Could not upload test file: Timeout [backup_destination_validate] version [1].
 
Last edited by a moderator:

Spirogg

Well-Known Member
Feb 21, 2018
696
151
43
chicago
cPanel Access Level
Root Administrator
Attempting to configure an S3Compatible backup destination in WHM. S3 server is running on MinIO, have tested the S3 server using the "mc" MinIO S3 client from the WHM server, this works fine. When attempting to configure the destination in cPanel, "Validation" always fails. I am pretty sure we're not falling into the trap of bucketname.domain.com where bucketname needs to be specified separately. Regardless of how we've attempted to configure this, the validation step fails. Logs are scarce as far as I can tell. The typical error we get is "Error: Validation for transport “cpbackup” failed: Could not upload test file: Timeout"; (cpbackup is the name we've given this destination). Has anyone succeeded in getting cPanel to use MinIO as a backup destination? What logs can I be checking? Are there any OS dependencies that should be present for S3 clients to work? Furthermore, tcpdump on the MinIO server while doing validation shows no packets actually hitting the S3 server from the cPanel server.

Any help would be appreciated.

[2022-04-18 11:31:23 -0600] info [xml-api] Validation for transport “cpbackup” failed: Could not upload test file: Timeout [backup_destination_validate] version [1].
not sure if you read the docs here might be helpful ?

Setting​
Description​
Destination Name
Required — Enter a destination name for your backup files. This name appears in your destination table.​
Transfer System Backups to this Destination
Select this checkbox to transfer system backups to this additional destination. To use this setting, you must also select the Back up System Files checkbox in the Backup Settings tab.​
Folder
Enter the name of the directory where you wish to store your backup.​
S3 Endpoint
Required — Enter the fully qualified domain name (FQDN) for the remote server where your bucket resides. The system backs up files to the destination server through a secured connection via SSL/TLS. The FQDN that you enter must be the same as the FQDN on the server’s SSL/TLS certificate, or the connection will fail.​
Bucket
Required — Enter the name of the bucket where you wish to store your backup. You must first create the bucket through your S3-compatible provider.​
Access Key ID
Required — Enter the access key ID in this text box. Your server uses the access key to authenticate with your S3-compatible account. You must first generate an access key ID through your S3-compatible provider.​
Secret Access Key
Required — Enter the secret access key in this text box. You must first generate a secret access key ID through your S3-compatible provider.​
Timeout
Required— The maximum amount of time, in seconds, that you want the server to wait for a response from the remote server before it generates errors.
  • You must enter a number between 30 and 300.
  • If the server does not respond before the time expires, it makes two additional attempts to contact the server.
  • If the server does not respond after those attempts, the system administrator receives an email that notes the failed attempts. The system will retry the transfer when it runs the backup process again.
 

nixon7606

Member
Aug 27, 2020
11
1
3
Centennial, CO
cPanel Access Level
Root Administrator
not sure if you read the docs here might be helpful ?

Setting​
Description​
Destination Name
Required — Enter a destination name for your backup files. This name appears in your destination table.​
Transfer System Backups to this Destination
Select this checkbox to transfer system backups to this additional destination. To use this setting, you must also select the Back up System Files checkbox in the Backup Settings tab.​
Folder
Enter the name of the directory where you wish to store your backup.​
S3 Endpoint
Required — Enter the fully qualified domain name (FQDN) for the remote server where your bucket resides. The system backs up files to the destination server through a secured connection via SSL/TLS. The FQDN that you enter must be the same as the FQDN on the server’s SSL/TLS certificate, or the connection will fail.​
Bucket
Required — Enter the name of the bucket where you wish to store your backup. You must first create the bucket through your S3-compatible provider.​
Access Key ID
Required — Enter the access key ID in this text box. Your server uses the access key to authenticate with your S3-compatible account. You must first generate an access key ID through your S3-compatible provider.​
Secret Access Key
Required — Enter the secret access key in this text box. You must first generate a secret access key ID through your S3-compatible provider.​
Timeout
Required— The maximum amount of time, in seconds, that you want the server to wait for a response from the remote server before it generates errors.​
  • You must enter a number between 30 and 300.
  • If the server does not respond before the time expires, it makes two additional attempts to contact the server.
  • If the server does not respond after those attempts, the system administrator receives an email that notes the failed attempts. The system will retry the transfer when it runs the backup process again.
We have read the documentation. Is there a way to increase the log level so that we can see what is happening? We aren't even seeing traffic via tcpdump from cpanel to the minio server. What S3 client does cpanel use? is there a way to use it via the command line for testing?
 

cPRex

Jurassic Moderator
Staff member
Oct 19, 2014
10,360
1,632
363
cPanel Access Level
Root Administrator
I don't have a good way to change the backup verbosity. I'm not sure there is a specific "client" that we used, although it is based around an S3 perl module.

This sounds like it would be a good request to open a ticket with our team so we can do some more investigating directly on an affected system.