In Progress [CPANEL-21358] Unable to validate rsync backup destination

cPanelMichael

Administrator
Staff member
Apr 11, 2011
47,909
2,227
463
Hello @Michael Svendsen,

I don't have a specific time frame to provide at this time. I'll update this thread as soon as a tentative release date is available.

Thank you.
 

feanorknd

Member
Sep 28, 2005
21
1
153
I am being affected by this serious bug.

Did purchase server with Cpanel using Hetzner provider, and did figure out being able to use incremental / rsync backups over Hetzner Backup Space, which is completelly functional if using rsync manually. This way, my hard drive space available is tight and fixed up to my account requirements.

Now, Cpanel is not able to validate a perfectly functional remote rsync source, although using it to copy temp files inside without problems. ¿?¿?

Now I cannot use incremental rsyncs because of this, which will harm me because of CPU/MEM requirements for backups, as well as required space.

And discovered Cpanel development team could not fix this bug for months... :(

Seriously?

Hello @Michael Svendsen,

I don't have a specific time frame to provide at this time. I'll update this thread as soon as a tentative release date is available.

Thank you.
 

cPanelMichael

Administrator
Staff member
Apr 11, 2011
47,909
2,227
463
Hello @feanorknd,

The Rsync backup destination type included with cPanel & WHM requires full SSH access to the remote destination due to the need to execute multiple commands over SSH during the backup transportation stage. For example, directory creation tasks and backup pruning tasks are performed via the execution of the mkdir and rm commands on the remote backup destination over SSH.

Backup providers such as Hetzner permit the use of the Rsync command, but they do not permit the execution of additional commands over SSH.

CPANEL-21358 remains open to request a feature improvement to the Rsync destination type so that the additional SSH commands are not required, however I don't have a firm time frame to offer on when we might see a change to the current functionality. I'll continue to monitor the case and update this thread with any new information as it becomes available.

Thank you.
 

feanorknd

Member
Sep 28, 2005
21
1
153
Great explanation, sincerely

Hope you could improve these in the future. Regards,


Hello @feanorknd,

The Rsync backup destination type included with cPanel & WHM requires full SSH access to the remote destination due to the need to execute multiple commands over SSH during the backup transportation stage. For example, directory creation tasks and backup pruning tasks are performed via the execution of the mkdir and rm commands on the remote backup destination over SSH.

Backup providers such as Hetzner permit the use of the Rsync command, but they do not permit the execution of additional commands over SSH.

CPANEL-21358 remains open to request a feature improvement to the Rsync destination type so that the additional SSH commands are not required, however I don't have a firm time frame to offer on when we might see a change to the current functionality. I'll continue to monitor the case and update this thread with any new information as it becomes available.

Thank you.
 
  • Like
Reactions: cPanelMichael

uk01

Well-Known Member
Dec 31, 2009
206
19
68
We are now testing Hetner storage. What does seem to work is if you backup to another vps with root access first, then rsync with the -H hardlinks preserved this works.
Then Hetner would be your secondary storage.

Also tried backblaze with rclone but doesn't support hardlinks!

EDIT: Of course if you use ESXi and have a server with SSD and HDD you could add a backup datastore then rsync from there to hetner for offsite.
 

hhvivek

Registered
Jul 12, 2020
1
0
1
INDIA
cPanel Access Level
Root Administrator
I am also facing the same issue, and I checked this thread and found people are facing it from 2+ years and yet the cpanel didn't solve it.
can you tell us any update on this thread?
 

cPAdminsMichael

Well-Known Member
Dec 19, 2016
176
62
103
Denmark
cPanel Access Level
Root Administrator
Yeah - We ended up doing a small ssh-fuse workaround, rsyncing backups to ssh-mounted volumes. Worked pretty good for the past years.
Today we just have local rsync storage servers which we also offer to customers of ours.
Other alternatives out there are rsync.net - but my experience is that the bandwidth to their servers varies a lot.
 
  • Like
Reactions: AzeDK

cPAdminsMichael

Well-Known Member
Dec 19, 2016
176
62
103
Denmark
cPanel Access Level
Root Administrator
Hi guys,

Small insider ... JetBackup is going to support Hetzner Storage Boxes (and other rsync-only storage devices) in upcoming major version (v5). Release planned at end of the year, so if you are JetBackup customer, expect a solution maybe within 6 months :)
 
  • Like
Reactions: AzeDK

ServerMechanic

Registered
Apr 28, 2021
1
0
1
Romania
cPanel Access Level
Root Administrator
Hi

Here is my solution that it works.

1) Mount the Hetzner backup as they describe it here Access with SAMBA/CIFS - Hetzner Docs - I have mounted in to /mnt/backup/cpanel
2) generate a new ssh-key and store the public key it on the same server, add it to the /USER/.ssh/authorized_keys
4) Symlink the /mnt/backup/cpanel to /USER/backup (uase any name you desire)
5) on the cpanel backup create new rsync backup, the backup path should be the symlink (in my case backup), IP/hostname address of the server (no localhost or 127.0.0.0) USER, select he private_key
6) test and validate

Note: I sued root user and ssh root login was disabledin ssh but I used the MatchAddress to allow root only from specific IP.

Enjoy!
 

cPRex

Jurassic Moderator
Staff member
Oct 19, 2014
6,810
895
313
cPanel Access Level
Root Administrator
@noticet - thanks for your patience on this. I spoke with one of our developers about the rsync issue and we'd decided to do the following:

-change the wording in the "Additional Destinations" interface to be "Rsync+SSH" for that destination type
-update the documentation to reflect this change
-change the existing case to a feature request to get a more native rsync implementation included in the product. That is on the team's radar as it's a known issue, but I can't provide any type of timeframe on that work.

The current issue is that our rsync tool in the backup system relies on SSH as well, so remote systems expecting "pure" rsync connections and protocols currently do not give the desired results.