I know legacy backup/restore is dead, but today I need restore a cpanel server with legacy restore, 1Tb of data.
All data stored on a backup hdd, without compression, without .tar or .tar.gz - my backup is incremental, file by file, folder by folder.
This is how legacy restore work:
1) copy the account backup to a temporary folder inside the backup disk;
2) copy the data copied to temporary folder, to the final destination (/home);
Repeat this 600 times, with 1Tb of data.
A big waste of time and resources, why legacy copy to a temporary location to after this restore the account? Why not restore direct from /backup/cpbackup/daily/account_login/ ????
This temporary copy double the time necessary to restore, and stress the backup disk with a lot of writes.
I dont know how the new restore system work, I hope not in the same way. This dont make sense, just lock backups while do restore.
Yes but I will lose more time, to compact the backup to .tar.gz, and when use /restorepkg /backup/cpbackup/daily/user.tar.gz this script will create temporary files again on /backup/cpabackup/daily/cpanelpkgrestore.TMP.work.SOMETHING/
Compact to .tar.gz to restore will not avoid cpanel to use the temporary folder.
cpanel legacy restore ALWAYS copy the backup to a temporary folder before restore. This step is not necessary since in a incremental backup all files are uncompressed.
You can browse to "WHM >> Backup Configuration", configure the "Incremental" backup type, configure the local backup directory to the partition where the incremental backups are stored, and then move the existing incremental backups to the "/$configured-backup-dir/date/accounts" location. Once you do this, the backups should appear in WHM's Backup Restoration interface under the appropriate account and date.
You can browse to "WHM >> Backup Configuration", configure the "Incremental" backup type, configure the local backup directory to the partition where the incremental backups are stored, and then move the existing incremental backups to the "/$configured-backup-dir/date/accounts" location. Once you do this, the backups should appear in WHM's Backup Restoration interface under the appropriate account and date.
Temporarily files are utilized, however the restore feature runs a check before starting the restoration process to ensure enough disk space is available. EX:
Code:
Target “/backup/2017-08-17/accounts” on host “123.hostname.tld” has 11.16 GB free and requires at least 67.4 MB free, which includes space for temporary files.
Today I run a test: using the new backup system, I restored one account with 14Gb.
Unfortunately the backup system first copie the entire backup (14Gb) to a temporary directory /backup/date/accounts/cpanelpkgrestore.TMP.work.RuFrBXh5
Well... I have the incremental backup, so all files on my backup are ready to be restored (no .tar or .tar.gz).
The restore system copy all data to a temporary folder, and only after this restore the account - dont make any sense with a incremental backup setup.
This "step" (copy to temporary folder) only add time to restore and unnecessarily generates writing on the backup disk, which may be undesirable, because when restoring a backup is a moment that can be critical and the last thing we want is to stress the backup disk.
I understand when the backup have the format .tar or .tar.gz is necessary copy to a temporary folder to extract the files, but with "incremental" option this temporary copy is useless.
Maybe someone from cpanel reads this and agrees with my argument, if any, thank you bring the matter to the developers. There could be an automatic scan if the backup is incremental so do not use temporary folder.
To finish: 1Tb restore of 600 accounts, in a new hardware (Xeon 1270 V6 + 32Gb RAM + Adaptec RAID1 7.2K) took 30 hours. Half this spent with cpanel copying backup to temporary folders unnecessarily.
However, I encourage you to open a separate feature request if you'd like to see new functionality that doesn't make use of a temporary directory for restoration purposes:
However, I encourage you to open a separate feature request if you'd like to see new functionality that doesn't make use of a temporary directory for restoration purposes:
I will do this, and I will pray that other users will need to do incremental backup restore faster, so maybe - with many votes - cpanel.net decides to make this small change.
We are struggling with the same problem. We have a file level backup from users accounts on a remote server. After we mount the remote server's disk, and try to restore the backup with /scripts/restorepkg /mnt/something/$USERNAME, the script make a copy from the given user's backup, which make a very big - and unnecessary - overhead.
I will do this, and I will pray that other users will need to do incremental backup restore faster, so maybe - with many votes - cpanel.net decides to make this small change.
In the meantime, how large are the individual accounts you are attempting to restore? You may want to consider restoring the accounts without the home directory, and then manually moving the home directory in place after the account itself is restored.
Our problem is a little bit different.
We have uncompressed backup on remote server. We attached the backup server's file system with NFS (we tried with ISCSI also, but the problem is the same).
We can restore the user from backup with /scripts/restorepkg /mnt/something/randomusername, where "randomusername" folder contains the uncompressed backup:
When we run the /scripts/restorepkg /mnt/something/randomusername the restorepkg make a full copy from /mnt/something/randomusername to /mnt/something/cpanelpkgrestoreXXXXX, see below:
[ 17059][RESTORE:1 ][A:randomusername ]: Target “/mnt/something/cpanelpkgrestore.TMP.work.86_1V5fk/unsafe_to_read_archive” on host “server.example.com” has 806.22 GB free and requires at least 491.08 MB free, which includes space for temporary files.
[ 17059][RESTORE:1 ][A:randomusername ]: Target “/mnt/something/cpanelpkgrestore.TMP.work.86_1V5fk/unsafe_to_read_archive” on host “server.example.com” has 60,503,178 inodes free and requires at least 1,472 inodes free, which includes space for temporary files.
During the restorepkg, I ran du -sh /mnt/something/cpanelpkgrestore.TMP.work.86_1V5fk which returned with nearly the same size as mentoined on the restorepkg's output (494 Mb).
After the copy, the /scripts/restorepkg restore the user, but this copy is clearly unneccessary and make big overhead, because the data copied twice. Why is it important to copy the uncompressed backup to another folder? Can the restorepkg skip this step?
Also, I checked the /scripts/restorepkg help, but it does not have any option to skip home directory:
--allow_reseller Restore reseller privileges if the archive contains them.
NOTE: This option is only available in unrestricted mode.
--force Force the package to restore and overwrite all account settings
and databases. If the account already exists on the system
this implies --skipaccount.
--ip Assign a dedicated IP address. You may specify a particular
IP address, or “y” to have one randomly assigned.
--newuser Specify the username to use for the restored account instead
of the one contained within the archive. Databases and DB
users will retain their names unless a conflict is detected.
This option cannot be used with --skipaccount.
--skipaccount Restore the package into the existing account with the same
username. This option cannot be used in conjunction with
--newuser.
--restricted Use the Restricted Restore feature to restore the package.
--unrestricted Restore the package in unrestricted mode. This flag is on by
default and overridden by the --restricted flag.
If we can skip the home directory, it would be nice.
The temporary directory is for staging purposes during the restoration process. It's how the integrity of the backup archive used for the restore is validated. You could open a separate feature request if you have a specific request to change this behavior. As far as restoring the account without the home directory in the situation you described, you'd need to move the account's public_html directory from the directory you are restoring it from to another location (outside of the account).
4 years later and the restore process is still wasting time and resources duplicating incremental backups.
Yesterday I needed to restore a 220GB account on a incremental backup. It took SIX hours just to duplicate HALF the backup on the "cpanelpkgrestore.TMP.work.xxx" folder!
As a workaround , you can move the huge folders to somewhere else outside the user backup, restore the account and then rsync the folder to the restored account manually.
I hope you find this thread before wasting a whole day waiting on a backup restoration.
4 years later and the restore process is still wasting time and resources duplicating incremental backups.
Yesterday I needed to restore a 220GB account on a incremental backup. It took SIX hours just to duplicate HALF the backup on the "cpanelpkgrestore.TMP.work.xxx" folder!
As a workaround , you can move the huge folders to somewhere else outside the user backup, restore the account and then rsync the folder to the restored account manually.
I hope you find this thread before wasting a whole day waiting on a backup restoration.
As Michael suggested previously, you could open a feature request using the link in my signature if you are interested in seeing an option for restoring backups without temporary space for an integrity check. If you do so, please update this thread.
