Please whitelist cPanel in your adblocker so that you’re able to see our version release promotions, thanks!

The Community Forums

Interact with an entire community of cPanel & WHM users!

[CPANEL-14896] Restoring Incremental Backups from NSFv4 mounts

Discussion in 'Data Protection' started by dzung042, Sep 19, 2017.

Tags:
  1. dzung042

    dzung042 Registered

    Joined:
    Feb 10, 2015
    Messages:
    2
    Likes Received:
    0
    Trophy Points:
    1
    cPanel Access Level:
    DataCenter Provider
    After upgrade I have error restore account,
    "Could not restore account “uyozcizu”: Account Restore Failed: “The system could not load the cPanel user attributes file from the archive because of an error: (XID 98vm2v) The system failed to lock (LOCK_EX, LOCK_NB) the file “/backup/2017-09-18/accounts/cpanelpkgrestore.TMP.work.O3eDqQLI/safe_to_read_archive/contents/cp/uyozcizu” because of an error: Bad file descriptor at /usr/local/cpanel/Whostmgr/Transfers/Utils.pm line 234."
    Any one help me resolve
     
  2. cPanelMichael

    cPanelMichael Technical Support Community Manager
    Staff Member

    Joined:
    Apr 11, 2011
    Messages:
    44,802
    Likes Received:
    1,895
    Trophy Points:
    363
    cPanel Access Level:
    Root Administrator
    Twitter:
    Hello,

    Can you verify if the location of your backup directory is configured as a NFS mount?

    Thank you.
     
    Stop hovering to collapse... Click to collapse... Hover to expand... Click to expand...
  3. feday2

    feday2 Registered

    Joined:
    Apr 5, 2016
    Messages:
    3
    Likes Received:
    0
    Trophy Points:
    1
    Location:
    Kiev, Ukraine
    cPanel Access Level:
    Root Administrator
    I have the same problem, and /backup/ - NFS mount disk.
     
  4. cPanelMichael

    cPanelMichael Technical Support Community Manager
    Staff Member

    Joined:
    Apr 11, 2011
    Messages:
    44,802
    Likes Received:
    1,895
    Trophy Points:
    363
    cPanel Access Level:
    Root Administrator
    Twitter:
    Hello,

    Internal case CPANEL-14896 is open to report an issue where it's not possible to restore incremental backups from a NFSv4 mount. I'll monitor this case and update this thread with the outcome. In the meantime, a temporary workaround is to move the incremental backup directory that you want to restore outside of the NFS mount and restore it using a command like this:

    Code:
    /scripts/restorepkg /path/to/inremental/user/
    One additional workaround to consider is to set the NFS mount options to "local_lock=all,nolock" to see if that helps.

    Thank you.
     
    Stop hovering to collapse... Click to collapse... Hover to expand... Click to expand...
  5. Intercloud

    Intercloud Registered

    Joined:
    Sep 25, 2017
    Messages:
    3
    Likes Received:
    0
    Trophy Points:
    1
    Location:
    Israel
    cPanel Access Level:
    DataCenter Provider
    Hi cPanelMichael,
    Even though I am using this forum for years, I have just created my first account solely to answer you.

    I am having the exact same problem with about 3 of my WHM servers (could be more, but these are the ones I found out about).

    The scenario in my case is a version 4 NFS server that is mounted to each of my WHM servers, and is used to save their backups that are created using WHM intergrated backup system. I am not using incremental backups but only full backups of all cPanel accounts.

    I also get this kind of message when trying to restore a backup:
    Could not restore account “rotemintercloud”: Account Restore Failed: “The system could not load the cPanel user attributes file from the archive because of an error: (XID k8mqzx) The system failed to lock (LOCK_EX, LOCK_NB) the file “/mnt/backup/2017-09-19/accounts/cpanelpkgrestore.TMP.work.lCXCWZG5/safe_to_read_archive/contents/cp/rotemintercloud” because of an error: Bad file descriptor at /usr/local/cpanel/Whostmgr/Transfers/Utils.pm line 234.


    I have also noticed that the WHM backup system's retention is no longer working and old backups are simply not being deleted.

    This issue has started about a month ago, following one of WHM's updates I believe. Seems like a bug to me since it worked perfectly in this exact way for years now (I have over 40 WHM servers that are configured in the exact same way).

    About your two temporary workaround:

    1. Copying the backup file localy and then running it using a script / "Restore a Full Backup/cpmove File" WHM tool is working perfectly, but it is not a solution since it is very time-consuming and my customers won't know how to do it themselves (They restore backups using WHM's web interface).

    2. Setting the NFS mount options to "local_lock=all,nolock" doesn't seem to be working on my case.

    3. Changing the mount's NFS version to 3 instead of 4 seem to be solving the problem completely, but I can only implement it to WHM servers that are mounted to one of my NFS servers that is supporting NFSv3 (I cannot change the other NFS server version since it is not supporting it).


    In conclusion, it seems to be a bug that have just recently started and should be fixed ASAP since it hurts backup retentions and restores.

    Thanks a lot for your attention and assistance on this matter.
     
  6. cPanelMichael

    cPanelMichael Technical Support Community Manager
    Staff Member

    Joined:
    Apr 11, 2011
    Messages:
    44,802
    Likes Received:
    1,895
    Trophy Points:
    363
    cPanel Access Level:
    Root Administrator
    Twitter:
    Stop hovering to collapse... Click to collapse... Hover to expand... Click to expand...
  7. orudge

    orudge Member

    Joined:
    Oct 31, 2004
    Messages:
    17
    Likes Received:
    2
    Trophy Points:
    153
    Location:
    United Kingdom
    We are also experiencing this issue, which only started in the past month or so. We have backup servers mounted via NFS, and our clients are unable to restore backups due to a "The system failed to lock (LOCK_EX, LOCK_NB)" error. Altering the backup mount in fstab to the suggested line unfortunately does not help. Nor does using nfsvers=3.

    Perhaps the cPanel code could be altered to store the temporarily extracted data in a different location (e.g., /home)?

    Thanks,
     
    Stop hovering to collapse... Click to collapse... Hover to expand... Click to expand...
  8. cPanelMichael

    cPanelMichael Technical Support Community Manager
    Staff Member

    Joined:
    Apr 11, 2011
    Messages:
    44,802
    Likes Received:
    1,895
    Trophy Points:
    363
    cPanel Access Level:
    Root Administrator
    Twitter:
    Hello,

    There's no update to report on the status of internal case CPANEL-14896 at this time. I'll continue to monitor the case and update this thread with more information as it becomes available. As far as a workaround, you should be able to move the backup archives to a non-NFS mount location and then proceed with the restore.

    Thank you.
     
    Stop hovering to collapse... Click to collapse... Hover to expand... Click to expand...
  9. Intercloud

    Intercloud Registered

    Joined:
    Sep 25, 2017
    Messages:
    3
    Likes Received:
    0
    Trophy Points:
    1
    Location:
    Israel
    cPanel Access Level:
    DataCenter Provider
    Hi cPanelMichael,
    Are there any news? Is there an estimated time for the fix to be released?

    The thing is that we have plenty of customer's WHM servers that are currently having many backup and restore problems.
    We are working with NFS based backups for our WHM servers for years now without any problems, and if this issue will take much to be solved we will need to start planning on finding an alternative solution.
    This will take a lot of time and efforts which we really prefer to avoid at the moment, so a rough estimation of the period of time for this issue to be fix will be really helpful.

    Thanks a lot for your assistance!
     
  10. cPanelMichael

    cPanelMichael Technical Support Community Manager
    Staff Member

    Joined:
    Apr 11, 2011
    Messages:
    44,802
    Likes Received:
    1,895
    Trophy Points:
    363
    cPanel Access Level:
    Root Administrator
    Twitter:
    Hello,

    It looks like we've completed work on a resolution, and it's now in under testing. There's currently no specific time frame for the publication of the resolution, but I'll monitor the case and update this thread as soon as more information is available.

    Thank you.
     
    Stop hovering to collapse... Click to collapse... Hover to expand... Click to expand...
  11. orudge

    orudge Member

    Joined:
    Oct 31, 2004
    Messages:
    17
    Likes Received:
    2
    Trophy Points:
    153
    Location:
    United Kingdom
    Hi Michael, just wondering if you have any further update on this case? I've not seen anything in any of the changelogs about it.

    Thanks,
     
    Stop hovering to collapse... Click to collapse... Hover to expand... Click to expand...
  12. cPanelMichael

    cPanelMichael Technical Support Community Manager
    Staff Member

    Joined:
    Apr 11, 2011
    Messages:
    44,802
    Likes Received:
    1,895
    Trophy Points:
    363
    cPanel Access Level:
    Root Administrator
    Twitter:
    Hi,

    Internal case CPANEL-14896 is still open, however we did publish a resolution for another NFS related issue that may help alleviate the issue on some systems:

    Fixed case CPANEL-16122: Improve cpmove restore compatibility with NFS.

    If the issue is still occurring, you are welcome to open a support ticket so we can take a closer look at your system and link your specific report to internal case CPANEL-14896.

    Thank you.
     
    Stop hovering to collapse... Click to collapse... Hover to expand... Click to expand...
Loading...

Share This Page

  1. This site uses cookies to help personalise content, tailor your experience and to keep you logged in if you register.
    By continuing to use this site, you are consenting to our use of cookies.
    Dismiss Notice