The Community Forums

Interact with an entire community of cPanel & WHM users!
  1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

S3 Backups not being deleted

Discussion in 'Data Protection' started by platinum, Jun 4, 2015.

  1. platinum

    platinum Active Member

    Joined:
    Nov 21, 2004
    Messages:
    30
    Likes Received:
    0
    Trophy Points:
    6
    There's a few threads I've seen, but as far as I could work out - the current version of cPanel with S3 backups is meant to behave the same was as regular backups, i.e. only retain the latest set of weekly/monthly/daily.

    My daily backups are just building up each day, causing a ton of space to be used. I don't really want to use the auto-expire feature of files, because it would be better to explicitly have them deleted by the backup system.

    grep BACKUP_DAILY_RETENTION /var/cpanel/backups/config = 1

    Also checked that my Amazon bucket policy allows for deletion of files, it's happening the same on 2 servers I'm testing with.

    Edit: should also add, on WHM 11.50.0 (build 9)
     
  2. cPanelMichael

    cPanelMichael Forums Analyst
    Staff Member

    Joined:
    Apr 11, 2011
    Messages:
    30,678
    Likes Received:
    648
    Trophy Points:
    113
    cPanel Access Level:
    Root Administrator
  3. platinum

    platinum Active Member

    Joined:
    Nov 21, 2004
    Messages:
    30
    Likes Received:
    0
    Trophy Points:
    6
    Ah I see - I thought that might have been implemented as the last reply on that feature request says it does work, and the most recent thread about it here was asking about how to stop it deleting files.

    I also get these messages in the transporter log, which say it's attempting to prune?

    Code:
    [2015-06-05 08:04:16 +1000] info [cpbackup_transporter] Starting a "prune" operation on the "Server-Name" destination ID XXXXX
     
  4. cPanelMichael

    cPanelMichael Forums Analyst
    Staff Member

    Joined:
    Apr 11, 2011
    Messages:
    30,678
    Likes Received:
    648
    Trophy Points:
    113
    cPanel Access Level:
    Root Administrator
    Hello,

    Yes, this message indicates it's pruning the expired backups on the remote destination.

    Thank you.
     
  5. platinum

    platinum Active Member

    Joined:
    Nov 21, 2004
    Messages:
    30
    Likes Received:
    0
    Trophy Points:
    6
    Well this is confusing haha.

    So it does prune expired backups on a remote destination (in this case S3) ?
    What qualifies an "expired" backup - as at the moment it seems like it's just storing everything.
     
  6. cPanelMichael

    cPanelMichael Forums Analyst
    Staff Member

    Joined:
    Apr 11, 2011
    Messages:
    30,678
    Likes Received:
    648
    Trophy Points:
    113
    cPanel Access Level:
    Root Administrator
    Yes, for instance, daily backups are replaced with new daily backups. There is an internal case open (#111053) to address an issue where Amazon backups are not being pruned from Amazon S3 destination servers, however you should see errors in /usr/local/cpanel/logs/cpbackup_transporter.log when this happens.

    Thank you.
     
  7. LostNerd

    LostNerd Well-Known Member

    Joined:
    Mar 12, 2014
    Messages:
    258
    Likes Received:
    11
    Trophy Points:
    18
    Location:
    Hastings, East Sussex, UK
    cPanel Access Level:
    Root Administrator
    Twitter:
    There is also another internal case open (#187065) which has recently been discovered to be relevant to the S3 platform, not just FTP:
    Code:
    187065 - Pruning remote backups can fail when FTP directory listing format is different than expected
    
    The definition I've received from P. Elsner is:
    For cPanel Staff:
    Support Ticket ID: 6578307 for more info on what I have quoted.
     
  8. platinum

    platinum Active Member

    Joined:
    Nov 21, 2004
    Messages:
    30
    Likes Received:
    0
    Trophy Points:
    6
    Here's my log:

    Code:
    [2015-06-11 12:08:11 +1000] info [cpbackup_transporter] Starting a "prune" operation on the "XXXXXX" destination ID "XXXXXX".
    [2015-06-11 12:08:12 +1000] info [cpbackup_transporter] cpbackup_transporter - Checking queue for tasks
    [2015-06-11 12:08:12 +1000] info [cpbackup_transporter] cpbackup_transporter - Processing next task
    [2015-06-11 12:08:12 +1000] info [cpbackup_transporter] cpbackup_transporter - Checking queue for tasks
    [2015-06-11 12:08:12 +1000] info [cpbackup_transporter] cpbackup_transporter - Exiting - the queue has been emptied; no more work to do
    [2015-06-11 12:08:12 +1000] info [cpbackup_transporter] cPanel Backup Transporter Queue Daemon is being stopped.
    
    Nothing has been removed, I've got a backlog of about 10 daily backups built up again.

    Is there anything else I can check or is this a bug?
     
  9. cPanelMichael

    cPanelMichael Forums Analyst
    Staff Member

    Joined:
    Apr 11, 2011
    Messages:
    30,678
    Likes Received:
    648
    Trophy Points:
    113
    cPanel Access Level:
    Root Administrator
    Feel free to open a support ticket using the link in my signature so we can take a closer look. You can post the ticket number here so we can update this thread with the outcome.

    Thank you.
     
Loading...

Share This Page