Empty weekly folders on destination

kabatak

Well-Known Member
Jun 10, 2009
148
8
68
I noticed the destination server contains a lot of empty weekly folders like the following screenshot:

cpbackup.png

Each of the folders (with dates as names) contains an "accounts" folder which is also empty e.g., 2018-06-27/accounts/

What are the purpose of these empty folders? Are they supposed to be automatically deleted at some point?
 

cPanelLauren

Product Owner
Staff member
Nov 14, 2017
13,295
1,257
313
Houston
Hi @kabatak

The folders should be removed along with data in accordance to your retention policy for automated backups. I would suggest looking at the backup transport logs to see if there is a noted issue with their removal. You can get to them by going to:

Code:
/usr/local/cpanel/logs/cpbackup_transporter/
 

kabatak

Well-Known Member
Jun 10, 2009
148
8
68
Hi @kabatak

The folders should be removed along with data in accordance to your retention policy for automated backups. I would suggest looking at the backup transport logs to see if there is a noted issue with their removal. You can get to them by going to:

Code:
/usr/local/cpanel/logs/cpbackup_transporter/
What sort of message should I be looking for?

Note: The retention policy works fine inside the weekly folder.

E.g., folder backup/ contains/weekly/ and weekly/ contains the actual backup files, so no problems with that.

What I'm talking about is the folder backup/ which also contains the empty folders as seen in the OPs screenshot.
 

cPanelLauren

Product Owner
Staff member
Nov 14, 2017
13,295
1,257
313
Houston
Hello @kabatak

I understood what you were referencing. You would be looking for errors in those logs. If you're unsure what constitutes an error you can post the transport log for the entire backup when it is removing data.

Thanks!
 

cPanelLauren

Product Owner
Staff member
Nov 14, 2017
13,295
1,257
313
Houston
Hi @kabatak

The entire prune operation is here:
Code:
[2018-09-05 08:01:55 +0800] info [cpbackup_transporter] Starting a "prune" operation on the "ovh_xxx_backup" destination ID "t3xWL7BqBZ6JpxQx6GSDaRmF".
[2018-09-05 08:01:55 +0800] info [cpbackup_transporter] Performing prune operation, retaining 2 items on:  ovh_xxx_backup
[2018-09-05 08:01:55 +0800] info [cpbackup_transporter] Net::FTP=GLOB(0x1b32018) >>>  PASV
[2018-09-05 08:01:55 +0800] info [cpbackup_transporter] Net::FTP=GLOB(0x1b32018) <<<  227 Entering Passive Mode (178,32,222,126,117,144).
[2018-09-05 08:01:55 +0800] info [cpbackup_transporter] Net::FTP=GLOB(0x1b32018) >>>  LIST weekly
[2018-09-05 08:01:55 +0800] info [cpbackup_transporter] Net::FTP=GLOB(0x1b32018) <<<  150 Opening BINARY mode data connection for file list
[2018-09-05 08:01:55 +0800] info [cpbackup_transporter] Net::FTP=GLOB(0x1b32018) <<<  226 Transfer complete
[2018-09-05 08:01:55 +0800] info [cpbackup_transporter] Net::FTP=GLOB(0x1b32018) >>>  PASV
[2018-09-05 08:01:55 +0800] info [cpbackup_transporter] Net::FTP=GLOB(0x1b32018) <<<  227 Entering Passive Mode (178,32,222,126,121,35).
[2018-09-05 08:01:55 +0800] info [cpbackup_transporter] Net::FTP=GLOB(0x1b32018) >>>  LIST weekly/2018-09-05
[2018-09-05 08:01:56 +0800] info [cpbackup_transporter] Net::FTP=GLOB(0x1b32018) <<<  150 Opening BINARY mode data connection for file list
[2018-09-05 08:01:56 +0800] info [cpbackup_transporter] Net::FTP=GLOB(0x1b32018) <<<  226 Transfer complete
[2018-09-05 08:01:56 +0800] info [cpbackup_transporter] Pruning backup directory:  weekly/2018-08-22, from ovh_xxx_backup
[2018-09-05 08:01:56 +0800] info [cpbackup_transporter] Net::FTP=GLOB(0x1b32018) >>>  RMD weekly/2018-08-22
[2018-09-05 08:01:56 +0800] info [cpbackup_transporter] Net::FTP=GLOB(0x1b32018) <<<  550 weekly/2018-08-22: Directory not empty
[2018-09-05 08:01:56 +0800] info [cpbackup_transporter] Net::FTP=GLOB(0x1b32018) >>>  PASV
[2018-09-05 08:01:56 +0800] info [cpbackup_transporter] Net::FTP=GLOB(0x1b32018) <<<  227 Entering Passive Mode (178,32,222,126,120,24).
[2018-09-05 08:01:56 +0800] info [cpbackup_transporter] Net::FTP=GLOB(0x1b32018) >>>  MLSD weekly/2018-08-22
[2018-09-05 08:01:56 +0800] info [cpbackup_transporter] Net::FTP=GLOB(0x1b32018) <<<  150 Opening ASCII mode data connection for MLSD
[2018-09-05 08:01:57 +0800] info [cpbackup_transporter] Net::FTP=GLOB(0x1b32018) <<<  226 Transfer complete
[2018-09-05 08:01:57 +0800] info [cpbackup_transporter] Net::FTP=GLOB(0x1b32018) >>>  DELE weekly/2018-08-22/system
[2018-09-05 08:01:57 +0800] info [cpbackup_transporter] Net::FTP=GLOB(0x1b32018) <<<  550 weekly/2018-08-22/system: Is a directory
[2018-09-05 08:01:57 +0800] info [cpbackup_transporter] Net::FTP=GLOB(0x1b32018) >>>  RMD weekly/2018-08-22/system
[2018-09-05 08:01:57 +0800] info [cpbackup_transporter] Net::FTP=GLOB(0x1b32018) <<<  550 weekly/2018-08-22/system: Directory not empty
[2018-09-05 08:01:57 +0800] info [cpbackup_transporter] Net::FTP=GLOB(0x1b32018) >>>  PASV
[2018-09-05 08:01:57 +0800] info [cpbackup_transporter] Net::FTP=GLOB(0x1b32018) <<<  227 Entering Passive Mode (178,32,222,126,123,53).
[2018-09-05 08:01:57 +0800] info [cpbackup_transporter] Net::FTP=GLOB(0x1b32018) >>>  MLSD weekly/2018-08-22/system
[2018-09-05 08:01:57 +0800] info [cpbackup_transporter] Net::FTP=GLOB(0x1b32018) <<<  150 Opening ASCII mode data connection for MLSD
[2018-09-05 08:01:57 +0800] info [cpbackup_transporter] Net::FTP=GLOB(0x1b32018) <<<  226 Transfer complete
[2018-09-05 08:01:57 +0800] info [cpbackup_transporter] Net::FTP=GLOB(0x1b32018) >>>  DELE weekly/2018-08-22/system/system_files.tar
[2018-09-05 08:01:58 +0800] info [cpbackup_transporter] Net::FTP=GLOB(0x1b32018) <<<  250 DELE command successful
[2018-09-05 08:01:58 +0800] info [cpbackup_transporter] Net::FTP=GLOB(0x1b32018) >>>  RMD weekly/2018-08-22/system
[2018-09-05 08:01:58 +0800] info [cpbackup_transporter] Net::FTP=GLOB(0x1b32018) <<<  250 RMD command successful
[2018-09-05 08:01:58 +0800] info [cpbackup_transporter] Net::FTP=GLOB(0x1b32018) >>>  DELE weekly/2018-08-22/accounts
[2018-09-05 08:01:58 +0800] info [cpbackup_transporter] Net::FTP=GLOB(0x1b32018) <<<  550 weekly/2018-08-22/accounts: Is a directory
[2018-09-05 08:01:58 +0800] info [cpbackup_transporter] Net::FTP=GLOB(0x1b32018) >>>  RMD weekly/2018-08-22/accounts
[2018-09-05 08:01:58 +0800] info [cpbackup_transporter] Net::FTP=GLOB(0x1b32018) <<<  550 weekly/2018-08-22/accounts: Directory not empty
[2018-09-05 08:01:58 +0800] info [cpbackup_transporter] Net::FTP=GLOB(0x1b32018) >>>  PASV
[2018-09-05 08:01:58 +0800] info [cpbackup_transporter] Net::FTP=GLOB(0x1b32018) <<<  227 Entering Passive Mode (178,32,222,126,117,135).
[2018-09-05 08:01:58 +0800] info [cpbackup_transporter] Net::FTP=GLOB(0x1b32018) >>>  MLSD weekly/2018-08-22/accounts
[2018-09-05 08:01:58 +0800] info [cpbackup_transporter] Net::FTP=GLOB(0x1b32018) <<<  150 Opening ASCII mode data connection for MLSD
[2018-09-05 08:01:59 +0800] info [cpbackup_transporter] Net::FTP=GLOB(0x1b32018) <<<  226 Transfer complete
[2018-09-05 08:01:59 +0800] info [cpbackup_transporter] Net::FTP=GLOB(0x1b32018) >>>  DELE weekly/2018-08-22/accounts/xxx_account.tar.gz
[2018-09-05 08:02:17 +0800] info [cpbackup_transporter] Net::FTP=GLOB(0x1b32018) <<<  250 DELE command successful
[2018-09-05 08:02:17 +0800] info [cpbackup_transporter] Net::FTP=GLOB(0x1b32018) >>>  RMD weekly/2018-08-22/accounts
[2018-09-05 08:02:17 +0800] info [cpbackup_transporter] Net::FTP=GLOB(0x1b32018) <<<  250 RMD command successful
[2018-09-05 08:02:17 +0800] info [cpbackup_transporter] Net::FTP=GLOB(0x1b32018) >>>  RMD weekly/2018-08-22
[2018-09-05 08:02:17 +0800] info [cpbackup_transporter] Net::FTP=GLOB(0x1b32018) <<<  250 RMD command successful
Ending with:
Code:
[2018-09-05 08:02:17 +0800] info [cpbackup_transporter] Net::FTP=GLOB(0x1b32018) >>>  RMD weekly/2018-08-22
[2018-09-05 08:02:17 +0800] info [cpbackup_transporter] Net::FTP=GLOB(0x1b32018) <<<  250 RMD command
This shows the pruning of the weekly though not the daily and the issue seems to be the daily backups that aren't getting pruned.

Can you run the following and paste the output in your response:

Code:
cat /var/cpanel/backups/config
You might look through all the backup logs for pruning as well:

Code:
grep -ir prun /usr/local/cpanel/logs/cpbackup_transporter/
 

kabatak

Well-Known Member
Jun 10, 2009
148
8
68
Here's the output:

Code:
[email protected] [/usr/local/cpanel/logs/cpbackup_transporter]# cat /var/cpanel/backups/config
---
BACKUPACCTS: 'yes'
BACKUPBWDATA: 'yes'
BACKUPDAYS: 0
BACKUPDIR: /backup
BACKUPENABLE: 'yes'
BACKUPFILES: 'yes'
BACKUPLOGS: 'yes'
BACKUPMOUNT: 'no'
BACKUPSUSPENDEDACCTS: 'yes'
BACKUPTYPE: compressed
BACKUP_DAILY_ENABLE: 'no'
BACKUP_DAILY_RETENTION: 3
BACKUP_MONTHLY_DATES: 1
BACKUP_MONTHLY_ENABLE: 'no'
BACKUP_MONTHLY_RETENTION: 1
BACKUP_WEEKLY_DAY: 3
BACKUP_WEEKLY_ENABLE: 'yes'
BACKUP_WEEKLY_RETENTION: 2
CHECK_MIN_FREE_SPACE: 1
DISABLE_METADATA: 'no'
ERRORTHRESHHOLD: 3
FORCE_PRUNE_DAILY: 0
FORCE_PRUNE_MONTHLY: 0
FORCE_PRUNE_WEEKLY: 0
GZIPRSYNCOPTS: --rsyncable
KEEPLOCAL: 0
LINKDEST: 0
LOCALZONESONLY: 'no'
MAXIMUM_RESTORE_TIMEOUT: 21600
MAXIMUM_TIMEOUT: 2700
MIN_FREE_SPACE: 5
MIN_FREE_SPACE_UNIT: percent
MYSQLBACKUP: both
POSTBACKUP: 'no'
PREBACKUP: -1
PSQLBACKUP: 'no'
USEBINARYPKGACCT: 'no'
 

cPanelLauren

Product Owner
Staff member
Nov 14, 2017
13,295
1,257
313
Houston
Hi @kabatak

This indicates that you don't have daily backups enabled
Code:
BACKUP_DAILY_ENABLE: 'no'
You only have weekly enabled and we can see folders in weekly/ are removed.

Code:
[2018-09-05 08:01:58 +0800] info [cpbackup_transporter] Net::FTP=GLOB(0x1b32018) <<<  227 Entering Passive Mode (178,32,222,126,117,135).
[2018-09-05 08:01:58 +0800] info [cpbackup_transporter] Net::FTP=GLOB(0x1b32018) >>>  MLSD weekly/2018-08-22/accounts
[2018-09-05 08:01:58 +0800] info [cpbackup_transporter] Net::FTP=GLOB(0x1b32018) <<<  150 Opening ASCII mode data connection for MLSD
[2018-09-05 08:01:59 +0800] info [cpbackup_transporter] Net::FTP=GLOB(0x1b32018) <<<  226 Transfer complete
[2018-09-05 08:01:59 +0800] info [cpbackup_transporter] Net::FTP=GLOB(0x1b32018) >>>  DELE weekly/2018-08-22/accounts/xxx_account.tar.gz
[2018-09-05 08:02:17 +0800] info [cpbackup_transporter] Net::FTP=GLOB(0x1b32018) <<<  250 DELE command successful
[2018-09-05 08:02:17 +0800] info [cpbackup_transporter] Net::FTP=GLOB(0x1b32018) >>>  RMD weekly/2018-08-22/accounts
[2018-09-05 08:02:17 +0800] info [cpbackup_transporter] Net::FTP=GLOB(0x1b32018) <<<  250 RMD command successful
[2018-09-05 08:02:17 +0800] info [cpbackup_transporter] Net::FTP=GLOB(0x1b32018) >>>  RMD weekly/2018-08-22
[2018-09-05 08:02:17 +0800] info [cpbackup_transporter] Net::FTP=GLOB(0x1b32018) <<<  250 RMD command successful
Did you recently make changes to the backup configuration? Are the empty folders still being created? When the backup is created what folder are they placed in
Code:
weekly/yyyy-mo-day/accounts/accountname.tar.gz
or are they placed elsewhere?

Based on what you're showing me it seems the backup is functioning as intended and I'm not sure why those folders exist there. In order to investigate this issue more in depth can you please open a ticket using the link in my signature? Once open please reply with the Ticket ID here so that we can update this thread with the resolution once the ticket is resolved.


Thanks!