How can I reduce size of staging backup drive

cPanel Community,

I am transporting my backup files to Amazon S3 for storage.

How can I reduce the size of my backup drive? It is empty most of the time. My backups are 330 GB and my backup drive is 600 GB. This mostly unused drive is costing me about $60 per month.

Thanks,

Jim

 

I was hoping we could figure out a way to write directly to Amazon S3, to store and transfer backups in smaller chunks, or transfer backups as they are being made, rather than later.

Any ideas?

Thanks,

Jim

 

Infopro,

What if I created an every-5-min cron job to periodically sweep the backup files to the Amazon S3 storage?

Like this:

aws s3 mv /backup s3://<bucket-name> --recursive

or just the tar balls:

for i in /backup/*/*tar; do aws s3 mv $i s3://<bucket-name>;done;
for i in /backup/*/accounts/*tar; do aws s3 mv $i s3://<bucket-name>;done;
for i in /backup/weekly/*/*tar; do aws s3 mv $i s3://<bucket-name>;done;
for i in /backup/weekly/*/accounts/*tar; do aws s3 mv $i s3://<bucket-name>;done;
***

Thanks,

Jim

 

I wasn't using compressed backups, which meant that my backups were completed is less time (3.5 hours) and that my backup files were larger (300 GB) which caused the cpbackup_transport process to take longer (9 hours).

I switched to compressed backups and the cpbackup_transport process was able to keep up with the backup process. The file size dropped to 220 GB and the both processes ended after 8 hours.

I dropped my backup drive size to 300 GB.

Thanks,

Jim