FTP Backups terminating with FTP time out for big accounts

[Kent Brockman]

Hi there people.

I've noticed that when you do a full backup (not incremental, compressed) to a remote FTP site, big accounts may not be correctly backed up because the long time consumed compressing the package may cause a time out in the FTP server, and cpbackup just bypass that package and continue with the next one instead of checking first if the ftp connection is alive. The problem I've encountered reading the cpbackup logs is that the remote sequence is:

1. connect to FTP
2. compress the account data
3. upload it to the FTP
4. close FTP connection and move on to the next account to backup

I think it actually should be:

1. compress the account data
2. connect to FTP
3. upload it to the FTP
4. close FTP connection and move on to the next account to backup

I recommend this change because sometimes there are very huge accounts that cause time out and not every admin may notice the need to extend the idle connection wait in their FTP, causing some accounts being uncopied. Also, there are many factors, like load balancing in the production server, load balancing in the backup server, internet traffic load in the production server, internet traffic load in the backup server, internet connection speed, etc, that may delay the backup of every big account far beyond the max expected FTP idle time.

Any solution?


ADDED Jan 14:
I've checked today that cPanel 11.28.64 is not ftping correctly the files. The compressed files are being stored with zero size and is creating only those belonging to small sized accounts (accounts consuming <200 MB). I see this in the cpbackup file for every ftp transaction every time an account is tried to be ftped:

Net::FTP=GLOB(0xa2b3c94)<<< 220 Microsoft FTP Service
Net::FTP=GLOB(0xa2b3c94)>>> USER Marcelo
Net::FTP=GLOB(0xa2b3c94)<<< 331 Password required for Marcelo.
Net::FTP=GLOB(0xa2b3c94)>>> PASS ....
Net::FTP=GLOB(0xa2b3c94)<<< 230-Directory has 55,442,214,912 bytes of disk space available.
Net::FTP=GLOB(0xa2b3c94)<<< 230 User logged in.
Net::FTP=GLOB(0xa2b3c94)>>> TYPE I
Net::FTP=GLOB(0xa2b3c94)<<< 200 Type set to I.
Net::FTP=GLOB(0xa2b3c94)>>> MKD Backups_Hosting
Net::FTP=GLOB(0xa2b3c94)<<< 550
Net::FTP=GLOB(0xa2b3c94)>>> MKD Backups_Hosting/daily
Net::FTP=GLOB(0xa2b3c94)<<< 550
Net::FTP=GLOB(0xa2b3c94)>>> CWD Backups_Hosting/daily
Net::FTP=GLOB(0xa2b3c94)<<< 250 CWD command successful.
Net::FTP=GLOB(0xa2b3c94)>>> ALLO 19826396
Net::FTP=GLOB(0xa2b3c94)<<< 200 ALLO command successful.
Net::FTP=GLOB(0xa2b3c94)>>> PASV
Net::FTP=GLOB(0xa2b3c94)<<< 227 Entering Passive Mode (190,190,183,95,216,175).
Net::FTP=GLOB(0xa2b3c94)>>> STOR dever.tar.gz
Net::FTP=GLOB(0xa2b3c94)<<< 150 Opening BINARY mode data connection.
Net::FTP=GLOB(0xa2b3c94)>>> QUIT
Net::FTP=GLOB(0xa2b3c94): Timeout at /scripts/cpbackup line 788

and when I go to see that line 788, it's inside this loop:

    if ( !$ftp->login( $ftpuser, $ftppass ) ) {
        $ftp->quit();
        my $msg = 'Unable to login to remote FTP server.';
        print "$msg\n";
        Cpanel::Logger::logger(
            {
                'message'   => $msg,
                'service'   => 'cpbackup',
                'level'     => 'die',
                'output'    => 2,
                'backtrace' => 0,
            }
        );
        return;
    }

Line 788 is the one with "'backtrace' => 0,"
May be due to a problem with Cpanel::Logger::logger?

[cPanelDavidG]

This sounds more like a bug report than a feature request. Has this been reported yet?

http://go.cPanel.net/bugs

[Kent Brockman]

ok, I will report it right now. Thanks