The Community Forums

Interact with an entire community of cPanel & WHM users!
  1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Multi-Threaded Remote FTP Backup?

Discussion in 'Data Protection' started by seenBEST, Jun 20, 2014.

  1. seenBEST

    seenBEST Registered

    Joined:
    Mar 24, 2013
    Messages:
    4
    Likes Received:
    0
    Trophy Points:
    1
    cPanel Access Level:
    Root Administrator
    I backup a VPS remotely to FTP via the "additional destinations" area of the built-in WHM Backup Configuration tool. It's slow! I find that it takes nearly 14 hours to transfer 40GB of backup data from the VPS to the FTP destination which works out to about 1MB/s sustained rate.

    The VPS is from a major provider with nearly unlimited speed and the remote FTP server has a 25MB connection, so I know that I'm not being limited by the pipe in either direction. However, the built-in FTP script used by the backup utility uploads a single file one at a time via FTP. This single thread file upload, for whatever reason, never exceeds 1MB/s.

    I've ran a test by downloading the entire 40GB backup directory via the same FTP server using a FileZilla client set to 6 simultaneous threads and it completes much faster (a couple of hours). So that leads to my question, is there a way to configure the WHM remote FTP backup script to upload files using multiple threads so that I'm not waiting for files 1 at a time?

    In addition does anyone know if there is some configuration somewhere from within WHM that could be limiting a single FTP upload to 1MB/s? If I could increase the speed at which the FTP uploads then it wouldn't be so bad.
     
  2. cPanelMichael

    cPanelMichael Forums Analyst
    Staff Member

    Joined:
    Apr 11, 2011
    Messages:
    30,854
    Likes Received:
    676
    Trophy Points:
    113
    cPanel Access Level:
    Root Administrator
    Hello :)

    cPanel does not limit the backup transfer speed with FTP. It simply uses the FTP protocol on your server as it's configured. In your test, did you use FTP to upload the files to the remote server from the cPanel server? That would be the most reliable way to determine the transfer speeds.

    Thank you.
     
  3. seenBEST

    seenBEST Registered

    Joined:
    Mar 24, 2013
    Messages:
    4
    Likes Received:
    0
    Trophy Points:
    1
    cPanel Access Level:
    Root Administrator
    Thank you for the reply, cPanelMichael.

    In my first speed test, I monitored the VPS server during the backup operation to get FTP speeds sent from the VPS to the remote server FTP. Transfers hovered around 1MB/s. This was my (PUSH) test.

    In my second test, I used a Filezilla client from the remote server FTP to download the backup files from the VPS server. Transfers again hovered around 1MB/s. This was my (PULL) test.

    Finally, I did a multi-threaded FTP download where I downloaded 6 of the backup files at a time using FileZilla client. Each thread obtained 1MB/s, which made it possible for me to transfer all of the backup files much faster than waiting for 1 file at a time.

    I am hoping there is some hidden variable somewhere that can be adjusted to make the Backup to Remote FTP option send backup files via FTP using multiple threads. It could reduce my time to transfer backups to FTP from 12 hours to 3.
     
  4. cPanelMichael

    cPanelMichael Forums Analyst
    Staff Member

    Joined:
    Apr 11, 2011
    Messages:
    30,854
    Likes Received:
    676
    Trophy Points:
    113
    cPanel Access Level:
    Root Administrator
Loading...

Share This Page