Faster cPanel Backup by replacing gzip with pigz [Case 55535]

[Compubuster]

Gzip doesn't make good use of multi-processor/multi-core machines, which results in long backup time.

Fortunately, there's an in-place replacement, pigz

pigz, which stands for parallel implementation of gzip, is a fully functional replacement for gzip that exploits multiple processors and multiple cores to the hilt when compressing data.

Also, files compressed with pigz can be decompressed with gunzip, and unpigz can decompress gzip compressed files.

I've done a couple of tests on an 8 cores machine, and here are the results:

[root@ns3 tests]# time gzip test1.tar

real    1m 7.176s
user    0m55.119s
sys     0m1.069s

[root@ns3 tests]# time pigz test2.tar

real    0m 12.096s
user    1m15.924s
sys     0m1.109s

[root@ns3 tests]# time pigz -1 test3.tar

real    0m 7.141s
user    0m44.020s
sys     0m1.349s


-rw-r--r--  1 root     root     315M Nov 10 04:54 test1.tar.gz
-rw-r--r--  1 root     root     315M Nov 10 04:54 test2.tar.gz
-rw-r--r--  1 root     root     346M Nov 10 04:54 test3.tar.gz
-rw-r--r--  1 root     root     788M Nov 10 04:54 test.tar

In layman's terms "to get more votes " .. pigz compressed a 788 MB file, using fastest compression and all 8 cores, in 7 seconds, while it took gzip 1 minute and 7 seconds .. you do the math.

BTW, pigz was written by Mark Adler.

Adler is the author of the Adler-32 hash function, a co-author of the zlib compression library and gzip, has contributed to Info-ZIP, and has participated in developing the Portable Network Graphics (PNG) image format. Adler was also the Spirit Mission Manager for the Mars Exploration Rover mission.

In short .. he knows what he's doing

[fkatzenb]

That is impressive

[Infopro]

This might just be the best Feature Request I've seen on these forums in a very long time. Not necessarily due to the request itself, but the style in which you've posted it.

Thanks for taking the time to do it right.

In layman's terms "to get more votes

This request has my vote, absolutely.

[Compubuster]

Thank you Infopro for your nice words ... I just love marketing

[fkatzenb]

Now how to figure out how to get Apache to use that in its place. Couldn't you just replace gzip with pigz? This could be awesome on high traffic web servers that use gzip to compress content. You can use gzip to uncompress the content from pigz correct?

[fkatzenb]

Doing some more research and my own testing, this is quite the powerful program and is completely compatible with gzip... obviously.

cPanel,
Please considering using this for all things gzip - log rotations, backup, etc. It is blazingly fast on my Xeon X3470. I could actually allow users to make their own backups now with this!

Thanks,
Frank

[cPanelDavidG]

I'll bring this up with the devs the next time I meet with them about new feature requests.

[Astral God]

Gzip doesn't make good use of multi-processor/multi-core machines, which results in long backup time.

Fortunately, there's an in-place replacement, pigz

Thank's for sharing ant +1 for this request.

[fkatzenb]

Yep. It was simple to install.... yum install pigz

[Compubuster]

Thank you guys for your positive feedback and votes

For those of you who have tried pigz, give it another try using "pigz -1 filename.tar". This makes pigz use the fastest compression option, which results in almost double the speed with just a slightly bigger compressed file.

You may revise my testing results, in the first post.

"pigz test2.tar" completed in 12 seconds, resulting in a 315 M compressed test2.tar.gz.
"pigz -1 test3.tar" completed in 7 seconds, resulting in a 346 M compressed test3.tar.gz.

That's why I recommend, using this level of compression with cPanel backup instead of using the normal compression level.

BTW, replacing gzip with pigz can be accomplished by simply adding some symbolic links, and that's it

Did I mention, it's an in-place replacement ? .. which means, pigz uses the exact same arguments used with gzip

[root@ns3 ~]# gzip -h

Usage: gzip [OPTION]... [FILE]...

  -c, --stdout      write on standard output, keep original files unchanged
  -d, --decompress  decompress
  -f, --force       force overwrite of output file and compress links
  -h, --help        give this help
  -l, --list        list compressed file contents
  -L, --license     display software license
  -n, --no-name     do not save or restore the original name and time stamp
  -N, --name        save or restore the original name and time stamp
  -q, --quiet       suppress all warnings
  -r, --recursive   operate recursively on directories
  -S, --suffix=SUF  use suffix SUF on compressed files
  -t, --test        test compressed file integrity
  -v, --verbose     verbose mode
  -V, --version     display version number
  -1, --fast        compress faster
  -9, --best        compress better
    --rsyncable   Make rsync-friendly archive

And for pigz, after re-arranging the arguments list for easy comparison :

[root@ns3 ~]# pigz -h

Usage: pigz [options] [files ...]

  -c, --stdout         Write all processed output to stdout (won't delete)
  -d, --decompress     Decompress the compressed input
  -f, --force          Force overwrite, compress .gz, links, and to terminal
  -h, --help           give this help "not mentioned in the help, but it works"
  -l, --list           List the contents of the compressed input
  -L, --license        display software license "not mentioned in the help, but it works"
  -n, --no-name        Do not store or restore file name in/from header
  -N, --name           Store/restore file name and mod time in/from header
  -q, --quiet          Print no messages, even on error
  -r, --recursive      Process the contents of all subdirectories
  -S, --suffix .sss    Use suffix .sss instead of .gz (for compression)
  -t, --test           Test the integrity of the compressed input
  -v, --verbose        Provide more verbose output
  -V, --version        display version number "not mentioned in the help, but it works"
  -0 to -9, --fast, --best   Compression levels, --fast is -1, --best is -9
  -R, --rsyncable      Input-determined block locations for rsync


   Additional arguments in pigz only :

  -p, --processes n    Allow up to n compression threads (default is the
                       number of online processors, or 8 if unknown)
  -b, --blocksize mmm  Set compression block size to mmmK (default 128K)
  -i, --independent    Compress blocks independently for damage recovery
  -z, --zlib           Compress to zlib (.zz) instead of gzip format
  -K, --zip            Compress to PKWare zip (.zip) single entry format
  -k, --keep           Do not delete original file after processing
  -T, --no-time        Do not store or restore mod time in/from header

[fkatzenb]

What would be ideal is if they setup a config page in WHM where you can configure the -p and -b for your server's setup. What would be the best configuration when it comes to blocksize?

[Compubuster]

fkatzenb, in a perfect world, we should have options in WHM to change compression level and number of processors/threads.

As for the blocksize, I didn't test it. So, I have no idea

[fkatzenb]

fkatzenb, in a perfect world, we should have options in WHM to change compression level and number of processors/threads.

As for the blocksize, I didn't test it. So, I have no idea

I did some testing just now.

Here is what I started with:

root@one [~/test]# ls -alh
total 2.8G
drwxr-xr-x  2 root root 4.0K Nov 12 08:22 ./
drwxr-x--- 36 root root 4.0K Nov 12 08:18 ../
-rw-r--r--  1 root root 257M Nov 12 08:18 0.tar
-rw-r--r--  1 root root 257M Nov 12 08:16 1.tar
-rw-r--r--  1 root root 257M Nov 12 08:16 2.tar
-rw-r--r--  1 root root 257M Nov 12 08:17 3.tar
-rw-r--r--  1 root root 257M Nov 12 08:17 4.tar
-rw-r--r--  1 root root 257M Nov 12 08:18 5.tar
-rw-r--r--  1 root root 257M Nov 12 08:18 6.tar
-rw-r--r--  1 root root 257M Nov 12 08:18 7.tar
-rw-r--r--  1 root root 257M Nov 12 08:18 8.tar
-rw-r--r--  1 root root 257M Nov 12 08:18 9.tar
-rw-r--r--  1 root root 257M Nov 12 08:15 test.tar

Here is my baseline of gzip followed by pigz and then fast compression:

root@one [~/test]# time gzip 0.tar
real    0m9.100s
user    0m8.805s
sys     0m0.294s

root@one [~/test]# time pigz 1.tar
real    0m3.511s
user    0m17.644s
sys     0m0.395s

root@one [~/test]# time pigz -1 2.tar
real    0m1.845s
user    0m13.679s
sys     0m0.427s

root@one [~/test]# ls -alh
total 2.5G
drwxr-xr-x  2 root root 4.0K Nov 12 08:28 ./
drwxr-x--- 36 root root 4.0K Nov 12 08:28 ../
-rw-r--r--  1 root root 167M Nov 12 08:18 0.tar.gz
-rw-r--r--  1 root root 167M Nov 12 08:16 1.tar.gz
-rw-r--r--  1 root root 171M Nov 12 08:16 2.tar.gz
-rw-r--r--  1 root root 257M Nov 12 08:17 3.tar
-rw-r--r--  1 root root 257M Nov 12 08:17 4.tar
-rw-r--r--  1 root root 257M Nov 12 08:18 5.tar
-rw-r--r--  1 root root 257M Nov 12 08:18 6.tar
-rw-r--r--  1 root root 257M Nov 12 08:18 7.tar
-rw-r--r--  1 root root 257M Nov 12 08:18 8.tar
-rw-r--r--  1 root root 257M Nov 12 08:18 9.tar
-rw-r--r--  1 root root 257M Nov 12 08:15 test.tar

Now I played with the block size. It is obvious why they use 128K:

root@one [~/test]# time pigz -1 -b 256 3.tar

real    0m10.356s
user    0m15.325s
sys     0m0.450s
root@one [~/test]# time pigz -1 -b 64 4.tar

real    0m14.330s
user    0m19.194s
sys     0m0.529s
root@one [~/test]# time pigz -1 5.tar

real    0m3.769s
user    0m14.665s
sys     0m0.422s

Now I played with the processor size. I have a Xeon 3470 with HT, so I was curious what results I would see as to how it detects processors and HT.

root@one [~/test]# time pigz -1 -p 4 6.tar
real    0m3.144s
user    0m10.310s
sys     0m0.442s

root@one [~/test]# time pigz -1 -p 2 7.tar
real    0m7.701s
user    0m9.985s
sys     0m0.469s

root@one [~/test]# time pigz -1 -p 6 8.tar
real    0m2.731s
user    0m12.368s
sys     0m0.437s

root@one [~/test]# time pigz -1 -p 8 9.tar
real    0m2.064s
user    0m13.805s
sys     0m0.404s

Just for reference:

root@one [~/test]# ls -alh
total 2.0G
drwxr-xr-x  2 root root 4.0K Nov 12 08:43 ./
drwxr-x--- 36 root root 4.0K Nov 12 08:40 ../
-rw-r--r--  1 root root 167M Nov 12 08:18 0.tar.gz
-rw-r--r--  1 root root 167M Nov 12 08:16 1.tar.gz
-rw-r--r--  1 root root 171M Nov 12 08:16 2.tar.gz
-rw-r--r--  1 root root 171M Nov 12 08:17 3.tar.gz
-rw-r--r--  1 root root 171M Nov 12 08:17 4.tar.gz
-rw-r--r--  1 root root 171M Nov 12 08:18 5.tar.gz
-rw-r--r--  1 root root 171M Nov 12 08:18 6.tar.gz
-rw-r--r--  1 root root 171M Nov 12 08:18 7.tar.gz
-rw-r--r--  1 root root 171M Nov 12 08:18 8.tar.gz
-rw-r--r--  1 root root 171M Nov 12 08:18 9.tar.gz
-rw-r--r--  1 root root 257M Nov 12 08:15 test.tar

It would be ideal to have a setting for at least number of processing threads for when cPanel implements this along with a nice setting.

[crazyaboutlinux]

Very Good!

[Compubuster]

I would just like to add;

cPanel shared hosting servers usually host 100 to 300 account per server.

let's assume that each account's tar file is around 800 MB "as in my first post", using 200 account per server as an average, on an 8 cores machine :

Using gzip :

200 account * 1 minute and 7 seconds = 3 hours, 43 minutes, 20 seconds

Using pigz with fastest compression :

200 account * 7 seconds = 23 minutes, 20 seconds

I rest my case


P.S: this is a bull****, non-scientific test