baseline/optimal pigz block size (chunks per compression unit)

[ottdev]

Some guidance please ??
with regards to chunks per compression work units - which I gather alters pigz' -b (block size) parameter.

Is there a calculation based on system memory that gives you a good number to start with?
The default is way up at 4096. I recently attempted to change the value in a VPS from 128 to 1024 and this caused backups times to more than quadruple and cause way higher server loads. Obviously not optimal to have backups now dragging into morning work hours. So I will set this one back.

Is there a rule of thumb?
Couldn't find any useful info in cpanel or pigz docs.

And why is it that pigz docs say default is 128, and cpanel has chosen 4096 as the default? <= how did that figure come to be chosen?

 

[cPanelMichael]

Hello,

As I understand, using a higher value for the "Number of kilobyte chunks per compression work unit" option will benefit systems with larger L2/L3 caches. There's no specific rule of thumb or guidelines to follow, however here's a third-party URL on L2/L3 cache sizes you may find helpful:

L2 Vs. L3 CPU cache speed and performance

Fine tuning the value often occurs through testing, and can depend on your specific system's existing performance and hardware specifications. It's generally something that a qualified system administrator would need to adjust:

System Administration Services | cPanel Forums

As far as the current default of 4096, it's the value that was found to offer the best performance across the widest variety of testing environments.

Thank you.