pkgacct script very slow since WHM 11.25.x

[cyon]

Hi

I noticed that the pkgacct script with the --skiphomedir option takes significantly longer to complete since WHM 11.25.x. After a little research I found what is causing the delay. The following code block was added to the pkgacct script in WHM 11.25.x.

/scripts/pkgacct Line 1237

if ($skiphomedir) {
my $du = qx( du -s $homedir );
my($homesize_kb) = ($du =~ m/^(\d+)/);
## FreeBSD does not have the -b option; performing the calculation manually.
my $homesize = $homesize_kb * 1024;
print "\nhomesize is: $homesize\n";
}

This code snippet is calculating the size of the users home directory. The "du -s /home/$user" command can be very slow to calculate the size of large accounts. It looks like the variable $homesize is only needed to "echo" the size of the home directory. If this is the case I would recommend using a faster method like "quota $user" to get the actual size of the users home directory.

Example script to calculate the size (MB) of a users homedir. using the quota command:
user_quota=$(quota $user | grep $volume | awk '{print $2}' | sed 's/*//g') | echo "$user: $user_quota $(echo "scale=1 ; ${user_quota}/1024" | bc)"

What do you guys think about this ?

Brgds

Nicolas

[BlackRain]

Hi

I noticed that the pkgacct script with the --skiphomedir option takes significantly longer to complete since WHM 11.25.x. After a little research I found what is causing the delay. The following code block was added to the pkgacct script in WHM 11.25.x.

/scripts/pkgacct Line 1237

if ($skiphomedir) {
my $du = qx( du -s $homedir );
my($homesize_kb) = ($du =~ m/^(\d+)/);
## FreeBSD does not have the -b option; performing the calculation manually.
my $homesize = $homesize_kb * 1024;
print "\nhomesize is: $homesize\n";
}

This code snippet is calculating the size of the users home directory. The "du -s /home/$user" command can be very slow to calculate the size of large accounts. It looks like the variable $homesize is only needed to "echo" the size of the home directory. If this is the case I would recommend using a faster method like "quota $user" to get the actual size of the users home directory.

Example script to calculate the size (MB) of a users homedir. using the quota command:
user_quota=$(quota $user | grep $volume | awk '{print $2}' | sed 's/*//g') | echo "$user: $user_quota $(echo "scale=1 ; ${user_quota}/1024" | bc)"

What do you guys think about this ?

Brgds

Nicolas

We are having the same problem with backups since the upgrade. Now, we are getting high server loads and incomplete backups.

cPanel 11.25.0-C43473 - WHM 11.25.0 - X 3.9
CENTOS 5.4 i686

[mohit]

I believe cPanel did some tweak to cpbackup script in FEB, and backup was real smooth, but since one week I am seeing too many problems when weekly backup is run.

I must say last month i was surprised to see backup didn't gave any high loads, but this month i had a bad sunday, cause 3 of servers were giving Load alerts during backup run.

[konrath]

same here... 48 hours to make 100GB of backups

[AlexAT]

No one from cPanel interesting??

Why at all to add not needed for backup information into the backup scripts?
If you need additional info then you can run backup with some option.
But by default backup should be simple, fast and with minimal load.

p.s. I still don't understand for years why backup makes copy of full account before tarring&gzipping. Is this additional MEGA-load really required?

p.p.s I think cPanel's QAs testing their backup on servers with 10 accounts with 5 files each and with 10Megs of total size.

[mohit]

p.s. I still don't understand for years why backup makes copy of full account before tarring&gzipping. Is this additional MEGA-load really required?

You can use Increamental backup's if you want to back only what has changed. but it would not compress the output.

If you are using compressed backup, whole account would be backed up until you edit and manually remove some dirs using. cpbackup-exclude.conf

[AlexAT]

You can use Increamental backup's if you want to back only what has changed. but it would not compress the output.

If you are using compressed backup, whole account would be backed up until you edit and manually remove some dirs using. cpbackup-exclude.conf

I said about not needed step (copying whole account into other place) and you suggest me to use incremental backup.
How this help ?

[cpanelkenneth]

Hi

I noticed that the pkgacct script with the --skiphomedir option takes significantly longer to complete since WHM 11.25.x. After a little research I found what is causing the delay. The following code block was added to the pkgacct script in WHM 11.25.x.

/scripts/pkgacct Line 1237

if ($skiphomedir) {
my $du = qx( du -s $homedir );
my($homesize_kb) = ($du =~ m/^(\d+)/);
## FreeBSD does not have the -b option; performing the calculation manually.
my $homesize = $homesize_kb * 1024;
print "\nhomesize is: $homesize\n";
}

This code snippet is calculating the size of the users home directory. The "du -s /home/$user" command can be very slow to calculate the size of large accounts. It looks like the variable $homesize is only needed to "echo" the size of the home directory. If this is the case I would recommend using a faster method like "quota $user" to get the actual size of the users home directory.

Example script to calculate the size (MB) of a users homedir. using the quota command:
user_quota=$(quota $user | grep $volume | awk '{print $2}' | sed 's/*//g') | echo "$user: $user_quota $(echo "scale=1 ; ${user_quota}/1024" | bc)"

What do you guys think about this ?

Brgds

Nicolas

The --skiphomedir parameter was implemented for use during account transfer. While it is useful for some backup scenarios that is not its intended design. The additional information added to the block in question for cPanel 11.25.0 was done to improve reporting and failure handling during account transfers.

The use of du was necessitated to provide accurate reporting information as it is very common for files in /home/user to be owned by the nobody user ( think PHP web apps ). The quota command will not reveal that information.