Backups cause processor usage to spike/perfomance to a crawl

[schwim]

Hi there everyone,

About a week or so ago, I began getting notices for 5 minute load averages being high with all questionable processes being cpbackup, lfd failures and my backups began finishing about 12 hours later than average. Logging into WHM to try to trace the problem, I noticed a major WHM update(agreement to a new TOS).

I submitted a ticket and cpanel worked on it for about three days. At the end of the process, they blamed lfd for false notices and issued the server a clean bill of health. They suggested lfd being at fault because the notices for the high server load were being generated by lfd. Please note that if you logged in after an lfd notice, you could see the high server load in WHM.

So, my problem is, there is a problem

This particular server is not carrying many accounts. Usually, you will find the processor usage at about .5 to 1. Backups always finished prior to start of business day. Right now, I'm looking at a processor usage of 7.62, the backups couldn't start today because the backups from yesterday are still running and page loads on this particular server are abysmally slow because all of my resources are being devoured by cpbackup. Up until this rash of problems, I had never once received a high server load notice, lfd had never failed and cpbackup had never run over.

I need to fix this. Since it's been decided that I don't have a problem, I am at the mercy of fellow forumgoers more knowledgeable than I

I would greatly appreciate any and all suggestions you might have in the matter.

thanks,
json

 

[schwim]

...... I am at the mercy of fellow forumgoers more knowledgeable than I

hehe... mom always said not to count on anyone but family

thanks,
json

 

[Infopro]

You might take a closer look at the accounts it's struggling with.

If an account for example has lots of files that don't change often, like videos or something you might add that directory on that account to the cpbackup-exclude.conf file in the root of that account. Then it won't be backed up every night savig some time and resources.

If the accounts got other issues like multiple bloated MySQL databases or large galleries packed with images owned by nobody you might sort those out.

Have you checked your backup logs for signs of trouble you can fix?

Not sure if that's helpful, but I'm also not sure if your Mom was being too helpful either, there. :p

Trust your common sense. Look closer at the accounts and error logs and you might see what it is you're missing...

 

[schwim]

Hi there infopro and thanks very much for your response.

I'm continuing to struggle with this and was wondering about a couple of your points. Concerning galleries owned by nobody, do you mean images owned by user: nobody? Does it cause a problem when files are owned by nobody?

Sorting out bloated mysql databases: If it's currently used, there's not a lot I can do, correct? Do you just mean retired db's?

As for exclusions, I'll look to see if there's anything I can afford to skip, but to be honest, I'm averse to skipping any of my clients' files, as it's their buns on the line if it all goes south.

I always thought the inability to perform incremental backups over FTP was pretty crappy. All it would require is a single local copy of the backup in addition to the remote backup. Perform checks on the local copy and ship the file to the remote server if it's different. The additional space would be much more well received than the resources used to transfer the complete account during every backup.

Reading my initial post, I see that it looks like I'm insinuating that cPanel is at fault. I'd like to clarify that they were very thorough in their search and that the problem more than likely lies with my accounts. The fact remains however that regardless of the reason, I need to try to resolve it.

thanks again for your help,
json

 

[Infopro]

Hi there infopro and thanks very much for your response.

I'm continuing to struggle with this and was wondering about a couple of your points. Concerning galleries owned by nobody, do you mean images owned by user: nobody? Does it cause a problem when files are owned by nobody?

The files themselves may not be a problem, but if they are owned by nobody:nobody then they may not be being counted as account space usage on that account. If a user has a 200MB account for example, there is a possibility that they could have 2GB of files on that account and you'd not know it. Inspecting the account might be helpful.

You can make sure the account's files are owned by that user by running this command as root, which would then give you a better idea of how large the accout is.

(this is an example)

chown -R accountname:accountname /home/accountname/public_html/thegalleryname

Sorting out bloated mysql databases: If it's currently used, there's not a lot I can do, correct? Do you just mean retired db's?

Yes. If a user backed up/copied a DB before an upgrade to a gallery for example, they might not need it any more and it should be backed up and removed. If the DB is really big, you might take a closer look at why and clean it up if possible. 4 large MySQL DBs take a lot more CPU and time to backup than 1, for example.

As for exclusions, I'll look to see if there's anything I can afford to skip, but to be honest, I'm averse to skipping any of my clients' files, as it's their buns on the line if it all goes south.

Yes of course. But if there is a static directory of files they don't need backed up every night, (I don't know if there is or not, just suggesting you find out) back it up, and then add it to be skipped, until it's needed again to be updated. It's just a suggestion to make your backups go a bit quicker. YMMV.

I always thought the inability to perform incremental backups over FTP was pretty crappy. All it would require is a single local copy of the backup in addition to the remote backup. Perform checks on the local copy and ship the file to the remote server if it's different. The additional space would be much more well received than the resources used to transfer the complete account during every backup.

Reading my initial post, I see that it looks like I'm insinuating that cPanel is at fault. I'd like to clarify that they were very thorough in their search and that the problem more than likely lies with my accounts. The fact remains however that regardless of the reason, I need to try to resolve it.

thanks again for your help,
json

Not sure if I'm any help at all. But that won't stop me from trying.

 

[schwim]

Hi there infopro,

Do you happen to know of the best way to view the space used by each user divided(mail, db, space, etc.)? When I log in as some clients, the space is quite large, but it can't be accounted for in their web space.

The problem is worsening now and I'm not making much headway, I'm afraid. Thanks for the help you've already provided.

thanks,
json