cpanel removing RPM's?

[xwing777]

I had a server that deleted all rpm's and messed up up2date as well. Couldnt get anything working so reinstalled redhat. Everything was running latest version and updated till this. Running WHM 10.1.0 cPanel 10.2.0-R82

Now i have the same problem on another server. Has anyone else had this happen? Saw a few unresolved questions for the same thing at cpanel forums and other sites. If your running cpanel and redhat ent, check in whm under install rpm's to see if there gone. Im wanting to know if this a bug and how to fix.

Updating the rpm's both whm and ssh seems to run but nothing showing up in whm afterwards.

 

[chirpy]

Never seen such a problem. When you say they're missing, have they actually been removed from the rpm database, or is it just that the page in WHM isn't showing anything? If the former and you know when it happened and what likely caused it, I'd suggest you log a ticket with cPanel. If the latter, you should log it in bugzilla if it isn't already there.

The only time I've ever seen cPanel updates removing anything is during initial installation.

 

[xwing777]

Thats what it is, no rpms and up2date is broke as well. Since a lot of people may not use either up2date or rpms they might not even know something is wrong. From these post it looks like something from cpanel or redhat did it from automatic updates.

http://forums.cpanel.net/showpost.php?p=188599&postcount=1

http://forums.cpanel.net/showpost.php?p=186503&postcount=1

*From my datacenters tech* "It would appear that up2date and many of its components have been removed from the system. I added the missing library but now you're missing binaries. I can't even register the system on our RHN due to missing binaries."

The missing libary was libwnck-1.so.4 same as first post on this so that leads me to believe there are many others with same problem but they just havent seen it yet.

Can anyone running redhat/cpanel check the rpms and see if they are missing?

I don't want to reload this machine if it may happen again. Is there a safe way to rebuild up2date for redhat ent?

 

[haze]

What version and release of redhat is this ?

 

[xwing777]

Red Hat Enterprise Linux ES release 3 (Taroon Update 5)

Is there any way to rebuild up2date and rpm database manually?

 

[chirpy]

Well,

You can rebuild your rpm database with:

rpm --rebuilddb -v -v

and then update up2date with the current rpm set:

up2date -p

 

[xwing777]

everytime i run rebuild rpms, it scroll the list and says added, but nothing put on the server. I cleared out the database rm -f /var/lib/rpm/__db*; then rpm --rebuilddb -v -v
but same thing happens. Running anything with up2date just comes back to bash with no error, nothing run

 

[chirpy]

Then you have a corrupt RPM database and, unless you have pretty much an identical server setup where you can copy that database from, you're looking at having to have an OS restore done.

 

[xwing777]

I have several other servers that are the same setup (all except 1 which is dual processor) so they should be the same files.

What is the correct method to transfer up2date and rpm's over to this one without corrupting anything further?

 

[chirpy]

I will start out by saying that there are risks in doing this, namely that the database doesn't truly reflect what is on this server. If the hardware is different (esp the CPU type) then you should go no further, otherwise you could render the server unbootable after future updates.

That warning aside:

On the server where everything is OK:

cd /var/lib
tar -czf rpm.tgz rpm/

Upload the resulting rpm.tgz to the server where things are messed up to /var/lib and:

cd /var/lib
mv rpm/ rpm.old/
tar -xzf rpm.tgz
rpm --rebuilddb -v -v

If that all runs through OK then do:

up2date -p

 

[xwing777]

The only difference in these 2 are harddrive sizes and memory. The processors, motherboard, etc.. is same. I do have a seperate drive for backups only on both though. Im gonna get a reload if this don't work anyway.

Now if i only knew what caused it i would feel a lot better. Thanks for help on this issue and i will post results afterwards.

Last question, there is no way to damage anything on the good server by doing this is there?

cd /var/lib
tar -czf rpm.tgz rpm/

 

[chirpy]

No, wouldn't have any affect on the OK server. Once you've done the copy work you might want to reboot the server to ensure all the processes are using the new clean rpm database incase they currently have the old one open (it can happen).

 

[xwing777]

Updated: I went to check the good server before i started and there missing too

Went back and check logs from last night on this server and got a lot of stuff like this:

etc/cron.daily/logrotate: line 3: /usr/sbin/logrotate: No such file or directory
/etc/cron.daily/makewhatis.cron:


Check and there is no logrotate on either server.

/etc/cron.daily/tmpwatch: line 1: /usr/sbin/tmpwatch: No such file or directory
/etc/cron.daily/tmpwatch: line 2: /usr/sbin/tmpwatch: No such file or directory
/etc/cron.daily/tmpwatch: line 5: /usr/sbin/tmpwatch: No such file or directory

Now im wondering if something using tmpwatch deleted all these things? if so is it a security breech on these servers? Everything is running fine but can't update anything because of the missing files. At this point is my only option to restore the drives fresh?

On the freshly restored server everything is fine except for this:
lsof: WARNING: can't stat() ext3 file system /var/tmp\040(deleted)

Im hoping this isnt anything like the other ones starting over again. Im running the same apps, same configuration as i have been for over a year on all servers without any problems until now. Im getting very concerned on whats going on.

 

[chirpy]

That doesn't look good at all. Whenever I've seen binaries go missing from /usr/sbin it's been due to hacker activity. I would have the server checked over thoroughly for an intrusion if I were you.