Deleted /home/virtfs! Completely Screwed. Help!

[yong]

Recently one of our servers was compromised. In performing a server audit, one of my guys noticed directories in /home that didn't quite seem right as they were not showing up in our WHM usernames.

Long story short, we did a rm -rf /home/virtfs

Upon deleting /home/virtfs the entire system would not accept any commands and we received the msg:

-bash: /bin/ls: /lib/ld-linux.so.2: bad ELF interpreter: No such file or directory

--------------------------

We realize now (a bit to late) that we've essentialy deleted a big chunk of the OS.

We have easy access to the DC but before making matters worse, is there a detailed process of getting the box back.

Questions:

1. Can we restore the system while maintaining our users home directories and any datahbases?
2. Has anybody else pulled such a kuncklhead move? If so, how did you recover the box, users, whm settings, etc..etc?

Any detailed help and explanation on resolving would be GREATLY appreciated!

 

[Website Rob]

Houston - we have a problem ...

Let me just say, the reason there is so much room in the boat is because there are no paddles -- if you know what I mean.

Over the years many of us have made the same mistake and there is only one sure way to correct the problem -- an OS reload.

You could send an eMail to cPanel and see if that has changed but best to start setting up a new Server or for doing an OS reload on your current one.


The /home/virtfs dir. is for Clients you have given Shell access to. Used by cPanel for Jailed & Regular Shell access, it stores things in a way that only cPanel knows and understands. Strictly a "look but don't touch" dir. To make any manual changes is a definite nay nay.

 

[jandafields]

/home/virtfs itself is nothing, it is the fact that all the links in it are HARDLINKS to a lot of the OS. So, deleting those links also deleted the actual files from /etc /usr ....

One thing to try... load your same OS and CPanel on another computer, and copy over the deleted files, or ask someone with your setup to tell you / give you those files in a structural tar so you can just untar them into place

 

[freedman]

/home/virtfs itself is nothing, it is the fact that all the links in it are HARDLINKS to a lot of the OS. So, deleting those links also deleted the actual files from /etc /usr ....

One thing to try... load your same OS and CPanel on another computer, and copy over the deleted files, or ask someone with your setup to tell you / give you those files in a structural tar so you can just untar them into place

They're even a strange kind of hardlink, as they can cross mounted filesystems, which normal hardlinks can't do.

you have a couple options for 'cleaning up' the virtfs.

1) Mount a new filesystem as /home/virtfs2
mv /home/virtfs/* /home/virtfs2
unmount /home/virtfs2 and reformat the partition.
now you have a clean virtfs, but it will fill up again, so here's how we solve that.

2) mount /home/virtfs as a tmpfs (or soft link /home/virtfs to something which is a tmpfs (we have /tmp and /var/tmp as tmpfs on our systems).

Now, they'll take up about 1 MB per user account so keep that in mind.
again, if you have the disk space, you could just keep it on another partition and just occasionally reformat it when you feel you need to clean it out.