Server hanged several times, run fsck and this happened...

[bsasninja]

Hi guys, today I had a server issue this night my server hanged I did a remote boot and I found that there was a problem with a inode at some part of the /home directory.
Ok I found the directory where the problem was and when I tried to ls the folder content, the server hanged again.
I booted it, went again to the same directory when I tried to remove the file said cant delete read-only file system. Once I figured out all the /home partition was in read only mode, all the others /, /tmp,/usr,/boot,/var,/etc and whatever were writable.

So I asked to the DC tech support to do a reboot and run fsck manually, once it ended and booted /home partition was writable again and when I wanted to delete the site directory, deleted all except for the ./spamasassin folder inside this are this files:


?--------- ? ? ? ? ? auto-whitelist
?--------- ? ? ? ? ? auto-whitelist.lock.server.site.com.10035
?--------- ? ? ? ? ? auto-whitelist.lock.server.site.com.15941
?--------- ? ? ? ? ? auto-whitelist.lock.server.site.com.16378
?--------- ? ? ? ? ? auto-whitelist.lock.server.site.com.16903
?--------- ? ? ? ? ? auto-whitelist.lock.server.site.com.16931
?--------- ? ? ? ? ? auto-whitelist.lock.server.site.com.16984
?--------- ? ? ? ? ? auto-whitelist.lock.server.site.com.18608
?--------- ? ? ? ? ? auto-whitelist.lock.server.site.com.18878
?--------- ? ? ? ? ? bayes.lock.server.site.28146
?--------- ? ? ? ? ? bayes.lock.server.site.com.31919
?--------- ? ? ? ? ? bayes.lock.server.site.com.3426
?--------- ? ? ? ? ? bayes.lock.server.site.com.4618
?--------- ? ? ? ? ? bayes.lock.server.site.com.4719
?--------- ? ? ? ? ? bayes.lock.server.site.com.4849
?--------- ? ? ? ? ? bayes.lock.server.site.com.5066
?--------- ? ? ? ? ? bayes.lock.server.site.com.723
?--------- ? ? ? ? ? bayes_seen
?--------- ? ? ? ? ? bayes_toks


all the files are in red color, no chmod specs, owner, date or size only a ?. Could it be that fsck marked this sector of the disk as bad and this is why appears like this? I tried to remove the files but says no file found, like some kind of ghost of what was there.

The only problem is that I have this structure I want to remove but I cant

/home
----/sitename
---./spamasassin

I want to get rid of this /sitename folder and the ./spamasassin folder that has the above "ghosted" files. Anyone had this issue before?

Thank you

Diego

[cPanelTristan]

Hello Diego,

Is there any way you are able to move the files? It is definitely corrupted. Any files missing inode numbers are corrupted.

You might end up having to run another fsck if moving these doesn't work.

Thanks!

[bsasninja]

the site has been removed from cpanel, only this folders remains. Im not able to remove it.

My question is, if there any problem if I leave the folder as it is? backups will backup it or the backups only backup sites loaded at apache only.

Thanks

[cPanelTristan]

If the account doesn't exist, cPanel isn't going to try to back it up.

[bsasninja]

ok, well as long it doesnt affect the server uptime I will stay with the "ghost" until I got fixed.

[bsasninja]

btw if I try to chmod any of the files above says: file not found.

[cPanelTristan]

The only command I was ever able to run on files missing inodes was to move the directory a level above the one with ?? for it and it would let me move it. That's why I suggested trying to move the file or directories containing the corruption to see if that worked. That's about all you will be able to do.

[bsasninja]

hey I moved the folder inside /tmp and voila! the files became writable with size, date and group owner. Now its erased completely.

Thanks!

[nibb]

I also had this recently on some cPanel server and I don´t know if this is a coincidence, but searching the inode that causes this they where also exactly the same like your case
auto-whitelist.lock.server.site

On a user account.