The Community Forums

Interact with an entire community of cPanel & WHM users!
  1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Server hanged several times, run fsck and this happened...

Discussion in 'Data Protection' started by bsasninja, Jun 27, 2012.

  1. bsasninja

    bsasninja Well-Known Member

    Joined:
    Sep 2, 2004
    Messages:
    528
    Likes Received:
    0
    Trophy Points:
    16
    Hi guys, today I had a server issue this night my server hanged I did a remote boot and I found that there was a problem with a inode at some part of the /home directory.
    Ok I found the directory where the problem was and when I tried to ls the folder content, the server hanged again.
    I booted it, went again to the same directory when I tried to remove the file said cant delete read-only file system. Once I figured out all the /home partition was in read only mode, all the others /, /tmp,/usr,/boot,/var,/etc and whatever were writable.

    So I asked to the DC tech support to do a reboot and run fsck manually, once it ended and booted /home partition was writable again and when I wanted to delete the site directory, deleted all except for the ./spamasassin folder inside this are this files:


    ?--------- ? ? ? ? ? auto-whitelist
    ?--------- ? ? ? ? ? auto-whitelist.lock.server.site.com.10035
    ?--------- ? ? ? ? ? auto-whitelist.lock.server.site.com.15941
    ?--------- ? ? ? ? ? auto-whitelist.lock.server.site.com.16378
    ?--------- ? ? ? ? ? auto-whitelist.lock.server.site.com.16903
    ?--------- ? ? ? ? ? auto-whitelist.lock.server.site.com.16931
    ?--------- ? ? ? ? ? auto-whitelist.lock.server.site.com.16984
    ?--------- ? ? ? ? ? auto-whitelist.lock.server.site.com.18608
    ?--------- ? ? ? ? ? auto-whitelist.lock.server.site.com.18878
    ?--------- ? ? ? ? ? bayes.lock.server.site.28146
    ?--------- ? ? ? ? ? bayes.lock.server.site.com.31919
    ?--------- ? ? ? ? ? bayes.lock.server.site.com.3426
    ?--------- ? ? ? ? ? bayes.lock.server.site.com.4618
    ?--------- ? ? ? ? ? bayes.lock.server.site.com.4719
    ?--------- ? ? ? ? ? bayes.lock.server.site.com.4849
    ?--------- ? ? ? ? ? bayes.lock.server.site.com.5066
    ?--------- ? ? ? ? ? bayes.lock.server.site.com.723
    ?--------- ? ? ? ? ? bayes_seen
    ?--------- ? ? ? ? ? bayes_toks


    all the files are in red color, no chmod specs, owner, date or size only a ?. Could it be that fsck marked this sector of the disk as bad and this is why appears like this? I tried to remove the files but says no file found, like some kind of ghost of what was there.

    The only problem is that I have this structure I want to remove but I cant

    /home
    ----/sitename
    ---./spamasassin

    I want to get rid of this /sitename folder and the ./spamasassin folder that has the above "ghosted" files. Anyone had this issue before?

    Thank you

    Diego
     
    #1 bsasninja, Jun 27, 2012
    Last edited: Jun 27, 2012
  2. cPanelTristan

    cPanelTristan Quality Assurance Analyst
    Staff Member

    Joined:
    Oct 2, 2010
    Messages:
    7,623
    Likes Received:
    21
    Trophy Points:
    38
    Location:
    somewhere over the rainbow
    cPanel Access Level:
    Root Administrator
    Hello Diego,

    Is there any way you are able to move the files? It is definitely corrupted. Any files missing inode numbers are corrupted.

    You might end up having to run another fsck if moving these doesn't work.

    Thanks!
     
  3. bsasninja

    bsasninja Well-Known Member

    Joined:
    Sep 2, 2004
    Messages:
    528
    Likes Received:
    0
    Trophy Points:
    16
    the site has been removed from cpanel, only this folders remains. Im not able to remove it.

    My question is, if there any problem if I leave the folder as it is? backups will backup it or the backups only backup sites loaded at apache only.

    Thanks
     
  4. cPanelTristan

    cPanelTristan Quality Assurance Analyst
    Staff Member

    Joined:
    Oct 2, 2010
    Messages:
    7,623
    Likes Received:
    21
    Trophy Points:
    38
    Location:
    somewhere over the rainbow
    cPanel Access Level:
    Root Administrator
    If the account doesn't exist, cPanel isn't going to try to back it up.
     
  5. bsasninja

    bsasninja Well-Known Member

    Joined:
    Sep 2, 2004
    Messages:
    528
    Likes Received:
    0
    Trophy Points:
    16
    ok, well as long it doesnt affect the server uptime I will stay with the "ghost" until I got fixed.
     
  6. bsasninja

    bsasninja Well-Known Member

    Joined:
    Sep 2, 2004
    Messages:
    528
    Likes Received:
    0
    Trophy Points:
    16
    btw if I try to chmod any of the files above says: file not found.
     
  7. cPanelTristan

    cPanelTristan Quality Assurance Analyst
    Staff Member

    Joined:
    Oct 2, 2010
    Messages:
    7,623
    Likes Received:
    21
    Trophy Points:
    38
    Location:
    somewhere over the rainbow
    cPanel Access Level:
    Root Administrator
    The only command I was ever able to run on files missing inodes was to move the directory a level above the one with ?? for it and it would let me move it. That's why I suggested trying to move the file or directories containing the corruption to see if that worked. That's about all you will be able to do.
     
  8. bsasninja

    bsasninja Well-Known Member

    Joined:
    Sep 2, 2004
    Messages:
    528
    Likes Received:
    0
    Trophy Points:
    16
    hey I moved the folder inside /tmp and voila! the files became writable with size, date and group owner. Now its erased completely.

    Thanks!
     
  9. nibb

    nibb Well-Known Member

    Joined:
    Mar 22, 2008
    Messages:
    301
    Likes Received:
    1
    Trophy Points:
    18
    I also had this recently on some cPanel server and I don´t know if this is a coincidence, but searching the inode that causes this they where also exactly the same like your case
    auto-whitelist.lock.server.site

    On a user account.
     
Loading...

Share This Page