The Community Forums

Interact with an entire community of cPanel & WHM users!
  1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

input/out error and bad file descriptors

Discussion in 'General Discussion' started by eva2000, Jul 19, 2003.

  1. eva2000

    eva2000 Well-Known Member

    Joined:
    Aug 14, 2001
    Messages:
    322
    Likes Received:
    10
    Trophy Points:
    18
    Location:
    Brisbane, Australia
    cPanel Access Level:
    Root Administrator
    Twitter:
    this all started to happen last night after 49 beautiful days uptime

    Running a WHM/CPanel 6.4.2 server which had the below problem, updating to latest right now and still same problem

    WHM 7.2.0 cPanel 7.2.1-S67
    RedHat 7.3 - WHM X v2.1
    Apache 1.3.28
    Linuxe 2.4.20-18.7smp
    PHP 4.3.2
    MySQL 4.0.13

    Dual AthlonMP 2000+
    2GB PC2100 DDRAM ECC
    2x 18GB 10k IBM SCSI
    1x 80GB EIDE

    I've tried recompiling apache using easyapache options 1 through to 6 as well

    Problem 1. apache keeps gracefully restarting everyone few minutes

    Problem 2.

    My vBulletin word table corrupted and trying to repair it with mysqlcheck resulted in:
    Code:
    mysqlcheck -s vbdbname word
    
    vbdbname.word
    warning  : Table is marked as crashed and last repair failed
    warning  : Size of indexfile is: 18095104      Should be: 10494976
    error    : Record at pos: 39392595 is not remove-marked
    error    : record delete-link-chain corrupted
    error    : Corrupt
    
    trying to use mysqlcheck to repair it I get an error about bad file descriptors
    Code:
    mysqlcheck -r vbdbname word
    
    vbdbname.word
    warning  : Duplicate key for record at 39392595 against record at 83820
    error    : 5 when fixing table
    error    : Can't copy datafile-header to tempfile, error 9
    status   : Operation failed
    
    ----------------------------
    my server current has the following limits
    Code:
    ulimit -aS
    core file size        (blocks, -c) 0
    data seg size         (kbytes, -d) unlimited
    file size             (blocks, -f) unlimited
    max locked memory     (kbytes, -l) unlimited
    max memory size       (kbytes, -m) unlimited
    open files                    (-n) 1024
    pipe size          (512 bytes, -p) 8
    stack size            (kbytes, -s) 8192
    cpu time             (seconds, -t) unlimited
    max user processes            (-u) 7168
    virtual memory        (kbytes, -v) unlimite
    
    
    ulimit -aH
    core file size        (blocks, -c) unlimited
    data seg size         (kbytes, -d) unlimited
    file size             (blocks, -f) unlimited
    max locked memory     (kbytes, -l) unlimited
    max memory size       (kbytes, -m) unlimited
    open files                    (-n) 1024
    pipe size          (512 bytes, -p) 8
    stack size            (kbytes, -s) unlimited
    cpu time             (seconds, -t) unlimited
    max user processes            (-u) 7168
    virtual memory        (kbytes, -v) unlimited
    
    lsof | grep httpd | wc -l
    
    returns = 8369
    
    ----------------------------
    So i wanted to attempt myisamchk repair by first shutting down mysql, and copying the actual database data files for backup but i get an error in the same word table, it gives me:
    Code:
    reading `vbdbname/word.MYI': Input/output error
    
    after the input/output error showed itself, I checked database sizes for original and backup
    Code:
    du /var/lib/mysql/vbdbname
    1435964 
    
    du /home/mysql190703/data/vbdbname-bak
    1428532 
    
    ------------------
    In my apache error log i'm getting errors after i just updated from Apache 1.3.27 to 1.3.28 via WHM/CPanel /scripts/easyapache

    error listed are:
    Apache seems to be gracefully restarting every few minutes :(

    -----
    trying to do myisamchk repair while mysql was shutdown, resulted in same error no5 : input/output error

    check resulted in
    Code:
    /usr/bin/myisamchk -s /var/lib/mysql/vbdbname/word.MYI
    myisamchk: MyISAM file /var/lib/mysql/vbdbname/word.MYI
    myisamchk: warning: Table is marked as crashed and last repair failed
    myisamchk: warning: Size of indexfile is: 18095104      Should be: 10494976
    myisamchk: error: Record at pos: 39392595 is not remove-marked
    myisamchk: error: record delete-link-chain corrupted
    myisamchk: error: Found 716628 keys of 716627
    MyISAM-table '/var/lib/mysql/vbdbname/word.MYI' is corrupted
    Fix it using switch "-r" or "-o"
    
    repair didn't work
    Code:
    /usr/bin/myisamchk -r /var/lib/mysql/vbdbname/word.MYI
    - recovering (with sort) MyISAM-table '/var/lib/mysql/vbdbname/word.MYI'
    Data records: 716627
    - Fixing index 1
    - Fixing index 2
    myisamchk: warning: Duplicate key for record at   39392595 against record at      83820
    /usr/bin/myisamchk: Error writing file '/var/lib/mysql/vbdbname/word.MYI' (Errcode: 5)
    myisamchk: error: 5 when fixing table
    MyISAM-table '/var/lib/mysql/vbdbname/word.MYI' is not fixed because of errors
    Try fixing it by using the --safe-recover (-o) or the --force (-f) option
    
    repair using safe recover mode didn't work
    Code:
    /usr/bin/myisamchk -o /var/lib/mysql/vbdbname/word.MYI
    - recovering (with keycache) MyISAM-table '/var/lib/mysql/vbdbname/word.MYI'
    Data records: 716627
    Duplicate key  2 for record at   39392595 against new record at      83820
    myisamchk: warning: 1 records have been removed
    myisamchk: error: 5 when trying to write bufferts
    MyISAM-table '/var/lib/mysql/vbdbname/word.MYI' is not fixed because of errors
    Try fixing it by using the --safe-recover (-o) or the --force (-f) option
    
    -----
    so is this actual filesystem or hard drive problem/corruption with the input/out error ?? would it be related to my graceful restarts in some way ?

    this is the first time I ever experienced this problem :(

    cPanel.net Support Ticket Number:
     
    #1 eva2000, Jul 19, 2003
    Last edited: Jul 19, 2003
  2. FWC

    FWC Well-Known Member

    Joined:
    May 13, 2002
    Messages:
    354
    Likes Received:
    0
    Trophy Points:
    16
    Location:
    Ontario, Canada
    If you have a backup of the /var/lib/mysql/vbdbname directory, try deleting it from SSH and replacing it. You have an error in the file structure in there. Tools working inside the existing directory will fail. I have fixed this before by deleting the entire directory and starting over.
     
  3. goodmove

    goodmove Well-Known Member

    Joined:
    May 12, 2003
    Messages:
    624
    Likes Received:
    0
    Trophy Points:
    16
    Eva, have you found a solution to the apache problem? I have a server that's showing the same symptoms. :(
     
  4. eva2000

    eva2000 Well-Known Member

    Joined:
    Aug 14, 2001
    Messages:
    322
    Likes Received:
    10
    Trophy Points:
    18
    Location:
    Brisbane, Australia
    cPanel Access Level:
    Root Administrator
    Twitter:
    thanks FWC!

    i was lucky enough to have copied the actual mysql data files after i couldn't repair the word table.. so recreating a new database and importing the copy, allowed me to run the repair tools on the word table :)

    cPanel.net Support Ticket Number:
     
  5. eva2000

    eva2000 Well-Known Member

    Joined:
    Aug 14, 2001
    Messages:
    322
    Likes Received:
    10
    Trophy Points:
    18
    Location:
    Brisbane, Australia
    cPanel Access Level:
    Root Administrator
    Twitter:
    Not yet, but i haven't been monitoring the apache logs since the database corruption was first priority

    cPanel.net Support Ticket Number:
     
  6. FWC

    FWC Well-Known Member

    Joined:
    May 13, 2002
    Messages:
    354
    Likes Received:
    0
    Trophy Points:
    16
    Location:
    Ontario, Canada
    I'm glad it worked, George. I've had the same error before in a non-mysql directory. I couldn't delete the bad files or anything. I finally tried nuking the directory and it worked. It's one of those things you don't forget. :)
     
    #6 FWC, Jul 19, 2003
    Last edited: Jul 19, 2003
Loading...

Share This Page