The Community Forums

Interact with an entire community of cPanel & WHM users!
  1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

slocate segmentation fault

Discussion in 'General Discussion' started by verdon, Oct 8, 2005.

  1. verdon

    verdon Well-Known Member

    Joined:
    Nov 1, 2003
    Messages:
    836
    Likes Received:
    2
    Trophy Points:
    18
    Location:
    Northern Ontario, Canada
    cPanel Access Level:
    Root Administrator
    Hi

    For the last 3 days, I've been getting slocate errors from cron.daily. The exact error is

    /etc/cron.daily/slocate.cron: line 3: 17746 Segmentation fault /usr/bin/updatedb -f "nfs,smbfs,ncpfs,proc,devpts" -e "/tmp,/var/tmp,/usr/tmp,/afs,/net"

    I tried renaming my /var/lib/slocate dir and creating a new one, then running /etc/cron.daily/slocate.cron manually, to see what would happen. I still get the error however.

    When I check out the file var/lib/slocate/slocate.db.tmp that is created during this process, it has 534 lines and the last one is ...

    ... hmmm the forum here won't accept the example I am trying to include, even wrapped in code tags

    Any ideas how I might proceed from here?

    Thanks,
     
    #1 verdon, Oct 8, 2005
    Last edited: Oct 8, 2005
  2. chirpy

    chirpy Well-Known Member

    Joined:
    Jun 15, 2002
    Messages:
    13,475
    Likes Received:
    20
    Trophy Points:
    38
    Location:
    Go on, have a guess
    A few ideas:

    1. Make sure that /var isn't out of space ;)

    2. Empty out /var/lib/slocate/* again and then check the perms/owner of /var/lig/slocate it should be:

    drwxr-x--- 2 root slocate 4096 Oct 8 04:32 /var/lib/slocate/

    3. Run updatedb with the -v qualifier so that you can see what files are being processed and see if it keels over at a particular point:

    /usr/bin/updatedb -fv "nfs,smbfs,ncpfs,proc,devpts" -e "/tmp,/var/tmp,/usr/tmp,/afs,/net"

    4. If it doesn't get as far as listing files before the seg fault from the verbose tag, try running it through strace, though you're likely to get a lot of output:

    strace /usr/bin/updatedb -f "nfs,smbfs,ncpfs,proc,devpts" -e "/tmp,/var/tmp,/usr/tmp,/afs,/net"

    5. Last try, remove the slocate rpm (package, depends what OS you're using) and try reinstalling it.
     
  3. verdon

    verdon Well-Known Member

    Joined:
    Nov 1, 2003
    Messages:
    836
    Likes Received:
    2
    Trophy Points:
    18
    Location:
    Northern Ontario, Canada
    cPanel Access Level:
    Root Administrator
    Yup, that's fine

    That's OK too

    No output

    Tons of output here. The final few lines before it dies are...
    Code:
    fchdir(5)                               = 0
    getdents64(5, /* 24 entries */, 4096)   = 672
    lstat64(".registry", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
    lstat64(".lock", {st_mode=S_IFREG|0644, st_size=0, ...}) = 0
    lstat64(".filemap", {st_mode=S_IFREG|0644, st_size=2312, ...}) = 0
    lstat64("Archive", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
    lstat64("doc", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
    lstat64("Console", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
    lstat64("data", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
    lstat64("pearcmd.php",  <unfinished ...>
    +++ killed by SIGSEGV +++
    
    I'll leave this as a last ditch effort. I am using CentOS 3.5. When I use WHM -> Install a RPM to have a look, I don't see anything for slocate though?

    Finally, one other thing I've noticed in the last day or 2 is that when running top, I can see a zombie process that won't clear and I don't seem to be able to kill.
    5 root 9 0 0 0 0 Z 0.0 0.0 195:06 0 kswapd <defunct>
    I don't know if this is related or not, but it's something I've not seen before in the 294 days of uptime that I've had this server.

    Chirpy, thanks for your advice on this issue :)
     
  4. chirpy

    chirpy Well-Known Member

    Joined:
    Jun 15, 2002
    Messages:
    13,475
    Likes Received:
    20
    Trophy Points:
    38
    Location:
    Go on, have a guess
    kswapd is a system process and shouldn't be killed. If the server has been up that long you should probably try a report first as you could be suffering from memory fragmentation. I would also assume that if it's been up that long you're due a kernel upgrade which might be an opportune time to do that if you're rebooting.

    Lastly, you could try and find out where that pearcmd.php is on disk and then look within the same directory and see if there's anything that looks like file system corruption. Either way, it would certainly be work forcing an FSCK of your disks when you reboot to check for any corruption just incase the file system is on it's way out.
     
  5. gemininetcom

    gemininetcom Active Member

    Joined:
    Nov 29, 2003
    Messages:
    36
    Likes Received:
    0
    Trophy Points:
    6
    Last night received same problem with run-parts /etc/cron.daily and segmentation fault:

    *** start error log

    /etc/cron.daily/slocate.cron:

    /etc/cron.daily/slocate.cron: line 3: 31164 Segmentation fault /usr/bin/updatedb -f "nfs,smbfs,ncpfs,proc,devpts" -e "/tmp,/var/tmp,/usr/tmp,/afs,/net"

    *** end error log

    Ran you suggestions 1 up/included 4 and last lines result of (4)
    strace /usr/bin/updatedb -f "nfs,smbfs,ncpfs,proc,devpts" -e "/tmp,/var/tmp,/usr/tmp,/afs,/net"
    is below


    *** start of strace

    ...
    ....
    fchdir(5) = 0
    close(5) = 0
    open("browse", O_RDONLY|O_NONBLOCK|O_LARGEFILE|O_DIRECTORY) = 5
    fstat64(5, {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
    fcntl64(5, F_SETFD, FD_CLOEXEC) = 0
    fstat64(5, {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
    fchdir(5) = 0
    getdents64(5, /* 13 entries */, 4096) = 448
    lstat64("column_footers.inc", {st_mode=S_IFREG|0644, st_size=645, ...}) = 0
    lstat64("actions.inc", {st_mode=S_IFREG|0644, st_size=3273, ...}) = 0
    lstat64("column_headers.inc", {st_mode=S_IFREG|0644, st_size=2075, ...}) = 0
    lstat64("contactrow.inc", {st_mode=S_IFREG|0644, st_size=2689, ...}) = 0
    lstat64("footer.inc", {st_mode=S_IFREG|0644, st_size=274, ...}) = 0
    lstat64("header.inc", {st_mode=S_IFREG|0644, st_size=65, ...}) = 0
    lstat64("javascript.inc", {st_mode=S_IFREG|0644, st_size=2504, ...}) = 0
    lstat64("search.inc", {st_mode=S_IFREG|0644, st_size=4576, ...}) = 0
    lstat64("select.inc", {st_mode=S_IFREG|0644, st_size=659, ...}) = 0
    lstat64("footerAlpha.inc", {st_mode=S_IFREG|0644, st_size=796, ...}) = 0
    lstat64("search_criteria.inc", {st_mode=S_IFREG|0644, st_size=325, ...}) = 0
    getdents64(5, /* 0 entries */, 4096) = 0
    close(5 <unfinished ...>
    +++ killed by SIGSEGV +++
    root@saturn [~]#

    *** end of strace

    Seems that strace at last "close 5(" got kicked out.

    System changes of last 7 days:
    - had last Sunday (16th) a full memory replacement
    - on Monday 17th no problem: upcp and cpbackup OK and no errors
    - on Tuesday 18th suddenly no cpbackup performed -- upcp was OK
    - today Wednesday 19th upcp and cpbackup OK but segmentation problem.

    Any ideas Jonathan ?
     
    #5 gemininetcom, Oct 19, 2005
    Last edited: Oct 19, 2005
  6. chirpy

    chirpy Well-Known Member

    Joined:
    Jun 15, 2002
    Messages:
    13,475
    Likes Received:
    20
    Trophy Points:
    38
    Location:
    Go on, have a guess
    Have you tried looking for the previous file search_criteria.inc and see if anything in that directory looks troublesome?

    My only other idea apart from those mentioned would be to reboot and run and FSCK to check for/correct and file system corruption.

    I have seen segfaults with slocate before and tracked it down to a corrupt file on the file system and simply put in an exclusion into the /etc/cron.daily/slocate.cron file.
     
  7. verdon

    verdon Well-Known Member

    Joined:
    Nov 1, 2003
    Messages:
    836
    Likes Received:
    2
    Trophy Points:
    18
    Location:
    Northern Ontario, Canada
    cPanel Access Level:
    Root Administrator
    Sorry, I should have put a summary here too... doing a reboot with fsck did the job for me. Check man for sure, but I think it was just....

    shutdown -rF
     
  8. gemininetcom

    gemininetcom Active Member

    Joined:
    Nov 29, 2003
    Messages:
    36
    Likes Received:
    0
    Trophy Points:
    6
    Located "search_criteria.inc" in [/usr/local/cpanel/base/horde/turba/templates/browse] and tried a list, but system came back immediately with "Segmentation fault". So can't even see what's in that directory.

    Tonight a reboot with FSCK is scheduled. My NOC suggested a restore, but find this the extreme :D
     
    #8 gemininetcom, Oct 20, 2005
    Last edited: Oct 20, 2005
  9. gemininetcom

    gemininetcom Active Member

    Joined:
    Nov 29, 2003
    Messages:
    36
    Likes Received:
    0
    Trophy Points:
    6
    You wouldn't believe it, but last Sunday the Segmentation fault mysteriously dispappeared :eek:

    Still using 10.8.1-RELEASE_4 both on last Saturday and Sunday according to upcp.

    Tested
    (1)
    /usr/bin/updatedb -fv "nfs,smbfs,ncpfs,proc,devpts" -e "/tmp,/var/tmp,/usr/tmp,/afs,/net" without errors
    (2)
    Directory /usr/local/cpanel/base/horde/turba/templates/browse with search_criteria.inc is accessable and contents can be listed

    ... thus my problem solved but it beats me what did the trick :D
     
Loading...

Share This Page