The Community Forums

Interact with an entire community of cPanel & WHM users!
  1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

sshd fails after upgrade to latest Release

Discussion in 'General Discussion' started by RavenSoul_, Jul 7, 2005.

  1. RavenSoul_

    RavenSoul_ Well-Known Member

    Joined:
    Nov 2, 2004
    Messages:
    95
    Likes Received:
    0
    Trophy Points:
    6
    Location:
    Belgium
    32bit sshd/pam rpms installed on x86_64 servers

    Hi,

    I just did an upgrade from cPanel 10.0.0 R11 to cPanel 10.2.0 R82 (it's been a while, I know)

    Putty denied my root password after the - otherwise - successful update.

    Restarting OpenSSH in WHM results in:

    --

    Waiting for sshd to restart.... . . . . . . . . . . finished.

    sshd status root 24840 0.6 0.9 38892 18596 ? S 07:28 0:00 /usr/local/cpanel/whostmgr/bin/whostmgr ./ressshd
    root 24858 0.1 0.0 20492 1896 ? S 07:28 0:00 /usr/sbin/sshd




    Jul 7 07:28:27 sunv20z sshd: sshd -TERM succeeded Jul 7 07:28:27 sunv20z sshd: succeeded sshd has failed, please contact the sysadmin.

    --

    Any known issues? I have Centos 3.5 x86_64. Dual Opteron.

    --

    Update: Solution at http://bugzilla.cpanel.net/show_bug.cgi?id=2802
     
    #1 RavenSoul_, Jul 7, 2005
    Last edited: Jul 22, 2005
  2. internetfab

    internetfab Well-Known Member
    PartnerNOC

    Joined:
    Feb 20, 2003
    Messages:
    336
    Likes Received:
    0
    Trophy Points:
    16
    Location:
    Gothenburg, Sweden
    cPanel Access Level:
    DataCenter Provider
    Exactly same problem and have got it on 2 servers now.
    After a reboot it works perfectly again - BUT after a while openssh goes down again and trying to connect to the server through ssh gets you to type username and password - but it just hangs right after password and just stands there forever.

    cPanel 10.2.0-S83 CentOS 3.5 i686
    cPanel 10.2.0-E137 CentOS 3.4 i686

    Same problem on those 2.

    It also seems that https in horde and mysqladmin just makes the pages stand still forever and load load load when openssh is failed/down. Switching over to http loads them normally.

    This is getting really annoying and needs to be resolved ASAP

    sshd: [ != SSH] Jul 7 09:01:05 uranos sshd: sshd -TERM succeeded Jul 7 09:01:05 uranos sshd: succeeded sshd has failed, please contact the sysadmin.

    *EDIT*
    A cpanel ticket is now filed: cPanel ID# 104727
    *EDIT*
     
    #2 internetfab, Jul 7, 2005
    Last edited: Jul 7, 2005
  3. internetfab

    internetfab Well-Known Member
    PartnerNOC

    Joined:
    Feb 20, 2003
    Messages:
    336
    Likes Received:
    0
    Trophy Points:
    16
    Location:
    Gothenburg, Sweden
    cPanel Access Level:
    DataCenter Provider
    Does anyone know where to possibly find logfiles as to why SSHD has failed? Been searching through /var/log/messages /var/log/security and well most of the other log files as well.
     
  4. AndyReed

    AndyReed Well-Known Member
    PartnerNOC

    Joined:
    May 29, 2004
    Messages:
    2,222
    Likes Received:
    3
    Trophy Points:
    38
    Location:
    Minneapolis, MN
    Try reinstalling OpenSSL and OpenSSHD on your server.
     
  5. RavenSoul_

    RavenSoul_ Well-Known Member

    Joined:
    Nov 2, 2004
    Messages:
    95
    Likes Received:
    0
    Trophy Points:
    6
    Location:
    Belgium
    Daily LogWatch:

    --------------------- SSHD Begin ------------------------

    SSHD Killed: 6 Time(s)

    SSHD Started: 8 Time(s)

    Failed logins from these:
    root/password from x.x.193.21: 8 Time(s)

    **Unmatched Entries**
    PAM unable to dlopen(/lib/security/pam_shells.so)
    PAM [dlerror: /lib/security/pam_shells.so: cannot open shared object file: No such file or directory]
    PAM adding faulty module: /lib/security/pam_shells.so
    PAM unable to dlopen(/lib/security/pam_shells.so)
    PAM [dlerror: /lib/security/pam_shells.so: cannot open shared object file: No such file or directory]
    PAM adding faulty module: /lib/security/pam_shells.so
    PAM unable to dlopen(/lib/security/pam_shells.so)
    PAM [dlerror: /lib/security/pam_shells.so: cannot open shared object file: No such file or directory]
    PAM adding faulty module: /lib/security/pam_shells.so
    PAM unable to dlopen(/lib/security/pam_shells.so)
    PAM [dlerror: /lib/security/pam_shells.so: cannot open shared object file: No such file or directory]
    PAM adding faulty module: /lib/security/pam_shells.so
    PAM unable to dlopen(/lib/security/pam_shells.so)
    PAM [dlerror: /lib/security/pam_shells.so: cannot open shared object file: No such file or directory]
    PAM adding faulty module: /lib/security/pam_shells.so
    PAM unable to dlopen(/lib/security/pam_shells.so)
    PAM [dlerror: /lib/security/pam_shells.so: cannot open shared object file: No such file or directory]
    PAM adding faulty module: /lib/security/pam_shells.so
    PAM unable to dlopen(/lib/security/pam_shells.so)
    PAM [dlerror: /lib/security/pam_shells.so: cannot open shared object file: No such file or directory]
    PAM adding faulty module: /lib/security/pam_shells.so
    PAM unable to dlopen(/lib/security/pam_shells.so)
    PAM [dlerror: /lib/security/pam_shells.so: cannot open shared object file: No such file or directory]
    PAM adding faulty module: /lib/security/pam_shells.so
    PAM unable to dlopen(/lib/security/pam_shells.so)
    PAM [dlerror: /lib/security/pam_shells.so: cannot open shared object file: No such file or directory]
    PAM adding faulty module: /lib/security/pam_shells.so
    PAM unable to dlopen(/lib/security/pam_shells.so)
    PAM [dlerror: /lib/security/pam_shells.so: cannot open shared object file: No such file or directory]
    PAM adding faulty module: /lib/security/pam_shells.so
    sshd shutdown failed
    sshd shutdown failed
    sshd -TERM failed

    ---------------------- SSHD End -------------------------

    Seems like SSH Daemon can't read out the passwords because of the PAM errors...
     
    #5 RavenSoul_, Jul 7, 2005
    Last edited: Jul 8, 2005
  6. ramiel

    ramiel Member

    Joined:
    Mar 14, 2005
    Messages:
    11
    Likes Received:
    0
    Trophy Points:
    1
    Location:
    Mountain View, CA, USA
    Help!

    I have the same issue. Running Red Hat Enterprise 3.0 ES with cPanel 10.2.0-S83...

    WHM 10.1.0 cPanel 10.2.0-S83
    RedHat Enterprise 3 i686 - WHM X v3.1.0

    I have had cPanel setup to do automatic stable updates. All of a sudden starting at 1AM from my external monitoring system I saw the load avg rise. However neither the bandwidth, disk capacity, or CPU utilization changed! The only thing I could see is that the amount of free memory buffers went down to 0. I create MRTG graphs of system resources via SNMP so that's the only reason I got to see them.

    When I try to SSH into the server I can't. Putty won't work, I also can't ssh from other linux boxes as root or as a regular user.

    I drove to the co-location facility, and I was also NOT able to login from the keyboard/monitor console!!!!

    Rebooted, modified the GRUB boot parm to single user mode and was able to get root there. Then I modifed the /etc/inittab file so that it gives me a /bin/bash on tty1. Then went to init 3 (default run level).

    Now I can see the shell, but still can't login from any other windows.

    I proceeded to uninstall all of SSH and PAM files using

    rpm -qa|grep -i pam
    rpm -qa|grep -i openssh

    then I removed them using `rpm -e openssh openssh-clients openssh-server`
    and same thing with pam, except for pam I had to give `rpm -e pam --nodeps`

    then I installed them fresh using `up2date -i openssh openssh-clients openssh-server`
    and `up2date -i pam` etc.

    Still no go!

    Another strange thing, if I run `crontab -l` it just freezes too!!!! I think it's because crontab looks up the auth on the system for the user's cron??

    I also can NOT kill crrond! even with `kill -9 pid` it won't die! ??

    I have kept the server always up to the latest updates... both OS and cpanel.

    server is only used for HTTP, SSH, FTP (pure-ftpd), SMTP (exim), and POP3 (qpopper).

    I checked on `netstat -taupen` and did not see anything unusual that didn't belong there.

    I also ran `rpm -V pam` and didn't see anything having been modified.

    I'm going to the colo facility again in the morning... 8 hours from now.

    Anyone have any IDEAS??????
     
  7. ramiel

    ramiel Member

    Joined:
    Mar 14, 2005
    Messages:
    11
    Likes Received:
    0
    Trophy Points:
    1
    Location:
    Mountain View, CA, USA
    Got SSH working!

    Got SSH and CROND working again via:

    /etc/init.d/audit stop
    /etc/init.d/crond stop
    rmmod audit
    echo alias audit off >> /etc/modules.conf
    /etc/init.d/crond start

    Console is working too.

    But in case for future:

    edit your /etc/inittab file and replace the tty1 line with:

    1:2345:respawn:/bin/sh

    On boot up it will give you a shell without asking for login....

    I hope cPanel figures out why this is happening!

    I think it's with the LaUS system!? What is that?
     
    #7 ramiel, Jul 12, 2005
    Last edited: Jul 12, 2005
  8. ensermo

    ensermo Active Member

    Joined:
    Apr 24, 2004
    Messages:
    32
    Likes Received:
    0
    Trophy Points:
    6
    Location:
    Delft
    Don't know if this applies to my problem but doesn't hurt to ask.

    I just co-located a Dual Opteron server and installed the minimum installation for CentOS 4.1 x86_64.

    I could login fine with ssh. After I started the CPanel installation it frooze halfway while downloading packages. Tried to log in via another SSH and got the username screen but when I fill in my password nothing happens. SSH just doesnt react. I thought it had something to do with iptables (which I don't have right now) so I tried from other machines with different ip's and still the same.

    Could this be related to the same problem you guys are having?

    Anyways I'm driving now to the datacenter to see if I can solve it there....
     
  9. RavenSoul_

    RavenSoul_ Well-Known Member

    Joined:
    Nov 2, 2004
    Messages:
    95
    Likes Received:
    0
    Trophy Points:
    6
    Location:
    Belgium
    Yup, same issue :)

    Join the club ;)

    I suggest you wait a bit to drive all the way: they are working on it atm...
     
    #9 RavenSoul_, Jul 12, 2005
    Last edited: Jul 12, 2005
  10. StevenC

    StevenC Well-Known Member

    Joined:
    Jan 1, 2004
    Messages:
    254
    Likes Received:
    0
    Trophy Points:
    16
    I had this happen on a centos 3.4 machine a few weeks back. Same issue. What happened was ... some rpms/binarys from the wrong arch i386 were installed instead of the x86_64 which caused pam to fail. Reinstalling openssh and all the related pam and dependecys fixed the issue.
     
  11. RavenSoul_

    RavenSoul_ Well-Known Member

    Joined:
    Nov 2, 2004
    Messages:
    95
    Likes Received:
    0
    Trophy Points:
    6
    Location:
    Belgium
    I saw both i386 and x86_64 pam files while debugging the Daily LogWatch errors, so that may be indeed the bad guy...


    On the other hand:

    No 64 bit...
     
    #11 RavenSoul_, Jul 13, 2005
    Last edited: Jul 13, 2005
  12. RavenSoul_

    RavenSoul_ Well-Known Member

    Joined:
    Nov 2, 2004
    Messages:
    95
    Likes Received:
    0
    Trophy Points:
    6
    Location:
    Belgium
  13. IberHosting

    IberHosting Well-Known Member

    Joined:
    Jun 1, 2005
    Messages:
    124
    Likes Received:
    0
    Trophy Points:
    16
    From a new installation, CentOS 4.0 and 4.1, same error. With FreeBSD too.

    Thanks,

    Antonio
     
  14. Paxuist

    Paxuist Member

    Joined:
    Jul 11, 2005
    Messages:
    9
    Likes Received:
    0
    Trophy Points:
    1
    crap , i have to install new fresh centos x86_64 tomorrow .
     
  15. ensermo

    ensermo Active Member

    Joined:
    Apr 24, 2004
    Messages:
    32
    Likes Received:
    0
    Trophy Points:
    6
    Location:
    Delft
    I just gave up with Centos 4.1 + CPanel. It gave me a lot of headaches on my hardware (Tyan B2882 different kinds of HD)
    - Problems with the sata controller
    - Problems with SSH
    - Problems installing CPanel (ex. not possible to do quota check)

    So finally I installed Centos 3.5 and it seems to be working now.
     
  16. internetfab

    internetfab Well-Known Member
    PartnerNOC

    Joined:
    Feb 20, 2003
    Messages:
    336
    Likes Received:
    0
    Trophy Points:
    16
    Location:
    Gothenburg, Sweden
    cPanel Access Level:
    DataCenter Provider
    Gonna try installing on a new CentOS 4.1 box on monday. Last time I tried it got stuck in the middle of cpanel install (during perl installaion) and hung there forever (left it for 6 hrs+ until doing init 0).

    The boxes that I had problems with at a start seems to be ok now after rebooting, logging in as root and removing cpup. Doing a /scripts/updatenow and then /scripts/cpup --force on latest stable (those boxes were centos 3.4 and 3.5). We're still keepin a watchful eye on them.

    Still confused over what made them rise like *** in load and not being able to log in through SSH (even at times when load was low - just hung after password prompt).

    The restart script for openssh gives an error - but does seem to restart ssh after a while on centos in whm. Not that it helped with those boxes that couldnt login through ssh. Only a reboot helped (graceful or forceful through WHM worked fine)
     
  17. IberHosting

    IberHosting Well-Known Member

    Joined:
    Jun 1, 2005
    Messages:
    124
    Likes Received:
    0
    Trophy Points:
    16
    Hello, If the port is default, 22, you get a sshd restarted ok from whm, but if the port isn't 22, is another port, you get the fail message.

    Thanks,

    Antonio
     
  18. RavenSoul_

    RavenSoul_ Well-Known Member

    Joined:
    Nov 2, 2004
    Messages:
    95
    Likes Received:
    0
    Trophy Points:
    6
    Location:
    Belgium
    #18 RavenSoul_, Jul 18, 2005
    Last edited: Jul 22, 2005
  19. jsnape

    jsnape Well-Known Member

    Joined:
    Mar 11, 2002
    Messages:
    174
    Likes Received:
    0
    Trophy Points:
    16
    I had the exact same problem and applied the fix and nothing happened. With each reboot the time until the next hanging login attempt seemed to be cut in half. Until no login was possible even a few minutes after rebooting. Contacted GNAX guys and they figured it out in 45 seconds.

    Luis - GNAX
    Sorry the article link is here:
    http://www.eth0.us/auditd

    I've disabled auditd from starting up as mentioned there.

    I've also cleared the logs for auditd

    root@serve [~]# !du
    du -sh /var/log/audit*
    0 /var/log/audit
    81M /var/log/audit.d

    This should be resolved.
    ****

    And it is resolved
     
Loading...

Share This Page