The Community Forums

Interact with an entire community of cPanel & WHM users!
  1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Problems since last night's update?

Discussion in 'General Discussion' started by dspillett, Sep 15, 2007.

  1. dspillett

    dspillett Active Member

    Joined:
    Oct 2, 2005
    Messages:
    26
    Likes Received:
    0
    Trophy Points:
    1
    My cPanel setup upgraded itself at approx 0500GTM (it is currently 1025GMT) and since then I'm seeing the following problems:
    • The load reading has been constantly higer than usual, even though the machine is practically idle most of the time. Running top shows no process using anything more than a fraction of CPU time, in eith busy or io-wait states. /proc/io_status confirms that there is little IO going on and the load readings of the host machine confirm that there is little CPU being used. I'd notmall expect to see load readings between 0.2 and 0.5 on this machine except the occasional burst of activity, and it is currently constantly above 3.5 despite apparently no tasks doing anything.
    • Some services keep failing, or at least the monitor things they do and keeps restarting them. This is usually exim and cpservd. Exim does seem to be running OK, as I'm getting the "service automatically restarted" messages to an external account.
    • WHM keeps stopping responding to HTTPS requests on port 2087, which lends some credence to the messages that cpservd is stopping, but it doesn't stop responding to HTTP requests on port 2086.
    • When poking arouns WHM over HTTP://<addr>:2086, the I keep seemingly getting logged out (so the browsers HTTP-Auth dialogue appears). Once I've re-authenticated I continue where I left off, but twice this has happened so many times in a short time that the brute-force detector blocks me for a few minutes.
    This instance of cPanel/RELEASE in running in a UserModeLinux based VM on CentOS 4.5.


    Is anyone else having similar trouble, or is it just me?

    Is there anywhere else I can check for what might be causing load to show higher values than it should when the machine it idle?

    My first worry was that the VM had been hacked, but as the problems seem to have started directly after the update (that is when I started getting the "service failed, restarting" emails I'm not currently thinking that this is the case. I don't think the unexpected load reading is due it UML as my UML kernel has not changed recently and has been running fine for months.

    I've tried forcing a complete update of cPanel (in case something odd happened in the overnight update that left a problem a forced update would clear), making sure that the underlying CentOS is uptodate, and rebooting - all several times.
     
  2. ahostli

    ahostli Active Member

    Joined:
    May 28, 2006
    Messages:
    40
    Likes Received:
    0
    Trophy Points:
    6
    No (maybe because Im using STABLE release), but after yerterdays updates im getting this error when trying to use phpMyadmin:
    /usr/local/cpanel/3rdparty/bin/php: /usr/lib/mysql/libmysqlclient.so.14: no version information available (required by /usr/local/cpanel/3rdparty/bin/php) /usr/local/cpanel/3rdparty/bin/php: relocation error: /usr/local/cpanel/3rdparty/bin/php: symbol __udivdi3, version libmysqlclient_14 not defined in file libmysqlclient.so.14 with link time reference
     
  3. dspillett

    dspillett Active Member

    Joined:
    Oct 2, 2005
    Messages:
    26
    Likes Received:
    0
    Trophy Points:
    1
    If I can't sort out the current issues quickly any other way, is downgrading from RELEASE to STABLE considered a safe operation? (though is you are having mysql problems on STABLE, maybe I'd just be swapping one problem for another...)
     
  4. nyjimbo

    nyjimbo Well-Known Member

    Joined:
    Jan 25, 2003
    Messages:
    1,125
    Likes Received:
    0
    Trophy Points:
    36
    Location:
    New York
    You might have something going on unrelated to the update/upgrade.

    Do a bunch of things like look for weird perl tasks:

    ps ax | grep "perl"

    Look for lots of ftp login attempts:

    pure-ftpwho

    look for excessive pop logins:

    ps ax| grep "pop"

    How many tasks are running in top, even if they show very little cpu usage each?

    Do a "netstat -m" to see if you are getting slammed on your network with anything that might not show up on top or other checks. If your mbuf usage is very high then some kind of attack could be happening.

    What does WHM show when you look for "Show mysql processes" and "Apache Status", anything weird going on.

    Did you try restarting some of the monsters like apache, exim, mysql, etc.. sometimes something might be stuck in a loop.

    Check "top" to see if you are running out of ram and doing alot of swapping.

    Do a "exiwhat | sort +6| more" to do a quick check to see if you are getting hammered by these new zombie spammers

    there is probably alot more you can check but some of these might give you a clue if anything weird is running that might be causing this load issue.
     
  5. dspillett

    dspillett Active Member

    Joined:
    Oct 2, 2005
    Messages:
    26
    Likes Received:
    0
    Trophy Points:
    1
    Though the the fact the problems started immediately after the update, or so it would seem, is bit of a coincidence.

    None of those, or my earlier scans for ssh/ftp/exim/apache logs showed anything out of the ordinary.

    -m does not seem to be an option on netstat. I'm fairly sure that it isn't network activity, as I'm not seeing lots of activity going through the host machine.

    None of those processes seem to be churning at all, and I've tried completely rebooting the VM as well as restarting the individual services.

    73 tasks, which I don't think is abnormal, none of which are showing much activity in top, and the machine isn't yet touching swap since the last reboot a few hours ago.

    Starting artificial tasks a short time apart and checking the PIDs allocated shows that processes don't seem to be being created and destroyed faster than top will see them.[/QUOTE]

    Whatever is happening is affecting secure variants of the services more than the insecure ones, as getting into http:2082 is far more reliable than getting into https:2083 (it works almost all the time rather than almost, but not quite, never). The same goes for the WHM and Webmail ports. On the http ports you do regularly need to re-authenticate, but I'm thinking that this is due to the precess monitor seeing the https version not responding and resetting the whole lot.

    Luckyly, Apache and the IMAP/POP services seem to be functioning OK, so people aren't losing mail and page views.
     
    #5 dspillett, Sep 15, 2007
    Last edited: Sep 15, 2007
  6. nyjimbo

    nyjimbo Well-Known Member

    Joined:
    Jan 25, 2003
    Messages:
    1,125
    Likes Received:
    0
    Trophy Points:
    36
    Location:
    New York
    Ouch, I guess thats a freebsd switch. I hate it when these O/S's have different syntax.
     
  7. dspillett

    dspillett Active Member

    Joined:
    Oct 2, 2005
    Messages:
    26
    Likes Received:
    0
    Trophy Points:
    1
    Aye. BSD and the tool variant usually found on Linux systems don't differ much, though sometimes they do in unexpected places.

    Could you send me the text describing the option from your system's man page? There is likely another tool in the standard set that does the same job under Linux (or even just a /proc entry that carries the appropriate data).
     
  8. nyjimbo

    nyjimbo Well-Known Member

    Joined:
    Jan 25, 2003
    Messages:
    1,125
    Likes Received:
    0
    Trophy Points:
    36
    Location:
    New York
    NAME
    netstat -- show network status

    DESCRIPTION
    The netstat command symbolically displays the contents of various net-
    work-related data structures. There are a number of output formats,
    depending on the options for the information presented.

    netstat [-AaLnSW] [-f protocol_family | -p protocol] [-M core]
    [-N system]
    Display a list of active sockets (protocol control blocks) for
    each network protocol, for a particular protocol_family, or for a
    single protocol. If -A is also present, show the address of a
    protocol control block (PCB) associated with a socket; used for
    debugging. If -a is also present, show the state of all sockets;
    normally sockets used by server processes are not shown. If -L
    is also present, show the size of the various listen queues. The
    first count shows the number of unaccepted connections, the sec-
    ond count shows the amount of unaccepted incomplete connections,
    and the third count is the maximum number of queued connections.
    If -S is also present, show network addresses as numbers (as with
    -n) but show ports symbolically.

    netstat -i | -I interface [-abdhntW] [-f address_family] [-M core]
    [-N system]
    Show the state of all network interfaces or a single interface
    which have been auto-configured (interfaces statically configured
    into a system, but not located at boot time are not shown). An
    asterisk (``*'') after an interface name indicates that the
    interface is ``down''. If -a is also present, multicast
    addresses currently in use are shown for each Ethernet interface
    and for each IP interface address. Multicast addresses are shown
    on separate lines following the interface address with which they
    are associated. If -b is also present, show the number of bytes
    in and out. If -d is also present, show the number of dropped
    packets. If -h is also present, print all counters in human
    readable form. If -t is also present, show the contents of
    watchdog timers. If -W is also present, print interface names
    using a wider field size.

    netstat -w wait [-I interface] [-d] [-M core] [-N system]
    At intervals of wait seconds, display the information regarding
    packet traffic on all configured network interfaces or a single
    interface. If -d is also present, show the number of dropped
    packets.

    netstat -s [-s] [-z] [-f protocol_family | -p protocol] [-M core]
    [-N system]
    Display system-wide statistics for each network protocol, for a
    particular protocol_family, or for a single protocol. If -s is
    repeated, counters with a value of zero are suppressed. If -z is
    also present, reset statistic counters after displaying them.

    netstat -i | -I interface -s [-f protocol_family | -p protocol] [-M core]
    [-N system]
    Display per-interface statistics for each network protocol, for a
    particular protocol_family, or for a single protocol.

    netstat -m [-M core] [-N system]
    Show statistics recorded by the memory management routines
    (mbuf(9)). The network manages a private pool of memory buffers.

    netstat -r [-AanW] [-f address_family] [-M core] [-N system]
    Display the contents of all routing tables, or a routing table
    for a particular address_family. If -A is also present, show the
    contents of the internal Patricia tree structures; used for
    debugging. If -a is also present, show protocol-cloned routes
    (routes generated by an RTF_PRCLONING parent route); normally
    these routes are not shown. When -W is also present, show the
    path MTU for each route, and print interface names with a wider
    field size.

    netstat -rs [-s] [-M core] [-N system]
    Display routing statistics. If -s is repeated, counters with a
    value of zero are suppressed.

    netstat -g [-W] [-f address_family] [-M core] [-N system]
    Show information related to multicast (group address) routing.
    By default, show the IP Multicast virtual-interface and routing
    tables, and multicast group memberships.

    netstat -gs [-s] [-f address_family] [-M core] [-N system]
    Show multicast routing statistics. If -s is repeated, counters
    with a value of zero are suppressed.

    Some options have the general meaning:

    -f address_family, -p protocol
    Limit display to those records of the specified address_family or a
    single protocol. The following address families and protocols are
    recognized:

    Family Protocols
    inet (AF_INET) bdg, divert, icmp, igmp, ip, ipsec,
    pim, tcp, udp
    inet6 (AF_INET6) bdg, icmp6, ip6, ipsec6, rip6, tcp, udp
    pfkey (PF_KEY) pfkey
    atalk (AF_APPLETALK) ddp
    netgraph, ng (AF_NETGRAPH) ctrl, data
    ipx (AF_IPX) ipx, spx
    unix (AF_UNIX)
    link (AF_LINK)

    -M Extract values associated with the name list from the specified
    core instead of the default /dev/kmem.

    -N Extract the name list from the specified system instead of the
    default, which is the kernel image the system has booted from.

    -n Show network addresses and ports as numbers. Normally netstat
    attempts to resolve addresses and ports, and display them symboli-
    cally.

    -W In certain displays, avoid truncating addresses even if this causes
    some fields to overflow.

    The default display, for active sockets, shows the local and remote
    addresses, send and receive queue sizes (in bytes), protocol, and the
    internal state of the protocol. Address formats are of the form
    ``host.port'' or ``network.port'' if a socket's address specifies a net-
    work but no specific host address. When known, the host and network
    addresses are displayed symbolically according to the databases hosts(5)
    and networks(5), respectively. If a symbolic name for an address is
    unknown, or if the -n option is specified, the address is printed numeri-
    cally, according to the address family.

    The interface display provides a table of cumulative statistics regarding
    packets transferred, errors, and collisions. The network addresses of
    the interface and the maximum transmission unit (``mtu'') are also dis-
    played.

    The routing table display indicates the available routes and their sta-
    tus. Each route consists of a destination host or network, and a gateway
    to use in forwarding packets. The flags field shows a collection of
    information about the route stored as binary choices. The individual
    flags are discussed in more detail in the route(8) and route(4) manual
    pages. The mapping between letters and flags is:

    1 RTF_PROTO1 Protocol specific routing flag #1
    2 RTF_PROTO2 Protocol specific routing flag #2
    3 RTF_PROTO3 Protocol specific routing flag #3
    B RTF_BLACKHOLE Just discard pkts (during updates)
    b RTF_BROADCAST The route represents a broadcast address
    C RTF_CLONING Generate new routes on use
    c RTF_PRCLONING Protocol-specified generate new routes on use
    D RTF_DYNAMIC Created dynamically (by redirect)
    G RTF_GATEWAY Destination requires forwarding by intermediary
    H RTF_HOST Host entry (net otherwise)
    L RTF_LLINFO Valid protocol to link address translation
    M RTF_MODIFIED Modified dynamically (by redirect)
    R RTF_REJECT Host or net unreachable
    S RTF_STATIC Manually added
    U RTF_UP Route usable
    W RTF_WASCLONED Route was generated as a result of cloning
    X RTF_XRESOLVE External daemon translates proto to link address

    Direct routes are created for each interface attached to the local host;
    the gateway field for such entries shows the address of the outgoing
    interface. The refcnt field gives the current number of active uses of
    the route. Connection oriented protocols normally hold on to a single
    route for the duration of a connection while connectionless protocols
    obtain a route while sending to the same destination. The use field pro-
    vides a count of the number of packets sent using that route. The inter-
    face entry indicates the network interface utilized for the route.

    When netstat is invoked with the -w option and a wait interval argument,
    it displays a running count of statistics related to network interfaces.
    An obsolescent version of this option used a numeric parameter with no
    option, and is currently supported for backward compatibility. By
    default, this display summarizes information for all interfaces. Infor-
    mation for a specific interface may be displayed with the -I option.

    The bpf(4) flags displayed when netstat is invoked with the -B option
    represents the underlying parameters of the bpf peer. Each flag is repre-
    sented as a single lower case letter. The mapping between the letters
    and flags in order of appearance are:

    p Set if listening promiscuously
    i BIOCIMMEDIATE has been set on the device
    f BIOCGHDRCMPLT status: source link addresses are being filled auto-
    matically
    s BIOCGSEESENT status: see packets originating locally and remotely on
    the interface.
    a Packet reception generates a signal
    l BIOCLOCK status: descriptor has been locked

    For more information about these flags, please refer to bpf(4).
     
  9. nat

    nat Well-Known Member

    Joined:
    Jan 16, 2003
    Messages:
    204
    Likes Received:
    0
    Trophy Points:
    16
    Same error for me since yesterday. Anyone find a solution to this?
     
  10. cPDan

    cPDan cPanel Staff
    Staff Member

    Joined:
    Mar 9, 2004
    Messages:
    711
    Likes Received:
    4
    Trophy Points:
    18
    Any ticket numbers?
     
  11. dashi

    dashi Member

    Joined:
    Aug 9, 2007
    Messages:
    8
    Likes Received:
    0
    Trophy Points:
    1
    Hi,

    after tomorrow update php session stop work
    also some time apache cant execute php files allow only save.
    please ASAP
     
  12. cPDan

    cPDan cPanel Staff
    Staff Member

    Joined:
    Mar 9, 2004
    Messages:
    711
    Likes Received:
    4
    Trophy Points:
    18
    Without a ticket I can only guess, but try:

    .php is configured
    Zend is resinstaleld or removed from php.ini
     
  13. nat

    nat Well-Known Member

    Joined:
    Jan 16, 2003
    Messages:
    204
    Likes Received:
    0
    Trophy Points:
    16
    /scripts/upcp --force on stable did not fix it. Upgrading to Release from Stable fixed it for me. Now 3rd party php scripts like fantastico are not obeying /usr/local/cpanel/3rdparty/lib/php.ini, they use /usr/local/lib/php.ini instead.
     
  14. cix

    cix Well-Known Member

    Joined:
    Nov 6, 2003
    Messages:
    74
    Likes Received:
    0
    Trophy Points:
    6
    My fantastico is not working:
    No input file specified. We use RELEASE version of cpanel.

    Could be the same related problem?
     
  15. nat

    nat Well-Known Member

    Joined:
    Jan 16, 2003
    Messages:
    204
    Likes Received:
    0
    Trophy Points:
    16
    dir -ald /tmp

    If it is not drwxrwxrwxt, change it back to drwxrwxrwxt

    chmod 1777 /tmp

    Also check to insure /tmp is not full.

    df -h



    My problem is not a fantastico problem as it happens with all 3rd party scripts.
     
Loading...

Share This Page