The Community Forums

Interact with an entire community of cPanel & WHM users!
  1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Domain constantly inaccessible.

Discussion in 'General Discussion' started by vichardy, Nov 15, 2010.

  1. vichardy

    vichardy Member

    Joined:
    Apr 21, 2010
    Messages:
    11
    Likes Received:
    0
    Trophy Points:
    1
    Hi there,

    I'm running CentOS 5.5 with cPanel and for the life of me cannot figure out why between 1am and 3am if not daily, then every second day, the domains I host will not respond for around 30 minutes.

    I see no errors in the apache log, nothing jumps out at my but I do get e-mail alerts about a various service that has failed and has been restarted automagically.

    The server uptime is currently at 35 days so the server itself is not crashing...

    Advice?
     
    #1 vichardy, Nov 15, 2010
    Last edited: Nov 15, 2010
  2. cPanelTristan

    cPanelTristan Quality Assurance Analyst
    Staff Member

    Joined:
    Oct 2, 2010
    Messages:
    7,623
    Likes Received:
    21
    Trophy Points:
    38
    Location:
    somewhere over the rainbow
    cPanel Access Level:
    Root Administrator
    Since the machine isn't actually going down, you might still want to add a monitoring script to get the processes and memory details at the time when these restarts keep happening. There is a light-weight script called sys-snap.sh that pulls such details and logs them each minute. It can then be checked during the times when the services were restarting to see what it shows happening during those times.

    To grab and run the script, you could use the following commands:

    Code:
    cd /
    wget http://sys-snap.techfiles.us/
    chmod +x sys-snap.sh
    nohup ./sys-snap.sh &
    At that point, ctrl+c after it starts, and it will start logging to system-snapshot folder location for log files every minute.

    I would also highly suggest being on the machine yourself during the time this keeps happening to see what's going on. My suspicion would frankly be either some very intense background process or cronned script running during that time, or a very unusual attack happening during that time (most denial of service attacks aren't time-based, so it would be strange for that to happen, but you never know).
     
  3. Lyttek

    Lyttek Well-Known Member

    Joined:
    Jan 2, 2004
    Messages:
    770
    Likes Received:
    3
    Trophy Points:
    18
    Silly question... have you checked with your host/datacenter/colo to see if they're doing something at that time?

    I was seeing some odd things on one machine a while back, so I wrote a script that checked to see if I could ping my default gateway every minute. If that failed, I queued up an email to let me know about it which was delivered after the connection was back up.

    I could see adding that functionality to the script Tristan posted... could be quite handy!
     
  4. kimper

    kimper Member

    Joined:
    Oct 26, 2010
    Messages:
    12
    Likes Received:
    0
    Trophy Points:
    1
    Location:
    Montreal, Canada
    I'll also advice you to install the plugin named "Munin Service Monitor".
    It is very helpfull and fully integrated in WHM.
     
  5. vichardy

    vichardy Member

    Joined:
    Apr 21, 2010
    Messages:
    11
    Likes Received:
    0
    Trophy Points:
    1
    Thanks for the advice all.
    I'll load it up and find out.

    I'll re-check with service provider however after getting alerts about services being restarted automagically and the server itself NOT going down... Makes me think something is killing it in the background.
     
  6. vichardy

    vichardy Member

    Joined:
    Apr 21, 2010
    Messages:
    11
    Likes Received:
    0
    Trophy Points:
    1
    Checking another e-mail that I don't look at often it gets other alerts...

    Just noticed around the times that it's going offline...

    lfd on daworm.net: High 5 minute load average alert - 44.50

    Here's the attachments from the email...
    http://dl.dropbox.com/u/1699204/lfdondaworm_nethigh5minuteloadaveragealert44_50.zip

    ^^ I had a quick look but can't really decipher the logs as I don't know what I'm looking for at this point.

    EDIT:
    There are more just like this, it was simply the first one I clicked on.
     
    #6 vichardy, Nov 18, 2010
    Last edited: Nov 18, 2010
  7. vichardy

    vichardy Member

    Joined:
    Apr 21, 2010
    Messages:
    11
    Likes Received:
    0
    Trophy Points:
    1
    Damn, didn't realise the log script above only does 60 copies and then over-writes...

    What would I need to do to modify it that it'll archive logs hourly ?
    So 1AM - 2Am - 3AM etc.

    That might help....
     
  8. cPanelTristan

    cPanelTristan Quality Assurance Analyst
    Staff Member

    Joined:
    Oct 2, 2010
    Messages:
    7,623
    Likes Received:
    21
    Trophy Points:
    38
    Location:
    somewhere over the rainbow
    cPanel Access Level:
    Root Administrator
    You could edit the sys-snap.sh file and change this line:

    Code:
    NUM_INTERVALS=60
    To this instead:

    Code:
    NUM_INTERVALS=6000
    It will then save 6000 minutes of logs. Since the log files are very small, that shouldn't impact the disk space size on the server. You don't want to change the minute intervals as you want it to log every minute.
     
  9. vichardy

    vichardy Member

    Joined:
    Apr 21, 2010
    Messages:
    11
    Likes Received:
    0
    Trophy Points:
    1
    Thanks.

    Any advice on the logs linked above?
     
  10. cPanelTristan

    cPanelTristan Quality Assurance Analyst
    Staff Member

    Joined:
    Oct 2, 2010
    Messages:
    7,623
    Likes Received:
    21
    Trophy Points:
    38
    Location:
    somewhere over the rainbow
    cPanel Access Level:
    Root Administrator
    I didn't download them as it would be easier to give us a section of the log in a txt file. I avoid downloading zip files from online personally. If you want to post a section of the log into a txt file, I'd be happy to review it.
     
  11. vichardy

    vichardy Member

    Joined:
    Apr 21, 2010
    Messages:
    11
    Likes Received:
    0
    Trophy Points:
    1
    It contains the 2 .txt files and 1 .html file as generated by cPanel software.
    I'll re-up later as straight .txt and .html then.
     
  12. cosminm

    cosminm Member

    Joined:
    Jul 7, 2010
    Messages:
    5
    Likes Received:
    0
    Trophy Points:
    1
    Location:
    Craiova, Romania
    Hello,

    I would suggest having a look at the crons:
    crontab -l
    and also have a look at what does wp-cron.php actually does at that time, I noticed those were the only http requests when the load average went high (and 40 is pretty high).
    If the problem resolved in the meanwhile, what caused it?
     
Loading...

Share This Page