The Community Forums

Interact with an entire community of cPanel & WHM users!
  1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

/var/log/messages, etc suddenly empty!?

Discussion in 'General Discussion' started by bataylor, Jun 15, 2004.

  1. bataylor

    bataylor Member

    Joined:
    Oct 4, 2003
    Messages:
    6
    Likes Received:
    0
    Trophy Points:
    1
    Location:
    Vancouver, BC, Canada
    Hi everyone,

    I'm experiencing some strange issues with /var/log/messages, boot.log, cron, exim_paniclog, maillog, secure, and spooler and /var/log/httpd/* -- that is, they're all empty (their size is 0). And what's interesting, is that this all happened at the same time -- 4:44am this past Sunday.

    I've restarted syslog, with no luck. Nothing is being logged correctly right now. And I'm starting to worry. I'm not sure how to make it start working properly again, PLUS the biggest concern, I don't know what caused this! If someone has broken into the system, there aren't any completely obvious signs. I've run chkrootkit, scanned for trojans, and the whole lot, and everything seems fine to me.

    Does anyone have any suggestions?

    This is really not a good situation...


    Thanks.
    Brett
     
  2. ialex03

    ialex03 Well-Known Member

    Joined:
    May 15, 2003
    Messages:
    56
    Likes Received:
    0
    Trophy Points:
    6
    Hi,

    got your message about this problem.
    But I am sorry, I don't remember how I fixed it. it was so long time ago ..

    As far as I remember that server was hacked and I reinstalled many RPMs. I think reinstalling sysklogd solved the problem.

    So my receipt is:
    1. reinstall sysklogd RPM
    2. make sure "syslog" is set to ON on your autostart ("ntsysv" command).
    3. reboot


    Of course I advise you to check the server by chkrootkit (http://www.chkrootkit.org/)

    Hope this helps.
     
  3. coastinc

    coastinc Well-Known Member

    Joined:
    Mar 13, 2002
    Messages:
    159
    Likes Received:
    0
    Trophy Points:
    16
    The first quick and easy thing to do is reboot the server, then see what's going on.

    If you need help with this, just drop us an e-mail.
     
  4. bataylor

    bataylor Member

    Joined:
    Oct 4, 2003
    Messages:
    6
    Likes Received:
    0
    Trophy Points:
    1
    Location:
    Vancouver, BC, Canada
    Still no luck with this. Trying to get support from my service provider (I'm on a VDS host) with no luck. I had it restarted, which didn't help. And reinstalling sysyklogd didn't help either.

    Any other ideas? Help!


    Brett
     
  5. coastinc

    coastinc Well-Known Member

    Joined:
    Mar 13, 2002
    Messages:
    159
    Likes Received:
    0
    Trophy Points:
    16
    Restarting VDS doesn't reboot the server really, have your provider check into it and try to give the server a fresh reboot.
     
  6. bataylor

    bataylor Member

    Joined:
    Oct 4, 2003
    Messages:
    6
    Likes Received:
    0
    Trophy Points:
    1
    Location:
    Vancouver, BC, Canada
    I don't expect my provider to restart the entire server on my request like that. Who knows how many clients it effects -- I don't think this is something they would readily do.
     
  7. coastinc

    coastinc Well-Known Member

    Joined:
    Mar 13, 2002
    Messages:
    159
    Likes Received:
    0
    Trophy Points:
    16
    I agree, however, if the technician is unable to diagnose the problem, you may be able to convince him to do a quick reboot.

    If you would like us to look into it for you, we'd be more than glad to do so, just drop us an e-mail.
     
  8. Justin_O

    Justin_O Registered

    Joined:
    May 13, 2004
    Messages:
    2
    Likes Received:
    0
    Trophy Points:
    1
    Hey, I have the same exact problem, only with things my script, Blah.pl is writting to (http://www.eblah.com/cgi-bin/bb/Blah.pl).

    It ONLY seems to happen on Friday/Saturday mornings at 6:04 AM (this is the time the server stamp is on the written file, it's at 1:04 AM CST). I've looked as far as I can look into this, and I have found no explaination. It recently started (just last week, June 11th, as far as I remember), and now if ANY file is written to durring that time the file is truncated to 0 (I *guess* this only happens from 1:03 AM to sometime around 1:04 AM). I've been told that, durring that time, the Google bot was on the site (which doesn't matter). I found out that a file had been edited at 1:04 AM, but the other file, chat.msg was truncated by the time the script got to that file.

    It did the same thing last week too. Also, don't say this is some faulty perl code, because it happened to a site I was hosting which wrote to a file using PHP. I'd really like answers on this. I do not have a cron list, or anything (I only have limited access to the WHM/cPanel).

    One further thing, if you notice http://www.eblah.com/cpanel gives a 500 error. That's also happened within the last 2 weeks or so. I don't know if cPanel was "upgraded", and then I started getting these errors or what. I know for a fact we couldn't have been hacked. So what could it have been?!


    - Justin
     
  9. Justin_O

    Justin_O Registered

    Joined:
    May 13, 2004
    Messages:
    2
    Likes Received:
    0
    Trophy Points:
    1
    I have found out the cause of this. I'm guessing some cron job is running at around 1:04 AM CST on the server, and does SOMETHING to the webspace quota for about 2 minutes, then goes back to normal. I know it's a quota issue (possibly a bug with cPanel), because of this:

    A few minutes later, I uploaded the file perfectly fine. I still have 90+ MB's of space on my account, so I have plently of space. So, I believe it's either a cPanel bug, or something that someone should explain. ;)


    - Justin
     
  10. Dillard

    Dillard Well-Known Member

    Joined:
    Feb 26, 2003
    Messages:
    114
    Likes Received:
    0
    Trophy Points:
    16
    Location:
    The Netherlands
    Sounds like the server (perhaps not your VDS but the mainbox) is hacked. As far as I recall in a VDS situation, you really using the syslogd from the system.

    Only solution: get an engineer to start looking at this problem (ie. does the other VDS have the same prob) and start downloading your backup's (just to be sure)

    Goodluck!
     
Loading...

Share This Page