The Community Forums

Interact with an entire community of cPanel & WHM users!
  1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Reliability of Raw Logs?

Discussion in 'General Discussion' started by Dr_Test, Jul 6, 2008.

  1. Dr_Test

    Dr_Test Registered

    Joined:
    Jul 6, 2008
    Messages:
    3
    Likes Received:
    0
    Trophy Points:
    0
    Hi all. My first post. :)

    I essentially signed up to look into a suspicion that I've been having recently, due to two issues (as follows):

    1: The Yahoo bot recently used up 20 gigs of bandwidth in one month, on my tiny site, and the only thing I could see it doing that would use THAT much bandwidth is downloading my huge collection of enormous backup RAR files. Only trouble is, there is no mention of this activity anywhere in my Raw logs, so I'm confused.

    2: When scouring my logs, I've noticed that many visits to my front page show someone downloading my header.htm, but not index.htm (huh?), or the other way around... it'll show them getting the index.htm, but no mention whatsoever of the header.htm.

    My *theory* is that maybe the raw logs don't report sudden bursts of activity, such as one user calling for multiple files at one. Does that make sense? Am I missing something here?

    I'm also suspecting a certain user of bombing my site with an app like UpdatePatrol or NeoDownloader to suck down my updated files, but of course I can't pinpoint this in my logs. Would it make any sense that this activity would be hidden if whatever app they use calls for tons and tons of my site's files/dirs at the same time? ...'Trying to wrap my head around this one.

    Thanks! :)
     
    #1 Dr_Test, Jul 6, 2008
    Last edited: Jul 6, 2008
  2. Infopro

    Infopro cPanel Sr. Product Evangelist
    Staff Member

    Joined:
    May 20, 2003
    Messages:
    14,447
    Likes Received:
    195
    Trophy Points:
    63
    Location:
    Pennsylvania
    cPanel Access Level:
    Root Administrator
    Twitter:
    You might find something in awstats to go on. Worth a closer look.
     
  3. Dr_Test

    Dr_Test Registered

    Joined:
    Jul 6, 2008
    Messages:
    3
    Likes Received:
    0
    Trophy Points:
    0
    Uhh, not sure what you mean. (btw, I have AWStats running on my computer... seems pretty generic though, as far as I can tell after 5 min of messing with it. Is there something aspect of it in particular I should look at?)

    -

    Btw, I DID manage to figure out for sure that my raw logs are not logging everything. My site has had 300 visits so far this month, so I did a text-search in the log for one of the images on the front page, and it only came up about 30 times. Also, I examined some visits carefully, and noticed that not all of the images on index.htm were reported as downloaded. Usually they were, but about 1/4 of the visitors are reported as downloading a PORTION of the images (which are a combined total of about 20k. It's a fast-loading page).
     
  4. chirpy

    chirpy Well-Known Member

    Joined:
    Jun 15, 2002
    Messages:
    13,475
    Likes Received:
    20
    Trophy Points:
    38
    Location:
    Go on, have a guess
    AFAIK, the raw logs are completely reliable, I've never seen or heard of apache not logging (indeed it will simply slow to a crawl doing this). It's much more likely that whatever is retrieving the page simply isn't downloading the images. Things like web crawlers and trawlers, some browser plugins and indeed DOS scripts probably wouldn't bother.
     
Loading...

Share This Page