The Community Forums

Interact with an entire community of cPanel & WHM users!
  1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

How does cpanellogd work?

Discussion in 'General Discussion' started by DuncanMcLord, Nov 17, 2002.

  1. DuncanMcLord

    DuncanMcLord Active Member

    Joined:
    Dec 12, 2001
    Messages:
    31
    Likes Received:
    0
    Trophy Points:
    6
    Hi,

    just a quick question, since i couldn't find any info about this, but how does cpanellogd work?
    I mean, does it work at runtime, so if a user wants to access his stats it parses those?
    Or does it wait for low-load times and then parse one log after the other?

    I am asking, because i would like to know if i could define the time when awstats get updated!

    Thanks,
    Duncan
     
  2. rich2

    rich2 Well-Known Member

    Joined:
    Apr 9, 2002
    Messages:
    64
    Likes Received:
    0
    Trophy Points:
    6
    *bump* this needs an answer, i'd like to know as well.
     
  3. golloween

    golloween Member

    Joined:
    Mar 5, 2003
    Messages:
    20
    Likes Received:
    0
    Trophy Points:
    1
    Does anybody know when exactly cpanellogd runs AWStats and Webalizer to process logs?
     
  4. pingo

    pingo Well-Known Member

    Joined:
    Nov 16, 2002
    Messages:
    430
    Likes Received:
    0
    Trophy Points:
    16
    Does cpanellogd run AWStats and Webalizer? - I wondered why process cpanellogd user root used more than 24 of the cpu resources for long period. While this is happening the server load is very high. So it is the stats and probably because there are more accounts now on the server. - Does anyone know if something can be done about this - I would also like to know if the time stats are processed could be changed?

    John
     
  5. pingo

    pingo Well-Known Member

    Joined:
    Nov 16, 2002
    Messages:
    430
    Likes Received:
    0
    Trophy Points:
    16
    I wonder why cpanel.net are not providing a faq for simple questions being asked again and again but not being answered here. :( - or does someone else care? :)

    John
     
  6. MarlboroMan

    MarlboroMan Well-Known Member

    Joined:
    Dec 7, 2001
    Messages:
    64
    Likes Received:
    0
    Trophy Points:
    6
    cpanellogd handles log processing and license updating (the main loop runs stats every other time it loops, and updates the license file every 25 loops)

    Every time through the main loop, it checks if the following files are over 5MB and rotates them:
    /usr/local/apache/logs/suexec_log,
    /usr/local/apache/access_log,
    /usr/local/apache/logs/error_log,
    /etc/httpd/logs/error_log

    If the logs are over 5MB and rotated - it restarts Apache with a SIGUSR1

    During the logrun loops, it first does something with FTP & transfer logs (haven't looked too deeply into this), then goes through /var/cpanel/users and runs logs and updates bandwidth usage for every user, catching subdomains and the like as each user is ran.

    After each loop through main, the daemon sleeps for 5 minutes.


    That's pretty much it for the daemon part of cpanellogd.


    It can also be called on a per-user basis - simply run /usr/local/cpanel/cpanellogd <username> to have it just run the logs for a specific user and exit.


    ::edit:: Slight correction
    In case you were wondering, cpanellogd doesn't actually get to run stats. it just collects information needed to run stats on all domains, and passes it to /usr/local/cpanel/bin/logrunner - which is an executable file, not a perl file, so I can't tell you what goes on in there. I'd imagine it just checks if stats have been run on the domains it is passed in the last 24 hours, and if not, spawns a child to run stats for that domain.

    All the load-checking and etc. have moved out of cpanellogd - I have to assume that it's in logrunner now.

    ::/edit::
     
    #6 MarlboroMan, Mar 7, 2003
    Last edited: Mar 7, 2003
  7. golloween

    golloween Member

    Joined:
    Mar 5, 2003
    Messages:
    20
    Likes Received:
    0
    Trophy Points:
    1
    MarlboroMan, thanks for the detailed answer! What's the difference between the "main loop" and the "logrun loop"?
     
  8. pingo

    pingo Well-Known Member

    Joined:
    Nov 16, 2002
    Messages:
    430
    Likes Received:
    0
    Trophy Points:
    16
    Yes, thanks alot for that explanation :)

    John
     
  9. MarlboroMan

    MarlboroMan Well-Known Member

    Joined:
    Dec 7, 2001
    Messages:
    64
    Likes Received:
    0
    Trophy Points:
    6
    cpanellogd has an internal counter - it increases by 1 each time the main loop runs. Each time through the main loop, if the counter is even, it runs logs. If it's odd, it doesn't run logs. If its divisible by 25, it updates your license file.
     
  10. golloween

    golloween Member

    Joined:
    Mar 5, 2003
    Messages:
    20
    Likes Received:
    0
    Trophy Points:
    1
    Ok, it sleeps for 5 minutes after each iteration of the main loop and runs logs on every 2nd pass. If this is correct, the time between running logs should be 10 minutes.

    But how these 10 minutes are related to the setting in WHM under the Tweak Settings menu: "Number of days between processing log files etc..."? Or this setting is handled by the Logrunner that simply skips loops if the time passed since the last logrun is lesser than the value of that setting?
     
  11. MarlboroMan

    MarlboroMan Well-Known Member

    Joined:
    Dec 7, 2001
    Messages:
    64
    Likes Received:
    0
    Trophy Points:
    6
    I would assume so - logrunner gets passed information about running logs for all domains every 10 minutes - but only runs logs after system time is 24 hours past that of the lastrun file in the stats directory.
     
  12. Curious Too

    Curious Too Well-Known Member

    Joined:
    Aug 31, 2001
    Messages:
    427
    Likes Received:
    0
    Trophy Points:
    16
    cPanel Access Level:
    Root Administrator
    Not on my machines. I have to manually delete the access_log, error_log, suexec_log and ssl_engine_log. I checked one machine tonight and saw:

    %cpu %mem Time command
    28.9 1.9 48:38 cpanellogd - updating logs

    When it "finished" I checked my logs. The error_log was nearly 15M, the suexec_log was 29M.
     
  13. MarlboroMan

    MarlboroMan Well-Known Member

    Joined:
    Dec 7, 2001
    Messages:
    64
    Likes Received:
    0
    Trophy Points:
    6
    Sorry - I read it wrong - if the filesize is over 500MB, it trims is to 5MB
     
  14. Curious Too

    Curious Too Well-Known Member

    Joined:
    Aug 31, 2001
    Messages:
    427
    Likes Received:
    0
    Trophy Points:
    16
    cPanel Access Level:
    Root Administrator
    What are the effects on log processing if you kill cpanellogd and restart it later?
     
  15. MarlboroMan

    MarlboroMan Well-Known Member

    Joined:
    Dec 7, 2001
    Messages:
    64
    Likes Received:
    0
    Trophy Points:
    6
    Logs don't get processed, bandwidth totals aren't updated, etc.

    I don't remember if CPanel runs a user's webalizer, etc stats if they request them through their CPanel and they haven't been run in the last 24 hours. It used to be that way, but I woulnd't put money on it anymore.
     
  16. Curious Too

    Curious Too Well-Known Member

    Joined:
    Aug 31, 2001
    Messages:
    427
    Likes Received:
    0
    Trophy Points:
    16
    cPanel Access Level:
    Root Administrator
    I've killed and restarted cpanellogd and the stats, logs etc do get updated. I haven't checked to see if there is any missing data but the stats do get updated.

    cpanellogd is a resource hog, not to mention the stats_log is growing so large so quickly now that it has to be deleted regualarly, and the only way to get the correct disk useage after deleting the stats_log is to restart cpanellogd.
     
  17. H2Hosting.com

    H2Hosting.com Well-Known Member

    Joined:
    Sep 4, 2001
    Messages:
    192
    Likes Received:
    0
    Trophy Points:
    16
    I think that something is wrong with cpanellogd after last updates :mad: as it takes too much resources of the server. We have dual P3-1130Mhz and 1000 1.5GbRAM boxes and cpanellogd use almost 20% of CPU!

    6580 root 20 19 117M 114M 114M R N 18.6 11.3 490:04 cpanellogd - updating logs

    26580 root 20 19 117M 114M 102M R N 25.3 11.3 490:49 cpanellogd - updating logs

    26580 root 20 19 117M 114M 25836 R N 22.3 11.3 490:51 cpanellogd - updating logs

    Cpanel programmers must fix it!!!
     
  18. FWC

    FWC Well-Known Member

    Joined:
    May 13, 2002
    Messages:
    354
    Likes Received:
    0
    Trophy Points:
    16
    Location:
    Ontario, Canada
    I've been seeing this since Cpanel updated the night of March 10th, too. It looks like it may be taking a long time to run rrdtool across all the domains.
     
  19. Brad

    Brad Well-Known Member

    Joined:
    Aug 16, 2001
    Messages:
    231
    Likes Received:
    0
    Trophy Points:
    16
    Ours has been having the same problem too.. Hight resource useage.
     
  20. Sash

    Sash Well-Known Member

    Joined:
    Feb 18, 2003
    Messages:
    252
    Likes Received:
    0
    Trophy Points:
    16
    Ditto.....

    Mike
     
Loading...

Share This Page