The Community Forums

Interact with an entire community of cPanel & WHM users!
  1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

webalizer reverse lookups borked on RedHat 9

Discussion in 'General Discussion' started by rbrown, Sep 23, 2003.

  1. rbrown

    rbrown Active Member

    Joined:
    Jun 11, 2003
    Messages:
    35
    Likes Received:
    0
    Trophy Points:
    6
    The way cPanel compiled webalizer does not support DNS on RedHat 9 distribution. I know it used to work because we had cPanel on RedHat 7.3 and the reverse lookups were computing fine. Now, all we ever get is a 4096 byte dns_cache.db stub file.

    I downloaded webalizer myself and tried to compile it and got this:

    ./configure --enable-dns
    [...]
    checking for dbopen... no
    checking for library containing dbopen... no
    configure: warning: You must have a V1.85 compatable DB library!
    configure: warning: DNS lookup code will be disabled...
    [...]

    It built a binary, but the behavior of this binary behaved exactly like the one installed in /usr/local/cpanel/3rdparty/bin/webalizer

    Then I recompiled it using the following options:

    ./configure --enable-dns --with-dblib=/lib

    and then the reverse lookups magically started working again. I copied over the good version of webalizer into /usr/local/cpanel/3rdparty/bin/. But when /scripts/upcp is run, the rsync bricks over my good webalizer with the old borked version without DNS support.

    Is there any way to force cPanel to use the good webalizer? Or can it be recompiled with that extra "--with-dblib=/lib" option on the rsync.cpanel.net end for the redhat9 distro? I'm not sure what other options were used, but maybe the --enable-static flag was used as well.

    Ideas?

    cPanel.net Support Ticket Number:
     
  2. rbrown

    rbrown Active Member

    Joined:
    Jun 11, 2003
    Messages:
    35
    Likes Received:
    0
    Trophy Points:
    6
    Okay, I found something called buildwebalizer which may have been used to create this bug. It assumes the linux version contains /(\d*\.\d*)/ which of course will NEVER match for "RedHat 9" since there is no "." anywhere.

    (I wonder how many other thing RedHat broke when they called it version "9" instead of "9.0" like how all the older versions used to be. :-/)

    I've attached the patch that repairs this cPanel bug so it will work on a RedHat 9 system now too.

    cPanel.net Support Ticket Number:
     

    Attached Files:

  3. rbrown

    rbrown Active Member

    Joined:
    Jun 11, 2003
    Messages:
    35
    Likes Received:
    0
    Trophy Points:
    6
    buildwebalizer fixes

    Support Ticket.
    Better patch.

    cPanel.net Support Ticket Number: 24845
     

    Attached Files:

  4. GuiPos

    GuiPos Well-Known Member

    Joined:
    Jul 9, 2003
    Messages:
    59
    Likes Received:
    0
    Trophy Points:
    6
    Location:
    Indonesia / Germany
    Re: buildwebalizer fixes

    Hello rbrown !

    thank you for pointing this out.
    just a simple question - is this bug responsible for :
    the 100% ?unknown Countries in the webalizer stats ?
    [ i looking for a fix to get the countries working in the webalizer ]

    if so would you please post a simple way to go through ?

    i am a newbie in server administration and need some little helping hands for this.

    thank you very much.
     
  5. rbrown

    rbrown Active Member

    Joined:
    Jun 11, 2003
    Messages:
    35
    Likes Received:
    0
    Trophy Points:
    6
    Re: webalizer 100% Unresolved Countries - RedHat9

    This cpanel misconfiguration is exactly the cause for getting the 100% Unresolved Countries in webalizer. But I've only verified this bug under RedHat 9. Do you have RedHat 7.3 or some other version?

    How to buttwag around this bug is to simply use the stock version of webalizer. Since cPanel statically compiled webalier using the --enable-static option, it just will not work. Removing this option makes it work. Until cPanel has a chance to recompile, your system will remain broken. I expect it will be another three to six months before cPanel will have this fixed.

    To fix your system now (REDHAT9 ONLY!), run this sequence:

    rpm --nodeps -Uvh ftp://rpmfind.net/linux/redhat/9/en/os/i386/RedHat/RPMS/webalizer-2.01_10-11.i386.rpm
    /bin/cp -p /usr/bin/webalizer /usr/local/cpanel/3rdparty/bin/.
    rpm -e webalizer


    You may need to run "/scripts/fixwebalizer username" to clear the old corrupted /home/username/tmp/webalizer/dns_cache.db file and to regenerate the stats. But then you will see the countries start working again.

    You may need to manually run this repair sequence after every time you update cPanel or just put these lines in your /scripts/postupcp until cPanel has it repaired on their end.

    cPanel.net Support Ticket Number: 24845
     
  6. sneader

    sneader Well-Known Member

    Joined:
    Aug 21, 2003
    Messages:
    1,126
    Likes Received:
    21
    Trophy Points:
    38
    Location:
    La Crosse, WI
    cPanel Access Level:
    Root Administrator
    rbrown, thank you for all the detective work to figure this out! Bravo !!

    Now, it seems like a fairly easy fix, why will it take 3 to 6 months for cPanel to fix this bug? What can I (or we) do to encourage a resolution sooner?

    - Scott
     
  7. GuiPos

    GuiPos Well-Known Member

    Joined:
    Jul 9, 2003
    Messages:
    59
    Likes Received:
    0
    Trophy Points:
    6
    Location:
    Indonesia / Germany
    Re: Re: webalizer 100% Unresolved Countries - RedHat9

    Thank you rbrown !

    its work fine by me too !

    just a question more - what should i write in the /scripts/postupcp ???

    like this: /scripts/fixwebalizer username and this for all clients ?
     
  8. rbrown

    rbrown Active Member

    Joined:
    Jun 11, 2003
    Messages:
    35
    Likes Received:
    0
    Trophy Points:
    6
    Maybe only two months now. The reason it will tend to take a long time is because cPanel does not like to use "rpm" very much. They'd rather just compile everything from source manually on each server. Plus, the cPanel guys hardly ever find the time to read anything in these forumss anyway.
     
  9. rbrown

    rbrown Active Member

    Joined:
    Jun 11, 2003
    Messages:
    35
    Likes Received:
    0
    Trophy Points:
    6
    Re: webalizer 100% Unresolved Countries - RedHat9

    You're welcome.

    No, never put any client username in the postupcp because the stats will never break as long as you always have the good webalizer installed.

    Example:

    cat <<FIXCPANEL > /scripts/postupcp
    #!/bin/sh

    rpm --nodeps -Uvh ftp://rpmfind.net/linux/redhat/9/en/os/i386/RedHat/RPMS/webalizer-2.01_10-11.i386.rpm
    /bin/cp -p /usr/bin/webalizer /usr/local/cpanel/3rdparty/bin/.
    rpm -e webalizer
    FIXCPANEL
    chmod 755 /scripts/postupcp


    Then you never have to worry about broken stats again whether you update cpanel or not.

    Good luck!

    Rob Brown
     
  10. dev.null

    dev.null Well-Known Member

    Joined:
    May 27, 2003
    Messages:
    75
    Likes Received:
    1
    Trophy Points:
    6
    Thanks for the great fix rbrown!

    I've been looking for this for several months.

    now I've ran into another problem as a result. Seems that the new webalizer doesn't like the old webalizer's history.

    When I run

    /scripts/runweblogs <name>

    I see this in the output:

    Error: Unable to save current run data
    Error: Unable to write history file /var/lib/webalizer/webalizer.hist

    Here's two other things I notice:

    1. The history is gone from the webalizer html pages. The only data in there is the data from what's left in the apache log. This is *real* bad.
    2. The /home/<name>/tmp/webalizer/webalizer.current and *.hist files are not changed, they still have yesterday's time and date on them. So it looks like webalizer can't read them and doesn't update them. I've checked the file permissions, they are the same as all the other files in the webalizer directory (owner:group set to the site's user ID).

    Any ideas why it's not keeping all the old history?
     
  11. dev.null

    dev.null Well-Known Member

    Joined:
    May 27, 2003
    Messages:
    75
    Likes Received:
    1
    Trophy Points:
    6
    UPDATE/FIX

    OK, when I first installed the rpm webalizer I was checking the stats page in for January on my own domain. I kept tweeking the /home/domain/webalizer.conf file, running `runweblogs domain`, and then refreshing the page in the browser to see the .conf changes take place. I had it set exactly like I wanted.

    I know it was working correctly back then with all the right stats, and it was finally doing the dns resolve and country pie graph (I finally got to see what it lookes like).

    All of the sudden it starts giving me a report on *another domain*, and only what was currently in the apache logs on top of that! I thought I had lost all my history (and my mind!).

    Well after doing the command line webalizer myself several times and reading the .conf instructions and the man page again and again I decided to leave nothing out. I went through every command and figured out what webalizer would do step by step. Then I realized in the man page it was telling me it would always try to use /etc/webalizer.conf if it were there. I checked, sure enough it was there. And it had it's HistoryName and IncrementalName pointing to the /var/lib/webalizer directory.

    I don't know when it slipped in, but whenever it did it was smooth. It had to have slipped in after I had already successfully ran webalizer on my own .conf file and saw my web stats the way I wanted them. The funny thing is I never did anthing (that I know of) that would have made this .conf file.

    So, after deleting the bad .conf, rm'ing the /var/lib/webalizer directory, and putting a webalizer.conf in each /home/<name> directory that specified their history and current webalizer files, a /scripts/runweblogs re-created all my stats the way I wanted them. Minus one thing - the dns names it had already looked up were now gone and there are only IPs in their place. Only new hits are getting their DNS names looked up. A minor glitch, at least I've got my real stats back.
     
    #11 dev.null, Jan 17, 2004
    Last edited: Jan 17, 2004
  12. rbrown

    rbrown Active Member

    Joined:
    Jun 11, 2003
    Messages:
    35
    Likes Received:
    0
    Trophy Points:
    6
    100% Unresolved Countries - RedHat 9

    Yes, I remember having problems like this too. That's why it is so important to remove the webalizer rpm. As shown in the example:

    cat <<FIXCPANEL > /scripts/postupcp
    #!/bin/sh

    rpm --nodeps -Uvh ftp://rpmfind.net/linux/redhat/9/en/os/i386/RedHat/RPMS/webalizer-2.01_10-11.i386.rpm
    /bin/cp -p /usr/bin/webalizer /usr/local/cpanel/3rdparty/bin/.
    rpm -e webalizer
    FIXCPANEL
    chmod 755 /scripts/postupcp

    Notice line 3 (rpm -e) which removes it. I've notices that cPanel usually removes the webalizer rpm whenever upgrading with /scripts/upcp for some reason but not always. Installing the rpm creates the evil /etc/webalizer.conf, but you really just needed the /usr/bin/webalizer. Once you copy that, you're safe to purge the rpm, which removes the /etc/webalizer.conf too.

    You should NOT need to configure all those /home/*/webalizer.conf files unless the users wish to customize their own stats, (like increasing the number of refers or something).

    Sorry for not explaining that before.

    Hopefully, cPanel can get their bug fixed soon so we all won't have to suffer with all these weird dances to workaround this issue anymore. But I still don't think they have a RedHat 9 box to test this on yet.
     
  13. silvernetuk

    silvernetuk Well-Known Member

    Joined:
    Sep 2, 2002
    Messages:
    311
    Likes Received:
    0
    Trophy Points:
    16
    Location:
    United Kingdom
    Hi,

    Does this also fix the problem with the IP's not resolving and just having the ip's in the stats that visited your site.

    Regards,
    Garry
     
  14. rbrown

    rbrown Active Member

    Joined:
    Jun 11, 2003
    Messages:
    35
    Likes Received:
    0
    Trophy Points:
    6
    Garry:

    Yes, the entire purpose of creating this postupcp script is to make sure the IPs resolve to reverse lookups on a RedHat 9 Linux cPanel box, (which is where the country stats come from too).

    --
    Rob Brown
     
  15. dev.null

    dev.null Well-Known Member

    Joined:
    May 27, 2003
    Messages:
    75
    Likes Received:
    1
    Trophy Points:
    6
    :) Rob Brown you are a great help - THANKS! :)

    Do you know how I can get webalizer to go back and do all the IPs from the hits it has already counted? It's only looking up the IPs for the hits that are new.

    I've thought about running webalizer and doing -o to another empty directory and the dns_cache.db to a file that doesn't yet exist, then copy that .db over top the .db in my webaizer directory. I think that it would not have any history to go by and would look up all the IPs and put them in the db, then the next real webalizer that runs would see the .db had the entries for the older IPs and would use the DNS names (I hope).

    Thanks again.
     
    #15 dev.null, Jan 18, 2004
    Last edited: Jan 18, 2004
  16. rbrown

    rbrown Active Member

    Joined:
    Jun 11, 2003
    Messages:
    35
    Likes Received:
    0
    Trophy Points:
    6
    Re: Rob Brown you are a great help - THANKS!

    Nah, that's too much work.

    Just delete the dns_cache.db file of the site that ran with the old webalizer. It will regenerate automatically. If you still have the log files, you can delete the webalizer.hist and webalizer.current files too and it will create everything from scratch without corruption.

    There is a program called /scripts/fixwebalizer that does all this for you and rebuilds the stats, but you'll lose the old stats that were generated from log files that have since been deleted (/usr/local/apache/domlogs/site.com). But from now on, it will never get corrupted again.

    -- Rob Brown
     
  17. dev.null

    dev.null Well-Known Member

    Joined:
    May 27, 2003
    Messages:
    75
    Likes Received:
    1
    Trophy Points:
    6
    Re: Re: Rob Brown you are a great help - THANKS!

    Did that already. It doesn't build the past days this month that it has already done (as noted in the hist file).

    Even worse, as some of the same IPs come in I see two entries on the page, one with all the IP hits and one with all the DNS lookup of that IP hits.

    I still have the log from the end of dec to now, but none of the previous and I really don't want to lose any past data, so deleting the webalizer.* files isn't an option.

    That's definately not going to work, I'd loose way to much past history.

    Thanks anyway.
     
    #17 dev.null, Jan 20, 2004
    Last edited: Jan 21, 2004
  18. dev.null

    dev.null Well-Known Member

    Joined:
    May 27, 2003
    Messages:
    75
    Likes Received:
    1
    Trophy Points:
    6
    OK, here's something that does work.

    I delete the dns_cache.db and the webalizer.current, re-run stats.

    And now my past data is there (because the .hist file is still around), and all of this month's data is w/ the dns is there because I still had this month's log file.

    By deleting the .current file I tricked webalizer into rebuilding the entire month again.

    I hadn't thought of doing that file alone, I had thought of doing hist and current together which wasn't something I wanted to do.

    Again, thanks to all your help.
     
  19. rbrown

    rbrown Active Member

    Joined:
    Jun 11, 2003
    Messages:
    35
    Likes Received:
    0
    Trophy Points:
    6
    webalizer reverse lookups borked on Redhat 9 is fixed in 8.8.0-EDGE_45

    It only took 3 months for those cPanel guys to get this fixed. I was expecting longer, but I guess those guys are really on the ball. SUPER!

    I just upgraded cPanel to 8.8.0-EDGE_45 and then did

    find /home/*/tmp -name dns_cache.db|xargs -i echo rm {}|sh

    to flush all the old corrupted cache files. Now everything suddenly started showing the real Countries and the reverse lookups are working again under the webalizer stats. YIPPY!

    (You should remove that /scripts/postupcp BEFORE upgrading if you ever created this script because of this issue because you don't need that hack anymore.)

    cPanel.net Support Ticket Number: 24845
     
  20. nettigritty

    nettigritty Well-Known Member
    PartnerNOC

    Joined:
    Jan 21, 2004
    Messages:
    194
    Likes Received:
    0
    Trophy Points:
    16
    Location:
    Bangalore, India
    I'm on
    WHM 8.8.0 cPanel 8.8.0-S74
    Red Hat Linux 9

    Webalizer doesnt show country data 100% unresolved countries. Help ..
     
Loading...

Share This Page