The Community Forums

Interact with an entire community of cPanel & WHM users!
  1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Better Error Log: Source Code

Discussion in 'cPanel Developers' started by stevenc317, Mar 2, 2009.

  1. stevenc317

    stevenc317 Well-Known Member

    Joined:
    Jan 27, 2009
    Messages:
    56
    Likes Received:
    0
    Trophy Points:
    6
    I personally cannot stand a pure text dump of the Apache error log as how cPanel shows it. I want additional information regarding how the information is displayed, so I came up with the following perl script.

    This script does a number of things for me. First it takes the IP addresses and checks to see if it is my static IP from my office (if you have dynamic IP then this part will not work - but will not *break* anything, don't worry), if it is my IP then it shows a company logo in the error log rather than just an IP. This makes it easy when we are debugging perl scripts, just look for your logo and that was something you did.

    Once it goes past that IP check if it is not my IP then it does a hostname lookup to see who the IP belongs to. If it is Googlebot or Yahoo Spider their logo is displayed next to that entry. This helps identify Error 404s and which spider is seeing it.

    The script is very simple there are a handful of variables at the top that need to be updated. Additionally once you upload the script (I recommend that you password protect the directory you place this script it because it WILL show path information to anyone that runs it.) just run it in your browser like this: errorlog.cgi?count=50 (replace 50 with the number or entries you want to show up on the screen)


    HTML:
    #!/usr/bin/perl
    
    use CGI qw/:standard/;
    $CGI::POST_MAX=1024 * 1;
    
    $count = param ('count');
    $tail_path = "/usr/bin/tail";
    $log_path = "/path/to/apache/error.log";
    $my_ip = "55.55.55.555";
    $my_pic_path = "http://www.yoursite.com/graphics/yourlogo.gif";
    $google_pic_path = "http://www.yoursite.com/graphics/google.gif";
    $yahoo_pic_path = "http://www.yoursite.com/graphics/yahoo.gif";
    
    if (-e $log_path)
    	{
    	$log_file = `$tail_path -$count $log_path`;
    	}
    else
    	{
    	print "error log file not found!";
    	exit;
    	}
    
    @lines = split (/\n/,$log_file);
    @lines = reverse (@lines);
    $x = 2;
    
    foreach $line (@lines)
    	{
    	if ($line ne "")
    		{
    		if ($x % 2 == 0)
    			{
    			$color = "#ddffff";
    			}
    		else
    			{
    			$color = "#ffffff";
    			}
    		## Gotta strip out any <b> html tags
    		$line =~ s/<b>//ig;
    		$ip = "";
    		$referer = "";
    		$remaining = "";
    		$host_info = "";
    		
    		@data = split (/] /,$line);
    		$curr_date = "$data[0]";
    		$curr_date =~ s/\[//;
    		unless ($date_block eq $curr_date)
    			{
    			$x = 2;
    			$color = "#ddffff";
    			$date_block = "$data[0]";
    			$date_block =~ s/\[//;
    			print "<div style=\"color:#008080; margin-top:15px;\">$date_block</div>\n";
    			$x++;
    			}
    		else
    			{
    			if ($x % 2 == 0)
    				{
    				$color = "#ddffff";
    				}
    			else
    				{
    				$color = "#ffffff";
    				}
    			$x++;
    			}
    
    		$error = "<span style=\"font-weight:bold\">$data[1]</span>";
    		$error =~ s/\[//;
    		$ip = $data[2];
    		$ip =~ s/\[client //;
    	
    		
    		if ($ip =~ m/\d+\.\d+\.\d+\.\d+/)
    			{
    			$ip =~ s/:\d+//;
    			if (($ip ne $my_ip) && ($ip ne ""))
    				{
    				$response = `host $ip`;
    				if (($response =~ m/google\.com\W*$/) || ($response =~ m/googlebot\.com\W*$/))
    					{
    					$ip = "<img src=$google_pic_path style=\"position:relative; top:7px; width:66px; height:22px;\">";
    					$host_info = "";
    					}
    				elsif (($response =~ m/yahoo\.net\W*$/) || ($response =~ m/yahoo\.com\W*$/))
    					{
    					$ip = "<img src=$yahoo_pic_path style=\"position:relative; top:4px; width:95px; height:18px;\">";
    					$host_info = "";
    					}
    				else
    					{
    					$ip = "[<b>$ip <a href=https://www.YOURDOMAIN.com:WHM_PORT/cgi/addon_csf.cgi?action=tempdeny&ip=$ip&timeout=900 target=_new>(block)</a></b>]";
    					$host_info = "<div style=\"margin-left:20px;\"><span style=\"font-weight:bold\">host info:</span> $response</div>";
    					}
    				}
    				
    			elsif ($ip eq $my_ip)
    				{
    				$ip = "<img src=$my_pic_path style=\"position:relative; top:4px; width:171px; height:18px;\">";
    				$host_info = "";
    				}
    				
    			else
    				{
    				$host_info = "(unknown - bad ip)";
    				}
    			}
    			
    		#
    		
    		if ($data[3] =~ m/(.*)(referer.*)/)
    			{
    			$remaining = $1;
    			$referer = $2;
    			$referer =~ s/^referer://;
    			$referer =~ s/^referer//;
    			}
    		else
    			{
    			$remaining = $data[3];
    			}
    		
    		print "<div style=\"padding:3px; background-color:$color;\">\n<div style=\"margin-left:20px;\"><span style=\"color:#dd0000;\">$error: </span>$ip $remaining</div>\n";
    		
    		if ($referer ne "")
    			{
    			print "<div style=\"margin-left:20px;\"><span style=\"font-weight:bold\">referer: </span>$referer</div>\n";
    			}
    		
    		if ($host_info ne "")
    			{
    			print "$host_info\n";
    			}
    		print "</div>\n\n";
    		}
    	}
    
    exit;
     

    Attached Files:

    #1 stevenc317, Mar 2, 2009
    Last edited: Mar 3, 2009
  2. stevenc317

    stevenc317 Well-Known Member

    Joined:
    Jan 27, 2009
    Messages:
    56
    Likes Received:
    0
    Trophy Points:
    6
    I added a tweak to the codes (update is posted above) to allow you to click a link and block all traffic from that specific IP address. For that to work you WILL need to have ConfigServer Security & Firewall installed.
     
  3. cPDan

    cPDan cPanel Staff
    Staff Member

    Joined:
    Mar 9, 2004
    Messages:
    711
    Likes Received:
    3
    Trophy Points:
    18
    WARNING SCRIPT HAS ARBITRARY SYSTEM COMMAND EXECUTION VULNERABILITY

    Code:
    $count = param ('count');
    ...
    `$tail_path -$count $log_path`;
    Solution (also defaults to valid data)

    Code:
    $count = int( param('count') ) || 50;
     
    #3 cPDan, Mar 4, 2009
    Last edited: Mar 4, 2009
  4. cPDan

    cPDan cPanel Staff
    Staff Member

    Joined:
    Mar 9, 2004
    Messages:
    711
    Likes Received:
    3
    Trophy Points:
    18
    Here is a slightly cleaned up version up to where the loop starts (the internals of that need some attention)

    Among other fixups, this one avoids slurping the entire output of the command into a variable only to slurp that into an array of lines.

    Code:
    #!/usr/bin/perl
    
    use strict;
    use warnings;
    use CGI;
    $CGI::POST_MAX        = 1024 * 100;    # max 100K posts
    $CGI::DISABLE_UPLOADS = 1;             # if we're worried about POST_MAX we might as well set this too
    
    my $tail_path       = "/usr/bin/tail";
    my $log_path        = "/path/to/apache/error.log";
    my $my_ip           = "55.55.55.555";
    my $my_pic_path     = "http://www.yoursite.com/graphics/yourlogo.gif";
    my $google_pic_path = "http://www.yoursite.com/graphics/google.gif";
    my $yahoo_pic_path  = "http://www.yoursite.com/graphics/yahoo.gif";
    
    my $cgi = CGI->new();
    my $count = int( $cgi->param('count') ) || 50;
    
    print $cgi->header();
    
    if ( !-r $log_path ) {
        print "error log file not found or not readble!";
        exit;
    }
    
    my $x = 2;
    
    foreach my $line ( reverse(`$tail_path -$count $log_path`) ) {
    ...
     
  5. cPDan

    cPDan cPanel Staff
    Staff Member

    Joined:
    Mar 9, 2004
    Messages:
    711
    Likes Received:
    3
    Trophy Points:
    18
    Here's the entire thing tidy'd up and tweaked a bit (untested past perl -Mstrict -wc)

    Code:
    #!/usr/bin/perl
    
    use strict;
    use warnings;
    use CGI;
    $CGI::POST_MAX        = 1024 * 100;    # max 100K posts
    $CGI::DISABLE_UPLOADS = 1;             # if we're worried about POST_MAX we might as well set this too
    
    my $tail_path       = "/usr/bin/tail";
    my $log_path        = "/path/to/apache/error.log";
    my $my_ip           = "55.55.55.555";
    my $my_pic_path     = "http://www.yoursite.com/graphics/yourlogo.gif";
    my $google_pic_path = "http://www.yoursite.com/graphics/google.gif";
    my $yahoo_pic_path  = "http://www.yoursite.com/graphics/yahoo.gif";
    
    my $cgi = CGI->new();
    my $count = int( $cgi->param('count') ) || 50;
    
    print $cgi->header();
    
    if ( !-r $log_path ) {
        print "error log file not found or not readble!";
        exit;
    }
    
    my $x = 2;
    
    foreach my $line ( reverse(`$tail_path -$count $log_path`) ) {
        if ( $line ne "" ) {
            my $color = "#ffffff";
    
            if ( $x % 2 == 0 ) {
                $color = "#ddffff";
            }
    
            ## Gotta strip out any <b> html tags
            $line =~ s/<b>//ig;
    
            my $referer    = "";
            my $host_info  = "";
            my $date_block = "";
    
            my ( $curr_date, $ip, $remaining ) = split( /] /, $line );
            $curr_date =~ s/\[//;
            $ip        =~ s/\[client //;
    
            if ( $date_block ne $curr_date ) {
                $x          = 2;
                $color      = "#ddffff";
                $date_block = $curr_date;
                $date_block =~ s/\[//;
                print "<div style=\"color:#008080; margin-top:15px;\">$date_block</div>\n";
            }
            $x++;
    
            my $error = "<span style=\"font-weight:bold\">$ip</span>";
            $error =~ s/\[//;
    
            if ( $ip =~ m/\d+\.\d+\.\d+\.\d+/ ) {
                $ip =~ s/:\d+//;
                if ( ( $ip ne $my_ip ) && ( $ip ne "" ) ) {
                    my $response = `host $ip`;
                    if ( ( $response =~ m/google\.com\W*$/ ) || ( $response =~ m/googlebot\.com\W*$/ ) ) {
                        $ip        = "<img src=$google_pic_path style=\"position:relative; top:7px; width:66px; height:22px;\">";
                        $host_info = "";
                    }
                    elsif ( ( $response =~ m/yahoo\.net\W*$/ ) || ( $response =~ m/yahoo\.com\W*$/ ) ) {
                        $ip        = "<img src=$yahoo_pic_path style=\"position:relative; top:4px; width:95px; height:18px;\">";
                        $host_info = "";
                    }
                    else {
                        $ip        = "[<b>$ip <a href=\"https://www.YOURDOMAIN.com:WHM_PORT/cgi/addon_csf.cgi?action=tempdeny&ip=$ip&timeout=900\" target=\"_blank\">(block)</a></b>]";
                        $host_info = "<div style=\"margin-left:20px;\"><span style=\"font-weight:bold\">host info:</span> $response</div>";
                    }
                }
    
                elsif ( $ip eq $my_ip ) {
                    $ip        = "<img src=$my_pic_path style=\"position:relative; top:4px; width:171px; height:18px;\">";
                    $host_info = "";
                }
    
                else {
                    $host_info = "(unknown - bad ip)";
                }
            }
    
            if ( $remaining =~ m/(.*)(referer.*)/ ) {
                $remaining = $1;
                $referer   = $2;
                $referer =~ s/^referer://;
                $referer =~ s/^referer//;
            }
    
            print "<div style=\"padding:3px; background-color:$color;\">\n<div style=\"margin-left:20px;\"><span style=\"color:#dd0000;\">$error: </span>$ip $remaining</div>\n";
    
            if ( $referer ne "" ) {
                print "<div style=\"margin-left:20px;\"><span style=\"font-weight:bold\">referer: </span>$referer</div>\n";
            }
    
            if ( $host_info ne "" ) {
                print "$host_info\n";
            }
            print "</div>\n\n";
        }
    }
     
    Infopro likes this.
  6. stevenc317

    stevenc317 Well-Known Member

    Joined:
    Jan 27, 2009
    Messages:
    56
    Likes Received:
    0
    Trophy Points:
    6
    Thanks for the tips, a few of those lines I have never thought to code in that manner (EX: $count = int( $cgi->param('count') ) || 50; ). Normally I would just use regex to clean out any non numerical values, but your way is much cleaner.

    Regarding the security issues, I must admit I was pretty lax on this script as compared to the ones I allow my customers and/or the public to see. This script was designed to be run exclusively from a controlled environment. In my case I am using an .htaccess to force user authentication via username/password and also blocking all traffic unless it comes from my office static IP address. Of course added security is never a bad thing.

    I did try running your tweaked codes in their entirety and while they did run (no error 500) they did not parse the log file correctly and display the results as expected. So what I did was take your tips and implemented many of them into my own script.

    I still have a few small little issues I am working out (formatting as I am picky about that) and will post my updated script for the community tomorrow.

    Look forward to any other tips or suggestions you guys have and feel free to use the codes if you find a use for them!


    Steven
     
  7. stevenc317

    stevenc317 Well-Known Member

    Joined:
    Jan 27, 2009
    Messages:
    56
    Likes Received:
    0
    Trophy Points:
    6
    I love my error log. Of course I iptables this guy pretty quick.

    (from stevencarlson.net/2009/03/04/nice-try-asshole )

    Well you need to give credit to this guy for a really cute set of spoofed hostnames.

    Wed Mar 04 21:43:24 2009
    error: [208.99.208.40] Invalid URI in request GET /../css/main.css HTTP/1.0
    host info: 40.208.99.208.in-addr.arpa domain name pointer the.cops.gona.get.u.if.u.smoke-weed.com.

    Wed Mar 04 21:42:54 2009
    error: [208.99.208.45] Invalid URI in request GET /../docs/credit.shtml HTTP/1.0
    host info: 45.208.99.208.in-addr.arpa domain name pointer iv.never.seen.a-pussy.biz.

    Wed Mar 04 21:42:47 2009
    error: [208.99.208.45] Invalid URI in request GET /../docs/contact.shtml HTTP/1.0
    host info: 45.208.99.208.in-addr.arpa domain name pointer iv.never.seen.a-pussy.biz.

    Wed Mar 04 21:42:27 2009
    error: [208.99.208.45] Invalid URI in request GET /../cat_pages/inventory_1.shtml HTTP/1.0
    host info: 45.208.99.208.in-addr.arpa domain name pointer iv.never.seen.a-pussy.biz.

    Wed Mar 04 21:42:10 2009
    error: [208.99.208.46] Invalid URI in request GET /../docs/request.shtml HTTP/1.0
    host info: 46.208.99.208.in-addr.arpa domain name pointer i.want.to.see.a-pussy.biz.
     
    #7 stevenc317, Mar 4, 2009
    Last edited by a moderator: Mar 5, 2009
  8. cPDan

    cPDan cPanel Staff
    Staff Member

    Joined:
    Mar 9, 2004
    Messages:
    711
    Likes Received:
    3
    Trophy Points:
    18
    You're welcome

    I figured as much, I was more focused on running it through perltidy and plugging some glaring holes.


    use Perl Best Practices as a guide and you will do very well
     
  9. stevenc317

    stevenc317 Well-Known Member

    Joined:
    Jan 27, 2009
    Messages:
    56
    Likes Received:
    0
    Trophy Points:
    6
    new codes -- new features!

    I have added a lot of additional functionality to this script including:

    1) Stripping all HTML tags < >
    2) Setting a maximum limit to tail
    2) Ability to click on an IP address in the error log and list all traffic (both from error log and access log) of that IP.
    3) Preform ARIN Whois searches on IPs to see the registered owner and abuse contact information
    4) Blocking IP for 15 minutes, 24 hours, and 2 weeks.
    5) Since I start with a fresh error & access log each day (archiving previous day's) I added a function &which_log=old to load the 'old' day's log.

    The script still needs some work, your comments are much appreciated.

    -- codes in next box as it was too long to fit 10,000 char limit --


    Also posted to --> http://www.stevencarlson.net/2009/03/05/a-better-apache-error-log-viewer/
     
    #9 stevenc317, Mar 5, 2009
    Last edited: Mar 5, 2009
  10. stevenc317

    stevenc317 Well-Known Member

    Joined:
    Jan 27, 2009
    Messages:
    56
    Likes Received:
    0
    Trophy Points:
    6
    here are the codes

    HTML:
    #!/usr/bin/perl
    
    use CGI qw/:standard/;
    $CGI::POST_MAX=1024 * 1;
    $CGI::DISABLE_UPLOADS = 1;
    $| = 1;
    
    ## Define the vars to be used in this script
    	
    	$tail_path = "/usr/bin/tail";
    	$error_log_path = "/path/to/logs/error.log";
    	$access_log_path = "/path/to/logs/access.log";
    	$my_ip = "55.55.55.555;
    	$my_pic_path = "http://www.YOU.com/graphics/cppsolutions.gif";
    	$google_pic_path = "http://www.YOU.com/graphics/google.gif";
    	$yahoo_pic_path = "http://www.YOU.com/graphics/yahoo.gif";
    	$block_URL = "https://www.YOU.com:PORT/cgi/addon_csf.cgi?action=tempdeny";
    	$arin_URL = "http://ws.arin.net/whois/?queryinput=";
    	$myself_URL = "http://www.YOU.com/CGI-DIRECTORY/NAME_OF_THIS_SCRIPT.cgi";	
    	$datetime = localtime();  
    
    
    ## Grab param strings
    	$cgi = CGI->new();
    	
    	# Line numbers to display
    	$count = int( $cgi->param('count') ) || 50;
    	if ($count >= 5000) {$count = 5000;}
    	
    	# Load param for which function I need to process
    	$which_function = param('function');
    	
    	
    ## Say hello
    
    print "Content-type: text/html\r\n\r\n";
    print "<html>\n<head>\n<title>WebServer Error Log </title>\n</head>\n<style>\n\ta:hover {text-decoration: underline;color:#FF0000;}\n\ta:link {text-decoration: none;}\n\ta:visited {text-decoration: none;}\n</style>\n<body>\n\n";
    
    
    
    # Is this todays or yesterday's log?
    if (param('which_log') eq "old"){$error_log_path = "$error_log_path\.old";} ## change to where your old log is located
    
    # Make sure the log is able to be read
    if ( (!-r $error_log_path) || (!-r $access_log_path))
    	{
        print "Someone fucked up!";
        exit;
    	}
    
    ### What you want me to do? ####
    if (param('which_function') eq "details") 
    	{
    	# This is for the 'full details' of a specific IP address
    	&showDetails;
    	}
    else
    	{
    	# Else do the standard Apache Log Processing
    	&apacheLog;
    }
    
    ## Say Goodbye
    print "</body>\n</html>\n";
    exit;
    
    sub showDetails
    	{
    	$bad_ip = param('ip');
    	print "<center><h3>Full Activity Report of IP: $bad_ip</h3>as of $datetime<hr align=center width=75%>";
    	print "<small>Quick Functions: [<a href=$block_URL&ip=$bad_ip&timeout=86400 target=_new>Block for 24 hours</a>] -- [<a href=$block_URL&ip=$bad_ip&timeout=1209600 target=_new>Block for 2 weeks</a>]</small></center> <br>\n\n";
    
    	###################
    	## Query whois for this IP ##
    	###################
    	print "<h3>ARIN WHOIS Report:</h3>\n\n";
    
    	$url = "$arin_URL";
    	$url .= "$bad_ip";
     	$Referer = "http://ws.arin.net/";
     	
     	## Setup LWP
     	use LWP::UserAgent;
     	$user_agent = new LWP::UserAgent;
     	$request = HTTP::Request->new(GET => $url);
     	$request->push_header('Referer', $Referer);
     	$response = $user_agent->request($request);
     
    	## Grab response and parse
     	$raw_whois = $response->{_content};
     	$raw_whois =~ m/<pre>(.*?)(?=<\/pre>)/s;
     	$whois = $1;
     	
     	## Clean up results prior to print
     	$whois =~ s/\n/<br>\n/g;
     	$whois =~ s/<br>//; ## only remove the first <br>
     	$whois =~ s/a href/a target=_new href/g;
     	$whois =~ s/\/whois\//http:\/\/ws.arin.net\/whois\//g;
     	
     	## Print it
     	print "<small>$whois</small>";
    	
    	
    	###################
    	## Traceroute the IP addr ##
    	###################
    
    ## -- I disabled this because it was very slow -- 
    ## anyone else have a better idea?
    
    #	$url = "http://network-tools.com/default.asp?prog=trace&base=yes&host=$bad_ip";
    # 	$Referer = "http://network-tools.com/";
    # 	
    # 	use LWP::UserAgent;
    # 	$user_agent = new LWP::UserAgent;
    # 	$request = HTTP::Request->new(GET => $url);
    # 	$request->push_header('Referer', $Referer);
    # 	$response = $user_agent->request($request);
    # 
    # 	$traceroute = $response->{_content};
    # 	
    # 	print $traceroute;
    	
    	
    	#########################
    	## Open & Parse Apache Access Log ##
    	#########################
    	# Tell the user what we are doing
    	print "<h3>Suspicious user activity from Apache access log:</h3>\n\n";
    	my $x = 2;
    	$accessLog = NULL;
    	
    	## Look at yesterday's log first
    	open(FILE,"$access_log_path\.old") or print "ERROR 404: Cannot open old error log file because $!<br>";
    	while(<FILE>)
    		{
    		$accessLog .= $_ if $. >= 0
    		}
    	close (FILE);
    	
    	## Now today's
    	open(FILE,"$access_log_path") or print "ERROR 404: Cannot open error log file because $!<br>";
    	while(<FILE>)
    		{
    		$accessLog .= $_ if $. >= 0
    		}
    	close (FILE);
    	
    	## Pop into an array and run
    	@lines = split (/\n/,$accessLog);
    	@lines = reverse (@lines);
    
    	
    	# process & print results
    	foreach $line (@lines)
    		{
    		&processAccessLine;
    		}
    		
    		
    	########################
    	## Open & Parse Apache Error Log ##
    	########################
    	# Tell the user what we are doing
    	print "<h3>Suspicious user activity from Apache error log:</h3>\n\n";
    	my $x = 2;
    	$lines = NULL;
    	@lines = NULL;
    	$errorLog = NULL;
    	
    	## Look at yesterday's log first
    	open(FILE,"$error_log_path\.old") or print "ERROR 404: Cannot open old error log file because $!<br>";
    	while(<FILE>)
    		{
    		$errorLog .= $_ if $. >= 0
    		}
    	close (FILE);
    	
    	## Now today's
    	open(FILE,"$error_log_path") or print "ERROR 404: Cannot open error log file because $!<br>";
    	while(<FILE>)
    		{
    		$errorLog .= $_ if $. >= 0
    		}
    	close (FILE);
    	
    	## Pop into an array and run
    	@lines = split (/\n/,$errorLog);
    	@lines = reverse (@lines);
    
    	
    	# process & print results
    	foreach $line (@lines)
    		{
    		&processErrorLine;
    		}
    		
    	return;
    	}
    	
    sub apacheLog
    	{
    	print "<h3>Last $count entries in webserver error log</h3>\n\n";
    	my $x = 2;
    	
    	foreach $line ( reverse(`$tail_path -$count $error_log_path`) )
    		{
    		&processErrorLine;
    		}
    	return;
    	}
    sub processAccessLine
    	{
    	if ($line =~ m/^$bad_ip/g)
    		{
    		print "$line<br>";
    		}
    	return;
    	}
    	
    sub processErrorLine
    	{
    	if ($line ne "")
    		{
    		$conent = "";
    		if ($x % 2 == 0)
    			{
    			$color = "#ddffff";
    			}
    		else
    			{
    			$color = "#ffffff";
    			}
    		## Gotta strip out any html tags
    		$line =~ s/<b>//ig;
    		$line =~ s/</&lt;/g;
    		$line =~ s/>/&gt;/g;
    		$line =~ s/\////g;
    		$line =~ s/\|/|/g;
    		
    		## Clear out the vars
    		$ip = "";
    		$referer = "";
    		$remaining = "";
    		$host_info = "";
    		
    		@data = split (/] /,$line);
    		$curr_date = "$data[0]";
    		$curr_date =~ s/\[//;
    		unless ($date_block eq $curr_date)
    			{
    			$x = 2;
    			$color = "#ddffff";
    			$date_block = "$data[0]";
    			$date_block =~ s/\[//;
    			$content = "<div style=\"color:#008080; margin-top:15px;\">$date_block</div>\n";
    			$x++;
    			}
    		else
    			{
    			if ($x % 2 == 0)
    				{
    				$color = "#ddffff";
    				}
    			else
    				{
    				$color = "#ffffff";
    				}
    			$x++;
    			}
    
    		$error = "<span style=\"font-weight:bold\">$data[1]</span>";
    		$error =~ s/\[//;
    		$ip = $data[2];
    		$ip =~ s/\[client //;
    		
    		if ((param('which_function') eq "details") && ($ip =~ m/$bad_ip/g))
    			{		
    			&processIPaddress;
    			}
    		elsif ((param('which_function') eq "details") && ($ip !~ m/$bad_ip/g))
    			{
    			#do not print anything and go back
    			return;
    			}
    		elsif (param('which_function') ne "details")
    			{
    			&processIPaddress;
    			}
    		else
    			{
    			print "we got a problem";
    			}
    		
    		if ($data[3] =~ m/(.*)(referer.*)/)
    			{
    			$remaining = $1;
    			$referer = $2;
    			$referer =~ s/^referer://;
    			$referer =~ s/^referer//;
    			}
    		else
    			{
    			$remaining = $data[3];
    			}
    		
    		$content .= "<div style=\"padding:3px; background-color:$color;\">\n<div style=\"margin-left:20px;\"><span style=\"color:#dd0000;\">$error: </span>$ip $remaining</div>\n";
    		
    		if ($referer ne "")
    			{
    			$content .= "<div style=\"margin-left:20px;\"><span style=\"font-weight:bold\">referer: </span>$referer</div>\n";
    			}
    		
    		if ($host_info ne "")
    			{
    			$content .= "$host_info\n";
    			}
    		$content .= "</div>\n\n";
    		}
    	print $content;
    	return;
    	}
    	
    sub processIPaddress
    	{
    	if ($ip =~ m/\d+\.\d+\.\d+\.\d+/)
    		{
    		$ip =~ s/:\d+//;
    		if (($ip ne $my_ip) && ($ip ne ""))
    			{
    			$response = `host $ip`;
    			if (($response =~ m/google\.com\W*$/) || ($response =~ m/googlebot\.com\W*$/))
    				{
    				$ip = "<a href=$myself_URL?which_function=details&ip=$ip target=_new><img border=0 src=$google_pic_path style=\"position:relative; top:7px; width:66px; height:22px;\"></a>";
    				$host_info = "";
    				}
    			elsif (($response =~ m/yahoo\.net\W*$/) || ($response =~ m/yahoo\.com\W*$/))
    				{
    				$ip = "<a href=$myself_URL?which_function=details&ip=$ip target=_new><img border=0 src=$yahoo_pic_path style=\"position:relative; top:4px; width:95px; height:18px;\"></a>";
    				$host_info = "";
    				}
    			else
    				{
    				$ip = "[<b><a href=$myself_URL?which_function=details&ip=$ip target=_new>$ip</a> <small>(<a href=$arin_URL$ip target=_new>whois</a>) (<a href=$block_URL&ip=$ip&timeout=900 target=_new>block 15</a>)</small></b>]";
    				$host_info = "<div style=\"margin-left:20px;\"><span style=\"font-weight:bold\">host info:</span> $response</div>";
    				}
    			}
    			
    		elsif ($ip eq $my_ip)
    			{
    			$ip = "<a href=$myself_URL?which_function=details&ip=$ip target=_new><img border=0 src=$my_pic_path style=\"position:relative; top:4px; width:171px; height:18px;\"></a>";
    			$host_info = "";
    			}
    			
    		else
    			{
    			$host_info = "(unknown - bad ip)";
    			}
    		}
    	return;
    	}
    
     
    #10 stevenc317, Mar 5, 2009
    Last edited: Mar 6, 2009
  11. stevenc317

    stevenc317 Well-Known Member

    Joined:
    Jan 27, 2009
    Messages:
    56
    Likes Received:
    0
    Trophy Points:
    6
    fixed small bug line 42
     
  12. stevenc317

    stevenc317 Well-Known Member

    Joined:
    Jan 27, 2009
    Messages:
    56
    Likes Received:
    0
    Trophy Points:
    6
    anyone else have any comments?
     
  13. IIIBradIII

    IIIBradIII Member

    Joined:
    Apr 13, 2003
    Messages:
    21
    Likes Received:
    0
    Trophy Points:
    1
    And to think that I popped into these forums today just to look around for a better error log! Thank you for this! I am so compulsive about watching my error log daily but the default cpanel log is definitely a wasteland - this could end the suffering!

    Is it safe to assume that we can add a comma separated list of IPs to be considered ours? We will need to use this in a multi-user environment and we are in different locations with various (albeit static) IPs.

    Edit: It looks like it's not setup to accept an array of "my" IPs. That would be a handy addition for sure.
     
    #13 IIIBradIII, Mar 17, 2009
    Last edited: Mar 17, 2009
  14. stevenc317

    stevenc317 Well-Known Member

    Joined:
    Jan 27, 2009
    Messages:
    56
    Likes Received:
    0
    Trophy Points:
    6
    Brad,

    It would be pretty simple to implement, you would need to create an array that contain all the IP addresses, then walk through the array and see if the current IP matches any value in the array.


    Steven
     
  15. IIIBradIII

    IIIBradIII Member

    Joined:
    Apr 13, 2003
    Messages:
    21
    Likes Received:
    0
    Trophy Points:
    1
    It's choking while stripping out html?

    Code:
    $line =~ s/\////g;
     
  16. stevenc317

    stevenc317 Well-Known Member

    Joined:
    Jan 27, 2009
    Messages:
    56
    Likes Received:
    0
    Trophy Points:
    6
    now that is just plain funny... this is a bug (I guess you would consider it that) with vBulletin and how it displays 'codes'. Rather than display the code as I entered it, it was replacing the & # 47; (spaces added so it wouldn't screw it up) with an '/', which of course screwed up the codes.

    I am guessing you are the only one to actually try out this script, well at least someone here appreciates my work.


    Anyways I took a screen shot of what the code should be for that line so you can manually type it.
     

    Attached Files:

    #16 stevenc317, Mar 19, 2009
    Last edited: Mar 19, 2009
  17. IIIBradIII

    IIIBradIII Member

    Joined:
    Apr 13, 2003
    Messages:
    21
    Likes Received:
    0
    Trophy Points:
    1
    Bingo - that did it. The & # 124; line after that was also altered by vB.

    I'm going to try to get my_ip in an array today if I get a chance - my perl mojo is low but if I get it working I'll post it here.

    Edit: Turns out my perl knowledge was even less than I anticipated. I may give it another go, but for now I'm stumped.

    Something else I was planning to add was an ignore array so that I could ignore things like favicon 404s, etc. I clearly need to improve my handle on perl first though...
     
    #17 IIIBradIII, Mar 20, 2009
    Last edited: Mar 21, 2009
  18. stevenc317

    stevenc317 Well-Known Member

    Joined:
    Jan 27, 2009
    Messages:
    56
    Likes Received:
    0
    Trophy Points:
    6
    Ahh... ok good, happy it is working

    Give this a shot -- this *should* work, I am pulling it out of my rear at the moment, but no reason it shouldn't work:
    Code:
    @myIPs = ("123.123.123.123", "123.123.123.124");
    
    foreach $myIPs (@myIPs)
       {
        if ($ip eq $myIPs)
          {
             print "match";
          }
       }
    In your: /usr/local/apache/conf/includes/errordocument.conf

    Add:
    Code:
    # Don't bother looking for favicon.ico
    Redirect 404 /favicon.ico
    Redirect 404 /favicon.gif
    
    # Don't bother sending the custom error page for favicon.ico
    <Location /favicon.ico>
        ErrorDocument 404 "No favicon.ico"
    </Location>
    
    <Location /favicon.gif>
        ErrorDocument 404 "No favicon here"
    </Location>
    
    This will prevent Apache from adding an 404 error in your log, additionally it responds to the browser/search engine "No favicon here"
     
  19. cPDan

    cPDan cPanel Staff
    Staff Member

    Joined:
    Mar 9, 2004
    Messages:
    711
    Likes Received:
    3
    Trophy Points:
    18
    Hello,

    A few things I noticed that would help make this script a little more solid...Since you asked :)

    why import all those functions when:
    a) you don't use them
    b) you have an object already that has access to every method available

    You already have an object, why call param() as a function?

    Also consider proper scoping: perldoc -f my

    And use modules :)

    for starters:
    use strict;
    use warnings;

    If you really want to strip them:
    http://search.cpan.org/perldoc?HTML::Obliterate

    If you really just want to make it XSS safe:
    my $html_safe_text = $cgi->escapeHTML($text);
    or to preserve whitespace as well (like pre but without pre tags)
    http://search.cpan.org/perldoc?Text::InHTML

    a lot of that sort of work would not be needed if you used scoping :)
     
  20. stevenc317

    stevenc317 Well-Known Member

    Joined:
    Jan 27, 2009
    Messages:
    56
    Likes Received:
    0
    Trophy Points:
    6
    Great! Honestly any and all feedback is great. When multiple programmers get together to make a project it becomes better than any one could ever do by him/herself.


    Because I use param ;-) it will not work with out CGI.pm

    Maybe (and it is possible) I do not understand you here, but I am using 'function' as more of a placeholder to tell the script if I want to process the entire log or show the details of one specific IP address. I probably should have documented that better in the codes.

    Do you mean use 'my' before the variables? I know you 'should' under proper coding rules, but since 99% of my programming is much more advanced than this with variables being passed between sub functions and scripts I need the portability. Also I (excluding as counters in a loop) do not use $x my vars are descriptive enough so that there is no confusion or overlapping elsewhere.

    I do not like strict, because it is that--too strict. On many occasions I need to preform creatively and push perl further than it was designed to go and strict is a pain. But I will admit, that it is not 'proper' to do it without it.

    Regarding warnings, I absolutely think this is the worst module ever created. The concept of printing errors back to the web browser where a web surfer could stumble across information is very dangerous. My errors to the user tell them nothing, they do not need to know!

    Why bring in a whole module of hundreds of lines of code to remove just a few simple things that can be done in about five lines in RegEx. Of course this module does a LOT more than my 5 lines, but they are not really needed in this case.

    Hmmm? I'm lost, could you explain a little more?

    thanks!
     
    #20 stevenc317, Mar 23, 2009
    Last edited: Mar 23, 2009
Loading...

Share This Page