/var/log/messages analysis with awk

bloatedstoat

Well-Known Member
Jun 14, 2012
183
24
68
Victoria, Australia
cPanel Access Level
Root Administrator
Hello,

I'm looking to refine the following one liner to extract IP addresses, sort and rank them.
I've got so far but need to get rid of the SRC= bit to leave the IP only.

The entry in the logs follows this (IP addresses obfuscated for privacy)
Code:
Mar 3 18:00:06 myserver kernel: [414786.179376] Firewall: *TCP_IN Blocked* IN=eth0 OUT= MAC=00:00:00:00:00:ec:00:00:00:00:00:00:00:00 SRC=xxx.xxx.xxx.xxx DST=xxx.xx.xxx.xxx LEN=44 TOS=0x00 PREC=0x00 TTL=235 ID=32940 PROTO=TCP SPT=52209 DPT=1433 WINDOW=1024 RES=0x00 SYN URGP=0
The one liner as it stands is:
Code:
cat /var/log/messages | awk '{ print $13 }' | sort | uniq -c | sort -n
The result is :
45 SRC=xxx.xxx.xxx.xxx
49 SRC=xxx.xxx.xxx.xxx
54 SRC=xxx.xxx.xxx.xxx
63 SRC=xxx.xxx.xxx.xxx

... and so on. Is there a tweak that can be made to get the IP only as well as the hits to the left?

Thanks for any help!
 

fuzzylogic

Well-Known Member
Nov 8, 2014
154
93
78
cPanel Access Level
Root Administrator
Code:
cat /var/log/messages | awk '/*TCP_IN Blocked*/ {print}' | awk '{ print $13 }' | awk '{ sub(/SRC=/, ""); print }' | sort | uniq -c | sort -n
Seems to work.
Someone who knows what they are doing may be able to acheive the same while only invoking awk once.

Also you could use sort -nr instead of sort -n if you want the major offenders at the top of the list.
 

cPanelLauren

Product Owner
Staff member
Nov 14, 2017
13,298
1,279
313
Houston
I have a different log level than you but using cut with a delimeter of the = and printing only what comes after should work so just add:
Code:
cat /var/log/messages | awk '{ print $13 }' | cut -d= -f2 |sort | uniq -c | sort -n
should work but you'll get a lot more than firewall blocks with that.

I looked for firewall and on my server the awk field I'm printing for the same thing is 12 instead of 13 so it looks like this:

Code:
[[email protected] backups]# grep Firewall /var/log/messages | awk '{ print $12 }' | cut -d= -f2 |sort | uniq -c | sort -nr
    474 185.156.73.54
    140 185.176.27.254
    137 185.156.73.52
    134 185.176.27.174
    122 194.26.29.114
You can add multiple fields to print with awk as well rather than piping them each out. For example if I want to see if they're connecting via TCP or UDP

Code:
[[email protected] backups]# grep Firewall /var/log/messages | awk '{ print $12,$7 }' | cut -d= -f2 |cut -d_ -f1|sort | uniq -c | sort -nr
    474 185.156.73.54 *TCP
    140 185.176.27.254 *TCP
    137 185.156.73.52 *TCP
    134 185.176.27.174 *TCP
    123 194.26.29.114 *TCP
You can also pretty effectively use sed for formatting as well but cut seemed easier to me for this
 
  • Like
Reactions: bloatedstoat

fuzzylogic

Well-Known Member
Nov 8, 2014
154
93
78
cPanel Access Level
Root Administrator
awk's default is to print all of each line that matches the pattern, so we can remove 2 instances of the print command.
Also results can be limited to have a count greater than a chosen number.
So here is a revised version...
Code:
cat /var/log/messages | awk '/*TCP_IN Blocked*/' | awk '{ print $13 }' | awk 'sub(/SRC=/, "")' | sort | uniq -c | sort -nr | awk '$1 >= 100'