/var/log/messages analysis with awk

[bloatedstoat]

Hello,

I'm looking to refine the following one liner to extract IP addresses, sort and rank them.
I've got so far but need to get rid of the SRC= bit to leave the IP only.

The entry in the logs follows this (IP addresses obfuscated for privacy)

Mar 3 18:00:06 myserver kernel: [414786.179376] Firewall: *TCP_IN Blocked* IN=eth0 OUT= MAC=00:00:00:00:00:ec:00:00:00:00:00:00:00:00 SRC=xxx.xxx.xxx.xxx DST=xxx.xx.xxx.xxx LEN=44 TOS=0x00 PREC=0x00 TTL=235 ID=32940 PROTO=TCP SPT=52209 DPT=1433 WINDOW=1024 RES=0x00 SYN URGP=0

The one liner as it stands is:

cat /var/log/messages | awk '{ print $13 }' | sort | uniq -c | sort -n

The result is :
45 SRC=xxx.xxx.xxx.xxx
49 SRC=xxx.xxx.xxx.xxx
54 SRC=xxx.xxx.xxx.xxx
63 SRC=xxx.xxx.xxx.xxx

... and so on. Is there a tweak that can be made to get the IP only as well as the hits to the left?

Thanks for any help!

 

[fuzzylogic]

cat /var/log/messages | awk '/*TCP_IN Blocked*/ {print}' | awk '{ print $13 }' | awk '{ sub(/SRC=/, ""); print }' | sort | uniq -c | sort -n

Seems to work.
Someone who knows what they are doing may be able to acheive the same while only invoking awk once.

Also you could use sort -nr instead of sort -n if you want the major offenders at the top of the list.

 

[cPanelLauren]

I have a different log level than you but using cut with a delimeter of the = and printing only what comes after should work so just add:

cat /var/log/messages | awk '{ print $13 }' | cut -d= -f2 |sort | uniq -c | sort -n

should work but you'll get a lot more than firewall blocks with that.

I looked for firewall and on my server the awk field I'm printing for the same thing is 12 instead of 13 so it looks like this:

[[email protected] backups]# grep Firewall /var/log/messages | awk '{ print $12 }' | cut -d= -f2 |sort | uniq -c | sort -nr
    474 185.156.73.54
    140 185.176.27.254
    137 185.156.73.52
    134 185.176.27.174
    122 194.26.29.114

You can add multiple fields to print with awk as well rather than piping them each out. For example if I want to see if they're connecting via TCP or UDP

[[email protected] backups]# grep Firewall /var/log/messages | awk '{ print $12,$7 }' | cut -d= -f2 |cut -d_ -f1|sort | uniq -c | sort -nr
    474 185.156.73.54 *TCP
    140 185.176.27.254 *TCP
    137 185.156.73.52 *TCP
    134 185.176.27.174 *TCP
    123 194.26.29.114 *TCP

You can also pretty effectively use sed for formatting as well but cut seemed easier to me for this

 

[bloatedstoat]

Thanks @fuzzylogic and @cPanelLauren for your help in such a short space of time.

The solution and the free lesson much appreciated.

 

[fuzzylogic]

awk's default is to print all of each line that matches the pattern, so we can remove 2 instances of the print command.
Also results can be limited to have a count greater than a chosen number.
So here is a revised version...

cat /var/log/messages | awk '/*TCP_IN Blocked*/' | awk '{ print $13 }' | awk 'sub(/SRC=/, "")' | sort | uniq -c | sort -nr | awk '$1 >= 100'