複習一下Linux命令awk的使用

好久不用linux命令，好多都忘記了，只是經常使用的記得比較比如如tail、 grep、 top、 touch、 ps-ef、sed等，用的少的日久失修，因此人腦不像計算機，計算機磁盤滿了手動清除才能存下新的文件。可是人腦畢竟不是計算機，遺忘是天然界賦予人的腦的自動清除算法。。。 下面回憶一下awk的統計日誌用法:

1.下面我有一個rockmq的日誌

hxr-mac:rocketmqlogs houxiurong$ cat namesrv_default.log
2019-09-27 17:51:49 INFO main - instrumented a special java.util.Set into: sun.nio.ch.KQueueSelectorImpl@1990a65e
2019-09-27 17:51:49 INFO main - instrumented a special java.util.Set into: sun.nio.ch.KQueueSelectorImpl@150c158
2019-09-27 17:51:49 INFO main - instrumented a special java.util.Set into: sun.nio.ch.KQueueSelectorImpl@4524411f
2019-09-27 17:51:49 INFO main - instrumented a special java.util.Set into: sun.nio.ch.KQueueSelectorImpl@401e7803
2019-09-27 17:53:02 INFO main - instrumented a special java.util.Set into: sun.nio.ch.KQueueSelectorImpl@1990a65e
2019-09-27 17:53:02 INFO main - instrumented a special java.util.Set into: sun.nio.ch.KQueueSelectorImpl@150c158
2019-09-27 17:53:02 INFO main - instrumented a special java.util.Set into: sun.nio.ch.KQueueSelectorImpl@4524411f
2019-09-27 17:53:02 INFO main - instrumented a special java.util.Set into: sun.nio.ch.KQueueSelectorImpl@401e7803
複製代碼

統計@後面字符串出現的數量：
好比:

2 | 401e7803
2 | 4524411f
複製代碼

下面是執行命令:

hxr-mac:rocketmqlogs houxiurong$ cat namesrv_default.log | awk -F '(into:|@)' '{print $3}'| sort | uniq -c
   2 150c158
   2 1990a65e
   2 401e7803
   2 4524411f
hxr-mac:rocketmqlogs houxiurong$   
複製代碼

總結

命令說明:
1.從每一行取出須要的字符串並分割 awk -F '(into:|@)' '{print $1}' 按照into: 和 @符號分割，輸出結果

hxr-mac:rocketmqlogs houxiurong$ cat namesrv_default.log |awk -F '(into:|@)' '{print $1}'
2019-09-27 17:51:49 INFO main - instrumented a special java.util.Set
2019-09-27 17:51:49 INFO main - instrumented a special java.util.Set
2019-09-27 17:51:49 INFO main - instrumented a special java.util.Set
2019-09-27 17:51:49 INFO main - instrumented a special java.util.Set
2019-09-27 17:53:02 INFO main - instrumented a special java.util.Set
2019-09-27 17:53:02 INFO main - instrumented a special java.util.Set
2019-09-27 17:53:02 INFO main - instrumented a special java.util.Set
2019-09-27 17:53:02 INFO main - instrumented a special java.util.Set
hxr-mac:rocketmqlogs houxiurong$
複製代碼

2.分割後第二個字段值

hxr-mac:rocketmqlogs houxiurong$ cat namesrv_default.log |awk -F '(into:|@)' '{print $2}'
 sun.nio.ch.KQueueSelectorImpl
 sun.nio.ch.KQueueSelectorImpl
 sun.nio.ch.KQueueSelectorImpl
 sun.nio.ch.KQueueSelectorImpl
 sun.nio.ch.KQueueSelectorImpl
 sun.nio.ch.KQueueSelectorImpl
 sun.nio.ch.KQueueSelectorImpl
 sun.nio.ch.KQueueSelectorImpl
hxr-mac:rocketmqlogs houxiurong$
複製代碼

3.分割後第三個字段值,這個字段纔是須要的字段

hxr-mac:rocketmqlogs houxiurong$ cat namesrv_default.log |awk -F '(into:|@)' '{print $3}'
1990a65e
150c158
4524411f
401e7803
1990a65e
150c158
4524411f
401e7803
hxr-mac:rocketmqlogs houxiurong$
複製代碼

4.第三個字段可是裏面有重複的值，要怎麼處理了？使用管道| sort排序

hxr-mac:rocketmqlogs houxiurong$ cat namesrv_default.log |awk -F '(into:|@)' '{print $3}' | sort
150c158
150c158
1990a65e
1990a65e
401e7803
401e7803
4524411f
4524411f
hxr-mac:rocketmqlogs houxiurong$
複製代碼

經過==分割三個字段的值並展現:

hxr-mac:rocketmqlogs houxiurong$ cat namesrv_default.log |awk -F '(into:|@)' '{print "=="$1,"=="$2,"=="$3}' | sort |uniq -c
   1 ==2019-09-27 17:51:49 INFO main - instrumented a special java.util.Set  == sun.nio.ch.KQueueSelectorImpl ==150c158
   1 ==2019-09-27 17:51:49 INFO main - instrumented a special java.util.Set  == sun.nio.ch.KQueueSelectorImpl ==1990a65e
   1 ==2019-09-27 17:51:49 INFO main - instrumented a special java.util.Set  == sun.nio.ch.KQueueSelectorImpl ==401e7803
   1 ==2019-09-27 17:51:49 INFO main - instrumented a special java.util.Set  == sun.nio.ch.KQueueSelectorImpl ==4524411f
   1 ==2019-09-27 17:53:02 INFO main - instrumented a special java.util.Set  == sun.nio.ch.KQueueSelectorImpl ==150c158
   1 ==2019-09-27 17:53:02 INFO main - instrumented a special java.util.Set  == sun.nio.ch.KQueueSelectorImpl ==1990a65e
   1 ==2019-09-27 17:53:02 INFO main - instrumented a special java.util.Set  == sun.nio.ch.KQueueSelectorImpl ==401e7803
   1 ==2019-09-27 17:53:02 INFO main - instrumented a special java.util.Set  == sun.nio.ch.KQueueSelectorImpl ==4524411f
複製代碼

sort 的 -n：依照數值的大小排序；-r 按照相反順序排列

排序後還不是我須要的值，須要統計這個值出現的次數，能夠使用
uniq經常使用命令字段:
-c 在輸出行前面加上每行在輸入文件中出現的次數。 -d 僅顯示重複行。 -u 僅顯示不重複的行。

5.將結果輸出到文件中

hxr-mac:rocketmqlogs houxiurong$ cat namesrv_default.log |awk -F '(into:|@)' '{print $3}' | sort |uniq -c | sort -nr > testfile
hxr-mac:rocketmqlogs houxiurong$ ls
namesrv_default.log	testfile
hxr-mac:rocketmqlogs houxiurong$ cat testfile
   2 4524411f
   2 401e7803
   2 1990a65e
   2 150c158
複製代碼

複習awk命令結束。。。