22. 三劍客之a(chǎn)wk

1. AWK基礎(chǔ)

1.1 AWK工作原理和基本用法說明

AWK：Aho, Weinberger, Kernighan样眠，報告生成器脑奠，格式化文本輸出图呢，GNU/Linux發(fā)布的AWK目前由自由軟件基金會（FSF）進(jìn)行開發(fā)和維護(hù)蜂筹，通常也稱它為 GNU AWK

有多種版本:

AWK：原先來源于 AT & T 實(shí)驗(yàn)室的的AWK
NAWK：New awk需纳，AT & T 實(shí)驗(yàn)室的AWK的升級版
GAWK：即GNU AWK. 所有的GNU/Linux發(fā)布版都自帶GAWK，它與AWK和NAWK完全兼容

gawk：模式掃描和處理語言艺挪，可以實(shí)現(xiàn)下面功能

文本處理
輸出格式化的文本報表
執(zhí)行算數(shù)運(yùn)算
執(zhí)行字符串操作

格式:

awk [options]   'program' var=value   file…
awk [options]   -f programfile var=value file…

說明：

program通常是被放在單引號中不翩，并可以由三種部分組成

BEGIN語句塊
模式匹配的通用語句塊
END語句塊

格式:

awk 選項(xiàng) PATTERN'BEGIN{BEGIN ACTION}{文本處理 ACTION}ENG{END ACTION}' 文件路徑

常見選項(xiàng):

-F “分隔符” 指明輸入時用到的字段分隔符，默認(rèn)的分隔符是若干個連續(xù)空白符
-v var=value 變量賦值; 即可定義內(nèi)置變量, 也可定義自定義變量

Program格式:

pattern{action statements;..}

pattern：決定動作語句何時觸發(fā)及觸發(fā)事件麻裳，比如：BEGIN,END,正則表達(dá)式等

如果省略了pattern, 那么就是對所有行做處理

action statements：對數(shù)據(jù)進(jìn)行處理口蝠，放在{}內(nèi)指明，常見：print, printf

如果省略了action, 那么就是對所有列做處理

范例: 省略pattern和action. 如果省略了action, 那么program內(nèi)的關(guān)系表達(dá)式必須返回真(非0值, 非空字符串). 否則不會對文本處理

[root@demo-c8 ~]# awk '' /etc/fstab

[root@demo-c8 ~]# awk '0' /etc/fstab 
[root@demo-c8 ~]# awk '1' /etc/fstab 

#
# /etc/fstab
# Created by anaconda on Mon Aug 15 16:52:19 2022
#
# Accessible filesystems, by reference, are maintained under '/dev/disk/'.
# See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info.
#
# After editing this file, run 'systemctl daemon-reload' to update systemd
# units generated from this file.
#
UUID=b1ab1ace-2582-4afd-8693-39bd9855041c /                       xfs     defaults        0 0
UUID=d5131695-82b3-4a23-bc28-5c8a4bf381a0 /boot                   ext4    defaults        1 2
UUID=bdd66510-e510-4fe7-ba71-e2a35e6dc492 /data                   xfs     defaults        0 0
UUID=05c944fb-d6f9-4544-ba10-8b7bf3cc8fed swap                    swap    defaults        0 0

[root@demo-c8 ~]# awk '"hello"' /etc/fstab 

#
# /etc/fstab
# Created by anaconda on Mon Aug 15 16:52:19 2022
#
# Accessible filesystems, by reference, are maintained under '/dev/disk/'.
# See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info.
#
# After editing this file, run 'systemctl daemon-reload' to update systemd
# units generated from this file.
#
UUID=b1ab1ace-2582-4afd-8693-39bd9855041c /                       xfs     defaults        0 0
UUID=d5131695-82b3-4a23-bc28-5c8a4bf381a0 /boot                   ext4    defaults        1 2
UUID=bdd66510-e510-4fe7-ba71-e2a35e6dc492 /data                   xfs     defaults        0 0
UUID=05c944fb-d6f9-4544-ba10-8b7bf3cc8fed swap                    swap    defaults        0 0

awk工作過程:

image.png

第一步：執(zhí)行BEGIN{action;… }語句塊中的語句

第二步：從文件或標(biāo)準(zhǔn)輸入(stdin)讀取一行津坑，然后執(zhí)行pattern{ action;… }語句塊妙蔗，逐行掃描文件，從第一行到最后一行重復(fù)這個過程疆瑰，直到文件全部被讀取完畢

第三步：當(dāng)讀至輸入流末尾時眉反，執(zhí)行END{action;…}語句塊

BEGIN語句塊在awk開始從輸入流中讀取行之前被執(zhí)行，這是一個可選的語句塊穆役，比如變量初始化寸五、打印輸出表格的表頭等語句通常可以寫在BEGIN語句塊中

END語句塊在awk從輸入流中讀取完所有的行之后即被執(zhí)行耿币，比如打印所有行的分析結(jié)果這類信息匯總都是在END語句塊中完成梳杏，它也是一個可選語句塊

pattern語句塊中的通用命令是最重要的部分，也是可選的. 如果沒有提供pattern語句塊掰读，則默認(rèn)執(zhí)行{ print }秘狞，即打印每一個讀取到的行，awk讀取的每一行都會執(zhí)行該語句塊

分隔符, 域和記錄:

awk會把讀入的文件或者標(biāo)準(zhǔn)輸入, 當(dāng)做一個表格格式來處理. 默認(rèn)按照\n來區(qū)分兩行, 當(dāng)然也可以自定義如何劃分不同的行. 比如: 自定義;為分隔符, 那么;前面的為一行, ;后面的為一行

由分隔符分隔的字段（列column,域field）標(biāo)記$1,$2...$n稱為域標(biāo)識蹈集，$0為所有域烁试，注意：和Shell中變量$符含義不同

$1: 第一列
$2: 第二列
...
$0: 所有列

文件的每一行稱為記錄record
如果省略action，則默認(rèn)執(zhí)行 print $0 的操作, 也就是對所有列, 做處理

常用的action分類:

output statements：print,printf
Expressions：算術(shù)拢肆，比較表達(dá)式等
Compound statements：組合語句
Control statements：if, while等
input statements

awk控制語句:

{ statements;… } 組合語句
if(condition) {statements;…}
if(condition) {statements;…} else {statements;…}
while(conditon) {statments;…}
do {statements;…} while(condition)
for(expr1;expr2;expr3) {statements;…}
break
continue
exit

1.2 動作print

格式:

print item1, item2, ...

說明:

逗號分隔符
輸出item可以是字符串减响，也可是數(shù)值靖诗；當(dāng)前記錄的字段、變量或awk的表達(dá)式
如果省略item支示，相當(dāng)于print $0
固定字符需要用""引起來刊橘，而變量和數(shù)字不需要

abc: 變量
"abc": 純字符串

范例: print默認(rèn)會對傳給awk的標(biāo)準(zhǔn)輸入做打印, 也就是打印整行, $0

root@u18:~# awk '{print}'
aa
aa
bb
bb
cc
cc

root@u18:~# cat /etc/fstab | awk '{print}'
# /etc/fstab: static file system information.
#
# Use 'blkid' to print the universally unique identifier for a
# device; this may be used with UUID= as a more robust way to name devices
# that works even if disks are added and removed. See fstab(5).
#
# <file system> <mount point>   <type>  <options>       <dump>  <pass>
# / was on /dev/sda1 during installation
UUID=f906b5aa-3e5b-4e12-8f11-55f55c41e1b0 /               ext4    errors=remount-ro 0       1
# /boot was on /dev/sda2 during installation
UUID=24239793-a342-4d7f-8773-e7381727a5dd /boot           ext4    defaults        0       2
# /data was on /dev/sda4 during installation
UUID=a328accf-9575-4343-976b-751c27cdb8ec /data           ext4    defaults        0       2
# swap was on /dev/sda5 during installation
UUID=0f94202e-4796-4835-b329-75425a807dcd none            swap    sw              0       0

root@u18:~# awk '{print}' < /etc/fstab 
# /etc/fstab: static file system information.
#
# Use 'blkid' to print the universally unique identifier for a
# device; this may be used with UUID= as a more robust way to name devices
# that works even if disks are added and removed. See fstab(5).
#
# <file system> <mount point>   <type>  <options>       <dump>  <pass>
# / was on /dev/sda1 during installation
UUID=f906b5aa-3e5b-4e12-8f11-55f55c41e1b0 /               ext4    errors=remount-ro 0       1
# /boot was on /dev/sda2 during installation
UUID=24239793-a342-4d7f-8773-e7381727a5dd /boot           ext4    defaults        0       2
# /data was on /dev/sda4 during installation
UUID=a328accf-9575-4343-976b-751c27cdb8ec /data           ext4    defaults        0       2
# swap was on /dev/sda5 during installation
UUID=0f94202e-4796-4835-b329-75425a807dcd none            swap    sw              0       0

范例: awk可以直接打印文件的全部內(nèi)容

root@u18:~# awk '{print}' /etc/fstab 
# /etc/fstab: static file system information.
#
# Use 'blkid' to print the universally unique identifier for a
# device; this may be used with UUID= as a more robust way to name devices
# that works even if disks are added and removed. See fstab(5).
#
# <file system> <mount point>   <type>  <options>       <dump>  <pass>
# / was on /dev/sda1 during installation
UUID=f906b5aa-3e5b-4e12-8f11-55f55c41e1b0 /               ext4    errors=remount-ro 0       1
# /boot was on /dev/sda2 during installation
UUID=24239793-a342-4d7f-8773-e7381727a5dd /boot           ext4    defaults        0       2
# /data was on /dev/sda4 during installation
UUID=a328accf-9575-4343-976b-751c27cdb8ec /data           ext4    defaults        0       2
# swap was on /dev/sda5 during installation
UUID=0f94202e-4796-4835-b329-75425a807dcd none            swap    sw              0       0

范例: 打印固定字符串. print接固定字符串, 就是打印固定內(nèi)容

root@u18:~# awk '{print "hello awk"}'
aaa
hello awk
vvv
hello awk
ccc
hello awk

# seq 10, 表示awk打印print的固定字符串10次
root@u18:~# seq 10 | awk '{print "hello awk"}'
hello awk
hello awk
hello awk
hello awk
hello awk
hello awk
hello awk
hello awk
hello awk
hello awk

范例: 域分隔符, 默認(rèn)為連續(xù)的空格. 分隔時默認(rèn)會把多個連續(xù)的空格, 壓縮成一個

root@u18:~# df | awk '{print $5}' 
Use%
0%
1%
3%
0%
0%
0%
9%
1%
0%

root@u18:~# df | awk '{print $5 }' | awk -F'%' '{print $1}'
Use
0
1
3
0
0
0
9
1
0

范例: 自定義域分隔符, -F選項(xiàng)

root@u18:~# awk -F":" '{print $1,$3}' /etc/passwd
root 0
daemon 1
bin 2
sys 3
sync 4
games 5
...

范例: 文本分隔后, 默認(rèn)會用空格作為列的分隔符. 也可以指定新的分隔符

# 新的分隔符在print里, 必須用雙引號括起來
root@u18:~# awk -F':' '{print $1":"$3}' /etc/passwd
root:0
daemon:1
bin:2
sys:3
sync:4
games:5
...

范例: 指定table鍵為輸出時的分隔符

root@u18:~# awk -F: '{print $1"\t"$3}' /etc/passwd
root    0
daemon  1
bin 2
sys 3
sync    4
games   5
man 6
lp  7
mail    8
news    9
...

范例: 統(tǒng)計一個網(wǎng)站訪問量最大的前5個ip

root@u18:~# sed -nr '1,5p' access_log 
172.18.118.91 - - [20/May/2018:08:09:59 +0800] "GET / HTTP/1.1" 200 912 "-" "Mozilla/4.0 (compatible; MSIE 9.0; Windows NT 5.1; Trident/5.0)"
172.18.118.91 - - [20/May/2018:08:09:59 +0800] "POST /webnoauth/model.cgi HTTP/1.1" 404 293 "http://172.18.0.1/webnoauth/model.cgi" "Mozilla/4.0 (compatible; MSIE 9.0; Windows NT 5.1; Trident/5.0)"
172.18.118.91 - - [20/May/2018:08:09:59 +0800] "GET /router/get_rand_key.cgi HTTP/1.1" 404 297 "http://172.18.0.1/router/get_rand_key.cgi" "Mozilla/4.0 (compatible; MSIE 9.0; Windows NT 5.1; Trident/5.0)"
172.18.118.91 - - [20/May/2018:08:09:59 +0800] "GET /router/get_rand_key.cgi HTTP/1.1" 404 297 "http://172.18.0.1/router/get_rand_key.cgi" "Mozilla/4.0 (compatible; MSIE 9.0; Windows NT 5.1; Trident/5.0)"
172.18.118.91 - - [20/May/2018:08:09:59 +0800] "GET /router/get_rand_key.cgi HTTP/1.1" 404 297 "http://172.18.0.1/router/get_rand_key.cgi" "Mozilla/4.0 (compatible; MSIE 9.0; Windows NT 5.1; Trident/5.0)"

root@u18:~# awk '{print $1}' access_log | sort  | uniq -c | sort -k1 -nr | head -n5 
   4870 172.20.116.228
   3429 172.20.116.208
   2834 172.20.0.222
   2613 172.20.112.14
   2267 172.20.0.227

范例: 分隔文本時, 可以指定多個分隔符. 此時, 凡是被指定分隔符隔開的, 都是單獨(dú)的一列. 這樣可以避免因?yàn)榉指舴煌? 需要多次對文本進(jìn)行處理的情況

# [[:space:]]+|%: 擴(kuò)展正則表達(dá)式, 表示一個以上包括一個空格, 或者%都作為分隔符
root@u18:~# df | awk -F"[[:space:]]+|%" '{print $5}'
Use
0
1
3
0
0
0
9
1
0

范例: 文件host_list.log 如下格式，請?zhí)崛　?xxx.com”前面的主機(jī)名部分并寫回到該文件中

1 www.xxx.com
2 blog.xxx.com
3 study.xxx.com
4 linux.xxx.com
5 python.xxx.com

root@u18:/opt# awk -F"[ .]" '{print $2}' host_list.org  
www
blog
study
linux
python
root@u18:/opt# awk -F"[ .]" '{print $2}' host_list.org  >> host_list.org
root@u18:/opt# cat host_list.org
1 www.xxx.com
2 blog.xxx.com
3 study.xxx.com
4 linux.xxx.com
5 python.xxx.com
www
blog
study
linux
python

1.3 AWK變量

awk中的變量分為：內(nèi)置和自定義變量

變量定義格式:

-v var=value 變量賦值; 即可定義內(nèi)置變量, 也可定義自定義變量

awk變量的引用格式:

-v FS=":"
引用awk變量不用寫$, 直接寫FS即可

常見的內(nèi)置變量:

FS(Field Separater): 輸入字段分隔符颂鸿，默認(rèn)為空格,功能相當(dāng)于 -F. 但是-F 和 FS變量功能一樣促绵，同時使用會沖突

范例: 指定FS為":"

root@u18:/opt# awk -v FS=":" '{print $1,$2}' /etc/passwd
root x
daemon x
bin x
sys x
sync x
games x
...

范例: 輸出分隔符變量也可引用輸入分隔符變量

root@u18:/opt# awk -v FS=":" '{print $1FS$2}' /etc/passwd
root:x
daemon:x
bin:x
sys:x
sync:x
games:x
...

范例: awk也可以引用Shell的變量

root@u18:/opt# var=":"; awk -v FS=$var '{print $1FS$2}' /etc/passwd
root:x
daemon:x
bin:x
sys:x
sync:x
games:x
...

OFS: 輸出字段分隔符，默認(rèn)為空格

范例: 指定OFS為"===="

root@u18:/opt# awk -F":" -v  OFS="======" '{print $1,$2}' /etc/passwd
root======x
daemon======x
bin======x
sys======x
sync======x
games======x
...

root@u18:/opt# awk -v FS=":" -v  OFS="======" '{print $1, $2}' /etc/passwd
root======x
daemon======x
bin======x
sys======x
...

RS：輸入記錄record分隔符嘴纺，指定輸入時的換行符. 默認(rèn)為\n

范例: 指定";"為record分隔符

root@u18:/opt# vim f1.txt
a,b,c;11,22
33,44;xx,yy,zz
m,n;xxx

# 指定分號為record分隔符, 那么a b c是一行, 11 22 33 44是一行, xx yy zz m n是一行, xxx是一行
# 但是因?yàn)?2, zz, xxx后本身就有換行符, 會繼續(xù)保留, 所以才又換一次行
root@u18:/opt# awk -v RS=";" '{print $0}' f1.txt 
a,b,c
11,22  # 文本中, 22后面有換行符, 會繼續(xù)保留
33,44
xx,yy,zz
m,n
xxx # xxx后有換行符

root@u18:/opt#

NR：打印record的編號

范例: NR變量可以顯示record編號, 用于區(qū)分awk的record和Shell本身的換行

root@u18:/opt# awk -v RS=";" '{print NR,$0}' f1.txt 
1 a,b,c  # record1
2 11,22 # record 2
33,44
3 xx,yy,zz # record3
m,n
4 xxx # record4

root@u18:/opt#

ORS：輸出記錄分隔符败晴，輸出時用指定符號代替換行符. 默認(rèn)會用換行符

范例: 指定"+++"為輸出時record換行符

root@u18:/opt# awk -v RS=";" -v ORS="+++" '{print NR,$0}' f1.txt 
1 a,b,c+++2 11,22
33,44+++3 xx,yy,zz
m,n+++4 xxx
+++root@u18:/opt#

NF：字段數(shù)量

范例: NF顯示的是, 根據(jù)輸入列分隔符分隔后, 每一行有多少個字段. 所以一共有幾行, 就會返回幾行

root@u18:/opt# awk -F":" '{print NF}' /etc/passwd
root@u18:/opt# awk -F":" '{print NF}' /etc/passwd
7
7
7
7
7
7
7
7
7
7
7
7
7
...

$NF: 代表最后一個字段

root@u18:/opt# awk -F":" '{print $NF}' /etc/passwd
/bin/bash
/usr/sbin/nologin
/usr/sbin/nologin
/usr/sbin/nologin
...

$(NF-1): 代表倒數(shù)第二個字段

root@u18:/opt# cat /etc/passwd
root:x:0:0:root:/root:/bin/bash
daemon:x:1:1:daemon:/usr/sbin:/usr/sbin/nologin
bin:x:2:2:bin:/bin:/usr/sbin/nologin
sys:x:3:3:sys:/dev:/usr/sbin/nologin
sync:x:4:65534:sync:/bin:/bin/sync
games:x:5:60:games:/usr/games:/usr/sbin/nologin
man:x:6:12:man:/var/cache/man:/usr/sbin/nologin
...

root@u18:/opt# awk -F":" '{print $(NF-1)}' /etc/passwd
/root
/usr/sbin
/bin
/dev
/bin
/usr/games
/var/cache/man
/var/spool/lpd
/var/mail
/var/spool/news
/var/spool/uucp
/bin
...

范例: 取源碼包的版本信息

root@u18:/opt# tar cvf app.v1.tar.gz /etc
root@u18:/opt# tar cvf app.v2.tar.gz /etc
root@u18:/opt# tar cvf app.v3.tar.gz /etc
root@u18:/opt# tar cvf app.v4.tar.gz /etc
root@u18:/opt# tar cvf app.v5.tar.gz /etc
root@u18:/opt# ll app*
-rw-r--r-- 1 root root 3225600 Sep 17 22:06 app.v1.tar.gz
-rw-r--r-- 1 root root 3225600 Sep 17 22:09 app.v2.tar.gz
-rw-r--r-- 1 root root 3225600 Sep 17 22:09 app.v3.tar.gz
-rw-r--r-- 1 root root 3225600 Sep 17 22:09 app.v4.tar.gz
-rw-r--r-- 1 root root 3225600 Sep 17 22:09 app.v5.tar.gz

# 以"."為輸入分隔符, 取倒數(shù)第三個字段, 即為版本號

root@u18:/opt# ls app.* | xargs -n1 |  awk -F"." '{print $(NF-2)}'
v1
v2
v3
v4
v5

# 這里其實(shí)不用xargs進(jìn)行轉(zhuǎn)置, 因?yàn)閘s的結(jié)果雖然是以橫行顯示, 但本身是豎著的

root@u18:/opt# ls app.* |  awk -F"." '{print $(NF-2)}'
v1
v2
v3
v4
v5

FNR：各文件分別計數(shù)記錄的編號. 默認(rèn)情況下, 多個文件會統(tǒng)一編號

范例: 利用FNR分別統(tǒng)計各文件記錄的編號

root@u18:/opt# awk -F":" '{print FNR,$1}' /etc/passwd /etc/group
1 root
2 daemon
3 bin
4 sys
5 sync
6 games
7 man
8 lp
9 mail
10 news
11 uucp
12 proxy
13 www-data
14 backup
15 list
16 irc
17 gnats
18 nobody
19 systemd-network
20 systemd-resolve
21 syslog
22 messagebus
23 _apt
24 lxd
25 uuidd
26 dnsmasq
27 landscape
28 sshd
29 pollinate
30 david
1 root
2 daemon
3 bin
4 sys
5 adm
6 tty
7 disk
8 lp
9 mail
10 news
11 uucp
12 man
13 proxy
14 kmem
15 dialout
16 fax
17 voice
18 cdrom
19 floppy
20 tape
21 sudo
22 audio
23 dip
24 www-data
25 backup
26 operator
27 list
28 irc
29 src
30 gnats
31 shadow
32 utmp
33 video
34 sasl
35 plugdev
36 staff
37 games
38 users
39 nogroup
40 systemd-journal
41 systemd-network
42 systemd-resolve
43 input
44 crontab
45 syslog
46 messagebus
47 lxd
48 mlocate
49 uuidd
50 ssh
51 landscape
52 david
53 lpadmin
54 sambashare

FILENAME：返回當(dāng)前文件名, 配合FNR使用, 返回的record編號會標(biāo)記屬于哪個文件

root@u18:/opt# awk -F":" '{print FNR,$1,FILENAME}' /etc/passwd /etc/group
1 root /etc/passwd
2 daemon /etc/passwd
3 bin /etc/passwd
4 sys /etc/passwd
5 sync /etc/passwd
6 games /etc/passwd
7 man /etc/passwd
8 lp /etc/passwd
9 mail /etc/passwd
10 news /etc/passwd
11 uucp /etc/passwd
12 proxy /etc/passwd
13 www-data /etc/passwd
14 backup /etc/passwd
15 list /etc/passwd
16 irc /etc/passwd
17 gnats /etc/passwd
18 nobody /etc/passwd
19 systemd-network /etc/passwd
20 systemd-resolve /etc/passwd
21 syslog /etc/passwd
22 messagebus /etc/passwd
23 _apt /etc/passwd
24 lxd /etc/passwd
25 uuidd /etc/passwd
26 dnsmasq /etc/passwd
27 landscape /etc/passwd
28 sshd /etc/passwd
29 pollinate /etc/passwd
30 david /etc/passwd
1 root /etc/group
2 daemon /etc/group
3 bin /etc/group
4 sys /etc/group
5 adm /etc/group
6 tty /etc/group
7 disk /etc/group
8 lp /etc/group
9 mail /etc/group
10 news /etc/group
11 uucp /etc/group
12 man /etc/group
13 proxy /etc/group
14 kmem /etc/group
15 dialout /etc/group
16 fax /etc/group
17 voice /etc/group
18 cdrom /etc/group
19 floppy /etc/group
20 tape /etc/group
21 sudo /etc/group
22 audio /etc/group
23 dip /etc/group
24 www-data /etc/group
25 backup /etc/group
26 operator /etc/group
27 list /etc/group
28 irc /etc/group
29 src /etc/group
30 gnats /etc/group
31 shadow /etc/group
32 utmp /etc/group
33 video /etc/group
34 sasl /etc/group
35 plugdev /etc/group
36 staff /etc/group
37 games /etc/group
38 users /etc/group
39 nogroup /etc/group
40 systemd-journal /etc/group
41 systemd-network /etc/group
42 systemd-resolve /etc/group
43 input /etc/group
44 crontab /etc/group
45 syslog /etc/group
46 messagebus /etc/group
47 lxd /etc/group
48 mlocate /etc/group
49 uuidd /etc/group
50 ssh /etc/group
51 landscape /etc/group
52 david /etc/group
53 lpadmin /etc/group
54 sambashare /etc/group

ARGC：返回awk命令行參數(shù)的個數(shù)

范例: ARGV返回awk命令參數(shù)個數(shù), '為第一個', /etc/passwd為第二個, /etc/group為第三個

root@u18:/opt# awk -F":" '{print ARGC,FNR,$1,FILENAME}' /etc/passwd /etc/group
3 1 root /etc/passwd
3 2 daemon /etc/passwd
3 3 bin /etc/passwd
3 4 sys /etc/passwd
3 5 sync /etc/passwd
3 6 games /etc/passwd
3 7 man /etc/passwd
3 8 lp /etc/passwd
3 9 mail /etc/passwd
3 10 news /etc/passwd
3 11 uucp /etc/passwd
3 12 proxy /etc/passwd
3 13 www-data /etc/passwd
3 14 backup /etc/passwd
3 15 list /etc/passwd
3 16 irc /etc/passwd
3 17 gnats /etc/passwd
3 18 nobody /etc/passwd
3 19 systemd-network /etc/passwd
3 20 systemd-resolve /etc/passwd
3 21 syslog /etc/passwd
3 22 messagebus /etc/passwd
3 23 _apt /etc/passwd
3 24 lxd /etc/passwd
3 25 uuidd /etc/passwd
3 26 dnsmasq /etc/passwd
3 27 landscape /etc/passwd
3 28 sshd /etc/passwd
3 29 pollinate /etc/passwd
3 30 david /etc/passwd
3 1 root /etc/group
3 2 daemon /etc/group
3 3 bin /etc/group
3 4 sys /etc/group
3 5 adm /etc/group
3 6 tty /etc/group
3 7 disk /etc/group
3 8 lp /etc/group
3 9 mail /etc/group
3 10 news /etc/group
3 11 uucp /etc/group
3 12 man /etc/group
3 13 proxy /etc/group
3 14 kmem /etc/group
3 15 dialout /etc/group
3 16 fax /etc/group
3 17 voice /etc/group
3 18 cdrom /etc/group
3 19 floppy /etc/group
3 20 tape /etc/group
3 21 sudo /etc/group
3 22 audio /etc/group
3 23 dip /etc/group
3 24 www-data /etc/group
3 25 backup /etc/group
3 26 operator /etc/group
3 27 list /etc/group
3 28 irc /etc/group
3 29 src /etc/group
3 30 gnats /etc/group
3 31 shadow /etc/group
3 32 utmp /etc/group
3 33 video /etc/group
3 34 sasl /etc/group
3 35 plugdev /etc/group
3 36 staff /etc/group
3 37 games /etc/group
3 38 users /etc/group
3 39 nogroup /etc/group
3 40 systemd-journal /etc/group
3 41 systemd-network /etc/group
3 42 systemd-resolve /etc/group
3 43 input /etc/group
3 44 crontab /etc/group
3 45 syslog /etc/group
3 46 messagebus /etc/group
3 47 lxd /etc/group
3 48 mlocate /etc/group
3 49 uuidd /etc/group
3 50 ssh /etc/group
3 51 landscape /etc/group
3 52 david /etc/group
3 53 lpadmin /etc/group
3 54 sambashare /etc/group

ARGV：數(shù)組，保存的是命令行所給定的各參數(shù)栽渴，返回某一個參數(shù)用：ARGV[0]尖坤，....... ARGV[0]返回awk命令本身, 超出參數(shù)個數(shù), 返回空白

范例: ARGV返回第n個參數(shù)

root@u18:/opt# awk -F":" '{print ARGV[0]}' /etc/passwd /etc/group
awk
awk
awk
awk
awk
awk
awk
awk
awk
awk
awk
awk
awk
awk
awk
awk
awk
awk
awk
awk
awk
awk
awk
awk
awk
awk
awk
awk
awk
awk
awk
awk
awk
awk
awk
awk
awk
awk
awk
awk
awk
awk
awk
awk
awk
awk
awk
awk
awk
awk
awk
awk
awk
awk
awk
awk
awk
awk
awk
awk
awk
awk
awk
awk
awk
awk
awk
awk
awk
awk
awk
awk
awk
awk
awk
awk
awk
awk
awk
awk
awk
awk
awk
awk
root@u18:/opt# awk -F":" '{print ARGV[1]}' /etc/passwd /etc/group
/etc/passwd
/etc/passwd
/etc/passwd
/etc/passwd
/etc/passwd
/etc/passwd
/etc/passwd
/etc/passwd
/etc/passwd
/etc/passwd
/etc/passwd
/etc/passwd
/etc/passwd
/etc/passwd
/etc/passwd
/etc/passwd
/etc/passwd
/etc/passwd
/etc/passwd
/etc/passwd
/etc/passwd
/etc/passwd
/etc/passwd
/etc/passwd
/etc/passwd
/etc/passwd
/etc/passwd
/etc/passwd
/etc/passwd
/etc/passwd
/etc/passwd
/etc/passwd
/etc/passwd
/etc/passwd
/etc/passwd
/etc/passwd
/etc/passwd
/etc/passwd
/etc/passwd
/etc/passwd
/etc/passwd
/etc/passwd
/etc/passwd
/etc/passwd
/etc/passwd
/etc/passwd
/etc/passwd
/etc/passwd
/etc/passwd
/etc/passwd
/etc/passwd
/etc/passwd
/etc/passwd
/etc/passwd
/etc/passwd
/etc/passwd
/etc/passwd
/etc/passwd
/etc/passwd
/etc/passwd
/etc/passwd
/etc/passwd
/etc/passwd
/etc/passwd
/etc/passwd
/etc/passwd
/etc/passwd
/etc/passwd
/etc/passwd
/etc/passwd
/etc/passwd
/etc/passwd
/etc/passwd
/etc/passwd
/etc/passwd
/etc/passwd
/etc/passwd
/etc/passwd
/etc/passwd
/etc/passwd
/etc/passwd
/etc/passwd
/etc/passwd
/etc/passwd
root@u18:/opt# awk -F":" '{print ARGV[2]}' /etc/passwd /etc/group
/etc/group
/etc/group
/etc/group
/etc/group
/etc/group
/etc/group
/etc/group
/etc/group
/etc/group
/etc/group
/etc/group
/etc/group
/etc/group
/etc/group
/etc/group
/etc/group
/etc/group
/etc/group
/etc/group
/etc/group
/etc/group
/etc/group
/etc/group
/etc/group
/etc/group
/etc/group
/etc/group
/etc/group
/etc/group
/etc/group
/etc/group
/etc/group
/etc/group
/etc/group
/etc/group
/etc/group
/etc/group
/etc/group
/etc/group
/etc/group
/etc/group
/etc/group
/etc/group
/etc/group
/etc/group
/etc/group
/etc/group
/etc/group
/etc/group
/etc/group
/etc/group
/etc/group
/etc/group
/etc/group
/etc/group
/etc/group
/etc/group
/etc/group
/etc/group
/etc/group
/etc/group
/etc/group
/etc/group
/etc/group
/etc/group
/etc/group
/etc/group
/etc/group
/etc/group
/etc/group
/etc/group
/etc/group
/etc/group
/etc/group
/etc/group
/etc/group
/etc/group
/etc/group
/etc/group
/etc/group
/etc/group
/etc/group
/etc/group
/etc/group

自定義變量(區(qū)分字符大小寫):

-v var=value 
在program中直接定義

范例: 在處理文件之前, 先打印一次變量, 用于制作表頭等情況. BEGIN{這里的內(nèi)容, 會在處理文本之前先執(zhí)行一次}

root@u18:/opt# awk 'BEGIN{test="hello,world"; print test}'
hello,world

root@u18:/opt# awk -v test="hello,awk" 'BEGIN{print test}'
hello,awk

root@u18:/opt# awk -v test="hello,awk" 'BEGIN{print test; test="hello,world";print test}'
hello,awk
hello,world

范例: 在awk的program中, 可以使用鏈?zhǔn)劫x值, 但是在awk外部不支持

root@u18:/opt# awk  'BEGIN{test1=test2="hello,world"; print test1; print test2; print "hello,awk"}'
hello,world
hello,world
hello,awk

范例: awk打印多個字符串, 可以只寫一個print命令

root@u18:/opt# awk  'BEGIN{test1=test2="hello,world"; print test1,test2,"hello,awk"}'
hello,world hello,world hello,awk

范例: awk的變量不支持先引用, 后賦值

root@u18:/opt# awk -F":" '{sex="male";print $1,sex,age;age=18}' /etc/passwd
root male # 第一行記錄的age為空, 說明處理第一行時, 運(yùn)行到print age發(fā)現(xiàn)age沒賦值, 所以返回空
daemon male 18 # 第二行開始有age的值, 因?yàn)榈谝恍刑幚硗? age=18就執(zhí)行了, 所以第二行開始都是有age值的
bin male 18
sys male 18
sync male 18
games male 18
man male 18
lp male 18
mail male 18
news male 18
uucp male 18
proxy male 18
www-data male 18
backup male 18
list male 18
irc male 18
gnats male 18
nobody male 18
systemd-network male 18
systemd-resolve male 18
syslog male 18
messagebus male 18
_apt male 18
lxd male 18
uuidd male 18
dnsmasq male 18
landscape male 18
sshd male 18
pollinate male 18
david male 18

范例: 可以把a(bǔ)wk的program部分寫到文件里, 之后在命令中通過-f選項(xiàng)調(diào)用

root@u18:/opt# vim awk.txt
{print $(NF-1)}

root@u18:/opt# awk -F":" -f awk.txt /etc/passwd
/root
/usr/sbin
/bin
/dev
/bin
/usr/games
/var/cache/man
/var/spool/lpd
/var/mail
/var/spool/news
/var/spool/uucp
/bin
/var/www
/var/backups
/var/list
/var/run/ircd
/var/lib/gnats
/nonexistent
/run/systemd/netif
/run/systemd/resolve
/home/syslog
/nonexistent
/nonexistent
/var/lib/lxd/
/run/uuidd
/var/lib/misc
/var/lib/landscape
/run/sshd
/var/cache/pollinate
/home/david

1.4 動作printf

printf可以實(shí)現(xiàn)格式化輸出

格式:

# 在'program中書寫'
printf "FORMAT", item1, item2,...

說明:

必須指定FORMAT
不會自動換行, 需要顯示給出換行控制符\n
FORMAT中需要分別為后面每個item指定格式符

格式符: 與item一一對應(yīng)

%c：顯示字符的ASCII碼
%d, %i：顯示十進(jìn)制整數(shù)
%e, %E：顯示科學(xué)計數(shù)法數(shù)值 
%f：顯示為浮點(diǎn)數(shù)
%g, %G：以科學(xué)計數(shù)法或浮點(diǎn)形式顯示數(shù)值
%s：顯示字符串
%u：無符號整數(shù)
%%：顯示%自身

修飾符:

#[.#] 第一個數(shù)字控制顯示的寬度；第二個#表示小數(shù)點(diǎn)后精度闲擦，如：%3.1f
-     左對齊（默認(rèn)右對齊） 如：%-15s
+     顯示數(shù)值的正負(fù)符號   如：%+d

范例:

awk -F:   '{printf "%s",$1}' /etc/passwd
awk -F:   '{printf "%s\n",$1}' /etc/passwd
awk -F:   '{printf "%20s\n",$1}' /etc/passwd
awk -F:   '{printf "%-20s\n",$1}' /etc/passwd
awk -F:   '{printf "%-20s %10d\n",$1,$3}' /etc/passwd
awk -F:   '{printf "Username: %s\n",$1}' /etc/passwd
awk -F:   '{printf “Username: %sUID:%d\n",$1,$3}' /etc/passwd
awk -F:   '{printf "Username: %25sUID:%d\n",$1,$3}' /etc/passwd
awk -F:   '{printf "Username: %-25sUID:%d\n",$1,$3}' /etc/passwd

%s指代的就是$1的內(nèi)容

[19:20:08 root@centos7 ~]#awk -F: '{printf "%s", $1}' /etc/passwd
rootbindaemonadmlpsyncshutdownhaltmailoperatorgamesftpnobodysystemd-networkdbuspolkitdsshdpostfixtcpdump[19:20:38 root@centos7 ~]#

換行符需要顯示指定\n

[19:20:03 root@centos7 ~]#awk -F: '{printf "%s\n", $1}' /etc/passwd
root
bin
daemon
adm
lp
sync
shutdown
halt
mail
operator
games
ftp
nobody
systemd-network
dbus
polkitd
sshd
postfix
tcpdump

[19:21:16 root@centos7 ~]#awk -F":" '{printf "%20s\n",$1}' /etc/passwd
                root
                 bin
              daemon
                 adm
                  lp
                sync
            shutdown
                halt
                mail
            operator
               games
                 ftp
              nobody
     systemd-network
                dbus
             polkitd
                sshd
             postfix
             tcpdump

[19:22:15 root@centos7 ~]#awk -F":" '{printf "%-20s\n",$1}' /etc/passwd
root                
bin                 
daemon              
adm                 
lp                  
sync                
shutdown            
halt                
mail                
operator            
games               
ftp                 
nobody              
systemd-network     
dbus                
polkitd             
sshd                
postfix             
tcpdump

[19:22:58 root@centos7 ~]#awk -F":" '{printf "Username: %s\n",$1}' /etc/passwd
Username: root
Username: bin
Username: daemon
Username: adm
Username: lp
Username: sync
Username: shutdown
Username: halt
Username: mail
Username: operator
Username: games
Username: ftp
Username: nobody
Username: systemd-network
Username: dbus
Username: polkitd
Username: sshd
Username: postfix
Username: tcpdump

打印多列, 需要分別指定格式, 并且指定列的輸出分隔符, 否則兩列會連在一起

[19:25:32 root@centos7 ~]#awk -F":" '{printf "Username: %s | UID: %d\n", $1, $3}' /etc/passwd
Username: root | UID: 0
Username: bin | UID: 1
Username: daemon | UID: 2
Username: adm | UID: 3
Username: lp | UID: 4
Username: sync | UID: 5
Username: shutdown | UID: 6
Username: halt | UID: 7
Username: mail | UID: 8
Username: operator | UID: 11
Username: games | UID: 12
Username: ftp | UID: 14
Username: nobody | UID: 99
Username: systemd-network | UID: 192
Username: dbus | UID: 81
Username: polkitd | UID: 999
Username: sshd | UID: 74
Username: postfix | UID: 89
Username: tcpdump | UID: 72

[19:26:34 root@centos7 ~]#awk -F":" '{printf "Username: %-25s UID: %d\n", $1, $3}' /etc/passwd
Username: root                      UID: 0
Username: bin                       UID: 1
Username: daemon                    UID: 2
Username: adm                       UID: 3
Username: lp                        UID: 4
Username: sync                      UID: 5
Username: shutdown                  UID: 6
Username: halt                      UID: 7
Username: mail                      UID: 8
Username: operator                  UID: 11
Username: games                     UID: 12
Username: ftp                       UID: 14
Username: nobody                    UID: 99
Username: systemd-network           UID: 192
Username: dbus                      UID: 81
Username: polkitd                   UID: 999
Username: sshd                      UID: 74
Username: postfix                   UID: 89
Username: tcpdump                   UID: 72

[root@demo-c8 ~]# awk -F":"  '{printf "%-20s %s\n", $1,$3 }' /etc/passwd
root                 0
bin                  1
daemon               2
adm                  3
lp                   4
sync                 5
shutdown             6
halt                 7
mail                 8
operator             11
games                12
ftp                  14
nobody               65534
dbus                 81
systemd-coredump     999
systemd-resolve      193
tss                  59
polkitd              998
geoclue              997
rtkit                172
pulse                171
libstoragemgmt       996
qemu                 107
usbmuxd              113
unbound              995
rpc                  32
gluster              994
chrony               993
setroubleshoot       992
pipewire             991
saslauth             990
dnsmasq              984
radvd                75
clevis               983
cockpit-ws           982
cockpit-wsinstance   981
sssd                 980
flatpak              979
colord               978
gdm                  42
rpcuser              29
gnome-initial-setup  977
sshd                 74
avahi                70
rngd                 976
tcpdump              72
wang                 1000

打印title, 湊出表格形式

[root@demo-c8 ~]# awk -F":"  'BEGIN{print "|---------------------------|\n|       Usename&UID         |\n-----------------------------"}{printf "|%-20s|%-6s|\n-----------------------------\n", $1,$3 }' /etc/passwd 
|---------------------------|
|       Usename&UID         |
-----------------------------
|root                |0     |
-----------------------------
|bin                 |1     |
-----------------------------
|daemon              |2     |
-----------------------------
|adm                 |3     |
-----------------------------
|lp                  |4     |
-----------------------------
|sync                |5     |
-----------------------------
|shutdown            |6     |
-----------------------------
|halt                |7     |
-----------------------------
|mail                |8     |
-----------------------------
|operator            |11    |
-----------------------------
|games               |12    |
-----------------------------
|ftp                 |14    |
-----------------------------
|nobody              |65534 |
-----------------------------
|dbus                |81    |
-----------------------------
|systemd-coredump    |999   |
-----------------------------
|systemd-resolve     |193   |
-----------------------------
|tss                 |59    |
-----------------------------
|polkitd             |998   |
-----------------------------
|geoclue             |997   |
-----------------------------
|rtkit               |172   |
-----------------------------
|pulse               |171   |
-----------------------------
|libstoragemgmt      |996   |
-----------------------------
|qemu                |107   |
-----------------------------
|usbmuxd             |113   |
-----------------------------
|unbound             |995   |
-----------------------------
|rpc                 |32    |
-----------------------------
|gluster             |994   |
-----------------------------
|chrony              |993   |
-----------------------------
|setroubleshoot      |992   |
-----------------------------
|pipewire            |991   |
-----------------------------
|saslauth            |990   |
-----------------------------
|dnsmasq             |984   |
-----------------------------
|radvd               |75    |
-----------------------------
|clevis              |983   |
-----------------------------
|cockpit-ws          |982   |
-----------------------------
|cockpit-wsinstance  |981   |
-----------------------------
|sssd                |980   |
-----------------------------
|flatpak             |979   |
-----------------------------
|colord              |978   |
-----------------------------
|gdm                 |42    |
-----------------------------
|rpcuser             |29    |
-----------------------------
|gnome-initial-setup |977   |
-----------------------------
|sshd                |74    |
-----------------------------
|avahi               |70    |
-----------------------------
|rngd                |976   |
-----------------------------
|tcpdump             |72    |
-----------------------------
|wang                |1000  |
-----------------------------

1.5 操作符

算數(shù)運(yùn)算符:

x+y, x-y, x*y, x/y, x^y, x%y
-x：轉(zhuǎn)換為負(fù)數(shù)
+x：將字符串轉(zhuǎn)換為數(shù)值

字符串操作符: 沒有符號的操作符, 字符串連接

賦值操作符:

=, +=, -=, *=, /=, %=, ^=慢味，++, --

范例: 自增

[19:29:30 root@centos7 ~]#awk 'BEGIN{i=0;print i++,i}' # 先打印i, 再自增, 然后重新打印
0 1
[19:29:49 root@centos7 ~]#awk 'BEGIN{i=0;print ++i,i}' # 先自增打印i, 然后再打印一遍i
1 1

比較操作符:

==, !=, >, >=, <, <=

范例: 打印/etc/issue文件的第二行record

[19:36:17 root@centos7 ~]#awk 'NR==2' /etc/issue
Kernel \r on an \m

范例: 打印UID>=1000的record

[19:36:26 root@centos7 ~]#awk '$3>=1000' /etc/passwd
systemd-network:x:192:192:systemd Network Management:/:/sbin/nologin
dbus:x:81:81:System message bus:/:/sbin/nologin
polkitd:x:999:998:User for polkitd:/:/sbin/nologin

范例: 取奇數(shù), 偶數(shù)行

[19:38:39 root@centos7 ~]#seq 10 | awk 'NR%2==0'
2
4
6
8
10
[19:39:16 root@centos7 ~]#seq 10 | awk 'NR%2==1'
1
3
5
7
9
[19:39:19 root@centos7 ~]#seq 10 | awk 'NR%2!=0'
1
3
5
7
9

范例: 取ip地址

[19:45:44 root@centos7 ~]#ifconfig eth0
eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 192.168.192.7  netmask 255.255.255.0  broadcast 192.168.192.255
        inet6 fe80::20c:29ff:fe0d:c854  prefixlen 64  scopeid 0x20<link>
        ether 00:0c:29:0d:c8:54  txqueuelen 1000  (Ethernet)
        RX packets 3731  bytes 324962 (317.3 KiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 1901  bytes 235371 (229.8 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

[19:45:29 root@centos7 ~]#ifconfig eth0 | awk 'NR==2{print $2}'
192.168.192.7

模式匹配符:

~ 左邊是否和右邊匹配，包含關(guān)系. 配合正則表達(dá)式, 判斷左邊內(nèi)容, 是否包含右邊內(nèi)容
!~ 是否不匹配, 左邊內(nèi)容, 是否不包含右邊內(nèi)容

格式:

awk 選項(xiàng) '$指定列范圍 ~ /指定的列范圍是否包含該字符串/{ACTION}' 文件路徑

范例:

[root@demo-c8 opt]# awk -F":" '$0 ~ /root/{print $1}' /etc/passwd
root
operator

條件表達(dá)式(三目表達(dá)式):

selector?if-true-expression:if-false-expression

select: 模式匹配
if-true-expression: 行匹配成功, 則執(zhí)行該命令
if-false-expression: 行匹配不成功, 則執(zhí)行該命令

范例: 計算每一行第三列的值是否大于等于1000, 如果是, 則給usertype賦值為"Common User", 如果不是, 賦值為"SysUser". 最后打印$1的內(nèi)容, 以及usertype變量的值

[root@demo-c8 opt]# awk -F: '{$3>=1000?usertype="Common User":usertype="SysUser";printf "%-20s:%12s\n",$1,usertype}'  /etc/passwd
root                :     SysUser
bin                 :     SysUser
daemon              :     SysUser
adm                 :     SysUser
lp                  :     SysUser
sync                :     SysUser
shutdown            :     SysUser
halt                :     SysUser
mail                :     SysUser
operator            :     SysUser
games               :     SysUser
ftp                 :     SysUser
nobody              : Common User
dbus                :     SysUser
systemd-coredump    :     SysUser
systemd-resolve     :     SysUser
tss                 :     SysUser
polkitd             :     SysUser
geoclue             :     SysUser
rtkit               :     SysUser
pulse               :     SysUser
libstoragemgmt      :     SysUser
qemu                :     SysUser
usbmuxd             :     SysUser
unbound             :     SysUser
rpc                 :     SysUser
gluster             :     SysUser
chrony              :     SysUser
setroubleshoot      :     SysUser
pipewire            :     SysUser
saslauth            :     SysUser
dnsmasq             :     SysUser
radvd               :     SysUser
clevis              :     SysUser
cockpit-ws          :     SysUser
cockpit-wsinstance  :     SysUser
sssd                :     SysUser
flatpak             :     SysUser
colord              :     SysUser
gdm                 :     SysUser
rpcuser             :     SysUser
gnome-initial-setup :     SysUser
sshd                :     SysUser
avahi               :     SysUser
rngd                :     SysUser
tcpdump             :     SysUser
wang                : Common User

1.6 模式PATTERN

PATTERN: 根據(jù)pattern條件墅冷，過濾匹配的行纯路，再做處理

如果未指定：空模式，匹配每一行

范例:

[root@demo-c8 ~]# awk -F":" '{print $1,$3}' /etc/passwd
root 0
bin 1
daemon 2
adm 3
lp 4
sync 5
shutdown 6
...

/regular expression/：僅處理能被模式匹配到的行寞忿，需要用/ /括起來

如果省略了action, 那么默認(rèn)會執(zhí)行print $0

[root@demo-c8 ~]# awk '/^UUID/' /etc/fstab
UUID=b1ab1ace-2582-4afd-8693-39bd9855041c /                       xfs     defaults        0 0
UUID=d5131695-82b3-4a23-bc28-5c8a4bf381a0 /boot                   ext4    defaults        1 2
UUID=bdd66510-e510-4fe7-ba71-e2a35e6dc492 /data                   xfs     defaults        0 0
UUID=05c944fb-d6f9-4544-ba10-8b7bf3cc8fed swap                    swap    defaults        0 0

范例: 取分區(qū)利用率數(shù)字

[root@demo-c8 ~]# df
Filesystem     1K-blocks    Used Available Use% Mounted on
devtmpfs         3957244       0   3957244   0% /dev
tmpfs            3985412       0   3985412   0% /dev/shm
tmpfs            3985412    9832   3975580   1% /run
tmpfs            3985412       0   3985412   0% /sys/fs/cgroup
/dev/sda2       41922560 4561428  37361132  11% /
/dev/sda5       41922560  325332  41597228   1% /data
/dev/sda1         999320  192552    737956  21% /boot
tmpfs             797080    1168    795912   1% /run/user/42
tmpfs             797080       4    797076   1% /run/user/0

[root@demo-c8 ~]# df | awk -F"[[:space:]]+|%" '/^\/dev\/sd/{print $5}'
11
1
21

[root@demo-c8 ~]# df | awk -F"[[:space:]]+|%" '/^\/dev\/sd/{print $5}' | sort |  df | awk -F"[[:space:]]+|%" '/^\/dev\/sd/{print $5}' | sort | tail -n1
21

relational expression: 關(guān)系表達(dá)式感昼，結(jié)果為“真”才會被處理

真：結(jié)果為非0值，非空字符串
假：結(jié)果為空字符串或0值

范例: !0表示真, !1表示假. 使用數(shù)字時, 不用加雙引號""

[root@demo-c8 ~]# awk '!0' /etc/passwd
root:x:0:0:root:/root:/bin/bash
bin:x:1:1:bin:/bin:/sbin/nologin
daemon:x:2:2:daemon:/sbin:/sbin/nologin
adm:x:3:4:adm:/var/adm:/sbin/nologin
lp:x:4:7:lp:/var/spool/lpd:/sbin/nologin
sync:x:5:0:sync:/sbin:/bin/sync
shutdown:x:6:0:shutdown:/sbin:/sbin/shutdown
halt:x:7:0:halt:/sbin:/sbin/halt
mail:x:8:12:mail:/var/spool/mail:/sbin/nologin
operator:x:11:0:operator:/root:/sbin/nologin
games:x:12:100:games:/usr/games:/sbin/nologin
ftp:x:14:50:FTP User:/var/ftp:/sbin/nologin
nobody:x:65534:65534:Kernel Overflow User:/:/sbin/nologin
dbus:x:81:81:System message bus:/:/sbin/nologin
systemd-coredump:x:999:997:systemd Core Dumper:/:/sbin/nologin
systemd-resolve:x:193:193:systemd Resolver:/:/sbin/nologin
tss:x:59:59:Account used by the trousers package to sandbox the tcsd daemon:/dev/null:/sbin/nologin
polkitd:x:998:996:User for polkitd:/:/sbin/nologin
geoclue:x:997:995:User for geoclue:/var/lib/geoclue:/sbin/nologin
rtkit:x:172:172:RealtimeKit:/proc:/sbin/nologin
pulse:x:171:171:PulseAudio System Daemon:/var/run/pulse:/sbin/nologin
libstoragemgmt:x:996:992:daemon account for libstoragemgmt:/var/run/lsm:/sbin/nologin
qemu:x:107:107:qemu user:/:/sbin/nologin
usbmuxd:x:113:113:usbmuxd user:/:/sbin/nologin
unbound:x:995:990:Unbound DNS resolver:/etc/unbound:/sbin/nologin
rpc:x:32:32:Rpcbind Daemon:/var/lib/rpcbind:/sbin/nologin
gluster:x:994:989:GlusterFS daemons:/run/gluster:/sbin/nologin
chrony:x:993:988::/var/lib/chrony:/sbin/nologin
setroubleshoot:x:992:986::/var/lib/setroubleshoot:/sbin/nologin
pipewire:x:991:985:PipeWire System Daemon:/var/run/pipewire:/sbin/nologin
saslauth:x:990:76:Saslauthd user:/run/saslauthd:/sbin/nologin
dnsmasq:x:984:984:Dnsmasq DHCP and DNS server:/var/lib/dnsmasq:/sbin/nologin
radvd:x:75:75:radvd user:/:/sbin/nologin
clevis:x:983:982:Clevis Decryption Framework unprivileged user:/var/cache/clevis:/sbin/nologin
cockpit-ws:x:982:980:User for cockpit web service:/nonexisting:/sbin/nologin
cockpit-wsinstance:x:981:979:User for cockpit-ws instances:/nonexisting:/sbin/nologin
sssd:x:980:978:User for sssd:/:/sbin/nologin
flatpak:x:979:977:User for flatpak system helper:/:/sbin/nologin
colord:x:978:976:User for colord:/var/lib/colord:/sbin/nologin
gdm:x:42:42::/var/lib/gdm:/sbin/nologin
rpcuser:x:29:29:RPC Service User:/var/lib/nfs:/sbin/nologin
gnome-initial-setup:x:977:975::/run/gnome-initial-setup/:/sbin/nologin
sshd:x:74:74:Privilege-separated SSH:/var/empty/sshd:/sbin/nologin
avahi:x:70:70:Avahi mDNS/DNS-SD Stack:/var/run/avahi-daemon:/sbin/nologin
rngd:x:976:974:Random Number Generator Daemon:/var/lib/rngd:/sbin/nologin
tcpdump:x:72:72::/:/sbin/nologin
wang:x:1000:1000:wang:/home/wang:/bin/bash

[root@demo-c8 ~]# awk '!1' /etc/passwd

范例: !n++表達(dá)式

awk先判斷n的值, n=0, !n=1, 為真, 所以會處理第一行
處理完第一行, 計算n++, n=2, !n=0, 為假, 則不處理第二行
第二行雖然不處理, 但是因?yàn)槭茄h(huán)處理每一行, 所以之后還要判斷n, n=2, n++=3, !n=0, 為假, 所以不處理第三行, 后續(xù)所有行因?yàn)閚都是為0, !n則為假, 也就都不處理了

[root@demo-c8 ~]# awk -v n=0 '!n++' /etc/passwd
root:x:0:0:root:/root:/bin/bash

范例: !++n表達(dá)式

n=0, 先計算++n, n=1, !n=0, 為假, 則不處理文本

[root@demo-c8 opt]# awk -v n=0 '!++n' /etc/passwd 
[root@demo-c8 opt]#

總結(jié): 當(dāng)pattern使用關(guān)系表達(dá)式時, 處理文本就相當(dāng)于循環(huán)處理每一行, 每次處理一行內(nèi)容之前, 要先判斷表達(dá)式的返回值, 返回真, 則處理, 返回假, 則不處理. 處理完一行, 接著再次判斷表達(dá)式的返回值, 然后再處理或不處理

范例: i=!i, 打印奇數(shù)行

i起初沒有賦值, 為假, 那么!i就為真, 所以會處理第一行, 此時i=真
之后, 因?yàn)?code>i為真, 那么!i就為假, 所以不會處理第二行, 此時i=假
接著再判斷表達(dá)式, 此時i=假, 那么!i則為真, 處理第三行, 也就是只處理奇數(shù)行

[root@demo-c8 opt]# seq 10 | awk 'i=!i' 
1
3
5
7
9

范例: 打印偶數(shù)行

先將i初始化為1, 真, 那么!i就是假, 和上面相反, 結(jié)果就是只處理偶數(shù)行

[root@demo-c8 opt]# seq 10 | awk -v i=1 'i=!i' 
2
4
6
8
10

[root@demo-c8 opt]# seq 10 | awk '!(i=!i)' 
2
4
6
8
10

line ranges：行范圍

不支持直接用行號罐脊，但可以使用變量NR間接指定行號

范例:

[root@demo-c8 opt]# seq 10 | awk 'NR>=3 && NR<=6'
3
4
5
6

/pat1/,/pat2/ 不支持直接給出數(shù)字格式, 但支持字符匹配

范例:

[root@demo-c8 opt]# awk '/^root/,/^nobody/' /etc/passwd
root:x:0:0:root:/root:/bin/bash
bin:x:1:1:bin:/bin:/sbin/nologin
daemon:x:2:2:daemon:/sbin:/sbin/nologin
adm:x:3:4:adm:/var/adm:/sbin/nologin
lp:x:4:7:lp:/var/spool/lpd:/sbin/nologin
sync:x:5:0:sync:/sbin:/bin/sync
shutdown:x:6:0:shutdown:/sbin:/sbin/shutdown
halt:x:7:0:halt:/sbin:/sbin/halt
mail:x:8:12:mail:/var/spool/mail:/sbin/nologin
operator:x:11:0:operator:/root:/sbin/nologin
games:x:12:100:games:/usr/games:/sbin/nologin
ftp:x:14:50:FTP User:/var/ftp:/sbin/nologin
nobody:x:65534:65534:Kernel Overflow User:/:/sbin/nologin

2. AWK條件判斷和循環(huán)控制

2.1 條件判斷if-else

語法:

if(condition)statement
if(condition){statement;…}[{else statement}
if(condition1){statement1}else if(condition2){statement2}else if(condition3){statement3}......else{statementN}

使用場景: 對awk取得的整行或某個字段做條件判斷

范例: '{if(condition)statement}'

[root@demo-c8 opt]# awk -F":" '{if($NF=="/bin/bash")print $1}' /etc/passwd
root
wang

[root@demo-c8 opt]# awk -F":" '{if($NF>5)print $0}' /etc/fstab 
UUID=b1ab1ace-2582-4afd-8693-39bd9855041c /                       xfs     defaults        0 0
UUID=d5131695-82b3-4a23-bc28-5c8a4bf381a0 /boot                   ext4    defaults        1 2
UUID=bdd66510-e510-4fe7-ba71-e2a35e6dc492 /data                   xfs     defaults        0 0
UUID=05c944fb-d6f9-4544-ba10-8b7bf3cc8fed swap                    swap    defaults        0 0

[root@demo-c8 opt]# awk -F":" '{if($3>=1000)print $1,$3}' /etc/passwd
nobody 65534
wang 1000

范例: '{if(condition1){statement1}else{statement2}}'

[root@demo-c8 opt]# awk -F":" '{if($3>=1000){printf "%-20s %s\n", $1,$3}else{printf "%-20s %s\n", $1, "common user"}}' /etc/passwd
root                 common user
bin                  common user
daemon               common user
adm                  common user
lp                   common user
sync                 common user
shutdown             common user
halt                 common user
mail                 common user
operator             common user
games                common user
ftp                  common user
nobody               65534
dbus                 common user
systemd-coredump     common user
systemd-resolve      common user
tss                  common user
polkitd              common user
geoclue              common user
rtkit                common user
pulse                common user
libstoragemgmt       common user
qemu                 common user
usbmuxd              common user
unbound              common user
rpc                  common user
gluster              common user
chrony               common user
setroubleshoot       common user
pipewire             common user
saslauth             common user
dnsmasq              common user
radvd                common user
clevis               common user
cockpit-ws           common user
cockpit-wsinstance   common user
sssd                 common user
flatpak              common user
colord               common user
gdm                  common user
rpcuser              common user
gnome-initial-setup  common user
sshd                 common user
avahi                common user
rngd                 common user
tcpdump              common user
wang                 1000

范例: awk取分區(qū)利用率大于10%的分區(qū)

[root@demo-c8 opt]# df 
Filesystem     1K-blocks    Used Available Use% Mounted on
devtmpfs         3957244       0   3957244   0% /dev
tmpfs            3985412       0   3985412   0% /dev/shm
tmpfs            3985412    9832   3975580   1% /run
tmpfs            3985412       0   3985412   0% /sys/fs/cgroup
/dev/sda2       41922560 4561796  37360764  11% /
/dev/sda5       41922560  325332  41597228   1% /data
/dev/sda1         999320  192552    737956  21% /boot
tmpfs             797080    1168    795912   1% /run/user/42
tmpfs             797080       4    797076   1% /run/user/0

[root@demo-c8 opt]# df | awk -F"[[:space:]]+|%" '/^\/dev\/sd/{if($5>10)print $1, $5}'
/dev/sda2 11
/dev/sda1 21

[ %]+ : 表示>=1個空格, 或者>=1個百分號
[[:space:]]+|% : 表示>=1個空格, 或者一個百分號
" +|%" : 表示>=1個空格, 或者>=1個百分號

2.2 switch語句

類似Shell中的case表達(dá)式

語法:

switch(expression) {case VALUE1 or /REGEXP/: statement1; case VALUE2 or 
/REGEXP2/: statement2; ...; default: statementn}

2.3 while循環(huán)

條件“真”，進(jìn)入循環(huán)蜕琴；條件“假”，退出循環(huán)
使用場景：

對一行內(nèi)的多個字段逐一類似處理時使用
對數(shù)組中的各元素逐一處理時使用

語法:

while (condition) {statement;…}

范例: length()內(nèi)置函數(shù), 返回字符個數(shù)

[root@demo-c8 opt]# awk  'BEGIN{print length("您好世界")}' 
4
[root@demo-c8 opt]# awk  'BEGIN{print length("hello world")}' 
11

范例: awk逐列處理

[20:08:29 root@centos-7-6 ~]#awk '/^[[:space:]]*linux16/{i=1;while(i<=NF){print $i,length($i); i++}}' /etc/grub2.cfg
linux16 7
/vmlinuz-3.10.0-1127.el7.x86_64 31
root=UUID=ffd4773d-a205-4937-bdff-4f80e73b6ad0 46
ro 2
crashkernel=auto 16
rhgb 4
quiet 5
net.ifnames=0 13
linux16 7
/vmlinuz-0-rescue-e5ad2c92a25b4c87b7cca719c77091ff 50
root=UUID=ffd4773d-a205-4937-bdff-4f80e73b6ad0 46
ro 2
crashkernel=auto 16
rhgb 4
quiet 5
net.ifnames=0 13

[20:08:37 root@centos-7-6 ~]#awk '/^[[:space:]]*linux16/{i=1;while(i<=NF) {if(length($i)>=10){print $i,length($i)}; i++}}' /etc/grub2.cfg
/vmlinuz-3.10.0-1127.el7.x86_64 31
root=UUID=ffd4773d-a205-4937-bdff-4f80e73b6ad0 46
crashkernel=auto 16
net.ifnames=0 13
/vmlinuz-0-rescue-e5ad2c92a25b4c87b7cca719c77091ff 50
root=UUID=ffd4773d-a205-4937-bdff-4f80e73b6ad0 46
crashkernel=auto 16
net.ifnames=0 13

范例: 打印1,2,..100

[root@demo-c8 opt]# awk -v i=1 'BEGIN{while(i<=100){print i;i++}}'
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100

注意:

1. awk在打印變量時, 不用加`$`符合. 位置變量除外

范例: 打印1-100的和

[root@demo-c8 opt]# awk -v i=1 -v sum=0 'BEGIN{while (i<=100){sum+=i; i++};print sum}'
5050

2.4 循環(huán)do-while

語法:

do {statement;…}while(condition)

意義: 無論真假，至少執(zhí)行一次循環(huán)體

do-while循環(huán)

語法：do {statement;…}while(condition)
意義：無論真假实蔽，至少執(zhí)行一次循環(huán)體

范例:

[root@demo-c8 opt]# awk 'BEGIN{ total=0;i=1;do{ total+=i;i++;}while(i<=100);print total}'
5050

2.5 循環(huán)for

語法:

for(expr1;expr2;expr3) {statement;…}

常見用法:

for(variable assignment;condition;iteration process) {for-body}

特殊用法：能夠遍歷數(shù)組中的元素

范例: awk的for循環(huán), 打印1-100

[root@demo-c8 opt]# awk 'BEGIN{for(i=1;i<=100;i++){print i}}'
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100

范例: awk的for循環(huán), 計算1-100的和

[root@demo-c8 opt]# awk 'BEGIN{sum=0;for(i=1;i<=100;i++){sum+=i};print sum}'
5050

范例: awk, Shell, bc效率對比, 計算1+...+1000000的和

[root@demo-c8 opt]# time (awk 'BEGIN{sum=0;for(i=1;i<=1000000;i++){sum+=i};print sum}')
500000500000

real    0m0.079s
user    0m0.076s
sys 0m0.003s

[root@demo-c8 opt]# time (sum=0; for((i=1;i<=1000000;i++));do let sum+=i;done;echo $sum)
500000500000

real    0m5.446s
user    0m5.408s
sys 0m0.000s

[root@demo-c8 opt]# time (seq -s+ 1000000 |bc)
500000500000

real    0m0.432s
user    0m0.343s
sys 0m0.150s

2.6 continue和break

continue 中斷本次循環(huán)
break 中斷整個循環(huán)

格式:

continue [n]
break [n]

范例: continue, 打印1-100奇數(shù)的和

[root@demo-c8 opt]# awk -v sum=0 'BEGIN{for(i=1;i<=100;i++){if(i%2==0)continue; sum+=i};print sum}'
2500

范例: break, 打印1-49

[root@demo-c8 opt]# awk 'BEGIN{for(i=1;i<=100;i++){if (i<=50){break}else{print i}}}'
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49

2.7 next

next 可以提前結(jié)束對本行處理而直接進(jìn)入下一行處理（awk自身循環(huán)）

范例: 打印UID是奇數(shù)的用戶名和UID

[root@demo-c8 opt]# awk -F":" '{if($3%2!=0)printf "%-20s %s\n", $1,$3}' /etc/passwd
bin                  1
adm                  3
sync                 5
halt                 7
operator             11
dbus                 81
systemd-coredump     999
systemd-resolve      193
tss                  59
geoclue              997
pulse                171
qemu                 107
usbmuxd              113
unbound              995
chrony               993
pipewire             991
radvd                75
clevis               983
cockpit-wsinstance   981
flatpak              979
rpcuser              29
gnome-initial-setup  977

3. 數(shù)組

awk的數(shù)組都是關(guān)聯(lián)數(shù)組

格式:

array_name[index-expression]

范例:

[root@demo-c8 opt]# weekdays["mon"]="monday"
[root@demo-c8 opt]# echo ${weekdays["mon"]}
monday

index-expression

利用數(shù)組寿冕，實(shí)現(xiàn) k/v 功能
可使用任意字符串；字符串要使用雙引號括起來
如果某數(shù)組元素(key)事先不存在雏搂，在引用時藕施，awk會自動創(chuàng)建此元素，并將其值初始化為空串("")
若要判斷數(shù)組中是否存在某元素凸郑，要使用index in array格式進(jìn)行遍歷. 存在返回1, 不存在返回0

注意: awk的關(guān)聯(lián)數(shù)組不支持[*], 取全部值

[root@demo-c8 opt]# awk 'BEGIN{weekdays["mon"]="monday";weekdays["tue"]="tuesday";weekdays["wed"]="wednesday";print weekdays["mon"]}'
monday

范例: 判斷數(shù)組是否包含某個key

[root@demo-c8 opt]# awk 'BEGIN{weekdays["mon"]="monday";weekdays["tue"]="tuesday";weekdays["wed"]="wednesday";print "thur" in weekdays; print "mon" in weekdays}'
0
1

若要遍歷數(shù)組中的每個value, 可以使用for循環(huán). for循環(huán)每次遍歷時, 會取出數(shù)組的key.

范例:

[root@demo-c8 opt]# awk 'BEGIN{user["name"]="root"; user["uid"]="0"; user["password"]="Y"; for(i in user){print user[i]}}'
Y
root
0

范例: 顯示主機(jī)的連接狀態(tài)出現(xiàn)的次數(shù), 并排序

ss命令輸出格式

[root@demo-c8 opt]# ss -nta | awk 'NR!=1{print $1}' | sort | uniq -c
      1 ESTAB
      7 LISTEN

state[$1]++: 創(chuàng)建一個awk數(shù)組, 名字為state. 按照$1, 把不同的狀態(tài), 作為數(shù)組的key, ++表示分別按照不同的狀態(tài), 進(jìn)行次數(shù)累計. 最后得到的數(shù)組就是每個狀態(tài)對應(yīng)的次數(shù)

[root@demo-c8 opt]# ss -ant | awk 'NR!=1{state[$1]++}END{for(i in state){print i,state[i]}}'
LISTEN 7
ESTAB 1

范例: 統(tǒng)計訪問網(wǎng)站的ip的次數(shù)

[root@demo-c8 opt]# cat access_log | head -n1
172.18.118.91 - - [20/May/2018:08:09:59 +0800] "GET / HTTP/1.1" 200 912 "-" "Mozilla/4.0 (compatible; MSIE 9.0; Windows NT 5.1; Trident/5.0)"

[root@demo-c8 opt]# awk '{ip[$1]++}END{for(i in ip){printf "%-20s %s\n", ip[i],  i}}' access_log  | sort -nr | head -n5
4870                 172.20.116.228
3429                 172.20.116.208
2834                 172.20.0.222
2613                 172.20.112.14
2267                 172.20.0.227

范例: 利用/etc/hosts.deny文件拒絕其他服務(wù)器的SSH訪問

/etc/hosts.deny文件利用的是tcpwrapper技術(shù), 但是僅在CentOS7和較老版本生效, CentOS8開始不支持

[20:10:41 root@centos-7-6 ~]#echo "sshd: 10.0.0.108" >> /etc/hosts.deny

[root@demo-c8 opt]# ssh root@10.0.0.187
kex_exchange_identification: read: Connection reset by peer

4. awk函數(shù)

awk 的函數(shù)分為內(nèi)置和自定義函數(shù)

4.1 常見內(nèi)置函數(shù)

數(shù)值處理:

rand()：返回0和1之間一個隨機(jī)數(shù)
srand()：配合rand() 函數(shù),生成隨機(jī)數(shù)的種子
int()：返回整數(shù)部分

范例: rand()和srand()要配套使用, 否則rand()只會返回固定數(shù)值. 并且, 此命令不能執(zhí)行過快, 否則只會返回相同的值

[22:42:27 root@centos7 ~]#awk 'BEGIN{srand();print rand()}'
0.482749
[22:42:40 root@centos7 ~]#awk 'BEGIN{srand();print rand()}'
0.12631
[22:42:42 root@centos7 ~]#awk 'BEGIN{srand();print rand()}'
0.12631
[22:42:42 root@centos7 ~]#awk 'BEGIN{srand();print rand()}'
0.12631
[22:42:42 root@centos7 ~]#awk 'BEGIN{srand();print rand()}'
0.0949145
[22:42:44 root@centos7 ~]#awk 'BEGIN{srand();print rand()}'
0.0949145
[22:42:44 root@centos7 ~]#awk 'BEGIN{srand();print rand()}'
0.25884

范例: 打印100以內(nèi)整數(shù)十次

[22:45:52 root@centos7 ~]#awk 'BEGIN{srand();for(i=1;i<=10;i++)print int(rand()*100)}'
13
5
63
15
36
82
91
33
86
31

字符串處理:

length([s])：返回指定字符串的長度
sub(r,s,[t])：對t字符串搜索r表示模式匹配的內(nèi)容裳食，并將第一個匹配內(nèi)容替換為s

范例: sub字符替換

[22:46:11 root@centos7 ~]#echo "2008:08:08 08:08:08" | awk 'sub(/:/,"-",$1)'
2008-08:08 08:08:08

[22:47:26 root@centos7 ~]#echo "2008:08:08 08:08:08" | awk '{sub(/:/,"-",$1);print $0}'
2008-08:08 08:08:08

gsub(r,s,[t])：對t字符串進(jìn)行搜索r表示的模式匹配的內(nèi)容，并全部替換為s所表示的內(nèi)容

范例:

[22:48:10 root@centos7 ~]#echo "2008:08:08 08:08:08" | awk 'gsub(/:/,"-",$0)'
2008-08-08 08-08-08
[22:49:00 root@centos7 ~]#echo "2008:08:08 08:08:08" | awk '{gsub(/:/,"-",$0);print $0}'
2008-08-08 08-08-08

split(s,array,[r])：以r為分隔符芙沥，切割字符串s诲祸，并將切割后的結(jié)果保存至array所表示的數(shù)組中浊吏，第一個索引值為1,第二個索引值為2,…

范例:

[22:49:47 root@centos7 ~]#netstat -nt | awk '/^tcp/{split($5,ip,":");count[ip[1]]++}END{for(i in count){print i,count[i]}}'
192.168.192.1 1

system 函數(shù)：可以awk中調(diào)用shell命令

空格是awk中的字符串連接符，如果system中需要使用awk中的變量可以使用空格分隔救氯，或者說除了awk的變量外其他一律用""引用起來

[22:50:43 root@centos7 ~]#awk 'BEGIN{system("hostname")}'
centos7.mac
[22:50:48 root@centos7 ~]#awk 'BEGIN{score=100; system("echo your score is " score) }'
your score is 100

范例: 統(tǒng)計連接次數(shù)大于等于10的ip, 并且添加到iptables

[root@centos8 ~]#netstat -tn | awk 
'/^tcp/{split($5,ip,":");count[ip[1]]++}END{for(i in count){if(count[i]>=10)
{system("iptables -A INPUT -s "i" -j REJECT")}}}'

4.2 自定義函數(shù)

自定義函數(shù)格式:

function name ( parameter, parameter, ... ) {
   statements
   return expression
}

范例: 統(tǒng)計兩個數(shù)誰大誰小

[22:53:20 root@centos7 ~]#vim func.awk

function max(x,y) {                                                             
 x>y?var=x:var=y
 return var
}
BEGIN{print max(a,b)}

[22:53:52 root@centos7 ~]#awk -v a=30 -v b=20 -f func.awk
30

5. awk腳本

將awk程序?qū)懗赡_本找田，直接調(diào)用或執(zhí)行

范例:

[22:56:52 root@centos7 ~]#vim passwd.awk

{if($3<=1000)print $1,$3}

[22:57:20 root@centos7 ~]#awk -F: -f passwd.awk /etc/passwd
root 0
bin 1
daemon 2
adm 3
lp 4
sync 5
shutdown 6
halt 7
mail 8
operator 11
games 12
ftp 14
nobody 99
systemd-network 192
dbus 81
polkitd 999
sshd 74
postfix 89
tcpdump 72

范例:

[22:57:21 root@centos7 ~]#vim test.awk

#!/bin/awk -f
#this is a awk script
{if($3<=1000)print $1,$3}

[22:58:45 root@centos7 ~]#./test.awk -F: /etc/passwd
root 0
bin 1
daemon 2
adm 3
lp 4
sync 5
shutdown 6
halt 7
mail 8
operator 11
games 12
ftp 14
nobody 99
systemd-network 192
dbus 81
polkitd 999
sshd 74
postfix 89
tcpdump 72

向awk腳本傳遞參數(shù)

格式:

awkfile var=value var2=value2... Inputfile

注意：在BEGIN過程中不可用。直到首行輸入完成以后着憨，變量才可用墩衙。可以通過-v 參數(shù)甲抖，讓awk在執(zhí)行BEGIN之前得到變量的值漆改。命令行中每一個指定的變量都需要一個-v參數(shù)

范例:

[22:58:46 root@centos7 ~]#vim test2.awk
#!/bin/awk -f
{if($3 >=min && $3<=max)print $1,$3}

[23:00:17 root@centos7 ~]#chmod +x test2.awk
[23:00:18 root@centos7 ~]#./test2.awk -F: min=100 max=200 /etc/passwd
systemd-network 192

練習(xí):

1、文件host_list.log 如下格式惧眠，請?zhí)崛　?magedu.com”前面的主機(jī)名部分并寫入到回到該文件中

1 www.magedu.com
2 blog.magedu.com
3 study.magedu.com
4 linux.magedu.com
5 python.magedu.com
......
999 study.magedu.com

2籽懦、統(tǒng)計/etc/fstab文件中每個文件系統(tǒng)類型出現(xiàn)的次數(shù)
3、統(tǒng)計/etc/fstab文件中每個單詞出現(xiàn)的次數(shù)
4氛魁、提取出字符串Yd$C@M05MB%9&Bdh7dq+YVixp3vpw中的所有數(shù)字
5暮顺、有一文件記錄了1-100000之間隨機(jī)的整數(shù)共5000個，存儲的格式100,50,35,89…請取出其中最大和最小的整數(shù)
6秀存、解決Dos攻擊生產(chǎn)案例：根據(jù)web日志或者或者網(wǎng)絡(luò)連接數(shù)捶码，監(jiān)控當(dāng)某個IP并發(fā)連接數(shù)或者短時內(nèi)PV達(dá)到100，即調(diào)用防火墻命令封掉對應(yīng)的IP或链，監(jiān)控頻率每隔5分鐘惫恼。防火墻命令為：iptables -A INPUT -s IP -j REJECT
7、將以下文件內(nèi)容中FQDN取出并根據(jù)其進(jìn)行計數(shù)從高到低排序

http://mail.magedu.com/index.html
http://www.magedu.com/test.html
http://study.magedu.com/index.html
http://blog.magedu.com/index.html
http://www.magedu.com/images/logo.jpg
http://blog.magedu.com/20080102.html
http://www.magedu.com/images/magedu.jpg

參考答案:

[root@centos8 ~]#awk -F"/" '{url[$3]++}END{for(i in url){print url[i],i}}' 
url.log |sort -nr
3 www.magedu.com
2 blog.magedu.com
1 study.magedu.com
1 mail.magedu.com

8澳盐、將以下文本以inode為標(biāo)記祈纯，對inode相同的counts進(jìn)行累加，并且統(tǒng)計出同一inode中叼耙，beginnumber的最小值和endnumber的最大值

inode|beginnumber|endnumber|counts|
106|3363120000|3363129999|10000|
106|3368560000|3368579999|20000|
310|3337000000|3337000100|101|
310|3342950000|3342959999|10000|
310|3362120960|3362120961|2|
311|3313460102|3313469999|9898|
311|3313470000|3313499999|30000|
311|3362120962|3362120963|2|

輸出的結(jié)果格式為：

310|3337000000|3362120961|10103|
311|3313460102|3362120963|39900|
106|3363120000|3368579999|30000|

?著作權(quán)歸作者所有,轉(zhuǎn)載或內(nèi)容合作請聯(lián)系作者

人面猴
序言：七十年代末腕窥，一起剝皮案震驚了整個濱河市，隨后出現(xiàn)的幾起案子筛婉，更是在濱河造成了極大的恐慌簇爆，老刑警劉巖，帶你破解...
沈念sama閱讀 211,123評論 6贊 490
死咒
序言：濱河連續(xù)發(fā)生了三起死亡事件爽撒，死亡現(xiàn)場離奇詭異入蛆，居然都是意外死亡，警方通過查閱死者的電腦和手機(jī)硕勿，發(fā)現(xiàn)死者居然都...
沈念sama閱讀 90,031評論 2贊 384
救了他兩次的神仙讓他今天三更去死
文/潘曉璐我一進(jìn)店門哨毁，熙熙樓的掌柜王于貴愁眉苦臉地迎上來，“玉大人源武，你說我怎么就攤上這事挑庶⊙越眨” “怎么了？”我有些...
開封第一講書人閱讀 156,723評論 0贊 345
道士緝兇錄：失蹤的賣姜人
文/不壞的土叔我叫張陵迎捺，是天一觀的道長举畸。經(jīng)常有香客問我，道長凳枝，這世上最難降的妖魔是什么抄沮？我笑而不...
開封第一講書人閱讀 56,357評論 1贊 283
?港島之戀（遺憾婚禮）
正文為了忘掉前任，我火速辦了婚禮岖瑰，結(jié)果婚禮上叛买，老公的妹妹穿的比我還像新娘。我一直安慰自己蹋订，他們只是感情好率挣，可當(dāng)我...
茶點(diǎn)故事閱讀 65,412評論 5贊 384
惡毒庶女頂嫁案：這布局不是一般人想出來的
文/花漫我一把揭開白布。她就那樣靜靜地躺著露戒，像睡著了一般椒功。火紅的嫁衣襯著肌膚如雪。梳的紋絲不亂的頭發(fā)上智什，一...
開封第一講書人閱讀 49,760評論 1贊 289
城市分裂傳說
那天动漾，我揣著相機(jī)與錄音，去河邊找鬼荠锭。笑死旱眯，一個胖子當(dāng)著我的面吹牛，可吹牛的內(nèi)容都是我干的证九。我是一名探鬼主播删豺，決...
沈念sama閱讀 38,904評論 3贊 405
雙鴛鴦連環(huán)套：你想象不到人心有多黑
文/蒼蘭香墨我猛地睜開眼，長吁一口氣：“原來是場噩夢啊……” “哼愧怜！你這毒婦竟也來了吼鳞？” 一聲冷哼從身側(cè)響起，我...
開封第一講書人閱讀 37,672評論 0贊 266
萬榮殺人案實(shí)錄
序言：老撾萬榮一對情侶失蹤叫搁，失蹤者是張志新（化名）和其女友劉穎，沒想到半個月后供炎，有當(dāng)?shù)厝嗽跇淞掷锇l(fā)現(xiàn)了一具尸體渴逻，經(jīng)...
沈念sama閱讀 44,118評論 1贊 303
?護(hù)林員之死
正文獨(dú)居荒郊野嶺守林人離奇死亡，尸身上長有42處帶血的膿包…… 初始之章·張勛以下內(nèi)容為張勛視角年9月15日...
茶點(diǎn)故事閱讀 36,456評論 2贊 325
?白月光啟示錄
正文我和宋清朗相戀三年音诫，在試婚紗的時候發(fā)現(xiàn)自己被綠了惨奕。大學(xué)時的朋友給我發(fā)了我未婚夫和他白月光在一起吃飯的照片。...
茶點(diǎn)故事閱讀 38,599評論 1贊 340
活死人
序言：一個原本活蹦亂跳的男人離奇死亡竭钝，死狀恐怖梨撞，靈堂內(nèi)的尸體忽然破棺而出雹洗，到底是詐尸還是另有隱情，我是刑警寧澤卧波，帶...
沈念sama閱讀 34,264評論 4贊 328
?日本核電站爆炸內(nèi)幕
正文年R本政府宣布时肿，位于F島的核電站，受9級特大地震影響港粱，放射性物質(zhì)發(fā)生泄漏螃成。R本人自食惡果不足惜，卻給世界環(huán)境...
茶點(diǎn)故事閱讀 39,857評論 3贊 312
男人毒藥：我在死后第九天來索命
文/蒙蒙一查坪、第九天我趴在偏房一處隱蔽的房頂上張望寸宏。院中可真熱鬧，春花似錦偿曙、人聲如沸氮凝。這莊子的主人今日做“春日...
開封第一講書人閱讀 30,731評論 0贊 21
一樁弒父案望忆，背后竟有這般陰謀
文/蒼蘭香墨我抬頭看了看天上的太陽罩阵。三九已至，卻和暖如春炭臭，著一層夾襖步出監(jiān)牢的瞬間永脓，已是汗流浹背。一陣腳步聲響...
開封第一講書人閱讀 31,956評論 1贊 264
情欲美人皮
我被黑心中介騙來泰國打工鞋仍，沒想到剛下飛機(jī)就差點(diǎn)兒被人妖公主榨干…… 1. 我叫王不留常摧，地道東北人。一個月前我還...
沈念sama閱讀 46,286評論 2贊 360
代替公主和親
正文我出身青樓威创，卻偏偏與公主長得像落午，于是被迫代替她去往敵國和親。傳聞我的和親對象是個殘疾皇子肚豺，可洞房花燭夜當(dāng)晚...
茶點(diǎn)故事閱讀 43,465評論 2贊 348