4.shell編程-文本處理三劍客之sed

4.1.sed的選項

 sed，流編輯器。對標準輸出或文件進行逐行處理。

語法格式

• 第一種：stdout | sed [option] "pattern command"
• 第二種：sed [option] "pattern command" file

 選項

• -n    只打印模式匹配行
• -e    直接在命令行進行sed編輯，默認選項
• -f    編輯動做保存在文件中，指定文件執行
• -r    支持擴展正則表達式
• -i    直接修改文件內容

實例

[root@VM_0_9_centos shell_learn]# cat test.txt 
I love python
I love PYTHON
Hadoop is bigdata frame
[root@VM_0_9_centos shell_learn]# sed -n 'p' test.txt 
I love python
I love PYTHON
Hadoop is bigdata frame
[root@VM_0_9_centos shell_learn]# sed -n '/python/p' test.txt 
I love python
[root@VM_0_9_centos shell_learn]# sed -n -e '/python/p' -e '/PYTHON/p' test.txt 
I love python
I love PYTHON
[root@VM_0_9_centos shell_learn]#

-f 選項，把編輯動做放到文本中

[root@VM_0_9_centos shell_learn]# cat edit.txt 
/python/p
[root@VM_0_9_centos shell_learn]# sed -n -f edit.txt test.txt 
I love python
[root@VM_0_9_centos shell_learn]#

 -i   修改

sed -i 's/love/like/g' test.txt

4.2.sed中的pattern詳解

pattern用發表

 

（1）直接指定行號

[root@VM_0_9_centos shell_learn]# sed -n '17p' /etc/passwd
dbus:x:81:81:System message bus:/:/sbin/nologin

（2）指定起始行號和結束行號

[root@VM_0_9_centos shell_learn]# sed -n '10,13p' /etc/passwd
operator:x:11:0:operator:/root:/sbin/nologin
games:x:12:100:games:/usr/games:/sbin/nologin
ftp:x:14:50:FTP User:/var/ftp:/sbin/nologin
nobody:x:99:99:Nobody:/:/sbin/nologin
[root@VM_0_9_centos shell_learn]# 

（3）指定起始行號，而後後面N行

[root@VM_0_9_centos shell_learn]# sed -n '10,+5p' /etc/passwd
operator:x:11:0:operator:/root:/sbin/nologin
games:x:12:100:games:/usr/games:/sbin/nologin
ftp:x:14:50:FTP User:/var/ftp:/sbin/nologin
nobody:x:99:99:Nobody:/:/sbin/nologin
avahi-autoipd:x:170:170:Avahi IPv4LL Stack:/var/lib/avahi-autoipd:/sbin/nologin
systemd-bus-proxy:x:999:997:systemd Bus Proxy:/:/sbin/nologin
[root@VM_0_9_centos shell_learn]# 

（4）/pattern1/    正則表達式匹配的行

[root@VM_0_9_centos shell_learn]# sed -n '/derek/p' /etc/passwd
derektest:x:1001:1001::/home/derektest:/bin/bash
derek:x:1002:1002::/home/derek:/bin/bash
[root@VM_0_9_centos shell_learn]# 

（5）/pattern1/,/pattern2/

[root@VM_0_9_centos shell_learn]# sed -n '/nginx/,/derek/p' /etc/passwd
nginx:x:993:991:Nginx web server:/var/lib/nginx:/sbin/nologin
memcached:x:992:990:Memcached daemon:/run/memcached:/sbin/nologin
redis:x:991:989:Redis Database Server:/var/lib/redis:/sbin/nologin
derektest:x:1001:1001::/home/derektest:/bin/bash
[root@VM_0_9_centos shell_learn]# 

（6）linenumber,/pattern1/

從第30行開始匹配，直到匹配到derek行結尾

[root@VM_0_9_centos shell_learn]# sed -n '30,/derek/p' /etc/passwd
mysql:x:27:27:MySQL Server:/var/lib/mysql:/bin/bash
nginx:x:993:991:Nginx web server:/var/lib/nginx:/sbin/nologin
memcached:x:992:990:Memcached daemon:/run/memcached:/sbin/nologin
redis:x:991:989:Redis Database Server:/var/lib/redis:/sbin/nologin
derektest:x:1001:1001::/home/derektest:/bin/bash
[root@VM_0_9_centos shell_learn]#

（7）/pattern1/,linenumber

從nginx行開始到35行結束

[root@VM_0_9_centos shell_learn]# sed -n '/nginx/,35p' /etc/passwd
nginx:x:993:991:Nginx web server:/var/lib/nginx:/sbin/nologin
memcached:x:992:990:Memcached daemon:/run/memcached:/sbin/nologin
redis:x:991:989:Redis Database Server:/var/lib/redis:/sbin/nologin
derektest:x:1001:1001::/home/derektest:/bin/bash
derek:x:1002:1002::/home/derek:/bin/bash
[root@VM_0_9_centos shell_learn]# 

4.3.sed中的刪除

（1）p

查詢

sed -n '1p' test.txt

（2）d  刪除

刪除1~3行

sed -i '1,3d' test.txt

刪除以 ‘’Beau‘’開頭，到以「Simp」開頭，中間全部的行

test.txt

[root@VM_0_9_centos shell_learn]# cat test.txt 
Beautiful is better than ugly.
Explicit is better than implicit.
Simple is better than complex.
Complex is better than complicated.
Flat is better than nested.
Sparse is better than dense.
Readability counts.
[root@VM_0_9_centos shell_learn]#

sed -i '/^Beau/,/^Simp/d' test.txt

[root@VM_0_9_centos shell_learn]# cat test.txt 
Complex is better than complicated.
Flat is better than nested.
Sparse is better than dense.
Readability counts

4.4.sed中的增長

（3）a

匹配到的行後追加內容

[root@VM_0_9_centos shell_learn]# cat test.txt 
Complex is better than complicated.
Flat is better than nested.
Sparse is better than dense.
Readability counts.
[root@VM_0_9_centos shell_learn]# sed -i '/Flat/a zhang-derek' test.txt 
[root@VM_0_9_centos shell_learn]# 
[root@VM_0_9_centos shell_learn]# cat test.txt 
Complex is better than complicated.
Flat is better than nested.
zhang-derek
Sparse is better than dense.
Readability counts.
[root@VM_0_9_centos shell_learn]# 

（4）i

匹配到的行前追加內容

[root@VM_0_9_centos shell_learn]# cat test.txt 
Complex is better than complicated.
Flat is better than nested.
Sparse is better than dense.
Readability counts.
[root@VM_0_9_centos shell_learn]# sed -i '/Flat/i zhang-derek' test.txt 
[root@VM_0_9_centos shell_learn]# 
[root@VM_0_9_centos shell_learn]# cat test.txt 
Complex is better than complicated.
zhang-derek
Flat is better than nested.
Sparse is better than dense.
Readability counts.
[root@VM_0_9_centos shell_learn]# 

（5）r

將後面指定文件的內容追加到爬匹配的行後面

[root@VM_0_9_centos shell_learn]# cat list.txt 
xxxxxxxxxxx
yyyyyyyyyyy
[root@VM_0_9_centos shell_learn]# cat test.txt 
Complex is better than complicated.
Flat is better than nested.
Sparse is better than dense.
Readability counts.
[root@VM_0_9_centos shell_learn]# sed -i '/Flat/r list.txt' test.txt 
[root@VM_0_9_centos shell_learn]# 
[root@VM_0_9_centos shell_learn]# cat test.txt 
Complex is better than complicated.
Flat is better than nested.
xxxxxxxxxxx
yyyyyyyyyyy
Sparse is better than dense.
Readability counts.
[root@VM_0_9_centos shell_learn]#

（6）w

將匹配到的行內容另存到其它文件

[root@VM_0_9_centos shell_learn]# cat test.txt 
Complex is better than complicated.
Flat is better than nested.
xxxxxxxxxxx
yyyyyyyyyyy
Sparse is better than dense.
Readability counts.
[root@VM_0_9_centos shell_learn]# touch 1.txt
[root@VM_0_9_centos shell_learn]# sed -i '/^xxx/,/^yyy/w 1.txt' test.txt 
[root@VM_0_9_centos shell_learn]# 
[root@VM_0_9_centos shell_learn]# cat 1.txt 
xxxxxxxxxxx
yyyyyyyyyyy
[root@VM_0_9_centos shell_learn]# 

4.5.sed中的修改

• s/pattern/string         只替換一行中的第一個
• s/pattern/string/g      所有行內所有替換
• s/pattern/string/ig      所有替換，而且不區分大小寫 

 實例

[root@VM_0_9_centos shell_learn]# cat 2.txt 
i like python
i like english
I like django
I like flask,flask,flask
[root@VM_0_9_centos shell_learn]# sed -i 's/flask/Flask/g' 2.txt 
[root@VM_0_9_centos shell_learn]# 
[root@VM_0_9_centos shell_learn]# cat 2.txt 
i like python
i like english
I like django
I like Flask,Flask,Flask

4.6.反向引用

實例一

[root@VM_0_9_centos shell_learn]# cat 3.txt 
hadAAp
hadBBp
hadCCp
hadDDp
[root@VM_0_9_centos shell_learn]# sed -i 's/had..p/&ss/g' 3.txt 
[root@VM_0_9_centos shell_learn]# 
[root@VM_0_9_centos shell_learn]# cat 3.txt 
hadAApss
hadBBpss
hadCCpss
hadDDpss
[root@VM_0_9_centos shell_learn]#

說明：「&」表示前面匹配到的內容，結果就是在匹配到的全部內容後面加上「ss」

實例二

[root@VM_0_9_centos shell_learn]# cat 3.txt 
hadAApss
hadBBpss
hadCCpss
hadDDpss
[root@VM_0_9_centos shell_learn]# sed -i 's/\(had\)...../\1derek/g' 3.txt 
[root@VM_0_9_centos shell_learn]# 
[root@VM_0_9_centos shell_learn]# cat 3.txt 
hadderek
hadderek
hadderek
hadderek
[root@VM_0_9_centos shell_learn]#

說明：「\1」和「&」的區別是「\1」能夠反向引用匹配到的內容的一部分，而後對其修改，「&」只能對匹配的內容總體修改，不能拆分