本篇主要寫一些shell腳本文本處理工具sed的使用。正則表達式
概述
sed（Stream EDitor）是一個強大而簡單的文本解析轉換工具，能夠讀取文本，並根據指定的條件對文本內容進行編輯（刪除、替換、添加、移動等），最後輸出全部行或者僅輸出處理的某些行。shell
sed也能夠在無交互的狀況下實現相複雜的文本處理操做，被普遍應用於Shell腳本中，用以完成各類自動化處理任務。express
sed的工做流程主要包括讀取、執行和顯示三個過程：vim
- 讀取：
sed從輸入流（文件、管道、標準輸入）中讀取一行內容並存儲到臨時的緩衝區中（又稱模式空間，pattern space） - 執行：默認狀況下，全部的
sed命令都在模式空間中順序地執行，除非指定了行的地址，不然sed命令將會在全部的行上依次執行 - 顯示：發送修改後的內容到輸出流。再發送數據後，模式空間將會被清空
在全部的文件內容都被處理完成以前，上述過程將重複執行，直至全部內容被處理完。bash
默認狀況下,全部的sed命令都是在模式空間內執行的，所以輸入的文件並不會發生任何變化，除非是用重定向存儲輸出。less
命令常見用法
- 命令有兩種格式：
sed [選項] '編輯指令' 文件1 文件2... sed [選項] -f 腳本文件 文件1 文件2...
- 常見的命令選項
-e或--expression=：表示用指定命令或者腳原本處理輸入的文本文件 -f或--file=：表示用指定的腳本文件來處理輸入的文本文件 -h或--help：顯示幫助 -n、--quiet或silent：表示僅顯示處理後的結果 -i：直接編輯文本文件dom
- 操做
一般狀況下是採用的[n1[,n2]]操做參數的格式。工具
a：增長，在當前行下面增長一行指定內容 c：替換，將選定行替換爲指定內容 d：刪除，刪除選定的行 i：插入，在選定行上面插入一行指定內容 p：打印，若是同時指定行，表示打印指定行；若是不指定行，則表示打印全部內容；若是有非打印字符，則以ASCII碼輸出。其一般與-n選項一塊兒使用 s：替換，替換指定字符 y：字符轉換ui
示例
輸出符合條件的文本「p」
- 輸出全部內容,等同於
cat test.txt
[root@localhost ~]# sed -n 'p' test.txt he was short and fat. He was wearing a blue polo shirt with black pants. The home of Football on BBC Sport online. the tongue is boneless but it breaks bones.12! google is the best tools for search keyword. The year ahead will test our political establishment to the limit. PI=3.141592653589793238462643383249901429 a wood cross! Actions speak louder than words #woood # #woooooood # AxyzxyzxyzxyzC I bet this place is really spooky late at night! Misfortunes never come alone/single. I shouldn't have lett so tast.
- 輸出第
3行
[root@localhost ~]# sed -n '3p' test.txt The home of Football on BBC Sport online.
- 輸出
3~5行
[root@localhost ~]# sed -n '3,5p' test.txt The home of Football on BBC Sport online. the tongue is boneless but it breaks bones.12! google is the best tools for search keyword.
- 輸出全部奇數行，
n表示讀入下一行資料
[root@localhost ~]# sed -n 'p;n' test.txt he was short and fat. The home of Football on BBC Sport online. google is the best tools for search keyword. PI=3.141592653589793238462643383249901429 Actions speak louder than words #woooooood # I bet this place is really spooky late at night! I shouldn't have lett so tast.
- 輸出全部偶數行，
n表示讀入下一行資料
[root@localhost ~]# sed -n 'n;p' test.txt He was wearing a blue polo shirt with black pants. the tongue is boneless but it breaks bones.12! The year ahead will test our political establishment to the limit. a wood cross! #woood # AxyzxyzxyzxyzC Misfortunes never come alone/single.
- 輸出第
1~5行之間的奇數行（1，3，5）
[root@localhost ~]# sed -n '1,5{n;p}' test.txt
He was wearing a blue polo shirt with black pants.
the tongue is boneless but it breaks bones.12!
The year ahead will test our political establishment to the limit.
- 輸出第
10行至文件尾之間的偶數行（11，13，15）
[root@localhost ~]# sed -n '10,${n;p}' test.txt
#woooooood #
I bet this place is really spooky late at night!
I shouldn't have lett so tast.
sed命令結合正則表達式時，格式略有不一樣，正則表達式以/包圍。this
- 輸出包含
the的行
[root@localhost ~]# sed -n '/the/p' test.txt the tongue is boneless but it breaks bones.12! google is the best tools for search keyword. The year ahead will test our political establishment to the limit.
- 輸出從第
4行至第一個包含the的行
[root@localhost ~]# sed -n '4,/the/p' test.txt the tongue is boneless but it breaks bones.12! google is the best tools for search keyword.
- 輸出包含
the的行所在的行號,等號=用來輸出行號
[root@localhost ~]# sed -n '/the/=' test.txt 4 5 6
- 輸出以
PI開頭的行
[root@localhost ~]# sed -n '/^PI/p' test.txt PI=3.141592653589793238462643383249901429
- 輸出以數字結尾的行
[root@localhost ~]# sed -n '/[0-9]$/p' test.txt PI=3.141592653589793238462643383249901429
- 輸出包含單詞
wood的行,\<、\>表明單詞邊界
[root@localhost ~]# sed -n '/\<wood\>/p' test.txt a wood cross!
刪除符合條件的文本「d」
nl命令用來計算文件的行數
- 刪除第
3行
[root@localhost ~]# nl test.txt | sed '3d'
1 he was short and fat.
2 He was wearing a blue polo shirt with black pants.
4 the tongue is boneless but it breaks bones.12!
5 google is the best tools for search keyword.
6 The year ahead will test our political establishment to the limit.
7 PI=3.141592653589793238462643383249901429
8 a wood cross!
9 Actions speak louder than words
10 #woood #
11 #woooooood #
12 AxyzxyzxyzxyzC
13 I bet this place is really spooky late at night!
14 Misfortunes never come alone/single.
15 I shouldn't have lett so tast.
- 刪除第
3~5行
[root@localhost ~]# nl test.txt | sed '3,5d'
1 he was short and fat.
2 He was wearing a blue polo shirt with black pants.
6 The year ahead will test our political establishment to the limit.
7 PI=3.141592653589793238462643383249901429
8 a wood cross!
9 Actions speak louder than words
10 #woood #
11 #woooooood #
12 AxyzxyzxyzxyzC
13 I bet this place is really spooky late at night!
14 Misfortunes never come alone/single.
15 I shouldn't have lett so tast.
- 刪除包含
cross的行,本來的第8行被刪除
[root@localhost ~]# nl test.txt | sed '/cross/d'
1 he was short and fat.
2 He was wearing a blue polo shirt with black pants.
3 The home of Football on BBC Sport online.
4 the tongue is boneless but it breaks bones.12!
5 google is the best tools for search keyword.
6 The year ahead will test our political establishment to the limit.
7 PI=3.141592653589793238462643383249901429
9 Actions speak louder than words
10 #woood #
11 #woooooood #
12 AxyzxyzxyzxyzC
13 I bet this place is really spooky late at night!
14 Misfortunes never come alone/single.
15 I shouldn't have lett so tast.
- 刪除不包含
cross的行,用!符號表示取反操做
[root@localhost ~]# nl test.txt | sed '/cross/!d'
8 a wood cross!
- 刪除以小寫字母開頭的行
[root@localhost ~]# sed '/^[a-z]/d' test.txt He was wearing a blue polo shirt with black pants. The home of Football on BBC Sport online. The year ahead will test our political establishment to the limit. PI=3.141592653589793238462643383249901429 Actions speak louder than words #woood # #woooooood # AxyzxyzxyzxyzC I bet this place is really spooky late at night! Misfortunes never come alone/single. I shouldn't have lett so tast.
- 刪除以
.結尾的行
[root@localhost ~]# sed '/\.$/d' test.txt the tongue is boneless but it breaks bones.12! PI=3.141592653589793238462643383249901429 a wood cross! Actions speak louder than words #woood # #woooooood # AxyzxyzxyzxyzC I bet this place is really spooky late at night!
- 刪除全部空行
[root@localhost ~]# sed '/^$/d' test.txt he was short and fat. He was wearing a blue polo shirt with black pants. The home of Football on BBC Sport online. the tongue is boneless but it breaks bones.12! google is the best tools for search keyword. The year ahead will test our political establishment to the limit. PI=3.141592653589793238462643383249901429 a wood cross! Actions speak louder than words #woood # #woooooood # AxyzxyzxyzxyzC I bet this place is really spooky late at night! Misfortunes never come alone/single. I shouldn't have lett so tast.
- 刪除重複的空行，即連續空行只保留一個
[root@localhost ~]# sed '/^$/{n;/^$/d}' test.txt
he was short and fat.
He was wearing a blue polo shirt with black pants.
The home of Football on BBC Sport online.
the tongue is boneless but it breaks bones.12!
google is the best tools for search keyword.
The year ahead will test our political establishment to the limit.
PI=3.141592653589793238462643383249901429
a wood cross!
Actions speak louder than words
#woood #
#woooooood #
AxyzxyzxyzxyzC
I bet this place is really spooky late at night!
Misfortunes never come alone/single.
I shouldn't have lett so tast.
[root@localhost ~]# cat -s test.txt he was short and fat. He was wearing a blue polo shirt with black pants. The home of Football on BBC Sport online. the tongue is boneless but it breaks bones.12! google is the best tools for search keyword. The year ahead will test our political establishment to the limit. PI=3.141592653589793238462643383249901429 a wood cross! Actions speak louder than words #woood # #woooooood # AxyzxyzxyzxyzC I bet this place is really spooky late at night! Misfortunes never come alone/single.
替換符合條件的文本
s（字符串替換） c（整行/整塊替換） y（字符轉換）
- 將每行中的第一個
the替換爲THE
[root@localhost ~]# sed 's/the/THE/' test.txt he was short and fat. He was wearing a blue polo shirt with black pants. The home of Football on BBC Sport online. THE tongue is boneless but it breaks bones.12! google is THE best tools for search keyword. The year ahead will test our political establishment to THE limit. PI=3.141592653589793238462643383249901429 a wood cross! Actions speak louder than words #woood # #woooooood # AxyzxyzxyzxyzC I bet this place is really spooky late at night! Misfortunes never come alone/single. I shouldn't have lett so tast.
- 將每行中的第
3個l替換爲L
[root@localhost ~]# sed 's/l/L/3' test.txt he was short and fat. He was wearing a blue polo shirt with bLack pants. The home of Football on BBC Sport onLine. the tongue is boneless but it breaks bones.12! google is the best tools for search keyword. The year ahead will test our poLitical establishment to the limit. PI=3.141592653589793238462643383249901429 a wood cross! Actions speak louder than words #woood # #woooooood # AxyzxyzxyzxyzC I bet this place is realLy spooky late at night! Misfortunes never come alone/single. I shouldn't have lett so tast.
- 將文件中的全部
the替換爲THE
[root@localhost ~]# sed 's/the/THE/g' test.txt he was short and fat. He was wearing a blue polo shirt with black pants. The home of Football on BBC Sport online. THE tongue is boneless but it breaks bones.12! google is THE best tools for search keyword. The year ahead will test our political establishment to THE limit. PI=3.141592653589793238462643383249901429 a wood cross! Actions speak louder than words #woood # #woooooood # AxyzxyzxyzxyzC I bet this place is really spooky late at night! Misfortunes never come alone/single. I shouldn't have lett so tast.
- 將文件中的全部
o刪除(替換爲空串)
[root@localhost ~]# sed 's/o//g' test.txt he was shrt and fat. He was wearing a blue pl shirt with black pants. The hme f Ftball n BBC Sprt nline. the tngue is bneless but it breaks bnes.12! ggle is the best tls fr search keywrd. The year ahead will test ur plitical establishment t the limit. PI=3.141592653589793238462643383249901429 a wd crss! Actins speak luder than wrds #wd # #wd # AxyzxyzxyzxyzC I bet this place is really spky late at night! Misfrtunes never cme alne/single. I shuldn't have lett s tast.
- 在每行行首插入
#號
[root@localhost ~]# sed 's/^/#/' test.txt #he was short and fat. #He was wearing a blue polo shirt with black pants. #The home of Football on BBC Sport online. #the tongue is boneless but it breaks bones.12! #google is the best tools for search keyword. #The year ahead will test our political establishment to the limit. #PI=3.141592653589793238462643383249901429 #a wood cross! #Actions speak louder than words # # ##woood # ##woooooood # #AxyzxyzxyzxyzC #I bet this place is really spooky late at night! #Misfortunes never come alone/single. #I shouldn't have lett so tast.
- 在包含
the的每行行首插入#號
[root@localhost ~]# sed '/the/s/^/#/' test.txt he was short and fat. He was wearing a blue polo shirt with black pants. The home of Football on BBC Sport online. #the tongue is boneless but it breaks bones.12! #google is the best tools for search keyword. #The year ahead will test our political establishment to the limit. PI=3.141592653589793238462643383249901429 a wood cross! Actions speak louder than words #woood # #woooooood # AxyzxyzxyzxyzC I bet this place is really spooky late at night! Misfortunes never come alone/single. I shouldn't have lett so tast.
- 在每行行尾插入字符串
EOF
[root@localhost ~]# sed 's/$/EOF/' test.txt he was short and fat.EOF He was wearing a blue polo shirt with black pants.EOF The home of Football on BBC Sport online.EOF the tongue is boneless but it breaks bones.12!EOF google is the best tools for search keyword.EOF The year ahead will test our political establishment to the limit.EOF PI=3.141592653589793238462643383249901429EOF a wood cross!EOF Actions speak louder than wordsEOF EOF EOF #woood #EOF #woooooood #EOF AxyzxyzxyzxyzCEOF I bet this place is really spooky late at night!EOF Misfortunes never come alone/single.EOF I shouldn't have lett so tast.EOF
- 將第
3~5行中的全部the替換爲THE
[root@localhost ~]# sed '3,5s/the/THE/g' test.txt he was short and fat. He was wearing a blue polo shirt with black pants. The home of Football on BBC Sport online. THE tongue is boneless but it breaks bones.12! google is THE best tools for search keyword. The year ahead will test our political establishment to the limit. PI=3.141592653589793238462643383249901429 a wood cross! Actions speak louder than words #woood # #woooooood # AxyzxyzxyzxyzC I bet this place is really spooky late at night! Misfortunes never come alone/single. I shouldn't have lett so tast.
- 將包含
the的全部行中的o都替換爲O
[root@localhost ~]# sed '/the/s/o/O/g' test.txt he was short and fat. He was wearing a blue polo shirt with black pants. The home of Football on BBC Sport online. the tOngue is bOneless but it breaks bOnes.12! gOOgle is the best tOOls fOr search keywOrd. The year ahead will test Our pOlitical establishment tO the limit. PI=3.141592653589793238462643383249901429 a wood cross! Actions speak louder than words #woood # #woooooood # AxyzxyzxyzxyzC I bet this place is really spooky late at night! Misfortunes never come alone/single. I shouldn't have lett so tast.
遷移符合條件的文本
H（複製到剪貼板） g、G（將剪貼板中的數據覆蓋/追加至指定行） w（保存爲文件） r（讀取指定文件） a（追加指定內容）
- 將包含
the的行遷移至文件末尾，{;}用於多個操做
[root@localhost ~]# sed '/the/{H;d};$G' test.txt
he was short and fat.
He was wearing a blue polo shirt with black pants.
The home of Football on BBC Sport online.
PI=3.141592653589793238462643383249901429
a wood cross!
Actions speak louder than words
#woood #
#woooooood #
AxyzxyzxyzxyzC
I bet this place is really spooky late at night!
Misfortunes never come alone/single.
I shouldn't have lett so tast.
the tongue is boneless but it breaks bones.12!
google is the best tools for search keyword.
The year ahead will test our political establishment to the limit.
- 將第
1~5行內容轉移至第17行後
[root@localhost ~]# sed '1,5{H;d};17G' test.txt
The year ahead will test our political establishment to the limit.
PI=3.141592653589793238462643383249901429
a wood cross!
Actions speak louder than words
#woood #
#woooooood #
AxyzxyzxyzxyzC
I bet this place is really spooky late at night!
Misfortunes never come alone/single.
I shouldn't have lett so tast.
he was short and fat.
He was wearing a blue polo shirt with black pants.
The home of Football on BBC Sport online.
the tongue is boneless but it breaks bones.12!
google is the best tools for search keyword.
- 將包含
the的行另存爲文件out.file
[root@localhost ~]# sed '/the/w out.file' test.txt he was short and fat. He was wearing a blue polo shirt with black pants. The home of Football on BBC Sport online. the tongue is boneless but it breaks bones.12! google is the best tools for search keyword. The year ahead will test our political establishment to the limit. PI=3.141592653589793238462643383249901429 a wood cross! Actions speak louder than words #woood # #woooooood # AxyzxyzxyzxyzC I bet this place is really spooky late at night! Misfortunes never come alone/single. I shouldn't have lett so tast.
[root@localhost ~]# cat out.file the tongue is boneless but it breaks bones.12! google is the best tools for search keyword. The year ahead will test our political establishment to the limit.
- 將文件
/etc/hostname的內容添加到包含the的每行之後
[root@localhost ~]# sed '/the/r /etc/hostname' test.txt he was short and fat. He was wearing a blue polo shirt with black pants. The home of Football on BBC Sport online. the tongue is boneless but it breaks bones.12! localhost.localdomain google is the best tools for search keyword. localhost.localdomain The year ahead will test our political establishment to the limit. localhost.localdomain PI=3.141592653589793238462643383249901429 a wood cross! Actions speak louder than words #woood # #woooooood # AxyzxyzxyzxyzC I bet this place is really spooky late at night! Misfortunes never come alone/single. I shouldn't have lett so tast.
- 在第
3行後插入一個新行,內容爲New
[root@localhost ~]# sed '3aNew' test.txt he was short and fat. He was wearing a blue polo shirt with black pants. The home of Football on BBC Sport online. New the tongue is boneless but it breaks bones.12! google is the best tools for search keyword. The year ahead will test our political establishment to the limit. PI=3.141592653589793238462643383249901429 a wood cross! Actions speak louder than words #woood # #woooooood # AxyzxyzxyzxyzC I bet this place is really spooky late at night! Misfortunes never come alone/single. I shouldn't have lett so tast.
- 在包含
the的每行後插入一個新行，內容爲New
[root@localhost ~]# sed '/the/aNew' test.txt he was short and fat. He was wearing a blue polo shirt with black pants. The home of Football on BBC Sport online. the tongue is boneless but it breaks bones.12! New google is the best tools for search keyword. New The year ahead will test our political establishment to the limit. New PI=3.141592653589793238462643383249901429 a wood cross! Actions speak louder than words #woood # #woooooood # AxyzxyzxyzxyzC I bet this place is really spooky late at night! Misfortunes never come alone/single. I shouldn't have lett so tast.
- 在第
3行後插入多行內容，中間的\n表示換行
[root@localhost ~]# sed '3aNew1\nNew2' test.txt he was short and fat. He was wearing a blue polo shirt with black pants. The home of Football on BBC Sport online. New1 New2 the tongue is boneless but it breaks bones.12! google is the best tools for search keyword. The year ahead will test our political establishment to the limit. PI=3.141592653589793238462643383249901429 a wood cross! Actions speak louder than words #woood # #woooooood # AxyzxyzxyzxyzC I bet this place is really spooky late at night! Misfortunes never come alone/single. I shouldn't have lett so tast.
使用腳本編輯文件
將多個編輯指令存放到文件中（每行一條編輯指令），經過-f選項來調用
- 將第
1~5行內容轉移至第17行後，以上操做能夠改用腳本文件方式：
[root@localhost ~]# vim opt.list 1,5H 1,5d 17G
[root@localhost ~]# sed -f opt.list test.txt The year ahead will test our political establishment to the limit. PI=3.141592653589793238462643383249901429 a wood cross! Actions speak louder than words #woood # #woooooood # AxyzxyzxyzxyzC I bet this place is really spooky late at night! Misfortunes never come alone/single. I shouldn't have lett so tast. he was short and fat. He was wearing a blue polo shirt with black pants. The home of Football on BBC Sport online. the tongue is boneless but it breaks bones.12! google is the best tools for search keyword.
直接操做文件示例
#!/bin/bash # 指定樣本文件路徑、配置文件路徑 SAMPLE="/usr/share/doc/vsftpd-3.0.2/EXAMPLE/INTERNET_SITE/vsftpd.conf" CONFIG="/etc/vsftpd/vsftpd.conf" # 備份原來的配置文件,檢測文件名爲/etc/vsftpd/vsftpd.conf.bak 備份文件是否存在, 若不存在則使用 cp 命令進行文件備份 [ ! -e "$CONFIG.bak" ] && cp $CONFIG $CONFIG.bak # 基於樣本配置進行調整,覆蓋現有文件 sed -e '/^anonymous_enable/s/YES/NO/g' $SAMPLE > $CONFIG sed -i -e '/^local_enable/s/NO/YES/g' -e '/^write_enable/s/NO/YES/g' $CONFIG grep "listen" $CONFIG || sed -i '$alisten=YES' $CONFIG # 啓動vsftpd 服務,並設爲開機後自動運行 systemctl restart vsftpd systemctl enable vsftpd