sed進階

下面這些命令未必常常會用到，但當須要時，知道這些確定是件好事。

1、多行命令

sed命令一般是對一行數據進行處理，而後下一行重複處理。

sed編輯器包含了三個可用來處理多行文本的特殊命令

• N：將數據流中的下一行加進來建立一個多行組來處理
• D：刪除多行組中的一行
• P：打印多行組中的一行

1.1 next命令

兩種刪除匹配的下一行的辦法：

cat data1.txt 
This is the header line.

This is a data line.

This is the last line.

sed '/^$/d' data1.txt 
This is the header line.
This is a data line.
This is the last line.

sed '/header/{n ; d}' data1.txt 
This is the header line.
This is a data line.

This is the last line.

空行處理

 

1.2 合併文本行

用大寫N，能夠一次性讀取兩行。區別與n是移動到下一行。

$ sed '/first/{ N ; s/\n/ / }' data2.txt 
This is the header line.
This is the first data line. This is the second data line.
This is the last line.

$ cat data2.txt 
This is the header line.
This is the first data line.
This is the second data line.
This is the last line.

$ cat data3.txt 
On Tuesday, the Linux System
Administrator's group meeting will be held.
All System Administrators should attend.
Thank you for your attendance.

$ sed 'N ; s/System.Administrator/Desktop User/' data3.txt
On Tuesday, the Linux Desktop User's group meeting will be held.
All Desktop Users should attend.
Thank you for your attendance.

N使用例子

在N後面最好匹配多行命令，而單行命令則能夠放在N前面，如：

$ sed 'N
> s/System\nAdministrator/Desktop\nUser/
> s/System Administrator/Desktop User/
> ' data4.txt
On Tuesday, the Linux Desktop
User's group meeting will be held.
All System Administrators should attend.

$ cat data4.txt 
On Tuesday, the Linux System
Administrator's group meeting will be held.
All System Administrators should attend.

$ sed '
> s/System Administrator/Desktop User/
> N
> s/System\nAdministrator/Desktop\nUser/
> ' data4.txt
On Tuesday, the Linux Desktop
User's group meeting will be held.
All Desktop Users should attend.

先後寫的例子

 

1.3 刪除多行命令

若是在N後面用d，就會把多行都一塊兒刪除。可是若是用D，就會刪除到\n爲止。

$ sed 'N ; /System\nAdministrator/d' data4.txt 
All System Administrators should attend.
$ cat data4.txt 
On Tuesday, the Linux System
Administrator's group meeting will be held.
All System Administrators should attend.
$ sed 'N ; /System\nAdministrator/D' data4.txt 
Administrator's group meeting will be held.
All System Administrators should attend.

d和D的對比

而後有刪除header前的空白行

$ sed '/^$/{N ; /header/D}' data5.txt 
This is the header line.
This is a date line.

This is the last line.

$ sed '{N ; /header/D}' data5.txt 
This is a date line.

This is the last line.
$ sed '{N ; /header/d}' data5.txt 
This is a date line.

This is the last line.
$ cat data5.txt 

This is the header line.
This is a date line.

This is the last line.

刪除header前的空行

相似的和D類似的，P也是會和N配合輸出第一行

$ cat data3.txt 
On Tuesday, the Linux System
Administrator's group meeting will be held.
All System Administrators should attend.
Thank you for your attendance.
$ sed -n 'N ; /System\nAdministrator/P' data3.txt
On Tuesday, the Linux System

P的使用

 

2、保持空間

sed編輯器有一塊稱做保持空間的緩衝區域。在處理某些行時，能夠用保持空間來臨時保存一些行。

• h：將模式空間那關鍵複製到保持空間
• H：將模式空間附加到保持空間
• g：將保持空間複製到模式空間
• G：將保持空間附加到模式空間
• x：交換模式空間和保持空間的內容

$ cat data2.txt 
This is the header line.
This is the first data line.
This is the second data line.
This is the last line.
$ sed -n '/first/ {h ; p; n ; p ; g ; p }' data2.txt 
This is the first data line.
This is the second data line.
This is the first data line.

$ sed -n '/first/ {h ; n ; p ; g ; p }' data2.txt 
This is the second data line.
This is the first data line.

保持空間

 

3、排除命令

感嘆號命令（！）用來排除命令

$ sed -n '/header/!p' data2.txt 
This is the first data line.
This is the second data line.
This is the last line.
$ sed 'N;
> s/System\nAdministrator/Desktop\nUser/
> s/System Administrator/Desktop User/
> ' data4.txt
On Tuesday, the Linux Desktop
User's group meeting will be held.
All System Administrators should attend.
$ sed '$!N;
> s/System\nAdministrator/Desktop\nUser/
> s/System Administrator/Desktop User/
> ' data4.txt
On Tuesday, the Linux Desktop
User's group meeting will be held.
All Desktop Users should attend.

！例子

而後是將文件反轉的例子：

$ cat data2.txt 
This is the header line.
This is the first data line.
This is the second data line.
This is the last line.
$ sed -n '{1!G ; h ; $p }' data2.txt 
This is the last line.
This is the second data line.
This is the first data line.
This is the header line.

反轉例子

 

4、改變流

4.1 分支指令

基於地址、地址模式或地址區間排除一整塊命令，這容許你只對數據流中的特定行執行一組命令。

分支命令b格式：[address]b [label]

address參數：決定了那些行的數據會觸發分支明林。

label參數：定義了要跳轉到的位置。

$ cat data2.txt 
This is the header line.
This is the first data line.
This is the second data line.
This is the last line.

$ sed '{2,3b ; s/This is/Is this/ ; s/line./test?/}' data2.txt
Is this the header test?
This is the first data line.
This is the second data line.
Is this the last test?

命令b使用例子

上面的例子裏，分支跳過了2,3兩行。若是不想直接跳轉到腳本結尾，可爲分支命令定義一個要跳轉到的標籤。

標籤最多長7個字符，例子：

chen@ubuntu:~/shell/ch21$ sed '{/first/b jump1 ; s/This is the/No jump on/
> :jump1
> s/This is the/Jump here on/}' data2.txt
No jump on header line.
Jump here on first data line.
No jump on second data line.
No jump on last line.

chen@ubuntu:~/shell/ch21$ cat data2.txt 
This is the header line.
This is the first data line.
This is the second data line.
This is the last line.

命令b跳轉例子

若是文中出現first，程序就跳轉到jump1腳本行。若是沒有匹配，sed則會繼續執行腳本中的命令，包括分支標籤後的命令。

chen@ubuntu:~/shell/ch21$ echo "This, is, a, test, to, remove, commas." | sed -n '{
> :start
> s/,//1p
> b start
> }'
This is, a, test, to, remove, commas.
This is a, test, to, remove, commas.
This is a test, to, remove, commas.
This is a test to, remove, commas.
This is a test to remove, commas.
This is a test to remove commas.
^C

命令b用做循環

要能中止循環，只要加上模式匹配在命令b前面就行了

chen@ubuntu:~/shell/ch21$ echo "This, is, a, test, to, remove, commas." | sed -n '{
> :start
> s/,//1p
> /,/b start
> }'
This is, a, test, to, remove, commas.
This is a, test, to, remove, commas.
This is a test, to, remove, commas.
This is a test to, remove, commas.
This is a test to remove, commas.
This is a test to remove commas.

改進例子

 

4.2 測試指令

相似於分支命令，測試test命令（t）也能夠用來改變sed編輯器腳本執行流程。

若是天幻命令成功匹配並替換了一個模式，測試命令就會跳轉到指定標籤。若是替換命令未能匹配指定模式，測試命令就不會跳轉。

測試命令格式與分支命令格式相同：[address]t [label]

和分支命令同樣，若是沒有標籤的狀況下，若是測試成功，sed會跳轉到腳本結尾。

測試命令基本上就是一個if-then

chen@ubuntu:~/shell/ch21$ sed '{
> s/first/matched/
> t
> s/This is the/No match on/
> }' data2.txt
No match on header line.
This is the matched data line.
No match on second data line.
No match on last line.
chen@ubuntu:~/shell/ch21$ cat data2.txt 
This is the header line.
This is the first data line.
This is the second data line.
This is the last line.

測試命令例子

測試命令的循環方法：

chen@ubuntu:~/shell/ch21$ echo "This, is, a, test, to, remove, commas." | sed -n '{
> :start
> s/,//1p
> t start
> }'
This is, a, test, to, remove, commas.
This is a, test, to, remove, commas.
This is a test, to, remove, commas.
This is a test to, remove, commas.
This is a test to remove, commas.
This is a test to remove commas.

測試命令循環方法

 

5、模式替代

 在使用通配符時，很難知道到底哪些文本會匹配模式。

chen@ubuntu:~/shell/ch21$ echo "The cat sleeps in his hat." | sed 's/cat/"cat"/'The "cat" sleeps in his hat.

chen@ubuntu:~/shell/ch21$ echo "The cat sleeps in his hat." | sed 's/.at/".at"/g'
The ".at" sleeps in his ".at".

通配符沒法再引號中使用

5.1 &符號

用&符號能夠用來表明替換命令中的匹配的模式。無論匹配出來什麼樣的文本，均可以使用&符號，來使用這段文本。

myfly2@ubuntu:~/shell/ch21$ echo "The cat sleeps in his hat." | sed 's/.at/"&"/g'
The "cat" sleeps in his "hat".

&符號例子

5.2 替代單獨的單詞

有時候咱們不須要整個字符串，只想提取字符串的一部分。

sed編輯器用圓括號來定義替換模式中的子模式，而後用特殊字符來引用每一個子模式。

chen@ubuntu:~/shell/ch21$ echo "The System Administrator manual" | sed '
> s/\(System\) Administrator/\1 User/'
The System User manual

chen@ubuntu:~/shell/ch21$ echo "The System Administrator manual" | sed '
s/System \(Administrator\)/\1 User/'
The Administrator User manual

chen@ubuntu:~/shell/ch21$ echo "That furry cat is pretty" | sed 's/furry \(.at\)/\1/'
That cat is pretty
chen@ubuntu:~/shell/ch21$ echo "That furry hat is pretty" | sed 's/furry \(.at\)/\1/'
That hat is pretty
#當須要在兩個或多個子模式間插入文本時，這個特性尤爲有用
chen@ubuntu:~/shell/ch21$ echo "1234567" | sed '{
> :start
> s/\(.*[0-9]\)\([0-9]\{3\}\)/\1,\2/
> t start
> }'
1,234,567
#分紅兩部分：
#.*[0-9]
#[0-9]{3}
#第一個子模式是以數字結尾的任意長度的字符。
#第二個子模式是若干組三位數字

子模式例子

 

6、在腳本中使用sed

使用包裝腳本，sed腳本過程繁瑣，若是腳本很長的話。能夠將sed編輯器命令放到shell包裝腳本中。

這樣能夠不用每次都鍵入腳本。

chen@ubuntu:~/shell/ch21$ cat reverse.sh
#!/bin/bash
# Shell wrapper for sed editor script.
#                               to reverse text file lines.
#
sed -n '{ 1!G ; h ; $p }' $1
#
chen@ubuntu:~/shell/ch21$ chmod +x reverse.sh
chen@ubuntu:~/shell/ch21$ ls
data2.txt  data4.txt  reverse.sh
chen@ubuntu:~/shell/ch21$ ./reverse.sh data2.txt
This is the last line.
This is the second data line.
This is the first data line.
This is the header line.

寫成文件形式的sed

6.2 重定向sed的輸出

chen@ubuntu:~/shell/ch21$ cat fact.sh
#!/bin/bash
# Add commas to number in factorial answer
#
factorial=1
counter=1
number=$1
#
while [ $counter -le $number ]
do
        factorial=$[ $factorial * $counter ]
        counter=$[ $counter + 1 ]
done
#
result=$(echo $factorial | sed '{
:start
s/\(.*[0-9]\)\([0-9]\{3\}\)/\1,\2/
t start
}')
#
echo "The result is $result"
#
chen@ubuntu:~/shell/ch21$ ./fact.sh 20
The result is 2,432,902,008,176,640,000

能夠發sed結果保存

 

7、建立sed實用工具

7.1 加倍行間距

chen@ubuntu:~/shell/ch21$ sed 'G' data2.txt
This is the header line.

This is the first data line.

This is the second data line.

This is the last line.

chen@ubuntu:~/shell/ch21$ sed '$!G' data2.txt
This is the header line.

This is the first data line.

This is the second data line.

This is the last line.

加倍行間距

 

7.2 對可能還有空白行的文件加倍行間距

#先刪除原來空行，而後再加空行
chen@ubuntu:~/shell/ch21$ sed '/^$/d;$!G' data6.txt
This is line one.

This is line two.

This is line three.

This is line four.
chen@ubuntu:~/shell/ch21$

加倍行間距例子

 

7.3 給文件中的行編行

chen@ubuntu:~/shell/ch21$ sed '=' data2.txt
1
This is the header line.
2
This is the first data line.
3
This is the second data line.
4
This is the last line.
chen@ubuntu:~/shell/ch21$ sed '=' data2.txt | sed 'N; s/\n/ /'
1 This is the header line.
2 This is the first data line.
3 This is the second data line.
4 This is the last line.
chen@ubuntu:~/shell/ch21$


#=號命令增長行號
#N命令將兩行合併成一行
#而後把換行符換成空格

增長行號的腳本

 

7.4 打印末尾行

chen@ubuntu:~/shell/ch21$ sed '{
:start
$q ; N ; 11,$D
b start
}' data7.txt
This is line 6.
This is line 7.
This is line 8.
This is line 9.
This is line 10.
This is line 11.
This is line 12.
This is line 13.
This is line 14.
This is line 15.
chen@ubuntu:~/shell/ch21$ cat data7.txt
This is line 1.
This is line 2.
This is line 3.
This is line 4.
This is line 5.
This is line 6.
This is line 7.
This is line 8.
This is line 9.
This is line 10.
This is line 11.
This is line 12.
This is line 13.
This is line 14.
This is line 15.

打印行尾

 

7.5 刪除行

刪除不須要的空白行，若是有選擇的刪除空白行，須要一點創造力。

7.5.1 刪除連續的空白行

chen@ubuntu:~/shell/ch21$ sed '/./,/^$/!d' data8.txt
This is line one.

This is line two.

This is line three.

This is line four.
chen@ubuntu:~/shell/ch21$ cat data8.txt
This is line one.

This is line two.

This is line three.


This is line four.

刪除連續的空白行例子

 

7.5.2 刪除開頭的空白行

chen@ubuntu:~/shell/ch21$ cat data9.txt


This is line one.

This is line two.
chen@ubuntu:~/shell/ch21$ sed '/./,$!d' data9.txt
This is line one.

This is line two.

腳本例子

 

7.5.3 刪除結尾的空白行

1 chen@ubuntu:~/shell/ch21$ sed '{
2 > :start
3 > /^\n*$/{$d ; N ; b start}
4 > }' data10.txt
5 This is the first line.
6 This is the seconde line.
7 chen@ubuntu:~/shell/ch21$ cat data10.txt
8 This is the first line.
9 This is the seconde line.

例子

 

7.6 刪除HTML標籤