經過《 The Linux Command Line》入門linux命令行

時間 2019-11-20

標籤經過 linux command line 入門命令行欄目 Linux 简体版

原文原文鏈接

原文發於個人獨立博客：經過《The Linux Command Line》入門linux命令行php

此書網站：The Linux Command Line ，它是免費的。css

它有中文版，並且還有一個好聽的名字：快樂的 Linux 命令行html

學習有兩種方法，一種是系統的學，一種是根據實際需求來學。兩種各有優缺點，前一種，知識不會有缺漏，對往後的融會貫通和舉一反三有好處，可是這樣慢，並且有可能不如針對性學習的精神集中。後一種，只找需求的學，可以任務完成就行，這樣快，它的缺點就是前一種的優勢。顯然通看整本書屬於第一種。node

做者寫的很好，娓娓道來，講的很清楚，就像一本故事書同樣。我很喜歡這樣的書。做者比較幽默，一些用詞看了會讓人發笑。linux

這本書的內容很淺，真的只是入門。nginx

正如做者所說，這是一場linux的journey(旅行)，就把看書當旅行吧。git

如下是隨手記的，只適合本身看。github

命令

date
cal-calendar
df-current amount of free space on your disk drives
free-display the amount of free memory
exit-closing the terminal emulator window

pwd - Print name of current working directory
cd - Change directory

cd             home directory
cd -           previous working directory
cd ~user_name  home directory of user
.              點號表明當前文件夾

ls - List directory contents

ls -a   all files
ls -l   long format
ls -lt  sort the result by the file's modification time
ls -lt --reverse

file – Determine file typesql
less – View file contentsshell

和cat的區別？參考

4 - Manipulating Files And Directories

cp

若是文件已經存在，直接覆蓋，沒有提示，要提示用-i
mv

move (rename) files
mkdir

mkdir dir1 dir2

rm – Remove Files And Directories

rm -r playground    # 遞歸刪除

ln – Create Links

ln myfile hard       # hard link
ln -s myfile soft    # symbolic link

軟連接」和「硬連接」的區別：硬連接指向磁盤區塊，軟連接指向絕對路徑。

hard links的缺點：

1.cannot span physical devices.

2.cannot reference directories, only files.

5 - Working With Commands

type – Indicate how a command name is interpreted

tsc@tsc:~$ type ls
ls is aliased to `ls --color=auto'  # alias

tsc@tsc:~$ type cd
cd is a shell builtin               # builtin

tsc@tsc:~$ type cp
cp is /bin/cp                       # executable program

which – Display An Executable’s Location

tsc@tsc:~$ which cp
/bin/cp

只用於exe

help – Get help for shell builtins

注意是shell builtin

tsc@tsc:/bin$ help cd
cd: cd [-L|[-P [-e]] [-@]] [dir]
    Change the shell working directory.

理解註釋：[]是可選，|是互斥。

--help – Display Usage Information

tsc@tsc:~$ mkdir --help

man – Display a command’s manual page

沒有example

tsc@tsc:~$ man ls

apropos – Display a list of appropriate commands
info – Display a command’s info entry
whatis – Display a very brief description of a command
alias – Create an alias for a command

起別名前先檢查是否存在，用type：

tsc@tsc:~$ type test
test is a shell builtin

使用：

alias name='string' # name後不能有空格
tsc@tsc:~$ alias foo='cd /usr; ls; cd -'
tsc@tsc:~$ foo

終端關閉後做用就消失了。

去除：

tsc@tsc:~$ unalias foo

trick:多條命令於一行

command1; command2; command3...
tsc@tsc:~$ cd /usr; ls

man .info .–help的用法和區別

help最簡單，info最詳細，man在二者之間。

6 – Redirection

cat - Concatenate files

cat file1 # 輸出文件到屏幕，沒有分頁
cat > file.txt # cat沒有參數時接受stdin
this is test.
               # ctrl+d 結束
# < 接受文件輸入
tsc@tsc:~$ cat < lazy_dog.txt

sort - Sort lines of text
uniq - Report or omit repeated lines

# -d 顯示重複的
tsc@tsc:~$ ls /bin /usr/bin | sort | uniq -d | less

grep - Print lines matching a pattern

「global regular expression print」,find功能。

tsc@tsc:~$ ls /bin /usr/bin | sort | uniq | grep zip

wc - Print newline, word, and byte counts for each file

聯合使用看條目個數：

tsc@tsc:~$ ls /bin /usr/bin | sort | uniq | wc -l

head - Output the first part of a file
tail - Output the last part of a file
tee - Read from standard input and write to standard output and files

這個名字頗有意思，tee有三通管的意思，配合pipe使用，它的做用是從stdin讀入，複製到stdout和文件。

# 一方面輸出到ls.txt，一方面傳給grep
tsc@tsc:~$ ls /usr/bin/ | tee ls.txt | grep zip

I/O redirection 的做用

I/O redirection allows us to change where output goes and where input comes from.
>重定向

tsc@tsc:~$ ls -l /usr/bin/ > ls-output.txt
tsc@tsc:~$ > ls-output2.txt  # 建立一個空文件

要注意重複>時，原來的文件會被清空。

>> 追加

tsc@tsc:~$ ls -l /usr/bin >> ls-output2.txt

文件不存在會新建。

輸出error

tsc@tsc:~$ ls -l /bin/usr 2> ls-error.txt

個人問題：在不知道有沒有error的狀況下，>和2>要如何同時執行？

都輸出到一個文件

有兩種方法：

# 1.老版本
# 順序很重要
tsc@tsc:~$ ls -l /bin/usr > ls-output.txt 2>&1

# 2.新，用&>
tsc@tsc:~$ ls -l /bin/usr &> ls-output.txt
tsc@tsc:~$ ls -l /bin/usr &>> ls-output.txt # 追加

Pipelines

tsc@tsc:~$ ls -l /usr/bin | less

千萬不要把>用成|，否則有些後果很嚴重。

filters

tsc@tsc:~$ ls /bin /usr/bin | sort | less

7 – Seeing The World As The Shell Sees It

這節講expansion，就是*、~之類的，本質就是變量。

echo – Display a line of text

It prints out its text arguments on standard output

tsc@tsc:~$ echo this is a test
this is a test

Tilde Expansion

tsc@tsc:~/playground$ echo ~
/home/tsc

Arithmetic Expansion

形式：

$((expression))

tsc@tsc:~/playground$ echo $((2+2))
4

brace expansion

括號中間不能有空格。

用途：批量建立有順序的文件。
Parameter Expansion
Command Substitution

command的結果也能夠做爲expansion

echo $(ls)

Double Quotes

「$」, 「\」 (backslash), and 「`」 (backquote) 失效，可是parameter expansion, arithmetic expansion, and command substitution 仍有效。

注意區別：

echo $(cal)
echo "$(cal)"

空格、tab、換行符都被用來分割，雙引號則抑制。因此前一個命令不會換行，後一個會。

Single Quotes

抑制全部expansions
escape character

和雙引號一塊兒使用，抑制部分符號

8 – Advanced Keyboard Tricks

這節講鍵盤技巧，提升效率。主要是command line和history的技巧。command line的：移動，刪除。history的上下一個命令。

clear – Clear the screen
history – Display the contents of the history list
Cursor Movement
completion

補全
history

history expansion-使用history中1209行的命令：

tsc@tsc:~$ !1209

History Commands快捷鍵

9 – Permissions

思考：多用戶爲何存在？想一想計算中心。

多用戶存在要解決的問題：

1.一個用戶的錯誤操做不能使計算機crash

2.一個用戶不能影響其餘用戶

幾個相關文件夾

/etc/passwd
User accounts .

/etc/group
groups

/etc/shadow
user's password

Owner Group World

id – Display user identity

$ id
uid=1000(tsc) gid=1000(tsc) groups=1000(tsc),4(adm),24(cdrom),27(sudo),30(dip),46(plugdev),113(lpadmin),128(sambashare)

# root
# id
uid=0(root) gid=0(root) groups=0(root)

chmod – Change File Mode
File Modes中八進制的妙用

$ chmod 600 foo.txt

umask – Set Default Permissions
su – Run A Shell With Substitute User And Group IDs

默認是superuser

$ su -
exit

sudo – Execute A Command As Another User

糾正本身的認識，su,sudo是改變用戶的，不必定是super，默認纔是。
chown – Change File Owner And Group
chgrp – Change Group Ownership
passwd - change user password

10 – Processes

進程

Processes are how Linux organizes the different programs waiting for their turn at the CPU.

ps - report a snapshot of the current processes.
top - display Linux processes
& - Putting A Process In The Background
jobs - list the jobs that have been launched from our terminal.
fg - Returning A Process To The Foreground

爲何要回到前臺來？後臺的程序不受terminal的控制。
Ctrl-z - Stopping (Pausing) A Process

注意和ctrl-c的區別，c是結束。
bg - resume the program’s execution in the background
kill - 「kill」 processes
killall - kill processes by name
halt
poweroff
reboot
shutdown

Part 2 – Configuration And The Environment

11 – The Environment

什麼是environment？

configuration的做用：store program settings

printenv - print all or part of environment

# 全部environment
tsc@tsc:~$ printenv | less

# 某一個environment
tsc@tsc:~$ printenv USER

tsc@tsc:~$ echo $HOME

printenv顯示的一些常見變量：

SHELL
PATH   # 全部的，最新的

set

will show both the shell and environment variables
printenv和set的區別

printenv只顯示environment variables，set顯示shell和environment variables.
追加

PATH=$PATH:$HOME/bin

export

tells the shell to make the contents of PATH available to child processes of this shell.
啓動順序

/etc/profile     # global
~/.bash_profile
~/.bash_login    # 若是上面那個不存在,讀這個
~/.profile       # 若是上面那個不存在,讀這個

.profile文件中有PATH，而且將$HOME/bin添加上了，因此啓動系統後$HOME/bin中的命令是直接能夠用的。$HOME是用戶目錄。

Non-Login Shell Sessions ：

/etc/bash.bashrc # global
~/.bashrc        # user's

Login和Non-Login的區別？

參考，
PATH的做用

命令從中找。
改環境變量時改哪一個文件？

.profile：

add directories

define additional environment variables

.bashrc：

everything else

Text Editors

有兩種：

1.graphical ：gedit

2.text based ：nano, vi, emacs

nano

ctrl-x : exit
ctrl-o : save, write out
# : comments

使.bashrc生效

首先要知道原理，啓動流程，在session一開始纔會read .bashrc，因此須要重啓terminal纔會生效。固然，能夠強行重讀.bashrc，用source命令：

source .bashrc

12 – A Gentle Introduction To vi

why vi?

always available

lightweight and fast
vi 操做

:q
:q!   # 強制退出
i     # insert
esc   # to command mode
:w    # save ,有:的命令叫ex command

# move
h, j, k, l
ctrl-f/b
numberG
gg # 第一個字符
G last line of the file
0 (zero) 行首
^ To the first non-whitespace character on the current line.
$ end of current line
w beginning of next word or punctuation
W ignoring punctuation
b beginning of previous word or punctuation
B ignoring punctuation


# edit
u    # undo last change
i    # insert
# append
a    # append, i是前插
A    # 直接到最後append
# opening a line
o    # 新建一行在下面
O    # 新建一行在上面

# delete
x    # current character
3x
dd   # current line,d有cut的做用
dw   # 
dW   # ignoring punctuation
d$   # current cursor to end
d0   # current cursor to beginning
dG   # current cursor to end of file
d20G # current cursor to 20 line，不是要刪除20行
5dd  # 5行
# Cutting, Copying, And Pasting
p    # 小寫粘貼在後一行（dd），普通在光標前
P    # 大寫粘貼在前一行（dd），普通在光標後

p    # paste after cursor
P    # upper case, paste before cursor
  # copy
yy   # copy current line
5yy  # copy 5 line
yW   # ignoring punctuation
yw   # 
y$   # current cursor to end of line
y0   # current cursor to beginning of line
y^   # current cursor to first non-whitespace char
yG  # current cursor to last line of file
y20G # current cursor to 20th line

# redo,undo
u  # undo
ctrl+r # redo

J    # join line

# search, replace
fa   # 行內搜索a，按;繼續搜索
/    # 文件內搜索，n繼續
:%s/line/Line/g    # 替換
:%s/line/Line/gc   # 有提醒的替換
%表明對全部內容進行操做
:1,2s/line/Line/gc # 對1-2行進行操做

Ctrl-e, Ctrl-y # Scroll down and scroll up

The leading tilde characters (」~」) indicate that no text exists on that line.

modal editor

command mode
Switching case of characters 大小寫轉換

You can change the case of text:

Toggle case 「HellO」 to 「hELLo」 with g~ then a movement.

Uppercase 「HellO」 to 「HELLO」 with gU then a movement.

Lowercase 「HellO」 to 「hello」 with gu then a movement.

Alternatively, you can visually select text then press ~to toggle case, or U to convert to uppercase, or u to convert to lowercase.

常規操做

# 到行尾
$  反：0
A  反：I
Word movement: w, e, b

Linux vi and vim editor: Tutorial and advanced features

13 – Customizing The Prompt

PS1-prompt string one

就是這個：

tsc@tsc:~$

更改它：

tsc@tsc:~$ PS1="<\u@\h \w>\$ "
<tsc@tsc ~>$ 
\u   # username of the current user.
\h   # Hostname of the local machine minus the trailing domain name.
\w   # Name of the current working directory.
\$   # This displays a 「$」 character unless we have superuser privileges. In that case, it displays a 「#」 instead.

改顏色：

經過escape code 實現，escape code 的開頭是八進制的033 ，例子：

\033[0;30m
\033是開頭
0   是optional character attribute，0表明text color是normal，1是bold
30m 是instruction
tsc@tsc:~$ PS1="\[\033[0;31m\]<\u@\h \w>\$\[\033[0m\] " \033[0m # 表明結束脩改

Part 3 – Common Tasks And Essential Tools

要知道package是什麼，如何install, remove, update

14 – Package Management

linux的哪一個發行版纔是最好的？考慮的因素不是界面，而是：

1.packaging system

2.vitality of the distribution’s support community

packaging 的兩大陣營

Debian 「.deb」 camp

Red Hat 「.rpm」 camp

Packaging System     Distributions (Partial Listing)
Debian Style (.deb)  Debian, Ubuntu, Linux Mint, Raspbian
Red Hat Style (.rpm) Fedora, CentOS, Red Hat Enterprise Linux, OpenSUSE

package files
shared libraries

軟件都是有依賴的，被依賴的就放在這裏

dependency
package tool

分爲high, low-level兩種，

high:metadata search, dependency

low-level: installing, removing

Red Hat系列使用同一low-lvel program(rpm), high-level則不一樣，yum使用者：Fedora, Red Hat Enter, CentOS.

Distributions	Low-Level Tools	High-Level Tools
Debian-Style	dpkg	apt-get, aptitude
Fedora, Red Hat, Enterprise Linux, CentOS	rpm	yum

- 查找和安裝package的方法
- What is the difference between apt-get update and upgrade?

15 – Storage Media

什麼是mounting?

attaching the device to the file system tree.

mount – Mount a file system

沒有參數時，顯示已經mounted的文件系統。

如何看？device on mount_point type file_system_type (options).

free - Display amount of free and used memory in the system
partition number 是什麼？
Unable to enumerate USB device under virtualbox

usb3.0插的3.0的口上在vm中讀不出來，改插在2.0的口上就能夠，真是。。。
磁盤、分區及Linux文件系統 [Disk, Partition, Linux File System]

inode的做用：實現文件存儲的關鍵。
tail的妙用

# ubuntu
tail -f /var/log/syslog

實時觀察系統在作什麼。

partition layout 是什麼？
IDE、SATA、SCSI、SAS、FC、SSD硬盤類型介紹
fsck - check and repair a Linux filesystem

16 – Networking

IP Addresses, Host Names, and Domain Names

明白host name與domain name的區別。
ping, ping6 - send ICMP ECHO_REQUEST to network hosts
traceroute - print the route packets trace to network host
ip - show / manipulate routing, devices, policy routing and tunnels

DHCP (Dynamic Host Configuration Protocol)
netstat - Print network connections, routing tables, interface statistics, masquerade connections, and multicast memberships
ftp — Internet file transfer program

ftp是明文傳輸。
Wget - The non-interactive network downloader.
ssh

rlogin and telnet 和ftp有同樣的缺陷。

ssh解決了兩大問題：First, it authenticates that the remote host is who it says it is (thus preventing so-called 「man in the
middle」 attacks), and second, it encrypts all of the communications between the local and
remote hosts.

ssh root@ip -p 端口號

windows下的ssh軟件

PuTTY

17 – Searching For Files

找文件。

locate - find files by name

locate查找的數據庫是如何更新的？由updatedb 程序更新，能夠手動更新。
locate vs find: usage, pros and cons of each other
find - search for files in a directory hierarchy

$ find ~ -type f | wc
# 文件名有jpg，大小大於1M
$ find ~ -type f -name "*.jpg" -size +1M | wc -l

test

$ find ~ \( -type f -not -perm 0600 \) -or \( -type d -not -perm 0700 \)

順序很重要。

action

把查找的結果做爲命令的輸入參數

$ find ~ -type f -name 'foo*' -exec ls -l '{}' ';'
$ find ~ -type f -name 'foo*' -exec ls -l '{}' +

$ find ~ -type f -name 'foo*' -print | xargs ls -l

{} is a symbolic representation of the current pathname

semicolon is a required delimiter indicating the end of the command.

+：the system only has to execute the ls command once

文件名中空格的處理

$ find ~ -type f -name '*.jpg' -print0 | xargs --null ls -l

touch - change file timestamps

文件不存在時會建立。
stat - display file or file system status

which is a kind of souped-up version of ls.

18 – Archiving And Backup

管理大量的文件-壓縮。

compression archiving區別？

本節講3個內容：

compression

archiving

synchronization

gzip, gunzip, zcat - compress or expand files

gzip壓縮，gunzip解壓。

gzip會替換原來的文件。

zcat - equivalent to gunzip with the -c option
zless
bzip2, bunzip2 - a block-sorting file compressor
linux壓縮和打包是兩個要領，分別對應zip和tar，window下面是否是沒有區分？
tar -

tape archive

.tar or .tgz, which indicate a 「plain」 tar archive and a gzipped archive, respectively.

# c create
$ tar cf playground.tar playground
# t list
$ tar tf playground.tar
# x extract
$ tar xf ../playground.tar
# r append

# extract a single file
$ tar xf archive.tar pathname

注意區分mode與option，c是mode，f是option。

mode要先寫。

incremental backups - 增量式備份
linux shell 重定向減號-的做用

studout或者stdin就能夠用減號（-）來替代

不解壓縮直接查看.gz和.bz2命令

使用zcat能夠查看*.gz文件內容，使用bzcat能夠直接查看*.bz2文件.

用ssh從遠程服務器複製文件到本地

$ ssh root@xxx.xxx.xx.xx -p 端口號 'tar cf - filename' | tar xf -
root@xxx.xxx.xx.xx's password:

先tar，傳回來再導出。

zip - 既壓縮也歸檔
zip,unzip的做用是交換indows文件，linux用tar,gzip
rsync - 備份與同步

rsync options source destination
$ rsync -av playground foo

19 – Regular Expressions

後面幾章都是關於文字處理的。先講regular expression.

metacharacters & literal characters

metacharacters是^ $ . [ ] { } - ? * + ( ) | \ ，其他就是literal characters。

The Any Character

就是點號，the dot or period character

anchors

The caret (^) (只匹配行首出來的)and dollar sign ($) （只匹配行尾出現的）

$ grep -h '^zip' dirlist*.txt
$ grep -h 'zip$' dirlist*.txt
$ grep -h '^zip$' dirlist*.txt

點號不是任意長度嗎？加上^$就表明一個？

$ grep -h '.zip' dirlist*.txt
$ grep -i '^..j.r$' /usr/share/dict/words

Bracket Expressions And Character Classes

[] 用來只匹配一個字符。

$ grep -h '[bg]zip' dirlist*.txt
^ : negation,只在第一個位置時有效
$ grep -h '[^bg]zip' dirlist*.txt # 注意出現的位置,表示取反
- : range
$ grep -h '^[A-Za-z0-9]' dirlist*.txt

# 要匹配-（短橫線），就把它放到第一位
$ grep -h '[A-Z]' dirlist*.txt
$ grep -h '[-AZ]' dirlist*.txt

POSIX Character Classes

出現的緣由：posix和ASCII不同。

alternation

[]是單個，它(|)是string

$ echo "AAA" | grep -E 'AAA|BBB|CCC'
$ grep -Eh '^(bz|gz|zip)' dirlist*.txt

Quantifiers

匹配的個數

? - Match An Element Zero Or One Time ,一個字符出現的次數，和word裏面的?不一樣，word裏面表明任意一個字符，這裏的?不能單獨使用，要跟在一個字符後面。

$ echo "(555) 123-4567" | grep -E '^\(?[0-9][0-9][0-9]\)? [0-9][0-9][0-9]-[0-9][0-9][0-9][0-9]$'

* - Match An Element Zero Or More Times

$ echo "This works." | grep -E '[[:upper:]][[:upper:][:lower:] ]*\.'

和?同樣，前面要有字符。

+ - Match An Element One Or More Times

至少出現一次。

$ echo "This that" | grep -E '^([[:alpha:]]+ ?)+$'

{ } - Match An Element A Specific Number Of Times

$ echo "(555) 123-4567" | grep -E '^\(?[0-9]{3}\)? [0-9]{3}-[0-9]{4}$'

find使用reg

$ find . -regex '.*[^-_./0-9a-zA-Z].*'
# 反 0-無窮個字符

locate使用reg

$ locate --regex 'bin/(bz|gz|zip)'

less和vi中也可使用

注意它們的區別，在vi中reg是basic

# less
/^\([0-9]{3}\) [0-9]{3}-[0-9]{4}

# vi
/([0-9]\{3\}) [0-9]\{3\}-[0-9]\{4\}

20 – Text Processing

真是一切皆文件。

cat的一些option

$ cat -A foo.txt # 顯示全部符號，包括控制符
$ cat -ns foo.txt # n顯示行號，s抑制多餘的空行

sort

$ sort > foo.txt
c
b
a

# 多個文件
$ sort file1.txt file2.txt file3.txt > final_sorted_list.txt

du - estimate file space usage

$ du -s /usr/share/* | head # 顯示前10個
$ du -s /usr/share/* | sort -nr | head # 大到小排序後顯示前10個

按指定列排序(k)：

$ ls -l /usr/bin | sort -nr -k 5 | head

多個key排序：

$ sort --key=1,1 --key=2n distros.txt
# 一個field內多個排序：
$ sort -k 3.7nbr -k 3.1nbr -k 3.4nbr distros.txt

指定分割符：

$ sort -t ':' -k 7 /etc/passwd | head

uniq

$ sort foo.txt | uniq

# 取第3列
$ cut -f 3 distros.txt

每一個field的長度可能不一樣，若是要取全部行中時間中的年，就不可能用字符位置去表示。而先把field取出來，就規整了。

$ cut -f 3 distros.txt | cut -c 7-10

expand - 把tab換爲等量空格

$ expand distros.txt | cut -c 23-

改cut的默認分割符，用d option

$ cut -d ':' -f 1 /etc/passwd | head

paste

adds one or more columns of text to a file

$ sort -k 3.7nbr -k 3.1nbr -k 3.4nbr distros.txt > distros-by-date.txt
$ cut -f 1,2 distros-by-date.txt > distros-versions.txt
$ cut -f 3 distros-by-date.txt > distros-dates.txt
$ paste distros-dates.txt distros-versions.txt

join

It joins data from multiple files based on a shared key field.

$ join distros-key-names.txt distros-key-vernums.txt | head

comm

$ comm file1.txt file2.txt
# -n 抑制列
$ comm -12 file1.txt file2.txt

diff

$ diff file1.txt file2.txt
# context format 看兩個各自都有什麼，沒有什麼
$ diff -c file1.txt file2.txt
# unified format 看從舊到新的變化
$ diff -u file1.txt file2.txt

patch

apply changes to text files

用來更新文件，好比代碼。

$ diff -Naur file1.txt file2.txt > patchfile.txt
$ patch < patchfile.txt

轉換

$ echo "lowercase letters" | tr a-z A-Z
LOWERCASE LETTERS
#　多對一的轉換
$ echo "lowercase letters" | tr [:lower:] A
# squeeze
$ echo "aaabbbccc" | tr -s ab
abccc

stream editor

# search-and-replace
$ echo "front" | sed 's/front/back/'
# 誰跟在command後面就是delimiter
$ echo "front" | sed 's_front_back_'
# 指定要處理的行數
$ echo "front" | sed '1s/front/back/'

# p：打印出來
# -n： not to print every line by default
$ sed -n '1,5p' distros.txt
$ sed -n '/SUSE/p' distros.txt
$ sed -n '/SUSE/!p' distros.txt

# 通常出輸出到屏幕，用i原地
$ sed -i 's/lazy/laxy/; s/jumped/jimped/' foo.txt

reg的back references

$ sed 's/\([0-9]\{2\}\)\/\([0-9]\{2\}\)\/\([0-9]\{4\}\)$/\3-\1-\2/' distros.txt

所有替換

$ echo "aaabbbccc" | sed 's/b/B/'
$ echo "aaabbbccc" | sed 's/b/B/g'

sed用script
aspell

拼寫檢查

$ aspell check foo.txt

# -H (HTML) checking-mode option
$ aspell -H check foo.txt

改完了會產生一個bak文件，存在原來的文本。

21 – Formatting Output

講文本輸出格式。

顯示行號

$ nl distros.txt | head
# 和cat -n 同樣

$ sort -k 1,1 -k 2n distros.txt | sed -f distros-nl.sed | nl

fold – Wrap Each Line To A Specified Length

$ echo "The quick brown fox jumped over the lazy dog." | fold -w 12
# 不打斷單詞
$ echo "The quick brown fox jumped over the lazy dog." | fold -w 12 -s

fmt – A Simple Text Formatter

it fills and joins lines in text while preserving blank lines and indentation.

$ fmt -w 50 fmt-info.txt | head
$ fmt -cw 50 fmt-info.txt
# 一行若是不足50個字符，會把第二行的補過來
# 只對'# '開頭的行操做
$ fmt -w 50 -p '# ' fmt-code.txt

pr – Format Text For Printing

used to paginate text.

# l:page length
# w:width
$ pr -l 15 -w 65 distros.txt

首尾會有空行。

printf – Format And Print Data

「print formatted」

使用：printf 「format」 arguments

$ printf "I formatted the string: %s\n" foo
$ printf "%d, %f, %o, %s, %x, %X\n" 380 380 380 380 380 380

Multiple optional components ：

%[flags][width][.precision]conversion_specification

Document Formatting Systems

tex後來取代了troff.

groff

$ zcat /usr/share/man/man1/ls.1.gz | head
$ man ls | head
$ zcat /usr/share/man/man1/ls.1.gz | groff -mandoc -T ascii | head

# 存爲postscript到桌面
$ zcat /usr/share/man/man1/ls.1.gz | groff -mandoc > ~/Desktop/foo.ps

# ps轉pdf
$ ps2pdf ~/Desktop/foo.ps ~/Desktop/ls.pdf

22 – Printing

介紹了ps的由來。

CUPS (Common Unix Printing System) ：provides print drivers and print-job management

Ghostscript： a PostScript interpreter, acts as a RIP.

pr – Convert Text Files For Printing

pr is used to adjust text to fit on a specific page size, with optional page headers and margins.

# 3列
$ ls /usr/bin | pr -3 -w 65 | head

CUPS 有兩種printing，lpr, lp.

lpr – Print Files (Berkeley Style)

$ ls /usr/bin | pr -3 | lpr
lpr: Error - no default destination available.
# 查看printer
$ lpstat -a
lpstat: No destinations added.

lp – Print Files (System V Style)
a2ps

「Anything to PostScript.」

$ ls /usr/bin | pr -3 -t | a2ps -o ~/Desktop/ls.ps -L 66 -R

lpstat – Display Print System Status

# 查詢printer狀態
$ lpstat -a
$ lpstat -a

lpq – Display Printer Queue Status

$ lpq
PDF is ready
no entries

lprm / cancel – Cancel Print Jobs

23 – Compiling Programs

compile

linker，解決共享問題，用共享庫。

$ which gcc
/usr/bin/gcc

用ftp下載source code

$ mkdir src
$ cd src
$ ftp ftp.gnu.org
Name (ftp.gnu.org:me): anonymous
ftp> cd gnu/diction
ftp> ls
ftp> get diction-1.11.tar.gz
ftp> bye

$ tar xzf diction-1.11.tar.gz
# 先查看是否是在一個文件夾裏，以避免解壓散開
$ tar tzvf diction-1.11.tar.gz | less

共享的頭文件在/usr/include ，The header files in this directory were installed when we installed the compiler.

Building The Program

通常須要兩步。

# 因爲script不在默認的目錄，須要./告訴shell它在當前目錄
$ ./configure
config.status: creating Makefile
config.status: creating diction.1
config.status: creating diction.texi
config.status: creating diction.spec
config.status: creating style.1
config.status: creating test/rundiction
config.status: creating config.h

Makefile is a configuration file that instructs the make program exactly how to build the program. Without it, make will refuse to run.

$ make

make只complie須要complie的文件，省時間。
Installing The Program

$ sudo make install

Part 4 – Writing Shell Scripts

24 – Writing Your First Script

script開頭的#!

shebang，指明interpreter，每一個script開頭都應該加這行。

讓script能夠執行

$ chmod 755 hello_world

owner的權限是x就能夠了。755全部人能夠執行，700只有owner能夠執行。必須是可讀的。

執行

$ ./hello_world

必需要加上./ ,由於文件位置的問題。自動識別的命令的目錄都寫在了PATH中，能夠查看它：

$ echo $PATH

把某個路徑添加到PATH

在.bashrc （它在~中）中添加：

# 把~/bin添加在PATH的前面
export PATH=~/bin:"$PATH"

# 在當前terminal中生效
$ . .bashrc
# 或者
$ source .bashrc

script放哪？

~/bin ： 本身用
 /usr/local/bin ： 同一個系統多人用
 /usr/local/sbin： 系統管理員用
 /usr/local：   locally supplied software，

line-continuations

就是\

vi中對script的配置

:syntax on
:set hlsearch
:set tabstop=4
:set autoindent

永久：
寫在~/.vimrc，沒有就建立

25 – Starting A Project

shell是動態語言

uppercase letters to designate constants and lower case
letters for true variables.

$ vim ~/bin/sys_info_page
$ chmod 755 ~/bin/sys_info_page
$ sys_info_page
$ sys_info_page > sys_info_page.html
$ firefox sys_info_page.html

Variables And Constants

uppercase letters to designate constants and lower case letters for true variables.

Note that in an assignment, there must be no spaces between the variable name, the equals sign, and the value.

a=z                  # Assign the string "z" to variable a.
b="a string"         # Embedded spaces must be within quotes.
c="a string and $b"  # Other expansions such as variables can be 
                     # expanded into the assignment.
d="$(ls -l foo.txt)" # Results of a command.
e=$((5 * 7))         # Arithmetic expansion.
f="\t\ta string\n"   # Escape sequences such as tabs and newlines.

大括號的做用

$ filename="myfile"
$ touch "$filename"
$ mv "$filename" "$filename1"
mv: cannot move 'myfile' to '': No such file or directory

$ mv "$filename" "${filename}1"

here document

command << token
text
token

# shell example
cat << _EOF_
<HTML>
    <HEAD>
        <TITLE>$TITLE</TITLE>
    </HEAD>
</HTML>
_EOF_

# terminal example
$ cat << _EOF_
> $foo
> "$foo"
> '$foo'
> \$foo
> _EOF_

26 – Top-Down Design

講程序設計的思想。

shell functions

兩種形式：

function name {
commands
return
}

and the simpler (and generally preferred) form:

name () {
commands
return
}

in order for function calls to be recognized as shell functions and not interpreted as the names of external programs, shell function definitions must appear in the script before they are called.

local variables

要加local

funct_1 () {
    local foo # variable foo local to funct_1
    foo=1
    echo "funct_1: foo = $foo"
}

27 – Flow Control: Branching With if

shell的if語句挺複雜的。

if commands; then
    commands
[elif commands; then
    commands...]
[else
    commands]
fi

x=5
if [ "$x" -eq 5 ]; then
    echo "x equals 5."
else
    echo "x does not equal 5."
fi

exit status

程序結果後返回給系統的。查詢：

$ ls -d /usr/bin
$ echo $?

true - do nothing, successfully

false - do nothing, unsuccessfully

test

語法：

test expression
and the more popular:
[ expression ]

expression是true時返回0，不然返回1

test和[ 本質上是同樣的。

使用：

#!/bin/bash 
# test-file: Evaluate the status of a file

FILE=~/.bashrc

if [ -e "$FILE" ]; then
    if [ -f "$FILE" ]; then
        echo "$FILE is a regular file."
    fi
    if [ -d "$FILE" ]; then
        echo "$FILE is a directory."
    fi
    if [ -r "$FILE" ]; then
        echo "$FILE is readable."
    fi
    if [ -w "$FILE" ]; then
        echo "$FILE is writable."
    fi
    if [ -x "$FILE" ]; then
        echo "$FILE is executable/searchable."
    fi
else
    echo "$FILE does not exist"
    exit 1
fi

exit  # ?

When a script 「runs off the end」 (reaches end of file), it terminates with an exit status of the last command executed.

String Expressions

#!/bin/bash
# test-string: evaluate the value of a string
ANSWER=maybe
if [ -z "$ANSWER" ]; then
    echo "There is no answer." >&2
    exit 1
fi
if [ "$ANSWER" = "yes" ]; then
    echo "The answer is YES."
elif [ "$ANSWER" = "no" ]; then
    echo "The answer is NO."
elif [ "$ANSWER" = "maybe" ]; then
    echo "The answer is MAYBE."
else
    echo "The answer is UNKNOWN."
fi

Integer Expressions
test的選項真多！！！
test用regex

[[]]

# 檢驗是否是數
if [[ "$INT" =~ ^-?[0-9]+$ ]]; then

(( )) - Designed For Integers

$ if ((1)); then echo "It is true."; fi
It is true.

if ((INT == 0)); 
if ((INT < 0)); 
if (( ((INT % 2)) == 0));

Combining Expressions

Operation   test   [[ ]] and (( ))
AND         -a           &&
OR          -o           ||
NOT         !            !

例子：

# [[]]
if [[ "$INT" -ge "$MIN_VAL" && "$INT" -le "$MAX_VAL" ]];
# test
if [ "$INT" -ge "$MIN_VAL" -a "$INT" -le "$MAX_VAL" ];

Control Operators: Another Way To Branch

The && (AND) and || (OR)

$ mkdir temp && cd temp
$ [[ -d temp ]] || mkdir temp

[]和[[]]的區別

[[]]和[]同樣，不過有兩個新特性，：

string1 =~ regex

== operator

[[]]和表達式之間要有空格

[[ "$count" -gt 5 ]]
[[ "$count" -gt 5]] # 會報錯，執行起來會很可怕

28 – Reading Keyboard Input

read – Read Values From Standard Input

read a single line of standard input.

read [-options] [variable...]

# -n option,suppresses the trailing newline on output
echo -n "Please enter an integer -> "
read int

read var1 var2 var3 var4 var5 # 多個

# -p prompt
read -p "Enter one or more values > "
# -t seconds
# -s Silent mode. 
if read -t 10 -sp "Enter secret passphrase > " secret_pass;

Internal Field Separator

file_info=$(grep "^$user_name:" $FILE)

# 寫在一行，IFS只改變跟在後面的 command
IFS=":" read user pw uid gid name home shell <<< "$file_info"

here string

The <<< operator indicates a here string.

You Can’t Pipe read

就像函數與子函數，凡是這一類都不能pip。

Validating Input
Menus

就是提示和if。

一個程序中出現多個exit是個很差的設計

那應該如何作？

29 – Flow Control: Looping With while / until

有3各loop

while

語法：
while commands; do commands; done

#!/bin/bash
# while-count: display a series of numbers
count=1
while [[ "$count" -le 5 ]]; do
    echo "$count"
    count=$((count + 1))
done
echo "Finished."

Breaking Out Of A Loop

break

continue

until

和while相反

count=1
until [[ "$count" -gt 5 ]]; do
    echo "$count count=$((count + 1)) done echo "Finished.

read file with loop

#!/bin/bash 
# while-read

while read distro version release; do
    printf "distro: %s\tversion: %s\treleased: %s\n" \
        "$distro" \
        "$version" \
        "$release"
done < distros.txt

30 – Troubleshooting

講錯誤的處理。

引號不對稱
特定符號缺失
意外的expansion

是否報錯取決於expansion的值。
雙引號的做用

# number爲空時會報錯
number=
if [ $number = 1 ]; then
# 這樣就不會了 「」=1
[ "$number" = 1 ]

Multi-word string也應該用雙引號括起來。

因此，老是用雙引號把變量括起來。

logical error
給出錯誤信息是比較好的作法
unix文件名稱什麼字符均可以用，有兩個例外

/ , null character(a zero byte)

the POSIX Portable Filename Character Set

uppercase letters A-Z, lowercase letters a-z, numerals 0-9, period (.), hyphen (-), and underscore(_).

檢查input
要會寫測試

利用echo輸出

在更改的地方加註釋，方便恢復。

test case

case要反映edge and corner

debug技術

commenting out :部分註釋

tracing :在一些關鍵點增長輸出，查看是否執行到這裏了

能夠在script的第一行加入x option:

#!/bin/bash -x

也能夠用set設置部分tracing

set -x # turn on
...
set +x # turn off

輸出中間過程當中變量的值
推薦《The Art of Unix Programming》

31 – Flow Control: Branching With `case`

case的用法

1 #!/bin/bash
  2 
  3 # case-menu
  4 
  5 clear
  6 echo " 7 please select: 8 9 1. display system information 10 2. display disk space 11 3. display home space utilization 12 0. quit 13 "
 14 read -p "enter selection [0-3] > "
 15 
 16 case "$REPLY" in
 17     0)  echo "program terminated"
 18         exit
 19         ;;
 20     1)  echo "hostname: $HOSTNAME"
 21         uptime
 22         ;;
 23     2)  df -h
 24         ;;
 25     3) if [[ "$(id -u)" -eq 0 ]]; then
 26             echo "home space utilization (all users)"
 27             du -sh /home/*
 28         else
 29             echo "home space utilization ($USER)"
 30             du -sh "$HOME"
 31         fi
 32         ;;
 33     *)  echo "invalid entry" >&2
 34         exit 1
 35         ;;
 36 esac

32 – Positional Parameters

講處理command的option

positional parameter

$0是當前命令的路徑。

注意它的含義：

Note that $0 always contains the full pathname of the first item on the command line (i.e., the name of the program) and does not contain the name of the shell function as we might expect.

是整個路徑，不是命令的名字。

大於9時用大括號括起來。

參數個數：$#
shift

讓$1下次變成$2，全部參數都移動，$#減1

basename

去除文件名前面的字符，只要文件名。

positional parameter也能夠用在function中
Difference between echo -e 「」 and echo $「」

e的意思就是擴展，支持轉義，e和$支持的符號部分不相同。
$*和$@的區別

不加雙引號時是同樣的，遇到空格就拆分，加雙引號時不同，$*會將全部參數放到一個字符串中，$@會將每一個輸入的參數分別看成參數，也就是不區分輸入時的空格。

$@常常用。

33 – Flow Control: Looping With for

講for

# 兩種風格
for variable [in words]; do
    commands
done

# c語言的
for (( expression1; expression2; expression3 )); do
    commands
done

例子：

[me@linuxbox ~]$ for i in A B C D; do echo $i; done

Create list:

itscs-MacBook-Pro:~ itsc$ for i in {A..D}; do echo $i; done
# 文件列表 pathname expansion
itscs-MacBook-Pro:learnCommandLine itsc$ for i in dis*.txt; do echo "$i"; done

34 – Strings And Numbers

通常狀況下將變量括起來，防止被拆分
parameter expansion是什麼？

變量是其中的一種

大括號的做用

$a
${a}
a="foo"
echo "${a}_file" # 和其它字符相連時防止造成不存在的變量

處理空的和不存在的字符

parameter爲空或者不存在就用word，存在就用它本身。
${parameter:-word}

和上面基本同樣，區別：要賦值。
${parameter:=word}

unset和empty就發發送word到error
${parameter:?word}

${parameter:+word}

String operation

${#parameter} # 換成長度

$ foo="This string is long."
$ echo "'$foo' is ${#foo} characters long. 'This string is long.' is 20 characters long ${parameter:offset} ${parameter:offset:length} [me@linuxbox ~]$ foo="This string is long." [me@linuxbox ~]$ echo ${foo:5} string is long. [me@linuxbox ~]$ echo ${foo:5:6} string 刪除一部分 ${parameter#pattern} ${parameter##pattern} [me@linuxbox ~]$ foo=file.txt.zip [me@linuxbox ~]$ echo ${foo#*.} txt.zip [me@linuxbox ~]$ echo ${foo##*.} zip 反着刪除 ${parameter%pattern} ${parameter%%pattern} [me@linuxbox ~]$ foo=file.txt.zip [me@linuxbox ~]$ echo ${foo%.*} file.txt [me@linuxbox ~]$ echo ${foo%%.*} file 替換 ${parameter/pattern/string} ${parameter//pattern/string} ${parameter/#pattern/string} ${parameter/%pattern/string}

能夠用expansion來提升script的效率
大小寫轉換

能夠用來作什麼？好比數據庫的查找，匹配的時候把輸入和數據庫中的都統一大小寫。

declare
shell不能進行浮點運算

itscs-MacBook-Pro:learnCommandLine itsc$ echo $((3.3+4.2))
-bash: 3.3+4.2: syntax error: invalid arithmetic operator (error token is ".3+4.2")

解決：perl, awk.書裏用最簡單的bc

爲何連浮點運算都不支持？真是麻煩。

35 – Arrays

scalar variables

Single value

shell的array只能一維

[me@linuxbox ~]$ a[1]=foo
[me@linuxbox ~]$ echo ${a[1]}
foo
[me@linuxbox ~]$ declare -a a

# assign
name[subscript]=value
name=(value1 value2 ...)
[me@linuxbox ~]$ days=(Sun Mon Tue Wed Thu Fri Sat)
[me@linuxbox ~]$ days=([0]=Sun [1]=Mon [2]=Tue [3]=Wed [4]=Thu [5]=Fri [6]=Sat)

遍歷array

[me@linuxbox ~]$ animals=("a dog" "a cat" "a fish")
[me@linuxbox ~]$ for i in ${animals[*]}; do echo $i; done
[me@linuxbox ~]$ for i in ${animals[@]}; do echo $i; done
[me@linuxbox ~]$ for i in "${animals[*]}"; do echo $i; done
[me@linuxbox ~]$ for i in "${animals[@]}"; do echo $i; done

bash的array不必定是要連續的

因此須要有方法知道哪些位置上有值

[me@linuxbox ~]$ foo=([2]=a [4]=b [6]=c)
[me@linuxbox ~]$ for i in "${foo[@]}"; do echo $i; done
a
b
c
[me@linuxbox ~]$ for i in "${!foo[@]}"; do echo $i; done
2
4
6

array的追加

$ foo=(a b c)
$ foo[100]=e
$ echo ${foo[@]}
a b c e
$ foo+=(k l)
$ echo ${foo[@]}
a b c e k l
$ for i in "${foo[@]}"; do echo $i; done
a
b
c
e
k
l
$ for i in "${!foo[@]}"; do echo $i; done
0
1
2
100
101
102

subscript不是連續的

associative arrays

下標能夠是字符

36 – Exotica

講一些不經常使用的，在特定場景下使用的

Group Commands And Subshells

Group command:
{ command1; command2; [command3; ...] }
Subshell:
(command1; command2; [command3;...])

它們是作什麼的？

manage redirection

{ ls -l; echo "Listing of foo.txt"; cat foo.txt; } > output.txt
(ls -l; echo "Listing of foo.txt"; cat foo.txt) > output.txt

結合pip

{ ls -l; echo "Listing of foo.txt"; cat foo.txt; } | lpr

注意大括號的使用

due to the way bash implements
group commands, the braces must be separated from the commands by a space and the
last command must be terminated with either a semicolon or a newline prior to the closing brace.

group和subshell的區別

subshell和它的名字同樣，返回時enviroment會丟失，因此通常狀況下用group.

echo "foo" | read
echo $REPLY # 這就是subshell的例子，reply是空的

commands in pipelines are always executed in subshells

process substitution

用來解決subshell的問題

read < <(echo "foo")
echo $REPLY

asynchronous execution
什麼是異步？

什麼是阻塞，非阻塞，同步，異步？

舉個打電話的例子：

阻塞/同步：打一個電話一直到有人接爲止
非阻塞：打一個電話沒人接，每隔10分鐘再打一次，知道有人接爲止
異步：打一個電話沒人接，轉到語音郵箱留言（註冊），而後等待對方回電（call back)

看起來異步是最高效，充分利用資源，能夠想像整個系統能支持大規模併發。但問題是調試很麻煩，不知道何時call back。

Named pipes

create a connection between two processes and can be
used just like other types of files.

named pipe是一個文件起着pipe的做用

[me@linuxbox ~]$ mkfifo pipe1
[me@linuxbox ~]$ ls -l pipe1
prw-r--r-- 1 me me 0 2009-07-17 06:41 pipe1

# terminal 1
[me@linuxbox ~]$ ls -l > pipe1

# terminal2
[me@linuxbox ~]$ cat < pipe1

問題

[[:upper:]]哪一節的？
每一個文件夾下的.和..的做用？爲何必需要有它們？

它們表明當前和前一級目錄，這個你們都知道，能夠爲何要有它們？設計原理是什麼？

參考：SO，只說了有，沒有說什麼爲何。
shell中各類括號[]、[[]]的做用與區別？

參考。

須要瞭解的知識

linux文件系統

磁盤、分區及Linux文件系統 [Disk, Partition, Linux File System]

硬盤接口

IDE、SATA、SCSI、SAS、FC、SSD硬盤類型介紹

Understand synopsis in manpage

能夠直接在命令man man中看。
vi color

ENABLING VI SYNTAX COLORS

cd ~
vi .vimrc
syntax on
colorscheme desert

vi indent

Redefine tab as 4 spaces

添加到vimrc中即永久：

set tabstop=4

標準錯誤輸出>&2輸出到哪了？

屏幕。
vim color

/usr/share/vim/vim74/colors/

我以爲還不錯的：

koehler

industry

murphy

ron

ls如何按文件類型排序？

How to make ls sort by file extension and then name?

ls -lX

平鋪式窗口管理器-Awesome
vi comment

Reveal the line numbers:

:set number

then

:5,$s/^/#/g comment

or this:

:5,$s/^/#/ uncomment

vim如何複製到clipboard?

Vim: copy selection to OS X clipboard still fail to work

To check if your Vim has clipboard support: vim --version | grep -i clipboard. You should see +clipboard. If you see -clipboard your Vim was compiled without clipboard support…

itscs-MacBook-Pro:learnCommandLine itsc$ vim --version | grep -i clipboard
-clipboard       +job             +path_extra      +user_commands
+eval            -mouse_dec       +statusline      -xterm_clipboard

個人是減號，說明不支持。

Getting Vim with +clipboard support

要先裝Homebrew，裝完後的vim在 /usr/local/bin/vim,原來的在/usr/bin，也存在，只不過PATH中，local的在前面，命令會先用它。

單引號、雙引號區別

echo $1
echo "$1"
echo '$1'

$ ./longest words
words
words
$1

雙引號與無引號在這裏是同樣的，雙引號的做用體如今字符串爲空的時候，爲空的時候是空字符串，能夠判斷，而無引號就麻煩了，好比等號，一邊爲空會報錯，說等號是個二元操做符兩邊都要有值。單引號的做用就是禁止轉義，所有爲字符串。

linux中shell變量#,@,0,1,$2的含義解釋

$#
添加到Shell的參數個數

$! shell parameter, which will always contain the process ID of the
last job put into the background.

網站

The Bash Hackers Wiki

相關標籤/搜索

每日一句

每一个你不满意的现在，都有一个你没有努力的曾经。