經過《 The Linux Command Line》入門linux命令行

時間  2019-11-20
標籤 經過 linux command line 入門 命令行 欄目 Linux 简体版
原文   原文鏈接

原文發於個人獨立博客：經過《The Linux Command Line》入門linux命令行php

此書網站：The Linux Command Line ，它是免費的。css

它有中文版，並且還有一個好聽的名字：快樂的 Linux 命令行html

學習有兩種方法，一種是系統的學，一種是根據實際需求來學。兩種各有優缺點，前一種，知識不會有缺漏，對往後的融會貫通和舉一反三有好處，可是這樣慢，並且有可能不如針對性學習的精神集中。後一種，只找需求的學，可以任務完成就行，這樣快，它的缺點就是前一種的優勢。顯然通看整本書屬於第一種。node

做者寫的很好，娓娓道來，講的很清楚，就像一本故事書同樣。我很喜歡這樣的書。做者比較幽默，一些用詞看了會讓人發笑。linux

這本書的內容很淺，真的只是入門。nginx

正如做者所說，這是一場linux的journey(旅行)，就把看書當旅行吧。git

如下是隨手記的，只適合本身看。github

命令

  • date
  • cal-calendar
  • df-current amount of free space on your disk drives
  • free-display the amount of free memory
  • exit-closing the terminal emulator window

2 - Navigation

  • pwd - Print name of current working directory
  • cd - Change directory
cd             home directory
cd -           previous working directory
cd ~user_name  home directory of user
.              點號表明當前文件夾
  • ls - List directory contents
ls -a   all files
ls -l   long format
ls -lt  sort the result by the file's modification time
ls -lt --reverse

  • file – Determine file typesql

  • less – View file contentsshell

    和cat的區別？參考

4 - Manipulating Files And Directories

  • cp

    若是文件已經存在，直接覆蓋，沒有提示，要提示用-i

  • mv

    move (rename) files

  • mkdir

mkdir dir1 dir2
  • rm – Remove Files And Directories
rm -r playground    # 遞歸刪除
  • ln – Create Links
ln myfile hard       # hard link
ln -s myfile soft    # symbolic link

軟連接」和「硬連接」的區別：硬連接指向磁盤區塊，軟連接指向絕對路徑。

hard links的缺點：

1.cannot span physical devices.

2.cannot reference directories, only files.

5 - Working With Commands

  • type – Indicate how a command name is interpreted
tsc@tsc:~$ type ls
ls is aliased to `ls --color=auto'  # alias

tsc@tsc:~$ type cd
cd is a shell builtin               # builtin

tsc@tsc:~$ type cp
cp is /bin/cp                       # executable program
  • which – Display An Executable’s Location
tsc@tsc:~$ which cp
/bin/cp

只用於exe

  • help – Get help for shell builtins

    注意是shell builtin

tsc@tsc:/bin$ help cd
cd: cd [-L|[-P [-e]] [-@]] [dir]
    Change the shell working directory.

理解註釋：[]是可選，|是互斥。

  • --help – Display Usage Information
tsc@tsc:~$ mkdir --help
  • man – Display a command’s manual page

沒有example

tsc@tsc:~$ man ls
  • apropos – Display a list of appropriate commands
  • info – Display a command’s info entry
  • whatis – Display a very brief description of a command
  • alias – Create an alias for a command

起別名前先檢查是否存在，用type：

tsc@tsc:~$ type test
test is a shell builtin

使用：

alias name='string' # name後不能有空格
tsc@tsc:~$ alias foo='cd /usr; ls; cd -'
tsc@tsc:~$ foo

終端關閉後做用就消失了。

去除：

tsc@tsc:~$ unalias foo
  • trick:多條命令於一行
command1; command2; command3...
tsc@tsc:~$ cd /usr; ls

6 – Redirection

  • cat - Concatenate files
cat file1 # 輸出文件到屏幕，沒有分頁
cat > file.txt # cat沒有參數時接受stdin
this is test.
               # ctrl+d 結束
# < 接受文件輸入
tsc@tsc:~$ cat < lazy_dog.txt
  • sort - Sort lines of text
  • uniq - Report or omit repeated lines
# -d 顯示重複的
tsc@tsc:~$ ls /bin /usr/bin | sort | uniq -d | less
  • grep - Print lines matching a pattern

「global regular expression print」,find功能。

tsc@tsc:~$ ls /bin /usr/bin | sort | uniq | grep zip
  • wc - Print newline, word, and byte counts for each file

聯合使用看條目個數：

tsc@tsc:~$ ls /bin /usr/bin | sort | uniq | wc -l
  • head - Output the first part of a file
  • tail - Output the last part of a file
  • tee - Read from standard input and write to standard output and files

這個名字頗有意思，tee有三通管的意思，配合pipe使用，它的做用是從stdin讀入，複製到stdout和文件。

# 一方面輸出到ls.txt，一方面傳給grep
tsc@tsc:~$ ls /usr/bin/ | tee ls.txt | grep zip

  • I/O redirection 的做用

    I/O redirection allows us to change where output goes and where input comes from.

  • >重定向

tsc@tsc:~$ ls -l /usr/bin/ > ls-output.txt
tsc@tsc:~$ > ls-output2.txt  # 建立一個空文件

要注意重複>時，原來的文件會被清空。

  • >> 追加
tsc@tsc:~$ ls -l /usr/bin >> ls-output2.txt

文件不存在會新建。

  • 輸出error
tsc@tsc:~$ ls -l /bin/usr 2> ls-error.txt

個人問題：在不知道有沒有error的狀況下，>和2>要如何同時執行？

  • 都輸出到一個文件

有兩種方法：

# 1.老版本
# 順序很重要
tsc@tsc:~$ ls -l /bin/usr > ls-output.txt 2>&1

# 2.新，用&>
tsc@tsc:~$ ls -l /bin/usr &> ls-output.txt
tsc@tsc:~$ ls -l /bin/usr &>> ls-output.txt # 追加
  • Pipelines
tsc@tsc:~$ ls -l /usr/bin | less

千萬不要把>用成|，否則有些後果很嚴重。

  • filters
tsc@tsc:~$ ls /bin /usr/bin | sort | less

7 – Seeing The World As The Shell Sees It

這節講expansion， 就是*、~之類的，本質就是變量。

  • echo – Display a line of text

It prints out its text arguments on standard output

tsc@tsc:~$ echo this is a test
this is a test
  • Tilde Expansion
tsc@tsc:~/playground$ echo ~
/home/tsc
  • Arithmetic Expansion

形式：

$((expression))
tsc@tsc:~/playground$ echo $((2+2))
4

  • brace expansion

    括號中間不能有空格。

    用途：批量建立有順序的文件。

  • Parameter Expansion

  • Command Substitution

    command的結果也能夠做爲expansion

echo $(ls)

  • Double Quotes

    「$」, 「\」 (backslash), and 「`」 (backquote) 失效，可是parameter expansion, arithmetic expansion, and command substitution 仍有效。

注意區別：

echo $(cal)
echo "$(cal)"

空格、tab、換行符都被用來分割，雙引號則抑制。因此前一個命令不會換行，後一個會。

  • Single Quotes

    抑制全部expansions

  • escape character

和雙引號一塊兒使用，抑制部分符號

8 – Advanced Keyboard Tricks

這節講鍵盤技巧，提升效率。主要是command line和history的技巧。command line的：移動，刪除。history的上下一個命令。

  • clear – Clear the screen

  • history – Display the contents of the history list

  • Cursor Movement

  • completion

    補全

  • history

    history expansion-使用history中1209行的命令：

tsc@tsc:~$ !1209
  • History Commands快捷鍵

9 – Permissions

思考：多用戶爲何存在？想一想計算中心。

多用戶存在要解決的問題：

1.一個用戶的錯誤操做不能使計算機crash

2.一個用戶不能影響其餘用戶

  • 幾個相關文件夾
/etc/passwd
User accounts .

/etc/group
groups

/etc/shadow
user's password
  • rwx

Owner Group World

  • id – Display user identity
$ id
uid=1000(tsc) gid=1000(tsc) groups=1000(tsc),4(adm),24(cdrom),27(sudo),30(dip),46(plugdev),113(lpadmin),128(sambashare)

# root
# id
uid=0(root) gid=0(root) groups=0(root)
  • chmod – Change File Mode
  • File Modes中八進制的妙用
$ chmod 600 foo.txt

  • umask – Set Default Permissions

  • su – Run A Shell With Substitute User And Group IDs

    默認是superuser

$ su -
exit

  • sudo – Execute A Command As Another User

    糾正本身的認識，su,sudo是改變用戶的，不必定是super，默認纔是。

  • chown – Change File Owner And Group

  • chgrp – Change Group Ownership

  • passwd - change user password

10 – Processes

進程

Processes are how Linux organizes the different programs waiting for their turn at the CPU.

  • ps - report a snapshot of the current processes.

  • top - display Linux processes

  • & - Putting A Process In The Background

  • jobs - list the jobs that have been launched from our terminal.

  • fg - Returning A Process To The Foreground

    爲何要回到前臺來？後臺的程序不受terminal的控制。

  • Ctrl-z - Stopping (Pausing) A Process

    注意和ctrl-c的區別，c是結束。

  • bg - resume the program’s execution in the background

  • kill - 「kill」 processes

  • killall - kill processes by name

  • halt

  • poweroff

  • reboot

  • shutdown

Part 2 – Configuration And The Environment

11 – The Environment

什麼是environment？

configuration的做用：store program settings

  • printenv - print all or part of environment
# 全部environment
tsc@tsc:~$ printenv | less

# 某一個environment
tsc@tsc:~$ printenv USER

tsc@tsc:~$ echo $HOME

printenv顯示的一些常見變量：

SHELL
PATH   # 全部的，最新的

  • set

    will show both the shell and environment variables

  • printenv和set的區別

    printenv只顯示environment variables，set顯示shell和environment variables.

  • 追加

PATH=$PATH:$HOME/bin

  • export

    tells the shell to make the contents of PATH available to child processes of this shell.

  • 啓動順序

Login Shell Sessions ：

/etc/profile     # global
~/.bash_profile
~/.bash_login    # 若是上面那個不存在,讀這個
~/.profile       # 若是上面那個不存在,讀這個

.profile文件中有PATH，而且將$HOME/bin添加上了，因此啓動系統後$HOME/bin中的命令是直接能夠用的。$HOME是用戶目錄。

Non-Login Shell Sessions ：

/etc/bash.bashrc # global
~/.bashrc        # user's

  • Login和Non-Login的區別？

    參考，

  • PATH的做用

    命令從中找。

  • 改環境變量時改哪一個文件？

.profile：

add directories

define additional environment variables

.bashrc：

everything else

  • Text Editors

有兩種：

1.graphical ：gedit

2.text based ：nano, vi, emacs

  • nano
ctrl-x : exit
ctrl-o : save, write out
# : comments
  • 使.bashrc生效

首先要知道原理，啓動流程，在session一開始纔會read .bashrc，因此須要重啓terminal纔會生效。固然，能夠強行重讀.bashrc，用source命令：

source .bashrc

12 – A Gentle Introduction To vi

  • why vi?

    always available

    lightweight and fast

  • vi 操做

:q
:q!   # 強制退出
i     # insert
esc   # to command mode
:w    # save ,有:的命令叫ex command

# move
h, j, k, l
ctrl-f/b
numberG
gg # 第一個字符
G last line of the file
0 (zero) 行首
^ To the first non-whitespace character on the current line.
$ end of current line
w beginning of next word or punctuation
W ignoring punctuation
b beginning of previous word or punctuation
B ignoring punctuation


# edit
u    # undo last change
i    # insert
# append
a    # append, i是前插
A    # 直接到最後append
# opening a line
o    # 新建一行在下面
O    # 新建一行在上面

# delete
x    # current character
3x
dd   # current line,d有cut的做用
dw   # 
dW   # ignoring punctuation
d$   # current cursor to end
d0   # current cursor to beginning
dG   # current cursor to end of file
d20G # current cursor to 20 line，不是要刪除20行
5dd  # 5行
# Cutting, Copying, And Pasting
p    # 小寫粘貼在後一行（dd），普通在光標前
P    # 大寫粘貼在前一行（dd），普通在光標後

p    # paste after cursor
P    # upper case, paste before cursor
  # copy
yy   # copy current line
5yy  # copy 5 line
yW   # ignoring punctuation
yw   # 
y$   # current cursor to end of line
y0   # current cursor to beginning of line
y^   # current cursor to first non-whitespace char
yG  # current cursor to last line of file
y20G # current cursor to 20th line

# redo,undo
u  # undo
ctrl+r # redo

J    # join line

# search, replace
fa   # 行內搜索a，按;繼續搜索
/    # 文件內搜索，n繼續
:%s/line/Line/g    # 替換
:%s/line/Line/gc   # 有提醒的替換
%表明對全部內容進行操做
:1,2s/line/Line/gc # 對1-2行進行操做

Ctrl-e, Ctrl-y # Scroll down and scroll up

The leading tilde characters (」~」) indicate that no text exists on that line.

You can change the case of text:

Toggle case 「HellO」 to 「hELLo」 with g~ then a movement.

Uppercase 「HellO」 to 「HELLO」 with gU then a movement.

Lowercase 「HellO」 to 「hello」 with gu then a movement.

Alternatively, you can visually select text then press ~to toggle case, or U to convert to uppercase, or u to convert to lowercase.

  • 常規操做
# 到行尾
$  反：0
A  反：I
Word movement: w, e, b

Linux vi and vim editor: Tutorial and advanced features

13 – Customizing The Prompt

  • PS1-prompt string one

就是這個：

tsc@tsc:~$

更改它：

tsc@tsc:~$ PS1="<\u@\h \w>\$ "
<tsc@tsc ~>$ 
\u   # username of the current user.
\h   # Hostname of the local machine minus the trailing domain name.
\w   # Name of the current working directory.
\$   # This displays a 「$」 character unless we have superuser privileges. In that case, it displays a 「#」 instead.

改顏色：

經過escape code 實現，escape code 的開頭是八進制的033 ，例子：

\033[0;30m
\033是開頭
0   是optional character attribute，0表明text color是normal，1是bold
30m 是instruction
tsc@tsc:~$ PS1="\[\033[0;31m\]<\u@\h \w>\$\[\033[0m\] " \033[0m # 表明結束脩改

Part 3 – Common Tasks And Essential Tools

要知道package是什麼，如何install, remove, update

14 – Package Management

linux的哪一個發行版纔是最好的？考慮的因素不是界面，而是：

1.packaging system

2.vitality of the distribution’s support community

  • packaging 的兩大陣營

Debian 「.deb」 camp

Red Hat 「.rpm」 camp

Packaging System     Distributions (Partial Listing)
Debian Style (.deb)  Debian, Ubuntu, Linux Mint, Raspbian
Red Hat Style (.rpm) Fedora, CentOS, Red Hat Enterprise Linux, OpenSUSE

  • package files

  • shared libraries

    軟件都是有依賴的，被依賴的就放在這裏

    dependency

  • package tool

    分爲high, low-level兩種，

    high:metadata search, dependency

    low-level: installing, removing

    Red Hat系列使用同一low-lvel program(rpm), high-level則不一樣，yum使用者：Fedora, Red Hat Enter, CentOS.

Distributions Low-Level Tools High-Level Tools
Debian-Style dpkg apt-get, aptitude
Fedora, Red Hat, Enterprise Linux, CentOS rpm yum

- 查找和安裝package的方法
- What is the difference between apt-get update and upgrade?

15 – Storage Media

  • 什麼是mounting?

attaching the device to the file system tree.

  • mount – Mount a file system

沒有參數時，顯示已經mounted的文件系統。

如何看？device on mount_point type file_system_type (options). 

# ubuntu
tail -f /var/log/syslog

實時觀察系統在作什麼。

16 – Networking

  • IP Addresses, Host Names, and Domain Names

    明白host name與domain name的區別。

  • ping, ping6 - send ICMP ECHO_REQUEST to network hosts

  • traceroute - print the route packets trace to network host

  • ip - show / manipulate routing, devices, policy routing and tunnels

    DHCP (Dynamic Host Configuration Protocol)

  • netstat - Print network connections, routing tables, interface statistics, masquerade connections, and multicast memberships

  • ftp — Internet file transfer program

    ftp是明文傳輸。

  • Wget - The non-interactive network downloader.

  • ssh

    rlogin and telnet 和ftp有同樣的缺陷。

    ssh解決了兩大問題：First, it authenticates that the remote host is who it says it is (thus preventing so-called 「man in the
    middle」 attacks), and second, it encrypts all of the communications between the local and
    remote hosts. 

ssh root@ip -p 端口號

  • windows下的ssh軟件

    PuTTY

17 – Searching For Files

找文件。

$ find ~ -type f | wc
# 文件名有jpg，大小大於1M
$ find ~ -type f -name "*.jpg" -size +1M | wc -l
  • test
$ find ~ \( -type f -not -perm 0600 \) -or \( -type d -not -perm 0700 \)

順序很重要。

  • action

把查找的結果做爲命令的輸入參數

$ find ~ -type f -name 'foo*' -exec ls -l '{}' ';'
$ find ~ -type f -name 'foo*' -exec ls -l '{}' +

$ find ~ -type f -name 'foo*' -print | xargs ls -l

{} is a symbolic representation of the current pathname

semicolon is a required delimiter indicating the end of the command.

+：the system only has to execute the ls command once

  • 文件名中空格的處理
$ find ~ -type f -name '*.jpg' -print0 | xargs --null ls -l

  • touch - change file timestamps

    文件不存在時會建立。

  • stat - display file or file system status

    which is a kind of souped-up version of ls.

18 – Archiving And Backup

管理大量的文件-壓縮。

  • compression archiving區別？

本節講3個內容：

compression

archiving

synchronization

  • gzip, gunzip, zcat - compress or expand files

gzip壓縮，gunzip解壓。

gzip會替換原來的文件。

  • zcat - equivalent to gunzip with the -c option
  • zless
  • bzip2, bunzip2 - a block-sorting file compressor
  • linux壓縮和打包是兩個要領，分別對應zip和tar，window下面是否是沒有區分？
  • tar -

tape archive

.tar or .tgz, which indicate a 「plain」 tar archive and a gzipped archive, respectively.

# c create
$ tar cf playground.tar playground
# t list
$ tar tf playground.tar
# x extract
$ tar xf ../playground.tar
# r append

# extract a single file
$ tar xf archive.tar pathname

注意區分mode與option，c是mode，f是option。

mode要先寫。

studout或者stdin就能夠用減號（-）來替代

  • 不解壓縮直接查看.gz和.bz2命令

使用zcat能夠查看*.gz文件內容，使用bzcat能夠直接查看*.bz2文件.

  • 用ssh從遠程服務器複製文件到本地
$ ssh root@xxx.xxx.xx.xx -p 端口號 'tar cf - filename' | tar xf -
root@xxx.xxx.xx.xx's password:

先tar，傳回來再導出。

  • zip - 既壓縮也歸檔
  • zip,unzip的做用是交換indows文件，linux用tar,gzip
  • rsync - 備份與同步
rsync options source destination
$ rsync -av playground foo

19 – Regular Expressions

後面幾章都是關於文字處理的。先講regular expression.

  • metacharacters & literal characters

metacharacters是^ $ . [ ] { } - ? * + ( ) | \ ，其他就是literal characters。

  • The Any Character

就是點號，the dot or period character

  • anchors

The caret (^) (只匹配行首出來的)and dollar sign ($) （只匹配行尾出現的）

$ grep -h '^zip' dirlist*.txt
$ grep -h 'zip$' dirlist*.txt
$ grep -h '^zip$' dirlist*.txt
  • 點號不是任意長度嗎？加上^$就表明一個？
$ grep -h '.zip' dirlist*.txt
$ grep -i '^..j.r$' /usr/share/dict/words
  • Bracket Expressions And Character Classes

[] 用來只匹配一個字符。

$ grep -h '[bg]zip' dirlist*.txt
^ : negation,只在第一個位置時有效
$ grep -h '[^bg]zip' dirlist*.txt # 注意出現的位置,表示取反
- : range
$ grep -h '^[A-Za-z0-9]' dirlist*.txt

# 要匹配-（短橫線），就把它放到第一位
$ grep -h '[A-Z]' dirlist*.txt
$ grep -h '[-AZ]' dirlist*.txt
  • POSIX Character Classes

出現的緣由：posix和ASCII不同。

  • alternation

[]是單個，它(|)是string

$ echo "AAA" | grep -E 'AAA|BBB|CCC'
$ grep -Eh '^(bz|gz|zip)' dirlist*.txt
  • Quantifiers

匹配的個數

? - Match An Element Zero Or One Time ,一個字符出現的次數，和word裏面的?不一樣，word裏面表明任意一個字符，這裏的?不能單獨使用，要跟在一個字符後面。

$ echo "(555) 123-4567" | grep -E '^\(?[0-9][0-9][0-9]\)? [0-9][0-9][0-9]-[0-9][0-9][0-9][0-9]$'

* - Match An Element Zero Or More Times

$ echo "This works." | grep -E '[[:upper:]][[:upper:][:lower:] ]*\.'

和?同樣，前面要有字符。

+ - Match An Element One Or More Times

至少出現一次。

$ echo "This that" | grep -E '^([[:alpha:]]+ ?)+$'

{ } - Match An Element A Specific Number Of Times

$ echo "(555) 123-4567" | grep -E '^\(?[0-9]{3}\)? [0-9]{3}-[0-9]{4}$'
  • find使用reg
$ find . -regex '.*[^-_./0-9a-zA-Z].*'
# 反 0-無窮個字符
  • locate使用reg
$ locate --regex 'bin/(bz|gz|zip)'
  • less和vi中也可使用

注意它們的區別，在vi中reg是basic

# less
/^\([0-9]{3}\) [0-9]{3}-[0-9]{4}

# vi
/([0-9]\{3\}) [0-9]\{3\}-[0-9]\{4\}

20 – Text Processing

真是一切皆文件。

  • cat的一些option
$ cat -A foo.txt # 顯示全部符號，包括控制符
$ cat -ns foo.txt # n顯示行號，s抑制多餘的空行
  • sort
$ sort > foo.txt
c
b
a

# 多個文件
$ sort file1.txt file2.txt file3.txt > final_sorted_list.txt

du - estimate file space usage

$ du -s /usr/share/* | head # 顯示前10個
$ du -s /usr/share/* | sort -nr | head # 大到小排序後顯示前10個

按指定列排序(k)：

$ ls -l /usr/bin | sort -nr -k 5 | head

多個key排序：

$ sort --key=1,1 --key=2n distros.txt
# 一個field內多個排序：
$ sort -k 3.7nbr -k 3.1nbr -k 3.4nbr distros.txt

指定分割符：

$ sort -t ':' -k 7 /etc/passwd | head
  • uniq
$ sort foo.txt | uniq
  • cut
# 取第3列
$ cut -f 3 distros.txt

每一個field的長度可能不一樣，若是要取全部行中時間中的年，就不可能用字符位置去表示。而先把field取出來，就規整了。

$ cut -f 3 distros.txt | cut -c 7-10
  • expand - 把tab換爲等量空格
$ expand distros.txt | cut -c 23-
  • 改cut的默認分割符，用d option
$ cut -d ':' -f 1 /etc/passwd | head
  • paste

adds one or more columns of text to a file

$ sort -k 3.7nbr -k 3.1nbr -k 3.4nbr distros.txt > distros-by-date.txt
$ cut -f 1,2 distros-by-date.txt > distros-versions.txt
$ cut -f 3 distros-by-date.txt > distros-dates.txt
$ paste distros-dates.txt distros-versions.txt
  • join

It joins data from multiple files based on a shared key field.

$ join distros-key-names.txt distros-key-vernums.txt | head
  • comm
$ comm file1.txt file2.txt
# -n 抑制列
$ comm -12 file1.txt file2.txt
  • diff
$ diff file1.txt file2.txt
# context format 看兩個各自都有什麼，沒有什麼
$ diff -c file1.txt file2.txt
# unified format 看從舊到新的變化
$ diff -u file1.txt file2.txt
  • patch

apply changes to text files

用來更新文件，好比代碼。

$ diff -Naur file1.txt file2.txt > patchfile.txt
$ patch < patchfile.txt
  • tr

轉換

$ echo "lowercase letters" | tr a-z A-Z
LOWERCASE LETTERS
#　多對一的轉換
$ echo "lowercase letters" | tr [:lower:] A
# squeeze
$ echo "aaabbbccc" | tr -s ab
abccc
  • sed

stream editor

# search-and-replace
$ echo "front" | sed 's/front/back/'
# 誰跟在command後面就是delimiter
$ echo "front" | sed 's_front_back_'
# 指定要處理的行數
$ echo "front" | sed '1s/front/back/'

# p：打印出來
# -n： not to print every line by default
$ sed -n '1,5p' distros.txt
$ sed -n '/SUSE/p' distros.txt
$ sed -n '/SUSE/!p' distros.txt

# 通常出輸出到屏幕，用i原地
$ sed -i 's/lazy/laxy/; s/jumped/jimped/' foo.txt
  • reg的back references
$ sed 's/\([0-9]\{2\}\)\/\([0-9]\{2\}\)\/\([0-9]\{4\}\)$/\3-\1-\2/' distros.txt
  • 所有替換
$ echo "aaabbbccc" | sed 's/b/B/'
$ echo "aaabbbccc" | sed 's/b/B/g'
  • sed用script
  • aspell

拼寫檢查

$ aspell check foo.txt

# -H (HTML) checking-mode option
$ aspell -H check foo.txt

改完了會產生一個bak文件，存在原來的文本。

21 – Formatting Output

講文本輸出格式。

  • nl

顯示行號

$ nl distros.txt | head
# 和cat -n 同樣

$ sort -k 1,1 -k 2n distros.txt | sed -f distros-nl.sed | nl
  • fold – Wrap Each Line To A Specified Length
$ echo "The quick brown fox jumped over the lazy dog." | fold -w 12
# 不打斷單詞
$ echo "The quick brown fox jumped over the lazy dog." | fold -w 12 -s
  • fmt – A Simple Text Formatter

it fills and joins lines in text while preserving blank lines and indentation.

$ fmt -w 50 fmt-info.txt | head
$ fmt -cw 50 fmt-info.txt
# 一行若是不足50個字符，會把第二行的補過來
# 只對'# '開頭的行操做
$ fmt -w 50 -p '# ' fmt-code.txt
  • pr – Format Text For Printing

used to paginate text.

# l:page length
# w:width
$ pr -l 15 -w 65 distros.txt

首尾會有空行。

  • printf – Format And Print Data

「print formatted」

使用：printf 「format」 arguments

$ printf "I formatted the string: %s\n" foo
$ printf "%d, %f, %o, %s, %x, %X\n" 380 380 380 380 380 380

Multiple optional components ：

%[flags][width][.precision]conversion_specification
  • Document Formatting Systems

tex後來取代了troff.

  • groff
$ zcat /usr/share/man/man1/ls.1.gz | head
$ man ls | head
$ zcat /usr/share/man/man1/ls.1.gz | groff -mandoc -T ascii | head

# 存爲postscript到桌面
$ zcat /usr/share/man/man1/ls.1.gz | groff -mandoc > ~/Desktop/foo.ps

# ps轉pdf
$ ps2pdf ~/Desktop/foo.ps ~/Desktop/ls.pdf
  • tbl

22 – Printing

介紹了ps的由來。

CUPS (Common Unix Printing System) ：provides print drivers and print-job management

Ghostscript： a PostScript interpreter, acts as a RIP.

  • pr – Convert Text Files For Printing

pr is used to adjust text to fit on a specific page size, with optional page headers and margins.

# 3列
$ ls /usr/bin | pr -3 -w 65 | head

CUPS 有兩種printing，lpr, lp.

  • lpr – Print Files (Berkeley Style)
$ ls /usr/bin | pr -3 | lpr
lpr: Error - no default destination available.
# 查看printer
$ lpstat -a
lpstat: No destinations added.
  • lp – Print Files (System V Style)
  • a2ps

「Anything to PostScript.」

$ ls /usr/bin | pr -3 -t | a2ps -o ~/Desktop/ls.ps -L 66 -R
  • lpstat – Display Print System Status
# 查詢printer狀態
$ lpstat -a
$ lpstat -a
  • lpq – Display Printer Queue Status
$ lpq
PDF is ready
no entries
  • lprm / cancel – Cancel Print Jobs

23 – Compiling Programs

compile

linker，解決共享問題，用共享庫。

$ which gcc
/usr/bin/gcc
  • 用ftp下載source code
$ mkdir src
$ cd src
$ ftp ftp.gnu.org
Name (ftp.gnu.org:me): anonymous
ftp> cd gnu/diction
ftp> ls
ftp> get diction-1.11.tar.gz
ftp> bye

$ tar xzf diction-1.11.tar.gz
# 先查看是否是在一個文件夾裏，以避免解壓散開
$ tar tzvf diction-1.11.tar.gz | less

共享的頭文件在/usr/include ，The header files in this directory were installed when we installed the compiler.

  • Building The Program

通常須要兩步。

# 因爲script不在默認的目錄，須要./告訴shell它在當前目錄
$ ./configure
config.status: creating Makefile
config.status: creating diction.1
config.status: creating diction.texi
config.status: creating diction.spec
config.status: creating style.1
config.status: creating test/rundiction
config.status: creating config.h

Makefile is a configuration file that instructs the make program exactly how to build the program. Without it, make will refuse to run.

$ make
  • make只complie須要complie的文件，省時間。
  • Installing The Program
$ sudo make install

Part 4 – Writing Shell Scripts

24 – Writing Your First Script

  • script開頭的#!

shebang，指明interpreter，每一個script開頭都應該加這行。

  • 讓script能夠執行
$ chmod 755 hello_world

owner的權限是x就能夠了。755全部人能夠執行，700只有owner能夠執行。必須是可讀的。

  • 執行
$ ./hello_world

必需要加上./ ,由於文件位置的問題。自動識別的命令的目錄都寫在了PATH中，能夠查看它：

$ echo $PATH
  • 把某個路徑添加到PATH

在.bashrc （它在~中）中添加：

# 把~/bin添加在PATH的前面
export PATH=~/bin:"$PATH"

# 在當前terminal中生效
$ . .bashrc
# 或者
$ source .bashrc
  • script放哪？
~/bin ： 本身用
 /usr/local/bin ： 同一個系統多人用
 /usr/local/sbin： 系統管理員用
 /usr/local：   locally supplied software，
  • line-continuations

就是\

  • vi中對script的配置
:syntax on
:set hlsearch
:set tabstop=4
:set autoindent

永久：
寫在~/.vimrc，沒有就建立

25 – Starting A Project

shell是動態語言

uppercase letters to designate constants and lower case
letters for true variables. 

$ vim ~/bin/sys_info_page
$ chmod 755 ~/bin/sys_info_page
$ sys_info_page
$ sys_info_page > sys_info_page.html
$ firefox sys_info_page.html
  • Variables And Constants

uppercase letters to designate constants and lower case letters for true variables.

Note that in an assignment, there must be no spaces between the variable name, the equals sign, and the value.

a=z                  # Assign the string "z" to variable a.
b="a string"         # Embedded spaces must be within quotes.
c="a string and $b"  # Other expansions such as variables can be 
                     # expanded into the assignment.
d="$(ls -l foo.txt)" # Results of a command.
e=$((5 * 7))         # Arithmetic expansion.
f="\t\ta string\n"   # Escape sequences such as tabs and newlines.

大括號的做用

$ filename="myfile"
$ touch "$filename"
$ mv "$filename" "$filename1"
mv: cannot move 'myfile' to '': No such file or directory

$ mv "$filename" "${filename}1"
  • here document
command << token
text
token

# shell example
cat << _EOF_
<HTML>
    <HEAD>
        <TITLE>$TITLE</TITLE>
    </HEAD>
</HTML>
_EOF_

# terminal example
$ cat << _EOF_
> $foo
> "$foo"
> '$foo'
> \$foo
> _EOF_

26 – Top-Down Design

講程序設計的思想。

  • shell functions

兩種形式：

function name {
commands
return
}

and the simpler (and generally preferred) form:

name () {
commands
return
}

in order for function calls to be recognized as shell functions and not interpreted as the names of external programs, shell function definitions must appear in the script before they are called.

  • local variables

要加local

funct_1 () {
    local foo # variable foo local to funct_1
    foo=1
    echo "funct_1: foo = $foo"
}

27 – Flow Control: Branching With if

shell的if語句挺複雜的。

if commands; then
    commands
[elif commands; then
    commands...]
[else
    commands]
fi

x=5
if [ "$x" -eq 5 ]; then
    echo "x equals 5."
else
    echo "x does not equal 5."
fi
  • exit status

程序結果後返回給系統的。查詢：

$ ls -d /usr/bin
$ echo $?

true - do nothing, successfully

false - do nothing, unsuccessfully

  • test

語法：

test expression
and the more popular:
[ expression ]

expression是true時返回0，不然返回1

test和[ 本質上是同樣的。

使用：

#!/bin/bash 
# test-file: Evaluate the status of a file

FILE=~/.bashrc

if [ -e "$FILE" ]; then
    if [ -f "$FILE" ]; then
        echo "$FILE is a regular file."
    fi
    if [ -d "$FILE" ]; then
        echo "$FILE is a directory."
    fi
    if [ -r "$FILE" ]; then
        echo "$FILE is readable."
    fi
    if [ -w "$FILE" ]; then
        echo "$FILE is writable."
    fi
    if [ -x "$FILE" ]; then
        echo "$FILE is executable/searchable."
    fi
else
    echo "$FILE does not exist"
    exit 1
fi

exit  # ?

When a script 「runs off the end」 (reaches end of file), it terminates with an exit status of the last command executed.

  • String Expressions
#!/bin/bash
# test-string: evaluate the value of a string
ANSWER=maybe
if [ -z "$ANSWER" ]; then
    echo "There is no answer." >&2
    exit 1
fi
if [ "$ANSWER" = "yes" ]; then
    echo "The answer is YES."
elif [ "$ANSWER" = "no" ]; then
    echo "The answer is NO."
elif [ "$ANSWER" = "maybe" ]; then
    echo "The answer is MAYBE."
else
    echo "The answer is UNKNOWN."
fi
  • Integer Expressions
  • test的選項真多！！！
  • test用regex

[[]]

# 檢驗是否是數
if [[ "$INT" =~ ^-?[0-9]+$ ]]; then
  • (( )) - Designed For Integers
$ if ((1)); then echo "It is true."; fi
It is true.

if ((INT == 0)); 
if ((INT < 0)); 
if (( ((INT % 2)) == 0));
  • Combining Expressions
Operation   test   [[ ]] and (( ))
AND         -a           &&
OR          -o           ||
NOT         !            !

例子：

# [[]]
if [[ "$INT" -ge "$MIN_VAL" && "$INT" -le "$MAX_VAL" ]];
# test
if [ "$INT" -ge "$MIN_VAL" -a "$INT" -le "$MAX_VAL" ];
  • Control Operators: Another Way To Branch

The && (AND) and || (OR)

$ mkdir temp && cd temp
$ [[ -d temp ]] || mkdir temp
  • []和[[]]的區別

[[]]和[]同樣，不過有兩個新特性，：

string1 =~ regex

== operator

  • [[]]和表達式之間要有空格
[[ "$count" -gt 5 ]]
[[ "$count" -gt 5]] # 會報錯，執行起來會很可怕

28 – Reading Keyboard Input

  • read – Read Values From Standard Input

read a single line of standard input.

read [-options] [variable...]

# -n option,suppresses the trailing newline on output
echo -n "Please enter an integer -> "
read int

read var1 var2 var3 var4 var5 # 多個

# -p prompt
read -p "Enter one or more values > "
# -t seconds
# -s Silent mode. 
if read -t 10 -sp "Enter secret passphrase > " secret_pass;
  • IFS

Internal Field Separator

file_info=$(grep "^$user_name:" $FILE)

# 寫在一行，IFS只改變跟在後面的 command
IFS=":" read user pw uid gid name home shell <<< "$file_info"
  • here string

The <<< operator indicates a here string.

  • You Can’t Pipe read

就像函數與子函數，凡是這一類都不能pip。

  • Validating Input
  • Menus

就是提示和if。

  • 一個程序中出現多個exit是個很差的設計

那應該如何作？

29 – Flow Control: Looping With while / until

有3各loop

  • while
語法：
while commands; do commands; done

#!/bin/bash
# while-count: display a series of numbers
count=1
while [[ "$count" -le 5 ]]; do
    echo "$count"
    count=$((count + 1))
done
echo "Finished."
  • Breaking Out Of A Loop

break

continue

  • until

和while相反

count=1
until [[ "$count" -gt 5 ]]; do
    echo "$count count=$((count + 1)) done echo "Finished.
  • read file with loop
#!/bin/bash 
# while-read

while read distro version release; do
    printf "distro: %s\tversion: %s\treleased: %s\n" \
        "$distro" \
        "$version" \
        "$release"
done < distros.txt

30 – Troubleshooting

講錯誤的處理。

  • 引號不對稱

  • 特定符號缺失

  • 意外的expansion

    是否報錯取決於expansion的值。

  • 雙引號的做用

# number爲空時會報錯
number=
if [ $number = 1 ]; then
# 這樣就不會了 「」=1
[ "$number" = 1 ]

Multi-word string也應該用雙引號括起來。

因此，老是用雙引號把變量括起來。

  • logical error
  • 給出錯誤信息是比較好的作法
  • unix文件名稱什麼字符均可以用，有兩個例外

/ , null character(a zero byte)

  • the POSIX Portable Filename Character Set

uppercase letters A-Z, lowercase letters a-z, numerals 0-9, period (.), hyphen (-), and underscore(_).

  • 檢查input
  • 要會寫測試

利用echo輸出

在更改的地方加註釋，方便恢復。

  • test case

case要反映edge and corner

  • debug技術

commenting out :部分註釋

tracing :在一些關鍵點增長輸出，查看是否執行到這裏了

能夠在script的第一行加入x option:

#!/bin/bash -x

也能夠用set設置部分tracing

set -x # turn on
...
set +x # turn off
  • 輸出中間過程當中變量的值
  • 推薦《The Art of Unix Programming》

31 – Flow Control: Branching With case

  • case的用法
1 #!/bin/bash
  2 
  3 # case-menu
  4 
  5 clear
  6 echo " 7 please select: 8 9 1. display system information 10 2. display disk space 11 3. display home space utilization 12 0. quit 13 "
 14 read -p "enter selection [0-3] > "
 15 
 16 case "$REPLY" in
 17     0)  echo "program terminated"
 18         exit
 19         ;;
 20     1)  echo "hostname: $HOSTNAME"
 21         uptime
 22         ;;
 23     2)  df -h
 24         ;;
 25     3) if [[ "$(id -u)" -eq 0 ]]; then
 26             echo "home space utilization (all users)"
 27             du -sh /home/*
 28         else
 29             echo "home space utilization ($USER)"
 30             du -sh "$HOME"
 31         fi
 32         ;;
 33     *)  echo "invalid entry" >&2
 34         exit 1
 35         ;;
 36 esac

32 – Positional Parameters

講處理command的option

  • positional parameter

$0是當前命令的路徑。

注意它的含義：

Note that $0 always contains the full pathname of the first item on the command line (i.e., the name of the program) and does not contain the name of the shell function as we might expect.

是整個路徑，不是命令的名字。

大於9時用大括號括起來。

  • 參數個數：$#
  • shift

讓$1下次變成$2，全部參數都移動，$#減1

  • basename

去除文件名前面的字符，只要文件名。

不加雙引號時是同樣的，遇到空格就拆分，加雙引號時不同，$*會將全部參數放到一個字符串中，$@會將每一個輸入的參數分別看成參數，也就是不區分輸入時的空格。

$@常常用。

33 – Flow Control: Looping With for

講for

# 兩種風格
for variable [in words]; do
    commands
done

# c語言的
for (( expression1; expression2; expression3 )); do
    commands
done

例子：

[me@linuxbox ~]$ for i in A B C D; do echo $i; done

Create list:

itscs-MacBook-Pro:~ itsc$ for i in {A..D}; do echo $i; done
# 文件列表 pathname expansion
itscs-MacBook-Pro:learnCommandLine itsc$ for i in dis*.txt; do echo "$i"; done

34 – Strings And Numbers

  • 通常狀況下將變量括起來，防止被拆分
  • parameter expansion是什麼？

變量是其中的一種

  • 大括號的做用
$a
${a}
a="foo"
echo "${a}_file" # 和其它字符相連時防止造成不存在的變量
  • 處理空的和不存在的字符
parameter爲空或者不存在就用word，存在就用它本身。
${parameter:-word}
和上面基本同樣，區別：要賦值。
${parameter:=word}
unset和empty就發發送word到error
${parameter:?word}
${parameter:+word}
  • String operation
${#parameter} # 換成長度

$ foo="This string is long."
$ echo "'$foo' is ${#foo} characters long. 'This string is long.' is 20 characters long ${parameter:offset} ${parameter:offset:length} [me@linuxbox ~]$ foo="This string is long." [me@linuxbox ~]$ echo ${foo:5} string is long. [me@linuxbox ~]$ echo ${foo:5:6} string 刪除一部分 ${parameter#pattern} ${parameter##pattern} [me@linuxbox ~]$ foo=file.txt.zip [me@linuxbox ~]$ echo ${foo#*.} txt.zip [me@linuxbox ~]$ echo ${foo##*.} zip 反着刪除 ${parameter%pattern} ${parameter%%pattern} [me@linuxbox ~]$ foo=file.txt.zip [me@linuxbox ~]$ echo ${foo%.*} file.txt [me@linuxbox ~]$ echo ${foo%%.*} file 替換 ${parameter/pattern/string} ${parameter//pattern/string} ${parameter/#pattern/string} ${parameter/%pattern/string}
  • 能夠用expansion來提升script的效率
  • 大小寫轉換

能夠用來作什麼？好比數據庫的查找，匹配的時候把輸入和數據庫中的都統一大小寫。

  • declare
  • shell不能進行浮點運算
itscs-MacBook-Pro:learnCommandLine itsc$ echo $((3.3+4.2))
-bash: 3.3+4.2: syntax error: invalid arithmetic operator (error token is ".3+4.2")

解決：perl, awk.書裏用最簡單的bc

爲何連浮點運算都不支持？真是麻煩。

35 – Arrays

  • scalar variables

Single value

  • shell的array只能一維
[me@linuxbox ~]$ a[1]=foo
[me@linuxbox ~]$ echo ${a[1]}
foo
[me@linuxbox ~]$ declare -a a

# assign
name[subscript]=value
name=(value1 value2 ...)
[me@linuxbox ~]$ days=(Sun Mon Tue Wed Thu Fri Sat)
[me@linuxbox ~]$ days=([0]=Sun [1]=Mon [2]=Tue [3]=Wed [4]=Thu [5]=Fri [6]=Sat)
  • 遍歷array
[me@linuxbox ~]$ animals=("a dog" "a cat" "a fish")
[me@linuxbox ~]$ for i in ${animals[*]}; do echo $i; done
[me@linuxbox ~]$ for i in ${animals[@]}; do echo $i; done
[me@linuxbox ~]$ for i in "${animals[*]}"; do echo $i; done
[me@linuxbox ~]$ for i in "${animals[@]}"; do echo $i; done
  • bash的array不必定是要連續的

因此須要有方法知道哪些位置上有值

[me@linuxbox ~]$ foo=([2]=a [4]=b [6]=c)
[me@linuxbox ~]$ for i in "${foo[@]}"; do echo $i; done
a
b
c
[me@linuxbox ~]$ for i in "${!foo[@]}"; do echo $i; done
2
4
6
  • array的追加
$ foo=(a b c)
$ foo[100]=e
$ echo ${foo[@]}
a b c e
$ foo+=(k l)
$ echo ${foo[@]}
a b c e k l
$ for i in "${foo[@]}"; do echo $i; done
a
b
c
e
k
l
$ for i in "${!foo[@]}"; do echo $i; done
0
1
2
100
101
102

subscript不是連續的

  • associative arrays

下標能夠是字符

36 – Exotica

講一些不經常使用的，在特定場景下使用的

  • Group Commands And Subshells
Group command:
{ command1; command2; [command3; ...] }
Subshell:
(command1; command2; [command3;...])

它們是作什麼的？

manage redirection

{ ls -l; echo "Listing of foo.txt"; cat foo.txt; } > output.txt
(ls -l; echo "Listing of foo.txt"; cat foo.txt) > output.txt

結合pip

{ ls -l; echo "Listing of foo.txt"; cat foo.txt; } | lpr
  • 注意大括號的使用

due to the way bash implements
group commands, the braces must be separated from the commands by a space and the
last command must be terminated with either a semicolon or a newline prior to the closing brace.

  • group和subshell的區別

subshell和它的名字同樣，返回時enviroment會丟失，因此通常狀況下用group.

echo "foo" | read
echo $REPLY # 這就是subshell的例子，reply是空的

commands in pipelines are always executed in subshells

  • process substitution

用來解決subshell的問題

read < <(echo "foo")
echo $REPLY
  • asynchronous execution
  • 什麼是異步？

什麼是阻塞，非阻塞，同步，異步？

舉個打電話的例子：

阻塞/同步：打一個電話一直到有人接爲止
非阻塞：打一個電話沒人接，每隔10分鐘再打一次，知道有人接爲止
異步：打一個電話沒人接，轉到語音郵箱留言（註冊），而後等待對方回電（call back)

看起來異步是最高效，充分利用資源，能夠想像整個系統能支持大規模併發。但問題是調試很麻煩，不知道何時call back。

  • Named pipes

create a connection between two processes and can be
used just like other types of files.

named pipe是一個文件起着pipe的做用

[me@linuxbox ~]$ mkfifo pipe1
[me@linuxbox ~]$ ls -l pipe1
prw-r--r-- 1 me me 0 2009-07-17 06:41 pipe1

# terminal 1
[me@linuxbox ~]$ ls -l > pipe1

# terminal2
[me@linuxbox ~]$ cat < pipe1

問題

  • [[:upper:]]哪一節的？

  • 每一個文件夾下的.和..的做用？爲何必需要有它們？

    它們表明當前和前一級目錄，這個你們都知道，能夠爲何要有它們？設計原理是什麼？

    參考：SO，只說了有，沒有說什麼爲何。

  • shell中各類括號[]、[[]]的做用與區別？

    參考。

須要瞭解的知識

  • linux文件系統

磁盤、分區及Linux文件系統 [Disk, Partition, Linux File System]

  • 硬盤接口

IDE、SATA、SCSI、SAS、FC、SSD硬盤類型介紹

ENABLING VI SYNTAX COLORS

cd ~
vi .vimrc
syntax on
colorscheme desert
  • vi indent

Redefine tab as 4 spaces

添加到vimrc中即永久：

set tabstop=4

  • 標準錯誤輸出>&2輸出到哪了？

    屏幕。

  • vim color

/usr/share/vim/vim74/colors/

我以爲還不錯的：

koehler

industry

murphy

ron

  • ls如何按文件類型排序？

How to make ls sort by file extension and then name?

ls -lX

Reveal the line numbers:

:set number

then

:5,$s/^/#/g comment

or this:

:5,$s/^/#/ uncomment
  • vim如何複製到clipboard?

Vim: copy selection to OS X clipboard still fail to work

To check if your Vim has clipboard support: vim --version | grep -i clipboard. You should see +clipboard. If you see -clipboard your Vim was compiled without clipboard support…

itscs-MacBook-Pro:learnCommandLine itsc$ vim --version | grep -i clipboard
-clipboard       +job             +path_extra      +user_commands
+eval            -mouse_dec       +statusline      -xterm_clipboard

個人是減號，說明不支持。

Getting Vim with +clipboard support

要先裝Homebrew，裝完後的vim在 /usr/local/bin/vim,原來的在/usr/bin，也存在，只不過PATH中，local的在前面，命令會先用它。

  • 單引號、雙引號區別
echo $1
echo "$1"
echo '$1'

$ ./longest words
words
words
$1

雙引號與無引號在這裏是同樣的，雙引號的做用體如今字符串爲空的時候，爲空的時候是空字符串，能夠判斷，而無引號就麻煩了，好比等號，一邊爲空會報錯，說等號是個二元操做符兩邊都要有值。單引號的做用就是禁止轉義，所有爲字符串。

$#
添加到Shell的參數個數

$! shell parameter, which will always contain the process ID of the
last job put into the background.

網站

The Bash Hackers Wiki

相關文章
相關標籤/搜索
每日一句
    每一个你不满意的现在，都有一个你没有努力的曾经。
本站公眾號
   歡迎關注本站公眾號,獲取更多信息
聯繫我們 最近搜索 最新文章 沪ICP备13005482号-10 MyBatis教程 SQL 教程 MySQL教程 Java 教程 Thymeleaf 教程 Hibernate教程 Spring教程 Redis教程