[筆記]The Linux command line

時間 2019-12-09

標籤筆記 linux command line 欄目 Linux 简体版

原文原文鏈接

Notes on The Linux Command Line (by W. E. Shotts Jr.)

edited by Gopher

感受博客園是否是搞了什麼CSS在裏頭……在博客園顯示效果挺蛋疼的。。。這本書至關棒，建議一讀html

File, file systems and commands

lsnode

-a(all) -d(directory) -h(human-readable)linux
less (show file content)git
FHSshell
1. /bin: Contains binaries (programs) that must be present for the system to boot and run.express
2. /boot: Contains the Linux kernel, initial RAM disk image (for drivers needed at boot time), and the boot loader.bash
  
  boot/grub/grub.conf or menu.lst, which are used to configure the boot loader
  /boot/vmlinuz, the Linux kernelapp
3. /dev: Device nodesless
4. /etc: The /etc directory contains all of the system-wide configuration files. It also contains a collection of shell scripts which start each of the system services at boot time. Everything in this directory should be readable text.ssh
5. /home: Ordinary users can only write files in their home directories
6. /lib: Shared library files.
7. /lost+found
8. /mnt
9. /opt: This is mainly used to hold commercial software products that may be installed on your system.
10. /proc
11. /root
12. /sbin: This directory contains 「system」 binaries. These are programs that perform vital system tasks that are generally reserved for the superuser.
13. /tmp
14. /usr: Unix software resources. It contains all the programs and support files used by regular users.
/usr/bin contains the executable programs installed by your Linux distribution. Shared libraries of programs contained in it are stored in /usr/lib.

The /usr/local tree is where programs that are not included with your distribution but are intended for system-wide use are installed. Programs compiled from source code are normally installed in /usr/local/bin.

/usr/sbin contains more system administration programs.
1. /var
/var/log records of various system activity

Wildcards

Wildcard	Meeaning
*	Matches any characters
?	Matches any single character
[characters]	Matches any character that is a member of the set characters
[!characters]	Matches any character that is not a member of the set characters
[[:class:]]	Matches any character that is a member of the specified class

Commonly used character classes

Character class	Meaning
[:alnum:]	Matches any alphanumeric character
[:alpha:]	Matches any alphabetic character
[:digit:]	Matches any numeral
[:lower:]	Matches any lowercase letter
[:upper:]	Matches any uppercase letter

cp

-a(--archive, copy with attributes) -u(--update, update only -- do not override)
Be careful with rm! ls first before you rm -rf.
ln

-s(soft/symbolic link)

Something with command
1. type – Indicate how a command name is interpreted
2. which – Display which executable program will be executed
  1. help – Get help for shell builtins
  2. man – Display a command's manual page
  3. apropos – Display a list of appropriate commands
  4. info – Display a command's info entry
  5. whatis – Display a very brief description of a command
  6. alias – Create an alias for a command

To learn more about Steve Bourne, father of the Bourne Shell, see this Wikipedia
article:
http://en.wikipedia.org/wiki/Steve_Bourne
Here is an article about the concept of shells in computing:
http://en.wikipedia.org/wiki/Shell_(computing)
The full version of the Linux Filesystem Hierarchy Standard can be found here:
http://www.pathname.com/fhs/
An article about the directory structure of Unix and Unix-like systems:
http://en.wikipedia.org/wiki/Unix_directory_structure
A detailed description of the ASCII text format:
http://en.wikipedia.org/wiki/ASCII
The Bash Reference Manual is a reference guide to the bash shell. It’s still a ref-
erence work but contains examples and is easier to read than the bash man page.
http://www.gnu.org/software/bash/manual/bashref.html
The Bash FAQ contains answers to frequently asked questions regarding bash.
This list is aimed at intermediate to advanced users, but contains a lot of good in-
formation.
http://mywiki.wooledge.org/BashFAQ
The GNU Project provides extensive documentation for its programs, which form
the core of the Linux command line experience. You can see a complete list here:
http://www.gnu.org/manual/manual.html
Wikipedia has an interesting article on man pages:
http://en.wikipedia.org/wiki/Man_page
A discussion of symbolic links:
http://en.wikipedia.org/wiki/Symbolic_link

Redirection

Commands
1. cat - Concatenate files
2. sort - Sort lines of text
3. uniq - Report or omit repeated lines
4. grep - Print lines matching a pattern
5. wc - Print newline, word, and byte counts for each file
6. head - Output the first part of a file
7. tail - Output the last part of a file
8. tee - Read from standard input and write to standard output and files
>:truncate / redirect stdout
2>: redirect stderr

2>&1: redirect stderr to (redirected) stdout

&>: redirect both stderr and stdout

&>>: redirect with append

2> /dev/null: redirect stderr to bit bucket (supress stderr)
The redirection operator silently creates or overwrites files, so you need to treat it with a lot of respect
e.g. (this will override/destroy less)
cd /usr/bin
ls > less

Shell basics and terminal keystrokes

~(tilde) = /home/user_name
Arithmetic expansion:$((expression)) (/ divide into integers, ** exponentiation, () should contains expressions like $(()))
{} expands ranges. e.g. {1-2000}-{a-z}
Parameter expansion: printenv, $env
Command substitution: command1 $(command2) or command1 `command2`
Quoting:

"" expand expression (treats space as space, e.g., try echo $(cal) and echo "$(cal)")
'' treat literally
\ escape carriage return
/ espace special characters

Keystrokes (Emacs style)

Key	Meaning
Ctrl-a	Move cursor to the beginning of the line.
Ctrl-e	Move cursor to the end of the line.
Ctrl-f	Move cursor forward one character; same as the right arrow key.
Ctrl-b	Move cursor backward one character; same as the left arrow key.
Alt-f	Move cursor forward one word.
Alt-b	Move cursor backward one word.
Ctrl-l	Clear the screen and move the cursor to the top left corner. The clear command does the same thing.
Ctrl-d	Delete the character at the cursor location
Ctrl-t	Transpose (exchange) the character at the cursor location with the one preceding it.
Alt-t	Transpose the word at the cursor location with the one preceding it.
Alt-l	Convert the characters from the cursor location to the end of the word to lowercase.
Alt-u	Convert the characters from the cursor location to the end of the word to uppercase.
Ctrl-k	Kill text from the cursor location to the end of line.
Ctrl-u	Kill text from the cursor location to the beginning of the line.
Alt-Backspace	Kill text from the cursor location to the beginning of the current word. If the cursor is at the beginning of a word, kill the previous word.
Ctrl-y	Yank text from the kill-ring and insert it at the cursor location.

history: C-r (Reverse incremental search.)

The bash man page has major sections on both expansion and quoting which cover these topics in a more formal manner.
The Bash Reference Manual also contains chapters on expansion and quoting:
http://www.gnu.org/software/bash/manual/bashref.html
The Wikipedia has a good article on computer terminals:
http://en.wikipedia.org/wiki/Computer_terminal

Permission and process

su – Run a shell with substitute user and group IDs
sudo – Execute a command as another user
r (4) w (2) x (1); u(user) o(other) g(group)

File types:

Type	Meaning
c	A character special file. This file type refers to a device that handles data as a stream of bytes, such as a terminal or modem.
b	A block special file. This file type refers to a device that handlesdata in blocks, such as a hard drive or CD-ROM drive.
l	A symbolic link. Notice that with symbolic links, the remaining file attributes are always 「rwxrwxrwx」 and are dummy values. The real file attributes are those of the file the symbolic link points to

Attribute	Files	Directory
r	Allows a file to be opened and read.	Allows a directory's contents to be listed if the execute attribute is also set.
w	Allows a file to be written to or truncated, however this attribute does not allow files to be renamed or deleted. The ability to delete or rename files is determined by directory attributes.	Allows files within a directory to be created, deleted, and renamed if the execute attribute is also set.
x	Allows a file to be treated as a program and executed. Program files written in scripting languages must also be set as readable to be executed.	Allows a directory to be entered, e.g., cd directory.

Special permissions:

setuid(4000 or u+s): When applied to an executable file, it sets the effective user ID from that of the real user (the user actually running the program) to that of the program's owner. e.g. let a program runs as root.
setgid(2000 or g+s): changes the effective group ID from the real group ID of the real user to that
of the file owner. If the setgid bit is set on a directory, newly created files in the directory will be given the group ownership of the directory rather the group ownership of the file's creator. This is useful in a shared directory when members of a common group need access to all the files in the directory, regardless of the file owner's primary group.
sticky(1000 or +t): If applied to a directory, it prevents users from deleting or renaming files unless the user is either the owner of the directory, the owner of the file, or the superuser. This is often used to control access to a shared directory, such as /tmp.
su: -l(or -, get login shell, default to root); -c(single command)
chown: (user):(group)
Process
ps – Report a snapshot of current processes (ps aux)
top – Display tasks
jobs – List active jobs
bg – Place a job in the background
fg – Place a job in the foreground
kill – Send a signal to a process
killall – Kill processes by name
shutdown – Shutdown or reboot the system

Process status:

Status	Meaning
R	Running. This means that the process is running or ready to run.
S	Sleeping.
D	Uninterruptible Sleep. Process is waiting for I/O such as a disk drive.
T	Stopped.
Z	A defunct or 「zombie」 process. This is a child process that has terminated, but has not been cleaned up by its parent.
<	A high priority process (richness)
N	A low priority process (nice)

top:
1. %us: user process
2. %sy: kernel process
3. %ni: nice process
4. %id: idle
5. %wa: waiting for I/O.
&: put in background; % specify job number (jobspec)
kill: -signal_number PID

process signal:

Number	Signal	Meaning
1	HUP	Hangup
2	INT	Interrupt
9	KILL	Kill immediately (cannot be ignored)
15	TERM	Terminate (do clean-up before terminate)
18	CONT	Continue, restore a process after a STOP
19	STOP	Stop. This signal causes a process to pause (cannot be ignored)
20	TSTP	Terminal stop, can be ignored

pstree: Outputs a process list arranged in a tree-like pattern showing the parent/child relationships between processes.
vmstat: Outputs a snapshot of system resource usage;
xload A graphical program that draws a graph showing system load over time.
tload: Similar to the xload program, but draws the graph in the terminal

The environment and packages (apt-get)

Login shell (/etc/profile, global, or ~/.bash_profile,~/.bash_login,~/.profile for local user) and non-login shell (/etc/bash.bashrc, global, or ~/.bashrc)

Environment variables:

Variable	Meaning
PATH	A colon-separated list of directories that are searched when you
PS1	Prompt String 1. This defines the contents of your shell prompt
TERM	Terminal type

Apt-get
1. Update repo: apt-get update
2. Search: apt-cache search search_string
3. Update: apt-get install package_name
4. Install/update (dpkg): dpkg --install package_file
5. Remove: apt-get remove package_name
6. List:dpkg --list
7. Status: dpkg --status package_name
8. Description: apt-cache show package_name
9. File identification: dpkg --search file_name

The Debian GNU/Linux FAQ chapter on package management provides an over-
view of package management on Debian systems :
http://www.debian.org/doc/FAQ/ch-pkgtools.en.html
The home page for the RPM project:
http://www.rpm.org
The home page for the YUM project at Duke University:
http://linux.duke.edu/projects/yum/
For a little background, the Wikipedia has an article on metadata:
http://en.wikipedia.org/wiki/Metadata

Storage and networking

Storage related commands:
1. mount – Mount a file system
2. umount – Unmount a file system
3. fsck – Check and repair a file system
4. fdisk – Partition table manipulator
5. mkfs – Create a file system
6. fdformat – Format a floppy disk
7. dd – Write block oriented data directly to a device
8. genisoimage (mkisofs) – Create an ISO 9660 image file
9. wodim (cdrecord) – Write data to optical storage media
10. md5sum – Calculate an MD5 checksum
Devices:
1. /dev/hd*: IDE (PATA)
2. /dev/sd*: SCSI (including PATA/SATA hard disks, flash drives, and USB mass storage devices
3. /dev/sr*: Optical disk
4. /dev/lp*: Printer
sudo tail -f /var/log/messages: System log (including history of removable devices)
dd if=input_file of=output_file [bs=block_size [count=blocks]]: Use with caution!
Networking:
1. ping - Send an ICMP ECHO_REQUEST to network hosts
2. traceroute - Print the route packets trace to a network host
3. netstat - Print network connections, routing tables, interface statistics, mas-
4. uerade connections, and multicast memberships
5. ftp - Internet file transfer program
6. wget - Non-interactive network downloader
7. ssh - OpenSSH SSH client (remote login program)
SFTP is based on SSH

For a broad (albeit dated) look at network administration, the Linux Documentation Project provides the Linux Network Administrator’s Guide:
http://tldp.org/LDP/nag2/index.html
Wikipedia contains many good networking articles. Here are some of the basics:
http://en.wikipedia.org/wiki/Internet_protocol_address
http://en.wikipedia.org/wiki/Host_name
http://en.wikipedia.org/wiki/Uniform_Resource_Identifier

Searching, archive and backup

Searching
1. locate – Find files by name
2. find – Search for files in a directory hierarchy
3. xargs – Build and execute command lines from standard input
4. touch – Change file times
5. stat – Display file or file system status
Find
1. Test: -type (f/d), -name "regex", -size +/-n(k/M/G etc), -iname(case insensitive),-cmin +/- n(modified more/less than n mins)
2. Operator:-not(n) -or(o) -and(a), -and is implicit, \ to escape (), the shell characters
3. Action: -delete -ls -print -quit
  1. User-defined: -exec command '{}' ';' , ' to escape shell characters, -exec command'{}' +, apply command on all output,use -ok in place of -exec to do execution interactively
4. Option: -depth(Direct find to process a directory’s files before the directory itself. This option is automatically applied when the -delete action is specified.), -maxdepth/mindepth levels, -iname: locate inode, can be used to delete inode with which files has weird name
5. xargs: accepts input from standard input and converts it into an argument list for a specified command . e.g., find ~ -iname '*.jpg' -print0 | xargs --null ls -l: Deal with those containing embedded spaces in file names by changing delimiter from space to null.
gzip/gunzip, bzip2/bunzip2
tar:-x(extract), t(content), c(create), r(append) ,f(followed by tar name), v(--verbose), z(gzip). e.g.,tar cf file_compressed.tar fils_to_compress
important: tar path is relative not absolute!
Using tar with find is a good way of creating incremental backups of a directory tree or an entire system. By using find to match files newer than a timestamp file, we could create an archive that only contains files newer than the last archive, assuming that the timestamp file is updated right after each archive is created.
ssh remote-sys 'tar cf - Documents' | tar xf - tar remote file and untar at local, - stands for stdin/stdout

Regex

grep: grep [options] regex [file...]

Option	Meaning
-i	--ignore-case
-v	--invert-match (print lines that don't match)
-l	--files-with-matches (match only file names)
-L	--files-without-match (invert match file names)
-n	--line-number (prefix each matching line with the number of the line within the file)

Metacharacters:
1. .: match any character in that character position
  
  .*accepts any sequence of characters, including an empty string.
2. ^ and $: cause the match to occur only if the regular expression is found at the beginning of the line (^) or at the end of the line ($). e.g., ^exact_matching$
3. [character_set]: [bg], b or g, [^bg] no b or g, [a-z], a to z
4. POSIX character classes

Class	Description
[:alnum:]	The alphanumeric characters. In ASCII, equivalent to:[A-Za-z0-9].
[:word:]	[:alnum:] + _
[:alpha:]	The alphabetic characters. In ASCII, equivalent to: [A-Za-z]
[:blank:]	Space and tab
[:digit:]	0-9
[:lower:]/[:upper:]	Lower/upper case of characters
[:punct:]	Punctuation:[-!"#$%&'()*+,./:;<=>?@[\]_`{\|}~]
[:space:]	space, tab, carriage, return, newline, vertical tab, and form feed([ \t\r\n\v\f])

Extended regex
1. egrep = grep -E: grep with extended regex
2. ?: match an element zero or one time
3. +: match an element one or more times
4. *: match an element zero or more times
  
  e.g. ^([[:alpha:]]+ ?)+$ :match lines consisting of groups of one or more alphabetic characters separated by single spaces
5. {m,n}: min/max repetition m/n. m or n can be omitted

Text processing

cat – Concatenate files and print on the standard output (-A display invisible symbols)
sort – Sort lines of text files
uniq – Report or omit repeated lines
cut – Remove sections from each line of files
expand - replace tabs with spaces
paste – Merge lines of files
join – Join lines of two files on a common field
comm – Compare two sorted files line by line
diff – Compare files line by line (-c context; -u unified; prepare for patch:diff -Naur)
patch – Apply a diff file to an original
tr – Translate(replace) or delete characters

tr -d '\r' < dos_file > unix_file >convert win to unix (EOL)
sed – Stream editor for filtering and transforming text
sed -option 'address/regex/replacement/flag': substitute regex with replacement
One of the flag, g performs substitution on every instance

Address	Meaning
n	line n
$	the last line
/regexp/	basic regex only
addr1,addr2	lines between addr1 and addr2
addr1~steps	the addr1 line and every "steps" lines after
addr1,+n	addr1 and n lines after
addr1!	except addr1

sed back references: If the sequence \n appears in replacement where n is a number from 1 to 9, the sequence will refer to the corresponding subexpression in the preceding regular expression.

Sed manipulation	Meaning
s	substitute
p	print
d	delete
a	append
i	insert in front of
y/set1/set2	Perform transliteration by converting characters from set1 to the corresponding characters in set2. `sed` requires that both sets be equal size

advanced: awk, perl
aspell – Interactive spell checker

Formatting output and printing

nl – Number lines
fold – Wrap each line to a specified length:fold -w(width) n
fmt – A simple text formatter
pr – Prepare text for printing
printf – Format and print data
groff – A document formatting system
pr – Convert text files for printing
lpr – Print files
a2ps – Format files for printing on a PostScript printer
lpstat – Show printer status information
lpq – Show printer queue status
lprm – Cancel print jobs
Page description language(PDL), Raster image processor (RIP)

Shell script

#!： shebang, tells the system the name of the interpreter
Use \ to escape special operators
var="value", use by $var
declare -r CONSTANT_VAR
local: LOCAL_VAR
Here command (feed text to command through stdin, single and double quotes within "here documents" lose their special meaning to the shell):
command << token
text
token
Shell function
function_name (parameter) {
commands
return
}
Control flow
if commands; then
commands
[elif commands; then
commands...]
[else
commands]
fi
What included in[] is called test statement. It returns either true or false.
Regex can be evaluated in [[]] (bash specification)
Integer operation can be done in (())
Exit status: $?, 0 for success. Check command man page for error code.
File operation: -nt(newer than) -ot(older than) -e(exist) -d(directory) -r(regular)
String operation : ==(equal), <,>(sorted after/before), -n(nonezero), -z(zero)
Integer operation: -eq, -ne, -le...
Logical: -a(and,&&), -o(or,||), !(not), =~(inequal)
Loop
While (statement); do
statement
done

for variable [in words]; do
commands
done

for (( expression1; expression2; expression3 )); do
commands
done
Branching:
case word in
pattern commands ;;
...
esac
Positional parameters: \$0(usually filename) \$#(num of args)
Parameter expansion: ${to_expand} or \$to_expand
Group command, subshell, wait, trap, named pipe(mkfifo)

相關標籤/搜索

每日一句

每一个你不满意的现在，都有一个你没有努力的曾经。