上下文管理、路徑操做、序列化

時間 2019-12-08

標籤上下文管理路徑序列简体版

原文原文鏈接

1、上下文管理

In [2]: lst = []

In [3]: for i in range(1000):
   ...:     lst.append(open('./hello.py'))
   ...:     

In [4]: len(lst)
Out[4]: 1000

In [5]: for i in range(1000):
   ...:     lst.append(open('./hello.py'))
   ...:     
---------------------------------------------------------------------------
OSError                                   Traceback (most recent call last)
<ipython-input-5-0b1b3d821ecd> in <module>()
      1 for i in range(1000):
----> 2     lst.append(open('./hello.py'))
      3 

OSError: [Errno 24] Too many open files: './hello.py'

In [6]: for x in lst:
   ...:     x.close()
   ...:     

In [7]: f = open('./hello.py')

In [8]: f.close()

In [9]:

一、上下文管理方法

with open() as f:
    pass

In [9]: f = open('./hello.py')      # 因爲第二步報錯，因此第三步的close操做沒有被正常執行。
   ...: f.write('abc')
   ...: f.close()
   ...: 
---------------------------------------------------------------------------
UnsupportedOperation                      Traceback (most recent call last)
<ipython-input-9-9d43d4b0758e> in <module>()
      1 f = open('./hello.py')
----> 2 f.write('abc')
      3 f.close()

UnsupportedOperation: not writable

In [10]: f.fileno()
Out[10]: 11

In [11]: f.closed                    # 因爲第二步報錯，因此第三步的close操做沒有被正常執行。
Out[11]: False

In [12]: f = open('./hello.py')      # 常規解決方法 
    ...: try:
    ...:     f.write('123')
    ...: finally:
    ...:     f.close()
    ...:     
---------------------------------------------------------------------------
UnsupportedOperation                      Traceback (most recent call last)
<ipython-input-12-0841de49e228> in <module>()
      1 f = open('./hello.py')               
      2 try:
----> 3     f.write('123')
      4 finally:
      5     f.close()

UnsupportedOperation: not writable

In [13]: f.closed
Out[13]: True

In [14]: 

In [14]: with open('hello.py') as f:    # python 解決方法
    ...:     pass
    ...: 

In [15]: f.closed
Out[15]: True

In [16]: f.name
Out[16]: 'hello.py'

In [17]: %pwd
Out[17]: '/home/magedu'

In [18]: with open('/home/magedu/hello.py') as f:
    ...:     pass
    ...: 

In [19]: f.name           # f.name 返回的並不是是 f 的名字；而是with...open() as f： 中open()內傳入的名字
Out[19]: '/home/magedu/hello.py'

In [20]:

In [6]: from io import StringIO

In [7]: sio = StringIO()        # File-Like對象，文本模式

In [8]: sio.readable()
Out[8]: True

In [9]: sio.write('abcd')
Out[9]: 4

In [10]: sio.read()
Out[10]: ''

In [11]: sio.flush()

In [12]: sio.read()
Out[12]: ''

In [13]: sio.seek(0)
Out[13]: 0

In [14]: sio.read()
Out[14]: 'abcd'

In [15]: sio.getvalue()
Out[15]: 'abcd'

In [16]: sio.write('qwe')
Out[16]: 3

In [17]: sio.getvalue()
Out[17]: 'abcdqwe'

In [18]: sio.close()

In [19]:

In [19]: from io import BytesIO  # 二進制模式

In [20]: bio = BytesIO()

In [21]: bio.write(b'abc')
Out[21]: 3

In [22]: bio.seek(0)
Out[22]: 0

In [23]: bio.read()
Out[23]: b'abc'

In [24]: bio.close()

In [25]: bio.getvalue()
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-25-6be49df4471b> in <module>()
----> 1 bio.getvalue()

ValueError: I/O operation on closed file.

In [26]:

StringIO 和 BytesIO 是子集，類文件對象是超集python

In [27]: bio = BytesIO()

In [28]: buf = bio.getbuffer()

In [29]: bio.close()
---------------------------------------------------------------------------
BufferError                               Traceback (most recent call last)
<ipython-input-29-dc4312aa1350> in <module>()
----> 1 bio.close()

BufferError: Existing exports of data: object cannot be re-sized

In [30]: buf.release()

In [31]: bio.close()

In [32]:

2、路徑的操做

In [33]: import os  # 以字符串的形式進行操做

In [34]: import pathlib # 使用OO 的方式進行操做，Object Oriented（面向對象） 

In [35]: cwd = pathlib.Path('.')

In [36]: cwd
Out[36]: PosixPath('.')

In [37]: help(cwd)

Help on PosixPath in module pathlib object:

Help on PosixPath in module pathlib object:

class PosixPath(Path, PurePosixPath)
 |  PurePath represents a filesystem path and offers operations which
 |  don't imply any actual filesystem I/O.  Depending on your system,
 |  instantiating a PurePath will return either a PurePosixPath or a
 |  PureWindowsPath object.  You can also instantiate either of these classes
 |  directly, regardless of your system.
 |  
 |  Method resolution order:
 |      PosixPath
 |      Path
 |      PurePosixPath
 |      PurePath
 |      builtins.object
 |  
 |  Methods inherited from Path:
 |  
 |  __enter__(self)
 |  
 |  __exit__(self, t, v, tb)
 |  
 |  absolute(self)
 |      Return an absolute version of this path.  This function works
 |      even if the path doesn't point to anything.
 |      
 |      No normalization is done, i.e. all '.' and '..' will be kept along.
 |      Use resolve() to get the canonical path to a file.
# ...略

In [39]: cwd.is_dir()   # 此路徑是否爲目錄
Out[39]: True

In [40]: cwd.iterdir()  # 使cwd 可遍歷         
Out[40]: <generator object Path.iterdir at 0x7f94d24421a8>

In [41]: for i in cwd:
    ...:     print(i)
    ...:     
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-41-5477fd409326> in <module>()
----> 1 for i in cwd:
      2     print(i)
      3 

TypeError: 'PosixPath' object is not iterable

In [42]: for i in cwd.iterdir():
    ...:     print(type(i), i)
    ...:     
<class 'pathlib.PosixPath'> .ICEauthority
<class 'pathlib.PosixPath'> .bash_logout
<class 'pathlib.PosixPath'> .sogouinput
<class 'pathlib.PosixPath'> 下載
<class 'pathlib.PosixPath'> .gnome2_private
<class 'pathlib.PosixPath'> 公共的
# ... 略

In [43]: cwd.mkdir('abcd')  # 在指定的路徑建立一個新目錄
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-43-632e6b8cf3ec> in <module>()
----> 1 cwd.mkdir('abcd')

~/.pyenv/versions/3.6.3/lib/python3.6/pathlib.py in mkdir(self, mode, parents, exist_ok)
   1224             self._raise_closed()
   1225         try:
-> 1226             self._accessor.mkdir(self, mode)
   1227         except FileNotFoundError:
   1228             if not parents or self.parent == self:

~/.pyenv/versions/3.6.3/lib/python3.6/pathlib.py in wrapped(pathobj, *args)
    385         @functools.wraps(strfunc)
    386         def wrapped(pathobj, *args):
--> 387             return strfunc(str(pathobj), *args)
    388         return staticmethod(wrapped)
    389 

TypeError: an integer is required (got type str)

In [44]: d = pathlib.Path('./abcd')

In [45]: d.exists()   # 判斷這條路徑 是否存在
Out[45]: False

In [46]: d.mkdir(0o755)

In [47]: %ls -ld ./abcd
drwxr-xr-x 2 root root 4096 11月 13 23:09 ./abcd/

In [48]: 0o755
Out[48]: 493

In [49]: 0o777      # 777 是一個 八進制的777， 輸出是十進制的數
Out[49]: 511

In [50]: help(d.mkdir)

Help on method mkdir in module pathlib:

mkdir(mode=511, parents=False, exist_ok=False) method of pathlib.PosixPath instance
    Create a new directory at this given path.
~

(END)

In [51]: 0xa
Out[51]: 10

In [52]: 0b10
Out[52]: 2

In [53]:

In [58]: d = pathlib.Path('./ab/cd/ef')

In [59]: d.mkdir(0o755, parents=True)  # parents 參數幫咱們實現級聯建立 
---------------------------------------------------------------------------
FileExistsError                           Traceback (most recent call last)
<ipython-input-59-e309af38104f> in <module>()
----> 1 d.mkdir(0o755, parents=True)  # parents 參數幫咱們實現級聯建立

~/.pyenv/versions/3.6.3/lib/python3.6/pathlib.py in mkdir(self, mode, parents, exist_ok)
   1224             self._raise_closed()
   1225         try:
-> 1226             self._accessor.mkdir(self, mode)
   1227         except FileNotFoundError:
   1228             if not parents or self.parent == self:

~/.pyenv/versions/3.6.3/lib/python3.6/pathlib.py in wrapped(pathobj, *args)
    385         @functools.wraps(strfunc)
    386         def wrapped(pathobj, *args):
--> 387             return strfunc(str(pathobj), *args)
    388         return staticmethod(wrapped)
    389 

FileExistsError: [Errno 17] File exists: 'ab/cd/ef'

In [60]: %ls -lR   
.:
總用量 1644
-rw-rw-r-- 1 magedu magedu  11964 11月  5 11:42 1.docx
drwxr-xr-x 3 root   root     4096 11月 12 22:59 ab/

# ... 略

./ab:
總用量 4
drwxr-xr-x 3 root root 4096 11月 12 22:59 cd/

./ab/cd:
總用量 4
drwxr-xr-x 2 root root 4096 11月 12 22:59 ef/

./ab/cd/ef:
總用量 0

./abcd:
總用量 0

./My Music:
總用量 0

./My Pictures:
總用量 0

./My Videos:
總用量 0
# ... 略  
   
In [68]: d.mkdir(0o755, parents=True, exist_ok=True)     # exist_ok 若文件夾存在，也不會報錯

In [69]: f = pathlib.Path('./ab/cd/a.txt')

In [70]: f.exists()     # f.exists() 能夠用來判斷 這個文件（或文件夾）是否存在。
Out[70]: False

In [71]: f.is_file()
Out[71]: False

In [72]: f.is_dir()
Out[72]: False

In [73]: f.is_absolute()
Out[73]: False

In [74]:

In [19]: d.rmdir()

In [20]: d = pathlib.Path('./ab/')

In [21]: d.rmdir()     # rmdir 刪除路徑的操做，要求必須是空目錄
---------------------------------------------------------------------------
OSError                                   Traceback (most recent call last)
<ipython-input-21-b1299da09e5c> in <module>()
----> 1 d.rmdir()

~/.pyenv/versions/3.6.3/lib/python3.6/pathlib.py in rmdir(self)
   1268         if self._closed:
   1269             self._raise_closed()
-> 1270         self._accessor.rmdir(self)
   1271 
   1272     def lstat(self):

~/.pyenv/versions/3.6.3/lib/python3.6/pathlib.py in wrapped(pathobj, *args)
    385         @functools.wraps(strfunc)
    386         def wrapped(pathobj, *args):
--> 387             return strfunc(str(pathobj), *args)
    388         return staticmethod(wrapped)
    389 

OSError: [Errno 39] Directory not empty: 'ab'

In [22]:

In [23]: f = pathlib.Path('./ab/cd/a.txt')

In [24]: f.exists()
Out[24]: True

In [25]: f.is_file()
Out[25]: True

In [26]: d = pathlib.Path('./ab/cd/d.txt')

In [27]: d.exists()               # 能夠用來 判斷該路徑是否存在！
Out[27]: False

In [28]: d.is_file()
Out[28]: False

In [29]: f.is_absolute()  # 若是是絕對路徑 則返回 True

驅動器）。
Out[29]: False

In [30]: d = pathlib.Path('/home/')

In [31]: d.is_absolute()  # 若是是絕對路徑 則返回 True
Out[31]: True

In [32]: f.absolute().as_uri() # 將路徑返回爲 file： URI
Out[32]: 'file:///home/magedu/ab/cd/a.txt'

In [33]: f.cwd()
Out[33]: PosixPath('/home/magedu')

In [34]: f.absolute()
Out[34]: PosixPath('/home/magedu/ab/cd/a.txt')

In [35]: f.expanduser()
Out[35]: PosixPath('ab/cd/a.txt')

In [36]: pathlib.Path('~').expanduser()
Out[36]: PosixPath('/root')

In [37]: f.home()
Out[37]: PosixPath('/root')

In [38]: f.lchmod  # 連接
Out[38]: <bound method Path.lchmod of PosixPath('ab/cd/a.txt')>

In [39]: f.name
Out[39]: 'a.txt'

In [40]: f.home().name()
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-40-cdc3e8c3b25b> in <module>()
----> 1 f.home().name()

TypeError: 'str' object is not callable

In [41]: f.home().name
Out[41]: 'root'

In [42]: f.owner()
Out[42]: 'root'

In [43]: f.parent
Out[43]: PosixPath('ab/cd')

In [44]: f.home().parent
Out[44]: PosixPath('/')

In [45]: f.parents
Out[45]: <PosixPath.parents>

In [46]: f.parts
Out[46]: ('ab', 'cd', 'a.txt')

In [47]: f.home().root
Out[47]: '/'

In [48]: f.suffix       # 文檔的後綴
Out[48]: '.txt'


In [50]: f.suffixes     # 將文檔的後綴放入一個列表中
Out[50]: ['.txt']

In [51]: f.touch        # 若是它不存在，就用指定的訪問模式建立這個文件
Out[51]: <bound method Path.touch of PosixPath('ab/cd/a.txt')>

In [52]:

一、start

In [61]: f.stat()
Out[61]: os.stat_result(st_mode=33188, st_ino=1185203, st_dev=66307, st_nlink=1, st_uid=0, st_gid=0, st_size=0, st_atime=1510587456, st_mtime=1510587456, st_ctime=1510587456)

In [62]: f.lstat()
Out[62]: os.stat_result(st_mode=33188, st_ino=1185203, st_dev=66307, st_nlink=1, st_uid=0, st_gid=0, st_size=0, st_atime=1510587456, st_mtime=1510587456, st_ctime=1510587456)

In [63]:

二、glob

In [64]: f
Out[64]: PosixPath('ab/cd/a.txt')

In [65]: d
Out[65]: PosixPath('/home')

In [66]: d = pathlib.Path('.')

In [67]: d.glob('**/*.py') 
Out[67]: <generator object Path.glob at 0x7f03ad69aba0>

In [68]: for x in d.glob('**/*.py')
  File "<ipython-input-68-2a1a184286ac>", line 1
    for x in d.glob('**/*.py')
                              ^
SyntaxError: invalid syntax


In [69]: for x in d.glob('**/*.py'):
    ...:     print(x)
    ...:     
hello.py
# ... 略


In [70]: help(d.glob)  # 迭代子目錄，生成指定類型的文件

Help on method glob in module pathlib:

glob(pattern) method of pathlib.PosixPath instance
    Iterate over this subtree and yield all existing files (of any
    kind, including directories) matching the given pattern.
~
(END)


In [72]: help(d.rglob)  # 遞歸生成 全部位置上的指定類型的文件。

Help on method rglob in module pathlib:

rglob(pattern) method of pathlib.PosixPath instance
    .
~

(END)


In [69]: for x in d.glob('**/*.py'):
    ...:     print(x)
    ...:     
hello.py
# ... 略

三、join

In [76]: p = '/' + 'miracle' + '/' + 'Project' + '/' + 'workspace'

In [77]: pathlib.Path('/', '/Users', 'miracleYoung', 'Project') 
Out[77]: PosixPath('/Users/miracleYoung/Project')
# 根據系統環境以及給出的內容，生成 str形式的 路徑。能夠修正輸入的錯誤
In [78]:

四、copy

In [81]: import shutil        # 權限，id，uid， gid

In [82]: shutil.copyfileobj   # 文件對象
Out[82]: <function shutil.copyfileobj>

In [83]: shutil.copyfile      # 文件內容
Out[83]: <function shutil.copyfile>

In [84]: shutil.copymode      # 權限
Out[84]: <function shutil.copymode>

In [85]: shutil.copystat      # 元數據
Out[85]: <function shutil.copystat>

In [86]: shutil.copy          # copyfile + copymode
Out[86]: <function shutil.copy>

In [87]: shutil.copy2         # copyfile + copystat
Out[87]: <function shutil.copy2>

In [88]: shutil.copytree      # 遞歸複製
Out[88]: <function shutil.copytree>

In [89]:

五、remove

In [90]: shutil.rmtree    # remove tree; 遞歸刪除目錄樹
Out[90]: <function shutil.rmtree>

In [91]: help(shutil.rmtree)

Help on function rmtree in module shutil:

rmtree(path, ignore_errors=False, onerror=None)
    Recursively delete a directory tree.  
    
    If ignore_errors is set, errors are ignored; otherwise, if onerror
    is set, it is called to handle the error with arguments (func,
    path, exc_info) where func is platform and implementation dependent;
    path is the argument to that function that caused it to fail; and
    exc_info is a tuple returned by sys.exc_info().  If ignore_errors
    is false and onerror is None, an exception is raised.

~
(END)

In [92]: shutil.move     # 遞歸地將文件或目錄移到另外一個位置。這是相似於UNIX「mv」命令。返回文件或目錄的目的地
Out[92]: <function shutil.move>

In [93]: help(shutil.move)

Help on function move in module shutil:

move(src, dst, copy_function=<function copy2 at 0x7f03b65a90d0>)
    Recursively move a file or directory to another location. This is
    similar to the Unix "mv" command. Return the file or directory's
    destination.  

    
    If the destination is a directory or a symlink to a directory, the source
    is moved inside the directory. The destination path must not already
    exist.
    
    If the destination already exists but is not a directory, it may be
    overwritten depending on os.rename() semantics.
    
    If the destination is on our current filesystem, then rename() is used.
    Otherwise, src is copied to the destination and then removed. Symlinks are
    recreated under the new name if os.rename() fails because of cross
    filesystem renames.
    
    The optional `copy_function` argument is a callable that will be used
    to copy the source or it will be delegated to `copytree`.
    By default, copy2() is used, but any function that supports the same
    signature (like copy()) can be used.
    
    A lot more could be done here...  A look at a mv.c shows a lot of
    the issues this implementation glosses over.

~
(END)

3、序列化與反序列化

序列化：將對象轉化爲數據
反序列化：將數據轉化爲對象

一、pickle模塊

In [95]: import pickle

In [96]: class A:
    ...:     def pr(self):
    ...:         print('xxx')
    ...:         

In [97]: a = A()

In [98]: b = pickle.dumps(a)            # dumps(object) -> string

In [99]: aa = pickle.loads(b)           # loads(string) -> object 

In [100]: a
Out[100]: <__main__.A at 0x7f03ad617b38>

In [101]: a.pr()
xxx

In [102]: aa.pr
Out[102]: <bound method A.pr of <__main__.A object at 0x7f03ac095438>>

In [103]: aa.pr()
xxx

In [104]: b
Out[104]: b'\x80\x03c__main__\nA\nq\x00)\x81q\x01.'

In [105]:

In [107]: obj = list(range(10))

In [108]: obj
Out[108]: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]

In [109]: pickle.dumps(obj)
Out[109]: b'\x80\x03]q\x00(K\x00K\x01K\x02K\x03K\x04K\x05K\x06K\x07K\x08K\te.'

In [110]: class RPC:
     ...:     def __init__(self):
     ...:         self.date = []
     ...:     def server(self):
     ...:         self.date = list(range(10))
     ...:     def client(self):
     ...:         print(self.date) 
     ...:                

In [111]: s = RPC()

In [112]: s.server()

In [113]: s.date
Out[113]: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]

In [114]: pickle.dumps(s)
Out[114]: b'\x80\x03c__main__\nRPC\nq\x00)\x81q\x01}q\x02X\x04\x00\x00\x00dateq\x03]q\x04(K\x00K\x01K\x02K\x03K\x04K\x05K\x06K\x07K\x08K\tesb.'

In [115]:

二、json

In [116]: import json

object
array
number
string
bool
null

dict
list
int/float
str
bool
None

In [118]: d = {'a': 1, 'b': [1, 2, 3]}

In [119]: s = json.dumps(d)

In [120]: type(s)
Out[120]: str

In [121]: json.loads(s)
Out[121]: {'a': 1, 'b': [1, 2, 3]}

In [122]: