把這裏的題目爭取刷一遍
博客記錄html
python的ORM框架peewee
SQLAlchemy
psycopg2
Djangojava
在1 的基礎上,重點突出本身之前沒注意的,作到精而不雜!!!python
Python是如何進行內存管理的c++
1).對象的引用計數機制git
Python內部使用引用計數,來保持追蹤內存中的對象,全部對象都有引用計數。github
引用計數增長的狀況:json
一個對象分配一個新名稱
將其放入一個容器中(如列表、元組或字典)
引用計數減小的狀況:api
使用del語句對對象別名顯示的銷燬
引用超出做用域或被從新賦值
2).垃圾回收緩存
當一個對象的引用計數歸零時,它將被垃圾收集機制處理掉。安全
3).內存池機制
Python提供了對內存的垃圾收集機制,可是它將不用的內存放到內存池而不是返回給操做系統:
Pymalloc機制:爲了加速Python的執行效率,Python引入了一個內存池機制,用於管理對小塊內存的申請和釋放。
對於Python對象,如整數,浮點數和List,都有其獨立的私有內存池,對象間不共享他們的內存池。也就是說若是你分配又釋放了大量的整數,用於緩存這些整數的內存就不能再分配給浮點數。
from io import StringIO def out_str(): return 'out\nstring\nhello\nworld' s = StringIO(out_str()) s.getvalue() s.seek(0,2) s.write('haha') s.getvalue() # 若是去掉 s.seek(0,2) 則意味着默認指針從 0 位置開始寫入,那麼haha將替換掉out\n 四個字符長度 # 因此這裏 seek , tell 是隱藏的倆有用的玩意兒, s.write() 調用完指針在當前位置即最後一個字符,StringIO(str) 這種指針在 0 處 ,改用seek時候用seek
# StringIO和BytesIO # stringIO 好比說,這時候,你須要對獲取到的數據進行操做,可是你並不想把數據寫到本地硬盤上,這時候你就能夠用stringIO from io import StringIO from io import BytesIO def outputstring(): return 'string \nfrom \noutputstring \nfunction' s = outputstring() # 將函數返回的數據在內存中讀 sio = StringIO(s) # 能夠用StringIO自己的方法 print(sio.getvalue()) # 也能夠用file-like object的方法 s = sio.readlines() for i in s: print(i.strip()) print('*'*100) # 將函數返回的數據在內存中寫 sio = StringIO() sio.write(''.join(s)) # 能夠用StringIO自己的方法查看 s=sio.getvalue() print(s) print('*'*100) # 若是你用file-like object的方法查看的時候,你會發現數據爲空 sio = StringIO() sio.write(s) # write 會把指針丟往最後 for i in sio.readlines(): print(i.strip()) print('*'*100) # 這時候咱們須要修改下文件的指針位置 # 咱們發現能夠打印出內容了 sio = StringIO() sio.write(s) sio.seek(0,0) print(sio.tell()) for i in sio.readlines(): print(i.strip()) # 這就涉及到了兩個方法seek 和 tell # tell 方法獲取當前文件讀取指針的位置 # seek 方法,用於移動文件讀寫指針到指定位置,有兩個參數,第一個offset: 偏移量,須要向前或向後的字節數,正爲向後,負爲向前;第二個whence: 可選值,默認爲0,表示文件開頭,1表示相對於當前的位置,2表示文件末尾 # 用seek方法時,需注意,若是你打開的文件沒有用'b'的方式打開,則offset沒法使用負值哦 print('*'*100) # stringIO 只能操做str,若是要操做二進制數據,就須要用到BytesIO # 上面的sio沒法用seek從當前位置向前移動,這時候,咱們用'b'的方式寫入數據,就能夠向前移動了 bio = BytesIO() bio.write(s.encode('utf-8')) print(bio.getvalue()) bio.seek(-36,1) print(bio.tell()) for i in bio.readlines(): print(i.strip().decode('utf-8')) print(len('string \nfrom \noutputstring \nfunction'))
from __future__ import division import sys,time j = '#' for i in range(1,61): j += '#' sys.stdout.write(str(int((i/60)*100))+'% ||'+j+'->'+"\r") sys.stdout.flush() time.sleep(0.1)
動態加載一個目錄下的全部模塊 目錄: ---test ----a.py ----b.py ---c.py c.py導入test下面的全部模塊: for path in ["test"]: for i in list(set([os.path.splitext(i)[0] for i in os.listdir("./"+path)])): if i!="__init__" and i!=".DS_Store": ##排除沒必要要的文件 import_string = "import path+"."+i+" exec import_string #執行字符串中的內容
# 三方法去重 # 方法一利用 set 集合不重複特性 >>> list1 = [2,3,8,4,9,5,6] >>> list2 = [5,6,10,17,11,2] >>> set(sorted(list1)+sorted(list2)) {2, 3, 4, 5, 6, 8, 9, 10, 11, 17} # 方法二利用 dict 的 keys 不能重複的特色 >>> {}.fromkeys(sorted(list1)+sorted(list2)).keys() dict_keys([2, 3, 4, 5, 6, 8, 9, 10, 11, 17]) # 方法三,自定義函數判斷若是重複則刪除 >>> def distinct(lst): ... lst.sort() ... last = lst[-1] ... for i in range(len(lst)-2,-1,-1): ... if last == lst[i]: ... del lst[i] ... else: ... last = lst[i] ... return lst ... >>> distinct(list1+list2) [2, 3, 4, 5, 6, 8, 9, 10, 11, 17]
python set集合運算(交集,並集,差集,對稱差集)
1》交集
x={1,2,3,4}
y={3,4,5,6}
x
set([1, 2, 3, 4])
y
set([3, 4, 5, 6])
x&y
set([3, 4])
x.intersection(y)
set([3, 4])
2》並集
x | y #集合並集
set([1, 2, 3, 4, 5, 6])
x.union(y)
set([1, 2, 3, 4, 5, 6])
3》差集
x-y # x與y的差集
set([1, 2])
x.difference(y)# x與y的差集
set([1, 2])
y-x # y與x的差集
set([5, 6])
y.difference(x)# y與x的差集
set([5, 6])
4》對稱差集
x^y
set([1, 2, 5, 6])
y^x
set([1, 2, 5, 6])
x.symmetric_difference(y)
set([1, 2, 5, 6])
y.symmetric_difference(x)
set([1, 2, 5, 6])
5》集合的子集和超集
x
set([1, 2, 3, 4])
z
set([1, 2, 3])
z.issubset(x)#z是x的子集
True
x.issuperset(z)#x是z的超集
True
下面的圖片形象地展現了set集合的各類運算:
集合 x <==> ① + ②
集合 x <==> ② + ③
交集 x&6 <==> ②
並集 x|y <==> ① + ② + ③
差集 x-y <==> ①
差集 y-x <==> ③
對稱差集 x^y == y^x <==> ① + ③
# 本身想的一個 隨機生成一個 max 位數的 生成器,也許沒什麼用,可是既然想了就寫上吧。。。 >>> import random >>> def g_rand_range(max): ... for i in range(1,max+1,1): ... yield random.randint(1,10) ... >>> for i in g_rand_range(10): ... print(i)
#下面的代碼會不會報錯 list = ['a', 'b', 'c', 'd', 'e'] print(list[10:]) 答案是並不會。。 當試圖訪問一個超過列表索引值的成員將致使 IndexError(好比訪問以上列表的 list[10])。儘管如此,試圖訪問一個列表的以超出列表長度數做爲開始索引的切片將不會致使 IndexError,而且將僅僅返回一個空列表
# 猜猜下面執行後的結果: def extendList(val, list=[]): list.append(val) return list list1 = extendList(10) # list 爲可變類型 10 list2 = extendList(123,[]) # 123 list3 = extendList('a') # 10,'a' 而這個地方 list3 再次操做了 list1 時候的那個 列表 連帶 list1 都變成了 10,'a' print ("list1 = %s" % list1) print ("list2 = %s" % list2) print ("list3 = %s" % list3) list1 = [10, 'a'] list2 = [123] list3 = [10, 'a'] 爲了不這樣的問題, 能夠 採用如下作法,參數列表裏賦值爲None def extendList(val, list=None): if list is None: list = [] list.append(val) return list list1 = extendList(10) # list 爲可變類型 10 list2 = extendList(123,[]) # 123 list3 = extendList('a') # 10,'a' print ("list1 = %s" % list1) print ("list2 = %s" % list2) print ("list3 = %s" % list3) list1 = [10] list2 = [123] list3 = ['a']
>>>all([1<2,3>2]) True >>> any([1<2,3<2]) True >>>
# 反序一下 a = [0, 1, 2, 3, 4, 5, 7, 8, 9, 10] >>> ','.join(map(str,a[::-1]))
## 問題 @classmethod 與 @staticmethod 的區別,爲啥要用這些裝飾器 # 我看完舉例後的淺顯理解是 classmethod 解決了 c++ , java 中重載的問題, #@staticmethod 能夠不用建立對象直接類調用而且可以被其子類輕易繼承。。。 class Date(object): def __init__(self, day=0, month=0, year=0): self.day = day self.month = month self.year = year @classmethod def from_string(cls, date_as_string): day, month, year = map(int, date_as_string.split('-')) date1 = cls(day, month, year) return date1 @staticmethod def is_date_valid(date_as_string): day, month, year = map(int, date_as_string.split('-')) return day <= 31 and month <= 12 and year <= 3999 date2 = Date.from_string('11-09-2012') is_date = Date.is_date_valid('11-09-2012') # 改寫一下 加深理解 import re class Date(object): def __init__(self,day=0,month=0,year=0): self.day = day self.month = month self.year = year @staticmethod def split_date_string(delimeter,date_as_string): day,month,year = map(int,date_as_string.split(delimeter)) return day,month,year @staticmethod def is_valid_date(date_as_string,delimeter): r_string = r'\d{1,2}%s\d{1,2}%s\d{4}' %(delimeter,delimeter) print(r_string) if re.match(r_string,date_as_string): day,month,year = Date.split_date_string(delimeter,date_as_string) else: raise ValueError('不支持的日期格式') return day<=31 and month<=12 and year<=3999 @classmethod def from_string(cls,date_as_string,delimeter): if cls.is_valid_date(date_as_string,delimeter): day,month,year = cls.split_date_string(delimeter,date_as_string) return cls(day,month,year) def __str__(self): return '/'.join(list(map(str,[self.day,self.month,self.year]))) date2 = Date.from_string('01/09/2012','/') date3 = Date.from_string('01-09-2012','-') print(date2) print(date3)
#new 相似於 java 的懶漢式 單例,判斷是否有,沒有就建立 # 第一種方法 new 方法 class Singleton(object): def __new__(cls,*args,**kw): if not hasattr(cls,'_instance'): cls._instance = super(Singleton,cls).__new__(cls,*args,**kw) return cls._instance s1 = Singleton() s2 = Singleton() s1 == s2 class Singleton(object): def __new__(cls, *args, **kw): if not hasattr(cls, '_instance'): orig = super(Singleton, cls) cls._instance = orig.__new__(cls, *args, **kw) return cls._instance def __init__(self,*args,**kw): pass s1 = Singleton() s2 = Singleton() print(s1==s2) print(s1 is s2) print(type(s1),'--',type(s2)) # 裝飾器 ## 函數式裝飾器 def singleton(cls,*args,**kw): def get_instance(): instance = {} if cls not in instance: instance[cls] = cls(*args,**kw) return instance[cls] return get_instance @singleton class Class_Demo(object): def __new__(cls,*args,**kw): pass c1 = Class_Demo() c2 = Class_Demo() c1 is c2 ## 類裝飾器 # 共享屬性 class Borg(object): _state = {} def __new__(cls, *args, **kw): ob = super(Borg, cls).__new__(cls, *args, **kw) ob.__dict__ = cls._state return ob class MyClass(Borg): def __new__(cls,*args,**kw): pass c1 = MyClass() c2 = MyClass() c1 is c2
Python中9種生成新對象的方法 先定義一個類: class Point: def __init__(self, x, y): self.x = x self.y = y 下面咱們使用9種方法來生成新的對象: point1 = Point(1, 2) point2 = eval("{}({}, {})".format("Point", 1, 2)) point3 = globals()["Point"](1, 2) point4 = locals()["Point"](1, 2) point5 = getattr(sys.modules[__name__], "Point")(1, 2) point6 = copy.deepcopy(point1) point7 = point1.__class__(1, 2) point8 = type('Point', (Point, ), {})(1, 2) point9 = types.new_class('Point', (Point, ), {})(1, 2
閱讀下面的代碼,它的輸出結果是什麼? class Node(object): def __init__(self,sName): self._lChildren = [] self.sName = sName def __repr__(self): return "<Node '{}'>".format(self.sName) def append(self,*args,**kwargs): self._lChildren.append(*args,**kwargs) def print_all_1(self): print self for oChild in self._lChildren: oChild.print_all_1() def print_all_2(self): def gen(o): lAll = [o,] while lAll: oNext = lAll.pop(0) lAll.extend(oNext._lChildren) yield oNext for oNode in gen(self): print oNode oRoot = Node("root") oChild1 = Node("child1") oChild2 = Node("child2") oChild3 = Node("child3") oChild4 = Node("child4") oChild5 = Node("child5") oChild6 = Node("child6") oChild7 = Node("child7") oChild8 = Node("child8") oChild9 = Node("child9") oChild10 = Node("child10") oRoot.append(oChild1) oRoot.append(oChild2) oRoot.append(oChild3) oChild1.append(oChild4) oChild1.append(oChild5) oChild2.append(oChild6) oChild4.append(oChild7) oChild3.append(oChild8) oChild3.append(oChild9) oChild6.append(oChild10) # 說明下面代碼的輸出結果 oRoot.print_all_1() oRoot.print_all_2() 答案 oRoot.print_all_1()會打印下面的結果: <Node 'root'> <Node 'child1'> <Node 'child4'> <Node 'child7'> <Node 'child5'> <Node 'child2'> <Node 'child6'> <Node 'child10'> <Node 'child3'> <Node 'child8'> <Node 'child9'> oRoot.print_all_1()會打印下面的結果: <Node 'root'> <Node 'child1'> <Node 'child2'> <Node 'child3'> <Node 'child4'> <Node 'child5'> <Node 'child6'> <Node 'child8'> <Node 'child9'> <Node 'child7'> <Node 'child10'> 爲何提這個問題? 由於對象的精髓就在於組合(composition)與對象構造(object construction)。對象須要有組合成分構成,並且得以某種方式初始化。這裏也涉及到遞歸和生成器(generator)的使用。 生成器是很棒的數據類型。你能夠只經過構造一個很長的列表,而後打印列表的內容,就能夠取得與print_all_2相似的功能。生成器還有一個好處,就是不用佔據不少內存。 有一點還值得指出,就是print_all_1會以深度優先(depth-first)的方式遍歷樹(tree),而print_all_2則是寬度優先(width-first)。有時候,一種遍歷方式比另外一種更合適。但這要看你的應用的具體狀況。
# 零寬斷言,舉例, (?=),(?<=),(?!),(?<!) import re s = 'abc123ebo456' # re.findall(r'[a-z]+(?=\d+)',s) # ['abc', 'ebo'] 先行零寬斷言 後面是 數字,findall 取出 字母部分 # re.findall(r'(?<=[a-z])\d+',s) # ['123', '456'] 後發零寬斷言 前面是 字母, findall 取出 \d+ 部分 # re.findall(r'[a-z]+(?!c)',s) # ['abc', 'ebo']負向零寬先行斷言 後面不是 c 這個 c 隨便取的,能夠爲 a只要不是數字 # re.findall(r'(?<!\d)\d+',s) # 負向零寬後發斷言 ['123', '456']
好比要匹配2個一樣的字符 '(\w)\1',\1是對(\w)的引用。 這樣有個弊端,就是分組不少,並且有嵌套的時候,很難分清\n引用的是哪一個分組。 用命名分組就直觀多了。 '(?P<word>\w)(?P=word)' import re s = 'abc1334oioj33435lljlj45' re.findall(r'(?P<repeatdigit>\d+).*?(?P=repeatdigit)',s) # ['334', '5']
import re s = 'abc' re.findall(r'(?:a)(.*)',s) # ['bc']
lock = threading.Lock() with lock: print('Citical part...do something...')
# -*- coding: utf-8 -*- def func_cache(func): cache = {} def inner_deco(*args): if args in cache: print('func {} is already cached with arguments {}'.format( func.__name__, args)) return cache[args] else: print('func {} is not cached with arguments {}'.format( func.__name__, args)) res = func(*args) cache[args] = res return res return inner_deco @func_cache def add_two_number(a, b): return a + b if __name__ == "__main__": print('1. add_two_number(1, 2)') add_two_number(1, 2) print('2. add_two_number(2, 3)') add_two_number(2, 3) print('3. add_two_number(1, 2)') add_two_number(1, 2)
import re from collections import Counter words_list = [] with open('/home/ml/work/notebook/Frank Li/words.txt','r') as fr: for line in fr: words_array = line.strip().split(" ") for w in words_array: words_list.append(w) words_list.remove('') Counter(words_list) {w:words_list.count(w) for w in words_list} >>> from collections import Counter >>> list_1 = [1,2,3,1,2,4,5,6,7,4,8,9] >>> [key for key,cnt in Counter(list_1).items() if cnt>1] [1, 2, 4]
import re from collections import Counter words_list = [] with open('/home/ml/work/notebook/Frank Li/words.txt','r') as fr: for line in fr: words_array = line.strip().split(" ") words_list.extend(words_array) # for w in words_array: # words_list.append(w) def flattern(words_lst): for lst in words_lst: if isinstance(lst,list): tmp = flattern(lst) for i in tmp: yield i else: yield lst words_list = list(flattern(words_list)) words_list.remove('') Counter(words_list) {w:words_list.count(w) for w in words_list}
BaseException +-- SystemExit +-- KeyboardInterrupt +-- GeneratorExit +-- Exception +-- StopIteration +-- StandardError | +-- BufferError | +-- ArithmeticError | | +-- FloatingPointError | | +-- OverflowError | | +-- ZeroDivisionError | +-- AssertionError | +-- AttributeError | +-- EnvironmentError | | +-- IOError | | +-- OSError | | +-- WindowsError (Windows) | | +-- VMSError (VMS) | +-- EOFError | +-- ImportError | +-- LookupError | | +-- IndexError | | +-- KeyError | +-- MemoryError | +-- NameError | | +-- UnboundLocalError | +-- ReferenceError | +-- RuntimeError | | +-- NotImplementedError | +-- SyntaxError | | +-- IndentationError | | +-- TabError | +-- SystemError | +-- TypeError | +-- ValueError | +-- UnicodeError | +-- UnicodeDecodeError | +-- UnicodeEncodeError | +-- UnicodeTranslateError +-- Warning +-- DeprecationWarning +-- PendingDeprecationWarning +-- RuntimeWarning +-- SyntaxWarning +-- UserWarning +-- FutureWarning +-- ImportWarning +-- UnicodeWarning +-- BytesWarning
# -*- coding: utf-8 -*- """ requests.api ~~~~~~~~~~~~ This module implements the Requests API. :copyright: (c) 2012 by Kenneth Reitz. :license: Apache2, see LICENSE for more details. """ from . import sessions def request(method, url, **kwargs): """Constructs and sends a :class:`Request <Request>`. :param method: method for the new :class:`Request` object. :param url: URL for the new :class:`Request` object. :param params: (optional) Dictionary, list of tuples or bytes to send in the body of the :class:`Request`. :param data: (optional) Dictionary, list of tuples, bytes, or file-like object to send in the body of the :class:`Request`. :param json: (optional) A JSON serializable Python object to send in the body of the :class:`Request`. :param headers: (optional) Dictionary of HTTP Headers to send with the :class:`Request`. :param cookies: (optional) Dict or CookieJar object to send with the :class:`Request`. :param files: (optional) Dictionary of ``'name': file-like-objects`` (or ``{'name': file-tuple}``) for multipart encoding upload. ``file-tuple`` can be a 2-tuple ``('filename', fileobj)``, 3-tuple ``('filename', fileobj, 'content_type')`` or a 4-tuple ``('filename', fileobj, 'content_type', custom_headers)``, where ``'content-type'`` is a string defining the content type of the given file and ``custom_headers`` a dict-like object containing additional headers to add for the file. :param auth: (optional) Auth tuple to enable Basic/Digest/Custom HTTP Auth. :param timeout: (optional) How many seconds to wait for the server to send data before giving up, as a float, or a :ref:`(connect timeout, read timeout) <timeouts>` tuple. :type timeout: float or tuple :param allow_redirects: (optional) Boolean. Enable/disable GET/OPTIONS/POST/PUT/PATCH/DELETE/HEAD redirection. Defaults to ``True``. :type allow_redirects: bool :param proxies: (optional) Dictionary mapping protocol to the URL of the proxy. :param verify: (optional) Either a boolean, in which case it controls whether we verify the server's TLS certificate, or a string, in which case it must be a path to a CA bundle to use. Defaults to ``True``. :param stream: (optional) if ``False``, the response content will be immediately downloaded. :param cert: (optional) if String, path to ssl client cert file (.pem). If Tuple, ('cert', 'key') pair. :return: :class:`Response <Response>` object :rtype: requests.Response Usage:: >>> import requests >>> req = requests.request('GET', 'http://httpbin.org/get') <Response [200]> """ # By using the 'with' statement we are sure the session is closed, thus we # avoid leaving sockets open which can trigger a ResourceWarning in some # cases, and look like a memory leak in others. with sessions.Session() as session: return session.request(method=method, url=url, **kwargs) def get(url, params=None, **kwargs): r"""Sends a GET request. :param url: URL for the new :class:`Request` object. :param params: (optional) Dictionary, list of tuples or bytes to send in the body of the :class:`Request`. :param \*\*kwargs: Optional arguments that ``request`` takes. :return: :class:`Response <Response>` object :rtype: requests.Response """ kwargs.setdefault('allow_redirects', True) return request('get', url, params=params, **kwargs) def options(url, **kwargs): r"""Sends an OPTIONS request. :param url: URL for the new :class:`Request` object. :param \*\*kwargs: Optional arguments that ``request`` takes. :return: :class:`Response <Response>` object :rtype: requests.Response """ kwargs.setdefault('allow_redirects', True) return request('options', url, **kwargs) def head(url, **kwargs): r"""Sends a HEAD request. :param url: URL for the new :class:`Request` object. :param \*\*kwargs: Optional arguments that ``request`` takes. :return: :class:`Response <Response>` object :rtype: requests.Response """ kwargs.setdefault('allow_redirects', False) return request('head', url, **kwargs) def post(url, data=None, json=None, **kwargs): r"""Sends a POST request. :param url: URL for the new :class:`Request` object. :param data: (optional) Dictionary, list of tuples, bytes, or file-like object to send in the body of the :class:`Request`. :param json: (optional) json data to send in the body of the :class:`Request`. :param \*\*kwargs: Optional arguments that ``request`` takes. :return: :class:`Response <Response>` object :rtype: requests.Response """ return request('post', url, data=data, json=json, **kwargs) def put(url, data=None, **kwargs): r"""Sends a PUT request. :param url: URL for the new :class:`Request` object. :param data: (optional) Dictionary, list of tuples, bytes, or file-like object to send in the body of the :class:`Request`. :param json: (optional) json data to send in the body of the :class:`Request`. :param \*\*kwargs: Optional arguments that ``request`` takes. :return: :class:`Response <Response>` object :rtype: requests.Response """ return request('put', url, data=data, **kwargs) def patch(url, data=None, **kwargs): r"""Sends a PATCH request. :param url: URL for the new :class:`Request` object. :param data: (optional) Dictionary, list of tuples, bytes, or file-like object to send in the body of the :class:`Request`. :param json: (optional) json data to send in the body of the :class:`Request`. :param \*\*kwargs: Optional arguments that ``request`` takes. :return: :class:`Response <Response>` object :rtype: requests.Response """ return request('patch', url, data=data, **kwargs) def delete(url, **kwargs): r"""Sends a DELETE request. :param url: URL for the new :class:`Request` object. :param \*\*kwargs: Optional arguments that ``request`` takes. :return: :class:`Response <Response>` object :rtype: requests.Response """ return request('delete', url, **kwargs)