總之網站上說得很好啦:http://msgpack.org/,又快,生成的文件又小等等,總之就是很牛逼的救世主咯python
由於忍受不了 pickle 的速度,因此嘗試切換一下。less
import pickle import msgpack import gc a = list(range(10000*10000)) def dump1(): with open('1', 'wb') as f: pickle.dump(a, f) def dump2(): with open('2', 'wb') as f: msgpack.dump(a, f) def dump3(): gc.disable() with open('2', 'wb') as f: msgpack.dump(a, f) gc.enable()
而後測速,有 dump3
的緣由是官方網站說關閉gc有奇效:oop
CPython's GC starts when growing allocated object. This means unpacking may cause useless GC. You can use gc.disable() when unpacking large message.測試
恩,先看測試結果(測試了10次):網站
dump1: 1 loop, best of 3: 3.98 s per loop dump2: 1 loop, best of 10: 5.22 s per loop dump3: 1 loop, best of 10: 5.13 s per loop
恩,完全打臉了。關閉gc也沒有奇效。code
而後看dump出來的文件 (=,=b):get
-rw-r--r-- 1 chendx staff 477M 8 11 18:43 1 -rw-r--r-- 1 chendx staff 477M 8 11 18:49 2
恩,浪費個人時間……it
而後不死心class
def load1(): with open('1', 'rb') as f: pickle.load(f) def load2(): with open('2', 'rb') as f: msgpack.load(f)
結果(仍是10次)import
load1: 1 loop, best of 10: 6.78 s per loop load2: 1 loop, best of 10: 5.91 s per loop
恩,真是雞肋啊