看我打臉Message Pack

總之網站上說得很好啦:http://msgpack.org/,又快,生成的文件又小等等,總之就是很牛逼的救世主咯python

由於忍受不了 pickle 的速度,因此嘗試切換一下。less

import pickle
import msgpack
import gc

a = list(range(10000*10000))

def dump1():
    with open('1', 'wb') as f:
        pickle.dump(a, f)

def dump2():
    with open('2', 'wb') as f:
        msgpack.dump(a, f)

def dump3():
    gc.disable()
    with open('2', 'wb') as f:
        msgpack.dump(a, f)
    gc.enable()

而後測速,有 dump3 的緣由是官方網站說關閉gc有奇效:oop

CPython's GC starts when growing allocated object. This means unpacking may cause useless GC. You can use gc.disable() when unpacking large message.測試

恩,先看測試結果(測試了10次):網站

dump1: 1 loop, best of 3: 3.98 s per loop
dump2: 1 loop, best of 10: 5.22 s per loop
dump3: 1 loop, best of 10: 5.13 s per loop

恩,完全打臉了。關閉gc也沒有奇效。code

而後看dump出來的文件 (=,=b):get

-rw-r--r--  1 chendx  staff   477M  8 11 18:43 1
-rw-r--r--  1 chendx  staff   477M  8 11 18:49 2

恩,浪費個人時間……it


而後不死心class

def load1():
    with open('1', 'rb') as f:
        pickle.load(f)

def load2():
    with open('2', 'rb') as f:
        msgpack.load(f)

結果(仍是10次)import

load1: 1 loop, best of 10: 6.78 s per loop
load2: 1 loop, best of 10: 5.91 s per loop

恩,真是雞肋啊

相關文章
相關標籤/搜索