python 2/3 joblib.dump() 和 joblib.load()

在python2中加載python3訓練和保存的模型時出錯：


ValueErrorTraceback (most recent call last)
--> 237 clf = joblib.load('clf300_all.model')
    238 pred_y = clf.predict_proba(X)

/usr/local/anaconda2/lib/python2.7/site-packages/sklearn/externals/joblib/numpy_pickle.pyc in load(filename, mmap_mode)
    576                     return load_compatibility(fobj)
    577 
--> 578                 obj = _unpickle(fobj, filename, mmap_mode)
    579 
    580     return obj

/usr/local/anaconda2/lib/python2.7/site-packages/sklearn/externals/joblib/numpy_pickle.pyc in _unpickle(fobj, filename, mmap_mode)


ValueError: unsupported pickle protocol: 3

　　

通過查閱資料：

跨python版本的 joblib.dump() 和 joblib.load() 

Compatibility across python versions

Compatibility of joblib pickles across python versions is not fully supported. Note that, for a very restricted set of objects, this may appear to work when saving a pickle with python 2 and loading it with python 3 but relying on it is strongly discouraged.

If you are switching between python versions, you will need to save a different joblib pickle for each python version.

Here are a few examples or exceptions:

• Saving joblib pickle with python 2, trying to load it with python 3:

  Traceback (most recent call last): File "/home/lesteve/dev/joblib/joblib/numpy_pickle.py", line 453, in load obj = unpickler.load() File "/home/lesteve/miniconda3/lib/python3.4/pickle.py", line 1038, in load dispatch[key[0]](self) File "/home/lesteve/miniconda3/lib/python3.4/pickle.py", line 1176, in load_binstring self.append(self._decode_string(data)) File "/home/lesteve/miniconda3/lib/python3.4/pickle.py", line 1158, in _decode_string return value.decode(self.encoding, self.errors) UnicodeDecodeError: 'ascii' codec can't decode byte 0x80 in position 1024: ordinal not in range(128) Traceback (most recent call last): File "<string>", line 1, in <module> File "/home/lesteve/dev/joblib/joblib/numpy_pickle.py", line 462, in load raise new_exc ValueError: You may be trying to read with python 3 a joblib pickle generated with python 2. This is not feature supported by joblib. 

• Saving joblib pickle with python 3, trying to load it with python 2:

  Traceback (most recent call last): File "<string>", line 1, in <module> File "joblib/numpy_pickle.py", line 453, in load obj = unpickler.load() File "/home/lesteve/miniconda3/envs/py27/lib/python2.7/pickle.py", line 858, in load dispatch[key](self) File "/home/lesteve/miniconda3/envs/py27/lib/python2.7/pickle.py", line 886, in load_proto raise ValueError, "unsupported pickle protocol: %d" % proto ValueError: unsupported pickle protocol: 3
  
  =================================================================================================================================================

  不徹底支持跨python版本的joblib pickle的兼容性。請注意，對於一組很是有限的對象，當使用python 2保存pickle並使用python 3加載它時，這可能會起做用，但強烈建議不要依賴它。 若是要在python版本之間切換，則須要爲每一個python版本保存不一樣的joblib pickle。
  ==================================================================================================================================================
  另外：不一樣python版本的pickle.dump()和pickle.load()是能夠相互轉換和支持的
  

  You should write the pickled data with a lower protocol number in Python 3. Python 3 introduced a new protocol with the number 3 (and uses it as default), so switch back to a value of 2 which can be read by Python 2.

  Check the protocolparameter in pickle.dump. Your resulting code will look like this.

  pickle.dump(your_object, your_file, protocol=2)

  There is no protocolparameter in pickle.load because pickle can determine the protocol from the file.

   

  Pickle uses different protocols to convert your data to a binary stream.

  • In python 2 there are 3 different protocols (0, 1, 2) and the default is 0.

  • In python 3 there are 5 different protocols (0, 1, 2, 3, 4) and the default is 3.

  You must specify in python 3 a protocol lower than 3 in order to be able to load the data in python 2. You can specify the protocol parameter when invoking pickle.dump.