python 字符編碼判斷 chardet評測

 

以前一直想找到一個模塊，針對字符判斷是什麼字符集編碼的庫

網上有chardet的blog，發現本身的環境有這個庫，因而就作了測試

>>> import chardet
>>> a = "也有".decode('gbk')
>>> a
u'\u6d94\u71b8\u6e41'
>>> a = "也有".decode('gbk').encode('gbk')
>>> a
'\xe4\xb9\x9f\xe6\x9c\x89'
>>> chardet.detect(a)
{'confidence': 0.7525, 'language': '', 'encoding': 'utf-8'}
>>> print(a)
也有>>> a = "也有".decode('gbk').encode('gbk')
>>> a = u"也有".encode('gbk')
>>> a
'\xd2\xb2\xd3\xd0'
>>> chardet.detect(a)
{'confidence': 0.7679697235616183, 'language': 'Russian', 'encoding': 'KOI8-R'}

In [5]: a = "asd算法".encode('utf-8').decode('gbk')

In [6]: import chardet

In [7]: chardet.detect(a)
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
<ipython-input-7-45a625c5d441> in <module>()
----> 1 chardet.detect(a)

c:\python3.6.3\lib\site-packages\chardet\__init__.py in detect(byte_str)
32 if not isinstance(byte_str, bytes):
33 raise TypeError('Expected object of type bytes or bytearray, got: '
---> 34 '{0}'.format(type(byte_str)))
35 else:
36 byte_str = bytearray(byte_str)

TypeError: Expected object of type bytes or bytearray, got: <class 'str'>

In [8]: a
Out[8]: 'asd綆楁硶'

 

總結髮現不是很準，放棄使用