Redis 鏈接編碼問題

Redis 鏈接編碼問題

不編碼狀況下，UnicodeDecodeError

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/python3/lib/python3.4/site-packages/redis/client.py", line 863, in get
    return self.execute_command('GET', name)
  File "/usr/local/python3/lib/python3.4/site-packages/redis_py_cluster-1.0.0-py3.4.egg/rediscluster/utils.py", line 85, in inner
    _keys = set(sum([list(obj.keys()) for obj in objs],[]))  
  File "/usr/local/python3/lib/python3.4/site-packages/redis_py_cluster-1.0.0-py3.4.egg/rediscluster/client.py", line 293, in execute_command
  File "/usr/local/python3/lib/python3.4/site-packages/redis/client.py", line 577, in parse_response
    response = connection.read_response()
  File "/usr/local/python3/lib/python3.4/site-packages/redis/connection.py", line 569, in read_response
    response = self._parser.read_response()
  File "/usr/local/python3/lib/python3.4/site-packages/redis/connection.py", line 266, in read_response
    response = response.decode()
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x8b in position 1: invalid start byte
複製代碼

代碼

>>> from redis import StrictRedis
>>> r = StrictRedis('localhost', 6380)
>>> r.set('hello', '你好')
True
>>> r.get('hello')
b'\xe4\xbd\xa0\xe5\xa5\xbd'
複製代碼

能夠看見，字符串輸入被編碼成utf8存儲在Redis裏了。而取出來的時候仍是被編碼後的bytes，須要顯示的decode才能變成字符串。

解決方案

Redis創建鏈接時有兩個參數，一個是encoding指定編碼，默認是utf8。一個是decode_responses，默認爲False，若是是True，會以encoding方式解碼，而後返回字符串。若是是字符串，就根據encoding編碼成bytes。

>>> from redis import StrictRedis
>>> r = StrictRedis('localhost', 6380, encoding='utf8', decode_responses=True)
>>> r.set('hello', '你好')
True
>>> r.get('hello')
'你好'
複製代碼

壓縮 (gzip, zlib, lz4) 狀況下，UnicodeDecodeError

依然報上面的錯

解決方案

decode_responses 設爲 False

鏈接 redis 時帶上 decode_responses=False 取消對返回結果的 decode('utf-8') 操做

>>import redis
>>import gzip
>>rdb = redis.Redis(host='127.0.0.1',6379,db=0,decode_responses=False)
>>rdb.set('test.zip',gzip.compress(b'xxxxxxxxxxxxx'))
>>rdb.get('test.zip')
b'\x1f\x8b\x08\x00\xd5;\xe1V\x02\xff\xab\xa8@\x02\x00\xd6\x82q\xe4\r\x00\x00\x00'
複製代碼

帶來的弊端的返回的結果是byte類型，要自行decode

網上還有人推薦修改源碼，我沒嘗試過，直接粘貼過來的 ，但也是一種方案嘛。

位置在 python安裝目錄/lib/python3.4/site-packages/redis/connection.py 266行 注：能夠先將egg包解壓，而後將egg包更個名 如此未用gzip壓縮的返回的是utf-8，而通過gzip壓縮過的返回的是byte

修改 redis 源碼

elif byte == '+':
            pass
        # int value
        elif byte == ':':
            response = long(response)
        # bulk response
        elif byte == '$':
            length = int(response)
            if length == -1:
                return None
            response = self._buffer.read(length)
        # multi-bulk response
        elif byte == '*':
            length = int(response)
            if length == -1:
                return None
            response = [self.read_response() for i in xrange(length)]
        if isinstance(response, bytes) and self.encoding:
            #此處加個異常處理就能夠了
            try:
                response = response.decode()
            except :
                pass
        return response
複製代碼

結果

>>import redis
>>import gzip
>>rdb = redis.Redis(host='127.0.0.1',6379,db=0,decode_responses=False)
>>rdb.set('test.zip',gzip.compress(b'xxxxxxxxxxxxx'))
>>rdb.get('test.zip')
b'\x1f\x8b\x08\x00\xd5;\xe1V\x02\xff\xab\xa8@\x02\x00\xd6\x82q\xe4\r\x00\x00\x00'
>>rdb.set('test.str','xxxx')
>>rdb.get('test.str')
'xxxx'
複製代碼