使用Java讀取「Python寫入redis」的數據踩坑記錄

時間 2019-11-13

標籤使用 java 讀取 python 寫入 redis 數據記錄欄目 Java 简体版

原文原文鏈接

前置環境聲明：

java8 + spring-boot-2.0.0.RELEASE + spring-boot-starter-data-redis（boot集成）java

python 3.7 64位python

redis版本4.0.1redis

在這個python腳本中寫入redis的數據是3個Student對象spring

代碼以下json

第一部分：python寫入

import redis
import json

redisOperator = redis.Redis(host='localhost', port=6379,password="666666")


class Student(object):
    def __init__(self, name, age, sex):
        self.name = name
        self.age = age
        self.sex = sex


student1  = Student("chenjun1", 18, "男")
student2  = Student("chenjun2", 19, "男")
student3  = Student("chenjun3", 20, "男")

studentList = []
studentList.append(student1)
studentList.append(student2)
studentList.append(student3)


print(json.dumps(student1.__dict__, ensure_ascii=False))
print(json.dumps(student2.__dict__, ensure_ascii=False))
print(json.dumps(student3.__dict__, ensure_ascii=False))



for u in studentList:
    redisOperator.hset("ThreeCodeInOne",u.name, json.dumps(u.__dict__, ensure_ascii=False))

第二部分，java讀取：

@SpringBootApplication
public class Application implements ApplicationContextAware {

	private static ApplicationContext applicationContext;

	@Override
	public void setApplicationContext(ApplicationContext applicationContext) throws BeansException {
		Application.applicationContext = applicationContext;
	}

	@SuppressWarnings("unchecked")
	public static <T> T getBean(String beanName) {
		assertApplicationContext();
		return (T) applicationContext.getBean(beanName);
	}

	public static <T> T getBean(Class<T> requiredType) {
		assertApplicationContext();
		return applicationContext.getBean(requiredType);
	}

	private static void assertApplicationContext() {
		if (Application.applicationContext == null) {
			throw new RuntimeException("applicaitonContext屬性爲null,請檢查是否注入了SpringContext!");
		}
	}

	public static void getStudents() {
		System.out.println("*************getStudentHash*************");
		RedisTemplate<String, String> redisTemplate  = getBean("redisTemplate");
		 redisTemplate.setValueSerializer(new GenericToStringSerializer<>(Object.class, Charset.forName("UTF-8")));
		 Object o = redisTemplate.opsForHash().get("ThreeCodeInOne", "chenjun1");
		System.out.println(o);
	}

    public static void main(String[] args) {
			SpringApplication.run(Application.class, args);
			assertApplicationContext();
            getStudents();
    }

}

然而，這樣是錯誤的！ 數組

然而，這樣是錯誤的！ app

然而，這樣是錯誤的！ dom

報錯信息：ide

*************getStudentHash*************
Exception in thread "main" 2019-06-13 11:10:03.813  INFO 7064 --- [       Thread-2] s.c.a.AnnotationConfigApplicationContext : Closing org.springframework.context.annotation.AnnotationConfigApplicationContext@51931956: startup date [Thu Jun 13 11:10:01 CST 2019]; root of context hierarchy
org.springframework.data.redis.serializer.SerializationException: Could not read JSON: Unexpected token (START_OBJECT), expected START_ARRAY: need JSON Array to contain As.WRAPPER_ARRAY type information for class java.lang.Object
 at [Source: (byte[])"{"name": "chenjun1", "age": 18, "sex": "男"}"; line: 1, column: 1]; nested exception is com.fasterxml.jackson.databind.exc.MismatchedInputException: Unexpected token (START_OBJECT), expected START_ARRAY: need JSON Array to contain As.WRAPPER_ARRAY type information for class java.lang.Object
 at [Source: (byte[])"{"name": "chenjun1", "age": 18, "sex": "男"}"; line: 1, column: 1]
	at org.springframework.data.redis.serializer.Jackson2JsonRedisSerializer.deserialize(Jackson2JsonRedisSerializer.java:75)
	at org.springframework.data.redis.core.AbstractOperations.deserializeHashValue(AbstractOperations.java:354)
	at org.springframework.data.redis.core.DefaultHashOperations.get(DefaultHashOperations.java:54)
	at com.redisexample.Application.getStudents(Application.java:139)
	at com.redisexample.Application.main(Application.java:97)
Caused by: com.fasterxml.jackson.databind.exc.MismatchedInputException: Unexpected token (START_OBJECT), expected START_ARRAY: need JSON Array to contain As.WRAPPER_ARRAY type information for class java.lang.Object
 at [Source: (byte[])"{"name": "chenjun1", "age": 18, "sex": "男"}"; line: 1, column: 1]
	at com.fasterxml.jackson.databind.exc.MismatchedInputException.from(MismatchedInputException.java:59)
	at com.fasterxml.jackson.databind.DeserializationContext.wrongTokenException(DeserializationContext.java:1498)
	at com.fasterxml.jackson.databind.DeserializationContext.reportWrongTokenException(DeserializationContext.java:1273)
	at com.fasterxml.jackson.databind.jsontype.impl.AsArrayTypeDeserializer._locateTypeId(AsArrayTypeDeserializer.java:137)
	at com.fasterxml.jackson.databind.jsontype.impl.AsArrayTypeDeserializer._deserialize(AsArrayTypeDeserializer.java:96)
	at com.fasterxml.jackson.databind.jsontype.impl.AsArrayTypeDeserializer.deserializeTypedFromAny(AsArrayTypeDeserializer.java:71)
	at com.fasterxml.jackson.databind.deser.std.UntypedObjectDeserializer$Vanilla.deserializeWithType(UntypedObjectDeserializer.java:712)
	at com.fasterxml.jackson.databind.deser.impl.TypeWrappedDeserializer.deserialize(TypeWrappedDeserializer.java:68)
	at com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:4001)
	at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3117)
	at org.springframework.data.redis.serializer.Jackson2JsonRedisSerializer.deserialize(Jackson2JsonRedisSerializer.java:73)
	... 4 more
2019-06-13 11:10:03.815  INFO 7064 --- [       Thread-2] o.s.j.e.a.AnnotationMBeanExporter        : Unregistering JMX-exposed beans on shutdown

這是爲何呢？ spring-boot

我打開redis控制檯看一下 :

發現寫入進去的是這樣的

看似沒錯，可是爲何使用java讀取就報錯了呢？

第三部分：排查問題

這時候排查問題的思路是：咱們來使用java寫入，而且使用java讀取，總不會出問題了吧？

咱們修改上述主類的方法以下:

public static void getStudents() {
		System.out.println("*************getStudentHash*************");
		RedisTemplate<String, String> redisTemplate  = getBean("redisTemplate");
		Student student = new Student("chenjun1", "18", "男");
		redisTemplate.opsForHash().put("ThreeCodeInOne", student.getName(), student);
		 Object o = redisTemplate.opsForHash().get("ThreeCodeInOne", "chenjun1");
		System.out.println(o);
	}

執行結果：

*************getStudentHash*************
Student [name=chenjun1, age=18, sex=男]
2019-06-13 11:16:03.877  INFO 188 --- [       Thread-2] s.c.a.AnnotationConfigApplicationContext : Closing org.springframework.context.annotation.AnnotationConfigApplicationContext@61d47554: startup date [Thu Jun 13 11:16:01 CST 2019]; root of context hierarchy
2019-06-13 11:16:03.880  INFO 188 --- [       Thread-2] o.s.j.e.a.AnnotationMBeanExporter        : Unregistering JMX-exposed beans on shutdown

這時候，再打開一下redis-cli控制檯看下存進去的是什麼格式

發現寫進去的是這樣的：

原來這和python寫進去的格式不同啊，那麼確定是各個語言本身的序列化機制的問題了

那麼大膽猜測一下啊，若是我使用python寫入的時候，拼湊成這種帶包名+類名的格式，是否是就能夠正常讀取了呢？

來仔細對比一下二者的差異吧：

後者竟然是一個json數組，那咱們在python代碼中試着拼湊一下，

我修改python代碼以下：

import redis
import json

redisOperator = redis.Redis(host='localhost', port=6379,password="666666")


class Student(object):
    def __init__(self, name, age, sex):
        self.name = name
        self.age = age
        self.sex = sex

def useJavaSerial(student):
    list = []
    list.append("com.redisexample.domain.Student")
    list.append(student.__dict__)
    return json.dumps(list,ensure_ascii=False)

student1  = Student("chenjun1", 18, "男")
student2  = Student("chenjun2", 19, "男")
student3  = Student("chenjun3", 20, "男")

studentList = []
studentList.append(student1)
studentList.append(student2)
studentList.append(student3)


print(json.dumps(student1.__dict__, ensure_ascii=False))
print(json.dumps(student2.__dict__, ensure_ascii=False))
print(json.dumps(student3.__dict__, ensure_ascii=False))



for u in studentList:
    redisOperator.hset("ThreeCodeInOne",u.name, useJavaSerial(u))

打開控制檯檢驗一下，看看寫進去的是什麼樣的

結構和java寫進去的如出一轍，完美

那接着來用java程序讀一下：

public static void getStudents() {
		System.out.println("*************getStudentHash*************");
		RedisTemplate<String, String> redisTemplate  = getBean("redisTemplate");
		 Object o = redisTemplate.opsForHash().get("ThreeCodeInOne", "chenjun1");
		System.out.println(o);
	}

輸出：

*************getStudentHash*************
Student [name=chenjun1, age=18, sex=男]
2019-06-13 11:28:19.852  INFO 15932 --- [       Thread-2] s.c.a.AnnotationConfigApplicationContext : Closing org.springframework.context.annotation.AnnotationConfigApplicationContext@564718df: startup date [Thu Jun 13 11:28:17 CST 2019]; root of context hierarchy
2019-06-13 11:28:19.853  INFO 15932 --- [       Thread-2] o.s.j.e.a.AnnotationMBeanExporter        : Unregistering JMX-exposed beans on shutdown

猜測成功驗證，結論：是不一樣語言序列化機制的差別致使的讀取和寫入形成格式不兼容

補充備註：

Spring-data-redis的redis序列化配置：

package com.redisexample.redisconf;

import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.data.redis.connection.RedisConnectionFactory;
import org.springframework.data.redis.core.RedisTemplate;
import org.springframework.data.redis.serializer.Jackson2JsonRedisSerializer;
import org.springframework.data.redis.serializer.StringRedisSerializer;

import com.fasterxml.jackson.annotation.JsonAutoDetect;
import com.fasterxml.jackson.annotation.PropertyAccessor;
import com.fasterxml.jackson.databind.ObjectMapper;

@Configuration
public class RedisConfig {
	
	/**
	 * 自定義統一RedisTemplate序列化機制
	 * @param factory
	 * @return
	 */
	@Bean
	@SuppressWarnings("all")
	public RedisTemplate<String, Object> redisTemplate(RedisConnectionFactory factory) {
		RedisTemplate<String, Object> template = new RedisTemplate<String, Object>();
		template.setConnectionFactory(factory);
		Jackson2JsonRedisSerializer jackson2JsonRedisSerializer = new Jackson2JsonRedisSerializer(Object.class);
		ObjectMapper om = new ObjectMapper();
		om.setVisibility(PropertyAccessor.ALL, JsonAutoDetect.Visibility.ANY);
		om.enableDefaultTyping(ObjectMapper.DefaultTyping.NON_FINAL);
		jackson2JsonRedisSerializer.setObjectMapper(om);
		StringRedisSerializer stringRedisSerializer = new StringRedisSerializer();
		// key採用String的序列化方式
		template.setKeySerializer(stringRedisSerializer);
		// hash的key也採用String的序列化方式
		template.setHashKeySerializer(stringRedisSerializer);
		// value序列化方式採用jackson
		template.setValueSerializer(jackson2JsonRedisSerializer);
		// hash的value序列化方式採用jackson
		template.setHashValueSerializer(jackson2JsonRedisSerializer);
		template.afterPropertiesSet();
		return template;
	}

}