Hadoop1.0 ObjectWritable

  Hadoop的ObjectWritable保存了一個能夠在Hadoop RPC裏面傳輸的對象和對象類型的說明信息。用於客戶端與服務器間傳輸的Writable對象,也是對RPC傳輸對象的封裝,由於RPC上交換的信息只能是JAVA的基礎數據類型,String或者Writable類型,而ObjectWritable是對其子類的抽象封裝,該類大體狀況以下:java

ObjectWritable會往流裏寫入以下信息:數組

對象類名,對象本身的串行化結果服務器

其序列化和反序列化方法以下:oop

public void readFields(DataInput in) throws IOException {
    readObject(in, this, this.conf);
  }
  
  public void write(DataOutput out) throws IOException {
    writeObject(out, instance, declaredClass, conf);
  }

 分別調用的是writeObject(out, instance, declaredClass, conf)和readObject(in, this, this.conf)方法:this

writeObject(out, instance, declaredClass, conf)方法:該方法逐一分析待xulie的數據類型而且實現序列化。spa

public static void writeObject(DataOutput out, Object instance,
                                 Class declaredClass, 
                                 Configuration conf) throws IOException {
	//對象爲空則抽象出內嵌數據類型NullInstance
    if (instance == null) {                       // null
      instance = new NullInstance(declaredClass, conf);
      declaredClass = Writable.class;
    }
    //先寫入類名
    UTF8.writeString(out, declaredClass.getName()); // always write declared
    /*
     * 封裝的對象爲數組類型,則逐個序列化(序列化爲length+對象的序列化內容)
     * 採用了迭代
     */
    
    if (declaredClass.isArray()) {                // array
      int length = Array.getLength(instance);
      out.writeInt(length);
      for (int i = 0; i < length; i++) {
        writeObject(out, Array.get(instance, i),
                    declaredClass.getComponentType(), conf);
      }
      //爲String類型直接寫入
    } else if (declaredClass == String.class) {   // String
      UTF8.writeString(out, (String)instance);
      
    }//基本數據類型寫入 
    else if (declaredClass.isPrimitive()) {     // primitive type

      if (declaredClass == Boolean.TYPE) {        // boolean
        out.writeBoolean(((Boolean)instance).booleanValue());
      } else if (declaredClass == Character.TYPE) { // char
        out.writeChar(((Character)instance).charValue());
      } else if (declaredClass == Byte.TYPE) {    // byte
        out.writeByte(((Byte)instance).byteValue());
      } else if (declaredClass == Short.TYPE) {   // short
        out.writeShort(((Short)instance).shortValue());
      } else if (declaredClass == Integer.TYPE) { // int
        out.writeInt(((Integer)instance).intValue());
      } else if (declaredClass == Long.TYPE) {    // long
        out.writeLong(((Long)instance).longValue());
      } else if (declaredClass == Float.TYPE) {   // float
        out.writeFloat(((Float)instance).floatValue());
      } else if (declaredClass == Double.TYPE) {  // double
        out.writeDouble(((Double)instance).doubleValue());
      } else if (declaredClass == Void.TYPE) {    // void
      } else {
        throw new IllegalArgumentException("Not a primitive: "+declaredClass);
      }
      //枚舉類型寫入
    } else if (declaredClass.isEnum()) {         // enum
      UTF8.writeString(out, ((Enum)instance).name());
      //hadoop的Writable類型寫入
    } else if (Writable.class.isAssignableFrom(declaredClass)) { // Writable
      UTF8.writeString(out, instance.getClass().getName());
      ((Writable)instance).write(out);

    } else {
      throw new IOException("Can't write: "+instance+" as "+declaredClass);
    }
  }


對於readObject(in, this, this.conf)方法:該方法用於反序列化,在ObjectWritable對象內部封裝了一個基本數據類型的註冊表.net

經過if語句條件實現基本數據類型的反序列化,而對於實現了Writbale接口的封裝數據,則是經過Conf對象獲取其類聲明,其反序列化的code

實例化也是調用WritableFactories工程去實例化,最後在存儲反序列化的值。對象

public static Object readObject(DataInput in, Configuration conf)
    throws IOException {
    return readObject(in, null, conf);
  }
    
  /** Read a {@link Writable}, {@link String}, primitive type, or an array of
   * the preceding. */
  @SuppressWarnings("unchecked")
  public static Object readObject(DataInput in, ObjectWritable objectWritable, Configuration conf)
    throws IOException {
	  //獲取反序列化的名字
    String className = UTF8.readString(in);
    //假設爲基本數據類型
    Class<?> declaredClass = PRIMITIVE_NAMES.get(className);
    /*
     * 判斷是否爲基本數據類型,不是則爲空,則爲Writable類型,
     * 對於Writable類型從Conf配置文件中讀取類名,
     * 在這裏只是獲取類名,而並無反序列化對象
     */
    
    if (declaredClass == null) {
      try {
        declaredClass = conf.getClassByName(className);
      } catch (ClassNotFoundException e) {
        throw new RuntimeException("readObject can't find class " + className, e);
      }
    }    
    //基本數據類型
    Object instance;
    //爲基本數據類型,逐一反序列化
    if (declaredClass.isPrimitive()) {            // primitive types

      if (declaredClass == Boolean.TYPE) {             // boolean
        instance = Boolean.valueOf(in.readBoolean());
      } else if (declaredClass == Character.TYPE) {    // char
        instance = Character.valueOf(in.readChar());
      } else if (declaredClass == Byte.TYPE) {         // byte
        instance = Byte.valueOf(in.readByte());
      } else if (declaredClass == Short.TYPE) {        // short
        instance = Short.valueOf(in.readShort());
      } else if (declaredClass == Integer.TYPE) {      // int
        instance = Integer.valueOf(in.readInt());
      } else if (declaredClass == Long.TYPE) {         // long
        instance = Long.valueOf(in.readLong());
      } else if (declaredClass == Float.TYPE) {        // float
        instance = Float.valueOf(in.readFloat());
      } else if (declaredClass == Double.TYPE) {       // double
        instance = Double.valueOf(in.readDouble());
      } else if (declaredClass == Void.TYPE) {         // void
        instance = null;
      } else {
        throw new IllegalArgumentException("Not a primitive: "+declaredClass);
      }

    } else if (declaredClass.isArray()) {              // array
      int length = in.readInt();
      instance = Array.newInstance(declaredClass.getComponentType(), length);
      for (int i = 0; i < length; i++) {
        Array.set(instance, i, readObject(in, conf));
      }
      
    } else if (declaredClass == String.class) {        // String類型的反序列化
      instance = UTF8.readString(in);
    } else if (declaredClass.isEnum()) {         // enum的反序列化
      instance = Enum.valueOf((Class<? extends Enum>) declaredClass, UTF8.readString(in));
    } else {                                      // Writable
      Class instanceClass = null;
      String str = "";
      try {
    	  //剩下的從Conf對象中獲取類型Class
        str = UTF8.readString(in);
        instanceClass = conf.getClassByName(str);
      } catch (ClassNotFoundException e) {
        throw new RuntimeException("readObject can't find class " + str, e);
      }
      /*
       * 帶用了WritableFactories工廠去new instanceClass(實現了Writable接口)對象出來
       * 在調用實現Writable對象自身的反序列化方法
       */
     
      Writable writable = WritableFactories.newInstance(instanceClass, conf);
      writable.readFields(in);
      instance = writable;

      if (instanceClass == NullInstance.class) {  // null
        declaredClass = ((NullInstance)instance).declaredClass;
        instance = null;
      }
    }
    //最後存儲反序列化後待封裝的ObjectWritable對象
    if (objectWritable != null) {                 // store values
      objectWritable.declaredClass = declaredClass;
      objectWritable.instance = instance;
    }

    return instance;
      
  }

對於Null類型的封裝,在ObjectWritable內部封裝了一個Nullinstance接口

private static class NullInstance extends Configured implements Writable {
    private Class<?> declaredClass;
    public NullInstance() { super(null); }
    public NullInstance(Class declaredClass, Configuration conf) {
      super(conf);
      this.declaredClass = declaredClass;
    }
相關文章
相關標籤/搜索