Hadoop detailed (13) serialization framework

Introduction to the Framework

MapReduce can only support writable do key,value? The answer is in the negative. In fact, all types are supported by just one small condition: each type is transmitted in binary streams. This Hadoop provides a serialization framework to support the types that writable can serve as mapreduce support in the Org.apache.hadoop.io.serializer package as well as the fact that there are not many classes, we start with several interfaces.

Serializer

Defines a set of interfaces, opens the stream, serializes, closes the stream

Public interface Serializer <T>  {  
    void open (Java.io.OutputStream outputstream) throws java.io.IOException;  
      
    void Serialize (T-T) throws java.io.IOException;  
      
    void Close () throws Java.io.IOException  
}

Deserializer

Defines a set of interfaces, opens the stream, deserializes it, and closes the stream

public interface 

Deserializer <T>  {  
    void open (Java.io.InputStream inputstream) throws java.io.IOException;  
      
    T Deserialize (T-T) throws java.io.IOException;  
      
    void Close () throws Java.io.IOException  
}

Serialization

Defines a set of interfaces that determine whether the input class is supported, and gives the serialization interface and the deserialization interface based on the input class

Public interface Serialization <T>  {  
    Boolean accept (java.lang.class<?> aclass);  
      
    Org.apache.hadoop.io.serializer.serializer<t> Getserializer (java.lang.class<t> 

tClass);  
      
    Org.apache.hadoop.io.serializer.deserializer<t> getdeserializer

(java.lang.class<t> tClass);  
}

Writableserialization

If you want to define a framework like writable yourself, then the first thing you need is to implement the three interfaces above, so let's take a look at how writable is implemented.

<pre style= "Word-wrap:break-word; White-space:pre-wrap;  
        
  ">public class Writableserialization extends configured implements serialization<writable> {  Static class Writabledeserializer extends configured implements deserializer<writable> {private  
    Class<?> Writableclass;  
          
    Private DataInputStream DataIn;  
      Public Writabledeserializer (Configuration conf, class<?> c) {setconf (conf);  
    This.writableclass = C; The public void open (InputStream in) {if (in instanceof datainputstream) {DataIn = (Dat  
      Ainputstream) in;  
      else {datain = new datainputstream (in);  
      } Public writable deserialize (writable W) throws IOException {writable writable;  
      if (w = = null) {writable = (writable) reflectionutils.newinstance (Writableclass, getconf ()); } else {Writable = W;  
      } writable.readfields (DataIn);  
    return writable;  
    public void Close () throws IOException {datain.close (); } Static class Writableserializer implements Serializer<writable> {Priva  
          
    Te DataOutputStream dataout;  public void Open (OutputStream out) {if (out instanceof dataoutputstream) {dataout = (DataOutputStream)  
      Out  
      else {dataout = new DataOutputStream (out);  
    } public void serialize (writable W) throws IOException {w.write (dataout);  
    public void Close () throws IOException {dataout.close ();  
  } public Boolean accept (class<?> c) {return Writable.class.isAssignableFrom (c); Deserializer<writable> Getdeserializer (class<writable> c) {return new Writabledeser Ializer (getconf(), c); Serializer<writable> Getserializer (class<writable> c) {return new writableserialize  
  R (); }}</pre>