Thrift source code analysis (3)-IDL and Code Generation Analysis

IDL is a service description component used by many RPC frameworks to support cross-language environment calls. It is generally defined in text format. For more thoughts on IDL, refer to "Understanding WSDL and IDL".

Different Versions of thrift define different IDL syntax. Here we use thrift-0.8.0 to introduce the IDL definition in Java.

1. namespace defines the package name

2. struct defines the parameters of the service interface, and the class structure used in the return value. If all interface parameters are of the basic type, you do not need to define struct.

3. Service-defined interface

In a simple example, The IDL file is suffixed with. thrift.

Demo. Thrift

1. defined the generated Java file package name com. Thrift. Test

2. Define a struct class structure as a parameter

3. A service interface is defined, and the return value is int. The method name is demomethod. There are three parameters. The first is the string type, the second is the parameter class defined above, and the third is the map type.

namespace java com.thrift.teststruct Parameter{1: required i32 id;2: required string name;}service DemoService{i32 demoMethod(1:string param1, 2:Parameter param2, 3:map<string,string> param3);}

The data types supported by IDL include the following:

Bool Boolean byte 8-bit integer I16-bit integer i32 32-bit integer i64 64-bit integer double-precision floating point string binary byte array list <I16> list set, the generic Map <string, string> map type must be specified. The generic set <i32> set must be specified.

With IDL, you can use thrift to automatically generate auxiliary code, including client code and serialized interface code.

thrift -r --gen java demo.thrift

The generated code is as follows:

Each struct will generate a separate class, and each service will generate a class.

Let's take a look at the specific structure of the generated class.

The generated class consists of five parts:

1. Interface TypeThe default name is iface. This interface type is used by the server and client. The server uses it as the top-level interface to write implementation classes. The client code uses it as the service interface for proxy generation.

There are two automatically generated interfaces: Synchronous call iface and asynchronous call asynciface. The asynchronous call interface has one more callback parameter.

public interface Iface {    public int demoMethod(String param1, Parameter param2, Map<String,String> param3) throws org.apache.thrift.TException;  }  public interface AsyncIface {    public void demoMethod(String param1, Parameter param2, Map<String,String> param3, org.apache.thrift.async.AsyncMethodCallback<AsyncClient.demoMethod_call> resultHandler) throws org.apache.thrift.TException;  }

2. Client type, A client for Synchronous calls and a client asyncclient for asynchronous calls

3. ProcessorTo support method calls. Each Service implementation class must be registered using processor, so that the server can locate the specific implementation class when calling the interface implementation. A special article will be introduced later

4. encapsulation class of method parameters, Named after "method name_args"

5. encapsulation class of the method Return Value, Named after "method name_result"

Let's take a look at the generated code for synchronously calling the Client client.

1. Provide a factory method to create a client object

2. The client proxy of the interface method only sends the method call request and receives the returned value.

The sending method call request does two things.

1. Create a method parameter object and encapsulate the method parameters

2. Call the sendbase method of the parent class to send messages. Before sending a message, use writemessagebegin to send the message header, call the write (tprotocol) method of the method parameter object to send the message body, and end sending.

Two things were done to accept the call return value.

1. Create a method return value object and encapsulate the method Return Value

2. Call the receivebase method of the parent class to receive the method return value. First, you can use receivemessage to receive the message body, handle exceptions, call the read (tprotocol) method of the method parameter object to receive the message body, and end the receipt.

Public static class Client extends Org. apache. thrift. tserviceclient implements iface {public static class factory implements Org. apache. thrift. tserviceclientfactory <client> {public Factory () {} public client getclient (Org. apache. thrift. protocol. tprotocol prot) {return new client (prot);} public client getclient (Org. apache. thrift. protocol. tprotocol iprot, org. apache. thrift. protocol. tprotocol opro T) {return new client (iprot, oprot) ;}} public int demomethod (string param1, parameter param2, Map <string, string> param3) throws Org. apache. thrift. texception {send_demomethod (param1, param2, param3); Return recv_demomethod ();} public void send_demomethod (string param1, parameter param2, Map <string, string> param3) throws Org. apache. thrift. texception {demomethod_args ARGs = new demomethod_args (); Args. setparam1 (param1); args. setparam2 (param2); args. setparam3 (param3); sendbase ("demomethod", argS);} public int recv_demomethod () throws Org. apache. thrift. texception {demomethod_result result = new demomethod_result (); receivebase (result, "demomethod"); If (result. issetsuccess () {return result. success;} Throw new Org. apache. thrift. tapplicationexception (Org. apache. thrift. tapplicationexcep Tion. missing_result, "demomethod failed: Unknown result") ;}// Org. apache. thrift. tserviceclient. sendbase, the client's parent class method protected void sendbase (string methodname, tbase ARGs) throws texception {// send the message header oprot _. writemessagebegin (New tmessage (methodname, tmessagetype. call, ++ seqid _); // sends the message body. The method parameter object processes the codec ARGs by itself. write (oprot _); oprot _. writemessageend (); oprot _. gettransport (). flush ();} protected void re Ceivebase (tbase result, string methodname) throws texception {// receive message header tmessage MSG = iprot _. readmessagebegin (); If (MSG. type = tmessagetype. exception) {tapplicationexception x = tapplicationexception. read (iprot _); iprot _. readmessageend (); throw X;} If (MSG. seqid! = Seqid _) {Throw new tapplicationexception (tapplicationexception. bad_sequence_id, methodname + "failed: out of sequence response");} // The result is encoded and decoded by the returned value object. read (iprot _); iprot _. readmessageend ();}

Let's take a look at the method parameter object.

The method parameters implement the tbase interface, which defines an encoding/decoding interface for an object under a certain protocol.

public interface TBase<T extends TBase<?,?>, F extends TFieldIdEnum> extends Comparable<T>,  Serializable {  public void read(TProtocol iprot) throws TException;  public void write(TProtocol oprot) throws TException;｝

The method parameter object mainly performs two tasks.

1. Create the metadata of each parameter, including the parameter type and sequence number. The sequence number is set when IDL is defined. It is used to identify the parameter location and is useful in codec.

2.Implement your own coding and decoding methods, including read (tprotocol) and write (tprotocol ). The specific codec function is delegated to the xxxscheme class.

Public static class demomethod_args implements Org. apache. thrift. tbase <demomethod_args, demomethod_args. _ fields>, Java. io. serializable, cloneable {Private Static final Org. apache. thrift. protocol. tstruct struct_desc = new Org. apache. thrift. protocol. tstruct ("demomethod_args"); Private Static final Org. apache. thrift. protocol. tfield paramdid field_desc = new Org. apache. thrift. protocol. tfield ("param1", O RG. apache. thrift. protocol. tType. string, (short) 1); Private Static final Org. apache. thrift. protocol. tfield param2_field_desc = new Org. apache. thrift. protocol. tfield ("param2", org. apache. thrift. protocol. tType. struct, (short) 2); Private Static final Org. apache. thrift. protocol. tfield param3_field_desc = new Org. apache. thrift. protocol. tfield ("param3", org. apache. thrift. protocol. tType. map, (short) 3); PR Ivate static final map <class <? Extends ischeme>, schemefactory> schemes = new hashmap <class <? Extends ischeme>, schemefactory> (); static {schemes. put (standardscheme. class, new demomethod_argsstandardschemefactory (); schemes. put (tuplescheme. class, new demomethod_arstupleschemefactory ();} Public String param1; // required public parameter param2; // required public Map <string, string> param3; // required/** the set of fields this struct contains, along with convenience methods for finding and manipulating them. */Public Enum _ fields implements Org. apache. thrift. tfieldidenum {param1 (short) 1, "param1"), param2 (short) 2, "param2"), param3 (short) 3, "param3 "); private Static final map <string, _ fields> byname = new hashmap <string, _ fields> (); static {for (_ fields field: enumset. allof (_ fields. class) {byname. put (field. getfieldname (), field) ;}// the object itself is responsible for decoding public void read (Org. apache. thrift. protocol. tprotocol iprot) throws Org. apache. thrift. texception {schemes. get (iprot. getscheme ()). getscheme (). read (iprot, this) ;}< pre name = "code" class = "Java"> // The object is encoded by itself.

Public void write (Org. Apache. Thrift. Protocol. tprotocol oprot) throws org. Apache. Thrift. texception {
Schemes. Get (oprot. getscheme (). getscheme (). Write (oprot, this );
}

Let's take a look at the xxxscheme class, which is also automatically generated, which is the internal class of the method parameter xxx_args.

There are two types of scheme. One is standardscheme, which uses the message header + message body method to codec the object. One is tuplescheme, which directly uses the Message Body Writing Method for coding and decoding, And the encoded byte stream is smaller.

Take demomethod_argsstandardscheme as an example,

1. Its encoding method is to write segments one by one starting from writestructbegin. before writing each field, it starts with writefieldbegin. The writing segment type and the sequence number of the field. If the field is a struct class, you can call this class's own encoding method write (tprotocol ). Thrift generates classes for each struct, which define the encoding and decoding methods of this class. End with writestructend.

2. the decoding method starts from readstructbegbegin, reads the field metadata readfieldbegin, reads the field type of 1 byte, And the byte sequence number of 2 fields. Then, based on the field type, to read data of the corresponding type length. It ends with readstructend.

 private static class demoMethod_argsStandardScheme extends StandardScheme<demoMethod_args> {      public void read(org.apache.thrift.protocol.TProtocol iprot, demoMethod_args struct) throws org.apache.thrift.TException {        org.apache.thrift.protocol.TField schemeField;        iprot.readStructBegin();        while (true)        {          schemeField = iprot.readFieldBegin();          if (schemeField.type == org.apache.thrift.protocol.TType.STOP) {             break;          }          switch (schemeField.id) {            case 1: // PARAM1              if (schemeField.type == org.apache.thrift.protocol.TType.STRING) {                struct.param1 = iprot.readString();                struct.setParam1IsSet(true);              } else {                 org.apache.thrift.protocol.TProtocolUtil.skip(iprot, schemeField.type);              }              break;            case 2: // PARAM2              if (schemeField.type == org.apache.thrift.protocol.TType.STRUCT) {                struct.param2 = new Parameter();                struct.param2.read(iprot);                struct.setParam2IsSet(true);              } else {                 org.apache.thrift.protocol.TProtocolUtil.skip(iprot, schemeField.type);              }              break;            case 3: // PARAM3              if (schemeField.type == org.apache.thrift.protocol.TType.MAP) {                {                  org.apache.thrift.protocol.TMap _map0 = iprot.readMapBegin();                  struct.param3 = new HashMap<String,String>(2*_map0.size);                  for (int _i1 = 0; _i1 < _map0.size; ++_i1)                  {                    String _key2; // required                    String _val3; // required                    _key2 = iprot.readString();                    _val3 = iprot.readString();                    struct.param3.put(_key2, _val3);                  }                  iprot.readMapEnd();                }                struct.setParam3IsSet(true);              } else {                 org.apache.thrift.protocol.TProtocolUtil.skip(iprot, schemeField.type);              }              break;            default:              org.apache.thrift.protocol.TProtocolUtil.skip(iprot, schemeField.type);          }          iprot.readFieldEnd();        }        iprot.readStructEnd();        // check for required fields of primitive type, which can't be checked in the validate method        struct.validate();      }      public void write(org.apache.thrift.protocol.TProtocol oprot, demoMethod_args struct) throws org.apache.thrift.TException {        struct.validate();        oprot.writeStructBegin(STRUCT_DESC);        if (struct.param1 != null) {          oprot.writeFieldBegin(PARAM1_FIELD_DESC);          oprot.writeString(struct.param1);          oprot.writeFieldEnd();        }        if (struct.param2 != null) {          oprot.writeFieldBegin(PARAM2_FIELD_DESC);          struct.param2.write(oprot);          oprot.writeFieldEnd();        }        if (struct.param3 != null) {          oprot.writeFieldBegin(PARAM3_FIELD_DESC);          {            oprot.writeMapBegin(new org.apache.thrift.protocol.TMap(org.apache.thrift.protocol.TType.STRING, org.apache.thrift.protocol.TType.STRING, struct.param3.size()));            for (Map.Entry<String, String> _iter4 : struct.param3.entrySet())            {              oprot.writeString(_iter4.getKey());              oprot.writeString(_iter4.getValue());            }            oprot.writeMapEnd();          }          oprot.writeFieldEnd();        }        oprot.writeFieldStop();        oprot.writeStructEnd();      }    }

 private static class demoMethod_argsTupleScheme extends TupleScheme<demoMethod_args> {      @Override      public void write(org.apache.thrift.protocol.TProtocol prot, demoMethod_args struct) throws org.apache.thrift.TException {        TTupleProtocol oprot = (TTupleProtocol) prot;        BitSet optionals = new BitSet();        if (struct.isSetParam1()) {          optionals.set(0);        }        if (struct.isSetParam2()) {          optionals.set(1);        }        if (struct.isSetParam3()) {          optionals.set(2);        }        oprot.writeBitSet(optionals, 3);        if (struct.isSetParam1()) {          oprot.writeString(struct.param1);        }        if (struct.isSetParam2()) {          struct.param2.write(oprot);        }        if (struct.isSetParam3()) {          {            oprot.writeI32(struct.param3.size());            for (Map.Entry<String, String> _iter5 : struct.param3.entrySet())            {              oprot.writeString(_iter5.getKey());              oprot.writeString(_iter5.getValue());            }          }        }      }      @Override      public void read(org.apache.thrift.protocol.TProtocol prot, demoMethod_args struct) throws org.apache.thrift.TException {        TTupleProtocol iprot = (TTupleProtocol) prot;        BitSet incoming = iprot.readBitSet(3);        if (incoming.get(0)) {          struct.param1 = iprot.readString();          struct.setParam1IsSet(true);        }        if (incoming.get(1)) {          struct.param2 = new Parameter();          struct.param2.read(iprot);          struct.setParam2IsSet(true);        }        if (incoming.get(2)) {          {            org.apache.thrift.protocol.TMap _map6 = new org.apache.thrift.protocol.TMap(org.apache.thrift.protocol.TType.STRING, org.apache.thrift.protocol.TType.STRING, iprot.readI32());            struct.param3 = new HashMap<String,String>(2*_map6.size);            for (int _i7 = 0; _i7 < _map6.size; ++_i7)            {              String _key8; // required              String _val9; // required              _key8 = iprot.readString();              _val9 = iprot.readString();              struct.param3.put(_key8, _val9);            }          }          struct.setParam3IsSet(true);        }      }    }  }

The structure and principle of the method return value encapsulation class are exactly the same as those of the method parameter encapsulation class.

　　

Thrift source code analysis (3)-IDL and Code Generation Analysis