A little understanding of thrift

a little understanding of thrift

This is a study of thrift notes that includes a brief description of How Thrift uses thrift Thrift The Knowledge points in the overall architecture Thrift
　　struct can set default values
　　the serialization mechanism in thrift
　　version control in thrift Brief Introduction to Thrift

It is an RPC communication framework, using C/S architecture, and has an efficient serialization mechanism. To use thrift, first we need to open the Thrift service on the remote server, and then the server-side process stays dormant until the client code calls.
One of the main reasons for thrift's wide application is that it supports a variety of mainstream languages, and that users who use it do not need to focus on how the server and the client communicate and how to serialize, only to consider how to implement the business logic that they need.
Thrift uses the interface language to define data structures and services, contains the most commonly used data types, corresponds to the basic types of languages, and defines enumerations and exceptions, among other things. How to use thrift

Thrift defines it fairly succinctly, so that our use of the process is also exceptionally convenient, in simple terms, the use of thrift process only requires the following four steps:
1. Design the data format (struct, enum, etc.) that requires interaction and specific services (service), define the Thrift interface description file, that is, the suffix name is. Thrift
2. Using the Thrift tool (I am using the older version of 0.5.0), generate the target language file based on the previously defined interface file (in this note, the client code and server code are all in the Java language)
3. Implement service code, and set the implementation of the business logic as the Thrift Server processing layer, select the port, the server to start listening, waiting for the client's connection request
4. The client uses the same port to connect the server to request service

The following is a brief introduction to the following Thrift Interface Description Language (IDL) type:
IDL contains several types of underlying types, structs, containers, exceptions, and services:
Base type: Includes Bool,byte, i16,i32,i64,double,string, each of which corresponds to the underlying type of each language
Structure: defined in thrift as struct, which resembles a struct in C, is a collection of underlying types, and each structure generates a separate class, similar to the Pojo in Java
Container: Thrift defines three commonly used container –list,set,map, in Java, each corresponding implementation is ArrayList, HashSet, HashMap, where the template type can be the underlying type or struct type
Exception: The definition of an exception is similar to struct, just replaced by a exception
Service: Service Similar to Java interface, need to define return type, parameter declaration, throw exception for each method signature in the service, exception thrown by method, every method will throw texception, in addition to own declaration, for return value is void type method, We can prefix the method signature with the oneway identifier, marking this method as asynchronous, that is, immediately after the call returns

Now, to better understand how to use thrift and how to use the types in IDL, I'll give you an example of course, this example is just for the demo process, not too much design, and there may be some impractical logic.
How to start writing this example. Yes, just follow the four steps of the thrift process described earlier: define an Interface description file (. Thrift)
　　Qinyi_student_model.thrift defines the data structure of student information and school information

/**
 * Qinyi Student Thrift Model
 * @author Qinyi
 * @since 2015-10-02/namespace

java Com.qinyi.thrift_study.thrift_example

enum Sex {
    Boy = 1;
    Girl = 2;
}

struct Studentinfo {
    1:required string name;
    2:required Sex Sex;
    3:required i32 age;
    4:optional list<string> Hobby;
    5:required map<string, i64> number;
}

struct School {
    1:required string name;
    2:required list<studentinfo> students;
    3:optional string description;
}

As you can see, we use namespace to define the namespace of the file, because the target code is the Java language, so the declaration after namespace Java represents the package name, and the struct structure has a digital ID tag before each attribute, once defined, It is best not to change, the specific reasons below will be specified, the attribute type has a required/optional declaration, representing this property must be set or can choose not to set, if this property is declared as required, but there is no set in the code, Thrift will think this is an exception, of course, we can set the default value of the attribute, that is, when the declaration of the value can be. The file begins with the Java-style annotation used in the section, which is also optional, thrift supports C,c++,shell,java style annotations, and how annotations are based on personal habits.

　　　　qinyi_student_exception.thrift Definition exception

/**
 * Qinyi Student Thrift Exception * @author Qinyi * @since 2015-10-02/namespace

java Com.qinyi . thrift_study.thrift_example

Exception studentexception {
    1:required i64 errorcode;
    2:required string description;
    3:optional string causeinfo;
}

We can see that the definition of the exception is very similar to the struct in the file above.

　　　　qinyi_student_service.thrift Definition Service

/** * Qinyi Student Service * @author Thrift * @since Qinyi/2015-10-02 java com . Qinyi.thrift_study.thrift_example include "Qinyi_student_model.thrift" include "qinyi_student_exception.thrift"// The definition of a service is semantically equivalent to an interface in object-oriented programming service Studentservice {//Add student to school bool Addstudenttoschool (1:qinyi_stu Dent_model. Studentinfo student) throws (1:qinyi_student_exception.

    Studentexception ex); Get student info by name List<qinyi_student_model. Studentinfo> getstudentinfobyname (1:string name) throws (1:qinyi_student_exception.

    Studentexception ex); Print single student info void Printstudentinfo (1:qinyi_student_model. Studentinfo student) throws (1:qinyi_student_exception.

    Studentexception ex); Print List students info void Printstudentsinfo (1:list<qinyi_student_model. Studentinfo> students) throws (1:qinyi_student_exception.
Studentexception ex); }

If you are familiar with C, you are certainly not unfamiliar with the include, thrift can also refer to other thrift files, and include a double quotation mark, in the file for the reference to other thrift files in the field also use the full name. using the Thrift tool to generate target code using IDL
As mentioned earlier, this is the second step in the thrift process, and here, for ease of operation, let's write a shell script:

#!/bin/bash

thrift_home= "{your_thrift_home}/thrift_version/bin"
thrift_file= "{your_thrift_idl_files}"

${thrift_home}/thrift--gen java ${thrift_file}/qinyi_student_exception.thrift
${thrift_home}/thrift-- Gen Java ${thrift_file}/qinyi_student_model.thrift
${thrift_home}/thrift--gen java ${thrift_file}/qinyi_ Student_service.thrift

Because the target language is Java and the namespace is defined in the thrift script, the resulting directory structure will be the same after you run the above script:
　　/gen-java/com/qinyi/thrift_study/thrift_example Implement service business logic and start service monitoring
The next step is to implement the business logic in the interface, and wait for the client to invoke these business logic, relatively simple, the business logic implementation file is: Studentserviceimpl.java

/** * Created by Qinyi on 10/2/15. */public class Studentserviceimpl implements Studentservice.iface {@Override public boolean addstudenttoschool ( Studentinfo student) throws Studentexception, texception {if (null = = Student) {throw new Studentexc
                    Eption (). Seterrorcode ( -1). SetDescription ("Addstudenttoschool (Studentinfo Student) error")
        . Setcauseinfo ("Student is null");
        } list<studentinfo> students = Schoolmock.getinstance (). getstudents ();
        Students.add (student);

        Schoolmock.getinstance (). setstudents (students);
    return true; @Override public list<studentinfo> getstudentinfobyname (String name) throws Studentexception, Texception {if (null = = name) {throw new Studentexception (). Seterrorcode ( -1). Setdescriptio
        N ("Getstudentinfobyname (String name) error"). Setcauseinfo ("name is null");} list<studentinfo> students = Schoolmock.getinstance (). getstudents ();
        list<studentinfo> results = new arraylist<studentinfo> (); for (Studentinfo student:students) {if (Student.getname (). Equals (name)) {Results.add (stude
            NT);
    } return results; @Override public void Printstudentinfo (Studentinfo student) throws Studentexception, texception {if (n ull = = Student) {throw new Studentexception (). Seterrorcode ( -1). SetDescription ("Printstud
        Entinfo (studentinfo Student) Error "). Setcauseinfo (" Student is null ");
        } StringBuilder builder = new StringBuilder ();
        Builder.append ("Name:"). Append (Student.getname ()). Append ("\ n");
        if (Student.getsex (). GetValue () = = 1) {builder.append ("Sex:boy"). Append ("\ n");
 else {builder.append ("Sex:girl"). Append ("\ n");       } builder.append ("Age:"). Append (Student.getage ()). Append ("\ n"); if (Student.issethobby ()) {for (String hobby:student.getHobby ()) {Builder.append ("hobby:
            "). Append (Hobby). Append (" \ n ");

        } builder.append ("ID:"). Append (Student.getnumber (). Get (Student.getname ()). Append ("\ n");
    System.out.println (Builder.tostring ()); 
        @Override public void Printstudentsinfo (list<studentinfo> students) throws Studentexception, Texception { if (null = = students) {throw new Studentexception (). Seterrorcode ( -1). Setdescrip
        tion ("Printstudentsinfo (list<studentinfo> students) error"). Setcauseinfo ("Students is null");
        for (Studentinfo student:students) {printstudentinfo (student); }
    }
}

As mentioned earlier, all of the service methods, in addition to throwing our custom exception, throw texception This check exception, where a Schoolmock object is used to get a school object to complete the simulated business logic, and here is the implementation code:
　　Schoolmock.java

/**
 * Created by Qinyi on 10/2/15.
 * * Public
class Schoolmock {

    private static School School;

    Private Schoolmock () {
    } public

    static synchronized School getinstance () {
        if (null = = School) {
            School = New School ();
            School.setname ("school");
            School.setdescription ("This is just a mock school");
            School.setstudents (New arraylist<studentinfo> ());
        }

        Return school
    }
}

Next, we need to do the server side of the last step, to open the server side of the monitoring, the implementation file is: Studentthriftserver.java, because the code has done a lot of comments, so, do not go too much explanation:

/** * Created by Qinyi on 10/2/15.

    * * Public class Studentthriftserver {public static final int server_port = 9527; public static void Main (string[] args) throws texception{/** * Servertransport: Setting the server's port * t Processor: The service implementation class of the associated processor * Server: Set the server (Tsimpleserver-single-threaded server side using standard blocking I/O, only for test development) * Server.serv
         E (): Open the service, is generally in the sleep state, until the request of the client * *//** * Here to open the Server service using the old API interface, where the thrift is 0.5.0
        * */Tserversocket Servertransport = new Tserversocket (server_port);
        Tprocessor tprocessor = new Studentservice.processor (new Studentserviceimpl ());
        /** * Single-threaded server-side using standard blocking I/O */tserver Server = new Tsimpleserver (tprocessor, Servertransport);
        SYSTEM.OUT.PRINTLN ("Start Server on port 9527 ...");

        Server.serve (); /** * thrift0.6.1 later versions (if I'm not mistaken), an internal static class is defined in the Tserver abstract class Args, user cascade software stack (Transport layer, protocol layer, processing layer) * public sTatic class Args extends abstractserverargs<args> {* Public Args (Tservertransport transport) {
         * SUPER (transport);
         *} * The interface call to open the Thrift service in the new interface is probably this: * Args series: Transport layer, Protocol layer, processing layer * * *//**
         * Tprocessor tprocessor = new studentservice.processor<studentservice.iface> (new Studentserviceimpl ());
         * Tserversocket servertransport = new Tserversocket (server_port);
         * Tserver.args Targs = new Tserver.args (servertransport);
         * Targs.processor (Tprocessor);
         * Targs.protocolfactory (new tbinaryprotocol.factory ());
         * Tserver Server = new Tsimpleserver (Targs);
         * SYSTEM.OUT.PRINTLN ("Start Server on port 9527 ...");
         * Server.serve ();
 */
    }
}

Yes, it's very simple to turn on the server-side code, because thrift does a lot of work and all we need is just to fill in the business logic we want and the implementation of each tier. Ok.
Finally, only the client connection is left to get the request. Client Connection Server request Service
The implementation of the client is also very simple, we just need to get a thrift for us to define the client, and then invoke the required business logic, the implementation code here is: Studentthriftclient.java

/** * Created by Zhanghu on 10/2/15.
    * * Public class Studentthriftclient {private static final String server_ip = "127.0.0.1";
    private static final int server_port = 9527;
    private static final int TIMEOUT = 5000;
    private static ttransport transport;

    private static studentservice.client Client;  The static {/** * Transport layer uses blocking I/O for transmission */transport = new Tsocket (server_ip, Server_port,
        TIMEOUT);
        /** * Defines the mapping between memory and network transport formats * binary: Fairly simple binary encoding: combining filed and corresponding value in a simple binary code tbinaryprotocol * * * *
        Tprotocol protocol = new Tbinaryprotocol (transport);
    Client = new Studentservice.client (protocol); } private static void Mockconstructstudent () throws Texception, studentexception {/** * Constructs objects to be aware of
         : * 1. If the field defined in the Thrift script file is required, then it must be set, otherwise it will be an error * 2. If the field defined in the Thrift script file is optional, you do not have to go to the set * */Studentinfo student1 = new STudentinfo ();
        Student1.setname ("Qinyi");
        Student1.setnumber (New hashmap<string, long> () {{put ("Qinyi", 21209184L);
        }});
        Student1.setage (25);
        Student1.setsex (Sex.boy);

        Student1.sethobby (New arraylist<string> (Arrays.aslist ("Ping pong", "swimming", "Tai Qiu"));
        Studentinfo Student2 = new Studentinfo ();
        Student2.setname ("Brucezhang");
        Student2.setnumber (New hashmap<string, long> () {{put ("Brucezhang", 8205050122L);
        }});
        Student2.setage (18);
Student2.setsex (Sex.boy);

        Student2.sethobby (New arraylist<string> () {//Add ("Game");//});
        Client.addstudenttoschool (STUDENT1);
        Client.addstudenttoschool (Student2); /** * The following call throws an exception: * The exception message printed in this example is as follows: * Studentexception (Errorcode:-1, Description:addstudenttos
       Chool (Studentinfo Student) error, causeinfo:student is null)  * */Client.addstudenttoschool (NULL);
        private static void Mockgetservice () throws Texception, studentexception {mockconstructstudent ();
        Client.printstudentsinfo (Client.getstudentinfobyname ("Qinyi"));
    Client.printstudentsinfo (Client.getstudentinfobyname ("Brucezhang")); public static void Main (string[] args) throws texception{/** * Transport: Setting Transport Channel * PR Otocol: Using binary Transport protocol * Client: Create client * Transport.open (): Open transmission Channel * Transport.close (): Turn off transmission pass
        Road * */Transport.open ();
        try {mockgetservice ();
            catch (Studentexception e) {System.out.println (E.getmessage ());
        E.printstacktrace ();
    } transport.close ();
 }
}

The important location is described in the code, and there is no explanation.
In this way, we have completed the four steps of the thrift process, and then we can start testing the RPC process, first of all, we need to run the server-side code, we will see the console will print out an output:Start server on port 9527, after Running client code and waiting for the client process to end, we go back to the server-side console and see the output defined in the business logic.
Haha, maybe you don't understand why I want to put the output on the server side, rather than the client, does not seem to be the right logical thinking, yes, here to explain, just because convenient, conveniently written on the server side, the actual application must be the method to return the client's query results, and then the client side to do their own parsing work. Thrift Overall Architecture

In fact, writing this part of the inevitable some of the powerless, this part of the entire thrift framework of the composition, my understanding of it is only the basis of the foundation, however, because it is learning notes, or record here.
Thrift is a four-tier architecture, so the advantage of this design is the freedom to choose the implementation of each layer to meet different service requirements, such as I in the above example, the server is a single-threaded blocking IO model (this is just a thrift implementation of the toy, The production process is not likely to use this service model, you can also change the other implementation model as needed, and the code part of the change is minimal, the separation of the architectural design makes each layer is transparent, regardless of the underlying implementation, only need an interface to complete the call. Below, I'll start with a rough introduction to each layer in the thrift from the bottom.

Ttransport Layer
The transport layer is implemented using protocols such as TCP and HTTP, which includes methods in various socket invocations, such as Open,close,read,write. Because it is the last layer in the framework, the most important part of the implementation is, of course, read and write data (read and write), which has a blocking and non-blocking implementation.

Tprotocol Layer
The protocol layer is the definition of how data will reach the transport layer in the form. It first defines each data structure in IDL, and defines the read and write methods for each type. We need to declare the same implementation protocol on both the server side and the client as a mapping between memory and network transmission formats.
Commonly used protocols have Tbinaryprotocol: it defines the data as binary transmission, it is the simplest implementation protocol, but also the most commonly used implementation protocol, very efficient; Tcompactprotocol: Its name is called the compressed binary protocol, Compared with the Tbinaryprotocol, it uses the compression algorithm to compress the data, reduce the actual amount of data transmitted and improve the transmission efficiency.

tprocessor Layer
The processing layer is the server-side defined processing business logic, whose main code is the Iface interface and processor class in the **service.java file.
Iface interface: All methods in this interface are user defined in the IDL file service method, it needs to throw texception This check exception, the server side needs to define the corresponding implementation class to implements * *. Iface interface to complete the business logic of the service.
Processor class: This class defines a processmap that contains the methods defined in the service, and the server side constructs the Processor object The only thing that needs to be done is to pass the object that implements the service (Iface) as a parameter to the processor constructor.

Server Layer
The server is the highest level in the thrift framework, creating and managing the three layers below, while providing the thread scheduling logic for client invocation.
The base class for the service layer is tserver, which is equivalent to a container containing the Tprocessor,ttransport,tprotocol and implementing management and scheduling of them. Tserver has a variety of implementations, for this example is the use of tsimpleserver, this is a single threaded blocking IO model, the actual production of most of the use of the tthreadselectorserver–-multi-threaded non-blocking IO model. knowledge points in the thrift struct can set default values

We can also define struct School by using the School example we defined earlier:

struct School {
    1:required string name = "School";
    2:required list<studentinfo> students;
    3:optional String Description = "This is just a mock school";
}

In this way, we can set these two fields without having to construct the school object, of course, if the default value is what you want. The advantage of this feature is that when there are more than one required field, and these fields are often invariant, we also have to define objects in the one by one set these fields, if you forget to set a certain, it will cause thrift throw an exception, it will be very troublesome, but, If we consider these defaults when we define the IDL file, we will not encounter those problems when we construct the object. the serialization mechanism in thrift

Before, it has been mentioned that each attribute in the struct must have a digital ID before it is defined and it is best not to change it, as explained here. To better illustrate the problem, let's take an example, assuming that we need to define a school structure in our program that contains two fields (string name, string address) as follows:

struct School {
    1:required string name;
    2:required string address;

After that, we used the thrift tool to generate the target code (which contained serialization), after which we constructed the school:

School School = new School ();
School.setname ("Dalian University"). Setaddress ("No. 2nd Ling Gong Road");

Then, we redefine school (thrift file):

struct School {
    2:required string name;
    1:required string address;

Then regenerate the target code and write the following procedure:

System.out.println (School.getname ());
System.out.println (School.getaddress ());

The question is, what kind of output will we get? Perhaps, as you can guess, the name and address are reversed, not as we've defined before, and to know why, we need to understand how thrift serializes objects.
The struct definition in thrift ultimately requires serialization, and the information it needs to use is the ID and type before the attribute, and the serialized stored procedure forms Such a mapping relationship:
　　Name:value--id + type:value
Therefore, the name of the attribute is not important, the actual process is not needed, so we use objects to get the value of the property is the process of mapping the relationship of a reverse process, according to the ID and type to obtain the corresponding value, then why the opposite result is clear.
So, in practical applications, if you have defined the fields in the struct, add no problem, only need to define a different ID value can be, try not to change the original attribute ID, also do not delete the fields that are no longer needed, so as to avoid the original ID used repeatedly, serialization will result in confusion. version control in thrift

This is a technique for designing thrift script files, which is struct for serialization mechanisms. We still explain in the form of examples, suppose we need to design a school structure (how always school, not dislike school. , which contains student information and teacher information (usually written in two different struct, just to illustrate the problem), it looks like the following:

struct School {
    1:required list<string> student_name;
    2:required map<string, i16> student_age;

    5:required list<string> Teacher_name;
    6:required map<string, i16> teacher_age;
}

It looks weird, why no ID is 3,4 's property field. This is because if our needs change, such as a field in which the student information needs to add an exam score (score), then, according to the previous version of IDL's design, "seamless access" can be achieved as follows:

struct School {
    1:required list<string> student_name;
    2:required map<string, i16> student_age;
    3:required map<string, set<i16>> score;

    5:required list<string> Teacher_name;
    6:required map<string, i16> teacher_age;
}

This ID design will not result in ambiguous semantics as the subsequent increase in information although we can define the ID of each field casually, it is better to define the ID of each field sequentially and set some reserved fields as needed in case of version upgrade, which is used in HBase , MySQL and other database tables are also very common.

Learning thrift time is not long, coupled with my reaction is dull, the level is limited, the ability to understand the new things slightly worse, but, happy to share, human kindness is the nature of the dictates, know how to share the fun, to better programming, no sharing, not programming.