"Go" Apache Thrift-Scalable cross-language service development Framework

Apache Thrift-Scalable cross-language service development Framework

Apache Thrift is an efficient framework for Facebook to implement remote service calls that support multiple programming languages. This article will detail the architecture, development, and deployment of Apache Thrift from the Java developer perspective, and give the corresponding Java instance for different transport protocols and service types, and introduce the implementation of Thrift asynchronous client in detail, and finally put forward some things to be aware of using Thrift.

12 Reviews

Huang, Intern, IBM

Zhang Jing, software engineer, IBM

Zhang Kai, senior software engineer, IBM

January 16, 2012

• Content

Develop and deploy your next application on the IBM Bluemix cloud platform.

Get started with your trial

Objective:

There are a number of popular service invocation methods, such as a WEB service based on a SOAP message format, a RESTful service based on a JSON message format, and so on. The data transmission methods used include Xml,json, but the XML is too large, the transmission efficiency is low, the JSON is small and novel, but it is not perfect. This article will introduce the remote service call framework Apache Thrift, developed by Facebook, which uses interface description language to define and create services to support extensible cross-language service development, including code generation engines that can be used in multiple languages such as C + +, Java, Python, PHP, Ruby, Erlang, Perl, Haskell, C #, Cocoa, Smalltalk, etc. create efficient, seamless services that transmit data in a binary format that is smaller in size relative to XML and JSON, and more advantageous for high concurrency, large data volumes, and a multilingual environment. This article will cover the use of Thrift in detail and provide a rich example code to explain to help users quickly build services.

Back to top of page

A simple example of Thrift

This article first introduces a simple Thrift implementation example that allows readers to quickly and intuitively understand what Thrift is and how to build a service using Thrift.

Create a simple service Hello. First write the script file Hello.thrift according to the syntax specification of Thrift, the code is as follows:

Listing 1. Hello.thrift

namespace Java Service.demo  service hello{   string hellostring (1:string para)   i32 helloint (1:i32 para)   bool Helloboolean (1:bool para)   void hellovoid ()   string Hellonull ()  }

There are five methods that define the service Hello, each containing a method name, a parameter list, and a return type. Each parameter consists of a parameter ordinal, a parameter type, and an argument name. Thrift is a specific implementation of the descriptive language of IDL (Interface Definition Language). Therefore, the above service description file is written using IDL syntax. Using the Thrift tool to compile the Hello.thrift, the corresponding Hello.java file is generated. The file contains the interface definition of the service Hello described in the Hello.thrift file, the Hello.iface interface, and the underlying communication details of the service invocation, including the client's call logic Hello.client and the server-side processing logic Hello.processor, which is used to build client and server-side functionality.

Create the Helloserviceimpl.java file and implement the Hello.iface interface in the Hello.java file, with the following code:

Listing 2. Helloserviceimpl.java

Package Service.demo;  Import org.apache.thrift.TException;  public class Helloserviceimpl implements Hello.iface {     @Override public     Boolean Helloboolean (Boolean para) Throws Texception {         return para;     }     @Override public     int helloint (int para) throws texception {         try {             thread.sleep (20000);         } catch ( Interruptedexception e) {             e.printstacktrace ();         }         return para;     }     @Override public     String Hellonull () throws texception {         return null;     }     @Override public     string hellostring (string para) throws texception {         return para;     }     @Override public     void Hellovoid () throws Texception {         System.out.println ("Hello World");     }  }

Create a server-side implementation code that passes Helloserviceimpl as a specific processor to the Thrift server with the following code:

Listing 3. Helloserviceserver.java

 Package service.server;  Import Org.apache.thrift.TProcessor;  Import Org.apache.thrift.protocol.TBinaryProtocol;  Import Org.apache.thrift.protocol.TBinaryProtocol.Factory;  Import Org.apache.thrift.server.TServer;  Import Org.apache.thrift.server.TThreadPoolServer;  Import Org.apache.thrift.transport.TServerSocket;  Import org.apache.thrift.transport.TTransportException;  Import Service.demo.Hello;  Import Service.demo.HelloServiceImpl; public class Helloserviceserver {/** * starts Thrift server * @param args */public static void main (Strin              G[] Args {try {//Set service port to 7911 tserversocket servertransport = new Tserversocket (7911);             Set the Protocol factory to tbinaryprotocol.factory Factory profactory = new Tbinaryprotocol.factory ();             The implementation of the associated processor and the Hello service Tprocessor processor = new Hello.processor (new Helloserviceimpl ());        Tserver Server = new Tthreadpoolserver (processor, Servertransport,             Profactory);             SYSTEM.OUT.PRINTLN ("Start Server on port 7911 ...");         Server.serve ();         } catch (Ttransportexception e) {e.printstacktrace (); }     }  }

Create the client implementation code, call the Hello.client to access the server side of the logical implementation, the code is as follows:

Listing 4. Helloserviceclient.java

 Package service.client;  Import org.apache.thrift.TException;  Import Org.apache.thrift.protocol.TBinaryProtocol;  Import Org.apache.thrift.protocol.TProtocol;  Import Org.apache.thrift.transport.TSocket;  Import Org.apache.thrift.transport.TTransport;  Import org.apache.thrift.transport.TTransportException;  Import Service.demo.Hello; public class Helloserviceclient {/** * call Hello Service * @param args */public static void Main (string[] A RGS) {try {//Set call service address is local, port is 7911 ttransport transport = new Tsocket ("localhost", 7911             );             Transport.open ();             Set the transport protocol to tbinaryprotocol Tprotocol protocol = new Tbinaryprotocol (transport);             Hello.client Client = new hello.client (protocol);             Call the Hellovoid method of the service client.hellovoid ();         Transport.close ();         } catch (Ttransportexception e) {e.printstacktrace (); } catch (Texception e) {E. Printstacktrace (); }     }  }

After the code is written, run the server, and then start the client Invoke service Hello method hellovoid, in the server-side console window output "Hello World" (Hellovoid method implementation in the console print string, no return value, So the client calls the method and does not return a value output, and the reader can try other calls with the return value method on its own, and the results can be printed in the client's console window.

Back to top of page

Thrift Architecture

The Thrift contains a complete stack structure for building the client and server side. Depicts the overall architecture of the Thrift.

Figure 1. Architecture diagram

, the yellow part of the figure is the user-implemented business logic, the brown part is based on the Thrift definition of the service interface profile generated by the client and server-side code framework, the red part is based on the Thrift file generated code to achieve data read and write operations. Red section below is the Thrift transport system, protocol, and underlying I/O communication, using Thrift to easily define a service and choose a different transport protocol and transport layer without having to regenerate the code.

The Thrift server contains the infrastructure for binding protocols and transport layers, which provides blocking, non-blocking, single-threaded, and multithreaded modes to run on the server and can be run with the server/container, and can be seamlessly combined with the existing Java EE Server/web container.

The service-side and client-specific invocation processes are as follows:

Figure 2. Server-side boot, service timing diagram (view larger image)

The diagram shows the process of Helloserviceserver startup and the response of the server when the service is called by the client. As we can see, after the program calls the Tthreadpoolserver serve method, the server enters the blocking listening state, which is blocked on the Tserversocket accept method. When a message is received from the client, the server initiates a new thread to process the message request and the original thread enters the blocking state again. In the new thread, the server reads the message content through the TBINARYPROTOCOL protocol, invokes the Helloserviceimpl hellovoid method, and writes the results back to the client in Hellovoid_result.

Figure 3. Client-side Invoke Service timing diagram (view larger image)

The figure shows the process of helloserviceclient invoking the service and processing the result after receiving the return value from the server side. As we can see, the program calls the Hello.client Hellovoid method, in the Hellovoid method, sends the call request to the service through the Send_hellovoid method, through the Recv_hellovoid method receives the result returned by the service when the request is processed.

Back to top of page

Data type

Thrift scripts can define data types that include the following types:

• Basic type:

• BOOL: Boolean, True or False, Boolean corresponding to Java
• A byte:8-bit signed integer that corresponds to the byte of Java
• I16:16-bit signed integer that corresponds to the short of Java
• I32:32-bit signed integer that corresponds to the int of Java
• i64:64-bit signed integer that corresponds to a long in Java
• double:64 bit floating point number, corresponding to the double of Java
• String: Unknown encoded text or binary string, corresponding to Java string

• struct type:
  • struct: Defines a common object, similar to a struct definition in C, which is a JavaBean in Java
• Container type:
  • List: ArrayList for Java
  • Set: corresponding to Java HashSet
  • Map: HashMap for Java
• Exception type:
  • Exception: Exception for Java
• Service Type:
  • Service: The class of the corresponding services

Back to top of page

Agreement

Thrift can let the user choose the type of transmission communication protocol between client and server, divide the transmission protocol into text and binary (binary) transmission protocol, in order to save bandwidth, improve transmission efficiency, and generally use binary type transmission protocol as the majority, Sometimes a text type-based protocol is used, depending on the actual requirements in the project/product. Common protocols include the following:

• tbinaryprotocol--binary encoded format for data transfer

  Use the method as shown in Listing 3 and listing 4.

• tcompactprotocol--high-efficiency, dense binary encoding format for data transfer

  The servers and clients that build the TCOMPACTPROTOCOL protocol simply replace the TBINARYPROTOCOL Protocol section in Listing 3 and listing 4 and replace it with the following code:

  Listing 5. Helloserviceserver.java built using the TCOMPACTPROTOCOL protocol

  Tcompactprotocol.factory profactory = new Tcompactprotocol.factory ();

  Listing 6. Helloserviceclient.java using the Tcompactprotocol protocol

  Tcompactprotocol protocol = new Tcompactprotocol (transport);

• tjsonprotocol--data transmission using JSON encoding protocol

  The servers and clients that build the TJSONPROTOCOL protocol simply replace the TBINARYPROTOCOL Protocol section in Listing 3 and listing 4 and replace it with the following code:

  Listing 7. Helloserviceserver.java built using the TJSONPROTOCOL protocol

  Tjsonprotocol.factory profactory = new Tjsonprotocol.factory ();

  Listing 8. Helloserviceclient.java using the Tjsonprotocol protocol

  Tjsonprotocol protocol = new Tjsonprotocol (transport);

• tsimplejsonprotocol--only provides JSON-only protocols for parsing through scripting languages

Back to top of page

Transport Layer

The common transport layers are as follows:

• tsocket--is the most common mode of transmission using blocking I/O

  Use the method as shown in Listing 4.

• tframedtransport--is transmitted in a block size, similar to NIO in Java, using a non-blocking method

  If the Tframedtransport transport layer is used, its server must be modified to a non-blocking service type, the client simply replaces the Ttransport section in Listing 4, the code is as follows, and the Tnonblockingservertransport class in Listing 9 is building a nonblocking so Cket abstract class, Tnonblockingserversocket class inheritance Tnonblockingservertransport

  Listing 9. Helloserviceserver.java built using the Tframedtransport transport layer

  Tnonblockingservertransport Servertransport;  Servertransport = new Tnonblockingserversocket (10005);  Hello.processor Processor = new Hello.processor (new Helloserviceimpl ());  Tserver Server = new Tnonblockingserver (processor, servertransport);  SYSTEM.OUT.PRINTLN ("Start Server on port 10005 ...");  Server.serve ();

  Listing 10. Using the Helloserviceclient.java of the Tframedtransport Transport layer

  Ttransport transport = new Tframedtransport (new Tsocket ("localhost", 10005));

• tnonblockingtransport--uses non-blocking methods for building asynchronous clients

  Please refer to Thrift Asynchronous client building for usage methods

Back to top of page

Service-side type

There are several common types of server services:

• tsimpleserver--single-threaded server side using standard blocking I/O

  The code is as follows:

  Listing 11. Helloserviceserver.java built using the Tsimpleserver server

  Tserversocket servertransport = new Tserversocket (7911);  Tprocessor processor = new Hello.processor (new Helloserviceimpl ());  Tserver Server = new Tsimpleserver (processor, servertransport);  SYSTEM.OUT.PRINTLN ("Start Server on port 7911 ...");  Server.serve ();

  The way the client is built can be referenced in Listing 4.

• tthreadpoolserver--multi-threaded server side using standard blocking I/O

  Use the method as shown in Listing 3.

• tnonblockingserver--multi-threaded server side using non-blocking I/O

  Please refer to Thrift Asynchronous client building for usage methods

Back to top of page

Thrift Asynchronous Client Building

The Thrift provides a non-blocking invocation method that builds an asynchronous client. In this way, Thrift provides a new class Tasyncclientmanager for managing client requests, tracing requests and responses on a thread, and passing standard parameters and callback objects through the interface Asyncclient, after the service call is complete, Callback provides methods for handling call results and exceptions.

First we look at the implementation of callback:

Implementation of Listing 12.CallBack: Methodcallback.java

Package service.callback;  Import Org.apache.thrift.async.AsyncMethodCallback;  public class Methodcallback implements Asyncmethodcallback {     Object response = null;     Public Object GetResult () {         //Returns the result value return        this.response;     }     The result value returned by the processing service    @Override public     void OnComplete (Object response) {         this.response = response;     }     Unhandled exception    @Override public     void OnError (Throwable throwable) {     }  } during the invocation of the service

As the code shows, the OnComplete method receives the result of the service's processing, where we assign the result response directly to callback's private property response. The OnError method receives the exception that is thrown during service processing, and the exception is not processed here.

Create a non-blocking server-side implementation code that passes Helloserviceimpl as a specific processor to an asynchronous Thrift server with the following code:

List 13.helloserviceasyncserver.java

 Package service.server;  Import Org.apache.thrift.server.TNonblockingServer;  Import Org.apache.thrift.server.TServer;  Import Org.apache.thrift.transport.TNonblockingServerSocket;  Import Org.apache.thrift.transport.TNonblockingServerTransport;  Import org.apache.thrift.transport.TTransportException;  Import Service.demo.Hello;  Import Service.demo.HelloServiceImpl; public class Helloserviceasyncserver {/** * starts Thrift asynchronous server * @param args */public static void Mai         N (string[] args) {Tnonblockingservertransport servertransport;             try {servertransport = new Tnonblockingserversocket (10005);             Hello.processor Processor = new Hello.processor (new Helloserviceimpl ());             Tserver Server = new Tnonblockingserver (processor, servertransport);             SYSTEM.OUT.PRINTLN ("Start Server on port 10005 ...");         Server.serve (); } catch (Ttransportexception e) {e.printstacktrace (); }     }  }

Helloserviceasyncserver creates a non-blocking server-side connection to wait for the client through Java.nio.channels.ServerSocketChannel.

Create the asynchronous client implementation code, call Hello.asyncclient to access the server's logical implementation, pass the Methodcallback object as a parameter into the calling method, and the code is as follows:

List 14.helloserviceasyncclient.java

 Package service.client;  Import java.io.IOException;  Import Org.apache.thrift.async.AsyncMethodCallback;  Import Org.apache.thrift.async.TAsyncClientManager;  Import Org.apache.thrift.protocol.TBinaryProtocol;  Import Org.apache.thrift.protocol.TProtocolFactory;  Import Org.apache.thrift.transport.TNonblockingSocket;  Import Org.apache.thrift.transport.TNonblockingTransport;  Import Service.callback.MethodCallback;  Import Service.demo.Hello; public class Helloserviceasyncclient {/** * call Hello Service * @param args */public static void main (St             Ring[] args) throws Exception {try {tasyncclientmanager Clientmanager = new Tasyncclientmanager ();             Tnonblockingtransport transport = new Tnonblockingsocket ("localhost", 10005);             Tprotocolfactory protocol = new Tbinaryprotocol.factory ();   Hello.asyncclient asyncclient = new Hello.asyncclient (protocol, Clientmanager, transport);          SYSTEM.OUT.PRINTLN ("Client calls ...");             Methodcallback callBack = new Methodcallback ();             Asyncclient.hellostring ("Hello World", CallBack);             Object res = Callback.getresult ();             while (res = = NULL) {res = Callback.getresult ();         } System.out.println (((Hello.AsyncClient.helloString_call) res). GetResult ());         } catch (IOException e) {e.printstacktrace (); }   }  }

Helloserviceasyncclient creates an asynchronous client with Java.nio.channels.Socketchannel to establish a connection to the server. In this article, the asynchronous client uses the following loop code to achieve the synchronization effect, the reader can remove this part of the code and then run the contrast.

Listing 15. Asynchronous client implementation synchronization effect code snippet

Object res = Callback.getresult ();//wait for the return result after the service call while (res = = null) {   res = Callback.getresult ();}

In comparison with the code in Listing 9 and listing 10, we can build a service side of the Tnonblockingserver service type, build a synchronization client for the Tframedtransport transport layer on the client and a Tnonblockingtransport The asynchronous client of the transport layer, a service can provide two different invocation modes through a socket port. Interested readers can try.

Back to top of page

FAQ NULL Problem

When we call a method on a service, there are times when the method returns a null value, and in Thrift, calling a method that returns a null value throws a Tapplicationexception exception. In Listing 2, Helloserviceimpl implements the Hellonull method, returns a null value, we add the code that calls the method in Helloserviceclient.java, and the exception appears as shown:

Figure 4. Tapplicationexception exception

In order to handle the return null value condition, we want to catch the exception and handle it accordingly, the specific client code is implemented as follows:

Listing 16. Code to process the service return value NULL

 Package service.client;  Import org.apache.thrift.TApplicationException;  Import org.apache.thrift.TException;  Import Org.apache.thrift.protocol.TBinaryProtocol;  Import Org.apache.thrift.protocol.TProtocol;  Import Org.apache.thrift.transport.TSocket;  Import Org.apache.thrift.transport.TTransport;  Import org.apache.thrift.transport.TTransportException;  Import Service.demo.Hello;  public class Helloserviceclient {/** * calls the Hello service and handles a null value problem * @param args */public static void             Main (string[] args) {try {ttransport transport = new Tsocket ("localhost", 7911);             Transport.open ();             Tprotocol protocol = new Tbinaryprotocol (transport);             Hello.client Client = new hello.client (protocol);             System.out.println (Client.hellonull ());         Transport.close ();         } catch (Ttransportexception e) {e.printstacktrace (); } catch (Texception e) {if (e instanceof TapplicaTionexception && ((tapplicationexception) e). GetType () = = T             Applicationexception.missing_result) {System.out.println ("The RESULT of Hellonull function is NULL"); }         }     }  }

After calling the Hellonull method, the Tapplicationexception exception is thrown, and the exception kind is missing_result, and this code shows that after capturing the exception, the RESULT of the Hellonull is printed directly in the console. function is NULL "information.

Back to top of page

Installation deployment

Apache Thrift's official website is: http://thrift.apache.org/tutorial/, the specific installation steps are as follows:

1. Download Thrift source files (HTTP://SVN.APACHE.ORG/REPOS/ASF/THRIFT/TAGS/THRIFT-0.6.1/)
2. Import the thrift source file into Eclipse, enter the/lib/java directory, and use the ant compilation Build.xml to get Libthrift-0.6.1-snapshot.jar
3. Import Libthrift-0.6.1-snapshot.jar, Slf4j-api-1.5.8.jar, Slf4j-log4j12-1.5.8.jar, and Log4j-1.2.14.jar into the Eclipse development environment
4. Download the thrift compiler tool, which compiles the thrift script file into a Java file: Http://apache.etoak.com//thrift/0.6.0/thrift-0.6.1.exe
5. Create the Hello.thrift script file, as described in the previous section, enter the directory where Thrift-0.6.1.exe is located, execute the command "Thrift-0.6.1.exe-gen Java x:\Hello.thrift", under the current run letter, You can see the Gen-java directory and enter the directory to see the generated Java code. For more thrift command content, please refer to the help command from thrift
6. Write the server and client code to complete the installation and deployment of thrift

The services generated based on the Apache Thrift framework include the client and server side, and the specific deployment pattern is as follows:

Figure 5. Deployment diagram

As we can see, the client and server-side deployments need to use common jar packages and Java files, the "Common file" area, where Hello.java is compiled by Hello.thrift. On the server side, the service must implement the Hello.iface interface and include the server's startup code Helloserviceserver.java. In the client, the code that includes the client call service Helloserviceclient.java. The client and server implement remote service invocation through the API provided by Hello.java.

Back to top of page

Summarize

This article introduces the installation and architecture of Apache Thrift, and introduces how to use Apache Thrift to build services in different situations, and introduces the construction of Thrift asynchronous client, and hopes to bring some help to the readers.

Reference Learning

• Apache Thrift official website: downloadable Thrift tools and source code.
• Thrift Features and Non-features:thrift of the functional characteristics and deficiencies.
• Apache Thrift Introduction: Describes the Thrift architecture, protocol, Transport layer, and service-side types, and compares them with other methods of building services such as REST.
• Installation deployment for Thrift: Thrift Installation Deployment Instructions
• Thrift:scalable cross-language Services Implementation:thrift official documentation detailing the design of Thrift
• Thrift API: About Apache Thrift 0.6.1 building the server and client API manuals
• Thrift Example: A simple application example of Thrift
• Fully Async Thrift Client in Java: Introduction to Thrift Asynchronous clients
• DeveloperWorks Java Technology Zone: Here are hundreds of articles on various aspects of Java programming.

Discuss

• Join DeveloperWorks Chinese community. View developer-driven blogs, forums, groups, and wikis, and communicate with other DeveloperWorks users.

"Go" Apache Thrift-Scalable cross-language service development Framework