Thrift Java Servers Compared

Thrift Java Servers Compared

Address: https://github.com/m1ch1/mapkeeper/wiki/Thrift-Java-Servers-Compared

This article is only about java servers. If you are interested in C ++ servers, please refer

Thrift is a cross-language serialization/rpc framework. It consists of three main components: protocol, transmission transport, and service server. The Protocol defines how messages are serialized. Transmission defines how messages are transmitted between the client and the server. The server receives the serialized message from transport, deserializes the message according to the protocol, then calls the user-defined message processing function, deserializes the response from the hander, and then writes it back to transport. Thrift's modular architecture makes it available to diverse servers. The list of available servers in java is as follows:

• TSimpleServer
• TNonblockingServer
• THsHaServer
• TThreadedSelectorServer
• TThreadPoolServer

  It is good to have a choice, but which server is suitable for you? In this article, I will describe the differences between servers and describe the performance characteristics through the benchmark results (detailed benchmark in appendix B ). Let's start with the simplest one: TSimpleServer

  TSimpleServer

  TSimpleServer receives a connection and processes the request on the connection until the client closes the connection. Because everything is in one thread and it is blocking I/O, it can only serve one connection at the same time, and other clients have to wait until they are received. TSimpleServer is mainly used for testing purposes and should not be used in the production environment.

  TNonblockingServer VS THsHaServer

  TNonblockingServer solves the problem with TSimpleServer of one client blocking all the other clients by using non-blocking I/O. it uses java. nio. channels. selector, which allows you to get blocked on multiple connections instead of a single connection by calling select (). the select () call returns when one ore more connections are ready to be accepted/read/written. TNonblockingServer handles those connections either by accepting it, reading data from it, or writing data to it, and callselect () again to wait for the next available connections. this way, multiple clients can be served without one client starving others.

  TNonblockingServer uses non-blocking I/O to solve the problem that a client of TSimpleServer blocks all other clients. It uses java. nio. channels. Selector and allows you to block multiple connections rather than one connection by calling select. Select () returns when one or more connections are accepted, read, and write. TNonblockingServer processes these connections either by accepting connections, or by reading or writing data from these connections, and then calling select () again to wait for the next available connection. This method can be installed to serve multiple clients without one client starving other clients.

  There is a catch, however. messages are processed by the same thread that callselect (). let's say there are 10 clients, and each message takes 100 MS to process. what wocould be the latency and throughput? While a message is being processed, 9 clients are waiting to be selected, so it takes 1 second for the clients to get the response back from the server, and throughput will be 10 requests/second. wouldn't it be great if multiple messages can be processed simultaneously?

  However, there is a tricky problem. Messages are processed by the select () thread. Let's assume that there are 10 clients, and each message needs to be processed in 100 ms. Latency and throughput? When a message is processed, the other nine clients are waiting to be selected. Therefore, 1 sclient is required to obtain the response returned by the server. The throughput is 10 req/s. Will it be much better if messages can be processed at the same time?

  This is where THsHaServer (Half-Sync/Half-Async server) comes into picture. it uses a single thread for network I/O, and a separate pool of worker threads to handle message processing. this way messages will get processed immediately if there is an idle worker threads, and multiple messages can be processed concurrently. using the example abve, now the latency is 100 MS and throughput will be 100 requests/sec.

  This is the starting point of THsHaServer (semi-synchronous/semi-asynchronous service. It uses a single thread to process network I/O and a separate working thread pool to process messages. In this way, if there is an idle working thread, the message will be immediately processed, and multiple messages can also be processed in parallel. Using the example above, the latency is now 100 ms, and the throughput prize is 100 req/s.

  To demonstrate this, I ran a benchmark with 10 clients and a modified message handler that simply sleeps for 100 MS before returning. I used THsHaServer with 10 worker threads. The handler looks something like this:

  To prove this, I ran a benchmark with 10 clients and a message processing function. The message processing was simply a simple sleep of MS and then returned. I use THsHaServer with 10 working threads. Handler is as follows:

  public ResponseCode sleep() throws TException{    try {        Thread.sleep(100);    } catch (Exception ex) {    }    return ResponseCode.Success;}

  The results are as expected. THsHaServer is able to process all the requests concurrently, while TNonblockingServer processes requests one at a time.

  The result is as expected. THsHaServer can process all requests concurrently, while TNonblockingServer processes one request at a time. <喎?http: www.bkjia.com kf ware vc " target="_blank" class="keylink"> VcD4KCgo8aDI + signature/LGjs9bBvbj2z9 + signature/Signature + MLnaW/Kx8a/Signature + y/Signature/PwrXExr2 + signature/Signature + cjxpbwcgc3jjjpq = "http://www.bkjia.com/uploadfile/Collfiles/20140120/20140120085935159.png" alt = "\">
  

  The result shows that TThreadedSelectorServer has much higher throughput than THsHaServer while maintaining lower latency.

  The results show that TThreadedSelectorServer has higher throughput and lower latency than THsHaServer.

  TThreadedSelectorServer vs. TThreadPoolServer

  Finally, there is TThreadPoolServer. TThreadPoolServer is different from the other 3 servers in that:

  Finally, TThreadPoolServer, TThreadPoolServer, and the other three servers are different in the following aspects:

  • There is a dedicated thread for accepting connections.
  • Once a connection is accepted, it gets scheduled to be processed by a worker thread in ThreadPoolExecutor.
  • The worker thread is tied to the specific client connection until it's closed. Once the connection is closed, the worker thread goes back to the thread pool.
  • You can configure both minimum and maximum number of threads in the thread pool. Default values are 5 and Integer. MAX_VALUE, respectively.
  • There is a dedicated thread that accepts the connection
  • Once a connection is accepted, it is scheduled to be handled by a worker thread in ThreadPoolExecutor.
  • This worker thread serves the specified client until the connection is closed. Once the connection is closed, the worker thread returns to the thread pool.
  • You can configure the minimum and maximum number of threads in the thread pool. Corresponding default values: 5 and Integer. MAX_VALUE

    This means that if there are 10000 concurrent client connections, you need to run 10000 threads. as such, it is not as resource friendly as other servers. also, if the number of clients exceeds the maximum number of threads in the thread pool, requests will be blocked until a worker thread becomes available.

    This means that if there are 10000 concurrent client connections, you need to run 10000 threads. In itself, this is not as resource-friendly as other servers. In addition, if the number of clients exceeds the maximum value of the thread pool, the request will be blocked until a working thread is available.

    Having said that, TThreadPoolServer performs very well; on the box I'm using it's able to support 10000 concurrent clients without any problem. if you know the number of clients that will be connecting to your server in advance and you don't mind running a lot of threads, TThreadPoolServer might be a good choice for you.

    Even so, TThreadPoolServer performs very well; I use it to support 10000 concurrent connections without any problems. If you know the number of your clients in advance and don't mind a little more threads, TThreadPoolServer may be a good choice for you.

    
    Conclusion

    I hope this article helps you decide which Thrift server is right for you. I think TThreadedSelectorServer wocould be a safe choice for most of the use cases. you might also want to consider TThreadPoolServer if you can afford to run lots of concurrent threads. feel free to send me email atmapkeeper-users@googlegroups.com or post your comments here if you have any questions/comments.

    I hope this article will help you decide which Thrift server suits you. I think TThreadedSelectorServer is a safe choice in most cases. If you can accept a large number of concurrent processes, you can also consider TThreadPoolServer.

    Appendix A: Hardware Configuration

    Processors:     2 x Xeon E5620 2.40GHz (HT enabled, 8 cores, 16 threads)Memory:         8GBNetwork:        1Gb/s 
         
          OS:             RHEL Server 5.4 Linux 2.6.18-164.2.1.el5 x86_64
         

    Appendix B: Benchmark Details

    Pass