Golang GRPC Thrift with Aerospike

This is a creation in Article, where the information may have evolved or changed.

Faced with hundreds of g of data, tens of thousands of of TPS has been looking for the right database and RPC framework. Recently, Aerospike and Google Grpc, Apache thrift were tested briefly.
The content of the test is relatively simple, that is, the client looks for the key to the server and returns value. To test simplicity, there are only a couple of key_value in the database. The AEROSPIKE,RPC framework used by the database is Grpc\thrift, the client and the server are all developed using the go language.
GRPC is defined as follows:

"proto3";package inf;message AuthRq {        string clientId = 1;        string token = 2;}message AuthRp {        1;}message DataTypeRq {        AuthRq authRq = 1;        string groupName  = 101;        string enName = 102;}message DataTypeRp {        1;        101;        102;}service Data {        rpc DataType (DataTypeRq) returns (DataTypeRp){};}

Thrift is defined as follows:

namespace go infnamespace java infnamespace cpp infstruct AuthRq {        1: string clientId;        2: string       token;}struct AuthRp {        1: required string      token;}struct DataTypeRq {        1: required AuthRq authrq;        2: required string groupName;        3: required string enName;}struct DataTypeRp {        1: required AuthRp authrp;        2: required i64 codeId;        3: required string CnName;}service Data {        DataTypeRp DataType (1: DataTypeRq dataTypeRq);}

The service-side and GRPC-related code are as follows:

DataType converts the request parameter to the DATATYPEM request parameter, invokes DATATYPEM, and returns the result to Func (T *data) DataType (CTX context. Context, Request *inf. Datatyperq) (Response *inf. Datatyperp, err error) {//log. Printf("APP Request:% #v", request)//If AUTHRSP, err: = t. Auth(CTX, request. AUTHRQ)Err = = Nil {var req datatypereq req. Enname= Request. EnnameReq. GroupName= Request. GroupNameresult, err: = DATATYPEM (req) response = &inf. Datatyperp{AUTHRP: &inf. AUTHRP{TOKEN:AUTHRSP. Token}, Cnname:result. Cnname, Codeid:result. Codeid,} return response, err//} else {//return nil, err//}}

The service-side and thrift-related code are as follows:

func (t *Data) DataType(request *inf.DataTypeRq) (response *inf.DataTypeRp, err error) {    //log.Printf("App request: %#v", request)    //  if authRsp, err := t.Auth(request.Authrq); err == nil {    var req DataTypeReq    req.EnName = request.EnName    req.GroupName = request.GroupName    result, err := dataTypeM(req)    response = inf.NewDataTypeRp()    response.Authrp.Token = authRsp.Token    response.CnName = result.CnName    response.CodeId = result.CodeId    //log.Printf("DataType response: %#v", response)    return response, err    //  } else {    //      return nil, err    //  }}

Server and Aerospike related code, datatype call the following function:

Func Datatypem (Request Datatypereq) (response datatypersp, err error) {//log. Printf ("Datatypem Request:% #v", request)Cachetable: ="Data_type_def"Key: = Request. GroupName +"|"+ Request. EnnameKeys, Err: = NewKey ("Test", Cachetable, key)result, Err: = Asclient.get (Nil,Keys)if result= = Nil { for{hostname, _: = OS. Hostname () ServerID: = Hostname + StrConv. Itoa (OS. Getpid ()) + StrConv. Itoa ( Time. Now (). Nanosecond ())ifOK: = TDP. Dlock (key, ServerID); OK {bin1: = Newbin ("Cnname","Voice on") Bin2: = Newbin ("code_id",101) Asclient.putbins (Nil,Keys, Bin1, Bin2)result, _ = Asclient.get (Nil,Keys) TDP. Undlock (key, ServerID) break}Else{ Time. Sleep (Trylocksleep * Time. Millisecond)result, _: = Asclient.get (Nil,Keys)if result! = nil {Break}}}}if result! = Nil {response. Cnname, _ =result. bins["Cnname"]. (string) tmp, _: =result. bins["code_id"]. (int)//log. Printf ("code_id:%d", TMP)Response. Codeid = Int64 (TMP)}Else{Log. Printf ("can ' t get result:% #v", request)}returnResponse, Err}

The client code is exactly the same as the RPC framework-independent code:

Func dataType (Client inf. Dataclient, t string) {var request inf. Datatyperqvar reqauth inf. AUTHRQReqauth. ClientId= ClientId Reqauth. Token= Token Request. AUTHRQ= &reqauth//Request. ClientId= clientId//Request. Token="1"Request. Enname="App_call_connect_success"Request. GroupName="User_event"Response, Err: = Client. DataType(Context. Background(), &request)//thrift is: response, err: = Client. DataType(request) If Err = = Nil {token = response. AUTHRP. TokenLog. Printf("token:%s", response. AUTHRP. Token) If response. Codeid!=101{Log. Printf("DataType% #v", response)}} else if err. Error() =="1"{Log. Printf("Auth Failer:% #v", err)}}

The test server is two, are two Intel E5 2630 CPU, the server on a servers, 192G memory, Aerospike and client on the B server, 128G memory, multi-client with Goroutine implementation.
The following is the number of client concurrency at 50 o'clock
GRPC:
Service side
Client

Thrift
Service side
Client

During the entire testing process, the number of client concurrency, whether 50 or 800, is not quite the same for resource usage. Overall, the thrift server or client, the CPU usage is very small, and the GRPC usage is too high, at the client 50 o'clock server GRPC more than thrift use 30% CPU.
The following are performance test results:

From the test, GRPC performance is generally better than thrift, but the performance difference between the two is very small. The best score is better than 17%, the worst is better than 3%. However, Grpc just came out, still in the alpha phase, the PROTOBUF3 protocol used, also in the alpha phase. Believe that after the official version, GPRC performance will have an upgrade.
In addition, from the development point of view, thrift and GRPC protocol in the use of the difference is very small, from the learning curve, from one side to the other side is relatively easy. At present Grpc-go relies on the third party package, most of them are on the googlecode, need to jailbreak access, very inconvenient.

Copyright NOTICE: Welcome reprint.