Golang: Implementation of thrift client-side Association security

This is a creation in Article, where the information may have evolved or changed.

Objective

Golang, as the main language of our server development, has realized many basic services, such as OAUTH2, account system, payment, customer service, etc. In the early stage of development, but also try to use Golang to do the page display, but each language has its own best field, let Golang to make the front end is a bit painful, finally we choose Php+golang way to serve as a whole structure.

So the question is, PHP and Golang, how does this pair of good friends play happily? The conclusion is that thrift is a good soap!

Throwing Bricks

There is a lot of soap on the market, the most famous is the good skin, then we do not need to skin good hair, and choose thrift ... Because it's sour and cool enough!

This acid is cool, only teeth good, can eat well fragrant. As we all know, thrift has a variety of models (transport protocol), such as home-type tdebugprotocol, long-lasting tbinaryprotocol and explosion-type Tcompactprotocol.

And we used the initial, take for granted the explosive type of tcompactprotocol this more can make the acid feel 10 increase in the model. But PHP teeth not very good, encountered Golang rubbed out of the 64-bit int, 1234567890 just to explode into 1234567891 (here is only an example, PHP in the process of processing Golang return int64 will have the wrong result). So, PHP and Golang this good base friend, thrift explosion acid cool good, than thrift lasting good. (it is said that the next version of thrift will fix this bug, please pay attention to it)

Lead Jade

A random, citing classic, found that the server side of the thrift generated is thread safe, but the client side is not. Therefore, multiple thread and server-side communication is required, and each thread needs to init one of its own client instances.

So the question is, how does the Golang implement the thrift client-side process security?

Practice

First of all, thrift implementation of Golang server side, relying on Golang cattle fork goroutine, only to achieve a similar tthreadedserver service model, so Mao teacher no longer have to worry about my drip use.

Func (P *tsimpleserver) Acceptloop () error {for    {        Select {case        <-p.quit:            return nil        default:< c5/>}        Client, err: = P.servertransport.accept ()        if err! = Nil {            log. Println ("Accept err:", err)        }        if client! = Nil {            go func () {//From new routine processing                if err: = P.processrequests (client); Err! = Nil {                    log. PRINTLN ("Error processing request:", Err)}}            ()        }    }

Second, the client side of thrift is thread insecure, then the problem comes, re-implement transport good, or in the existing transport based on the use of pool good?

Still when I think about how to modify the implementation of the transport, Mao has taken care of the pool, then the conclusion of the transport based on the use of pool good ... Even if the re-implementation is nothing more than a pool, so it has to change the thrift of the client implementation, it is time-consuming and difficult to please. Thrift the default implementation of the transport has the basic reading and writing functions, dropped into the pool to swim to swim.

The following is a pool implemented by teacher Mao, with features such as basic timeout checking, maximum activations, and idle numbers.

Type Pool struct {//Dial is a application supplied function for creating new connections.    Dial func () (interface{}, error)//Close is a application supplied functoin for closeing connections. Close func (c interface{}) error//Testonborrow is a optional application supplied function for checking//the HEA Lth of an idle connection before the connection are used again by//the application. Argument T is the time, the connection was returned//to the pool.    If the function returns an error, then the connection is//closed. Testonborrow func (c interface{}, T time.    Time) error//Maximum number of idle connections in the pool.    Maxidle int//Maximum number of connections allocated by the pool at a given time.    When zero, there are no limit on the number of connections in the pool. maxactive int//Close connections after remaining idle for this duration. If the value//is zero and then idle connections was not closed. Applications should set//the timeout to a value less than the server ' s timeout. IdleTimeout time.    Duration//MU protects fields defined below. Mu sync.    Mutex closed bool Active INT//Stack of Idleconn with most recently used at the front. Idle list. List}type idleconn struct {c interface{} t time. time}//new creates a new pool. This function is deprecated.  Applications should//Initialize the Pool fields directly as shown in Example.func New (DIALFN func () (interface{}, error), Closefn func (c interface{}) error, Maxidle int) *pool {return &AMP;POOL{DIAL:DIALFN, Close:closefn, Maxidle:maxidl e}}//Get gets a connection. The application must close the returned connection.//this method always returns a valid connection so that applications C    An defer//error handling to the first use of the Connection.func (P *pool) Get () (interface{}, error) {P.mu.lock () If closed if p.closed {p.mu.unlock () return nil, errpoolclosed}//Prune stale connections. If timeout: = P.idletimeout; Timeout > 0 {for i, N: = 0, P.idle.len (); i < n; i++ {e: = P.idle.back () If E = Nil {break} IC: = E.value. (idleconn) if Ic.t.add (timeout). After (Nowfunc ()) {break} p.idle.remove (e) p.active-= 1 p.mu. Unlock ()//Ic.c.close () p.close (IC.C) P.mu.lock ()}}//Get idle Connectio N. For i, N: = 0, P.idle.len (); I 0 && p.active >= p.maxactive {p.mu.unlock () return nil, errpoolexhausted}//No idle C    Onnection, create new.  Dial: = p.dial p.active + = 1 p.mu.unlock () c, err: = Dial () if err! = Nil {p.mu.lock () p.active  -= 1 P.mu.unlock () c = nil} return c, err}//Put adds Conn back to the pool, use Forceclose to close The connection Forcelyfunc (P *pool) Put (c Interface{}, forceclose bool) error {if!forceclose {p.mu.lock () if!p.closed {P.idle.pushfront (i                Dleconn{t:nowfunc (), c:c}) if P.idle.len () > P.maxidle {//remove exceed Conn c = P.idle.remove (P.idle.back ()).  (Idleconn). C} else {c = nil}} p.mu.unlock ()}//close exceed Conn if c! = nil {p.mu.lock () p.active-= 1 P.mu.unlock () return P.close (c)} retur N nil}//Activecount Returns the number of active connections in the Pool.func (P *pool) activecount () int {P.mu.lock ( ) Active: = P.active p.mu.unlock () return active}//Relaase releases the resources used by the Pool.func (P *pool Release () error {P.mu.lock () idle: = P.idle p.idle.init () p.closed = true p.active-= idle. Len () P.mu.unlock () for e: = idle. Front (); E! = nil; E = E.next () {p.close (E.value) ( idleconn). C)} RetuRN Nil} 

Finally, in the actual use of thrift-related settings seems to be only time-out, then the question is, under the pool, thrift the timeout time is good?

Because a client is created using each routine before using the pool, the time-out setting is very short, and the server side and client are 15 seconds. After changing the way the pool was used, the time did not change, that is, we gave the timeout to thrift to manage it, but found that the recurrent EOF I/O error occurred. After tracking found that in the case of small requests, 15 seconds is too short, the pool will be easy to appear free time more than 15 seconds of the connection, and when we get out of use, because the timeout, resulting in EOF.

After practice, the server side of the time must be long enough, we set the 8h,client end of the timeout is given to the pool management, or the pool may also have a timeout connection.

Server transportfactory: = Thrift. Newtframedtransportfactory (Thrift. Newttransportfactory ()) Protocolfactory: = Thrift. Newtbinaryprotocolfactorydefault () Servertransport, err: = Thrift. Newtserversockettimeout (BIND, thriftcalltimeout) if err! = Nil {log. EXITF ("Start Thrift RPC Error (%V)", err)}//Thrift RPC Service Handler: = NEWTHRIFTRPC () Processor: = Thrift Rpc.newrpcserviceprocessor (handler) Server: = Thrift. NewTSimpleServer4 (processor, Servertransport, Transportfactory, protocolfactory) thriftserver = append (Thriftserver, Server) log. Info ("Start Thrift RPC Listen addr:%s", bind) go server. Serve ()//Clientthriftpool = &pool. pool{Dial:func () (interface{}, error) {addr: = Conf. Myconf.thriftoauth2addr[rand. INTN (len (conf). MYCONF.THRIFTOAUTH2ADDR)] sock, err: = Thrift. Newtsocket (addr)//Client side does not set timeout if err! = Nil {log. Error ("Thrift. Newtsockettimeout (%s) error (%V) ", addr, err) return NIL, err} TF: = Thrift. Newtframedtransportfactory (Thrift. Newttransportfactory ()) PF: = Thrift. Newtbinaryprotocolfactorydefault () Client: = RPC. Newrpcserviceclientfactory (Tf.gettransport (sock), PF) If Err = client. Transport.open (); Err! = Nil {log. Error ("Client. Transport.open () error (%V) ", err) return nil, err} return client, nil}, Close:func (v int erface{}) Error {v. (*RPC). rpcserviceclient). Transport.close () return nil}, maxactive:conf. Myconf.thriftmaxactive, maxidle:conf. Myconf.thriftmaxidle, idletimeout:conf. Myconf.thriftidletimeout,}pool.idletimeout 7h//Pool maximum idle time, setting is smaller than server side, setting 8h, may also have a timeout connection