This is a creation in Article, where the information may have evolved or changed. Update log:
Vitess code updated to 2012.4.8
The new code adds the conditional variables for synchronization, and queues no longer use channel to synchronize when there are no idle resources (the code can be ported easily by classmates in other programming languages),
Use condition variable instead. No longer use MU. Rlock, unified use lock, no longer tangled. The overall code is a lot clearer.
In order to further improve performance and code reuse, Vitess also provides a common pool management, roundrobin.go in the implementation of a common resource pool, easy to manage the total number of resources in the pool, time-out.
First course: What role does roundrobin play in the entire vitess architecture? Personally feel the importance of roundrobin in the vitess can also be counted as a confraternity of several elders, as the core of the other pool, such as Cache_pool.go is Roundrobin + memcache client implementation, and Conn_ The pool.go is a connection pool implemented with Roundrobin. First look at the basic principles of roundrobin.go, the usual, or from the data structure to start // Factory is a function so can be used to create a resource.
Type Factory func () (Resource, error) // factory methods, resource creation functions, such as connection
// every resource needs to suport the resource interface.
Type Resource Interface{
Close ()
IsClosed () BOOL
}
It is important to note that objects managed using the Roundrobin resource pool must implement the Resouce interface.
Why do you want to implement these two interfaces? Because the resource pool needs to know how close the resource is timed out,
and which resources are idle and which are closed.
// Roundrobin allows you and use a pool of resources in a round robin fashion.
Type Roundrobin struct{
// mu Controls Resources & Factory
// Use Lock to modify, Rlock otherwise
Mu sync. Rwmutex
Resources Chan Fifowrapper // use Chan to simulate a FIFO queue, first to serve
Factory Factory
// Use sync/atomic to access the following VARs
Size Int64 // LQ: Total size of the pool
Waitcount Int64 // LQ: How many people are waiting for free resources in the pool
WaitTime Int64 // LQ: How long does it take to wait for free resources
IdleTimeout Int64 // LQ: Maximum allowed resource idle time, more than close idle resource
}
Type Fifowrapper struct{
Resource Resource
Timeused time.timeused // LQ: Used to control timeout, the initial value is the time the resource last entered the pool (see put function)
}
I'm sorry it's almost the first episode of the TV show. The Golang inside the heavy component channel, a simple understanding can be thought that channel is a
Supports the synchronization queue of the producer consumer model and is a queue that does not need to be destroyed, and Golang automatically reclaims the channel that is no longer used when it is garbage collected.
Actually the pool is also the producer consumer model. Therefore, the method of providing production resources externally is required, that is, the factory interface above. This can be understood as a simple function pointer in C language. The implementation logic of Roundrobin is this, if there are resources within the pool, you can get a success, if not, but there is capacity, then create a resource with factory. If there are no idle resources in the pool, then silly, know that there are free resources available locations. To understand the operational status of a resource pool, it also records how many customers are waiting. The total amount of time to wait, with these can easily assess the effect of roundrobin. The source code is also written according to this idea.
Not much to say, on the code, the details are shown in the annotated code comment
// Get would return the next available resource. If None is available, and capacity
// Have not been reached, it would create a new one using the factory. Otherwise,
// it'll indefinitely wait till the next resource becomes available.
Func (self *roundrobin) Get () (Resource resource, err error) {
returnSelf. Get( true)
}
// Tryget would return the next available resource. If None is available, and capacity
// Have not been reached, it would create a new one using the factory. Otherwise,
// It'll return nil with no error.
Func (self *roundrobin) tryget () (Resource resource, err error) {
returnSelf. Get( false)
}
Func (self *roundrobin) Get(Wait BOOL) (Resource resource, err error) {
Self.mu.Lock ()
Defer Self.mu.Unlock ()
// Any waits in this loop would release the lock, and it'll be
// reacquired before the waits return.
for{
Select{
CaseFW: = <-self.resources:
// Found a free resource in the channel
ifSelf.idletimeout > 0&& fw.timeUsed.Add (self.idletimeout). Sub (time. Now ()) < 0{
// resource have been idle for too long. Discard & Go for next.
Go Fw.resource.Close ()
self.size--
Continue
}
returnFw.resource, Nil
default:
// resource channel is empty
ifSelf.size >= Int64 (Cap (self.resources)) {
// The pool is full
ifWait {
Start: = time. Now ()
Self.available.Wait () There is no spare resources, wait for it, not as good as the code of the previous version of the natural AH
Self.recordwait (Start)
Continue
}
returnNil, Nil
}
// Pool isn't full. Create a resource.
ifResource, err = Self.waitforcreate (); Err = = Nil {
// Creation successful. account for this by incrementing size.
self.size++
}
returnResource, err
}
}
Panic " Unreachable ")
}
Func (self *roundrobin) recordwait (start time. Time) {
self.waitcount++
Self.waittime + = time. Now (). Sub (Start)
}
// LQ: Here the increment and decrement should be redundant, do not see what the author is what the purpose, and surprise group have what relationship
// to avoid self.factory () time-consuming, unlock is necessary to perform self.factory
Func (self *roundrobin) waitforcreate () (Resource resource, err error) {
// Prevent thundering herd:increment size before creating resource, and decrement after.
self.size++
Self.mu.Unlock ()
Defer func () {
Self.mu.Lock ()
self.size--
}()
returnSelf.factory ()
}
In the code comments can be seen, in order to avoid the surprise group effect, the method used here is first increment, I do not quite understand, why this can avoid the surprise group effect.
Also please the familiar friends to enlighten.
After reading the get find queued waiting is so natural, a line of code things. Looking at the put function again, we will find that the wake is so concise.
// Put'll return a resource to the pool. You must return every resource to the pool,
// Even if it ' s closed. If A resource is closed, Put would discard it. Thread Synchronization
// between Close () and IsClosed () is the caller ' s responsibility.
Func (self *roundrobin) Put (resource Resource) {
Self.mu.Lock ()
Defer Self.mu.Unlock ()
Defer self.available.Signal () // LQ: The brother in line should wake up.
ifSelf.size > Int64 (Cap (self.resources)) {
Go resource. Close ()
self.size--
} Else ifResource. IsClosed () {
self.size--
} Else{
Self.resources <-Fifowrapper{resource, time. Now ()}
}
}
The next series will analyze the implementations of several other pools that depend on Roundrobin, and then analyze how the various functions of vitess are implemented. Rest and rest for a while.
