GOLANG40-line code implements a common Federation pool

Code Warehouse

Goroutine-pool

Golang's co-process management

The Golang mechanism is very convenient to solve the problem of concurrent programming, but the process is not without overhead, so we also need to limit the number of appropriate.

Code that does not use the pool (sample code is implemented with Chan and the code is verbose)

func (p *converter) upload(bytes [][]byte) ([]string, error) {    ch := make(chan struct{}, 4)    wg := &sync.WaitGroup{}    wg.Add(len(bytes))    ret := make([]string, len(bytes))    // 上传    for index, item := range bytes {        ch <- struct{}{}        go func(index int, imageData []byte) {            defer func() {                wg.Done()                <-ch            }()            link, err := qiniu.UploadBinary(imageData, fmt.Sprintf("%d.png", time.Now().UnixNano()))            if err != nil {                log.Println("上传图片失败", err.Error())                return            }            ret[index] = link        }(index, item)    }    wg.Wait()    return ret, nil}

There are two requirements that need to be fulfilled:

1. Limit the maximum number of threads, this example is 4
2. Wait for all the processes to complete, this example isbytes切片长度

Code that uses the Federation pool

func (p *converter) upload(bytes [][]byte) ([]string, error) {    ret := make([]string, len(bytes))    pool := goroutine_pool.New(4, len(bytes))    for index, item := range bytes {        index := index        item := item        pool.Submit(func() {            link, err := qiniu.UploadBinary(item, fmt.Sprintf("%d.png", time.Now().UnixNano()))            if err != nil {                log.Println("上传图片失败", err.Error())                return            }            ret[index] = link        })    }    pool.Wait()    return ret, nil}

You can see that the biggest difference is that you only need to focus on the business logic, concurrency control and wait have been taken over by the pool

Written in the last

Hopefully this article will relieve you of the burden of controlling the process.