This is a creation in
Article, where the information may have evolved or changed. I used to think that the performance overhead of using Goroutine and channel was largely negligible-especially compared to the performance overhead of IO-but recently I did an experiment that actually verified the next. I'm making a [toy-related database] (HTTPS://GITHUB.COM/ROBOT-DREAMS/ZDB2) for [My Course Project] (https://bradfieldcs.com/courses/databases/). At first, I loaded the data table from the CSV file, and later I needed to add a binary table structure. Unfortunately, the first attempt (loading a binary table) is far worse than loading a CSV file. "$./csv_scan_benchmark-path table.csvdone scanning 20000263 Records after 15.69807191s$./binary_scan_benchmark- Path Table.btdone scanning 20000263 records after 27.01220384s "scanned the binary table twice as slow! This is too unusual because the binary table structure is simpler and does not require string conversions. Fortunately, I used [a few good go performance testing tools] (HTTPS://BLOG.GOLANG.ORG/PROFILING-GO-PROGRAMS) to study the problem. Scan the CSV table CPU profile looks reasonable, most of the time wasted on the IO-related system calls;! [] (https://raw.githubusercontent.com/studygolang/gctt-images/master/goroutine-channel/1_ R52ht8a94yg06h6ndnj0mq.png) > Scan the CPU profile of the CSV table but the CPU profile that scanned the binary table looks unreasonable; only a very small portion of the time is spent on IO. [] (https://raw.githubusercontent.com/studygolang/gctt-images/master/goroutine-channel/1_ W6cww8qfydaeszseeb3vya.png) > Scan the CPU profile of the binary table (first experiment results) Originally, this unreasonable CPU profile, because I used the concurrent prototype of go. At the time, I wanted to decouple producers and consumers with Goroutine and channel, simplifying the code structure of scanner. Create scanner start producer Goroutine, used to implement IO operation/decode, return results to channel: "' Gofunc newbinaryscan (Path string) (*binaryscan, error) {... go s . Producer () return S, Nil}func (S *binaryscan) producer () {for {record, err = S.readanddecoderecord (S.bufferedreader) S.resultchan <-&result{record, err}if err! = nil {return}}} ' ' Consumer goroutine, by repeatedly calling ' NextRecord ' to get results, simply from result ch Annel read data: ' Gofunc (S *binaryscan) NextRecord (Record, error) {result: = <-s.resultchan:return Result.record, resu Lt.err} "However, from the CPU profile, in this step, Goroutine spent a lot of time blocking the channel operation, go in the runtime wasted a lot of resources on the dispatch/concurrency prototype. I rewrote the binary form scan snippet and did all the work directly on the consumer goroutine: "Gofunc newbinaryscan (Path string) (*binaryscan, error) {...//No more Producer.return S, Nil}func (S *binaryscan) NextRecord () (Record, error) {return S.readanddecoderecord (s.bufferedreader } "However, this small change has a very big impact on performance. The following is the updated CPU profile, which looks more reasonable than before:! [] (Https://raw.githubusercontent.com/studygolang/gctt-iMages/master/goroutine-channel/1_cxcsd3eamiktt4grtsp2ba.png) > Scan the CPU profile of the binary table (improved) The following is the updated benchmark result: "$ ./binary_scan_benchmark-path Ratings.btdone scanning 20000263 Records after 8.160765247s "is twice times faster than reading a CSV file, 3 times times faster than the first experiment, That's the same! I always thought that reasonable use of goroutine and channel can make the code concise, in most cases, it is impossible to be the root cause of performance problems. But this experiment gave me a wake-up, that is, go super concurrency model can not be abused casually. **[Revised]** thanked Raghav for his advice, and he provided more benchmark test examples! -Add ' context ' to ' select '. Context ', reduce producer timeout: 59.4s-in ' select ' Add ' context. Context ', reducing the consumer timeout: 58.0s-using a buffer of 1 channel:23.5s-using a buffer of 1000 channel:17.4s-in the ' done ' channel to remove ' select ' ( The ' done ' channel is not in the code snippet above): 19.9s-' Select ' is removed from the ' done ' channel while using a buffer of 1000 channel:14.3s It seems that we have learned one more thing: Channel buffer size and ' select ' The complexity of the statement has a significant impact on performance. **[proofreading 2]** Thanks to [Stuart Carnie] (https://medium.com/@stuartcarnie) Recommendation, he recommends sending batch records simultaneously through the channel, rather than sending only one at a time! Here are the benchmark results I got using a different batch size:-1:28.83s-10:12.36s-100:8.92s-1,000:8.68s-10,000:8.74s-100,000:9.32s It seems, as Stuart , the number of channel write operations is reduced by 3 orders of magnitude, which also has a significant impact here.
via:https://medium.com/@robot_dreams/goroutines-and-channels-arent-free-a8684f3b6560
Author: Elliott Jin Translator: arisaries proofreading: polaris1119
This article by GCTT original compilation, go language Chinese network honor launches
This article was originally translated by GCTT and the Go Language Chinese network. Also want to join the ranks of translators, for open source to do some of their own contribution? Welcome to join Gctt!
Translation work and translations are published only for the purpose of learning and communication, translation work in accordance with the provisions of the CC-BY-NC-SA agreement, if our work has violated your interests, please contact us promptly.
Welcome to the CC-BY-NC-SA agreement, please mark and keep the original/translation link and author/translator information in the text.
The article only represents the author's knowledge and views, if there are different points of view, please line up downstairs to spit groove
395 reads ∙1 likes
