Golang Concurrency Sequential pipeline mode

This is a creation in Article, where the information may have evolved or changed.

2013-12-14 WCDJ

This article describes how to use the pipeline to calculate concurrency, it is important to note that the go pipeline is bidirectional, and the UNIX pipeline is unidirectional.

PS: In the test, you set up a file with the suffix _test.go, and the following error will be prompted after build:

One word explanation: in go, the filename suffix _test.go is the unit test file.

For specific reference: http://segmentfault.com/q/1010000000159135

This example refers to an example of the seventh chapter of the Go Language program and adds some comments.

Package Mainimport ("Flag", "FMT" "Log" "OS" "Path/filepath" "Runtime" "strings") func main () {//Use all the machine ' s Coresruntime. Gomaxprocs (runtime. NUMCPU ()) log. SetFlags (0)//processing command line parameters algorithm, minSize, maxSize, suffixes, files: = Handlecommandline ()//Start calculation operation if algorithm = 1 {//algorithm 1 is parallel computing by creating each of the goroutine//step1: The file list is processed by the source function, and the results are returned to the pipeline//step2: Place the suffix-compliant file in the pipeline//step3: Place the file in the pipeline//STEP4: Get the result data from the pipeline sink (Filtersize (MinSize, maxSize, filtersuffixes (suffixes, source (files)))} else {//algorithm 2 is a serial calculation channel1: = Source (Files) Channel2: = filtersuffixes (suffixes, channel1) Channel3: = Filtersize (MinSize, maxSize, Channel2) sink ( CHANNEL3)}}//command-line argument resolution operation Func Handlecommandline () (algorithm int, minSize, maxSize int64,suffixes, files []string) {//Command line The parameter is bound to the corresponding variable//algorithm default is 1flag. Intvar (&algorithm, "algorithm", 1, "1 or 2")//MinSize and maxsize default to-1, indicating no limit to flag. Int64var (&minsize, "min",-1, "Minimum file size ( -1 means no minimum)") flag. Int64var (&maxsize, "Max",-1, "Maximum file size (-1 means no maximum)//suffixes suffix list default is empty var suffixesopt *string = flag. String ("Suffixes", "", "comma-separated List of File suffixes")//command line preprocessing flag. Parse () if algorithm! = 1 && Algorithm! = 2 {algorithm = 1}if minSize > maxSize && maxSize! =-1 {//Fat Alln is equivalent to PRINTLN () followed by a call to OS. Exit (1) log. Fatalln ("Minimum size must be < maximum size")}//separates the suffix list with a comma, returning the suffixes suffix slice suffixes = []string{}if *suffixesopt! =] " {suffixes = strings. Split (*suffixesopt, ",")}//Args returns the Non-flag command-line arguments//that the parameters of the non-command option are all file parameters files = flag. Args () return algorithm, minSize, maxSize, suffixes, files}//create the pipeline, process the file list and return the results to the pipeline Func source (Files []string) <-chan string {out: = Make (chan string, +) go func () {for _, FileName: = Range files {out <-filename}close (Out)} () return OU t}//the suffix file into the pipe//The file is processed in the pipeline according to the suffix slice, and the result is returned to the pipeline again/make the buffer the same size as for files to maximize Throughputfunc Filtersuffixes (suffixes []string, in <-chan StRing) <-chan String {out: = Yes (Chan string, Cap (in)) go func () {to filename: = range in {///without a limit suffix, then plug the file directly into the pipe if l En (suffixes) = = 0 {out <-filenamecontinue}//get the suffix of the file list and convert all to lowercase//Ext Returns the file name extension used by path. The extension is the suffix beginning at the final dot in the final element of path; It is the empty if there is no dotext: = Strings. ToLower (filepath. EXT (filename)) for _, Suffix: = range suffixes {if ext = = suffix {out <-filenamebreak}}}close (Out)} () return out}//will conform to the text The size of the file into the pipe//make the buffer the same size as for files to maximize Throughputfunc filtersize (minimum, maximum int64, in <-chan string) <-chan string {out: = Yes (Chan string, Cap (in)) go func () {for filename: = range in {//) No limit on file size, direct Plug the file into the pipe if minimum = =-1 && Maximum = =-1 {out <-filename//Don't do a stat call it's not neededcontinue}//use the operator Make the system interface get the file size information//Stat returns a FileInfo describing the named file. If There is an error, it would be of type *patherror/*Type FileInfo Interface {name () string//base Name of the file Size () Int64//length in Byte s for regular files; System-dependent for others mode () FileMode//File Mode bits Modtime () time. Time//Modification Time isdir () bool//abbreviation for Mode (). Isdir () Sys () interface{}//Underlying data source (can return nil)}*/finfo, err: = OS. Stat (filename) if err! = Nil {continue///Ignore files We can ' t process}size: = Finfo. Size () if (minimum = =-1 | | minimum > 1 && minimum <= size) && (maximum = = 1 | | maximum >-1 &amp ;& maximum >= size) {out <-filename}}close (Out)} () return out}//get result data from pipe func sink (in <-chan string) {for fi Lename: = range in {fmt. PRINTLN (filename)}}/*output:mba:go gerryyang$./filter_t-min 1-suffixes ". cpp". /c++11/range_for.cpp. /c++11/test. /c++11/test.cpp Routines.go. /c++11/range_for.cpp. /c++11/test.cppmba:go gerryyang$/filter_t-min 1-max-2-SUffixes ". cpp". /c++11/range_for.cpp. /c++11/test. /c++11/test.cpp routines.go JJJJ Minimum size must be < maximum size*/

Find a Go learning site on the Internet, you can stroll down: http://blog.studygolang.com/