Use the go language (golang) to write a simple Crawler

I wrote a crawler using Scala last time. Recently, when I learned the go language, I transplanted the crawler written in Scala with go. The Code is as follows:

package mainimport ("fmt""io/ioutil""net/http""regexp")var (ptnIndexItem    = regexp.MustCompile(`<a target="_blank" href="(.+\.html)" title=".+" >(.+)</a>`)ptnContentRough = regexp.MustCompile(`(?s).*<div class="artcontent">(.*)<div id="zhanwei">.*`)ptnBrTag        = regexp.MustCompile(`<br>`)ptnHTMLTag      = regexp.MustCompile(`(?s)</?.*?>`)ptnSpace        = regexp.MustCompile(`(^\s+)|( )`))func Get(url string) (content string, statusCode int) {resp, err1 := http.Get(url)if err1 != nil {statusCode = -100return}defer resp.Body.Close()data, err2 := ioutil.ReadAll(resp.Body)if err2 != nil {statusCode = -200return}statusCode = resp.StatusCodecontent = string(data)return}type IndexItem struct {url   stringtitle string}func findIndex(content string) (index []IndexItem, err error) {matches := ptnIndexItem.FindAllStringSubmatch(content, 10000)index = make([]IndexItem, len(matches))for i, item := range matches {index[i] = IndexItem{"http://www.yifan100.com" + item[1], item[2]}}return}func readContent(url string) (content string) {raw, statusCode := Get(url)if statusCode != 200 {fmt.Print("Fail to get the raw data from", url, "\n")return}match := ptnContentRough.FindStringSubmatch(raw)if match != nil {content = match[1]} else {return}content = ptnBrTag.ReplaceAllString(content, "\r\n")content = ptnHTMLTag.ReplaceAllString(content, "")content = ptnSpace.ReplaceAllString(content, "")return}func main() {fmt.Println(`Get index ...`)s, statusCode := Get("http://www.yifan100.com/dir/15136/")if statusCode != 200 {return}index, _ := findIndex(s)fmt.Println(`Get contents and write to file ...`)for _, item := range index {fmt.Printf("Get content %s from %s and write to file.\n", item.title, item.url)fileName := fmt.Sprintf("%s.txt", item.title)content := readContent(item.url)ioutil.WriteFile(fileName, []byte(content), 0644)fmt.Printf("Finish writing to %s.\n", fileName)}}

The number of lines of code is somewhat higher than that of Scala, mainly due to the following reasons:
1 golang emphasizes code writing standards, or code formats, which are fixed and even troublesome in many places. For example, even if the if statement is true, there is only one sentence. According to the code specification, three lines with braces should be written. In Scala and many other languages, one line will work;
2 golang's strings package and Regexp package provide methods that are not particularly useful, especially compared with Scala, it feels much more comfortable to use Scala's regular expressions and string processing;
3 Scala crawlers use the NLP classes and methods in the scala standard library. Although they are not made up of syntax, they are used as syntactic sugar. Many methods are related to functional programming, golang's functional programming has not been carefully studied.

Of course, the crawler of golang also has an advantage, that is, the Compilation speed is very fast, and the execution speed does not reflect the advantage in the current writing; the goroutine of golang is not used here, this code will be continuously improved in the future.