Golang read MONGOB data written to SQL Server

Source: Internet
Author: User
Tags email string
This is a creation in Article, where the information may have evolved or changed.

Today there is a job that reads data from MongoDB and then puts it into SQL Server, and of course this program must be done with go.

Prepare the mongdb third-party driver package first Http://labix.org/mgo

ODBC third-party driver package Https://bitbucket.org/miquella/mgodbc

The conditions for reading MongoDB data are based on the date range, as well as the string condition. On the code.

  

Package Mainimport (_"Bitbucket.org/miquella/mgodbc" "Database/sql" "Encoding/json" "FMT" "Labix.org/v2/mgo" "labix.org/v2/mgo/b Son "OS" "Runtime" "Time") type serverslice struct {Servers []string sqlconn string Start string End string}type user struct {UserName string"UserName"Password String"Password"Email String"Email"Phone String"Phone"subtime time. time"Subtime"}varconfig ServerslicevarWorker =Runtime. NUMCPU ()//Initialize Configurationfunc init () {file, _:= OS. Open ("Config.json") defer file. Close () BUF:= Make ([]byte, 2048) N, _:=file. Read (BUF) Err:= json. Unmarshal (Buf[:n], &config)ifErr! =Nil {panic (err) fmt. PRINTLN (Err)}}func main () {runtime. Gomaxprocs (runtime. NUMCPU ())varChanuser =Make (chan user)//Mark CompleteDones: =Make (Chan struct{}, worker) go Readmongodb (chanuser) forI: = 0; I < worker; i++{Go writesql (Chanuser, Dones)} awaitforcloseresult (Dones) fmt. Println (Complete)}//Reading MongoDB DataFunc Readmongodb (Chanuser chan<-user) {Start, _:=Time . Parse (Layout, config. Start) End, _:=Time . Parse (Layout, config. END) for_, Server: =Range CONFIG. Servers {session, err:=MgO. Dial (server)ifErr! =Nil {fmt. Println ("Open", Server, "failed") Panic (ERR)} defer session. Close ()varRegex =Bson. regex{} regex. Pattern= "^139.*"//the query condition is that Start<=subtime<=end;email=nil;phone does not start with 139        varquery = Bson. m{"Subtime": Bson. m{"$gte": Start, "$lte": End}, "Email": Nil,"Phone": Bson. m{"$not": Regex}} //Sqlreader SimilarITER: = Session. DB ("DB"). C ("Users"). Find (query). Iter () Message:=user{} forIter. Next (&message) {Chanuser<-message}//Close ChannelClose (Chanuser)}}//write to SQL ServerFunc writesql (chanuser <-chan user, Dones chan<-struct{}) {con, err:= SQL. Open ("Mgodbc", CONFIG. sqlconn)ifErr! =Nil {fmt. PRINTLN (ERR)return} defer con. Close ()//con. Exec (Preparesql)     forMessage: =Range Chanuser {//Assembling SQL        varsql =FMT. Sprintf (insertsql, message. UserName, message. Password, message. Email, message. Phone, message. Subtime.format (Layout) _, Err=con. Exec (SQL)ifErr! =Nil {fmt. PRINTLN (Err)}} dones<-struct{}{}}/*wait for operation to complete*/func awaitforcloseresult (dones<-Chan struct{}) {     for {        <-dones worker--ifWorker <= 0 {            return        }    }}varLayout = "2006-01-02 15:04:05"varInsertsql string =' INSERT into [dbo].                [User]                ([UserName], [Password], [Email], [Phone] , [Subtime]) VALUES ('%s ', '%s ', '%s ', '%s ', '%s '); `

The corresponding configuration file Config.json

{    "Servers":    [        "127.0.0.1:27017"    ],    "sqlconn": "Driver={sql Server}; Server=localhost; UID=AA; Pwd=bbb;database=auth ",    " start ":" 2013-05-17 00:00:00 ",    " End ":" 2013-06-01 00:00:00 " }

Related Article

Contact Us

The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion; products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the content of the page makes you feel confusing, please write us an email, we will handle the problem within 5 days after receiving your email.

If you find any instances of plagiarism from the community, please send an email to: info-contact@alibabacloud.com and provide relevant evidence. A staff member will contact you within 5 working days.

A Free Trial That Lets You Build Big!

Start building with 50+ products and up to 12 months usage for Elastic Compute Service

  • Sales Support

    1 on 1 presale consultation

  • After-Sales Support

    24/7 Technical Support 6 Free Tickets per Quarter Faster Response

  • Alibaba Cloud offers highly flexible support services tailored to meet your exact needs.