Golang + PROTOBUF Construction Communication protocol

This is a creation in Article, where the information may have evolved or changed.

Golang is very convenient for writing network communication, where message communication may use cross-language protobuf. This article describes how to construct a Golang communication protocol using PROTOBUF.

(1) Protobuf needs Protoc, this can be installed as long as the GitHub download compiles. The Debian department can also be installed via apt.
(2) The second is Go plug-in, the same download source code, the execution of Go build and go install can be installed under the $gopath path, and then add the folder $path can be used.
(3) Write the proto file, please refer to the official documentation for details. It looks like this:

// poster.protopackage poster;message Mail {    required bytes type = 1;    required bytes md5 = 2;    required bytes path = 3;}

The above is the message format I used in the socket, define the TYPE,MD5 value of the message, the path path also has the size of the file, it can be used for the next MD5 test, the continuation of the function, and according to the message type to customize different signals. It is not necessary to use bytes here, but I will define the length of each field separately in my code, and byte is more intuitive to me.

Then, it becomes a library of Golang import into its own code, the resulting command is:

protoc --go_out=. poster.proto

Then I get a poster.pb.go file, and then I can import the library in my Code. But this library only provides the ability to define formats and read information. To package the information into a protobuf or unpacking requires another library to support, which can be achieved by the following command:

go get github.com/golang/protobuf/proto

About the most basic marshal and unmarshal method I do not explain, but here to stick out the struct and protoc with the use of examples, but do not copy and paste, here is a utils library is written by itself, mainly to the int64 into byte method. Also note that since I have fixed three fields in the proto file as byte types, I will first convert the original type to byte type, but the struct is the appropriate type for easy invocation:

// 这里定义长度主要是因为socket通讯等消息包假设采用固定长度来接收const (    typeSize   = 8    md5Size    = 32    pathSize   = 80    bufferSize = 126)type msgBuffer struct {    bufType  int    bufMd5   string    bufPath  string}// func (msgBuf *msgBuffer) NewMsg() []byte {    bufType := make([]byte, typeSize)    bufMd5 := make([]byte, md5Size)    bufPath := make([]byte, pathSize)    copy(bufType, utils.Int64ToByte(int64(msgBuf.bufType)))    copy(bufMd5, []byte(msgBuf.bufMd5))    copy(bufPath, []byte(msgBuf.bufPath))    dataText := &poster.Mail {        Type: bufType,        Md5 : bufMd5,        Path: bufPath,    }    data, err := proto.Marshal(dataText)    checkError(err)    return data}

This data is a byte array that can be transferred randomly. Finally, the receiver will be able to unpack the message and invoke it in a friendly manner, as shown below. (Note utils. The firstnull automatically intercepts the character segment starting from the first byte to the last corresponding ASCII non 0, because if fixed length is set and value is not so long, it will be filled with null.

func msgParser(data []byte) *msgBuffer {    dataText := &poster.Mail{}    err := proto.Unmarshal(data, dataText)    checkError(err)    msgType := utils.ByteToInt64(dataText.GetType())    msgMd5 := string(dataText.GetMd5())    msgPath := string(dataText.GetPath()[:utils.FirstNull(dataText.GetPath())])    msg := &msgBuffer{int(msgType), msgMd5, msgPath}    return msg}

The whole process is simple, the most troublesome place is probably the protobuf is not so convenient.

Finish