Processing of sticky packet problem for TCP protocol in Go language

This is a creation in Article, where the information may have evolved or changed.

In the development of artificial customer service system with Golang encountered the problem of sticky bag, then what is sticky bag it? For example, we and the Client Convention data interaction format is a JSON-formatted string:

{"Id": 1, "Name": "Golang", "message": "Message"}

When the client sends the data to the server, if the server does not receive it in time, and the client sends a data, the server receives two consecutive strings, such as:

{"Id": 1, "Name": "Golang", "message": "Message"} {"Id": 1, "Name": "Golang", "message": "Message"}

If the receive buffer is full, then it is possible to receive half of the JSON string, Jiangzi, how to use JSON decoding it? That's a headache. The following Golang simulated the production of the next adhesive packet.

Note: The following code can be run on Golang 1.3.1, if you find a problem can contact me.

Sticky Pack Example

Server.go

Sticky Pack Problem Demo Service-side package Mainimport ("FMT" "NET" "OS") Func main () {netlisten, err: = Net. Listen ("TCP", ": 9988") checkerror (Err) defer netlisten.close () Log ("Waiting for Clients") for {conn, err: = Netlisten.accept () if err! = Nil {continue}log (conn. Remoteaddr (). String (), "TCP Connect Success") go Handleconnection (conn)}}func handleconnection (conn net. Conn) {buffer: = make ([]byte, 1024x768) for {n, err: = Conn. Read (buffer) if err! = Nil {LOG (conn. Remoteaddr (). String (), "Connection error:", err) Return}log (Conn. Remoteaddr (). String (), "Receive data length:", N) Log (Conn. Remoteaddr (). String (), "Receive data:", Buffer[:n]) LOG (Conn. Remoteaddr (). String (), "Receive data string:", String (Buffer[:n]))}}func Log (v. ... interface{}) {FMT. Println (v ...)} Func checkerror (err error) {if err! = Nil {fmt. fprintf (OS. Stderr, "Fatal Error:%s", err. Error ()) OS. Exit (1)}}

Client.go

Sticky Pack Problem Demo Client Package Mainimport ("FMT" "NET" "OS" "Time") Func sender (conn net). Conn) {for i: = 0; i <; i++ {words: = "{\" id\ ": 1,\" name\ ": \" golang\ ", \" message\ ": \" message\ "}" Conn. Write ([]byte (words))}}func main () {server: = "127.0.0.1:9988" tcpaddr, err: = Net. RESOLVETCPADDR ("TCP4", server) if err! = Nil {fmt. fprintf (OS. Stderr, "Fatal Error:%s", err. Error ()) OS. Exit (1)}conn, err: = Net. DIALTCP ("TCP", nil, tcpaddr) if err! = Nil {fmt. fprintf (OS. Stderr, "Fatal Error:%s", err. Error ()) OS. Exit (1)}defer Conn. Close () fmt. PRINTLN ("Connect success") go Sender (conn) for {time. Sleep (1 * 1e9)}}

View the server output after running:

Golang Sticky Bag Problem Demo

Can see the JSON format of the strings are glued together, there is a faint sadness-the headache of the matter again.

Causes of sticky bag production

About the cause of the sticky bag There are many related instructions on the Internet, the main reason is that the TCP data transfer mode is the flow mode, in the long connection can be carried out many times to receive and send. If you want to drill down, you can look at what the TCP protocol is about. It is recommended that the bird's private dishes are very easy to understand.

Solutions for Sticky packs

There are two main ways of doing this:

1, the client is sent once disconnected, need to send data when the connection again, typically such as HTTP. The following is a demonstration of this process with Golang, there is really no sticky packet problem.

Client-side code that demonstrates a disconnected package mainimport ("FMT" "NET" "OS" "Time") that sends data once, func main () {server: = "127.0.0.1:9988" for I: = 0; i < 10000; i++ {tcpaddr, err: = Net. RESOLVETCPADDR ("TCP4", server) if err! = Nil {fmt. fprintf (OS. Stderr, "Fatal Error:%s", err. Error ()) OS. Exit (1)}conn, err: = Net. DIALTCP ("TCP", nil, tcpaddr) if err! = Nil {fmt. fprintf (OS. Stderr, "Fatal Error:%s", err. Error ()) OS. Exit (1)}words: = "{\" id\ ": 1,\" name\ ": \" golang\ ", \" message\ ": \" Message\ "}" conn. Write ([]byte (words)) Conn. Close ()}for {time. Sleep (1 * 1e9)}}

Service-side code reference above demonstrates the service-side code of the sticky-package generation process

2, Baotou + data format, according to Baotou information read to the data needed to analyze. Forms such as:

Golang Sticky Packet problem header definition

When you read data from a data stream, you can fetch the data you want based on the header and the length of the data. This is actually the usual protocol (protocol), but this data transfer protocol is very simple, unlike TCP, IP and other protocols have more definitions. In the actual process, the Protocol class or protocol file is usually defined to encapsulate the packet-and-package process. The following code demonstrates the process of marshaling and wrapping:

Protocol.go

Communication protocol processing, the process of processing packets and packages mainly package Protocolimport ("bytes" "Encoding/binary") const (Constheader = "www.01happy.com" Consth Eaderlength = 15ConstSaveDataLength = 4)//packet func Packet (message []byte) []byte {return append (append ([]byte] (Constheader ), Inttobytes (len (message)) ...), message ...)} Unpacking func Unpack (buffer []byte, Readerchannel Chan []byte) []byte {length: = Len (buffer) var i intfor i = 0; i < length; i = i + 1 {if length < I+constheaderlength+constsavedatalength {break}if string (buffer[i:i+constheaderlength]) = = Const Header {messagelength: = Bytestoint (Buffer[i+constheaderlength:i+constheaderlength+constsavedatalength]) if length < I+constheaderlength+constsavedatalength+messagelength {break}data: = buffer[i+constheaderlength+ Constsavedatalength:i+constheaderlength+constsavedatalength+messagelength]readerchannel <-Datai + = Constheaderlength + constsavedatalength + messagelength-1}}if i = = length {return make ([]byte, 0)}return buffer[i:]}//shaping Convert to byte func inttobytes (n iNT) []byte {x: = Int32 (n) Bytesbuffer: = bytes. Newbuffer ([]byte{}) binary. Write (Bytesbuffer, Binary. Bigendian, X) return bytesbuffer.bytes ()}//bytes are converted into shaping func bytestoint (b []byte) int {bytesbuffer: = Bytes. Newbuffer (b) var x int32binary. Read (Bytesbuffer, Binary. Bigendian, &x) return int (x)}

Tips: In the process of unpacking should pay attention to the problem of array out of bounds, in addition to pay attention to the uniqueness of Baotou.

Server.go

The service-side unpacking process Package Mainimport ("./protocol" "FMT" "NET" "OS") Func main () {netlisten, err: = Net. Listen ("TCP", ": 9988") checkerror (Err) defer netlisten.close () Log ("Waiting for Clients") for {conn, err: = Netlisten.accept () if err! = Nil {continue}log (conn. Remoteaddr (). String (), "TCP Connect Success") go Handleconnection (conn)}}func handleconnection (conn net. Conn) {//declares a temporary buffer to store truncated data tmpbuffer: = Make ([]byte, 0)//declares a pipeline to receive unpacked data readerchannel: = Make (chan []byte, +) go Reader (readerchannel) Buffer: = Make ([]byte, 1024x768) for {n, err: = conn. Read (buffer) if err! = Nil {LOG (conn. Remoteaddr (). String (), "Connection error:", err) Return}tmpbuffer = protocol. Unpack (Append (Tmpbuffer, buffer[:n] ...), Readerchannel)}}func reader (Readerchannel Chan []byte) {for {select {case data] : = <-readerchannel:log (String (data))}}}func Log (v. ... interface{}) {FMT. Println (v ...)} Func checkerror (err error) {if err! = Nil {fmt. fprintf (OS. Stderr, "Fatal Error:%s", err. Error ()) OS. Exit (1)}}

Client.go

The client sends a packet Mainimport ("./protocol" "FMT" "NET" "OS" "Time") Func sender (conn net. Conn) {for i: = 0; i <; i++ {words: = "{\" id\ ": 1,\" name\ ": \" golang\ ", \" message\ ": \" message\ "}" Conn. Write (protocol. Packet ([]byte (words)))}fmt. PRINTLN ("Send Over")}func main () {server: = "127.0.0.1:9988" tcpaddr, err: = Net. RESOLVETCPADDR ("TCP4", server) if err! = Nil {fmt. fprintf (OS. Stderr, "Fatal Error:%s", err. Error ()) OS. Exit (1)}conn, err: = Net. DIALTCP ("TCP", nil, tcpaddr) if err! = Nil {fmt. fprintf (OS. Stderr, "Fatal Error:%s", err. Error ()) OS. Exit (1)}defer Conn. Close () fmt. PRINTLN ("Connect success") go Sender (conn) for {time. Sleep (1 * 1e9)}}

Run this program to see the server side good to get to the desired JSON format data. Full code demo download: Golang Sticky pack Problem Solving example

At last

The two methods shown above apply to different scenarios. The first method is more suitable for passive scenarios, such as opening a Web page where the user is requested to process the interaction. The second method is suitable for the type of the active push, such as instant chat system, because it is necessary to push the message to the user immediately, it is unavoidable to keep the long connection, this way.

Reprint Please specify: Happy programming»golang TCP socket sticky packet problem and processing