A simple TCP communication framework with GO implementation

This is a creation in Article, where the information may have evolved or changed.

After contacting go, the Go network support is very like. Go implements the syntax to maintain synchronization semantics, but does not sacrifice too much performance, the bottom of the same use of Io path multiplexing, such as the use of Linux under the Epoll, under Windows using IOCP.

However, in the development of the service-side program, many are passive trigger, are sent by the client request to be processed. is inherently a event-based program. In go, because concurrency is part of the language, Goroutine, channel, and so on, makes it easy for programmers to convert between synchronous and asynchronous when they implement functionality.

Because of my own needs, I made a simple package for the service side of the event-based scene. See here for specific code.

Design principles

Because of the IO mechanism of go and the native support of concurrent primitives, coupled with the encapsulation of the network API, programmers can easily implement an efficient server or client program. The general implementation is to call net. Listen ("TCP4", address) gets a net. Listener, and then infinite loop calls net. Listener.accept, you can then get a net.conn, you can call the Net.conn interface to set the send and receive buffer size, you can set the KeepAlive and so on. Because of the duplex nature of TCP, it is possible to target one net. Conn can specifically start a goroutine to the infinite loop to receive the data sent to the end, and then unpack and so on.

My idea is to make a thin package on the basis of this simple implementation, to make it as streamlined as possible, but without losing flexibility. Want to be able to adapt to different protocols, to the user to create the minimum constraints.

Session Object

The object is to net. Conn is a simple package that can be swnet. Server.acceptloop, you can also create new objects through swnet.newsession, which is typically used in a client context. After you get the session object, you can call the Start method to start working. It also exposes a method called start because there may be some requirements under the server, such as an ACL set for IP, so the start behavior is given to the consumer to decide how to invoke it. However, it is important to note that if the user does not want to start, the user is responsible for their own close off, otherwise it will cause resource leaks.

After start, two goroutine are started, one dedicated to receiving the data sent to the end, one dedicated to sending data to the peer. To send data to the peer, you can use the Asyncsend method, which queues the data to be sent to the Send channel. The reason to use the channel here is because in the context of the server, it is necessary to queue the data sent, to prevent the transmission of fast, but the end of the reception is very slow, or too many calls to the Asyncsend method, resulting in the accumulation of too much data, increased memory pressure. I think it is reasonable to control the transmission rate by channel. At the same time, the method can be used to modify the channel length, one is to call NewSession when passing the specified size, The second is to call the session.setsendchannelsize to set the size, but note that this method must be called before start to complete, or an error will result. The reason for this is also because there is no need to dynamically change the Send channel size.

If the Send channel is full, the Asyncsend method returns errsendchanblocking. Adding this error type is also due to the above design. Without returning this error, there is no way for the user to get the chance to deal with the problem. If the user gets the error, they can try to analyze the cause of the problem themselves, or they can try to send it in a loop or discard the sending data directly. In short, it allows users to get their own treatment opportunities.

If the session object is already close, calling Asyncsend will return a errstoped error. In addition, since asyncsend is queuing data to send the channel, it is the responsibility of the user to ensure that the data sent is not modified until the transmission is complete.

If the data is sent to fail, or for other reasons, my implementation is directly rude to close the session.

Also, there may be some use case scenarios, will send a larger packet, such as 64K size, or 32K size of data, and so on to avoid repeated application of memory, the session is hereby added Setsendcallback method. You can set a callback function to invoke the callback after the send is complete, giving the user the opportunity to reclaim the data object, such as with sync. Pool use. Although I did not have much effect on my own test.

To make it easier for users to set up some net. The conn parameter adds a Rawconn method that can be obtained to net. An instance of Conn. This is actually quite tangled. Because exposing this internal resource, it gives the user a very large degree of flexibility. It can directly bypass the session's sending channel and play its own. But for the convenience of users, I still do. The user shall bear the corresponding responsibility. In fact, it can also be like net. HTTP adds a hijack method that lets the user take over net. Conn, play your own.

Many of the Set/get methods in the session are not locked. On the one hand, many operations are done before start, or the data for get is not so tight.

Sometimes, if a session is closed, you may need to know this behavior. The Setclosecallback method is provided so that the method can be set. It doesn't matter if you don't set it. Calling Closecallback ensures that it is called only once.

Protocol Serialization Abstraction

Because one of the goals is to be able to isolate specific protocol formats. So the protocol was abstracted. You only need to implement the Packetprotocol interface:

//Packetreader is used to unmarshal a complete packet from buffType PacketreaderInterface {    //Read data from Conn and build a complete packet. //How to-read from Conn are up to you .    You can set the Read timeout or other option. //If Buff ' s capacity is small, you can make a new buff and then return it,//So can reuse to reduce memory overhead.Readpacket (Conn net. Conn, Buff []byte) (Interface{}, []byte, error)}//Packetwriter is used to marshal packet into buffType PacketwriterInterface {    //Build a complete packet. If Buff ' s capacity is too small, you can make a new one//and return it to reuse.Buildpacket (PacketInterface{}, Buff []byte) ([]byte, error)//How to write the data to Conn are up to you . So can set write timeout or other option.Writepacket (Conn net. Conn, Buff []byte) Error}//Packetprotocol just a composite interfaceType PacketprotocolInterface{Packetreader Packetwriter}

That is to implement packetreader/packetwriter two interfaces. In order for memory to be reused as much as possible to reduce memory pressure, it is necessary to return a slice in the return value of the Readpacket method and the Buildpacket method. The framework passes a default-sized slice to both methods at the first call, and if the capacity is insufficient, the consumer can re-establish the slice himself and then return the slice after writing the data. Use this returned slice the next time you are practical.

Where the Readpacket method is called in a goroutine that is dedicated to receiving data. The implementation can read itself according to its own policy because it is passed in net. Conn, so the user can set the I/O Timeout themselves. It is the responsibility of the implementing person to return a complete request package. If there is an error in the middle, it is necessary to return an error. When an error is found, the session is closed. The reason for this is that when reading or building a request package fails, it could be a data error, a link error, or other reason, in short, the individual does not think it is necessary to continue processing in this case, directly close the link. And here's another thing to be aware of, the data in the request package returned if there is data that contains the slice type, it is recommended that you reassign a slice and then copy it from the buff and try not to reuse the buff slices, otherwise you may incur additional bugs.

The Buildpacket method is called in a goroutine that is dedicated to handling the send. When the sending Goroutine receives the packet, the buildpacket is called, and the implementation can serialize it in its own private format. Similarly, the buff is not enough, you can reconstruct a buff yourself, then populate the data and return the buff.

Writepacket is the need to personalize the delivery to the person who realizes it. It is possible for the implementing person to set I/O Timeout.

Request Packet Routing

Based on the implementation of event-based, there is always the thing to do is to forward a request packet to the corresponding processing function. However, how to do it depends on the specific use case scenario and implementation. So what I do here is very simple, which is to define a Packethandler interface:

// Packethandler is used to process packet this recved from remote session Interface {    // when got a valid packet from Packetreader, you can dispatch it.    Interface {})}

The user can implement the corresponding handle method. When Goroutine receives data from the end and calls Packetreader.readpacket, the handle method is called, passing in the session instance to the request package. The purpose of the session is to make it easier for users not to maintain a session instance. Because some programmers want to implement the logic may be relatively simple, he just use the session to meet his needs, he only need to implement the corresponding processing function. After processing is complete, call Session.asyncsend to send the response package.

Here you can actually provide a simple default version of the implementation. However, given the different protocol, it will lead to different key scheduling, so let users play it by themselves.

The user actually has a lot of freedom here, he can do the callback distribution logic based on the map relation, can also make a simple implementation logic, and then make the corresponding implementation through the type assert. It depends on the individual tastes. I prefer the latter, can reduce a lot of register, realize the actor Model + Pattern match flavor of things.

Server object

Here is a simple package for the service side. The implementation of the server is very simple, is repeated to accept, and then construct a session, then call the user passed in the callback function, the end of the live. Users can pass in net themselves. Listener, you can pass in Packetprotocol, Packethandler and Sendchansize. These parameters are passed in when the session is constructed, which can reduce the duplication of code implementations. Server.acceptloop does not close the constructed session, the user is responsible for the completion of this matter!

Disadvantages

The whole is very shabby, just a model. In my own unpublished code, I actually realized the agreement of my company and realized the Packetprotocol. A code generator is also specifically written for this purpose.

There is newserver need to pass a net.listener, compare egg pain. Later decide whether to kill. NewSession need to pass in Net.conn, in fact, is the product of compromise, because Net.listener return is net.conn, this instance needs to be handed over to session use, but here is embarrassing is, the client use, need to go net.dial, get a N Et. Conn, maybe it's time to provide a swnet. Dial method.

Summarize

The code I published was modified on the basis of the original code, and was inspired by the https://github.com/funny/link of Dada, but it was a lot different. Thanks again for Dada's contribution.