Implementation of Event-loop-based network processing in go language

This is a creation in Article, where the information may have evolved or changed.

We know that the go language provides a neat way of programming for concurrent programming, and you can execute code concurrently with a "synchronous" programming style, such as using go a keyword to open a new goroutine. For network programming, the Go standard library and the runtime are used internally epoll/kqueue/IoCompletionPort to implement event-loop asynchronous processing based on the network, but netpoll in a way that provides synchronous access to the outside. Specific code can refer to Runtime/netpoll, net, and Internal/poll.

Package Poll supports non-blocking I/O on the file descriptors with polling.
This supports I/O operations that block is only a goroutine and not a thread.
This was used by the net and OS packages.
It uses a poller built into the runtime with the
Runtime Scheduler.

Of course, we usually do not design the details of these packages, the normal use net of the package is easy to develop network programs, but, if we want to implement based on epoll the event-loop network program it?

Simple program based on Epoll

man epollYou can see an introduction to Epoll. The following example is from Tevino, which edge-triggered handles events in a way.

It uses,,, syscall.Socket syscall.SetNonblock syscall.Bind syscall.Listen system to listen to the port, and then use syscall.EpollCreate1 , syscall.EpollCtl syscall.EpollWait to correlate this listener file descriptor, once there are new connected events come over, using the syscall.Accept receive connection request, And this connection to the file descriptor calls the syscall.EpollCtl Listen data event. Once the connection has data ready, the call syscall.Read reads the data and invokes the syscall.Write write data.

from https://gist.github.com/tevino/3a4f4ec4ea9d0ca66d4f

1234567891011121314151617181920212223242526272829303132333435363738394041424344454647484950515253545556575859606162636465 6667686970717273747576777879808182838485868788899091929394

PackageMainImport("FMT""NET""OS""Syscall")Const(Epollet =1<< toMaxepollevents = +)funcEcho (FDint) {deferSyscall. Close (FD)varBuf[*1024x768]byte for{nbytes, E: = Syscall. Read (FD, buf[:])ifNbytes >0{FMT. Printf (">>>%s", buf) Syscall. Write (FD, buf[:nbytes]) fmt. Printf ("<<<%s", BUF)}ifE! =Nil{ Break}}}funcMain () {varEvent Syscall. Epolleventvarevents [Maxepollevents]syscall. EPOLLEVENTFD, err: = Syscall. Socket (Syscall.af_inet, Syscall. O_nonblock|syscall. Sock_stream,0)ifErr! =Nil{FMT. PRINTLN (ERR) OS. Exit(1)}deferSyscall. Close (FD)ifErr = Syscall. Setnonblock (FD,true); Err! =Nil{FMT. Println ("Setnonblock1:", err) OS. Exit(1)}addr: = Syscall. Sockaddrinet4{port: -}Copy(addr. addr[:], net. PARSEIP ("0.0.0.0"). To4 ()) Syscall. Bind (FD, &AMP;ADDR) syscall. Listen (FD,Ten) EPFD, E: = Syscall. EpollCreate1(0)ifE! =Nil{FMT. Println ("Epoll_create1:", e) OS. Exit(1)}deferSyscall. Close (EPFD) event. Events = Syscall. EPOLLINEVENT.FD =Int32(FD)ifE = Syscall. Epollctl (EPFD, Syscall. Epoll_ctl_add, FD, &event); E! =Nil{FMT. Println ("Epoll_ctl:", e) OS. Exit(1)} for{nevents, E: = Syscall. Epollwait (EPFD, events[:],-1)ifE! =Nil{FMT. Println ("epoll_wait:"E Break} forEV: =0; EV < nevents; ev++ {if int(Events[ev]. FD) = = FD {CONNFD, _, Err: = Syscall. Accept (FD)ifErr! =Nil{FMT. Println ("Accept:", err)Continue}syscall. Setnonblock (FD,true) event. Events = Syscall. Epollin | EPOLLETEVENT.FD =Int32(CONNFD)ifERR: = Syscall. Epollctl (EPFD, Syscall. Epoll_ctl_add, CONNFD, &event); Err! =Nil{FMT. Print ("Epoll_ctl:", CONNFD, err) os. Exit(1)}}Else{GoEchoint(Events[ev]. Fd))}}}}

The above based on epoll just a simple event-loop processing prototype, and on some platforms (MAC OS) can not be executed, the processing of the event is also very rough, if you want to implement a complete event-loop network program, you can refer to the following section of the library.

Evio

Evio is a high performance Event-loop network library with simple code and powerful functions. It uses direct epoll and kqueue system calls, in addition to the GO Standard Net library provides another way of thinking, similar to LIBUV and Libevent.

This library implements the same packet processing mechanism as Redis and haproxy, but does not want to completely replace the standard net package. For a long-running request (greater than 1 milliseconds), such as database access, authentication, etc., it is recommended that you use the Go net/http library.

As you may know, many Event-loop-based programs, such as Nginx, Haproxy, Redis, memcached, and so on, are very good performance, and they are all single-threaded, very fast.

This library also has a benefit that you can handle multiple network binding in one event-loop.

A simple example:

1234567891011121314

Package mainimport"Github.com/tidwall/evio"func Main () {varfunc int, in []byte) (out []byte, action Evio. Action) {out = inreturn}if"tcp://localhost:5000" "tcp://192.168.0.10:5001" "tcp://192.168.0.10:5002","Unix://socket"nil {panic(err. Error ())}}

The performance of the author is very good compared with the performance.