Multi libvent TCP Server: a High-performance TCP servers

MRIOTCP, Super Mario, as the name suggests, he is not only efficient, but also super easy and fun. At the same time he can be a very concise Linux C Development Learning project. It is no exaggeration to say that if you master this project, you will become a Linux C cow; Of course, you can also learn from the mario.c of the source code (MARITCP Server sample program), you can quickly get started with Linux C development.

After two months of testing (writing C + + client testing and Tuning system parameters), the test results are maximum single machine bandwidth throughput 1000M, testing the highest TCP long connection 1 million, the number of processing connections per second up to 40,000, at this time the system pressure load value is very low. In short, it can perform the maximum of one server to provide the highest performance service, and, after complete testing, runs stably and consumes very little system resources.

He is an Open-source project built on the SourceForge, initiated by Feng Jianhua, author of source code (Johnfong). Source code can be downloaded on sourceforge.

SourceForge Download: https://sourceforge.net/projects/mariotcp/files/latest/download

CSDN Download: http://download.csdn.net/detail/everlastinging/6195605

51cto Download: http://down.51cto.com/data/935913

Getting Started

Using MARIOTCP to build a powerful TCP server is easy!

The project source package is a very concise example of a TCP server program: MARITCP.

In the source package:

MARIO.C is a simple example of the main program, direct make can be compiled MARITCP, a TCP server, business logic has only one function: statistics at the same time the number of online sockets, every 1 minutes output.

Mario folder, Mariotcp's core code, make can directly compile the static libmario.a. MARIOTCP Core architecture follow-up will be introduced.

Test folder, is a slightly rudimentary client testing program, through the connection with the server, send login to the server, the MARITCP server will be online plus 1, the client disconnect when the number of servers online minus 1.

Now let's talk about how to customize a TCP server for your own business logic in just five steps:

1. Initialize Server

SERVER *server = Init_server (Conf->port, Conf->workernum, Conf->connnum, Conf->timeout, Conf->timeout) ;

Incoming parameters are:

Server listen port, number of worker threads, number of connections supported by each thread, read timeout, write timeout time.

Workernum * Connum is the number of long connections supported by the server, and a worker can easily support 100,000 long connections.

2. Realize business logic function and register

For specific business logic functions, see function module. You can register by using a function named "regist_*" defined in mario.h.

* *  registration Business processing function
/
void Regist_akg_func (uint16 ID, func_ptr func);
The ID can be any number 0-65535, encapsulated in the MARIOTCP protocol (see the end of this article). The scope of the
ID, which can be customized according to business logic, such as maritcp through the CMD structure defined in Protocol.h:
typedef enum _CMD {
cmd_function_base = 0x6100,< c9/> cmd_function_login = 0x6101
} CMD;

If you want to add a "Say_hello" service to MARITCP, you can do this:

1 Add in CMD: Cmd_function_say_hello = 0x602

2 adding functions in function:

Sint32 Say_hello (CONN c) {

I) through Conn To resolve the client sent the requested parameters

II) Set "Hello" to C->out_buf

III) Bufferevent_write (C->bufev, C->out_buf, Hrsp->pkglen);

IV) return 0;

}

3) Increase in MARIO.C: Regist_akg_func (Cmd_function_say_hello, Say_hello);

What do you think? Custom business logic, or very simple and efficient bar!

3. Start log thread Start_log_thread ()

The MARIOTCP log feature encapsulation is not good enough to continue the discussion on the "Go on 1.0.0" page ...

4, start the server Start_server ((void*) servers);

OK, a TCP server that can support 1 million or more long connections is born!

Go on 1.0.0

The first release is 0.9.9, and even with this package, a stable and efficient TCP server that customizes your business logic can be achieved in a few minutes, but there are a lot of mariotcp to be perfected, so let's solve the following problems as soon as possible. Let MarioTCP-1.0.0 release as soon as possible!

1. How to optimize the MARIOTCP protocol

In order for MARIOTCP to be safe enough, a simple mariotcp protocol is provided, which is connected to the client of the mariotcp after three handshakes, and the following packets require the format to be "Head+data" and the head structure is defined in the Mario_ In Akg.h:

typedef struct _HEAD {

UInt32 STX;

UInt16 Pkglen;

UInt16 akg_id;

} head;

Pkglen is the length of the entire package, the "Head+data".

IDs for akg_id and custom business logic functions, such as Cmd_function_say_hello on the "Getting Started" page

STX is your custom protocol cipher, registered through REGIST_STX (UInt32 STX) (see MARIO.C)

Although the MARIOTCP protocol is simple enough, and the beginning of the protocol can be customized, but whether it can be simpler or no agreement to maximize the convenience of development and use, need everyone's advice and help!

2, the log system is too rigid

MARIOTCP has a system of its own log function, but more obscure difficult to understand.

then expand ...

3. Business logic Stability Support

MARIOTCP is highly efficient and stable for network connectivity and reading and writing.

But the MARIOTCP thread pool is a fixed number, and is globally uniquely initialized, the dead thread cannot be restarted; The master thread that allocates the network task does not have the ability to monitor the worker, a thread dies, and the task is still assigned, causing the service to accumulate and not handle. This problem can occur if the business logic is very complex and inefficient.

This column more highlights: http://www.bianceng.cnhttp://www.bianceng.cn/Servers/zs/

In large online projects, the use of mariotcp places, will be through the business logic module monitoring, Alarm and program automatic processing to avoid the above problems. This feature has not been abstracted into mariotcp because of the time problem.

I will deal with this matter in the near future. Also hope to have friends advice and help!!

Why Supper

One, why super efficient

1, network services used in all the structure and memory is initiated when the program initialization, no destruction, no recovery.

No destruction good understanding, no explanation.

No recycling, refers to all the memory units to be used, run out and can, do not have to do reset operation.

2, a master thread to accept

After testing, multiple processes or threads are found to be accept and a process or thread accept, with little difference under extreme pressure.

A master is better than multiple master to solve the synchronization problem without having to lock it again.

3, master and worker when the single producer consumer mode, completely without lock communication

Not only accept, connection and subsequent conncetion processing are unlocked.

Even the business logic (see example MARITCP statistics online number function), MARIOTCP log System (this is also a reason that the log system is not abstract enough, before the design is too dependent on the overall architecture) are no lock processing!

4, a worker a set of libevent environment

Libevent handles 100,000 long connection network read and write event, its performance achieves the maximization.

Each worker has a separate set of Libevent, which has been tested and found to be very inexpensive and highly performing.

Second, single million long connection, 40,000 CPS (connection per second) How to do the test

1, set the system maximum number of files for unlimited

2, set the system's TCP memory kernel parameters to more than 256M

3, set the system IP to 15, then the number of long connections can be served in theory at least 15* (65535-1024)

4. Use Epoll or libevent to open a client program that can connect to 5w at the same time; The program also implements a random selection of 1000 connections per second and then creates 1000 new connections. In addition randomly select thousands of connection contracts.

At the same time more than one machine on the 20 client, then is the 100w long connection, 2w per second connection broken, 2w New connection to join in, and a number of packets sent over.

5, set the server-side reusable syn_wait connection; the way the client disconnects is actively broken (prevents client program ports from piling up)

In short, a very frustrating test, about 2 months before the test finished.

The above content is written by memory, afraid of mistakes or omissions, looking back in order to publish the test code and test results to everyone, will again develop, test and adjust the above content.