Common server models under Linux

Common server models under Linux

To today in the company for 8 months, completed from the campus people to the turn of the field. Born in society, feeling a lot, all involuntarily. In the work of happy, anxious, disappointed, to the last heart cool, this all the way down to grow a lot. University graduation design involves network knowledge, at that time learned some basic network knowledge, work soon, transferred to new project group HMS Do product IPTV just and network related, in recent months has been looking at "Linux High Performance Server Programming", in the online also read a lot of articles, have been want to write a summary.

Basis

Figure 1 C/S architecture communication

Figure 1 is a simplified TCP communication process, the left side of the "Server side", the right side of the "client", the TCP protocol itself does not define the service side, the client these concepts, because a large number of resources on the left side of the communication, the request points to it, there is the service side, the client these concepts. At the beginning of the network development, the network transmits the most is the text file, does not resemble now the massive dynamic graph, the video and so on multimedia resources. After the client establishes a connection, submits a resource request, the server receives the request after it resolves the requested resource, then sends the client the requested resource, disconnects the client after it receives the requested resource, and ends the communication. The content length of the text file is not large, generally below the KB, the number of times the server sends the data depends on the link MTU, the early Xerox is 576, and the Ethernet is 1500 bytes, and the number of times is 1, 2 times. With the popularization of the network, the development of multimedia technology, browser upgrade, the introduction of new protocols, the early server architecture can not meet the requirements, the server model has been a derivative. Here are a few common server models.

Iterative Models

The iterative model is the earliest server model, its core implementation is to each user, and then for this user service to the end, the process does not accept any new user requests, a single server service a user, its flowchart 1.

Core code:

Bind (LISTENFD);

Listen (LISTENFD);

for (;;)

{

CONNFD = Accept (LISTENFD, ...); Accept connections from clients

while (User_oline)

{

Read (Connfd,recv_buf, ...); Reading data from the client

Release_request (RECV_BUF); Resolve Customer Requests

Write (Connfd,send_buf, ...) Sending data to the client

}

Close (CONNFD)

}

The biggest problem with this model is that a few of the main operations are blocked, such as accept, and if there is no user coming, then the process is stuck here. There is also a read operation, which assumes that the user is sending a resource request after the connection is established, but the user does not send it? If TCP keepalive detection is turned on, it is also a matter of a few minutes before you can turn off this malicious connection, and if you do not turn on TCP keepalive detection, set a connection validity time. Even so, the server is completely idle for a few minutes in the middle, but cannot accept new user connections.

Multi-process Model

In order to resolve the above operation blocking, new user connections cannot be accepted, and the multi-process model is used. The core idea of this model is that the main process accepts the user connection and processes the business in the subprocess, so that the new user connection is not blocked.

Figure 2 Multi-process model

Core code:

Bind (LISTENFD);

Listen (LISTENFD);

for (;;)

{

CONNFD = Accept (LISTENFD, ...); Start accepting connections from clients

PID = fork ();

Switch (PID)

{

Case-1:

Do_err ();

Break

Case 0://child process

Client_handler (User_info);

break;

Default://Parent process

Close (CONNFD);

Continue;

}

}

Voidclient_handler (User_info)

{

while ()

{

Read (Connfd,recv_buf,...); Reading data from the client

Dosomthingonbuf (RECV_BUF); Resolving user Requests

Write (CONNFD,SEND_BUF)//Send data to Client

}

Shutsown (CONNFD)

}

The disadvantage of this pattern is given in multiple threads.

Multithreaded model

Linux above thread is also called lightweight process, it and the main thread share the entire process of data, thread switching overhead is much smaller than the process. The core idea of multithreaded model is that each user connects to create a thread for the user, whose flowchart is 2, just change the fork to pthread_create.

Core code:

Bind (LISTENFD);

Listen (LISTENFD);

for (;;)

{

CONNFD = Accept (LISTENFD, ...); Start accepting connections from clients

ret = Pthread_create (, worker,, User_info);

}

void Worker (User_info)

{

while ()

{

Read (Connfd,recv_buf,...); Reading data from the client

Dosomthingonbuf (RECV_BUF); Resolving user Requests

Write (CONNFD,SEND_BUF)//Send data to Client

}

Shutsown (CONNFD)

}

Disadvantages of multi-process models and multithreaded models:

1, the process, the creation of threads, destruction at some times will cause a great consumption, for example: now the main terminal Equipment watch video transmission protocol is based on the HTTP protocol HLS,  IPTV latest version of a single server out of the stream 60G, in the case of HLS short connection out of the stream to hit 80 percent 48G, According to the 700K bitrate, then the number of users close to 7W, the server's caps according to 2W calculation. In accordance with the multi-threaded, multiple-process model, 1s build, destroy tens of thousands of of threads, the server simply can't carry. So the majority of the design will implement the process, the thread pool, reduce this part of the overhead. Process, thread pool can not avoid the preemption of resources, in the process pool, the use of semaphores, shared memory to achieve the allocation of resources, the use of mutual exclusion locks in the thread pool or conditional variables to achieve resource allocation.

2. In multi-threaded situations, if a thread has a problem, it may cause the process to hang up.

3. Multi-process and multi-threading with no meaning in multi-core case. As mentioned above, a server out of the stream 60G, if the user is a SD 2M bitrate, then the number of online users is 30000. With  latest architecture, the RH5288 V3 configures Intel 2658 48-core hardware, the number of processes and threads on a single core is 30000/48=625. A kernel running so many functions exactly the same process, or thread is completely meaningless, this is the default single server can run so many processes, threads. White, it is absolutely impossible for a user to use a process or thread.

Port Multiplexing Select/poll

Aside from the framework, Select/poll is used to solve the problem that one user uses a process, thread, Select/poll can listen to multiple file handles in a process, thread.

Code:

Bind (LISTENFD);

Listen (LISTENFD);

Fd_zero (&set);

Fd_set (LISTENFD, &aset);

for (;;)

{

Loop to add all file descriptors

for ()

Select (...);

if (Fd_isset (LISTENFD, &rset));

{

CONNFD = Accept ();

User_info [] = CONNFD;

Fd_set (CONNFD, &set);

}

else If ...

Loop detection File Descriptor

for (;;)

{

FD = USER_INFO[I].FD;

if (Fd_isset (FD, &rset))

DoSomething ();

else If ...

}

}

Select listens to multiple file descriptors, set the essence of a shape array, each bit of the array represents a file descriptor, select can listen to the file descriptor read events, write events, exception events, when something happens on the file descriptor, the system call will be the corresponding position 1. Select is the most criticized there are two points, one is each call to select before the file descriptor to add to the array, not to mention the time to go through the addition, the data from the user state copy to the kernel state is very consumption performance. The second is that the select call returns the array must be traversed again to see if there is an event generated by the file descriptor and an O (n) operation. In fact, the data of the select operation user must be saved separately and the user data cannot be saved in the select call.

Bind (LISTENFD);

Listen (LISTENFD);

Addtopoll (LISTENFD);

for (;;)

{

Number = poll ();

if (event.revents &pollin && fd = = LISTENFD)

{

CONNFD = Accept ();

event [] = CONNFD;

Addtopoll (CONNFD);

}

else If ...

Loop detection File Descriptor

for (;;)

{

FD = EVENT.FD;

if (Event.revents&pollin)

DoSomething ();

else If ...

}

}

The implementation mechanism of poll is to write a file descriptor and an event of interest to this file descriptor to a struct, the poll call returns, and the operating system writes the event that the file descriptor has occurred to a variable. Poll the optimization of a select is that it does not have to copy the file descriptor every time, writes the event to a variable, unlike select uses three variables, the poll call can only save the file descriptor. But the return of the poll call must also traverse the array again, which is also an O (n) operation.

Epoll

The most common use of Linux IO multiplexing is that the Epoll,epoll implementation is similar to poll, but it does a two-point improvement. The first is that the struct used in the Epoll call can hold the user data (not only), and the Epoll returns the number of file descriptors that actually occurred, and these corresponding events are written to the returned array. This is very meaningful for some scenarios, such as the RTSP protocol in the use of UDP transmission of data, its signaling data using TCP transmission, signaling data is much less than business data, generally a longer time to have a signaling interaction. In 60G scenario, the user data 2M bitrate, then need to listen to the file descriptor is 30000, a time a file descriptor generated events, if the poll call will need to traverse the length of 30000 array, and epoll only 1 times.

Code:

Bind (LISTENFD);

Listen (LISTENFD);

Addtoepoll (LISTENFD);

for (;;)

{

Number = Epoll_wait ();

if (event.events &epollin && fd = = LISTENFD)

{

CONNFD = Accept ();

event [].DATA.FD = CONNFD;

Addtoepoll (CONNFD);

}

else If ...

Loop detection File Descriptor

for (;;)

{

FD = EVENT.DATA.FD;

if (Event.events&epollin)

DoSomething ();

else If ...

}

}

Design for IO multiplexing

Obviously with the IO multiplexing feature, the original multi-process, multi-threaded mode design process has not been suitable. In all previous processes, accepting a new user connection (accept) is done in the main process or the main thread, but in some cases single-process, single-threaded processing will encounter bottlenecks, in the previous short connection example, a single process, the thread of the caps is less than 1s 2W. The division of the main process, thread and worker processes, threads must be clear, who is responsible for connecting, who is responsible for business processing, who is responsible for reading and writing.

In order to solve the accept bottleneck, some modes are to put processing accept into each process, thread, and some companies develop kernel modules on Linux, using port NAT technology, each core listens on a separate port. The good news is that Linux 3.7 and above support the Portreuse feature, where multiple processes can listen to a port at the same time without generating a panic effect.

The work process is responsible for all work

Model 3, a worker thread can accept a new user connection, and the main process creates multiple processes after listen.

Figure 3

Core code:

Bind (LISTENFD);

Listen (LISTENFD);

Typically create a number of sub-processes with CPUs

Master = 1;

for (;<cpu_number;)

{

PID = fork ();

ASSERT (PID >= 0);

if (PID > 0)

{

Continue

}

Else

{

Master =-1;

Break

}

}

if (1 = = Master)

{

Run_master ();

}

Else

{

Run_worker ();

}

void Run_worker ()

{

Epoll_create ();

for (;;)

{

Number = Epoll_wait ();

If (event.events &epollin && fd = = LISTENFD)

{

connfd= accept ();

EVENT[].DATA.FD = CONNFD;

Addtoepoll (CONNFD);

}

else If ...

Loop detection File Descriptor

for (;;)

{

FD = EVENT.DATA.FD;

if (event.events &epollin)

DoSomething ();

else If ...

}

}

You can see that the main process has created more than one word process and then created its own Epoll file descriptor in the subprocess, some of which are not forked until the main process Epoll created, and the individual is not very fond of this practice. The more and more fire Nginx also uses a similar pattern, in order to avoid the surprise group effect, it implements a mutex with shared memory, must obtain this mutex before calling accept.

Master process notifies child process accept

"Linux High Performance Server Programming" wrote a lock-free work process accept mode, the specific implementation is the child process Epoll does not join the listener handle. At the beginning of the process creation, the pipeline is created, and the parent process Epoll listens to LISTENFD, but does not do the accept operation, but notifies a child process through the pipeline to accept.

Figure 4 Lock-free process Accept

Core code

void Run_worker ()

{

Epoll_create ();

for (;;)

{

Number = Epoll_wait ();

If the parent process is notified by the pipeline, go to the Accept

If (event.events &epollin && FD ==pipefd[0])

{

connfd= accept ();

EVENT[].DATA.FD = CONNFD;

Addtoepoll (CONNFD);

}

else If ...

Loop detection File Descriptor

for (;;)

{

FD = EVENT.DATA.FD;

if (event.events &epollin)

DoSomething ();

else If ...

}

}

Threading mode

From many practical applications, it is very rare to use threads to accept operations in sub-threads, the general practice is that the main thread only takes the accept operation, and then the child thread is responsible for reading and writing the data. This programming is also the simplest, but very easy to appear the main thread accept the bottleneck.

Figure 5 Multi-thread IO multiplexing

As shown in 5, before the main thread accept, some threads and corresponding number of epoll are created, and each thread is assigned a epoll. After the main thread accepts the new user, the user is added directly to the epoll of a thread because it is the same process.

Core code:

Bind (LISTENFD);

Listen (LISTENFD);

Create a number of processes with CPUs

for (;<cpu_number;)

{

Pthread_create (, worker,,);

}

Create the same number of CPUs plus 1 EOPLL

for (idx = 0;<cpu_number + 1;)

{

THREAD_EPOLL[IDX] = epoll_create (number);;

}

Add a listener file descriptor to the main thread Epoll

Epoll_ctl (, LISTENFD,)

while (1)

{

Number = Epoll_wait ();

If (event.events &epollin && fd = = LISTENFD)

{

CONNFD = Accept ();

event [].DATA.FD = CONNFD;

Polling or some sort of algorithm that locks to a thread

Addtoepoll (CONNFD);

}

else If ...

{

}

}

void* worker ()

{

for (;;)

{

Number = Epoll_wait ();

Loop detection File Descriptor

for (;;)

{

FD = EVENT.DATA.FD;

if (event.events &epollin)

{

DoSomething ();

}

ElseIf ...

}

}

Epoll thread pool also has another approach, the main thread is responsible for the accept, responsible for distributing tasks, it will put the user's scoket into the list, and then multiple lines threads to compete with the linked list, get the list of threads to remove the head node and then release ownership, the worker thread only business processing, no epoll operation. This approach has two shortcomings, one is that the main thread to deal with the user connection request, but also distribute the task, causing the main thread busy, child threads idle dead phenomenon, completely did not play epoll and multithreading characteristics.

Figure 6

Summary server design model is more, now learning are relatively basic content, such as event triggering, streaming, port mapping, port forwarding, semi-asynchronous mode, reactor mode, follower mode, etc. are not mentioned, in short, there are a lot of learning.

Common server models under Linux