Python Network programming Advanced Chapter Three

In the advanced Chapter two, we explain the 5 commonly used IO model, understand these commonly used IO model, for writing server program has great help, can improve our concurrency speed! Because the main part of communication in the network is IO operations. In this article we will focus on the IO multiplexing model referred to in the second article, which is the select mechanism. In fact, select Mechanism has some defects, and later produced a more efficient mechanism epoll, will explain later!

First, select mechanism

1. Principle: Select can be understood as a listener, can listen to multiple file descriptors. When the state of a file descriptor has changed (readable/writable), the operating system sends a message to the application to process the data.

2. Advantages: Almost all platforms support, cross-platform support is better.

3. Disadvantages:

(1). The number of file descriptors that can be monitored by a process/thread is limited.

(2). The scan of the file descriptor is linear and is polled, each time it is scanned from the beginning to the end, and when the list of file descriptors becomes larger, it is quite a waste of time and CPU

(3). Copying an array containing a large number of file descriptors from the kernel space to the user space may be fine when the array is small, but as the array grows, it becomes a waste of resources.

4. Horizontal Trigger:

When select () reports a file descriptor that has changed status to the process, the next select () also reports these file descriptors if the process has not done any processing.

Second, Epoll

Epoll can be seen as an enhanced version of Select/poll (essentially select), breaking many of the constraints of select and adding some other features!

1. Why is epoll high efficiency?

The biggest feature of Epoll is that it only tells the server which file descriptor (FD) has changed. If the server does not process the corresponding FD, then the operating system will discard this FD, no longer send messages to the server (Edge trigger)! In addition, Epoll is the use of event monitoring method notification, which is also the charm of Epoll!

2. Principle:

　　　　

(1). Register the file descriptor in Epoll, the operating system's event listener will listen to the file descriptor collection (Fd_set)

(2). If there is a change in FD, then event snooping will report the change to the operating system on the FD

(3). The operating system sends a message to the server informing it that the FD you are interested in has changed, so deal with it.

(4). Now the server is going to read the data in shared memory!

3. Advantages:

(1). There is no limit on the maximum number of connections.

(2). Do not use polling to deal with FD, but the use of event monitoring, that is, which FD has an event, the OS notifies the server to use the corresponding callback function to process the FD

(3). Memory copy: When data arrives, the operating system sends a notification to the server to process the data. Speed up the transfer of user space and kernel space messages by using shared memory.

4. Misunderstanding:

Not in any case, epoll is more efficient than select/poll, and only when many connection requests arrive!

Three, Epoll programming model

(1). Create 1 Epoll objects

(2). Tell the Epoll object to listen for the specified event on the specified FD

(3). Ask the Epoll object what events have occurred on the FD since the last query

(4). Perform some operations on these FD

(5). Tell the Epoll object, modify the FD list or register the event, and monitor

(6). Repeat step 3-5 until you are finished

(7). Destroying Epoll objects

　　

Iv. implementation of the Code

1 """2 use non-blocking and epoll to implement a server3 """4 ImportSocket5 6 ImportSelect7 8 9 classWebServer:Ten     """define a Web server""" One  A     def __init__(self): -         #1. Create a TCP server -Self.tcp_server =Socket.socket (socket.af_inet, socket. SOCK_STREAM) the         #multiplexing Ports -Self.tcp_server.setsockopt (socket. Sol_socket, SOCKET. SO_REUSEADDR, 1) -         #2. Binding Port -Self.tcp_server.bind (("', 6767)) +         #3. Set as passive socket -Self.tcp_server.listen (128) +  A     defRun (self): at         """running a server""" -         #1. Set the server to non-blocking mode - self.tcp_server.setblocking (False) -         #2. Create an Epoll object and register an acceptable connection event for the server -Epoll =Select.epoll () - Epoll.register (Self.tcp_server.fileno (), select. Epollin) inClient_dict = Dict ()#to associate FD with the client -         #3. The server accepts requests from clients to          whileTrue: +             #4. Monitor which FD in the Epoll has occurred -Epoll_list =Epoll.poll () the              forFD, Eventinchepoll_list: *                 ifFD = =Self.tcp_server.fileno (): $                     #a client to connect to the passive socket serverPanax NotoginsengClient, addr =self.tcp_server.accept () -                     #print (addr) the                     #registering the client in Epoll + Epoll.register (Client.fileno (), select. Epollin) A                     #Add the client and client-side FD to the clients dictionary. theClient_dict[client.fileno ()] =Client +                 Else: -                     #a client sends the data, but how do I get the client?  $data = CLIENT_DICT[FD].RECV (1024x768). Decode ('Utf-8') $                     ifData: -                         #instructions The client sent the data. -                         Print(data) theClient_dict[fd].send ('I have received your data! \ n'. Encode ('Utf-8')) -                     Else:Wuyi                         #indicates that the client is closed the client_dict[fd].close () -                          Wu Client_dict.popitem () -                         #need to cancel the event registered by the client About Epoll.unregister (FD) $  -             #iterate through each client in the clients dictionary for the corresponding FD -              forIteminchClient_dict.items (): -                 A                 Print('fd:{}--->addr:{}'. Format (item[0], item[1])) +                 Print('-'*50) the         #shutting down the server - self.tcp_server.close () $  the  the  the defMain (): the     #1. Initializing a TCP server -Server =WebServer () in     #2. Running a server the Server.run () the  About  the if __name__=='__main__': theMain ()

Python Network programming Advanced Chapter Three