Python's socket programming

First, client/server architecture

C/S Architecture:

1. Hardware C/S Architecture (printer)

2. Software c/S Architecture (Web service)

Learn socket is also to write a C/s architecture software

Second, Socket introduction

Preface: We want to develop a set of C/S architecture, first of all to understand the OSI seven layer, simply said the OSI seven layer includes: should, table, will, biography, net, number, object

The main focus, we developed a C/s architecture based on an application (belonging to the application layer), which is communicated through the network, and the core of the network is a large stack of protocols, TCP/IP protocol, Ethernet protocol, if the development of the software of A/D architecture, must be based on these standards. Now let's briefly introduce these standards

Only the TCP/IP layer associated with the socket is described: The TCP/IP protocol family includes the transport layer, the network layer, and the link layer.

Now you're wondering, is the socket working on that floor? See:

Comprehensive:

A socket is an intermediate software abstraction layer that the application layer communicates with the TCP/IP protocol family, which is a set of interfaces. In design mode, thesocket is actually a façade mode, it is the complex TCP/IP protocol family hidden behind the socket interface, for the user, a set of simple interface is all, let the socket to organize data to meet the specified protocol.

Third, socket sockets history and classification:

Sockets originated in the the 1970s UC Berkeley version of Unix, which is what people call BSD Unix. Therefore, sometimes people also refer to sockets as "Berkeley sockets" or "BSD sockets". Initially, sockets are designed to be used for communication between multiple applications on the same host. This is also called interprocess communication, or IPC. There are two types of sockets (or two races), which are file-based and network-based.

Socket family based on file type

Socket family name: Af_unix

Unix all files, file-based sockets are called by the underlying file system to fetch data, two sockets process running on the same machine, you can access the same file system to complete the communication indirectly

Socket family based on network type

Socket family name: af_inet

(There are also af_inet6 used for IPv6 and some other address families, but they are either used only on a platform, or have been discarded, or are rarely used, or are not implemented at all, and Af_inet is the most widely used one in all address families, Python supports a variety of address families, but since we only care about network programming, most of the time I use af_inet only)

Socket Workflow:

For connection sockets:

1. The server first uses the socket function to establish a socket, using this socket to complete the communication monitoring.
2. Bind a port number and IP address with the BIND function. Because the local computer may have multiple URLs and IPs, each IP and port has multiple ports. You need to specify an IP and port for listening.
3. The server calls the Listen function so that the server's port and IP are listening, waiting for the client to connect.
4. The client uses the socket function to set up a socket, setting the remote IP and port.
5. The client calls the Connect function to connect to the port specified by the remote computer.
6. The server accepts the connection from the remote computer with the Accept function and establishes communication with the client.
7. After the connection is established, the client writes the data to the socket with the write function. You can also use the Read function to read the data sent by the server.
8. The server reads the data sent by the client using the Read function, or it can use the Write function to send the data.
9. When the communication is complete, close the socket connection with the close function.

For non-connected sockets

Non-connected communication does not require a connection between the client and the server, so there is no process to establish a connection in the program. Before you can communicate, you need to establish a network socket. The server needs to bind to a port on which to listen for information received. The client needs to set up the remote IP and port, and the required information needs to be sent to this IP and port.

Detailed

Service-side socket function S.bind ()    binding (host, port number) to Socket S.listen ()  starts TCP listener s.accept ()  passively accepts TCP client connections, (blocked) The arrival of the waiting connection client socket function S.connect ()     actively initializes an extended version of the TCP server Connection S.CONNECT_EX ()  connect () function, returning an error code when an error occurs, Instead of throwing an exception to the public-use socket function s.recv ()            receives TCP Data s.send ()            sends TCP data (send data is lost when the amount of data to be sent is greater than the remaining space in the cache) S.sendall ()         Send complete TCP data (essentially circular call Send,sendall when the amount of data to be sent is greater than the remaining space in the cache, the data is not lost, the loop calls send until the end of the Send) S.recvfrom ()        receive UDP data s.sendto ()          Send UDP data S.getpeername () the address of the     remote that is connected to the current socket S.getsockname ()     address of the current socket s.getsockopt ()      Returns the parameter of the specified socket s.setsockopt ()      sets the parameter of the specified socket S.close ()           closes the socket for the lock socket Method s.setblocking ()     Sets the blocking and non-blocking mode for sockets S.settimeout () to      set the timeout time for a blocked socket operation S.gettimeout ()      Gets the timeout time for a blocked socket operation the function of a socket for a file S.fileno ()          The socket's file descriptor S.makefile ()        creates a file associated with the socket

TCP-based sockets

Socket Service side:

#!/usr/bin/env python#-*-coding:utf-8-*-ImportSocketserver=Socket.socket (Socket.af_inet,socket. SOCK_STREAM) #创建服务端套接字server. Bind ('127.0.0.1', 8080)) #绑定ip +portserver.listen (5) #监听连接conn, clinet_addr=server.accept () #接受客户端连接Print(conn,clinet_addr) Client_data=CONN.RECV (1024) #接受客户端升级Print('client data is%s'%client_data) Conn.send (Client_data.upper ()) #发送数据给客户端conn. Close () #关闭客户端套接字连接
Server.close () #关闭服务端连接

Socket client:

#!/usr/bin/env python#-*-coding:utf-8-*-Importsocketclient=Socket.socket (Socket.af_inet,socket. SOCK_STREAM) #创建客户端套接字client. Connect ('127.0.0.1', 8080) #连接服务端client. Send ('Hello'. Encode ('Utf-8')) #发送byte格式数据client_date=CLIENT.RECV (1024) #接收服务端数据Print(client_date) client.close () #关闭连接

Evolution version:

Server Side

#!/usr/bin/env python#-*-coding:utf-8-*-ImportSocketserver=Socket.socket (Socket.af_inet,socket. SOCK_STREAM) server.setsockopt (socket. Sol_socket,socket. SO_REUSEADDR,1)#If the service side still exists four times the time_wait state of the wave is occupied address, need to use this releaseServer.bind (('127.0.0.1', 8080)) Server.listen (5) #backlog: Maximum can suspend several connections whileTrue:#Connecting LoopsConn,clinet_addr=server.accept ()Print(CONN,CLINET_ADDR) whileTrue:#Communication Cycle        Try:#exception Handling is a problem that resolves if a client disconnects and the server crashesCLIENT_DATA=CONN.RECV (1024) #收多少个字节Print('client data is%s'%client_data) Conn.send (Client_data.upper ())exceptException: Breakconn.close () server.close ( )

Client:

#!/usr/bin/env python#-*-coding:utf-8-*-Importsocketclient=Socket.socket (Socket.af_inet,socket. Sock_stream) Client.connect (('127.0.0.1', 8080)) whiletrue:msg=input ('>>'). Strip ()ifLen (msg) = = 0:Continueclient.send (Msg.encode ('Utf-8')) Client_date=CLIENT.RECV (1024) #收取1024个字节Print(client_date) client.close ()

UDP-based sockets

Server side:

#!/usr/bin/env python#-*-coding:utf-8-*-ImportSocketserver=Socket.socket (Socket.af_inet,socket. SOCK_DGRAM) Server.bind (('127.0.0.1', 8080)) whileTrue:data,client_addr_port=server.recvfrom (1024)    Print(Data.decode ('Utf-8')) msg=input ('>>'). Strip () Server.sendto (Msg.encode ('Utf-8'), Client_addr_port) server.close ()

Client

#!/usr/bin/env python#-*-coding:utf-8-*-Importsocketclient=Socket.socket (Socket.af_inet,socket. SOCK_DGRAM) whiletrue:msg=input ('>>'). Strip () Client.sendto (Msg.encode ('Utf-8'),('127.0.0.1', 8080)) Data,server_addr_port=client.recvfrom (1024)    Print(Data.decode ('Utf-8') ) Client.close ()

The difference between recv and Recvfrom&&send and SendTo 1. Disclaimer: Sends the data to the sending buffer of the client, and the message is received from the buffer in its own

1.1:tcp:send Send Message, recv receive message

1.2:udp:sendto Send Message, recvfrom receive message

2.send and SendTo

TCP is based on data flow, and UDP is datagram-based:

1. Send (Bytes_data): sends the data stream, the data stream Bytes_data if is empty, own this part of the buffer also is empty, the operating system does not control the TCP protocol to send the empty packet
2. SendTo (Bytes_data,ip_port): Send a datagram, Bytes_data is empty, and ip_port, all even send empty bytes_data, the datagram is not empty, their own buffer to receive the contents of the end, The operating system will control the UDP protocol packet.

3.recv and Recvfrom

1.TCP Protocol:

(1) If the data in the message buffer is empty, then the recv will be blocked (blocking is simple, is waiting to receive)

(2) only the TCP protocol client send an empty data is really empty data, the client even if there are infinite send empty, also with no one.

(3) TCP link-based communication

• Based on the link, the listen is required, specifying the size of the half-connection pool
• Based on the link, the server must run first, and then the client initiates the link request
• For Mac Systems: If one end of the link is broken, the other end of the link is also finished recv will not block, received is empty (workaround is: The service side after receiving the message plus if judgment, the empty message will break off the communication loop)
• For Windows/linux Systems: If one end of the link is broken, then the other end of the link is also finished recv will not be blocked, received is empty (workaround is: server-side communication loop with exception handling, catch the exception after the break off the communication loop)

2.UDP protocol

(1) If the data in the message buffer is "empty", Recvfrom will also block

(2) Only the UDP protocol client sendinto an empty data is not really empty data (including: empty data + address information, the resulting report will still not be empty), so as long as the client has a sendinto (whether or not to send empty data, is not really empty data), The server can recvfrom to the data.

(3) UDP no link

• No links, so no listen, no more connection pool
• No link, UDP sendinto no tube whether there is a running server, you can always send messages, but data loss
• Recvfrom the data received is less than the data sent by Sendinto, the data is lost directly on Mac and Linux system, and it is sent on the Windows system more direct error than received
• Only sendinto send data without recvfrom data, data loss

Summary: you run the UDP client alone, and will not error, but TCP will error, because the UDP protocol is only responsible for sending out the package, the other side can not receive, I do not care, and TCP is based on the link, there must be a service side first run, The client goes to establish a link with the server and then relies on the link to deliver the message, and any attempt to destroy the link will cause the other program to crash.

Five, sticky bag phenomenon

View Sticky pack behavior by remote execution of commands

Server Side

#!/usr/bin/env python#-*-coding:utf-8-*-ImportSocketImportSubprocessserver=Socket.socket (Socket.af_inet,socket. SOCK_STREAM) server.setsockopt (socket. Sol_socket,socket. SO_REUSEADDR,1)#If the service side still exists four times the time_wait state of the wave is occupied address, need to use this releaseServer.bind (('127.0.0.1', 8080)) Server.listen (5) whileTrue:#Connecting LoopsConn,clinet_addr=server.accept ()Print(CONN,CLINET_ADDR) whileTrue:#Communication Cycle        Try:#exception Handling is a problem that resolves if a client disconnects and the server crashesCMD=CONN.RECV (1024)            if  notCmd: Breakcmd=cmd.decode ('Utf-8') Cmd_res=subprocess. Popen (cmd,shell=True, stderr=subprocess. PIPE, stdout=subprocess. PIPE) Err=Cmd_res.stderr.read ()ifErr:res=ErrElse: Res=Cmd_res.stdout.read () conn.send (res)exceptException: Breakconn.close () server.close ( )

Client side:

#!/usr/bin/env python#-*-coding:utf-8-*-Importsocketclient=Socket.socket (Socket.af_inet,socket. Sock_stream) Client.connect (('127.0.0.1', 8080)) whileTrue:cmd=input ('>>'). Strip ()ifLen (cmd) = = 0:Continueclient.send (bytes (cmd,encoding='Utf-8')) Client_date=CLIENT.RECV (1024)    Print(Client_date.decode ('GBK') ) Client.close ()

Take a closer look at the results:

See execution results: When the input ipconfig, execution is still the dir command execution results, then this is the result of the occurrence of sticky packet phenomenon, why this phenomenon occurs? This behavior occurs because the client side accepts the execution result when it accepts 1024 bytes of data, and the data that executes the result is greater than 1024 bytes.

Six, what is sticky bag

Statement: The fifth is seen in the phenomenon of TCP sticky packets, and the phenomenon of sticky packets will only occur in the TCP connection, why? Take a look at the decomposition below

The sender can be a K-K to send the data, and the receiving side of the application can be two K two k to take the data, of course, it is possible to take 3 K or 6K data at a time, or only a few bytes of data at a time, that is, the application sees the data is a whole, or a stream (stream), The number of bytes of a message is not visible to the application, so the TCP protocol is a stream-oriented protocol, which is also the cause of the sticky packet problem. And UDP is a message-oriented protocol, each UDP segment is a message, the application must be in the message to extract data, not one time to extract arbitrary bytes of data, which is very different from TCP. How do you define a message? Can think of the other one-time Write/send data for a message, it is necessary to understand that when the other side send a message, regardless of the underlying how fragmented shards, the TCP protocol layer will constitute the entire message of the data segment is completed before rendering in the kernel buffer.

For example, the TCP-based socket client to the server to upload files, the content of the file is sent in accordance with a paragraph of the stream of bytes sent, in the receiver looked at, do not know where the file's byte stream from where to start, where to end

The so-called sticky packet problem is mainly because the receiver does not know the boundary between the message, do not know how many bytes of data extracted at once.

In addition, the packet caused by the sender is caused by the TCP protocol itself, TCP to improve transmission efficiency, the sender often to collect enough data before sending a TCP segment. If there are few data to send in a few consecutive times, TCP will usually send the data to a TCP segment based on the optimization algorithm, and the receiver receives the sticky packet data.

1. TCP (Transport Control Protocol, transmission Protocol) is connection-oriented, stream-oriented and provides high reliability services. Both ends of the transceiver (client and server side) have one by one pairs of sockets, so the sending side in order to send multiple packets to the receiver, more efficient to the other side, the use of the optimization method (Nagle algorithm), the multiple interval small and small data volume data, combined into a large block of data, and then to the packet. In this way, the receiving end, it is difficult to distinguish out, must provide a scientific unpacking mechanism. That is, stream-oriented communication is a non-message-protected boundary.
2. UDP (User Datagram Protocol, Subscriber Datagram Protocol) is non-connected, message-oriented, providing efficient service. The Block merging optimization algorithm is not used, because UDP supports a one-to-many pattern, so the receiver Skbuff (socket buffer) uses a chain structure to record each incoming UDP packet, in each UDP packet there is a message header (message source address, port and other information), so for the receiving end , it is easy to distinguish between the processing. that is, message-oriented communication is a message-protected boundary.

Seven, the solution of the adhesive package method One: Method two: Eighth, socketserver implementation concurrency

Python's socket programming