Python Automation Transport Koriyuki 14, network programming Socker (), Sockerserver ()

First, TCP/IP-related knowledge tcp/udp provide process address, two Protocols independent protocol Tcp:transmission Control Protocol Transmission Protocol, connection-oriented protocol, communications before communication channels (virtual link) need to be established, After the end of the removal link, streaming data protocol, reliable connection Udp:user Datagram Protocol User Datagram Protocol, no connection protocol, unreliable connection IP is the host to the host, in the transmission process is not changed, can not exceed MT U: Maximum transmission unit MAC is device to device communication, in the transmission will continue to encapsulate and unpack, will continue to change the data transfer process, TCP/IP protocols will be cut into data packets, IP protocol is non-connected, unreliable, but the upload of the Transport Layer protocol TCP is reliable, secure , IP may be in the segment, the maximum data message can not exceed mtu,1500 bytes, the IP message left 1400 or so bytes

Application layer, presentation layer, Session layer (resource subnet, user process) Transport layer, network layer, data link layer, physical layer (communication subnet, kernel space)features of the TCP protocol:

• Set up a connection, three-time handshake
• Package data into segments, checksum (CRC-32 cyclic redundancy check) to verify the integrity of the message
• Confirmation, retransmission, and timeouts
• Sort, logical Sequence number
• Flow control: Prevent fast-moving slow-receiving, sliding window (algorithm) to achieve
• Congestion control: Slow start and congestion avoidance algorithms to avoid congestion

Second, socket

Sockets are also commonly referred to as "sockets," which describe IP addresses and ports, and are a handle to a chain of communication, where applications usually make requests to the network through "sockets" or respond to network requests.
Sockets originate from UNIX, and one of the basic philosophies of unix/linux is "Everything is file", and the file is operated with "open" "Read and Write" "Off" mode. Socket is an implementation of this pattern, the socket is a special kind of file, some of the socket function is the operation of it (read/write Io, open, close)

The difference between a socket and file:

• The file module is "open", "Read and write" "Close" for a specified document
• The socket module is "open", "Read and write" "Off" for server-side and client sockets

Socket-related knowledge application process to use a socket to register the application port to the kernel, such as HTTP listening on port 80, the client Web browser process is open a random port 41052-65535 between the port, connect HTTP 80 port. Socket: Allows communication (data exchange) between processes between different processes on different hosts (even on the same host) socket API, early use in BSD

• SOCK_STREAM:TCP sockets
• SOCK_DGRAM:UDP sockets
• SOCK: Bare Socket

Socket domain: Socket field (based on the address it is using)

• Af_inet:address Family,ipv4
• Af_inet6:ipv6
• Af_unix: Communication of different processes on the same host, without TCP/IP, calling socket communication directly through the kernel, shared memory communication

Each type of socket provides at least two sockets: a stream, a data message

• Flow: TCP Reliable delivery, connection-oriented, borderless data transfer
• Data message: UDP unreliable delivery, bounded, no connection

The IANA (Internet Digital Distribution Agency) stipulates that:

• 0-1023: It is well known that the permanent allocation to the fixed application use, the privileged port, only administrator root has permission to use; 22/tcp (SSH), 80/tcp (HTTP), 443/TCP (HTTPS)
• 1024-41951: also registered port, but the requirements are not particularly strict, assigned to the program registered for an application to use
• 41952-65535: The client program randomly uses a port, called a dynamic port, or a private port whose scope is defined:/proc/sys/net/ipv4/ip_local_port_range, A larger number of concurrent scenarios can be modified as long as they do not contain a port before 1023. 1024x768 65535

The socket module is "open", "Read and write" "Close" for both the server side and the client socket, and a complete socket model diagram is shown: 1. Simple application

(1) Simple implementation of a TCP socket C/S architecture, s server, C client:

socker_server.py

#!/usr/bin/env python#-*-coding:utf-8-*-import socketip_add= (' 127.0.0.1 ', 9000) s = Socket.socket ()     # Create socket Object S.bind (ip_add)          # bound IP and port must be tuple S.listen (5)             # Set the number of connection pool hangs while True:    conn,addr = s.accept ()  # Accept client connections, Conn is an electrical signal from the Client connection server, addr client ip,port while    True:        try:            recv_data = Conn.recv (1024x768)    # CONN.RECV receive client information 1024 allowed bytes Maximum 8k            if Len (recv_data) = = 0:break            send_data = Recv_data.upper ()            conn.send ( Send_data)   # conn.send Send message        except Exception:            break    Conn.close ()

socker_client.py

#!/usr/bin/env python#-*-coding:utf-8-*-import socketip_add= (' 127.0.0.1 ', 9000) s = Socket.socket ()  # Create socket Object S.connect (ip_add)    # s.connect Connection server while True:    send_data = input (">>>:")        if send_data = = ' Exit ': Break    If Len (send_data) = = 0:continue    s.send (bytes (send_data,encoding= "Utf-8"))   # s.send Send Message    Recv_data = s.recv (1024x768)    # S.RECV message sent by the receiving Server    print (str (recv_data,encoding= "Utf-8")) S.close ()

According to the flowchart of the socket simple implementation, client-server interaction, the work flow diagram of the socket is very important, the above flow chart is TCP, UDP simple not so much interaction

(2) simple implementation of a UDP socket C/s architecture, s server, C client:

# server Import Socketip_port = (' 127.0.0.1 ', 9999) SK = Socket.socket (socket.af_inet,socket. sock_dgram,0) Sk.bind (ip_port) while True:    data, (host,port) = Sk.recvfrom (1024x768)    print (Data,host,port)    Sk.sendto (bytes (' OK ', encoding= ' Utf-8 '), (host,port)) #客户端import Socketip_port = (' 127.0.0.1 ', 9999) SK = Socket.socket ( Socket.af_inet,socket. sock_dgram,0) while True:    INP = input (' data: '). Strip ()    if InP = = ' exit ': Break    sk.sendto (bytes (INP, encoding= ' Utf-8 '), ip_port)    data = Sk.recvfrom (1024x768)    print (data) sk.close ()

2, Socker () class related methods:

SK = Socket.socket (socket.af_inet,socket. sock_stream,0)

Parameter one: Address cluster

Socket.af_inet IPv4 (default)
Socket.af_inet6 IPV6

Socket.af_unix can only be used for single UNIX system interprocess communication

Parameter two: type

Socket. Sock_stream streaming socket, for TCP (default)
Socket. SOCK_DGRAM datagram socket, for UDP

Socket. Sock_raw the original socket, the ordinary socket can not handle ICMP, IGMP and other network messages, and Sock_raw May, second, Sock_raw can also handle special IPV4 messages, in addition, the use of the original socket, can be IP_ The HDRINCL socket option constructs an IP header by the user.
Socket. SOCK_RDM is a reliable form of UDP that guarantees the delivery of datagrams but does not guarantee the order. Sock_ram is used to provide low-level access to the original protocol, which is used when certain special operations are required, such as sending ICMP packets. Sock_ram is typically used only by advanced users or by programs that are run by administrators.
Socket. Sock_seqpacket Reliable Continuous Packet service

Parameter three: protocol

0 (default) protocol related to a particular address family, if 0, the system will automatically select an appropriate protocol based on the address format and socket category.

Sk.bind (Address)

S.bind binds the socket to the address. The format of address addresses depends on the address family. Under Af_inet, address is represented as a tuple (host,port).

Sk.listen (Backlog)

Start listening for incoming connections. The backlog specifies the maximum number of connections that can be suspended before a connection is rejected.

The backlog equals 5, indicating that the kernel has received a connection request, but the server has not yet called the accept to process the maximum number of connections is 5
This value cannot be infinitely large because the connection queue is maintained in the kernel

Sk.setblocking (BOOL)

Whether blocking (default true), if set to false, then the accept and recv when there is no data, the error.

Sk.accept ()

Accepts the connection and returns (Conn,address), where Conn is a new socket object that can be used to receive and send data. Address is the location of the connection client.

Incoming TCP Client connection (blocked) waiting for connection

Sk.connect (Address)

The socket that is connected to the address. Generally, address is in the form of a tuple (Hostname,port) and returns a socket.error error if there is an error in the connection.

SK.CONNECT_EX (Address)

Ditto, but there will be a return value, the connection succeeds when the return 0, the connection fails when the return encoding, for example: 10061

Sk.close ()

Close socket

SK.RECV (Bufsize[,flag])

Accepts the data for the socket. The data is returned as a string, and bufsize specifies the maximum quantity that can be received. Flag provides additional information about the message, which can usually be ignored.

Sk.recvfrom (Bufsize[.flag])

Similar to recv (), but the return value is (data,address). Where data is the string that contains the received information, address is the socket addressing that sent the data.

Sk.send (Bytes[,flag])

Sends data from a string to a connected socket. The return value is the number of bytes to send, which may be less than the byte size of the string. That is, the specified content may not be sent all.

Sk.sendall (bytes[, Flag])

Sends data from a string to a connected socket, but attempts to send all data before returning. Successful return none, Failure throws an exception.

Internally, the send is called recursively, sending all the content.

Sk.sendto (bytes[, flag],address)

Sends the data to the socket, address is a tuple in the form of (Ipaddr,port), specifying the remote address. The return value is the number of bytes sent. This function is mainly used for UDP protocol.

Sk.settimeout (Timeout)

Sets the timeout period for the socket operation, and timeout is a floating-point number in seconds. A value of None indicates no over-time. In general, hyper-times should be set when a socket is just created, as they may be used for connected operations (such as client connections waiting up to 5s)

Sk.getpeername ()

Returns the remote address of the connection socket. The return value is typically a tuple (ipaddr,port).

Sk.getsockname ()

Returns the socket's own address. Typically a tuple (ipaddr,port)

Sk.fileno ()

File descriptor for sockets

3. Complex application

Use socket () to implement C/s and SSH related operations and solve sticky packet problems:

Service side before sending data, the length of the data sent to the client, to send how much data, and then the client based on the length of the data loop received to solve the sticky packet, transfer process: server: (1). Send #数据长度 (4). recv #收到确认信息, start next to send Sen D #发送数据 Client: (2). Recv #获取数据长度 (3). Send #发送确认信息 recv #循环接收

socket_ssh_nianbao_server.py

#!/usr/bin/env python#-*-coding:utf-8-*-import socketimport subprocessip_        add= (' 127.0.0.1 ', 9001) s = Socket.socket () s.bind (Ip_add) S.listen (5) While true:conn,addr = S.accept () while True: Try:recv_data = CONN.RECV (1024x768) If Len (recv_data) = = 0:break p = subprocess. Popen (str (recv_data,encoding= "Utf-8"), shell=true,stdout=subprocess. Pipe,stderr=subprocess. PIPE) res = P.stdout.read () Err = P.stderr.read () If Len (res) = = 0:conn.s End (err) Else:send_data = bytes ("ready|%s"%len (res), encoding= "Utf-8") Conn.send (Sen D_data) Feed_back = Conn.recv (1024x768) Start_tag = str (feed_back,encoding= "Utf-8") if Start_ Tag.startswith ("Start"): Conn.send (res) except Exception:break conn.close () 

socket_ssh_nianbao_client.py

#!/usr/bin/env python#-*-coding:utf-8-*-import socketip_add= (' 127.0.0.1 ', 9001) s = Socket.socket () s.connect (Ip_add) While True:    send_data = input (">>>:")    if Send_data = = ' exit ': Break    If Len (send_data) = = 0:continue    s.send (bytes (send_data,encoding= "Utf-8"))    Recv_tag = S.recv (1024x768)    Recv_tag = str (recv_tag,encoding=) Utf-8 ")    if Recv_tag.startswith (" Ready "):        msg_size = Int (Recv_tag.split (" | ") [1])    S.send (Bytes ("Start", encoding= "Utf-8"))    recv_size = 0    msg_data = B ' while    recv_size! = msg_size:        RECV_DATA=S.RECV (Recv_size+=len) (        recv_data)        msg_data+=recv_data        print (' message size%s recv Size%s '% (msg_size,recv_size))    print (str (msg_data,encoding= "Utf-8")) S.close ()

4. Set socket

SetSockOpt () and getsockopt (), one is the SET option, and the other is the one that gets set. Here the main use is setsockopt (), setsockopt (Level,optname,value), level defines which option will be used. Usually it is sol_socket, which means the SOCKET option is being used.

The optname parameter provides special options for use. The settings for the available options vary slightly depending on the operating system. If level Sol_socket is selected, some common options are shown in the following table:

A more common usage is that of setsockopt (socket. Sol_socket,socket. so_reuseaddr,1) Here value is set to 1, which means that the SO_REUSEADDR is marked as true, and the operating system releases the port of the server immediately after the server socket is closed or the server process terminates, otherwise the operating system retains the port for a few minutes.

Socketserver Socketserver uses IO multiplexing and "multithreading" and "multi-process" internally to enable the socket service side to process multiple client requests concurrently. That is, when each client requests a connection to the server, the socket server is creating a "thread" or "process" dedicated to all requests from the current client. The bottom or socket is encapsulated and joined threads, the process is implemented Socketserver, the following will be a source code analysis of Socketserver

Threadingtcpserver

Threadingtcpserver is the class that implements the socket server, which creates a " thread " For each client that is used to interact with the client.

Threadingtcpserver:

• Create a class that inherits from Socketserver.baserequesthandler
• A method called handle must be defined in a class
• Start Threadingtcpserver

1, Socketserver simple application, realize C/s interactive communication socketserver_server.py

#!/usr/bin/env python#-*-coding:utf-8-*-import socketserverimport subprocessimport osclass MyServer (socketserver. Baserequesthandler):    def handle (self):        conn = self.request        Conn.sendall (' Welcome to call 10086, please enter 1xxx, 0 to Manual service. ', encoding= "Utf-8")) while        True:            data = CONN.RECV (page)            if len (data) = = 0:break            cmd = Subprocess. Popen (Data.decode (), shell=true,stdout=subprocess. Pipe,stderr=subprocess. PIPE)            send_data = Cmd.stdout.read ()            err_data = Cmd.stderr.read ()            If Len (send_data) = = 0:                send_data = Bytes ("Current dirctory is%s"% os.getcwd (), encoding= "Utf-8")            if err_data:                send_data = Err_data            Conn.sendall (send_data) if __name__ = = ' __main__ ':    server = Socketserver. Threadingtcpserver ((' 127.0.0.1 ', 9000), MyServer)    server.serve_forever ()

socketserver_client.py

#!/usr/bin/env python#-*-coding:utf-8-*-import socketip_add= (' 127.0.0.1 ', 9000) s = Socket.socket () s.connect (Ip_add) Welcome_msg = S.RECV (1024x768) print (Welcome_msg.decode ()) while True:    send_data = input (">>>:")    if Send_ data = = ' exit ': Break    If Len (send_data) = = 0:continue    s.send (bytes (send_data,encoding= "Utf-8"))    recv_ data = S.RECV (1024x768)    print (str (recv_data,encoding= "Utf-8")) S.close ()

2, Socketserver Complex application: To achieve file upload similar

socketserver_ftp_server.py

#!/usr/bin/env python#-*-coding:utf-8-*-import socketserverimport jsonclass MyServer (socketserver. Baserequesthandler): def handle (self): conn = self.request Conn.sendall (' Welcome to call 10086, please enter the bytes to manual service. ', encoding= "Utf-8")) while True:data = Conn.recv () If Len (data) = = 0:break T            Ask_data = Json.loads (Data.decode ()) print (task_data) task_action = Task_data.get ("action") If Hasattr (self, "task_%s"%task_action): Func = GetAttr (self, "task_%s"%task_action) func (task_data) Else:print ("Task action is not supported", task_action) def task_put (self,*args,* *kwargs): Print ("put", Args,kwargs) file_size = Args[0].get (' filesize ') filename = args[0].get (' Filena        Me ') Server_response = {"Status": $ self.request.send (bytes (Json.dumps (server_response), encoding= "UTF")) With open ('/tmp/' + filename, ' WB ') as F: recv_size = 0 while recv_size! = File_size:data = Self.request.recv (4096) F.write (data) Recv_size + = len (data) print ("FileSize:%s recvsize:%s"% (File_size,rec v_size)) Print ("recv success") if __name__ = = ' __main__ ': Server = Socketserver. Threadingtcpserver ((' 127.0.0.1 ', 9000), MyServer) Server.serve_forever ()

socketserver_ftp_client.py

#!/usr/bin/env python#-*-coding:utf-8-*-import socketimport osimport jsonip_add= (' 127.0.0.1 ', 9000) s = Socket.socket ( ) S.connect (ip_add) welcome_msg = S.recv (1024x768) print (Welcome_msg.decode ()) While true:send_data = input ("&GT;&GT;&GT;:" ). Strip () if send_data = = ' exit ': Break If Len (send_data) = = 0:continue Cmd_list = Send_data.split () If Len (cm D_list) < 2:print ("Usage:put path/to/file") Continue Task_type = cmd_list[0] if Task_type = = ' Put ': Abs_filepath = cmd_list[1] If Os.path.isfile (abs_filepath): File_size = Os.stat (Abs_filepath).            st_size filename = abs_filepath.split ('/') [-1] Print (' file:%s size:%s '% (abs_filepath,file_size)) Msg_data = {"Action": "Put", "filename": filename, "filesize": file_size} s.send (bytes (json.dumps (msg_data) , encoding= "Utf-8")) Server_confirmation_msg = S.recv (1024x768) Confirm_data = Json.loads (Server_confirma            Tion_msg.decode ())If confirm_data[' status '] = = 200:print ("Start sending file", filename) with open (Abs_filepat H, ' RB ') as F:for line in F:s.send (line) print ("Send file does        Ne ") else:print (" \033[31m file [%s] is not exist \033[0m "%abs_filepath) Continue else: Print ("T ' support%s command"% task_type) Continue recv_data = S.RECV (1024x768) print (str (recv_data,enc oding= "Utf-8")) S.close ()

　　

　　

Python Automation Transport Koriyuki 14, network programming Socker (), Sockerserver ()