Analog ssh, sticky pack, Hashlib module

First, Analog SSH

1. subprocess Module

The Subprocess module is a module introduced by Python starting with version 2.4. It is mainly used to replace some old module methods, such as Os.system, os.spawn*, os.popen*, commands.* and so on. The subprocess module can be used to generate a process and connect to the process's input/output/Error output pipeline and get the return value of the process.

    Import subprocess    res = subprocess. Popen ("dir",                            shell=true,                            stderr=subprocess. PIPE,                            stdout=subprocess. PIPE)    # pipe represents pipeline    # Stdin,stdout,stderr: Represents the program's standard input, standard output, standard error, respectively.    print (Res.stdout.read (). Decode ("GBK")    # The encoding of the result is based on the current system, if it is Windows,    # then Res.stdout.read () Read out is GBK encoded, at the receiving end need to use GBK decoding

　　

2, write a simple simulation SSH program, that is, the implementation of the client to the server to send a terminal command, the service side executes this command, the execution results are returned to the client, the following example:

    ImportSocketImportsubprocess Server=socket.socket () Server.bind ('127.0.0.1', 8008)) Server.listen (5) conn,addr=server.accept () while1: Data= Conn.recv (1024x768). Decode ('UTF8') Res=subprocess. Popen (data, Shell=True, stderr=subprocess. PIPE, stdout=subprocess. PIPE) Out_result= Res.stdout.read ()#results after an error has occurredErr_result = Res.stderr.read ()#command the correct result        ifout_result:conn.send (Out_result)Print('Sent to client','the number of bytes is', Len (out_result))eliferr_result:conn.send (Err_result)Print('error occurred','the number of bytes is', Len (Err_result))

Service Side

    ImportSocket Client=socket.socket () Client.connect ('127.0.0.1', 8008))     while1: Comm= Input ('Please enter a command >>>'). Encode ('UTF8') Client.send (comm) Data= CLIENT.RECV (1024)        Print(Data.decode ('GBK'))

Client

Analysis: Try running the above example to analyze if there is a problem?

Second, sticky bag

1, sticky packet phenomenon: After executing multiple commands at the same time, the result is likely to be only part of the results, when executing other commands and receiving another part of the result of the previous execution, this phenomenon is sticky packet. The last server-side code in the simulated SSH example above may appear sticky, because the service-side code writes two consecutive send (a send length, a Send data content), send does not block, so two send will form a packet sent in the past, The receiver cannot tell which data is sent, nor can it get the length and content. So how to solve the sticky bag phenomenon?

2. The origin of sticky bag

A, data transfer in the TCP protocol

TCP protocol unpacking mechanism: When the length of the transmit buffer is greater than the MTU of the NIC, TCP will split the sent data into several data packets sent out. The MTU (Maximum transmission Unit) means the largest packet transmitted over the network. The unit of the MTU is bytes. Most network devices have an MTU of 1500, and if the MTU of the machine is larger than the MTU of the gateway, large packets are removed and sent, which can result in a lot of packet fragmentation, increased packet loss, and reduced network speed.

Stream-oriented communication features and Nagle algorithms: TCP (Transport Control Protocol, transmission Protocol) is connection-oriented, stream-oriented and provides high reliability services. Both ends of the transceiver (client and server side) have one by one pairs of sockets, so the sending side in order to send multiple packets to the receiver, more efficient to the other side, the use of the optimization method (Nagle algorithm), the multiple interval small and small data volume data, combined into a large block of data, and then to the packet. In this way, the receiving end is difficult to distinguish, must provide a scientific unpacking mechanism. That is, stream-oriented communication is a non-message-protected boundary. For the empty message: TCP is based on data flow, so send and receive messages can not be empty, which requires the client and the server to add a null message processing mechanism to prevent the program stuck, and UDP is based on the datagram, even if you enter empty content (direct carriage), can also be sent, The UDP protocol will help you encapsulate the message hair sent over. Reliable Packet TCP protocol: TCP protocol data will not be lost, the packet is not finished, the next time it receives, will continue to receive the last time, one end will always receive an ACK when the buffer content is clear. The data is reliable, but sticky.

The causes of sticky packet phenomenon based on TCP protocol characteristics:

：

　　　　　　　　　　

The     sender can be a K-K to send the data, and the receiving side of the application can be two K two k to take the data, of course, it is possible to take 3 K or 6K data at a time, or only a few bytes of data at a time. That is, the data the application sees is a whole, or a stream, and how many bytes of a message are not visible to the application, so the TCP protocol is a stream-oriented protocol, which is also the cause of the sticky-packet problem. And    UDP is a message-oriented protocol, each UDP segment is a message, the application must be in the message to extract data, not one time to extract arbitrary bytes of data, which is very different from TCP.    How do you define a message? You can think of the other party write/send data as a message, it is necessary to understand that when the other side send a message, regardless of the underlying how fragmented shards, the TCP protocol layer will constitute the entire message of the data segment is completed before rendering in the kernel buffer.

The cause of the sticky bag phenomenon

For example, the TCP-based socket client uploads a file to the server, sending the file content is sent in accordance with a paragraph of the byte stream, in the view of the receiver, there is no way to know where the text stream from where the file ends;

In addition, the packet caused by the sender is caused by the TCP protocol itself, TCP to improve transmission efficiency, the sender often to collect enough data before sending a TCP segment. If there are few data to send in a few consecutive times, TCP will usually send the data to a TCP segment based on the optimization algorithm, and the receiver receives the sticky packet data.

B, UDP does not occur sticky packets

UDP (User Datagram Protocol, Subscriber Datagram Protocol) is non-connected, message-oriented, providing efficient service. The Block merging optimization algorithm is not used, because UDP supports a one-to-many pattern, so the receiver Skbuff (socket buffer) uses a chain structure to record each incoming UDP packet, in each UDP packet there is a message header (message source address, port and other information), so for the receiving end , it is easy to distinguish between the processing. That is, message-oriented communication is a message-protected boundary.

For the empty message: TCP is based on data flow, so send and receive messages can not be empty, which requires the client and the server to add a null message processing mechanism, to prevent the program stuck, and UDP is based on the datagram, even if you enter the empty content (direct carriage), can also be sent, The UDP protocol will help you encapsulate the message hair sent over.

Unreliable non-sticky UDP protocol: UDP recvfrom is blocked, a recvfrom (x) must be the only one sendinto (y), the data after the X-byte is completed, if the y;x data is lost, which means that UDP is not sticky packets, but will lose data, Not reliable.

Summary: Sticky packets occur only in the TCP protocol:

1) on the surface, the problem of sticky packet is mainly due to the caching mechanism of sender and receiver, and the characteristic of the TCP protocol oriented stream communication.

2) In fact, mainly because the receiver does not know the boundary between the message, do not know how many bytes of data extraction caused;

3. struct MODULE

This module converts the length of data to be sent to a fixed-length bytes (bytes), as shown in the following example:

    import struct    res1 = struct.pack ("i", 23767)    print (res1)  # b ' \xd7\\\x00\x00 '    print (len (res1))  # 4       fixed length is 4 bytes    obj1 = Struct.unpack ("i", res1)    print (Obj1, obj1[0])  # (23767,) 23767

　

4. Solutions for Sticky packs

We can use the struct module, which converts the length of data to be sent to a fixed-length byte. In this way, each time a client receives a message, it takes a fixed-length byte to take a look at the size of the information that is to be received, so that the final accepted data is stopped as soon as it reaches that value, and it will be able to receive the complete data in just a few more.

　

When sending	When receiving
Send a struct converted data length 4 bytes First	Accept 4 bytes To get the length of the data to receive by using a struct to convert to a number
Send data again	And then receive the data by length

So the above simulated SSH example can eventually be changed to the following code:

Service side

Import Socket    Import subprocess    import struct    server = Socket.socket ()    server.bind (' 127.0.0.1 ', 8008))    Server.listen (5)    conn,addr = Server.accept () while    1:        data = CONN.RECV (1024x768). Decode (' UTF8 ')        res = subprocess. Popen (data,                           shell = True,                           stderr=subprocess. PIPE,                           stdout=subprocess. PIPE        )        Out_result = Res.stdout.read ()          Err_result = Res.stderr.read ()          if Out_result:            Conn.send (Struct.pack (' I ', Len (Out_result)) # Build Header            conn.send (out_result)  # Send Message            print (' Sent to client ', ' Number of bytes is ', Len (out_result))        elif Err_result:            conn.send (Struct.pack (' I ', Len (Err_result))  # Build Header            Conn.send (Err_result)  # Send Message            print (' Error ', ' byte Count ', Len (Err_result))

Client

Import socket    Import struct    client = Socket.socket ()    client.connect ((' 127.0.0.1 ', 8008)) while    1:        comm = input (' Please enter Command >>> '). Encode (' UTF8 ')        client.send (comm)        Data_len = Struct.unpack (' i ', CLIENT.RECV (4)) [0]        Data_all = B ' while        len (Data_all) < data_len:            data = CLIENT.RECV (1024x768)            Data_ All + = Data        print (Data_all.decode (' GBK '))

　　

Three, Hashlib module

1, Algorithm Introduction

Python's hashlib provides a common digest algorithm, such as MD5,SHA1 and so on.

What is a digest algorithm? Abstract the algorithm is also called hash algorithm and hashing algorithm. It uses a function to convert any length of data into a fixed length data string (usually represented by a 16-binary string).

Abstract algorithm is to find out whether the original data has been tampered with as long as the function F1 () calculates the fixed-length summary digest for the data of arbitrary length.

Abstract the algorithm can point out whether the data has been tampered with, because the digest function is a one-way function, computing (data) is very easy, but it is very difficult to push data through digest, and to make a bit of the original data modification, will result in a completely different calculation of the summary.

We've learned a simple example of using MD5 encryption, as follows:

Import hashlib    MD5 = HASHLIB.MD5 ()    md5.update (b ' How to use MD5 in Python hashlib? ')    Print (Md5.hexdigest ())  # d26a53750bc40b38b65a520292f69306

If you have a large amount of data, you can call Update () multiple times, and the result is the same, as in the following example:

    Import hashlib    MD5 = HASHLIB.MD5 ()    md5.update (b ' How to use MD5 in ')    md5.update (b ' python hashlib? ')    Print (Md5.hexdigest ())  # d26a53750bc40b38b65a520292f69306

MD5 is the most common digest algorithm and is fast enough to generate a fixed 128bit byte, typically represented by a 32-bit 16-character string. Another common as long as the algorithm is SHA1, calling SHA1 and calling MD5 are exactly like the following example:

    Import hashlib    SHA1 = HASHLIB.SHA1 ()    sha1.update (b ' How to use SHA1 in ')    sha1.update (b ' python hashlib? ')    print (Sha1.hexdigest ())  #     2c76b57293ce30acef38d98f6046927161b46a44

The result of the SHA1 is a bit byte, which is usually represented by a 40-bit 16 binary string. Algorithms that are more secure than SHA1 are SHA256 and SHA512, but the more secure the algorithm is, the slower it is, and the longer the digest length.

2. Algorithm Application

Any site that allows users to log on will store the user name and password that the user is logged on to. How do I store a user name and password? method is stored in the database table:

　

    Name    | password    --------+----------    Michael | 123456    Bob     | abc999    alice   | alice2008

If the user password is saved in clear text, if the database is compromised, all users ' passwords fall into the hands of the hacker. In addition, the site operators can access the database, that is, to get all the user's password. The correct way to save a password is not to store the user's plaintext password, but instead to store a digest of the user's password, such as MD5:

　

    Username | Password    ---------+---------------------------------    Michael  | e10adc3949ba59abbe56e057f20f883e    Bob      | 878ef96e86145580c38c87f0410ad153    Alice    | 99b1c2188db85afee403b1536010c2c9

Consider such a situation, many users like to use 123456,888888,password these simple password, so, the hacker can calculate in advance these common password MD5 value, get a counter-push table:

    ' e10adc3949ba59abbe56e057f20f883e ': ' 123456 '    21218cca77804d2ba1922c33e0151105 ': ' 888888 '    5f4dcc3b5aa765d61d8327deb882cf99 ': ' Password '

This way, no need to crack, only need to compare the database MD5, hackers get the use of common password user account.

For the user, of course, do not use too simple password. But can we enhance the protection of simple passwords in program design?

Because the MD5 value of the common password is easy to calculate, so to ensure that the Stored user password is not the MD5 of the commonly used password, this method is implemented by adding a complex string to the original password, commonly known as "Add salt":

    HASHLIB.MD5 ("Salt". Encode ("Utf-8"))

　

Salt processing of the MD5 password, as long as the salt is not known by hackers, even if the user entered a simple password, it is difficult to MD5 the plaintext password.

But if two users all use the same simple password such as 123456, in the database, two identical MD5 values will be stored, which means that the password of the two users is the same. Is there a way for users with the same password to store different MD5?

If the user cannot modify the login, it is possible to calculate the MD5 by using the login as part of the salt, so that users who implement the same password also store different MD5.

Abstract algorithms are widely used in many places. Note that the digest algorithm is not an encryption algorithm and cannot be used for encryption (because plaintext cannot be reversed by the digest), but it is only used for tamper-proof, but its one-way computing feature determines that the user's password can be verified without storing the plaintext password.

Iv. Supplementary Knowledge

data = CONN.RECV (1024) # Blocking

Listener SOCKET Object Conn Disconnected:

For Windows: Exception error.

For Linux: Receive an empty data.

　

　　

Analog ssh, sticky pack, Hashlib module