The Ssl,tls,https of Python Web Learning notes

First, SSL1. About SSL

The SSL protocol is located between the TCP/IP protocol and various application layer protocols, providing security support for data communication. The SSL protocol can be divided into two tiers:

SSL recording Protocol (SSL record Protocol): It is based on a reliable transport protocol (such as TCP) to provide high-level protocol data encapsulation, compression, encryption and other basic functions of support.

SSL Handshake Protocol (SSL handshake Protocol): It is based on the SSL logging Protocol, which is used to authenticate, negotiate cryptographic algorithms, exchange encryption keys, etc. before the actual data transfer begins.

The SSL protocol provides services such as: 1) authentication of users and servers, ensuring that data is sent to the correct client and server, 2) encrypting data to prevent data from being stolen, 3) maintaining data integrity and ensuring that data is not altered during transmission. the SL protocol uses asymmetric encryption technology to realize the secure transmission of information between two sessions. It can realize the confidentiality and integrity of information transmission, and the two sides can identify each other.

The goal is to provide a private message and reliability between two communication applications. This process is done with 3 elements:

1, handshake protocol. This protocol is responsible for negotiating the cryptographic parameters that are used for sessions between the client and server. When an SSL client and server begin communication for the first time, they agree on a protocol version, select the encryption algorithm, select mutual authentication, and use public key technology to generate the shared key.

2, record the agreement. This protocol is used to Exchange application layer data. Application messages are segmented into manageable chunks of data, compressed, and applied to a MAC (message authentication code), and the results are encrypted and transmitted. The receiving party accepts the data and decrypts it, verifies the Mac, unzip and re-assemble it, and submits the results to the application protocol.

3, warning agreement. This protocol is used to indicate when an error occurred, or when a session between two hosts was terminated.

The purpose of shaking hands

1. The client and server need to agree on a set of algorithms for protecting data;
2. They need to establish a set of cryptographic keys that are used by those algorithms;
3. The handshake can also optionally authenticate the client.

2. SSL process

Server Authentication phase:

1) The client sends a start message "Hello" to the server to start a new session connection;

"Negotiates a message encryption algorithm for encryption and a hash function for integrity checking. Typically, the client provides a list of all the algorithms it supports, and then the server chooses the most robust encryption algorithm. 】

2) The server determines whether a new master key needs to be generated based on the customer's information and, if necessary, the server will contain the information required to generate the master key in response to the customer's "Hello" information;

3) The customer generates a master key based on the received server response information and encrypts the server's public key to the server;

4) The server replies to the master key and returns it to the client to authenticate the server with a master key authentication message.

User Authentication phase:

Prior to this, the server has passed the customer certification, this phase is mainly to complete the customer certification.

The certified server sends a question to the customer, and the customer returns a (digital) signature question and its public key, thereby providing authentication to the server.

Detailed steps:

1. The user's browser sends its SSL version number, encryption setting parameters, session-related data, and other necessary information to the server. (negotiates a message encryption algorithm for encryption and a hash function for integrity checking.) Usually the client provides a list of all the algorithms it supports, and then the server chooses the most robust encryption algorithm.

2. The server sends its SSL version number, encryption setting parameters, session-related data, and some other necessary information to the browser, as well as the server's certificate to the browser. If you configure the server's SSL to authenticate the user, a request is made to require the browser to provide a user certificate.
3. The client checks the server certificate and if the check fails, the prompt cannot establish an SSL connection. If successful, then continue.
4. The client browser generates Pre-master secret (such as a random number) for this session, and encrypts it with the server public key and sends it to the server.
5. If the server requires authentication of the client, the client also signs additional data and sends it to the server along with the client certificate.
6. If the server requires authentication of the customer, check that the CA that signed the client certificate is trustworthy. If you are not in the trust list, end this session. If the check passes, the server decrypts the received pre-master secret with its own private key and uses it to generate the master secret for this session through certain algorithms.
7. Both the client and the server use this master secret to generate the session key (symmetric key) for this session. This session key is used when passing any message after the end of both SSL handshake. The main reason for this is that symmetric encryption is more than an order of magnitude lower than asymmetric encryption and can significantly increase the computational speed of both sessions.
8. The client notifies the server that the message sent thereafter is encrypted with this session key. and notifies the server that the client has completed this SSL handshake.
9. The server notifies the client that the message sent thereafter is encrypted with this session key. and notifies the client server that the SSL handshake has been completed.
10. The handshake process is complete and the session has been established. Both parties use the same session key to add and decrypt the information sent and received separately.

The advantage of the SSL protocol is that it provides connection security with 3 basic properties:

L The connection is private. After the initial handshake has defined a key, the encryption algorithm is used. Symmetric encryption (such as Des and RC4) is used for data encryption.

L can use asymmetric encryption or public-key cryptography (such as RSA and DSS) to verify the identity of the peer entity.

L reliable when connected. Message transfer uses a key for the MAC, including message integrity checks. A secure hash function (such as SHA and MD5) is used for Mac computing.

The degree of acceptance of SSL is limited to HTTP only. It has been shown to be available in other protocols, but it has not been widely used.

Second, TLS1. About TLS

TLS: Secure Transport Layer Protocol
(tls:transport Layer Security Protocol)
Secure Transport Layer Protocol (TLS) is used to provide confidentiality and data integrity between two communication applications. The protocol consists of two tiers: the TLS recording Protocol (TLS record) and the TLS Handshake Protocol (TLS handshake). The lower layer is the TLS logging protocol, located on a reliable transport protocol (for example, TCP). The TLS logging protocol provides two basic features for connection security:

Private-symmetric encryption for data encryption (DES, RC4, etc.). The keys generated by symmetric encryption are unique to each connection, and this key is negotiated based on another protocol, such as a handshake protocol.   Logging protocols can also be used without encryption.
Reliable-Information transfer includes the use of a key MAC for information integrity checks. The Secure hash function (SHA, MD5, and so on) is used for MAC computing.   The logging protocol can operate without a MAC, but it is generally only used in this mode, where another protocol is using the logging protocol to transport negotiated security parameters.
The TLS logging protocol is used to encapsulate various high-level protocols. The handshake protocol as one of these encapsulation protocols allows the server and the client to authenticate each other before the application protocol transmits and receives its first data byte, negotiating the encryption algorithm and encryption key. The TLS Handshake Protocol provides connection security with three basic properties:

The identity of the peer can be authenticated using an asymmetric, or public key cipher. This authentication is optional, but requires at least one node side.
The negotiation of a shared encryption key is secure. Negotiating encryption is hard to get for the thief. Additionally, authenticated connections cannot be encrypted, even if the attacker enters the middle of the connection.
the negotiations are reliable. No attacker can modify the communication negotiation without the detection of the communication party members.
the biggest advantage of TLS is that TLS is independent of the application protocol. High-level protocols can be transparently distributed over the TLS protocol. However, the TLS standard does not dictate how the application can increase security on TLS, and it leaves the Protocol's designer and implementers with the decision on how to start the TLS handshake protocol and how to interpret the Exchange's authentication certificate.


protocol Structure

The TLS protocol consists of two protocol groups ――tls logging protocol and TLS handshake protocol--each group has many different formats of information. In this document we only list the summary of the agreement and do not make a specific resolution. Specific content can be referred to the relevant documents.

The TLS logging protocol is a layered protocol. The information in each layer may contain fields such as length, description, and content. The logging protocol supports information transfer, segmentation of data into processing blocks, compression of data, application of MAC, encryption, and transmission results. The data received is decrypted, verified, decompressed, reorganized, etc., and then transmitted to the high-level client.

The TLS connection state refers to the operating environment of the TLS logging protocol. It specifies the compression algorithm, the encryption algorithm and the MAC algorithm.

The TLS record layer receives continuous data from a high level without empty blocks of any size. Key calculation: The logging protocol generates keys, IV, and MAC keys from the security parameters provided by the handshake protocol through the algorithm. The TLS handshake protocol consists of three sub-protocol groups, allowing the peer to agree on the security parameters of the recording layer, self-authentication, sample negotiation security parameters, and reporting error conditions to each other.

2. TLS processSimple TLS Handshake

A simple connection example follows, illustrating a handshake where the servers (but not the the client) are authenticated by it S Certificate:

1. Negotiation Phase:
   • A client sends a ClientHello message specifying the highest TLS protocol version it supports, a random number, a List of suggested Ciphersuites and suggested compression methods. If the client is attempting to perform a resumed handshake, it may send a session ID.
   • The server responds with a Serverhello message, containing the chosen protocol version, a random number, Ciphersu Ite and compression method from the choices offered by the client. To confirm or allow resumed handshakes the server may send a session ID. The chosen protocol version should is the highest that both the client and server support. For example, if the client supports TLS1.1 and the server supports TLS1.2, TLS1.1 should is selected; SSL 3.0 should not being selected.
   • The server sends its Certificate message (depending in the selected cipher suite, this could be omitted by the Serv ER). ^[+]
   • The server sends a serverhellodone message, indicating it is do with handshake negotiation.
   • The client responds with a clientkeyexchange message, which could contain a premastersecret, public key, O R nothing. (Again, this depends on the selected cipher.) This Premastersecret are encrypted using the public key of the server certificate.
   • The client and server then use the random numbers and Premastersecret to compute a common secret, called the "Mas ter Secret ". All other key data for this connection are derived from this master secret (and the Client-and server-generated random Val UES), which is passed through a carefully designed pseudorandom function.
2. • Finally, the client sends an authenticated and encrypted  finished  message, Containi Ng a hash and MAC over the previous handshake messages.
   • the server would attempt to decrypt the client ' s  finished  message and verify the hash and MAC. If the decryption or verification fails, the handshake is considered to has failed and the connection should being torn down .
3. finally, the server sends a  Changecipherspec , telling the client, "Everything I tell you from now on would be authenticated (and Encry Pted, if encryption was negotiated). " Application Phase:at This point, the "handshake" are complete and the application protocol are enabled, with content type O F 23. Application messages exchanged between client and server would also be authenticated and optionally encrypted exactly like In their  finished  message. Otherwise, the content type would return and the client would not authenticate. 
4. the client performs the same decryption and verification .

Swap key:

All processes

the difference between **ssl and TLS

The latest version of TLS (Transport layer Security, transport-layer secure protocol) is a new protocol developed by the IETF (Internet Engineering Task force,internet Engineering Task Force), which is built on SSL 3.0 protocol specification, is a subsequent version of SSL 3.0. There is a significant difference between TLS and SSL3.0, mainly because they support different cryptographic algorithms, so TLS and SSL3.0 cannot interoperate.
1. The difference between TLS and SSL
1) Version number: The TLS record format is the same as the SSL record format, but the version number has a different value, and TLS version 1.0 uses the version number SSLv3.1.
2) message authentication code: MAC algorithm of SSLv3.0 and TLS and Mac computing range are different. TLS uses the HMAC algorithm defined by the RFC-2104. SSLV3.0 uses a similar algorithm, the difference is in the SSLv3.0, the padding between the bytes and the key is the connection operation, and the HMAC algorithm is an XOR operation. But the degree of security is the same.
3) Pseudo-random function: TLS is a more secure way to use pseudo-random functions called PRF to extend a key into a block of data.
4) Alarm code: TLS supports almost all SSLv3.0 alarm codes, and TLS also complements the definition of many alarm codes, such as decryption failure (decryption_failed), record overflow (record_overflow), unknown CA (UNKNOWN_CA) , Access Denied (access_denied), and so on.
5) Ciphertext and client certificates: There is a small difference between SSLv3.0 and TLS that TLS does not support Fortezza key exchange, cryptographic algorithms, and client certificates.
6) Certificate_verify and finished messages: SSLv3.0 and TLS calculate a little difference in the calculated input when calculating finished and MD5 hash codes with certificate_verify and SHA-1 messages, but with equal security.
7) Encryption calculation: TLS and SSLv3.0 are used in different ways when calculating the master secret.
8) padding: The padding bytes that need to be added before user data is encrypted. In SSL, the length of the filled data is the smallest integer multiple of the length of the ciphertext block. In TLS, the length of the filled data can be any integer multiple of the length of the cipher block (but the maximum length of the padding is 255 bytes), which prevents attacks based on parsing the length of the message.
2. Key enhancements to TLS
The main goal of TLS is to make SSL more secure and to make the protocol more precise and sophisticated. Based on SSL v3.0, TLS provides the following enhancements:
1) more secure Mac algorithm;
2) tighter alarms;
3) A clearer definition of the "grey area" specification;
3. TLS improvements for security
1) Use key hashing for message authentication: TLS uses the "key hashing of Message authentication Code" (HMAC), which ensures that records are not changed when recorded on an open network (such as the Internet). SSLV3.0 also provides keyed message authentication, but HMAC is more secure than the SSLV3.0 (message authentication code) MAC feature used.
2) Enhanced pseudo-random function (PRF): PRF generates key data. In TLS, the HMAC defines the PRF. PRF uses two hashing algorithms to ensure its security. If any of the algorithms are exposed, the data remains secure as long as the second algorithm is not exposed.
3) Improved completed message validation: Both TLS and SSLv3.0 provide completed messages to two endpoints, and the message that authenticated the interchange has not been changed. However, TLS bases this completed message on the PRF and HMAC values, which is also more secure than SSLv3.0.
4) Consistent certificate processing: Unlike SSLv3.0, TLS attempts to specify the type of certificate that must be exchanged between TLS.
5) Specific Alert message: TLS provides more specific and additional alerts to indicate an issue detected by any of the session endpoints. TLS also records when certain alerts should be sent.

Third, HTTPS1. Brief introduction

Used to compress and decompress the data, and return the results that are sent back on the network. HTTPS actually applies the Netscape Full Sockets Layer (SSL) as a sub-layer of the HTTP application layer. (HTTPS uses port 443 instead of using port 80来 and TCP/IP to communicate like HTTP.) SSL uses 40-bit keywords as the RC4 stream encryption algorithm, which is appropriate for the encryption of business information. HTTPS and SSL support use of the digital authentication of the number, and if necessary, the user can confirm who the sender is.

HTTPS is a security-targeted HTTP channel, which is simply the security version of HTTP. That is, the SSL layer is added under HTTP, the security base of HTTPS is SSL, so the details of encryption see SSL.  It is a URI scheme (abstract identifier system) with syntax similar to http: System. For secure HTTP data transfer. Https:url indicates that it uses HTTP, but HTTPS has a different default port than HTTP and an encryption/authentication layer (between HTTP and TCP). The initial development of the system, conducted by Netscape, provides an authentication and encryption method of communication, which is now widely used in security-sensitive communications on the World Wide Web, such as transaction payments.  

The main idea of HTTPS is to create a secure channel on an insecure network and to provide reasonable protection against eavesdropping and inter-person attacks when using the appropriate cryptographic suite and server certificates can be verified and trusted .

The trust inheritance for HTTPS is based on a certificate authority that is preinstalled in the browser (such as VeriSign, Microsoft, and so on) (meaning "I trust the certification authority to tell me that I should trust"). Therefore, an HTTPS connection to a Web site can be trusted, if and only if:

1. Users believe that their browsers correctly implement HTTPS and that the correct certificate authority is installed;
2. The user trusts that the certification authority trusts only legitimate websites;
3. The website being visited provides a valid certificate, meaning that it is issued by a trusted certificate authority (most browsers warn of invalid certificates);
4. The certificate correctly validates the visited website (for example, when it was accessedhttps://exampleto "Example Inc"). Rather than the certificates of other organizations);
5. or the relevant node on the Internet is trustworthy, or the user believes that the encryption layer (TLS or SSL) of this protocol cannot be compromised by the listener.

2. The difference from HTTP

The URL to HTTP starts with "http://" and differs by default using port 80, and the URL for httpshttps://starts with "" and uses port 443 by default.

HTTP is unsafe, and attackers can gain access to website accounts and sensitive information by means of eavesdropping and inter-person attacks. HTTPS is designed to prevent the aforementioned attacks and is considered secure (when no older versions of SSL are used).

3. Network layer

HTTP works at the application layer (the highest layer of the OSI model), but the security protocol works on a lower sub-layer: Encrypts the HTTP message before it is transmitted and decrypts it when it arrives. Strictly speaking, HTTPS is not a separate protocol, but rather a salutation to the regular HTTP protocol that works on an encrypted connection (TLS or SSL).

Everything in the HTTPS message is encrypted, including all headers and loads. In addition to the possible CCA (see Restrictions section), an attacker can only know the fact that there is a connection between the two.

4. Server Settings

For a network server to be ready to accept HTTPS connections, the administrator must create a digital certificate and be signed by the certification authority to allow the browser to accept it. The certification authority verifies that the digital certificate holder and its claims are the same person. The browser usually has the certificate authority preinstalled, so they can verify the signature.

5. Obtaining a certificate

Certificates issued by a certification authority are free of charge ^{[3][4] and also have a yearly fee of $13 [5] to $1500 [6].}

An organization may also have its own certification authority, especially when setting up a browser to access their own website (for example, a website running on a corporate LAN, or university). They can easily add their own certificates to the browser.

In addition, there is a person-to-person certification authority, CAcert.

6. As Access control

HTTPS can also be used as a client authentication tool to restrict some information to legitimate users. To do this, the administrator typically creates a certificate for each user (typically containing the user's name and e-mail address). This certificate is placed in the browser and is checked by the server each time it connects to the server.

7. When the private key is compromised

The certificate can be revoked before it expires, usually because the private key of the certificate has been compromised. Newer browsers such as Google Chrome, Firefox^{[7], Opera[8], and Internet Explorer [9] running on Windows Vistaimplement the online certificate Status Protocol ( English :Online Certificate Status Protocol) (OCSP) to exclude this scenario: The browser sends the serial number of the certificate provided by the Web site to the certification authority via OCSP, which tells the browser whether the certificate is still valid. [ten]}

8. Limitations

TLS has two policies: simple policy and interactive policy. Interactive policies are more secure, but require users to install personal certificates in their browsers for authentication.

Regardless of which strategy is used, the protection provided by the Protocol is always strongly dependent on the implementation of the browser and the cryptographic algorithms supported by the server software.

HTTPS does not prevent sites from being crawled by web spiders. In some cases, the URL of an encrypted resource can be pushed only by the size of the intercept request and response,^{[11] which allows the attacker to know both plaintext (exposed static content) and ciphertext (plaintext encrypted), making it possible to select ciphertext attacks.}

Because SSL works under HTTP and knows nothing about the upper layer protocol, the SSL server can only provide a single certificate for an IP address/port combination. ^{[12] This means that in most cases, it is impractical to support name-based virtual hosts while using HTTPS. A scenario called Server name Indication( English :server name Indication) (SNI) resolves this issue by sending a host name to the server before the encrypted connection is created.}

The Ssl,tls,https of Python Web Learning notes