Using SSL (HTTPS) in Python

There are many scenarios for using SSL on Python, and my main concern is to use Python to access HTTPS resources and to use Python to provide HTTPS services. (HTTPS is one of the SSL applications on the Web)

I. Using python to access HTTPS websites

This should be the simplest and most common scenario. We use Python as a client to access the public Web site, and this site for the transmission of security (to avoid hijacking or eavesdropping) using the HTTPS service, the transfer process content has been SSL encryption. Here's a look at the specific Python code, which uses python2.7.11, which is the Python-urllib2. Of course you can also use requests or Pycurl and other third-party libraries, all can, the principle is the same, but the means to achieve some differences.

Import Urllib2

Import SSL

if __name__ = = ' __main__ ':

Myurl= "Https://www.baidu.com"

req = Urllib2. Request (Myurl)

Try

Response = Urllib2.urlopen (req)

Print "HTTP return code:%d"% Response.getcode ()

strresult= Response.read ()

Print Strresult

Except Exception, ex:

Print "Found Error:%s"% str (ex)

Run it and we can get the content on this website. And the transfer process is encrypted. is not very simple. The entire transfer process is probably the case, the client requests the SSL connection handshake (which will be with the server some SSL protocol interaction between, this we do not have to study in detail) the server to pass its own certificate to the client, The client authenticates the certificate (the certificate is installed by default on each client computer, which is used by the certificate authority), and the Python SSL module automatically loads the authoritative CA certificates, which you can also specify yourself, which will be discussed later. Ensure that the certificate is issued by a trusted CA (the client authenticates the service-side certificate, which is from python2.7.9, the previous version is not authenticated), and then uses the certificate to exchange the key to encrypt the transmission using the key.

Most of the internet on the regular website will have their own certificate, these certificates are authoritative CA certification, can be considered credible (in fact, this is not necessarily, some time ago there was a Google banned Symantec CA issued by the certificate of the event, interested can go online search to see). But there are also many sites, for a variety of reasons without these authoritative CA-certified certificates, after all, the application of these certificate process is more troublesome, and to pay a certain fee. These sites can also provide secure HTTPS services, but because their certificates are self-signed or issued by a non-authoritative CA (both of which are discussed later), they are considered untrustworthy. When using a browser to open these sites, the browser will remind you that this is not a secure connection, of course, this insecurity refers to the site is not authoritative certification, so the site itself may be unsafe, but on the network transmission level, the transmission is safe. When we visit these sites, using the above code will have an error, the following code is to access a This is my own establishment of an HTTPS website https://127.0.0.1:8443, the site uses a self-signed certificate. If you use Google Browser Access will prompt you: Your connection is not a private connection, if you force the connection will be prompted unsafe

Import Urllib2

Import SSL

if __name__ = = ' __main__ ':

Myurl= "https://127.0.0.1:8443"

req = Urllib2. Request (Myurl)

Try

Response = Urllib2.urlopen (req)

Print "HTTP return code:%d"% Response.getcode ()

strresult= Response.read ()

Print Strresult

Except Exception, ex:

Print "Found Error:%s"% str (ex)

Running this will cause an error:

Prompt for certificate validation errors. At this point, if we want to access such a website, the client's certificate check is turned off. In front add a sentence: Ssl._create_default_https_context = Ssl._create_unverified_context can

Import Urllib2

Import SSL

Ssl._create_default_https_context = Ssl._create_unverified_context

if __name__ = = ' __main__ ':

Myurl= "https://127.0.0.1:8443"

req = Urllib2. Request (Myurl)

Try

Response = Urllib2.urlopen (req)

Print "HTTP return code:%d"% Response.getcode ()

strresult= Response.read ()

Print Strresult

Except Exception, ex:

Print "Found Error:%s"% str (ex)

Of course, you can create a non-verifiable SSL context yourself, and then reference that context to open the URL

CTX = Ssl._create_unverified_context ()

And then

Response = Urllib2.urlopen (REQ,CONTEXT=CTX)

Ii. using OpenSSL to generate certificates

1. Generate a self-signed certificate

Before using Python to provide HTTPS services, we need a certificate from Mr. Https://www.cnblogs.com/wucao/archive/2017/02/27/6474711.html, please refer to the This article explains how to generate a certificate using OpenSSL. On the Windows platform remember to set the environment variables, refer to my blog post http://blog.sina.com.cn/s/blog_5d18f85f0102xdm7.html.

Use command: OpenSSL req-x509-newkey rsa:2048-nodes-days 365-keyout private.pem-out cert.crt

You will then be asked to enter some organization information, you can fill in according to your actual situation. The self-signed certificate cert.crt is then generated, and the private key is Private.pem

2. Establish a CA (certificate issuing authority), and then use your own CA to issue certificates. This will be used when we discuss two-way authentication later. For more information, please refer to http://blog.csdn.net/gx_1983/article/details/47866537

Here, I'm simply writing my own steps.

1) Ensure that the bin directory of OpenSSL is inside the PATH environment variable. Create a working directory, where I use the CA directory. Then create the directory Democa below the CA. After creating a blank text file Index.txt and serial under Democa, and opening the serial write character 01

2) first to cause CA key and certificate

OpenSSL req-new-x509-days 36500-key ca.key-out ca.crt

After the implementation of the prompt, the main requirement is to enter some information on the organization, please fill in the required

Successful execution will result in Ca.key and ca.crt two files, Ca.key for the private key needs to be properly kept, not easily to others. CA.CRT can be freely propagated for certificates.

3) generate the private key and certificate for the server, where the certificate is signed with the CA's private key:

L Generate server Private key

OpenSSL genrsa-out Server.key 2048

L Generate server-side certificate request file using server private key

OpenSSL Req-new-key server.key-out SERVER.CSR

Need to answer some organizational questions as well.

L Use the server certificate request file to generate a CA-signed certificate from the CA

OpenSSL ca-in server.csr-out server.crt-cert ca.crt-keyfile Ca.key

Verifying server certificates

OpenSSL verify-cafile ca.crt server.crt

This will get two files: one is the private key server.key; the other is the certificate server.crt

4) using the same method as 3 to generate the client's Client.key and client.crt the two files will be used in the following two-way authentication

Third, using Python to provide HTTPS services

Now that we have the certificate and the private key, we can formally use Python to build an HTTPS website. Here I use the framework to implement, using the twisted. The certificate used is a self-signed certificate that was previously generated with OpenSSL.

#-*-coding:utf-8-*-

From Twisted.web import server, Resource

From twisted.internet import Reactor,ssl

Class Mainresource (Resource. Resource):

IsLeaf = True

# for handling Get type requests

def render_get (self, request):

# NAME parameter

name = ' World '

If Request.args.has_key (' name '):

name = request.args[' name '][0]

# Set Response encoding

Request.responseHeaders.addRawHeader ("Content-type", "text/html; Charset=utf-8 ")

# The contents of the response are returned directly

Return "

Hello, "+ Name +" "

if __name__ = = ' __main__ ':

Sslcontext = SSL. Defaultopensslcontextfactory (

' C:/ca/private.pem, # private key

' C:/CA/CERT.CRT ' # certificate

)

site = Server. Site (Mainresource ())

Reactor.listenssl (8080, site, sslcontext)

Print "Listening port: 8080"

Reactor.run ()

Remember to turn off certificate validation before using Client access. You can now see the smooth access. If you use the browser access, you can also access, but the security prompt, ignoring this security prompt can also be accessed normally.

Four, SSL bidirectional check

The above example is a client-side check of the server certificate. This is also the most common one-way check. This is where the client verifies the server-side certificate (and, of course, it can choose not to verify). The usual Internet service is this way. In this way, the client does not need to have a certificate. The server side also does not validate the client's certificate. There is also a bidirectional check mode. In this mode, the client also has a certificate and is provided to the server side during the negotiation process. The server also verifies the client's certificate. This pattern is typically applied to environments where security requirements are high, and both the server and the client require certificates, and both must be validated by the authoritative ca. Look at the following server-side code:

#-*-coding:utf-8-*-

From Twisted.web import server, Resource

From twisted.internet import Reactor,ssl

From OpenSSL import SSL

def verifycallback (connection, X509, Errnum, errdepth, OK):

If not OK:

print ' Invalid cert from subject: ', X509.get_subject ()

Return False

Else

Print "Certs is fine", X509.get_subject ()

Return True

Class Mainresource (Resource. Resource):

IsLeaf = True

# for handling Get type requests

def render_get (self, request):

# NAME parameter

name = ' World '

If Request.args.has_key (' name '):

name = request.args[' name '][0]

# Set Response encoding

Request.responseHeaders.addRawHeader ("Content-type", "text/html; Charset=utf-8 ")

# The contents of the response are returned directly

Return "

Hello, "+ Name +" "

if __name__ = = ' __main__ ':

Sslcontext = SSL. Defaultopensslcontextfactory (

' C:/ca/private.pem, # private key

' C:/CA/CERT.CRT ' # certificate

)

CTX = Sslcontext.getcontext ()

#这里改为了双向校验模式

Ctx.set_verify (

Ssl. Verify_peer | Ssl. Verify_fail_if_no_peer_cert,

Verifycallback

)

site = Server. Site (Mainresource ())

Reactor.listenssl (8080, site, sslcontext)

Print "Listening port: 8080"

Reactor.run ()

Then use the client to visit. You will see an error occurred with the error:

Tip SSLv3 protocol handshake failed. The reason is that the server side requires two-way verification, but the client does not provide its own certificate to the server, so handshake negotiation failed.

OK, let's change the client code to fit this bidirectional check mode. The certificate used by the client here is the client certificate that was generated prior to using OpenSSL, and is signed with a CA that we have established ourselves.

Import Urllib2

Import SSL

Key_file= "C:/ca/client.key"

Cert_file= "C:/CA/CLIENT.CRT"

context = SSL. Sslcontext (SSL. PROTOCOL_SSLV23)

Context.check_hostname = False

Context.load_cert_chain (Certfile=cert_file,keyfile=key_file)

Context.verify_mode=ssl. Cert_required

if __name__ = = ' __main__ ':

Myurl= "HTTPS://127.0.0.1:8080/?NAME=QH"

req = Urllib2. Request (Myurl)

Try

Response = Urllib2.urlopen (Req,context=context)

#response = Urllib2.urlopen (req)

Print "HTTP return code:%d"% Response.getcode ()

strresult= Response.read ()

Print Strresult

Except Exception, ex:

Print "Found Error in auth phase:%s"% str (ex)

Error message appears here

Certificate validation failed. This is because the server-supplied certificate does not pass the client's checksum.

This is because the server-side certificate is using a self-signed certificate, which of course does not pass the client check. Here we'll change the server-side certificate to the certificate we signed with the CA. Modify the server-side code:

Sslcontext = SSL. Defaultopensslcontextfactory (

' C:/ca/server.key ', # private key

' C:/CA/SERVER.CRT ' # certificate

)

If you try again after you have changed, you can still make the same mistake, what's the matter? The original server-side certificate was signed with our self-built CA, not the authoritative CA, so the checksum failed. What about this, we can specify the CA's certificate and set our CA certificate to be trusted. Modify the client, plus one sentence:

Ca_file= "C:/CA/CA.CRT"

Context.load_verify_locations (Ca_file)

Note that CA.CRT is the CA certificate we established before using OpenSSL (refer to the second part of the previous section). Re-run after modification, or error is found:

This error has changed to indicate that the CA is not well known. The callback function on the server side also has an error output: Invalid cert from subject:

This indicates that the server-side check client certificate failed, because of course the server side we did not set our own CA to be trusted. So the same on the server side to modify, plus the following two sentences:

Cafile= "C:/CA/CA.CRT"

Ctx.load_verify_locations (Cafile)

Re-run, this is successful.

The complete code is sent below:

Server-side:

#-*-coding:utf-8-*-

‘‘‘

Created on 2018-1-16

@author: QH

‘‘‘

From twisted.internet import Iocpreactor as Iocpreactor

Try

Iocpreactor.install ()

Except Exception, E:

Print "IOCP install failed:%s"% str (e)

From Twisted.web import server, Resource

From twisted.internet import Reactor,ssl

From OpenSSL import SSL

Cafile= "C:/CA/CA.CRT"

Class Mainresource (Resource. Resource):

IsLeaf = True

# for handling Get type requests

def render_get (self, request):

# NAME parameter

name = ' World '

If Request.args.has_key (' name '):

name = request.args[' name '][0]

# Set Response encoding

Request.responseHeaders.addRawHeader ("Content-type", "text/html; Charset=utf-8 ")

# The contents of the response are returned directly

Return "

Hello, "+ Name +" "

if __name__ = = ' __main__ ':

Sslcontext = SSL. Defaultopensslcontextfactory (

' C:/ca/server.key ', # private key

' C:/CA/SERVER.CRT ' # certificate

)

CTX = Sslcontext.getcontext ()

Ctx.set_verify (

Ssl. Verify_peer | Ssl. Verify_fail_if_no_peer_cert,

Verifycallback

)

Ctx.load_verify_locations (Cafile)

site = Server. Site (Mainresource ())

Reactor.listenssl (8080, site, sslcontext)

Print "Listening port: 8080"

Reactor.run ()

Client:

‘‘‘

Created on 2018-3-2

@author: QH

‘‘‘

Import Urllib2

Import SSL

Key_file= "C:/ca/client.key"

Cert_file= "C:/CA/CLIENT.CRT"

Ca_file= "C:/CA/CA.CRT"

context = SSL. Sslcontext (SSL. PROTOCOL_SSLV23)

Context.check_hostname = False

Context.load_cert_chain (Certfile=cert_file,keyfile=key_file)

Context.load_verify_locations (Ca_file)

Context.verify_mode=ssl. Cert_required

if __name__ = = ' __main__ ':

req = Urllib2. Request (Myurl)

Try

Response = Urllib2.urlopen (Req,context=context)

#response = Urllib2.urlopen (req)

Print "HTTP return code:%d"% Response.getcode ()

strresult= Response.read ()

Print Strresult

Except Exception, ex:

Print "Found Error in auth phase:%s"% str (ex)

V. Other

1, Context.load_cert_chain (certfile=cert_file,keyfile=key_file)

This function can provide only one CertFile file, which requires the cert_file to include the private key. In fact, both the certificate file and the private key file are simple text files, you can copy the contents of the private key file to the certificate file, so that you can only provide a file in this function.

2. How to read the client information inside the request object?

All client information can be read in the request object, including the client's certificate information, see below, all you need is the basic

def render_get (self, request):

Print Request.content.read ()

Print Request.getallheaders ()

Print Request.getclientip ()

Print Request.gethost ()

Print Request.transport.getPeer ()

Cl=request.transport.getpeercertificate ()

Certificate = SSL. Certificate (CL)

Print Certificate.getissuer ()

Print Certificate.getsubject ()

3. Use the Post method to send JSON data

def urllib2_open (Myurl):

headers = {' Content-type ': ' Application/json '}

data={' test ': ' Hello '}

req = Urllib2. Request (Myurl,headers=headers, Data=json.dumps (data))

Try

Response = Urllib2.urlopen (Req,context=context)

Print "HTTP return code:%d"% Response.getcode ()

strresult= Response.read ()

Print Strresult

Except Exception, ex:

Print "Found Error in auth phase:%s"% str (ex)

4, there is a place I am not particularly aware of the study: By default, Python is where to find the authoritative CA certificate, whether a folder is still through the registry, or directly through the operating system provided by the API.

Writing so much finally finished the writing. Before I just wanted to use Python to access an HTTPS site. But it was not smooth, no matter how all appeared sslv3_alert_handshake_failure, at that time on the Internet to find a lot of similar solutions, what to enable SSLV3 Ah, encrypt the way to all Ah, Third-party components, such as requests and pycurl, are also used. But all attempts were not successful. It took me several days. Finally found that the original site needs two-way authentication. From here feel oneself to SSL understanding too little, go too many detours. So we have a good study of SSL, which has the above content. The content is still superficial and is limited to application, but it is enough for me. Share to everybody, let everybody less walk I pass the detour.

Using SSL (HTTPS) in Python