HTTPS principle and Golang basic implementation

This is a creation in Article, where the information may have evolved or changed.

About HTTPS

Background knowledge

Some basic knowledge of cryptography

It is broadly divided into two types, key-based encryption algorithm and non-key-based encryption algorithm. Now the algorithm is basically based on key, key on a string of random number, after the replacement of key, the algorithm can continue to use.

Key-based encryption algorithms are divided into two categories, symmetric encryption and asymmetric encryption, such as Des,aes, the two sides of the communication with key encryption, the other side with the same key to reverse the operation can be decrypted.

Asymmetric encryption is the most famous RSA, when encryption has a public key and a private key, the public key can be handed to each other, a to B to send information, a with their own private key encryption, B with a public key decryption, conversely, B to a send information, b with its own private key encryption.

Before the communication, we need to go through some handshake process, the two sides exchange the public key, this is the process of key exchange, the beginning of the HTTPS phase contains the key exchange process, the general principle is that, some places are slightly more complicated.

Digital certificates and CAS

A digital certificate is equivalent to an "id" of the server that uniquely identifies a server. Generally speaking, the digital certificate from the Trusted Authority certification authority (certification authority, certification authority) bought (very little free), the browser is generally built into a number of authoritative CA, when using HTTPS, as long as these CA-issued certificates, Browsers are certified, if you are communicating with the server, you receive a certificate without an authoritative CA certification, you will be notified of an untrusted certificate error, like logging in 123,061, but can also choose to accept.

In some of their projects, usually you issue a CA root certificate, then this root certificate issued a SERVER.CRT, and Server.key to the server,Server.key is the server's private key, SERVER.CRT contains the public key of the server There are some identity information on the server side. When communicating with the client and the server (especially when using code-written client access), to specify the CA root certificate is equivalent to the authoritative certificate built into the browser for the service-side identity detection.

Format of the certificate:

The CA certificate refers to this in the approximate process of signing the SERVER.CRT certificate (Http://www.tuicool.com/articles/aymYbmM):

A digital certificate consists of two parts:

1, C: certificate-related Information (object name + Expiration Time + Certificate Publisher + Certificate signing algorithm ...)

2, S: Digital signature of the certificate (generated by the CA certificate through the encryption algorithm)

The digital signature is obtained through the formula S = F (Digest (C)).

Digest is a digest function, which is a one-way hashing algorithm, such as MD5, sha-1, or sha256, used to convert an infinite input value into a finite-length "condensed" output value. For example, we often use the MD5 value to verify that the large downloaded file is complete. The content of a large file is an infinite input. When large files are placed on the site for download, the site will do a MD5 calculation of large files, and a 128bit value is placed on the website as a summary of the large files. After the user downloads the file, after the downloaded file again the local MD5 computation, uses the value to compare with the MD5 value on the website, if is consistent, the big file downloads intact, otherwise the download process big file content has the corruption or the source file to be tampered with. There is also a small trick often used to copy between machines or download compressed files can also use the md5sum command to test, to see if the file is complete.

F is the signature function. The CA's own private key is uniquely identifying the CA signature, so the signature function that the CA uses to generate the digital certificate must take its own private key as an input parameter. In RSA encryption system, the decryption function of the sending side is a function that takes the private key as the parameter, so it is often used as a signature function. So the CA uses the private key decryption function as F, encrypts the private key in the CA certificate, generates the final digital signature, just as the last part of the practice gives the certificate generation process, When generating SERVER.CRT, CA.CRT (information containing the root certificate) and Ca.key (the private key of the root certificate) are required to be added.

After receiving the server-side digital certificate, how to verify that the signature carried on the digital certificate is the signature of this CA? Of course, the receiving end needs to specify the corresponding CA, the receiver will use the following algorithm to verify the signature of the digital certificate:
F ' (S)? = Digest (C)

The receiver carries out two calculations and calculates the result:

1, first through the Digest (C), the receiver calculates the content of the certificate (in addition to the signature) summary, C content is clear can be seen.

2. The signature carried by the digital certificate is the result of the CA encrypting digest through the CA key, so the receiving end "decrypts" s through a decryption function f '. As it was first introduced, in an RSA system, the receiver uses the CA public key (contained in CA.CRT) to "decrypt" s, which is the inverse of the CA's "encryption" of s with the private key.

Comparing the results of the two operations above, if it is consistent, the signature does belong to the CA, the certificate is valid, or the certificate is either not the CA or is tampered with in the middle.

For self-signed (self-signed) certificates, the receiver does not have your SELF-CA digital certificate, that is, there is no CA public key, there is no way to verify the signature of the digital certificate. Therefore, if you want to write a self-signed certificate can be verified to the receiving end of the program, the first thing we have to do is to establish a own CA, with the CA to sign our server-side certificate, and then send a message to the client, the root certificate needs to be specified, Then verify the method as above.

You can use the OpenSSL x509-text-in client.crt-noout to view the specific information that a certificate file contains.

Overview of the HTTPS basic process

The HTTPS protocol is a secure protocol based on the HTTP protocol. The main features include two areas:

1 identity authentication for both sides of the communication

2 communication process encryption on both sides of the communication

The following is a detailed analysis of the HTTPS communication process to explain these two features.

Refer to these two articles specifically:

Http://www.fenesky.com/blog/2014/07/19/how-https-works.html
Http://www.ruanyifeng.com/blog/2014/02/ssl_tls.html

1, the client sends SayHello to the server side, explains the cryptographic suite supported by the client, and a random number of 1.
2, the server sends SayHello to the client side, the end sends the SERVER.CRT sends the customer, SERVER.CRT uses also has a random number 2.
3, the client side generated Premaster key This is a random number 3, followed by three random numbers together to generate Mastersecret, then generate session secret, using the specified CA for identity authentication, as described earlier, all normal, Switch to encryption mode.
4. The client side uses the public key in the SERVER.CRT to encrypt the Premastersecret, if two-way authentication, the client side will send the CLIENT.CRT in the past, the server side accepts the data, after decryption, also has three random numbers, uses the same way, three random The session secret used by the generation of communications. The structure of the specific session secret can refer to the two blogs listed earlier. After the server completes the related work, it sends a changecipherspec to the client, notifies the client that he has switched to the relevant encryption and decryption mode, and then sends an encrypted message to the client to see if it is normal.
5, the client side decryption is normal, then you can follow the previous protocol, using session secret to encrypt the communication.

Overall look, the beginning of the process of building a handshake is the process of identity authentication, after the certification is complete, is the process of encrypting communications, HTTPS two of the main use to achieve.

Related practices

Compare the process of typical certificate generation:

openssl genrsa -out ca.key 2048#这里可以使用 -subj 不用进行交互 当然还可以添加更多的信息openssl req -x509 -new -nodes -key ca.key -subj "/CN=zju.com" -days 5000 -out ca.crtopenssl genrsa -out server.key 2048#这里的/cn可以是必须添加的 是服务端的域名 或者是etc/hosts中的ip别名openssl req -new -key server.key -subj "/CN=server" -out server.csropenssl x509 -req -in server.csr -CA ca.crt -CAkey ca.key -CAcreateserial -out server.crt -days 5000

Note When generating a client-side certificate, be aware of adding one more field, Golang's server-side certification program will authenticate this field:

HTTPS client and server one-way check

This section refers to this (Http://www.tuicool.com/articles/aymYbmM
), the inside code part is more detailed.

The server uses the certificate, the client uses the ordinary way to access:

//server端代码package mainimport ("fmt""net/http""os")func handler(w http.ResponseWriter, r *http.Request) {fmt.Fprintf(w,"Hi, This is an example of https service in golang!")}func main() {http.HandleFunc("/", handler)//http.ListenAndServe(":8080", nil)_, err := os.Open("cert_server/server.crt")if err != nil {panic(err)}http.ListenAndServeTLS(":8081", "cert_server/server.crt","cert_server/server.key", nil)}

The client side sends the request directly, does not add, will report the following error:

2015/07/11 18:13:50 http: TLS handshake error from 10.183.47.203:58042: remote error: bad certificate

Use the browser direct access, then click on the Trust certificate, this time can be normal get to the message

or using Curl-k https://for row access, the equivalent of ignoring the first step of authentication work.
If you do not add-K, using the Curl-v parameter to print out detailed information, you will see the following error:

curl: (60) SSL certificate problem: Invalid certificate chain

The explanation is that the authentication did not pass, because the client side does not provide a trustworthy root certificate to the server sent over the certificate,/CN using the direct IP address, will report the following error:

Get https://10.183.47.206:8081: x509: cannot validate certificate for 10.183.47.206 because it doesn't contain any IP SANs

It is better to use a domain name when generating a certificate, or to add a corresponding mapping in/etc/hosts.

The code for the client that can send the request is as follows, note how to import the root certificate:

package mainimport (//"io"//"log""crypto/tls""crypto/x509"//"encoding/json""fmt""io/ioutil""net/http"//"strings")func main() {//x509.Certificate.pool := x509.NewCertPool()//caCertPath := "etcdcerts/ca.crt"caCertPath := "certs/cert_server/ca.crt"caCrt, err := ioutil.ReadFile(caCertPath)if err != nil {fmt.Println("ReadFile err:", err)return}pool.AppendCertsFromPEM(caCrt)//pool.AddCert(caCrt)tr := &http.Transport{TLSClientConfig:    &tls.Config{RootCAs: pool},DisableCompression: true,}client := &http.Client{Transport: tr}resp, err := client.Get("https://server:8081")if err != nil {panic(err)}body, _ := ioutil.ReadAll(resp.Body)fmt.Println(string(body))fmt.Println(resp.Status)}

With the Curl command, add the--CACRT CA.CRT certificate, which is equivalent to adding a trustworthy certificate, and the authentication operation can succeed.

For example, the generation of server-side certificate/cn is written when the client is sent to send to https://server:8081, but in the local/etc/hosts to add the corresponding mapping.

Bidirectional checksum on the client and server side:

In the previous way, the client generates the certificate, and the root certificate presses the previous one:

openssl genrsa -out client.key 2048openssl req -new -key client.key -subj "/CN=client" -out client.csropenssl x509 -req -in client.csr -CA ca.crt -CAkey ca.key -CAcreateserial -out client.crt -days 5000

The server-side code is improved to add a trusted root certificate.

// gohttps/6-dual-verify-certs/server.gopackage mainimport ("crypto/tls""crypto/x509""fmt""io/ioutil""net/http")type myhandler struct {}func (h *myhandler) ServeHTTP(w http.ResponseWriter,r *http.Request) {fmt.Fprintf(w,"Hi, This is an example of http service in golang!\n")}func main() {pool := x509.NewCertPool()caCertPath := "cert_server/ca.crt"caCrt, err := ioutil.ReadFile(caCertPath)if err != nil {fmt.Println("ReadFile err:", err)return}pool.AppendCertsFromPEM(caCrt)s := &http.Server{Addr:    ":8081",Handler: &myhandler{},TLSConfig: &tls.Config{ClientCAs:  pool,ClientAuth: tls.RequireAndVerifyClientCert,},}err = s.ListenAndServeTLS("cert_server/server.crt", "cert_server/server.key")if err != nil {fmt.Println("ListenAndServeTLS err:", err)}}

Client code improvements, when sending the specified client side of the CLIENT.CRT and Client.key

// gohttps/6-dual-verify-certs/client.gopackage mainimport ("crypto/tls""crypto/x509""fmt""io/ioutil""net/http")func main() {pool := x509.NewCertPool()caCertPath := "certs/cert_server/ca.crt"caCrt, err := ioutil.ReadFile(caCertPath)if err != nil {fmt.Println("ReadFile err:", err)return}pool.AppendCertsFromPEM(caCrt)cliCrt, err := tls.LoadX509KeyPair("certs/cert_server/client.crt", "certs/cert_server/client.key")if err != nil {fmt.Println("Loadx509keypair err:", err)return}tr := &http.Transport{TLSClientConfig: &tls.Config{RootCAs:      pool,Certificates: []tls.Certificate{cliCrt},},}client := &http.Client{Transport: tr}resp, err := client.Get("https://server:8081")if err != nil {fmt.Println("Get error:", err)return}defer resp.Body.Close()body, err := ioutil.ReadAll(resp.Body)fmt.Println(string(body))}

But in fact, this is not possible, the server side will report such an error:

client's certificate's extended key usage doesn't permit it to be used for client authentication

Because the client's certificate generation method is a bit different, to begin with, GOALNG for the client side of the authentication to one more parameter, when the certificate is generated, to add a separate authentication information:

is to add more information about an authentication file, and then use the new certificate to implement two-way authentication, so that only those clients that hold the certified certificate can send the request to the server.

Configuration of the Etcd HTTPS

The HTTPS configuration of Docker

K8 configuration of the Apiserver HTTPS

Related references

Http://www.fenesky.com/blog/2014/07/19/how-https-works.html
Http://www.ruanyifeng.com/blog/2014/02/ssl_tls.html
Http://www.tuicool.com/articles/aymYbmM