Golang implementation of TLS1.3 protocol--clienthello

Objective

At the time of this writing, TLS1.3 RFC has been released to version 28. Previously wrote a bit of cryptography and TLS related articles, in order to further understand the TLS1.3 protocol, this time will try to use the go language to implement it. There are already some sites on the network that support Tls1.3,chrome browsers through settings that support TLS1.3 (draft23), which can be used to verify whether the protocols we implement are standard.

The complete implementation TLS1.3 a lot of work, with a lot of concepts and details (feel like digging a hole again). This article begins with ClientHello, and may be considered later: Authentication, cryptographic computations, 0-rtt.

5G Future

Every time the infrastructure upgrade is a prelude to the change of the Times. In the mobile internet 2g/3g era, many innovations have been confined to the WiFi, the mobile internet into the 4G era, the outbreak of a variety of live, short video and other innovations. Today's IoT and mobile Internet are slightly similar in the first half, and when 5G matures, it is believed that a series of innovations will erupt.

The secure transmission of information after network interconnection is also a problem that cannot be neglected. TLS1.3 has fixed many security traps compared to previous versions, reducing the number of handshakes and improving efficiency.

Privatization

Many of the designs in TLS1.3 are designed to be backwards compatible with TLS1.2 or TLS1.1 (after all, an open network protocol), and if you want to create a privatized security layer, you can eliminate the compatibility design by following the key points in the standard, optimizing the packet structure to reduce the amount of data transferred. For example, by referring to the RLP encoding of the Ethereum, the size of the packet can be further reduced by combining length and data.

Build a secure and efficient "secure Session layer" with TLS1.3,
1, down can be added to the connection Management module, heartbeat control module, such as: the realization of multi-level TCP channel rapid reconnection and so on, to solve the weak network status of various problems.
2, up can serve a variety of different network types such as peer-to or classic C/S,
3, the higher layer can serve some middleware such as: RPC, MQTT and other support business.

Connect.001.jpeg

Below we will write the code as much as possible with reference to the structure defined by TLS1.3, while using the Wireshark capture package to view the package situation.

Starting from ClientHello

The TLS1.3 handshake process starts with the client sending ClientHello, which carries the key to negotiate the necessary data. The server-side replies to Serverhello after receiving the message. We will send to a site that has the TLS1.3 protocol enabled, the ClientHello message that we implement to see if we can receive Serverhello reply.

Look at the results first.

Image.png

Wireshark is the result of the capture, ALI-BF is my native computer, server is a TLS1.3-enabled network server.

After you can see three TCP handshakes, we emit a clienthello message with a length of 348 bytes, and after an ACK, the Server Hello message is successfully received.

Encoding implementation

TLS1.3 contains a series of sub-protocols such as Record Protocol, handshake Protocol, Alert Protocol, ApplicationData Protocol, etc.
The three-person relationship
20180504150105.png

Sending a clienthello requires at least the following modules to be implemented
20180504151618.png

Record Layer

ClientHello is transmitted in plaintext, so it is encapsulated in Tlsplaintext

ContentType enum {...};
Type ContentType byte

Const (
Invalid ContentType = 0
Changecipherspec ContentType = 20
Alert ContentType = 21
Handshake ContentType = 22
ApplicationData ContentType = 23
)

Tlsplaintext plaintext on record layer
Type Tlsplaintext struct {
ContentType ContentType
Legacyrecordversion ProtocolVersion//static
Length UInt16
Fragment syntax. Vector
}

Type Tlsciphertext Tlsplaintext
The value of Legacyrecordversion is 0x0303 in order to be compatible with TLS1.2 version.

Define an interface for serialization, and all subsequent structs will implement the interface

Type Encoder Interface {
Encode coding object into the Writer,
Encode (w io). Writer) Error

//ByteCount return the byte length of all ObjectByteCount() int

}

Type Vector Encoder
Serialization of Tlsplaintext

Func Generatetlsplaintext (ContentType contentType, fragment syntax. Vector) Tlsplaintext {
Return tlsplaintext{
Contenttype:contenttype,
Legacyrecordversion:tls1_2,
Length:uint16 (fragment. ByteCount ()),
Fragment:fragment,
}
}

Func (t *tlsplaintext) Encode (w io). Writer) Error {

if uint16(t.length) > 2<<14 {    return errors.New("overflow fragment")}err := syntax.Encode(w,    syntax.WriteTo(t.contentType),    syntax.WriteTo(t.legacyRecordVersion),    syntax.WriteTo(t.length))if err != nil {    return err}err = t.fragment.Encode(w)return err

}

Func (t *tlsplaintext) ByteCount () int {
return T.fragment.bytecount () + 5
}
The main purpose of this paper is to learn TLS1.3 protocol in a comprehensible sense, so it is not very concerned with performance and efficiency problems and some abnormal situations.

Handshake

TLS1.3 supports three ways of key negotiation: psk-only, (EC) DHE, and PSK with (EC) DHE, this article focuses on ECDHE.
The handshake corresponds to the Tlsplaintext fragment because it implements the vector interface

Type handshake struct {
Msgtype Handshaketype/ handshake type /
Length Uint24
Handshakedata syntax. Vector
}

Func Generatehandshake (msgtype handshaketype, data syntax. Vector) Handshake {
L: = data. ByteCount ()
Return handshake{
Msgtype:msgtype,
Length:uint24{byte (l >>), byte (L >> 8), Byte (L)},
Handshakedata:data,
}
}
ClientHello

ClientHello corresponds to the handshakedata of handshake.

Type Extensionsvector struct {
Length UInt16
extensions []extension. Extension
}

Type ClientHello struct {
Legacyversion TLS. ProtocolVersion
Random [32]byte
Legacysessionid Legacysessionid
Ciphersuites Ciphersuitevector
Legacycompressionmethods Legacycompressionmethods
Extensions Extensionsvector
}
Legacyversion, legacysessionidlegacycompressionmethods are defined to be compatible with older versions, and therefore require random, ciphersuites, and extensions.

Ciphersuite indicates that the cryptographic suites that the client can support, such as tls_aes_128_gcm_sha256, tls_aes_256_gcm_sha384, and so on, only support the Aead cryptographic algorithm suite.

Func Generateclienthello (ciphersuites []ciphersuite, exts ... extension. Extension) ClientHello {
var R [32]byte
Rand. Read (r[:])
Extbytes: = 0
For _, Ext: = Range Exts {
Extbytes + = ext. ByteCount ()
}
Return clienthello{
Legacyversion:tls. Tls1_2,
Random:r,
Legacysessionid:legacysessionid{0, nil},
Ciphersuites:newciphersuite (Ciphersuites ...),
Legacycompressionmethods:generatelegacycompressionmethods ([]byte{0}),
Extensions:generateextensions (exts ...),
}
}

Func generateextensions (exts ... extension. Extension) Extensionsvector {
L: = 0
For _, Ext: = Range Exts {
L + = ext. ByteCount ()
}
Return extensionsvector{
Length:uint16 (L),
Extensions:exts,
}
}

Func Newclienthellohandshake (ciphersuites []ciphersuite, exts ... extension. Extension) Handshake {
CL: = Generateclienthello (Ciphersuites, exts ...)
Return Generatehandshake (ClientHello, &CL)
}
Various extension

ClientHello mainly through the extension transfer key to negotiate the necessary material, the first ClientHello need to contain at least the following 5 extension:
1. ServerName: The requested hostname

Typically, a single server hosts multiple sites, that is, multiple Web servers of the same IP. Because the TLS layer has not yet completed the handshake, there is no HTTP request at this time (host head) and the specific site cannot be known. Subsequent handshakes will make it difficult for the server side to determine which certificate to send.

Although the use of common name and SubjectAltName can support multiple domain names, but it is not possible to know all the future domain names, once the changes have to re-apply for the certificate. Therefore, adding the servername extension in ClientHello to indicate the host to be accessed, view the details RFC6066.

This has a drawback, ClientHello is the plaintext transmission, the intermediary can clearly detect the destination of the traffic. Like WhatApp and Signal, a technique called "domain Front" is used to circumvent the problem.

2, Supportedversions: Can support the TLS version such as: TLS1.1, TLS1.2, TLS1.3, etc.

Used to negotiate the final adoption of the TLS version, in the list of supported ClientHello, put the highest priority support in the first place.

3. SIGNATUREALGORITHMS: Supported signature algorithms

such as: ecdsa_secp256r1_sha256, ecdsa_secp384r1_sha384, etc.

4. Supportedgroups: For key exchange

This paper focuses on elliptic curves such as: SECP256R1, SECP384R1, SECP521R1, etc.

5. Keyshare: The public key exchanged during key negotiation

Each supportedgroup needs to have a corresponding keyshare.

Extension Golang Implementation

Extension Basic is to have Request/response mode communication, the client sends request, the server side through Response way to reply to the selected.

Type Extension struct {
ExtensionType ExtensionType
Length UInt16
ExtensionData syntax. Vector
}
The Golang implementations of Supportedversions and Keyshare are listed below, and other extension implementations are similar.

ClientHello's supportedversions is relatively simple, as long as it contains a protocolversions array.

Type supportedversions struct {
Length uint8//Byte count of array protocolversions
protocolversions []tls. ProtocolVersion
}

Func generatesupportedversions (protocolversions. TLS). ProtocolVersion) Supportedversions {
Return supportedversions{
Length:uint8 (len (protocolversions) * 2),
Protocolversions:protocolversions,
}
}

Newsupportedversionsextension create a supported versions extension
Func newsupportedversionsextension (protocolversions. TLS). ProtocolVersion) Extension {
SV: = Generatesupportedversions (Protocolversions ...)
Return Generateextension (supportedversions, &SV)
}
In this paper, the P-256, P-384, and P-521 curves of ECC are implemented to show that if the corresponding Keyshare is generated

Type keyshareentry struct {
Group Namedgroup
Length UInt16
Keyexchange []byte
}

Func generatekeyshareentry (group Namedgroup) (Keyshareentry, []byte) {
var curve elliptic. Curve
Switch Group {
Case SECP256R1:
Curve = elliptic. P256 ()
Break
Case SECP384R1:
Curve = elliptic. P384 ()
Break
Case SECP521R1:
Curve = elliptic. P521 ()
Break
}

priv, x, y, err := elliptic.GenerateKey(curve, rand.Reader)if err != nil {    return KeyShareEntry{}, nil}nu := generateUncompressedPointRepresentation(x.Bytes(), y.Bytes())buffer := new(bytes.Buffer)nu.Encode(buffer)ks := KeyShareEntry{    group:       group,    length:      uint16(len(nu.X)+len(nu.Y)) + 1,    keyExchange: buffer.Bytes(),}return ks, priv

}

Type Keyshareclienthello struct {
Length UInt16
Clientshares []keyshareentry
}

Func Generatekeyshareclienthello (Enters ... Keyshareentry) Keyshareclienthello {
var l uint16
For _, K: = Range enters {
L + = K.length + 4
}
Return keyshareclienthello{
Length:l,
Clientshares:enters,
}
}

Func newkeyshareclientextension (Groups ... Namedgroup) (Extension, [][]byte) {

keyShareList := make([]KeyShareEntry, len(groups))privateList := make([][]byte, len(groups))for i, g := range groups {    ks, priv := generateKeyShareEntry(g)    keyShareList[i] = ks    privateList[i] = priv}kscl := generateKeyShareClientHello(keyShareList...)return generateExtension(keyShare, &kscl), privateList

}

Type uncompressedpointrepresentation struct {
Legacyform uint8
X []byte
Y []byte
}

Func generateuncompressedpointrepresentation (x, y []byte) uncompressedpointrepresentation {
Return uncompressedpointrepresentation{
Legacyform:4,
X:x,
Y:y,
}
}

Send ClientHello

Combine the various types above, build the ClientHello, encode and send to the remote server (the real site), TLS1.3 using the big endian byte sequence.

Func Firstclienthello (conn net. Conn, host string) {
Supportedversion: = extension. Newsupportedversionsextension (TLS. TLS1 3)
Supportedgroup: = extension. Newsupportedgroupextension (extension. SECP256R1, extension. SECP384R1)
Keyshare, : = extension. Newkeyshareclientextension (extension. SECP256R1, extension. SECP384R1)
Signaturescheme: = extension. Newsignatureschemeextension (extension. ecdsa_secp256r1_sha256, extension. ecdsa_secp384r1_sha384)
ServerName: = extension. Newservernameextension (host)

clientHelloHandshake := handshake.NewClientHelloHandshake([]handshake.CipherSuite{    handshake.TLS_AES_128_GCM_SHA256,    handshake.TLS_AES_128_CCM_SHA256,    handshake.TLS_AES_256_GCM_SHA384}, serverName, supportedVersion, signatureScheme, supportedGroup, keyShare)clientHelloRecord := tls.NewHandshakeRecord(&clientHelloHandshake)clientHelloRecord.Encode(outputBuffer)_, err := outputBuffer.WriteTo(conn)if err != nil {    fmt.Println(err)    return}

}
View Wireshark's packet capture data
20180507115326.png
Contains the data we have built and several Extenison

Server-side reply:
20180507115434.png
You can see that the Server selected the aes_256_gcm_sha384 cipher suite, Keyshare selected the p-256 curve. Also found that the server side has started to transfer partially encrypted data "ApplicationData".

Read the data structure of TLS1.3

The TLS1.3 RFC in English is very difficult to read. In order to better understand the protocol (and the series of RFCs it references) it is necessary to understand its markup language first

Presentation Language

TLS1.3 defines some presentation Language to describe the structure and serialization of the data.

1, type of alias T T '
T ' is a type alias for t
For example: UInt16 protocolversion, ProtocolVersion is the alias of UInt16, which indicates that ProtocolVersion occupies 2 bytes in the transmission. Definitions in Golang:

Type ProtocolVersion uint16
2. Fixed-length array type T t ' [n]
T ' [] is expressed as an array of T, it is important to note that N does not represent the number of T, but the number of bytes that the T ' type occupies,
such as: Opaque random[32], which means that the Random type occupies 32 bytes. Opaque represents an opaque data structure that can be understood as a byte array

3. Variable length array type T T '
T ' <> consists of two parts: Head+body,
The head value indicates how many bytes the body occupies.
The body value is the real payload, which is the array of T. The number of bytes that the head itself occupies is determined.
Such as:
Ciphersuite cipher_suites<2..2^16-2>
Represents an array of type Ciphersuite, whose head occupies 2byte and uint16 can tolerate 2^16 bytes.

This can be said in Golang

Type Ciphersuitevector struct {
Length UInt16
Ciphersuites []ciphersuite
}
4. Enum type enum {e1 (v1), E2 (v2), ..., en (VN) [[, (N]]]} Te;
E1 represents the value of an enumeration type. The last [[, (N)]] n represents the maximum value, which can infer the number of bytes occupied by the enumeration type Te
Such as:

enum {
Client_hello (1),
Server_hello (2),
New_session_ticket (4),
End_of_early_data (5),
Encrypted_extensions (8),
Certificate (11),
Certificate_request (13),
Certificate_verify (15),
Finished (20),
Key_update (24),
Message_hash (254),
(255)
} Handshaketype;
The Handshaketype type occupies a byte 2^8
This can be defined in Golang.

Handshaketype alies
Type Handshaketype byte

Const (
ClientHello Handshaketype = 1
Serverhello Handshaketype = 2
Newsessionticket Handshaketype = 4
Endofearlydata Handshaketype = 5
Encryptedextensions Handshaketype = 8
Certificate Handshaketype = 11
Certificaterequest Handshaketype = 13
Certificateverify Handshaketype = 15
Finished Handshaketype = 20
Keyupdate Handshaketype = 24
Messagehash Handshaketype = 254
)
5. Constant representation
In TLS 1.3, some of the fields in the protocol must be set to fixed values, primarily to be compatible with older versions, so you need to define a representation of the constants.

struct {
T1 f1 = 8; / T.F1 must always be 8/
T2 F2;
T
Here the value of T.F1 is fixed to 8
For example:

struct {
ProtocolVersion legacy_version = 0x0303; / TLS v1.2 /
Random random;
Opaque legacy_session_id<0..32>;
Ciphersuite cipher_suites<2..2^16-2>;
Opaque legacy_compression_methods<1..2^8-1>;
Extension extensions<8..2^16-1>;
} ClientHello;
The value of the clienthello.legacy_version is fixed to 0x0303, in order to be backwards compatible.

6. Variable definition

struct {
T1 F1;
T2 F2;
....
TN fn;
Select (E) {
Case E1:te1 [[Fe1]];
Case E2:te2 [[Fe2]];
....
Case En:ten [[Fen]];
};
} Tv;
TV.E is a variable that follows the specific case value.
For example

struct {
Select (Handshake.msg_type) {
Case Client_hello:
ProtocolVersion versions<2..254>;

          case server_hello: /* and HelloRetryRequest */               ProtocolVersion selected_version;      };

} supportedversions;
here if it is in ClientHello supportedversions is a Vector type, and in Serverhello it is a protocolversion