HD Wallet Learning Summary

Wallets are used on public chains such as Bitcoin/Ethereum. Wallets are primarily used to manage the user's private key, and the user's digital currency on the chain, which is the signature of the transaction with the private key. The private key can be used to generate a signature for a particular message, which can be validated using the public key without disclosing the private key.

Because the private key is extremely important, once a leak means that the ownership of the digital asset is in the hands of others. In theory, the private key can be arbitrary string of random numbers, not only difficult to remember and no rules to follow, it is necessary to use a number of cryptographic methods to manage the key pair (a key pair including a private key and the corresponding public key), both convenient management and adequate security.

1. Non-deterministic wallets

If you are simply randomly generating a number string as the private key, you can use the cryptographic security pseudo-random number generator (csprng,cryptographically secure pseudorandom numbers generator, Cryptography Security Pseudo-random number generator). These private keys are completely independent and the corresponding public key is irrelevant, and the purse that manages such a key pair is called a non-deterministic wallet (nondeterministic wallet). Early Bitcoin address non-deterministic wallet. The biggest trouble with a non-deterministic wallet is that when the key pair is imported and exported, all key pairs in the wallet must be manipulated individually.

2. Deterministic Wallets

In order to facilitate the application, this paper presents a method of generating key pair for non-deterministic wallets: The key pair is deduced from a primitive seed master key. The most common derivation is the tree-level derivation (hierarchical deterministic) short for HD. The wallet key pair generated by this method is also known as a deterministic wallet (deterministic wallet).

By a common seed can be deduced out of n multi-private key , the seed derivation of the private key using an irreversible hashing algorithm. When you need to back up your wallet's private key, just back up the seed (most of the time, for the sake of transcription, the seed is generated by 12 mnemonic words), the wallet simply imports the mnemonic to import all the private keys. The HD wallet can generate a large number of public keys without needing to know the private key, and this feature is ideal for services that are only responsible for collections.

We introduce the creation of HD wallets separately:

熵(128位)→助记词(12个)→种子(512位)→私钥→公钥→地址。

2.1 Mnemonic words and entropy

As the name implies, mnemonic is to facilitate the recording of a long sequence of irregular numbers and mapping to facilitate transcription and memory of mnemonic words. Since there are 2048 of mnemonic words in the mnemonic thesaurus, the index of the 11-bit length (2^11=2048) can be used to locate all mnemonic words. Remembering these n mnemonic words and their order, you can combine their index values intoa number string of (n 11) bit length *.

Conversely, the (n*11) bit length of the string cut into n copies, each length of 11 bits, respectively, as a mnemonic index, according to the index from the mnemonic thesaurus to obtain a mnemonic word and permanent record.

The following is the process of generating a string of numbers for this n*11 and converting them into mnemonic words:

Image.png

1. Generates a random serial number string of length 128/160/192/224/256 bits (bits), called entropy ;
2. The entropy is hashed, and the first 4/5/6/7/8 bit of the data string after hash is taken as the checksum (length = entropy length/32);
3. The entropy and checksum are combined, that is, the total length is 132/165/198/231/264 bit;
4. Every 11 bits of the above results are cut to get 12/15/18/21/24 index of mnemonic words.
5. A complete set of mnemonic words is obtained according to the words of mnemonic Word index matching mnemonic thesaurus;

The above procedure is represented by an example of a length of 128 bits of entropy:

[Image upload failed ...] (image-4d4e04-1532831639402)]

2.2 Seeds

A mnemonic can be used to derive an entropy of 128 to 256 bits in length. The PBKDF2 function allows you to derive a longer (512-bit) seed from entropy.

Note: PBKDF2 (password-based key derivation Function 2) is one of the commonly used key stretching algorithms. In cryptography, key stretching technology is used to enhance the security of weak keys, increasing the attack difficulty of brute force (Brute-force attack) attempts to break each possible key. The basic principle is to use a random function (such as an HMAC function), the plaintext and salt values as input parameters, and then repeat the operation to eventually generate the key. As shown in the following:

Image.png

The PBKDF2 function is implemented as follows:

DK = PBKDF2(PRF, Password, Salt, c, dkLen)实现：DK = T1 || T2 || ... || Tdklen/hlenTi = F(Password, Salt, c, i) = U1 ^ U2 ^ ... ^ UcU1 = PRF(Password, Salt || INT_32_BE(i))U2 = PRF(Password, U1)...Uc = PRF(Password, Uc-1)

The function has several input parameters:

• PRF: is a pseudo-random function, such as the hmac-sha512 function, which outputs the result of a length of hlen.
• Password: The original text used to produce a secret key, a string of mnemonic words .
• "Mnemonic" + user-entered password passphrase as a salt, the password is optional.
• C: Number of repetitions, e.g. 2048
• Dklen: Output secret key length

2.3 Primary private key and main chain code

The master key (master key) and the primary chain Code (master chain code) can be generated from the root seed. The calculation method is simple, the root seed is calculated once by the hmac-sha512 function, the left 256 bits is the primary private key, the right 256 bits is the main chain code, the master private key is pushed to the main public key by the Elliptic curve algorithm, the main public key and the master key pair. The main chain code is used as the entropy to derive the subordinate key.

Image.png

The algorithm for generating the public key of the private key is described below in the Elliptic Curve algorithm section. The main public key length here is 264 bits because the format is the compressed format public key (prefix (8 bit) +x axis direction coordinates (256 bits)), as described in the following subsections.

2.4 Sub-key

The child key can be deduced from the parent key (the Children keys), and the CKD function makes one-way scatter column hash for the following three inputs.

• Parent key (parent private key or parent public key), if the parent private key is entered, is also converted to the parent public key
• Chain code as Entropy
• Index number

The calculation method is described in Ethereum HD wallet below.

Image.png

The number of index numbers is 2^32, and each parent key can derive a half-size sub-key (the index number from 0x00 to 0X7FFFFFFF (0~2^31-1) generates a normal key; the index number from 0x80000000 to 0xFFFFFFFF generates an enhanced key ）。

The hmac-sha512 non-reversible encryption algorithm is deduced, and the sub-key cannot deduce the parent key upward, nor can it deduce the key at the same level. The left 256 bits of the generated 512-bit data are the child private keys , and the right 256 bits are the sub-chain codes .

2.5 extension Key

CKD deduces three elements of the subkey, where the parent key and the chain code are collectively known as the extended key (Extended keys).

1. The extended key containing the private key is used to derive the child private key, and the corresponding public key and the Bitcoin address can be deduced from the child private key;
2. The extended key containing the public key is used to derive the child public key ;

Extension keys use base58check encoding with specific prefix encodings:

• The prefix containing the private key is XPRV
• The extension key prefix that contains the public key is xpub

The extended key encoding is 512 or 513 bits longer than the public private key of the Bitcoin.

2.6 Child Public Key

One of the most useful features of the HD wallet is to derive the sub-public key from the public key under the premise of hiding the private key, greatly enhancing security. It is very useful in scenarios where it is only necessary to generate addresses to accept bitcoins without consumption, and a public key extension key can generate an infinite public key and a bitcoin address. The child public key derivation process is as follows:

Image.png

This method can be used to create a very secret public-key-only public key. Can be used to receive bitcoins but not to spend any bitcoin in this address. At the same time, on another, more insured server, the extended private key can derive all the corresponding tradable and private keys for the money.

Note: The child public key and the child private key deduced separately are a pair of secret keys. In the algorithm implementation, the derivation of the sub-public key needs to calculate the child private key, and then calculate the child public key.

2.7 Enhanced Extension key

The key needs to be kept in order to avoid leakage, the leakage of the private key means that the corresponding address of the currency can be transferred away, leaking public key means that the privacy of HD wallet is compromised. Enhanced key derivation (hardened child key derivation) addresses two issues:

1. Although leaking the public key does not result in the loss of money, an extended key leak with a public key can cause the extended public key deduced from this root node to leak all, to some extent, to undermine privacy.
2. If a leak extends the public key (which contains a chain code) and the child private key, it can be used to derive all the other child private keys, because the sub-chain code can be obtained by traversing the index. Worse, the child private key and the parent chain code can be used to infer the parent private key.

In this, the BIP32 protocol changes the CKD function to HKD (hardened key derivation formula) to generate an enhanced key deduction function. "Break" the relationship between the parent public key and the child chain code. The HKD is almost the same as the general derived child private key, but the parent private key is used as input instead of the parent public key.

The CKD function is derived from the serial number of the extended key (0x00 to 0X7FFFFFFF), the parent chain code and the parent public key , and the child private key is deduced from the parent private key, while the HKD is passed through the parent private Key, the parent chain code, and the enhanced extension Key's serial number ( 0x80000000 to 0xFFFFFFFF) derive the enhanced child private key and the enhanced sub-chain code.

Image.png

3. Elliptic Curve algorithm

The Elliptic curve algorithm can be used to calculate the public key from the private key, which is an irreversible process, the formula is as follows:k = k * G . where k is the private key, and G is the constant point called the generating points , all bitcoin users generate the same point , and K is the resulting public key. Its inverse operation, known as "finding the discrete logarithm"-known as the public key K to find out the private key k---is very difficult, as is to experiment with all possible K-values, namely brute force search.

A special elliptic curve as defined by the secp256k1 standard used is as follows:

Image.png

The corresponding formula is

y^2 mod p = (x^3 + 7) mod p其中 p = 2^256 – 2^32 – 2^9– 2^8 – 2^7 – 2^6 – 2^4 – 1

X, y is defined in the real range, and if x, y all take integers, then only some discrete coordinate values conform to the SECP256K1 elliptic curve, for example, when p=17, the discrete coordinates of x, Y are as follows:

Image.png

Defines the addition of elliptic curve coordinates: two points P1 (x1,y1) and P2 (X2,y2) on a given elliptic curve must have a 3rd P3 (x3,y3) = P1 + P2 on the elliptic curve. In the geometry, the 3rd P3 can be determined by drawing a line between P1 and P2. This line intersects exactly one point on the elliptic curve. This point is recorded as P3 ' = (x, y). Then, do the mapping on the x-axis to get p3= (x,-y). the multiplication of elliptic curve coordinates is well understood and decomposed into multiple additions.

Image.png

4. Public key

A public key is a point on an elliptic curve that consists of a pair of coordinates (x, y).

The public key is usually represented as a prefix of 04 followed by two 256-bit digits. One of the 256-bit numbers is the x-coordinate of the public key, and the other 256-bit number is the y-coordinate. For example:

K = 04F028892BAD7ED57D2FB57BF33081D5CFCF6F9ED3D3D7F159C2E2FFF579DC341A07CF33DA18BD734C600B96A72BBC4749D5141C90EC8AC328AE52DDFE2E505BDB

Why is there a prefix 04 in front of the coordinate address? Because prefix 04 is used to represent a non-compressed public key, that is, it has the full x, y coordinates, and the compressed format public key begins with 02 or 03.

4.1 compressed format public key

The compressed public key is introduced to reduce the number of bytes in bitcoin transactions, thereby saving the disk space of the nodes running the blockchain database. The point on the elliptic curve is actually a solution of the mathematical equation. Therefore, if we know the x-coordinate of the public key, we can get the y-coordinate by solving the equation y2 mod p = (x3 + 7) mod p. This scheme allows us to store only the x-coordinate of the public key, omitting the y-coordinate, thus reducing the size and storage space of the public key by 256 bits, which greatly saves a lot of data transmission and storage.

Compressed format Public key Why is there a 02 or 32 prefix? Because the solution of y comes from a square root, which is positive or negative. The y-coordinate may be an odd or even number, corresponding to positive or negative respectively. So when generating the compressed public key, if Y is even, 02 is used as the prefix, and 03 is used if Y is an odd number. If the above k is represented in a compressed format:

K = 03F028892BAD7ED57D2FB57BF33081D5CFCF6F9ED3D3D7F159C2E2FFF579DC341A

However, there is a problem: a private key can generate two different formats of the public key--the compressed format and the non-compressed format , generating two different bitcoin addresses. The default format for the newer Bitcoin client is now in the compressed format public key.

When you export a private key from your wallet, there are 2 kinds of wif formats (Wallet Import format):

• The new Bitcoin client can only be exported as a private key in the BASE58 encoded wif compression format prefixed with K or L.
• The older wallet that does not implement the compressed public key is exported as a 5-prefix BASE58 encoded private key in wif format .
• For the 16 binary source of the private key, the above 2 export format differences are the WIF compressed private key is appended with the suffix 01.

Examples are as follows:

私钥hex：1E99423A4ED27608A15A2616A2B0E9E52CED330AC530EDCC32C8FFC6A526AEDD对应WIF格式：5J3mBbAH58CpQ3Y5RNJpUKPE62SQ5tfcvU2JpbnkeyhfsYB1Jcn私钥Hex-compressed：1E99423A4ED27608A15A2616A2B0E9E52CED330AC530EDCC32C8FFC6A526AEDD01对应WIF-compressed：KxFC1jmwwCoACiCAWZ3eXa96mBM6tb3TYzGmf6YwgdGWZgawvrtJ

The BASE58 code is described in the following subsections.

5. Account Address

5.1 Bitcoin address

The Bitcoin address is a string of numbers and letters that begins with the number "1". For example:

1J7mdg5rbQyUHENYdx39WVWK7fsLpEoXZy

The algorithm for generating addresses from public keys is as follows:

ADDR = RIPEMD160(SHA256(PUBKEY))  //双hash

In order to improve readability, avoid ambiguity and effectively prevent errors in address transcription and input. The Bitcoin address also has to be "base58check" encoded.

ACCOUNT_ADDR = Base58Check(ADDR)

5.2 Encoding

BASE64 uses 26 lowercase letters, 26 capitals, 10 numbers, and 2 symbols (such as "+" and "/"), which are typically used to encode attachments in messages. BASE58 is a subset of the BASE64 encoded format, and also uses uppercase and lowercase letters and 10 numbers, but does not contain 0 (number 0), O (capital letter O), L (lowercase L), I (Uppercase letter i), and two characters of "+" and "/", which are not included in Base64. The Alphabet of Base58 is:

123456789ABCDEFGHJKLMNPQRSTUVWXYZabcdefghijkmnopqrstuvwxyz

Base58check is a commonly used BASE58 encoding format in bitcoin, which adds error checksums to check for errors in transcription of data. The check code is 4 bytes long and is added after the data that needs to be encoded. The checksum is obtained from the hash value of the data that needs to be encoded, so it can be used to detect and avoid errors generated in the input.

Before encoding, we first add a prefix called "version byte" to the data, which is used to identify the type of data that needs to be encoded. The prefixes are as follows:

Image.png

Attention:

• The same key is encoded in a different format, although the results may look different, but the text encoded in the key has not changed.
• The number string that adds a prefix is not limited to a Bitcoin address, such as a private key/public key.

The verification code is calculated as follows:

checksum = SHA256(SHA256(prefix+data))

Take the first 4 bytes of the result as a checksum. This gets 3 parts: prefix, data, and checksum, and then use the Base58 alphabet code described earlier:

Image.png

At this point, complete the Bitcoin address generation process.

6. Multi-currency and multi-account

Allows the same seed to support multiple currencies, multiple accounts , and more. The layers are defined as follows:

m / purpose' / coin_type' / account' / change / address_index

Where the purporse' fixation is 44' , the representative uses BIP44. and coin_type' used to denote different currencies, such as Bitcoin 0' , Ethereum is 60' .

7. Ethereum Certainty Wallet

There is a Golang implementation of the Ethereum Certainty Wallet project: Https://github.com/shiqinfeng1/go-ethereum-hdwallet.

The project provides a wallet-related interface with:

NewMnemonic(bits): 通过熵生成BIP-39助记词,bits一般等于128位NewSeedFromMnemonic(mnemonic):使用助记词生成BIP-39种子,不带密码NewSeed(): 不使用助记词,通过rand包直接生成512位长度的BIP-39种子NewSeedFromMnemonic(mnemonic): 助记词转换为种子NewFromSeed(seed): 通过种子创建一个新的钱包NewFromMnemonic(mnemonic): 通过助记词创建一个新的钱包

Definition of wallet structure:

type Wallet struct {    mnemonic  string   //助记词(可选)    masterKey *hdkeychain.ExtendedKey   //主秘钥,包含主私钥和主链码    seed      []byte  //种子    url       accounts.URL //HD钱包的生成路径    paths     map[common.Address]accounts.DerivationPath //每个账户的派生路径    accounts  []accounts.Account  //保存所有当前钱包的账户    stateLock sync.RWMutex   //钱包操作锁}

7.1 Generating Wallets

Seeds are generated first, can be generated directly, or generated based on entropy and mnemonic words.

According to the seed through Hdkeychain. Newmaster (seed) generates a primary private key and a primary chain code and hashes the seed once:

HMAC-SHA512(Key = "Bitcoin seed", Data = Seed)

The format of the account derivation path, Derivationpath, is this:

m / purpose' / coin_type' / account' / change / address_index

which

• Purpose=44 or 0x8000002c (for digital crypto currency)
• Coin_type=60 or 0x8000003c* (said to be Ethereum)

So the root path of Ethereum ism/44'/60'/0'/0

7.2 Wallet Interface

The Wallet wallet provides the following interfaces:

Accounts(): 返回当前钱包所有账户列表Contains(account): 检查指定账户是否在本钱包里Unpin(account): 取消固定指定的账户: 从钱包中删除该账户Derive(path, pin): 根据path派生一个账户地址: path->派生私钥->公钥->地址PrivateKey(account): 获取账户私钥PublicKey(account): 获取账户公钥Address(account): 获取账户地址Path(account): 账户路径SignHash(account, hash): 签名hashSignTx(account, tx, chainID): 签名交易

In the wallet interface, the most important is derived Derive.

The algorithm that derives the child private key and the child public key uses the same interface: ExtendedKey.Child(i) , which derives the child key of the specified index. If the input index is greater than 0x80000000, the derivation of the enhanced key, or the derivation of the normal key.

• Only the private key can derive the enhanced child private key; The public key can not derive the enhanced child public key; The derivation of a child public key does not need to know the parent private key, which can be derived directly from the parent public key;

The sub-key is also obtained through the following hmac-sha512 function:

HMAC-SHA512(Key = chainCode, Data = data)

• Chaincode is the link code for the parent key

• The data format of the hardened child private key is: 0x00 || ser256(parentKey) || ser32(i) ;

• The data format of the ordinary child private key/sub-public key is: serP(parentPubKey) || ser32(i) ;

  Note: The child private key is also used as a parameter for the parent public key.

• The result is a 512-bit length string of numbers I , where:

  • Ilthe child private key is computed by the left 256 bits: childKey = parse256(Il) + parentKey ,
  • The child public key is computed by the left 256-bit Il : childKey = serP(point(parse256(Il)) + parentKey) . In fact, the IL is used as the private key, and the public key coordinates are computed.
  • Right 256 bits Ir as the sub-chain code.

Report:

Image.png