What is hard fork, what is soft fork, what is consensus?

Chapter No. 0 Introduction

Bitcoin is a set of software, the software code to modify the upgrade will often involve two concepts, a called hard fork, a soft fork. What the hell does that mean? The most common definition in the community is the modification of consensus. But it is difficult to understand what "consensus" refers to. We still have a lot of practical changes to the code level to look at their definitions.

1th. Bitcoin transaction format and transaction history

Before we understand what a hard fork and a soft fork are, we need to know what the two things are going to change.

Bitcoin trading has a double meaning in Chinese, which is a key to the often misunderstood hard forks and soft forks.

One of the meanings of Bitcoin transactions is that we send Bitcoin in a unified data structure, which is a set of rules that all of us send Bitcoins, no matter what wallet software you use to comply with this set of rules. It is defined as a data structure in which each field represents what it means. All wallet software can understand this meaning.

Another implication of Bitcoin trading is the event that people using bitcoins send bitcoins to each other. Like I sent 0.12345BTC to you, this is a deal. All these bitcoin-sending events constitute the Bitcoin transaction history, which is the main content of more than 80 g of data that currently makes up the bitcoin full node. All of these records can be found on the Blockchain browser, as long as you type your Bitcoin address on the Blockchain browser to find all your transaction records.

Make a metaphor to illustrate. The trading format is similar to the one we went to the bank to send the remittance to fill out the money order, the remittance is clearly defined in the name of the sender, ID card, mobile phone number, account, amount, payee name, account ... and other information. The bitcoin transaction format also specifies the sender's information and the payee's information, but this information is not related to the identity of the person, bitcoin transaction provisions to fill in the specific information please see the next chapter.

2nd. Bitcoin transaction format and block format

Bitcoin trading History This concept is easy to understand, do not start to say, there is nothing to talk about. But bitcoin's trading format, which is the data structure of the transaction, needs to be interpreted with emphasis, otherwise it cannot understand what is called hard fork and soft fork.

A bitcoin transaction is a data structure that contains input and output values that are embedded in code information that transfers a sum of money from the initial point (input value) to the destination address (the output value). This data structure contains some fields, which we take a look at in detail in the 5th chapter of "Mastering Bitcoin."

All Bitcoin wallets have to send Bitcoins in the form specified in this table, otherwise they will not be recognized on the Bitcoin network, which is an invalid transaction. It's like you go to a bank transfer, you don't fill in the money order, the bank will not accept the same.

The Bitcoin network collects transactions from each node in an average of 10 minutes and then hits a packet, calls chunks, covers timestamps, and then adds them to the blockchain. What is a chunk?

A chunk is a container data structure that aggregates transaction information (that is, bitcoin transactions that are made on the network over an average of 10 minutes). It consists of a chunk header containing metadata and a long list of transactions immediately following the constituent block body. The data structure of a chunk is defined as follows.

All mining node wallets have to be packaged in the form specified in this table, otherwise the Bitcoin network will not be recognized by other complete nodes (SPV nodes are not counted), which is considered to be invalid transactions.

Chapter 3rd "official" definition of hard fork and soft fork

Hard forks and soft forks are defined on the bitcoin.org, but the definition is vague, and to prevent me from babbling (questioning the definition on bitcoin.org, it really makes me very scared.) ), I extract the original text:

A hard fork is defined as such:

A permanent divergence in the the The block chain, commonly occurs when non-upgraded nodes can ' t validate blocks created by up Graded nodes that follow newer consensus rules.

Https://bitcoin.org/en/glossary/hard-fork

The blockchain has a permanent divergence, and after the new consensus rule is released, some nodes that are not upgraded cannot verify the chunks that have been upgraded by the node, usually hard forks occur.

The definition of a soft fork is this:

A temporary fork in the block chain which commonly occurs when miners using non-upgraded nodes violate a new consensus Rul e their nodes don ' t know about.

Https://bitcoin.org/en/glossary/soft-fork

When the new consensus rule is released, nodes that do not upgrade will produce a temporary fork because they do not know the new consensus rules and produce illegal chunks.

Community on the expansion of the block is the implementation of the hard fork or soft fork dispute, I guess bitcoin.org on the two vague definitions should bear the primary responsibility. The biggest responsibility is the "consensus rules" definition in the original text is unclear, if we go to bitcoin.org to find "consensus rules" what is meant, the original is this:

The block validation rules that full nodes follow to stay in consensus with other nodes.

Https://bitcoin.org/en/glossary/consensus-rules

Block validation rules that complete nodes follow in order to maintain consensus with other nodes.

This definition of consensus logically commits the fallacy of circular authentication, which means "consensus is to maintain the rule of consensus". This is the same as what Xu Sando said: "There is meaning to live well." "Then what is a good life?" "Living well is doing something meaningful." "In fact, he simply did not answer what is good living, and what is meaningful."

In addition to these two definitions, the hard fork is defined as "permanent divergence (permanent fork)", and the soft fork is defined as "temporary fork (Temporary Fork)", which is almost a definition of the word game, so that many people in the community are afraid of hard fork. It is a terrible thing to think of a hard fork.

Truth be told, this article has been written in my mind for a long time, but I have been afraid to write because I doubt the definition of Holy Land bitcoin.org. I struggled to read the definition of the Bitcoin protocol upgrade on bitcoin.org, and I couldn't find a description of what I was trying to do. I have asked many large domestic V, also did not get a complete answer. The last few days the community of soft fork enthusiasts sent a lot of articles to promote the soft fork, called for resisting hard fork, after I read, I reasoned, in my knowledge system, I am sure they are wrong. The source of their mistakes comes from the understanding of "consensus" is a kind of unreal description, do not know what the specific refers to, as long as the "consensus", is sacred inviolable, all the logic and reasoning to the "consensus" stopped.

Since there is no abstract definition of soft forks and hard forks, let's start by analyzing what they have done.

the 4th chapter on the Bitcoin blockchain implementation of the soft Fork upgrade specific cases

The Bitcoin blockchain recently performed a soft fork upgrade several months ago, making it bip68/112/113 formally in the Bitcoin agreement, called the CSV Soft fork by the community. The upgrade was completed around June 2016. Where does this soft fork change the bitcoin code? To answer this question, we need to find the answer from the bitcoin transaction data structure.

What happened to the bitcoin transaction data structure before and after the CSV Soft fork.

The red word bold in the figure is the main place where the CSV soft fork is modified. Before the CSV soft fork, this field is the "Serial number (trade substitution function not currently used)" as shown below:

It is because this field is not used in the Bitcoin transaction data structure, or it is a fuzzy definition at the time, so it can be redefined by use. This undefined field is not carefully verified on the previous version of the Bitcoin full node, the new version of the node is produced in accordance with the defined rules, and can also be accepted by the previous version of the node validation. This is the concrete process of the soft fork.

But this is obviously not the long-term, because this field is one, you use it once, and then it is gone.

Bitcoin also had an important upgrade in 2012, called P2sh, and was called a multi-signature soft fork by the community. This is a change to the Bitcoin transaction signature script, which makes bitcoin easy to send transactions through multiple signatures. What did you change that time?

The red word bold in the figure is the main modification of the multi-signature soft fork. At the beginning of the definition of this field is called P2pkh (Pay-to-public-key-hash, paid to the public key), currently this field can have 5 kinds of scripts.

Since the multi-signature soft fork, the nodes that are not upgraded will also be validated for this new P2sh lock script as they validate the chunks generated by the upgraded nodes, so this is also a soft fork.

The above two soft forks are changes to the bitcoin transaction data structure. Soft forks can also modify Bitcoin's trading history in disguise.

On August 15, 2010, a hacker exploited a loophole in the Bitcoin code to brush a deal on the No. 74638 height block with 184.4 billion bitcoins. It was then discovered by the developer in a half-day period and a patch was released to invalidate the output of the transaction. But this is not simply to define the transaction itself as invalid, and a class called "negative output" of the transaction is defined as invalid, just that transaction is the use of Bitcoin before the prohibition of negative output value of the vulnerability, but defined as long as the total output amount can not be higher than the total amount of input. In the patch (should be published in the book) fixed the bug, specifically modified what it is.

The red-letter bold field in the figure is where the bug patch is mostly modified. The rule before the change is that "total" cannot be higher than "transaction", and the modified rule is that adding "total" cannot be a negative value. This modification of a node that has not been upgraded verifies the chunks generated by the upgraded nodes, so it is also a soft fork.

Now we abstract these three cases and give the definition of a soft fork: The soft fork is the data structure of the Bitcoin transaction (which is the widely circulated "consensus") changes, the non-upgraded node can verify the nodes that have been upgraded to produce the block, The nodes that have been upgraded can also validate the chunks produced by the non-upgraded nodes.

Soft fork changes to the "consensus" must also include changes to the block format, but the three cases here are modifications to the transaction format.

Let's look at the hard fork case.

5th specific case of hard fork execution on bitcoin blockchain

On March 12, 2013, it was bitcoin QT 0.8. version 0 The software was released, and the 0.8 version adopted a new database level db. Some miners have upgraded the Bitcoin QT 0.8 version, and some miners continue to use the Bitcoin qt0.7 version of the software. Each side produces blocks, but the chunks produced by the new database Bitcoin QT 0.8 are rejected by the qt0.7 version node. The specific reason is that the old database is sometimes not acceptable for chunks over 800Kb. Therefore, the block height of 225430 Bitcoin blockchain is divided into two chains, resulting in a Bitcoin blockchain generated two chains, one is a chain larger than 800kb chunks, and the other is to refuse to admit that these contain larger chunks of the chain, which occurs a hard fork.

The miners who used the bitcoin QT 0.8 release the chain they dug and went back to the bitcoin QT 0.7 release to continue digging.

The hard fork was an accident, a bug in the bitcoin QT 0.8 version of the software, which led to the use of the old software node to refuse to validate chunks produced by the new software node. But the reason for the hard fork is that the nodes that use the old software version refuse to validate the chunks produced by the new software version of the node, and then each side digs.

On July 4, 2015, a hard fork occurred in the Bitcoin blockchain at block height 363731. It was the bitcoin core developer who added Bip 66 to the new version of Bitcoin core 0.10.0. This was originally a soft fork modification, the main mine pool in the Bitcoin network using the 0.10 version of the software, but there is a pool BTC Nuggets did not upgrade, resulting in the BTC Nuggets dug out of the two blocks other miners refused to lose, and then each of the respective mining continuation of their own think is the right blockchain , resulting in a hard fork, divided into two chains.

The bitcoin.org then released an announcement calling for miners to upgrade to Bitcoin core 0.10.2 to eliminate Forks.

This is also an accident, the cause of the hard fork is the use of the new software version of the node refused to verify the use of the old software version of the node production block, and then each mining.

So far these two hard forks are accidents, but the cause of the hard fork is because the new block format on the different nodes of the divergence, the latter is because the transaction format on different nodes on the divergence. But because of the accident, the community did not discuss enough information, I also do not know the specific block or the transaction of which field was modified resulting from the fork.

Here, we can abstract these two cases, give a definition of a hard fork: hard fork refers to the Bitcoin block format or transaction format (this is widely circulated "consensus") changes, the non-upgraded node refused to verify the upgraded nodes produced by the block, However, the upgraded nodes can verify the blocks produced by the non-upgraded nodes, and then each of them continues to maintain the correct chain, so it is divided into two chains.

Let's look at the soft forks and hard forks that the community is planning but not yet activated.

the 6th chapter is planning the expansion of the block hard fork

The Bitcoin community is now planning a hard fork, because the current chunk is filled with transactions, and in order to accommodate more transactions in a block of 10 minutes, you need to modify the data structure of the block. Let's look at the data structure of the block first.

At present, the main complete node software of the Bitcoin network is bitcoin Core 0.12, the software stipulates that the "Chunk size" field in Table 6 has a maximum value of 1M. This leads to a limited number of Bitcoin transactions that the last field "trade" can hold, a transaction of at least 250 bytes, and a maximum of more than 4,000 trades in 1Mb, with an average of 7 trades per second only. And because the actual trades tend to reach a size of 500 bytes, in fact the average per second can only hold 3 trades.

So someone raised the maximum value of this field, such as Bitcoin Classic This software will be the maximum value of this field to 2M, and later there is a plan to take the first 2016 chunk size of the median and then multiply a contract to determine the size of the next chunk of the upper limit. The Bitcoin XT modifies this value to 20M and doubles every two years until the upper limit reaches 8.3G. The Bitcoin Unlimited, however, modifies this field directly to determine how large the pool is to be packaged.

The problem is that the chunks produced by the nodes that use the software that modifies this field are incompatible with nodes that are not upgraded and will refuse to validate the chunks produced by these new nodes. This results in a hard fork.

Here, we basically know that the so-called hard fork needs to modify the "consensus" refers to the modification of the block data structure format, or modify the transaction data structure format.

7th Chapter Isolation Witness Soft fork

There is also a block in the disguised expansion of the scheme is a quarantine witness, this solution is to modify the bitcoin transaction data structure, we still look at the figure.

The red field in the diagram is where the quarantine witness is primarily hands-off, which is to move this part of the data out of the transaction definition's data structure, and also to move out of the data structure defined by the block. As a result, there is less data on the transaction so that each transaction will be small in size, and the entire chunk will be able to accommodate more transactions in 1M space.

However, removing this field will also result in the failure of the upgraded node to validate the blocks produced by these upgraded nodes. But there is a way to let them do not refuse, but the code is very very complex to write, anyway, this code has been written for more than a year, and it has not been developed. The more complex the code, the more potential loopholes are, think of the Bip 66 is a soft fork caused by the hard fork, and this isolation witness more complex.

Here, once again, it is verified that the "consensus" of the so-called soft fork is to modify the block data structure format, or modify the transaction format.

8th Chapter Soft Fork and Hard fork the "consensus" to be modified is the modification of the data structure

Now we can summarize what the soft fork and hard fork are going to do, especially the difference between the soft fork and the hard fork to avoid being knocked out by the "consensus" that the definition is inaccurate.

The "consensus" of soft fork modification refers to the modification of the bitcoin transaction data structure or the modification of the Bitcoin block structure.

The "consensus" of the hard fork modification refers to the modification of the bitcoin transaction data structure, or the modification of the Bitcoin block structure.

At the object level of the specific modification, soft forks and hard forks are completely indistinguishable, so far as we see all that has happened, and the branching that is planned to happen, they all modify or attempt to modify the transaction data structure, or chunk structure.

The current, or planned, hard forks and soft forks will never modify the transaction history. Even if it is August 15, 2010, the issue of the amount of the day to brush out the currency, the change is also the transaction data structure, but the accompanying role is to invalidate a chunk of the transaction, because the transaction is in the definition of new transaction data structure is illegal.

So there is no difference between soft fork and hard fork in modifying the "consensus" level.

What's the difference between a soft fork and a hard fork?

9th Chapter The main difference between soft fork and hard fork is the compatibility between old and new nodes

The difference is the compatibility aspect of the old and new nodes. Soft fork Modify the data structure, the new node production of transactions and blocks can be verified and accepted by the old node, hard fork can not.

Just because the hard fork modifies the data structure, the new node blocks are rejected by the old node, and if the old node refuses to upgrade the software and insists on digging with the old data structure, the Bitcoin will generate two chains.

The soft fork does not show two chains because the node that is not upgraded can accept new data produced by the new node. But the soft fork in order to make the new node and the old node production of data is completely compatible, it is very difficult, because it is essentially different data structure we have to recognize each other, once there is a node refused to verify the different transactions or chunks, it will become a hard fork. This is why the Bip 66 soft fork eventually becomes a hard fork.

From the 4th chapter, we see that the modified method of the soft fork is to redefine the previously defined field, such as a multi-signature soft fork. Or a definition of a field that was originally left, such as a CSV soft fork.

But now, judging from the data structure of the transaction, all the fields are already occupied and defined accurately and in detail, and if you are going to do a soft fork again, you can only erase and redefine some of the original definitions of the fields. This can lead to the loss of functionality, and if this function is necessary, it will result in hard forks and new and old nodes rejecting each other. So the soft fork should be especially careful. This is also the practice of isolating witnesses. The quarantine witness removes a field directly, but in order to ensure compatibility between the old and new nodes, that is a waste of Saad.

The hard fork does not take into account the fact that no upgraded nodes will reject the data and code produced by the upgraded nodes, which is much simpler.

Let's look at the difference between the compatibility of a soft fork and a hard fork.

Soft fork and hard fork to achieve new and old node production of old and new data related to the major changes in a total of six:

1. No upgrade of the old node;

2. Upgrade the new node;

3. Old node transactions in the old trading format;

4. Trading of new trading format for new node;

5. Old blocks produced by old nodes that contain only old block formats;

6. New blocks are produced in the block format of the zone.

In fact, we should consider more complicated other factors, including complete node, SPV node, new and old node calculating force ratio, SPV mining node, unconfirmed transaction and multiple confirmation transactions, CSV trading, RBF trading ... All of these are factors to consider for compatibility. But in order to simplify, I only consider the above 6 major changes, we first make a list.

Soft forks need to ensure perfect compatibility requires to achieve the following two kinds of situations:

1. Upgraded nodes accept transactions and chunks produced by nodes that are not upgraded (backwards compatible);

2. Nodes that have not been upgraded accept liters