Building a block chain with Go--Part 3: Persistence and command-line interface _ tutorial

The translation of the series of articles I have put on the GitHub: blockchain-tutorial, follow-up updates will be on the GitHub, may not be synchronized here. If you want to run the code directly, you can also clone GitHub on the Tutorial warehouse and go to the SRC directory to execute make. Introduction

So far, we've built a block chain with a work-proof mechanism. With the work proved, the excavation also has a landing. Although the current implementation is getting closer to a block chain with a complete function, it still lacks some important features. In today's content, we will persist the chunk chain to a database, and then provide a simple command-line interface to do some interaction with the block chain. In essence, the chunk chain is a distributed database, but for the moment we ignore the "distributed" part and focus on "storage" only. Select Database

At present, our block chain implementation does not use the database, but in each run of the program, simply store the block chain in memory. So once the program exits, all the content disappears. We have no way to use this chain again, and there is no way to share it with others, so we need to store it on disk.

So, which database are we going to use? In fact, any database is OK. In Bitcoin's original paper, there is no mention of which particular database to use, and it all depends on how the developer chooses. Bitcoin Core, originally from the middle of the release, is now bitcoin a reference implementation, it is the use of leveldb. And what we're going to use is ... Boltdb

Because of it: very simple and minimalist use go implementation does not need to run a server to allow us to construct the desired data structure

Boltdb GitHub on the README is that: Bolt is a pure key-value store go database, inspired by Howard Chu's Lmdb. It is designed to provide a simple, fast and reliable database for projects that do not need a full database server such as Postgres and MySQL.

Because Bolt is intended to provide some low-level functionality, simplicity becomes the key. It's
There are not many APIs and only focus on getting and setting values. That's all.

Sounds like a perfect fit for our needs. To get a quick look:

Bolt uses a key value store, which means that it does not have tables like SQL RDBMS (Mysql,postgresql, etc.), no rows and columns. Instead, the data is stored as a key-value pair (Key-value pair, just like Golang's map). Key-value pairs are stored in bucket in order to group similar key-value pairs (similar to tables in an RDBMS). So, in order to get a value, you need to know a bucket and a key (key).

One thing to note is that the Bolt database has no data type: The key and value are byte arrays (byte array). Given the need to store go structures inside (exactly, that is, storage (block) blocks), we need to serialize them, in other words, to implement a mechanism for converting from the went struct to a byte array, as well as from a byte array to Convert back to go struct. Although we will use Encoding/gob to accomplish this goal, we can actually choose to use JSON, XML, Protocol buffers , and so on. The reason why you choose to use Encoding/gobis because it is simple and is part of the Go standard library. Database Structure

Before we begin implementing the logic of persistence, we first need to decide how to store it in the database. To this end, we can refer to the Bitcoin Core approach:

In simple terms, Bitcoin Core uses two "bucket" to store data: one of the bucket is blocks, which stores metadata that describes all the blocks in a chain, and another bucket is chainstate, which stores The state of a chain, that is, all the current unused transaction output, and some meta data

In addition, for performance reasons, Bitcoin Core stores each chunk (block) as a different file on disk. In this way, you do not need to load all (or part) of the block into memory just to read a single block. But, for the sake of simplicity, we are not going to achieve this.

In blocks ,key-> value is:

Key	value
B + 32 byte block hash	Block Index record
F + 4 bytes of file number	File Information record
L + 4 bytes of file number	The last block file number used
R + 1-byte Boolean	Are you reindex
F + 1-byte flag name length + flag name string	1 byte boolean:various flags that can is on or off
T + 32-byte transaction hash	Transaction index Record

In chainstate,key-> value is:

Key	value
C + 32-byte transaction hash	Unspent transaction output record for that transaction
B	32-byte block hash:the blocks hash up to which the database represents the unspent transaction outputs

Details can be seen here: _data_storage).

Because there is no deal at the moment, so we just need to blocks bucket. In addition, as mentioned above, we will store the entire database as a single file rather than storing chunks in separate files. Therefore, we do not need the document number (file numbers) related things. In the end, we'll use a key-value pair that has: 32-byte Block-hash-> block structure L-> chain the last chunk of the hash

That's all you need to know to implement the persistence mechanism. Serialization of

As mentioned above, in Boltdb, the value can only be []byte type, but we want to store the block structure. So, we need to use ENCODING/GOB to serialize these structures.

Let's implement the Serialize method of block (error handling is omitted here for brevity):

Func (b *block) Serialize () []byte {
    var result bytes. Buffer
    Encoder: = gob. Newencoder (&result)

    err: = Encoder. Encode (b) return result

    . Bytes ()
}

This section is more intuitive: first, we define a buffer to store the data after serialization. We then initialize a GOB encoder and encode the block, and the result is returned as a byte array.

Next, we need a serializable function that takes a byte array as input and returns a block. It is not a method, but rather a separate function (functions):

Func Deserializeblock (d []byte) *block {
    var block block

    Decoder: = Gob. Newdecoder (bytes. Newreader (d))
    err: = decoder. Decode (&block) return

    &block
}

This is the content of the serialization section. Persistence of

Let's start with the Newblockchain function. In the previous implementation, it would create a new
Blockchain example, and add the Genesis block to it. And now, what we want it to do is to open a database file to check if a block chain is already stored in the file.

If a chunk chain has been stored: Create a new Blockchain instance set the tip of the blockchain instance to the hash of the last block stored in the database

If there is no chunk chain: Create a creation block store to the database to save the creation block hash as the hash of the last block create a new blockchain instance with tip pointing to the Genesis block (Tip has tail, tip meaning, where tip stores the hash of the last block)

The code is probably like this:

Func Newblockchain () *blockchain {
    var tip []byte
    db, err: = Bolt. Open (DBFile, 0600, nil)

    err = db. Update (func (TX *bolt). TX) Error {
        b: = Tx. Bucket ([]byte (Blocksbucket))

        if b = = Nil {
            Genesis: = Newgenesisblock ()
            B, err: = Tx. Createbucket ([]byte (Blocksbucket))
            err = B.put (Genesis. Hash, Genesis. Serialize ())
            err = B.put ([]byte ("L"), Genesis. Hash)
            tip = Genesis. Hash
        } else {
            tip = B.get ([]byte ("L")
        } return

        nil
    })

    BC: = blockchain{tip, db}

    Return &BC
}

For a while, look at the code:

DB, err: = Bolt. Open (DBFile, 0600, nil)

This is the standard procedure for opening a boltdb file. Note that even if no such file exists, it will not return an error.

Err = db. Update (func (TX *bolt). Tx) Error {
...
})

In BOLTDB, database operations operate through a single transaction (transaction). There are two types of transactions: Read-only (read-only) and read-write (read-write). Here, a read-write transaction (DB) is turned on. Update (...) Because we might add a Genesis block to the database.

B: = Tx. Bucket ([]byte (Blocksbucket))

if b = = Nil {
    Genesis: = Newgenesisblock ()
    B, err: = Tx. Createbucket ([]byte (Blocksbucket))
    err = B.put (Genesis. Hash, Genesis. Serialize ())
    err = B.put ([]byte ("L"), Genesis. Hash)
    tip = Genesis. Hash
} else {
    tip = B.get ([]byte ("L")
}

Here is the core of the function. Here we first get the bucket of the storage block: If it exists, it reads the L key, if it does not exist, generates the creation block, creates the bucket, saves the chunk to it, and then updates the L key to the hash of the last block in the chain.

Also, note that creating a blockchain is a new way: