Derek interprets Bytom Source-persistent storage leveldb

Author: Derek

Brief introduction

GitHub Address: Https://github.com/Bytom/bytom

Gitee Address: Https://gitee.com/BytomBlockc ...

This chapter describes Derek's interpretation of-bytom source analysis-Persistent storage leveldb

The author uses the MacOS operating system, and the other platforms are similar

Golang version:1.8

LEVELDB Introduction

The LEVELDB database is used by default over the original chain. Leveldb is a very efficient KV database implemented by Google. Leveldb is a single-process service with very high performance on a 4-core Q6600 CPU machine, with write data exceeding 40w per second, while random read performance exceeds 10w per second.

由于Leveldb是单进程服务，不能同时有多个进程进行对一个数据库进行读写。同一时间只能有一个进程，或一个进程多并发的方式进行读写。

Store all the information on the chain address, asset transactions, etc. on the data storage layer than the original chain.

Leveldb to delete and change the operation

Leveldb is a high-performance k/v storage developed by Google, this section we introduce leveldb how to change the leveldb additions and deletions.

package mainimport (    "fmt"    dbm "github.com/tendermint/tmlibs/db")var (    Key        = "TESTKEY"    LevelDBDir = "/tmp/data")func main() {    db := dbm.NewDB("test", "leveldb", LevelDBDir)    defer db.Close()    db.Set([]byte(Key), []byte("This is a test."))    value := db.Get([]byte(Key))    if value == nil {        return    }    fmt.Printf("key:%v, value:%v\n", Key, string(value))    db.Delete([]byte(Key))}// Output// key:TESTKEY, value:This is a test.

The above output is the result of executing the program.

The program has made additions and deletions to the Leveld operation. Dbm. Newdb gets the DB object, a directory called Test.db is generated under the/tmp/data directory. This directory holds all the data for that database.
Db. Set sets the value of key, the key does not exist, the new, and the key exists is modified.
Db. Get gets the value data in key.
Db. Delete Deletes the data for key and value.

Database than the original chain

By default, the Datastore directory is in the data directory under the--home parameter. In the case of the Darwin platform, the default database is stored in the $HOME/library/bytom/data.

• Accesstoken.db token information (Wallet access control permission)
  Core.db Core database, store the data of the main chain. including block information, transaction information, asset information, etc.
  Discover.db end-to-end node information in a distributed network
• Trusthistory.db
  TXDB.DB Store Transaction Related information
  Txfeeds.db currently does not use this feature over the original chain code version and does not introduce
  WALLET.DB Local wallet database. Store information such as users, assets, transactions, Utox, etc.

All of the above databases are managed by the database module

Compared to the original database interface

The data persistence store is managed by the database module in the comparison chain, but the persistence-related interface is in Protocol/store.go

type Store interface {    BlockExist(*bc.Hash) bool    GetBlock(*bc.Hash) (*types.Block, error)    GetStoreStatus() *BlockStoreState    GetTransactionStatus(*bc.Hash) (*bc.TransactionStatus, error)    GetTransactionsUtxo(*state.UtxoViewpoint, []*bc.Tx) error    GetUtxo(*bc.Hash) (*storage.UtxoEntry, error)    LoadBlockIndex() (*state.BlockIndex, error)    SaveBlock(*types.Block, *bc.TransactionStatus) error    SaveChainStatus(*state.BlockNode, *state.UtxoViewpoint) error}

• Blockexist the existence of chunks according to the hash
• Getblock the chunk according to the hash
• Getstorestatus get store's storage status
• Gettransactionstatus the status of all trades in the block according to the hash
• Gettransactionsutxo caches all utxo associated with the input TXs
• Getutxo (*BC. Hash) gets all the Utxo in the block according to the hash
• Loadblockindex Load block index, read all block header information from DB and cache in memory
• Saveblock storage block and transaction status
• Savechainstatus sets the state of the main chain, and when the node first starts, the node determines whether the main chain is initialized based on the content of key Blockstore.

Key prefix than the original chain database

Database/leveldb/store.go

var (    blockStoreKey     = []byte("blockStore")    blockPrefix       = []byte("B:")    blockHeaderPrefix = []byte("BH:")    txStatusPrefix    = []byte("BTS:"))

• Blockstorekey Main chain status prefix
• Blockprefix Block Information prefix
• Blockheaderprefix Header information Prefix
• Txstatusprefix Trade Status Prefix

Getblock Query Block Process analysis

Database/leveldb/store.go

func (s *Store) GetBlock(hash *bc.Hash) (*types.Block, error) {    return s.cache.lookup(hash)}

Database/leveldb/cache.go

func (c *blockCache) lookup(hash *bc.Hash) (*types.Block, error) {    if b, ok := c.get(hash); ok {        return b, nil    }    block, err := c.single.Do(hash.String(), func() (interface{}, error) {        b := c.fillFn(hash)        if b == nil {            return nil, fmt.Errorf("There are no block with given hash %s", hash.String())        }        c.add(b)        return b, nil    })    if err != nil {        return nil, err    }    return block.(*types.Block), nil}

The Getblock function eventually executes the lookup function. The lookup function has two steps in total:

• Hash value is queried from the cache and returned if found
• The callback FILLFN callback function if it is queried from the cache. The FILLFN callback function stores the information from the disk that the block information is stored in the cache and returns the block.

The FILLFN callback function actually fetches the getblock under Database/leveldb/store.go, which takes the block information from the disk and returns it.