Building a blockchain based on the Java language (iii)--Persistence & command line

Introduction
In the previous article, we implemented the blockchain workload proof mechanism (POW), as much as possible to achieve the mining. But there are many important features that are not implemented from real blockchain applications. Today we will implement the storage mechanism of blockchain data and save the blockchain data generated each time. One thing to note is that the blockchain is essentially a distributed database, and we don't implement "distributed" in this case, focusing only on the data storage part.

Recommend a Java in-house learning Group: 725633148, the group to find management free to receive learning materials and videos. Nothing wrong is free to collect! Big man small white are welcome, we learn together progress together!

Database selection

So far, there is no block storage in our implementation mechanism, which causes our chunks to be saved in memory after each build. It's not easy for us to re-use the blockchain, to generate chunks from scratch every time, and not to share our blockchain with others, so we need to store it on disk.

Which database should we choose? In fact, the Bitcoin white Paper does not explicitly specify which database to use, so it is up to the developer to decide. In the Bitcoin Core developed by the Chinese, leveldb is used. Original Building Blockchain in Go. Part 3:persistence and CLI are used with BOLTDB, which is better for go language support.

But we are using Java to implement, BOLTDB does not support Java, here we choose Rocksdb

Rocksdb is a key-value storage engine developed and maintained by the Facebook database engineering team, more powerful than LEVELDB, and detailed information about ROCKSDB, please move to the official documentation: https://github.com/ FACEBOOK/R, there is not much to introduce.
Data
Before we begin to implement data persistence, let's first determine how we can store our data. To do this, let's first look at how Bitcoin is done.

In short, Bitcoin uses two "buckets (buckets)" to store data:

Blocks. Describes the metadata for all the blocks on the chain.
Chainstate. The state of the storage blockchain refers to all current Utxo (no transaction output is spent) and some meta data.
"In the Bitcoin world there are no accounts, no balances, only utxo that are scattered into the blockchain. ”
See: The No. 06 chapter of the second edition of "Mastering Bitcoin"-input and output of transactions
In addition, each chunk of data is stored on disk in a separate file format. This is done for performance reasons: When reading a single chunk of data, it is not necessary to load all chunk data into memory.

In blocks this bucket, the stored key value pairs:

' B ' + 32-byte block hash, block index record
Index records for chunks
' F ' + 4-byte file number--File information record
File Information record
' l ', 4-byte file number:the last block file number used
The file encoding used by the latest chunk
' R '-1-byte boolean:whether we ' re in the process of reindexing
is in the process of rebuilding the index
' F ' + 1-byte flag name length + flag name string--1 byte boolean:various flags that can is on or off
Various flag flags that can be turned on or off
' t ' + 32-byte transaction Hash--Transaction index record
Trading Index Records
In chainstate this bucket, the stored key value pairs:

' C ' + 32-byte transaction Hash--Unspent transaction output record for that transaction
A utxo record of a transaction
' B '-32-byte block hash:the block hash up to which the database represents the unspent transaction outputs
The database represents the chunk hash of the utxo (sorry, I haven't figured it out yet ...). ）
Since we have not yet implemented trading-related features, we only use block buckets here. In addition, as mentioned earlier, we do not realize that each chunk of data is stored separately on separate files, but in a single file. Therefore, we do not store data related to file encoding, so that the key value pairs we use are simplified to:

32-byte Block-hash, block structure (serialized)
Key-value pairs of chunk data and chunk hash
' l ', the hash of the last block in a chain
Key value pairs for the latest chunk hash
Serialization of
Rocksdb key and value can only be stored in the form of byte[], where we need to use the serialization and deserialization library Kryo, the code is as follows:

Package one.wangwei.blockchain.util;

Import Com.esotericsoftware.kryo.Kryo;
Import Com.esotericsoftware.kryo.io.Input;
Import Com.esotericsoftware.kryo.io.Output;

/**

• Serialization Tool class
• @author wangwei

• @date 2018/02/07
  */
  public class Serializeutils {

  /**

  • deserialization
  • @param byte array corresponding to bytes object
  • @return
    */
    Public stat IC Object Deserialize (byte[] bytes) {
    Input input = new input (bytes);
    Object obj = new Kryo (). Readclassandobject (input);
    Input.close ();
    Return obj;
    }

  /**

  • serialization
  • @param object needs to be serialized
  • @return
    */public Static byte[] Serialize (Object object) {
    Output output = new output (4096,-1);
    New Kryo (). Writeclassandobject (output, object);
    byte[] bytes = Output.tobytes ();
    Output.close ();
    Return bytes;
    }
    }
    Persistent
    As mentioned above, we use rocksdb here, we first write a related tool class Rocksdbutils, the main functions are as follows:

Putlastblockhash: Save the hash value of the latest chunk
Getlastblockhash: Query The hash value of the latest chunk
Putblock: Saving chunks
Getblock: Querying chunks
Note: Boltdb supports Bucket features, and Rocksdb does not, and we use a uniform prefix for processing.
Rocksdbutils
Package one.wangwei.blockchain.util;

Import Lombok. Getter;
Import One.wangwei.blockchain.block.Block;
Import org.rocksdb.Options;
Import Org.rocksdb.RocksDB;
Import org.rocksdb.RocksDBException;

/**

• Rocksdb Tool Class
• @author Wangwei

• @date 2018/02/27
  */
  public class Rocksdbutils {

  /**

  • Blockchain data files
    */
    private static final String Db_file = "blockchain.db";
    /**
  • Chunk Bucket Prefix
    */
    private static final String blocks_bucketPREFIX = "BLOCKS";

  Private volatile static rocksdbutils instance;

  public static Rocksdbutils getinstance () {
  if (instance = = null) {
  Synchronized (Rocksdbutils.class) {
  if (instance = = null) {
  Instance = new Rocksdbutils ();
  }
  }
  }
  return instance;
  }

  @Getter
  Private Rocksdb Rocksdb;

  Private Rocksdbutils () {
  Initrocksdb ();
  }

  /**

  • Initialize ROCKSDB
    */
    private void Initrocksdb () {
    try {
    Rocksdb = Rocksdb.open (new Options (). Setcreateifmissing (True), db_file);
    } catch (Rocksdbexception e) {
    E.printstacktrace ();
    }
    }

  /**

  • Save the hash value of the latest chunk
  • @param tipblockhash
    */
    public void Putlastblockhash (String tipblockhash) throws Exception {
    Rocksdb.put (serializeutils.serialize (Blocks_bucket_prefix + "L"), Serializeutils.serialize (TipBlockHash));
    }

  /**

  • Query the hash value of the latest chunk
  • @return
    */
    Public String Getlastblockhash () throws Exception {
    byte[] lastblockhashbytes = Rocksdb.get (serializeutils.serialize (Blocks_bucket_prefix + "L"));
    if (lastblockhashbytes! = null) {
    Return (String) serializeutils.deserialize (lastblockhashbytes);
    }
    Return "";
    }

  /**

  • Save chunks
  • @param block
    */
    public void Putblock (block block) throws Exception {
    byte[] key = serializeutils.serialize (Blocks_bucket_prefix + block.gethash ());
    Rocksdb.put (Key, Serializeutils.serialize (block));
    }

  /**

  • Query chunks
  • @param blockhash
  • @return
    */
    Public Block Getblock (String blockhash) throws Exception {
    byte[] key = serializeutils.serialize (Blocks_bucket_prefix + blockhash);
    Return (Block) serializeutils.deserialize (Rocksdb.get (key));
    }

}
Create a chunk chain
Now let's optimize the code logic for the Blockchain.newblockchain interface, and change to the following logic:

The code is as follows:

/**

• <p> Create a block chain </p>
• @return
  */
  public static Blockchain Newblockchain () throws Exception {
  String Lastblockhash = Rocksdbutils.getinstance (). Getlastblockhash ();
  if (Stringutils.isblank (Lastblockhash)) {
  Block Genesisblock = Block.newgenesisblock ();
  Lastblockhash = Genesisblock.gethash ();
  Rocksdbutils.getinstance (). Putblock (Genesisblock);
  Rocksdbutils.getinstance (). Putlastblockhash (Lastblockhash);
  }
  return new Blockchain (Lastblockhash);
  }
  Modify the Blockchain data structure to record only the hash value of the latest blockchain

public class Blockchain {

@Getterprivate String lastBlockHash;private Blockchain(String lastBlockHash) {    this.lastBlockHash = lastBlockHash;}

}
After each mining, we also need to save the latest chunk information and update the latest blockchain hash value:

/**

• <p> Add Chunks </p>
• @param data
  */
  public void Addblock (String data) throws Exception {
  String Lastblockhash = Rocksdbutils.getinstance (). Getlastblockhash ();
  if (Stringutils.isblank (Lastblockhash)) {
  throw new Exception ("Fail to add block into blockchain!");
  }
  This.addblock (Block.newblock (Lastblockhash, data));
  }

/**

• <p> Add Chunks </p>
• @param block
  */
  public void Addblock (block block) throws Exception {
  Rocksdbutils.getinstance (). Putlastblockhash (Block.gethash ());
  Rocksdbutils.getinstance (). Putblock (block);
  This.lastblockhash = Block.gethash ();
  }
  Here, the function of the storage section is complete, we also lack a function:

Retrieving a block chain
Now that all of our chunks are saved to the database, we are able to reopen the existing blockchain and add new chunks to it. But it also led us to no longer be able to print out all the chunks in the blockchain, because we didn't store the chunks in the array. Let's fix this flaw!

We create an inner class blockchainiterator in Blockchain, which, as an iterator to the blockchain, iterates the output chunk information sequentially by the hash connection before the block, with the following code:

Public class Blockchain {

/** * Blockchain iterator */public class Blockchainiterator {private String currentblockhash;    Public Blockchainiterator (String currentblockhash) {this.currentblockhash = Currentblockhash; }/** * Whether there is a next chunk * * @return */public Boolean hashnext () throws Exception {if (stringutils.        IsBlank (Currentblockhash)) {return false;        } Block Lastblock = Rocksdbutils.getinstance (). Getblock (Currentblockhash);        if (Lastblock = = null) {return false;        }//Creation block Direct release if (Lastblock.getprevblockhash (). Length () = = 0) {return true;    } return Rocksdbutils.getinstance (). Getblock (Lastblock.getprevblockhash ())! = NULL; }/** * returns block * * @return */public block Next () throws Exception {Block Currentblock = Rocks        Dbutils.getinstance (). Getblock (Currentblockhash); if (currentblock! = null) {This.currentblockhash = Currentblock.getprevblockhash();        return currentblock;    } return null;     }}   ....

}
Test
/**

• Test
• @author Wangwei

• @date 2018/02/05
  */
  public class Blockchaintest {

  public static void Main (string[] args) {
  try {
  Blockchain Blockchain = Blockchain.newblockchain ();

      blockchain.addBlock("Send 1.0 BTC to wangwei");    blockchain.addBlock("Send 2.5 more BTC to wangwei");    blockchain.addBlock("Send 3.5 more BTC to wangwei");    for (Blockchain.BlockchainIterator iterator = blockchain.getBlockchainIterator(); iterator.hashNext(); ) {        Block block = iterator.next();        if (block != null) {            boolean validate = ProofOfWork.newProofOfWork(block).validate();            System.out.println(block.toString() + ", validate = " + validate);        }    }} catch (Exception e) {    e.printStackTrace();}

  }
  }

/ Output /

Block{hash= ' 0000012f87a0510dd0ee7048a6bd52db3002bae7d661126dc28287bd6c23189a ', prevBlockHash= ' 0000024b2c23c4fb06c2e2c1349275d415efe17a51db24cd4883da0067300ddf ', Data= ' Send 3.5 more BTC to Wangwei ', timeStamp= 1519724875, nonce=369110}, validate = True
block{hash= ' 0000024b2c23c4fb06c2e2c1349275d415efe17a51db24cd4883da0067300ddf ', prevblockhash= ' 00000b14fefb51ba2a7428549d469bcf3efae338315e7289d3e6dc4caf589d79 ', data= ' Send 2.5 more BTC to Wangwei ', timeStamp= 1519724872, nonce=896348}, validate = True
block{hash= ' 00000b14fefb51ba2a7428549d469bcf3efae338315e7289d3e6dc4caf589d79 ', prevblockhash= ' 0000099ced1b02f40c750c5468bb8c4fd800ec9f46fea5d8b033e5d054f0f703 ', data= ' Send 1.0 BTC to Wangwei ', timeStamp= 1519724869, nonce=673955}, validate = True
block{hash= ' 0000099ced1b02f40c750c5468bb8c4fd800ec9f46fea5d8b033e5d054f0f703 ', prevblockhash= ', data= ' Genesis Block ', timestamp=1519724866, nonce=840247}, validate = True
Command line interface
CLI part of the content, here do not do a detailed introduction, specifically to see the end of the GitHub source link. The approximate steps are as follows:

Configuration

Add Pom.xml Configuration

<project>

...<dependency> <groupId>commons-cli</groupId> <artifactid>commons-cli</artifactid > <version>1.4</version></dependency>...<plugin> <groupId> Org.apache.maven.plugins</groupid> <artifactId>maven-assembly-plugin</artifactId> <version >3.1.0</version> <configuration> <archive> <manifest> <a                Ddclasspath>true</addclasspath> <classpathPrefix>lib/</classpathPrefix>        <mainClass>one.wangwei.blockchain.cli.Main</mainClass> </manifest> </archive> <descriptorRefs> <descriptorRef>jar-with-dependencies</descriptorRef> &LT;/DESCR iptorrefs> </configuration> <executions> <execution> <id>make-assembly& Lt;/id> <!--This was used for inheritance meRges-<phase>package</phase> <!--Specifies that a jar package merge operation is performed on the packaging node--<goals > <goal>single</goal> </goals> </execution> </executions ></plugin&gt, .....

</project>
Project Engineering Package

$ mvn Clean && MVN Package
Execute command

Printing Help information

$ Java-jar Blockchain-java-jar-with-dependencies.jar-h

Add a chunk

$ Java-jar Blockchain-java-jar-with-dependencies.jar-add "Send 1.5 BTC to Wangwei"
$ java-jar blockchain-java-jar-with-dependencies.jar-add "Send 2.5 BTC to Wangwei"
$ Java-jar Blockchain-java-jar-with-dependencies.jar-add "Send 3.5 BTC to Wangwei"

Print Blockchain

$ Java-jar Blockchain-java-jar-with-dependencies.jar-print
Recommend a Java in-house learning Group: 725633148, the group to find management free to receive learning materials and videos. Nothing wrong is free to collect! Big man small white are welcome, we learn together progress together!

Summarize
In this article we implement the Blockchain storage function, next we will implement the address, transaction, wallet, some of the functions of the column.

Building a blockchain based on the Java language (iii)--Persistence & command line