Python ethereum Code Analysis "2"

python ethereum Code Analysis "2" python version of the ether square Pyethapp Module

This chapter mainly introduces some key concepts of Chainservice and Powservice in Pyethapp module.

Totaldifficulty Total Difficulty:
Total difficulty difficulty is the sum of the difficulties of all the blocks in a current chain, and difficulty is used to indicate the longest chain, and if a node wants to sync data from another node, he chooses Difficulty the largest chain to synchronize data. It is noteworthy that the difficulty of Uncle Block has also been counted into Totaldifficulty, and the Phantom Agreement in the ether White paper explains why this design

Uncle Block Uncle Blocks: It is also a mine dug out of the block, it is also legal, but found late, or network transmission slightly slower, but not to become the longest chain on the block
The more than 10-second block spacing of the etheric square greatly increases the generation of solitary blocks and reduces security. By encouraging the reference of tertiary blocks, the reference to the main chain to obtain more security guarantees (because the lone block itself is also legal)
Blocks can be referenced without reference, or up to two tertiary blocks
The tertiary block must be the direct sub block of the first 2 layers of the block to the first 7 levels of the ancestors.
The referenced tertiary block cannot be referenced repeatedly
Using the block of the tert-block, you can get 1/32 of the mining compensation, namely 5*1/32=0.15625 ether. Get 2*0.15625=0.3125 ether max.
Reference http://blog.csdn.net/superswords/article/details/76445278
https://zhuanlan.zhihu.com/p/28928827

Head_candidate candidate Block: Miner's local candidate head block, equivalent to a temporary block, Head_candidate has been in existence and the mine trade union has been updated. The miners packed the deal into the block, calculating random numbers and broadcasting the secondary blocks as new blocks.

Contract: The contract itself is also an account. Whenever a transaction points to a contract account, the data property of the transaction transaction the contract as input to the contract.

Some basic concepts of the etheric square: The relationship between http://ethdocs.org/en/latest/contracts-and-transactions/index.html Chainservice and Powservice

The chain service is responsible for the synchronization and updating of the block chain, processing of the ETH_PROTOCOL packets of the connected nodes, and the broadcast of the transactions and blocks; POW service is the miner's mining services, calculated ' Lucky value ' after informing chain Service,chain The service writes chunks to the block chain and broadcasts them out.

Miners package the deal, update head_candidate

    @property
    def head_candidate (self):
        if self._head_candidate_needs_updating:
            self._head_candidate_ needs_updating = False
            # Make a copy of Self.transaction_queue because
            # make_head_candidate modifies it.
            Txqueue = Copy.deepcopy (self.transaction_queue)
            #将交易打包, referencing the Uncle block, executing the contract in the transaction, updating the block status
            Self._head_candidate, Self._head_candidate_state = Make_head_candidate (
                self.chain, Txqueue, Timestamp=int (Time.time ()))
        return Self._head_candidate

_on_new_head Callback for chain instance

    def _on_new_head (self, block):
        log.debug ("New head CBS", Num=len (SELF.ON_NEW_HEAD_CBS))
        Self.transaction_ Queue = Self.transaction_queue.diff (
            block.transactions)
        self._head_candidate_needs_updating = True
        # CB is POW Service callback function mine_head_candidate, update head_candidate and start digging for
        CB in SELF.ON_NEW_HEAD_CBS:
            CB (Block)

Powservice's Callback

    def mine_head_candidate (self, _=none):
        #打包当前交易队里中的交易并更新head_candidate
        HC = Self.chain.head_candidate
        If not self.active or self.chain.is_syncing:
            return
        elif (Hc.transaction_count = = 0 and not
              self.app.config [' POW '] [' Mine_empty_blocks ']):
            return

        log.debug (' mining ', difficulty=hc.difficulty)
        #开始挖矿
        Self.ppipe.put (' Mine ', Dict (Mining_hash=hc.mining_hash,
                                     block_number=hc.number,
                                     difficulty= hc.difficulty)))

After calculating the lucky value, call Chain.add_mined_block Write block chain and broadcast

 #成功找到幸运值 def recv_found_nonce (self, bin_nonce, Mixhash, Mining_hash): Log.info (' n Once found ', Mining_hash=mining_hash.encode (' hex ') #再次打包交易, update head_candidate block = Self.chain.head_candi Date if Block.mining_hash!= mining_hash:log.warn (' Mining_hash does not match ') return Fa LSE Block.header.mixhash = Mixhash Block.header.nonce = bin_nonce #添加新块并广播 if Self.chain.a Dd_mined_block (block): Log.debug (' mined blocks%d (%s) added to chain '% (Block.number, encode _hex (Block.hash[:8])) return True Else:log.debug (' failed to add mined blocks%d (%s) to C Hain '% (Block.number, Encode_hex (Block.hash[:8])) return False 

    def Add_mined_block (self, block):
        log.debug (' adding mined blocks ', block=block)
        assert isinstance (block, block
        #添加新块
        if Self.chain.add_block (block):
            log.debug (' added ', Block=block, Ts=time.time ())
            assert block = = Self.chain.head
            self.transaction_queue = Self.transaction_queue.diff (block.transactions)
            self._ head_candidate_needs_updating = True
            #广播新块
            self.broadcast_newblock (block, chain_difficulty= Self.chain.get_score (block)) return
            True
        log.debug (' Failed to add ', Block=block, Ts=time.time ())
        Return False

After writing the new block, call the NEW_HEAD_CB callback and start digging again
So far, the process of the whole cycle is the process by which miners continue to pack and trade and to broadcast new blocks Chainservice

Service Start Services started
Add the callback function for command in the Eth_protocol (Ether Square protocol) to the Eth_protocol instance at service startup

First look at the Eth_protocol protocol.
Ethernet Square Protocol Definition eth_protocol.py

Each peer object has a Eth_protocol instance. In Peer-to-peer protocol, when a node receives a Hello packet, all protocol instances are initialized, including Eth_protocol
Peer Object Receive Hello
Initialize Eth_protocol

    def connect_service (self, Service):
        assert isinstance (service, wiredservice)
        Protocol_class = service.wire_ Protocol
        assert Issubclass (Protocol_class, Baseprotocol)
        # Create Protcol instance which connects peer with Serivce
        protocol = Protocol_class (self, Service)
        # Register Protocol
        assert Protocol_class not in Self.protocols
        Log.debug (' Registering protocol ', Protocol=protocol.name, peer=self)
        self.protocols[ Protocol_class] = Protocol
        self.mux.add_protocol (protocol.protocol_id)
        Protocol.start () #调用chain_ Service.on_wire_protocol_start (self)

Chain service adds a callback function to the instance and sends the status packet to the other node

    def on_wire_protocol_start (self, proto): Log.debug ('----------------------------------') log.debug (' On
        _wire_protocol_start ', Proto=proto) assert isinstance (Proto, Self.wire_protocol) # Register Callbacks Proto.receive_status_callbacks.append (self.on_receive_status) #处理status数据包 Proto.receive_newblockhashes_callbac
        Ks.append (self.on_newblockhashes) proto.receive_transactions_callbacks.append (self.on_receive_transactions) Proto.receive_getblockheaders_callbacks.append (self.on_receive_getblockheaders) Proto.receive_blockheaders_call Backs.append (self.on_receive_blockheaders) proto.receive_getblockbodies_callbacks.append (self.on_receive_ getblockbodies) proto.receive_blockbodies_callbacks.append (self.on_receive_blockbodies) PROTO.RECEIVE_NEWB Lock_callbacks.append (self.on_receive_newblock) # Send status once connected to each other send their own block chain state status head = Self.chain . Head Proto.send_status (CHAin_difficulty=self.chain.get_score (head), Chain_head_hash=head.hash, genesis_hash=self.chain.ge Nesis.hash)

The Eth_protocol agreement includes
Status after establishing a connection with the new node, send each other's own block chain state
Newblockhashes to broadcast a batch of new chunks to the network hash
Transactions packet containing a batch of transactions
Getblockhashes starts with the specified hash and requests a batch of blockhashes
Blockhashes return getblockhashes Request
Getblocks starts with the specified hash and requests a batch block
Blocks return Getblocks Request
Newblock miners after mining after the broadcast new Area block, node to the block after verification to add to the local

1. Status Received:
A peer node that has already completed the handshake sends his current block chain network status Ethereum state,status packet is the first packet received after the connection is established between nodes.
The status packet is used to get the latest blocks in the network and to update the local block chain

    Class status (Baseprotocol.command): "" "protocolversion:the version of the Ethereum protocol this PE ER implements.
        At present. Networkid:the Network version of Ethereum for this peer.
        0 for the official testnet. Totaldifficulty:total difficulty of the best chain.
        Integer, as found in block header.
        Latesthash:the Hash of the "block" and the highest validated total difficulty.
        Genesishash:the Hash of the Genesis block.
            "" "cmd_id = 0 sent = False structure = [(' Eth_version ', rlp.sedes.big_endian_int), (' network_id ', Rlp.sedes.big_endian_int), (' Chain_difficulty ', rlp.sedes.big_endian_int), #totalDif Ficulty, the total difficulty of the chain (' Chain_head_hash ', rlp.sedes.binary), #latestHash, header block Hash (' Genesis_hash ', Rlp.sede S.binary)] #初始区块hash def create (self, proto, Chain_difficulty, Chain_head_hash, Genesis_hash): self.se NT = True Network_id = proto.service.app.config[' eth '].get (' network_id ', proto.network_id) return [Proto.version, network _id, Chain_difficulty, Chain_head_hash, Genesis_hash]

Receive packet received packets
Receive status resolution packet navigates to status handler function
Previously registered callback Eth_protocol instance initialized
On receive status Receive status processing function

Notice here that the version before the DAO event is not a forked version (or the normal logic first.) ）

    def on_receive_status (self, proto, Eth_version, network_id, Chain_difficulty, Chain_head_hash, Genesis_hash): Log.debug ('----------------------------------') log.debug (' status received ', Proto=proto , eth_version=eth_version) assert eth_version = = Proto.version, (eth_version, proto.version) #必须是同一个networ  Kid if network_id!= self.config[' eth '].get (' network_id ', proto.network_id): Log.warn ("Invalid network ID ", remote_network_id=network_id, expected_network_id=self.config[' eth '].get (' network_id ', proto.ne twork_id)) Raise Eth_protocol. Ethprotocolerror (' Wrong network_id ') # Check Genesis #初始区块hash必须一致 if Genesis_hash!=.
            Genesis.hash:log.warn ("Invalid Genesis hash", Remote_id=proto, Genesis=genesis_hash.encode (' hex ')) Raise Eth_protocol. Ethprotocolerror (' Wrong Genesis Block ') # Request chain #调用同步器同步数据
        Self.synchronizer.receive_status (Proto, Chain_head_hash, chain_difficulty) # Send Transactions #
            Gets the transactions known in the network but not yet counted into the blocks transactions transactions = Self.chain.get_transactions () If transactions: Log.debug ("Sending transactions", Remote_id=proto) #将这些交易告知对方 proto.send_transactions (*transaction S

Synchronizer Synchronizer synchronizes data receive_status:

    def receive_status (self, proto, Blockhash, chain_difficulty): "Called if a new peer is connected" log.
        Debug (' Status received ', Proto=proto, chain_difficulty=chain_difficulty) # memorize Proto with difficulty #将改节点区块链总难度记下 Self._protocols[proto] = chain_difficulty #对方头区块hash已经在本地存在, the If Self.chainservi is ignored Ce.knows_block (Blockhash) or self.synctask:log.debug (' existing task or known hash, discarding ') r Eturn if self.force_sync:blockhash, chain_difficulty = Self.force_sync log.debug (' Starti Ng forced Syctask ', Blockhash=blockhash.encode (' hex ') self.synctask = Synctask (self, proto, Blockhash, Chain_ Difficulty) #如果这条链总难度比自己的大, it is considered that the chain state is updated a bit, and from the header block hash he gave begins to synchronize the block, synchronization of the total difficulty from the connected node Chain_ Difficulty the biggest one begins to sync (but this does not guarantee that the chain of synchronization must be the main chain, followed by the analysis if the chain is not the main chain) elif chain_difficulty > Self.chain.head.chain_diffic
 Ulty (): Log.debug (' sufficient difficulty ')           Self.synctask = Synctask (self, proto, Blockhash, chain_difficulty) 

Fetch_hashchain requests to the network to specify a hash as a chunk hash of the header block

    def fetch_hashchain (self): Log_st.debug (' fetching hashchain ') Blockhashes_chain = [Self.blockhash] # Youngest to Oldest # for testing Purposes:skip the hash downoading stage # import AST # Blockhash Es_chain = ast.literal_eval (open ('/home/vub/blockhashes.pyast '). Read ()) [: 299000] Blockhash = Self.blockhash = Blo
        Ckhashes_chain[-1] Assert Blockhash not in Self.chain # Get block hashes until we found a known one
            Max_blockhashes_per_request = Self.initial_blockhashes_per_request while Blockhash not in Self.chain:

            # Proto with Highest_difficulty should is the proto we got the newblock from blockhashes_batch = [] # try with Protos protocols = Self.protocols if not Protocols:log_st.warn (' No protocols available ') return Self.exit (Success=false) #这里的protocols是各个已连接节点peer对象的eth_proto
Col instance, sorted by chain total difficulty size, most difficult in front            For Proto in Protocols:log.debug (' syncing with ', Proto=proto) if Proto.is_sto
                Pped:continue # Request assert Proto not in self.requests
                deferred = AsyncResult () Self.requests[proto] = Deferred #从指定hash开始, requesting a batch of blockhashes
                    Proto.send_getblockhashes (Blockhash, Max_blockhashes_per_request) Try:
                                                     #获取到这批BlockHashes Blockhashes_batch = Deferred.get (Block=true, timeout=self.blockhashes_request_timeout) except Gevent. Timeout:log_st.warn (' Syncing Hashchain timed out ') continue final
                    LY: # is also executed ' on the way out ' when any other clause of the TRY statement
     # is-left via a-break, continue or return statement.               Del Self.requests[proto] If not blockhashes_batch:log_st.warn (' empty Getblockhashes result ') Continue if isn't all (isinstance (BH, bytes) to BH in Blockhashe S_batch): Log_st.warn (' Got wrong data type ', expected= ' bytes ', receive
                D=type (Blockhashes_batch[0])) Continue break if not blockhashes_batch: Log_st.warn (' Syncing failed with all peers ', Num_protos=len (protocols)) return Self.exit (suc Cess=false #在获取到的这批blockhashes中直到找到一个blockhash是自己数据库中有的, notice here just find one of your own blockhash, and this blockhash is not necessarily your own local cha In the header block because the local header block may not be on the main chain block, as mentioned above.
            Nodes will always sync the longest chain, and may be from their own local chain of the parent block of a chunk of the beginning of the synchronization, which provides a certain error-correcting mechanism, so that the node can be corrected to the main chain up.
                For Blockhash in Blockhashes_batch: # youngest to Oldest assert utils.is_string (Blockhash) If Blockhash notIn Self.chain:blockhashes_chain.append (Blockhash) else:log_st.de Bugs (' found known Blockhash ', Blockhash=utils.encode_hex (Blockhash), Is_genesis=bool (block hash = = Self.chain.genesis.hash)) Break log_st.debug (' downloaded ' + str (len (blockhashes_c Hain)) + ' block hashes, ending with%s '% Utils.encode_hex (blockhashes_chain[-1]) max_blockhashes_per_reques
 t = self.max_blockhashes_per_request #取到最长链的这批blockhash后, starting sync block Self.fetch_blocks (blockhashes_chain)

Fetch blocks sync This batch of blocks to a network request

    def fetch_blocks (self, Blockhashes_chain): # Fetch blocks (no parallelism here) log_st.debug (' fetching Blocks ', Num=len (Blockhashes_chain)) assert Blockhashes_chain Blockhashes_chain.reverse () # Oldest to Yo Ungest num_blocks = Len (blockhashes_chain) num_fetched = 0 while blockhashes_chain:b Lockhashes_batch = blockhashes_chain[:self.max_blocks_per_request] T_blocks = [] # try with Proto s protocols = Self.protocols if not Protocols:log_st.warn (' No protocols available
                ') return Self.exit (Success=false) #向每个节点请求 to Proto in protocols: If proto.is_stopped:continue assert proto not in Self.requests # RE Quest Log_st.debug (' Requesting blocks ', Num=len (blockhashes_batch)) deferred = AsyncResult () self.Requests[proto] = deferred #向网络请求这批hash值对应的区块 proto.send_getblocks (*blockhashes_batch)
                Try:t_blocks = Deferred.get (block=true, Timeout=self.blocks_request_timeout) Except Gevent.
                Timeout:log_st.warn (' Getblocks timed out, trying next proto ') continue Finally:del Self.requests[proto] If not t_blocks:log_st.wa RN (' empty getblocks reply, trying next proto ') Continue elif not isinstance (t_blocks[0 ], Transientblock): Log_st.warn (' Received unexpected data ', Data=repr (t_blocks)) t  _blocks = [] Continue # We have results if not [B.header.hash to B in 
    T_blocks] = = Blockhashes_batch[:len (t_blocks)]: Log_st.warn (' Received, wrong blocks, should ban peer ')                T_blocks = [] Continue break # Add received T_blocks num_fetched = = Len (t_blocks) Log_st.debug (' Received blocks ', Num=len (t_blocks), Num_fetched=num_fetche
                D, Total=num_blocks, missing=num_blocks-num_fetched) if not t_blocks:

            Log_st.warn (' failed to fetch blocks ', Missing=len (Blockhashes_chain)) return Self.exit (Success=false)
            ts = Time.time () log_st.debug (' adding blocks ', qsize=self.chainservice.block_queue.qsize ())  For t_block in t_blocks:b = Blockhashes_chain.pop (0) assert T_block.header.hash = =
                B assert T_block.header.hash not in Blockhashes_chain #将获取的block添加到队列, last added to local database Self.chainservice.add_block (T_block, Proto) # This blocks if the queue is full log_st.debug (' adding B Locks done ', took=time.tIME ()-TS) # Done last_block = T_block assert