python ethereum 程式碼分析《2》

python ethereum 程式碼分析 《2》 python 版本以太坊 pyethapp 模組

本章主要介紹pyethapp 模組中的ChainService 和 PoWService 一些關鍵概念

totalDifficulty 總難度：
total difficulty總難度是當前某條鏈所有區塊難度的總和，total difficulty被用來指示最長的那一條鏈，某個節點如果想從其他節點同步資料的話，他會選擇total difficulty最大的那一條鏈來同步資料。值得注意的是uncle block的difficulty也被計算到totalDifficulty中去，以太坊白皮書中的幽靈協議闡述了為什麼這麼設計

uncle block 叔叔區塊：也是礦工挖出來的區塊，它也是合法的，但是發現的稍晚，或者是網路傳輸稍慢，而沒有能成為最長的鏈上的區塊
以太坊十幾秒的出塊間隔，大大增加了孤塊的產生，並且降低了安全性。通過鼓勵引用叔塊，使引用主鏈獲得更多的安全保證(因為孤塊本身也是合法的)
區塊可以不引用，或者最多引用兩個叔塊
叔塊必須是區塊的前2層~前7層的祖先的直接的子塊
被引用過的叔塊不能重複引用
引用叔塊的區塊，可以獲得挖礦報酬的1/32，也就是5*1/32=0.15625 Ether。最多獲得2*0.15625=0.3125 Ether
參考資料http://blog.csdn.net/superswords/article/details/76445278
https://zhuanlan.zhihu.com/p/28928827

head_candidate 候選頭區塊: 礦工本地的候選頭區塊，相當於一個臨時區塊，head_candidate一直存在並且礦工會一直更新。礦工將交易打包進這個區塊，計算出隨機數後將次區塊作為新區塊廣播出去。

contract ：合約，本身也是一個賬戶。每當一個交易指向一個合約賬戶時，該交易transaction的data屬性作為該合約的輸入執行該合約。

以太坊基本的一些概念：http://ethdocs.org/en/latest/contracts-and-transactions/index.html ChainService 和 PowService的關係

chain service負責區塊鏈的同步和更新，處理串連的各個節點的eth_protocol資料包，以及交易和區塊的廣播； pow service是礦工挖礦的服務，計算出’幸運值’後告知chain service，chain service 將區塊寫入區塊鏈並廣播出去。

礦工將交易打包，更新head_candidate

    @property    def head_candidate(self):        if self._head_candidate_needs_updating:            self._head_candidate_needs_updating = False            # Make a copy of self.transaction_queue because            # make_head_candidate modifies it.            txqueue = copy.deepcopy(self.transaction_queue)            #將交易打包，引用uncle區塊，執行交易中的合約，更新區塊狀態            self._head_candidate, self._head_candidate_state = make_head_candidate(                self.chain, txqueue, timestamp=int(time.time()))        return self._head_candidate

chain 執行個體的_on_new_head回調

    def _on_new_head(self, block):        log.debug('new head cbs', num=len(self.on_new_head_cbs))        self.transaction_queue = self.transaction_queue.diff(            block.transactions)        self._head_candidate_needs_updating = True        #cb是pow service的回呼函數mine_head_candidate，更新head_candidate並開始挖礦        for cb in self.on_new_head_cbs:            cb(block)

powservice的回調

    def mine_head_candidate(self, _=None):        #打包當前交易隊裡中的交易並更新head_candidate        hc = self.chain.head_candidate        if not self.active or self.chain.is_syncing:            return        elif (hc.transaction_count == 0 and              not self.app.config['pow']['mine_empty_blocks']):            return        log.debug('mining', difficulty=hc.difficulty)        #開始挖礦        self.ppipe.put(('mine', dict(mining_hash=hc.mining_hash,                                     block_number=hc.number,                                     difficulty=hc.difficulty)))

計算出幸運值後，調用chain.add_mined_block寫入區塊鏈並廣播

    #成功找到幸運值    def recv_found_nonce(self, bin_nonce, mixhash, mining_hash):        log.info('nonce found', mining_hash=mining_hash.encode('hex'))        #再次打包交易，更新head_candidate        block = self.chain.head_candidate        if block.mining_hash != mining_hash:            log.warn('mining_hash does not match')            return False        block.header.mixhash = mixhash        block.header.nonce = bin_nonce        #添加新塊並廣播        if self.chain.add_mined_block(block):            log.debug('mined block %d (%s) added to chain' % (                block.number, encode_hex(block.hash[:8])))            return True        else:            log.debug('failed to add mined block %d (%s) to chain' % (                block.number, encode_hex(block.hash[:8])))            return False

    def add_mined_block(self, block):        log.debug('adding mined block', block=block)        assert isinstance(block, Block)        #添加新塊        if self.chain.add_block(block):            log.debug('added', block=block, ts=time.time())            assert block == self.chain.head            self.transaction_queue = self.transaction_queue.diff(block.transactions)            self._head_candidate_needs_updating = True            #廣播新塊            self.broadcast_newblock(block, chain_difficulty=self.chain.get_score(block))            return True        log.debug('failed to add', block=block, ts=time.time())        return False

在寫入新塊完成後，調用new_head_cb回調，再次開始挖礦
至此，整個迴圈的過程就是礦工不斷打包交易挖礦到廣播新塊的過程 ChainService

service start 服務啟動
服務啟動時將eth_protocol（以太坊協議）中command對應的回呼函數添加到eth_protocol執行個體中

先看看eth_protocol協議
以太坊協議定義eth_protocol.py

每個peer對象都有一個eth_protocol執行個體。在p2p協議中，當節點收到hello資料包的時候，便初始化所有協議執行個體，其中包括eth_protocol
peer對象receive hello
初始化eth_protocol

    def connect_service(self, service):        assert isinstance(service, WiredService)        protocol_class = service.wire_protocol        assert issubclass(protocol_class, BaseProtocol)        # create protcol instance which connects peer with serivce        protocol = protocol_class(self, service)        # register protocol        assert protocol_class not in self.protocols        log.debug('registering protocol', protocol=protocol.name, peer=self)        self.protocols[protocol_class] = protocol        self.mux.add_protocol(protocol.protocol_id)        protocol.start()#調用chain_service.on_wire_protocol_start(self)

chain service為該執行個體添加回呼函數，並向對方節點發送status資料包

    def on_wire_protocol_start(self, proto):        log.debug('----------------------------------')        log.debug('on_wire_protocol_start', proto=proto)        assert isinstance(proto, self.wire_protocol)        # register callbacks        proto.receive_status_callbacks.append(self.on_receive_status)#處理status資料包        proto.receive_newblockhashes_callbacks.append(self.on_newblockhashes)        proto.receive_transactions_callbacks.append(self.on_receive_transactions)        proto.receive_getblockheaders_callbacks.append(self.on_receive_getblockheaders)        proto.receive_blockheaders_callbacks.append(self.on_receive_blockheaders)        proto.receive_getblockbodies_callbacks.append(self.on_receive_getblockbodies)        proto.receive_blockbodies_callbacks.append(self.on_receive_blockbodies)        proto.receive_newblock_callbacks.append(self.on_receive_newblock)        # send status  一旦串連就向對方發送自己的區塊鏈狀態status        head = self.chain.head        proto.send_status(chain_difficulty=self.chain.get_score(head), chain_head_hash=head.hash,                          genesis_hash=self.chain.genesis.hash)

eth_protocol協議中包括
Status 與新節點建立串連後，互相發送自己的區塊鏈狀態
NewBlockHashes 向網路中廣播一批新區塊的hash
Transactions 包含一批交易的資料包
GetBlockHashes 從指定hash開始，請求一批BlockHashes
BlockHashes 返回GetBlockHashes請求
GetBlocks 從指定hash開始，請求一批Block
Blocks 返回GetBlocks請求
NewBlock 礦工挖礦後廣播新區塊，節點接到該區塊後驗證後添加到本地

1.收到status：
某個已經完成握手的peer節點發送他當前的區塊鏈網路狀態ethereum state，status資料包是節點之間建立串連後收到的第一個資料包。
通過status資料包來擷取網路中最新的區塊並更新本地的區塊鏈

    class status(BaseProtocol.command):        """        protocolVersion: The version of the Ethereum protocol this peer implements. 30 at present.        networkID: The network version of Ethereum for this peer. 0 for the official testnet.        totalDifficulty: Total Difficulty of the best chain. Integer, as found in block header.        latestHash: The hash of the block with the highest validated total difficulty.        GenesisHash: The hash of the Genesis block.        """        cmd_id = 0        sent = False        structure = [            ('eth_version', rlp.sedes.big_endian_int),            ('network_id', rlp.sedes.big_endian_int),            ('chain_difficulty', rlp.sedes.big_endian_int),#totalDifficulty，該鏈的總難度            ('chain_head_hash', rlp.sedes.binary),#latestHash，頭區塊hash            ('genesis_hash', rlp.sedes.binary)]#初始區塊hash        def create(self, proto, chain_difficulty, chain_head_hash, genesis_hash):            self.sent = True            network_id = proto.service.app.config['eth'].get('network_id', proto.network_id)            return [proto.version, network_id, chain_difficulty, chain_head_hash, genesis_hash]

receive packet收到資料包
receive status解析資料包定位到status處理函數
之前登入回調eth_protocol執行個體初始化時登入
on receive status receive status處理函數

這裡注意一點，這裡看的是DAO事件之前的版本，不是分叉過後的版本（還是先按正常邏輯來。。）

    def on_receive_status(self, proto, eth_version, network_id, chain_difficulty, chain_head_hash,                          genesis_hash):        log.debug('----------------------------------')        log.debug('status received', proto=proto, eth_version=eth_version)        assert eth_version == proto.version, (eth_version, proto.version)        #必須是同一個networkid        if network_id != self.config['eth'].get('network_id', proto.network_id):            log.warn("invalid network id", remote_network_id=network_id,                     expected_network_id=self.config['eth'].get('network_id', proto.network_id))            raise eth_protocol.ETHProtocolError('wrong network_id')        # check genesis        #初始區塊hash必須一致        if genesis_hash != self.chain.genesis.hash:            log.warn("invalid genesis hash", remote_id=proto, genesis=genesis_hash.encode('hex'))            raise eth_protocol.ETHProtocolError('wrong genesis block')        # request chain        #調用同步器同步資料        self.synchronizer.receive_status(proto, chain_head_hash, chain_difficulty)        # send transactions        #擷取網路中已知的但尚未被計入區塊的交易transactions        transactions = self.chain.get_transactions()        if transactions:            log.debug("sending transactions", remote_id=proto)            #將這些交易告知對方            proto.send_transactions(*transactions)

synchronizer同步器同步資料receive_status：

    def receive_status(self, proto, blockhash, chain_difficulty):        "called if a new peer is connected"        log.debug('status received', proto=proto, chain_difficulty=chain_difficulty)        # memorize proto with difficulty        #將改節點區塊鏈總難度記下        self._protocols[proto] = chain_difficulty        #對方頭區塊hash已經在本地存在，則忽略        if self.chainservice.knows_block(blockhash) or self.synctask:            log.debug('existing task or known hash, discarding')            return        if self.force_sync:            blockhash, chain_difficulty = self.force_sync            log.debug('starting forced syctask', blockhash=blockhash.encode('hex'))            self.synctask = SyncTask(self, proto, blockhash, chain_difficulty)        #如果這條鏈總難度比自己的大，則認為這條鏈狀態更新一點，並從他給出的頭區塊hash開始同步區塊，同步的時候從已串連節點中總難度chain_difficulty最大的那個開始同步（但是這並不能保證同步的鏈就一定是主鏈，後面會分析如果同步的鏈不是主鏈的情況）        elif chain_difficulty > self.chain.head.chain_difficulty():            log.debug('sufficient difficulty')            self.synctask = SyncTask(self, proto, blockhash, chain_difficulty)

fetch_hashchain 向網路請求以指定hash作為頭區塊的一批區塊hash

    def fetch_hashchain(self):        log_st.debug('fetching hashchain')        blockhashes_chain = [self.blockhash]  # youngest to oldest        # For testing purposes: skip the hash downoading stage        # import ast        # blockhashes_chain = ast.literal_eval(open('/home/vub/blockhashes.pyast').read())[:299000]        blockhash = self.blockhash = blockhashes_chain[-1]        assert blockhash not in self.chain        # get block hashes until we found a known one        max_blockhashes_per_request = self.initial_blockhashes_per_request        while blockhash not in self.chain:            # proto with highest_difficulty should be the proto we got the newblock from            blockhashes_batch = []            # try with protos            protocols = self.protocols            if not protocols:                log_st.warn('no protocols available')                return self.exit(success=False)            #這裡的protocols是各個已串連節點peer對象的eth_protocol執行個體，按鏈總難度大小排序，難度最大在前面            for proto in protocols:                log.debug('syncing with', proto=proto)                if proto.is_stopped:                    continue                # request                assert proto not in self.requests                deferred = AsyncResult()                self.requests[proto] = deferred                #從指定hash開始，請求一批BlockHashes                proto.send_getblockhashes(blockhash, max_blockhashes_per_request)                try:                    #擷取到這批BlockHashes                    blockhashes_batch = deferred.get(block=True,                                                     timeout=self.blockhashes_request_timeout)                except gevent.Timeout:                    log_st.warn('syncing hashchain timed out')                    continue                finally:                    # is also executed 'on the way out' when any other clause of the try statement                    # is left via a break, continue or return statement.                    del self.requests[proto]                if not blockhashes_batch:                    log_st.warn('empty getblockhashes result')                    continue                if not all(isinstance(bh, bytes) for bh in blockhashes_batch):                    log_st.warn('got wrong data type', expected='bytes',                                received=type(blockhashes_batch[0]))                    continue                break            if not blockhashes_batch:                log_st.warn('syncing failed with all peers', num_protos=len(protocols))                return self.exit(success=False)            #在擷取到的這批blockhashes中直到找到一個blockhash是自己資料庫中有的，注意這裡只要找到一個自己有的blockhash就可以了，而且這個blockhash不一定是自己本地chain的頭區塊，因為本地的頭區塊有可能不是在主鏈上的區塊，上面已經提到過這點。節點總是會去同步最長的那條鏈，而且可能是從自己本地chain的頭區塊的父輩區塊的某個區塊開始同步，這樣就提供了一定的錯誤修正機制，讓節點可以糾正到主鏈上去。            for blockhash in blockhashes_batch:  # youngest to oldest                assert utils.is_string(blockhash)                if blockhash not in self.chain:                    blockhashes_chain.append(blockhash)                else:                    log_st.debug('found known blockhash', blockhash=utils.encode_hex(blockhash),                                 is_genesis=bool(blockhash == self.chain.genesis.hash))                    break            log_st.debug('downloaded ' + str(len(blockhashes_chain)) + ' block hashes, ending with %s' % utils.encode_hex(blockhashes_chain[-1]))            max_blockhashes_per_request = self.max_blockhashes_per_request        #取到最長鏈的這批blockhash後，開始同步區塊        self.fetch_blocks(blockhashes_chain)

fetch blocks 向網路請求同步這批區塊

    def fetch_blocks(self, blockhashes_chain):        # fetch blocks (no parallelism here)        log_st.debug('fetching blocks', num=len(blockhashes_chain))        assert blockhashes_chain        blockhashes_chain.reverse()  # oldest to youngest        num_blocks = len(blockhashes_chain)        num_fetched = 0        while blockhashes_chain:            blockhashes_batch = blockhashes_chain[:self.max_blocks_per_request]            t_blocks = []            # try with protos            protocols = self.protocols            if not protocols:                log_st.warn('no protocols available')                return self.exit(success=False)            #向每個節點請求            for proto in protocols:                if proto.is_stopped:                    continue                assert proto not in self.requests                # request                log_st.debug('requesting blocks', num=len(blockhashes_batch))                deferred = AsyncResult()                self.requests[proto] = deferred                #向網路請求這批hash值對應的區塊                proto.send_getblocks(*blockhashes_batch)                try:                    t_blocks = deferred.get(block=True, timeout=self.blocks_request_timeout)                except gevent.Timeout:                    log_st.warn('getblocks timed out, trying next proto')                    continue                finally:                    del self.requests[proto]                if not t_blocks:                    log_st.warn('empty getblocks reply, trying next proto')                    continue                elif not isinstance(t_blocks[0], TransientBlock):                    log_st.warn('received unexpected data', data=repr(t_blocks))                    t_blocks = []                    continue                # we have results                if not [b.header.hash for b in t_blocks] == blockhashes_batch[:len(t_blocks)]:                    log_st.warn('received wrong blocks, should ban peer')                    t_blocks = []                    continue                break            # add received t_blocks            num_fetched += len(t_blocks)            log_st.debug('received blocks', num=len(t_blocks), num_fetched=num_fetched,                         total=num_blocks, missing=num_blocks - num_fetched)            if not t_blocks:                log_st.warn('failed to fetch blocks', missing=len(blockhashes_chain))                return self.exit(success=False)            ts = time.time()            log_st.debug('adding blocks', qsize=self.chainservice.block_queue.qsize())            for t_block in t_blocks:                b = blockhashes_chain.pop(0)                assert t_block.header.hash == b                assert t_block.header.hash not in blockhashes_chain                #將擷取的block添加到隊列，最後添加到本機資料庫                self.chainservice.add_block(t_block, proto)  # this blocks if the queue is full            log_st.debug('adding blocks done', took=time.time() - ts)        # done        last_block = t_block        assert