Implementation and consideration of Go game program (6) -- Construction of game tree

To implement uctAlgorithmYou need to build a game tree in the memory. each node in the game tree is a situation in the game process.

An obvious method is to build a tree-like data structure. Each node stores three types of information: 1) what this node represents. 2) Situation Information (current benefit value and number of attempts ). 3) The index of the subnode.

For the following two reasons, I have not used this method.

1) reduce memory usage as much as possible.

2) in detail, the topology structure generated during the Go game is not a tree, but a graph-the situation under different branches may overlap, although the frequency of this diamond coincidence is very small.

I used a hash table to save the necessary information, instead of logically having a tree (containing a small number of diamond overlapping data structures. The table items in the hash table store two types of information: 1) hash value. 2) situation information. Due to the extremely low weight of the hash algorithm used, I assume that the hash values in different situations will not overlap-even if there is overlap, the probability algorithms such as uct will not be very elegant. In this hash table, the hash value is not only the index information but also the unique identifier of the situation.

The advantages and disadvantages of the two methods are unknown.

Here we post the uct algorithm described earlier:

1) Search down from the root point of the game tree and execute 2 ).

2) If node A has a child node that has never been evaluated, execute 3); otherwise, execute 4 ).

3) Evaluate the subnode by using the Monte Carlo method, obtain the benefit value, and then update the average benefit value of all nodes from the subnode to the Root Node path. Execute 1 ).

4) Calculate the UCB value of each subnode, and use the subnode with the highest UCB value as node A. Execute 2 ).

5) the algorithm can be terminated at any time, usually after a specified time or number of attempts.

The child node with the highest average return value under the root node serves as the output of the algorithm.

In step 2), determine whether the sub-node does not exist in the hash table. In Steps 3) and 4), update the situation information in the hash table.

at last, I mentioned the hash algorithm I used-Zobrist hashing. This is a powerful tool for hash chess games. Wikipedia Zobrist hashing entry: Zobrist hashing.