I saw the recommendation of this article in roundup of week 1, 17th on aigamedev.com. Out of my interest in Chinese chess and computer games, although I cannot steal go and go AI, however, I carefully read this article and think that the content here is different from the knowledge I have learned about computer games. So translate it to help you better understand the knowledge. I am very poor at English. Please kindly advise if you have any translation errors. In addition, the author of this article should be Chinese. I really hope that he will write it into Chinese in the future. ==================================================== Monte Carlo Method in game AIS
Application of Monte Carlo algorithms in game (GO) AIFriday, 255.l 25th, 2008 PM written by: qqqchn Author: qqqchn Translation: Lai Yong Hao (Love butterfly) original address: http://expertvoices.nsdl.org/cornell-cs322/2008/04/25/monte-carlo-method-in-game-ais/As of my classmates have posted, the Monte Carlo method isn' t actually any single method, but actually represents an entire class of methods which involve taking random samples to find a result. an interesting application my partner Nd I found for the Monte Carlo method was for one of the Go AIS we made for one of our other projects. (Go is an English ent Chinese board game that is still very popular today in East Asia, the rules and details can be found here) as many of my colleagues have said, the Monte Carlo algorithm is not an algorithm, but a series of algorithms about random sampling. My partner and I found an interesting Monte Carlo algorithm application: using it in go AI. (Go is an ancient intellectual game from China. It is still very popular in East Asia today. Refer to here) one of the reasons we chose to use the Monte Carlo method was because the immense number of possible moves in Go made using the minimax algorithm (one of the more common methods used for finding the next" best "move in wrong game AIS like chess by consecutively maximizing and minimizing the score for a player up to a certain depth, more details here) far too com Putationally intensive when looking at more than 2 or 3 moves ahead (looking only 4 moves ahead on a mere 9 × 9 Board takes about 81 ^ 4> 4 million Board evaluations ). an interesting quote into strating the computational intensity of GO games on a full 19 × 19 board is that "the number of possible go Games far exceeds the number of atoms in the universe" (more details and derivation here) interesting fa CTS: Lower Bound on number of possible go games on 19 × 19 board is about 10 ^ (10 ^ 48 ). upper Bound is 10 ^ (10 ^ 171 ). one of the reasons for choosing the Monte Carlo algorithm is that go uses extremely small algorithms (Minimax algorithm, an algorithm commonly used in chess and games to select the "best" Method for the next step, refer to here) it takes a lot of effort to calculate the number of steps after two or three steps. (81 ^ 4 (more than 4 million) is required to calculate the number of steps after 9x9) secondary disk valuation ). The computation complexity of Go (19x19) is much greater than the number of all atoms in the universe (refer to here ). In fact, go (19x19) calculates the lower limit of 10 ^ (10 ^ 48) and the upper limit is 10 ^ (10 ^ 171 ). So another way we used to evaluate how "good" a move is was to use the Monte Carlo method. what the Monte Carlo method does in this case to estimate how good or bad a certain move is for a given board position is to play "virtual games" without strating what wowould happen if two random AIS (AIS playing completely randomly) played out those moves. the way it does this is to start from this board position And play each of the viable moves in a fixed number of games with all subsequent moves being completely random. then after all of the "virtual games" are finished, we wocould average the total scores of each game and let it represent the "goodness" of the original move which spawned that game. finally by choosing the move with the highest average score, the Monte Carlo AI wowould then play this move In the actual game itself, based on the assumption that the moves which score better over a large number of random games wocould be "better" moves in general. therefore, we use the Monte Carlo algorithm to evaluate how good a method is (poor ). The Monte Carlo algorithm evaluates how good a method is (poor) by using two random AI (the method selected is completely random) to play a number of virtual games on a given disk ". Starting from a given disk surface, and then calculating a specified number of subsequent moves on each reachable interface, the system creates a completely random "virtual chess ". Then, we calculate the average values of all feasible steps to reflect the "good" approach. Finally, select the method with the highest average value. Monte Carlo AI applies this method in real chess and games. This is based on the assumption that this high score method is generally better than the outcome of other choices. For our project, we let our AI play about 500 virtual games for each move, which on slower computers actually can take a while, but it is still far faster than trying to use the minimax algorithm to look ahead just 4 moves (just over 1 million evaluations compared to 4 million + ). in addition, the results of the Monte Carlo AI are pretty good as it can generally defeat most of our other AIS (minima X ai looking 2 or 3 moves ahead and random AIS), and it even put up a decent fight against some beginner human players as well. in our project, we asked AI to play "virtual chess" for every 500 games ". This also has a lot of computing workload. if the machine is "broken down", it may take a long time to calculate. However, it is still a four-step forward computation than using a very small and extremely large algorithm (the calculation workload is about four steps for 9x9 Board computation (about + disks need to be evaluated, as shown in the previous article) it is much faster. Monte Carlo AI works well. It can usually defeat extremely small algorithms such as AI (two or three steps of computing) and random AI, which are similar to humans who are new to go.
This article was first published on the blog of liyong Hao (Lianhua butterfly.
Worth noting is that one very important factor for how well the Monte Carlo method works in this case is the scoring function which you use to decide a player's score given a certain board position. the one we used which is very straightforward and relatively simple in that it just assigns an empty spot to whoever has the closest stones to that spot, with ties being broken by number of stones near It. this isn' t the most accurate or valid tive scoring method, but it worked decently well enough for our purposes. it is worth noting that the Monte Carlo algorithm depends on a very important factor, that is, the valuation function for a specific disk. We use a simple function: assign an empty point to the nearest piece. If there are multiple pieces, they are evenly divided. It may not be accurate and efficient, but it is enough for us. The AI we developped using Monte Carlo methods was one of the better AIS we made, but it is still nowhere near the capabilities of a decently experienced amateur human player. especially, the AI starts losing out near the end game when tactics mean a lot more than overall strategy (which Monte Carlo and minimax seem to do well ). and the fact that we are using random moves to play each "virtual g AME "means that we can get very different results each time we play it, especially near the end game where results of moves really depend on the quality of subsequent moves, which in this case are completely random. the Monte Carlo algorithm AI we developed is a better one in our initial AI, but it is still far from well-trained players. When the game ends, tactics are more important than strategy, and AI is easy to lose (this problem exists with Monte Carlo and extremely small algorithms ). We use a random method to play "virtual chess" in every game, so we get different results each time. The final results depend on the quality of subsequent methods when the final outcome is approaching, and the subsequent methods are completely random, so the effect is unsatisfactory. Go is considered by prior to be the most complicated game we know of to date, and it is very unlikely that we will be able to come even marginally close to solving the game anytime soon (want to even try writing out 10 ^ (10 ^ 48 )?). But it seems equally unlikely that people will give up on trying anytime soon either, as has been proven by human tenacity in the face of other "insurmountable" odds in the past (landing on the Moon ...). Go is regarded as the most complex game so far, and we cannot solve it in the near future. But everyone will not give up, because it has proved that humans are indomitable in the face of "insurmountable" problems (such as the moon landing ). Note: When I said "random" in this post, I naturally mean the specified udorandom number generators computers use, which isn' t really random, but was more than close enough for our project. note: The "random" in this article refers to the pseudo-random number used by the computer, rather than the real random number, but it is good from the project perspective. Citations: Reference http://en.wikipedia.org/wiki/Monte_Carlo_method http://en.wikipedia.org/wiki/Go_%28board_game%29 http://en.wikipedia.org/wiki/Go_complexity http://en.wikipedia.org/wiki/Minimax go AI project cs478 (Gordon Briggs, Qin Chen) -Unfortunately not finished yet so don't really have any statistics yet to cite-Go AI project cs478 (Gordon Briggs, Qin Chen) has not been completed yet, so real statistics cannot be provided. --For Chinese Reference: Monte Carlo algorithm: Love butterfly) blog, ghost.
