Neat algorithm--essentially genetic algorithm for automatic construction of neural networks

Implementation of Mario AI based on neat algorithm

The so-called neat algorithm is an evolutionary neural network (evolving neural Networks through augmenting) that enhances the topology, unlike the traditional neural networks we discussed earlier, which not only train and modify the weights of the network, It also modifies the topology of the network, including operations such as adding nodes and deleting nodes.
Several core concepts of the neat algorithm are:

• Genes: Connections in the network
• Genome: a collection of genes
• Species: A collection of groups of similar genomes
• Fitness: Somewhat similar to the reward function in reinforcement learning
• Generation: A set of trained genomes that, after each generation of training, will eliminate the genome based on fitness and new genomes by asexual reproduction and sexual reproduction
• Genetic variation: In the course of a newly generated genome, it is possible to change the weight of the network, increase the prominence of connections or neurons, or disable synapses or enable synapses

We showed the algorithm from the very beginning of the simple neural network, training to the late network

The basic idea of using neat algorithm to realize Mario's only customs clearance is to use the basic viewpoint of the above neat algorithm, to get real-time game data from game memory, to judge whether Mario is dead busy, calculate fitness value, judge whether Mario is cleared, etc., so as to use these as neural network input, The final output of Mario's operations, including up and down jumps, and other operations, such as:

Most of this algorithm realizes Mario's intelligent customs clearance relies on the simulator, uses the Lua language to write the corresponding script, obtains the game data and the Operation Mario. Neuroevolution with MARI/O. The implementation is as follows:

The implementation of Mario AI based on deep reinforcement learning

Neat algorithm is a relatively early algorithm, in 2013 the famous DeepMind proposed a depth enhancement learning algorithm, the algorithm is mainly based on the above discussed in the CNN and q-learning two kinds of algorithms, DeepMind's researchers applied the algorithm to AI clearance in a variety of small games in the Atari game console.
The core of its basic algorithm is the CNN we introduced before and the q-learning of enhanced learning, the basic flow of game intelligent customs clearance, such as:

Use CNN to identify the state of the game's total Mario and use the enhanced learning algorithm to make the action selection, then calculate the reward function based on the new return state and the historical state to give feedback to the Q function for iteration, and keep training until the game can be cleared. After training a game, the researchers found that the same parameters were used in other games, which showed that the algorithm had some universality. Reflects a learning process.

In the same way, DRL is applied to Mario and there is an open source implementation on GitHub: Aleju/mario-ai
Its final implementation is as follows:

We found that in the CNN identification process, every 4 frames of images will be a CNN recognition, this is the problem of recognition rate, the graph reflects the direct return function and the introduction of return function.

Excerpt from: http://www.cnblogs.com/Leo_wl/p/5852010.html

Neat algorithm--essentially genetic algorithm for automatic construction of neural networks