Neural networks from being fooled to being fooled (iii)

Introduction

In the previous chapter, although the BP neural network has made great progress, but it has some unavoidable problems, one of which is more confused is the problem of local optimal solution.

It is risky to touch only those things you already like, that you may be involved in a self-centered whirlpool that ignores anything that is slightly different from your standards, even if you would have liked it. This phenomenon is known as the "Filter Bubble" (bubble), and the technical term is "overfitting". -Inevitable

The problem with the so-called local optimal solution is that it is stuck in a small high position, but considers itself at the highest point, leading to the end of training.

Many optimization algorithms have been put forward, at present, the more fiery may be considered a combination of multiple algorithms. In the 2015 "Machine Learning" review, neural networks that evolved using genetic algorithms developed algorithms that could play Super Mario on their own.

Genetic algorithm

In the 9th century century, Darwin founded the scientific theory of biological evolution, with natural selection as the core of Darwinian theory of evolution, for the first time, the entire biological field of the occurrence and development of a materialistic, regular interpretation, overturned the special creationism and other idealistic metaphysics in the dominant position in biology, so that the biological revolution has occurred.

Darwin

In the important book, Origin of Species: he used the data accumulated in the 1830 's survey of global science to try to prove that species evolution was achieved through natural selection and artificial selection.

In 1967, Holland's student J.d.bagley, for the first time in his doctoral dissertation, coined the term "genetic algorithm (genetic algorithms)" as GA, seizing two important rules of the Origin of Species: crossover and mutation. The two processes are perfectly interpreted in the way of algorithms.

Super Mario World's AI

In 2002, Kenneth O Stanley was published in Massachusetts Institute of technology, evolving neural Networks through
Augmenting topologies (neural network evolutionary topological structure), referred to as neat.

In simple terms, the neural network evolutionary topology is based on the combination of genetic algorithm and neural network, which can overcome the problem of the local minimum value of the neural network to a maximum extent.

2015 sethbling used him to develop an AI that could play Super Mario world. and open source of their own code, just over 1000 lines of code, you can learn, and can clear the Super Mary.

The code is written in Lua, with more than 1000 lines of code, and the structure is very clear.

The code calculates weights using a neural network algorithm, and uses genetic algorithms (crossover and mutation) to optimize weight.

Weight calculation process for neural networks:

function evaluateNetwork(network, inputs)    table.insert(inputs, 1)    if #inputs ~= Inputs then        console.writeline("Incorrect number of neural network inputs.")        return {}    end    for i=1,Inputs do        network.neurons[i].value = inputs[i]    end    for _,neuron in pairs(network.neurons) do        local sum = 0        for j = 1,#neuron.incoming do            local incoming = neuron.incoming[j]            local other = network.neurons[incoming.into]            sum = sum + incoming.weight * other.value        end        if #neuron.incoming > 0 then            neuron.value = sigmoid(sum)        end    end    local outputs = {}    for o=1,Outputs do        local button = "P1 " .. ButtonNames[o]        if network.neurons[MaxNodes+o].value > 0 then            outputs[button] = true        else            outputs[button] = false        end    end    return outputsend

Cross:

function crossover(g1, g2)    -- Make sure g1 is the higher fitness genome    if g2.fitness > g1.fitness then        tempg = g1        g1 = g2        g2 = tempg    end    local child = newGenome()    local innovations2 = {}    for i=1,#g2.genes do        local gene = g2.genes[i]        innovations2[gene.innovation] = gene    end    for i=1,#g1.genes do        local gene1 = g1.genes[i]        local gene2 = innovations2[gene1.innovation]        if gene2 ~= nil and math.random(2) == 1 and gene2.enabled then            table.insert(child.genes, copyGene(gene2))        else            table.insert(child.genes, copyGene(gene1))        end    end    child.maxneuron = math.max(g1.maxneuron,g2.maxneuron)    for mutation,rate in pairs(g1.mutationRates) do        child.mutationRates[mutation] = rate    end    return childend

Invisible effort and optimization

Super Mario World Training process is relatively concise, training after a night, can smoothly pass. Again using the game method, see the power of the neural network. Through the neural network training weights, and the genetic algorithm so that their maximum limit does not fall into the local minimum value, can train a lot of excellent projects.

But this kind of application is at the beginning, the present stage can only use in the entertainment project, actually can the practical thing, also needs many efforts.

From:http://datartisan.com/article/detail/117.html

Neural networks from being fooled to being fooled (iii)