Ant colony algorithm, genetic algorithm, simulated annealing algorithm Introduction _ Algorithm

Source: Internet
Author: User
Tags new set

Introduction of Ant colony algorithm, genetic algorithm and simulated annealing algorithm

Exhaustive method

Enumerate all the possibilities and go on to get the best results. As figure one, you need to go straight from point A to point G to know that F is the highest (best solution). The optimal solution obtained by this algorithm is certainly the best, but it is also the least efficient.



Although the best solution can be obtained by exhaustive method, the efficiency is extremely low. In order to improve efficiency, you can not enumerate all the results, only part of the result set, if a solution in this decomposition is optimal, then consider it as the optimal solution. Obviously, this may not get the real optimal solution, but the efficiency is much higher than the exhaustive method.

There are many ways to enumerate only partial solutions.

Greedy method

When enumerating all the solutions, when the solution encountered is optimal in the present case, it is considered to be the optimal solution. As shown in figure I, when from point A to point B, the B point is considered to be the optimal solution because B is better than the solution of point A.

Obviously, the efficiency is very high, but the optimal solution quality is very poor.

Climbing method

The greedy method is only compared with the previous one, in order to improve the quality of the optimal solution, not only compared to the previous solution, but also to the last solution, if better than the previous and subsequent solutions, then it is considered to be the optimal solution. As shown in figure one, when it comes to point C, it is found to be better than the preceding B and D-points, so it is considered to be the best solution.

Simulated annealing algorithm

Mountain climbing algorithm is simple to achieve, its main disadvantage is that it will fall into local optimal solution, and not necessarily can search the global optimal solution.

As shown in figure one, search is stopped after a point has been searched. If we can jump out of the local optimal solution, the quality of the optimal solution will be much better. If you search for a point, jump to another place with a certain probability. It is possible to jump out of the local optimal solution A. If you jump to point e after a certain number of jumps, you will find the optimal solution of the global.

If the probability does not change, then it will continue to jump, will not end. Can make this probability gradually smaller, to the end to stabilize. The probability of this is gradually reduced, similar to the annealing process of metal smelting, so called simulated annealing algorithm.

The simulated annealing algorithm (simulated ANNEALING,SA), which was first applied to combinatorial optimization by Kirkpatrick, is a random optimization algorithm based on Mente-carlo iterative solution strategy, The starting point is based on the similarity between the annealing process of solid matter in physics and the general combinatorial optimization problem. The simulated annealing algorithm is based on a higher initial temperature, with the decreasing of the temperature parameter, and the probability jump characteristic in the solution space to find the global optimal solution of the objective function randomly, that is, the local optimal solution can jump out of probability and finally tend to the global optimal.

The key of the simulated annealing algorithm is to control the temperature (probability) to reduce the speed of the parameter R, this parameter range is 0<r<1. If the parameter r is too large, the search to the global optimal solution may be higher, but the search process is longer. If R is too small, the search process will be fast, but it may eventually reach a local optimal value.

The simulated annealing algorithm can not guarantee the real optimal solution, but the optimal solution with high quality is obtained under the condition of good efficiency.

Genetic algorithm

Genetic algorithm is a kind of evolutionary algorithm, which is used in computational mathematics to solve the optimal search algorithm. The evolutionary algorithm was originally developed by reference to some phenomena in evolutionary biology, the process of reproduction and development, through breeding, genetic crossover, Gene mutation, the adaptation of individuals will be phased out, and more adaptable individuals will be more and more. Then after the n generation of natural selection, the preserved individuals are highly adaptable.

The genetic algorithm is initially a poor solution of the solution set population, through genetic crossover to reproduce the next generation of the solution set population. In the process of crossover, there is a certain probability that a gene mutation occurs. In the next generation of solutions, by natural selection of the fittest, the poorer solutions (individuals) are eliminated, and only the better solution (individual) can reproduce, thus generating a population representing a new set of solutions. This process will result in a population that is more adaptable to the environment than the previous generation, as naturally evolving epigenetic populations. After many generations of breeding and natural selection, solutions that are close to the real optimal solution can be obtained.

The principle of elitism can be used to optimize the basic genetic algorithm. The so-called elitism principle is to prevent the optimal solution in the evolution process from being destroyed by crossover and mutation, and the optimal solution of each generation can be reproduced intact to the next generation.

Ant Colony algorithm

Ant colony Algorithm (Ant Colony Optimization, ACO), also known as Ant Algorithm, is a probabilistic algorithm for finding optimal paths in graphs. It was proposed by Marco Dorigo in his doctoral dissertation in 1992, inspired by the behavior of ants discovering pathways during the search for food.

As the ant moves along the path, it chooses the path it will take, based on the secretions left by the ants that pass by. The probability of selecting a path is proportional to the strength of the secretions on the path. Therefore, the collective behavior of groups composed of large numbers of ants actually constitutes a positive feedback phenomenon of learning information: The more ants a path passes, the greater the likelihood that the ant will choose that path. Through this information exchange, the ant's individual seeks the shortest path to the food.

Ant colony algorithm is based on this feature, by imitating the behavior of ants, so as to achieve optimization. When the program first finds the target, the path is almost impossible to be optimal, and may even contain countless wrong choices and be extremely verbose. However, the program can go through the pheromone principle of the ants looking for food, constantly correcting the original route, making the whole route shorter, and finally finding the best route.

The essence of this optimization process is:

Selection mechanism: The more pheromone path, the greater the probability of being chosen.

Update mechanism: The pheromone on the path increases with the passage of the ant, and fades away over time.

Coordination mechanism: ants are actually through secretions to communicate with each other and work together. Finally, the optimal solution is found by exchanging information and cooperating with each other, so that it has a strong ability to find better solutions.

Error mechanism: It is obvious that if ants move to a place with more pheromone, the problem of local optimal solution will be caused. However, there are always some rebellious spirit of the ants, will not move to more information, so you can jump out of the local optimal solution, find the global optimal solution.


Summarize:

Genetic algorithm: The advantage is that it can handle the constraint well, can jump out of the local optimal, and finally get the global optimal solution, the global search ability is strong, the disadvantage is slow convergence, weak local search ability, long running time, and easy to be affected by parameters.
Simulated annealing: The advantage is that the local search ability is strong, the running time is short; the disadvantage is that the global search ability is poor and susceptible to the influence of parameters.
Mountain climbing algorithm: It is obvious that mountain climbing algorithm is simpler and more efficient, but it can not get better solution when dealing with large scale problems with many constraints.

Contact Us

The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion; products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the content of the page makes you feel confusing, please write us an email, we will handle the problem within 5 days after receiving your email.

If you find any instances of plagiarism from the community, please send an email to: info-contact@alibabacloud.com and provide relevant evidence. A staff member will contact you within 5 working days.

A Free Trial That Lets You Build Big!

Start building with 50+ products and up to 12 months usage for Elastic Compute Service

  • Sales Support

    1 on 1 presale consultation

  • After-Sales Support

    24/7 Technical Support 6 Free Tickets per Quarter Faster Response

  • Alibaba Cloud offers highly flexible support services tailored to meet your exact needs.