Brief introduction of simulated annealing method for C # numerical calculation (i)

Summary

This paper introduces the basic idea of simulated annealing, in order to choose the main parameters in simulation, then gives a specific problem and solution for solving the extremum of two-dimensional function, and gives the C # source code.





L Overview

There are a large number of combinatorial optimization finish in the fields of management science, computer science, molecular physics and biology, as well as VLSI design, code design, image processing and electronic engineering. Many of these problems, such as traveling salesman problem, graph coloring problem, equipment layout problem and wiring problem, have not found an effective polynomial time algorithm so far. These problems have proved to be NP-complete problems.

In the 1982, Kirkpatrick introduced the annealing idea into combinatorial optimization, and proposed an algorithm for solving large-scale combinatorial optimization problems, which is especially effective for NP complete combinatorial optimization. This is due to the annealing process of the solid, that is, the temperature is first added to very high, and then slowly cooling (ie annealing), so as to achieve the lowest energy. If the rapid cooling (that is, quenching) can not reach the lowest point.

interpreted in [1] as: Simulation annealing is a technique which can be applied to any minimisation or learning process based on succes sive Update steps (either random or deterministic) where the update step length was proportional to a arbitrarily set para Meter which can play the role of a temperature. Then, in analogy with the annealing of metals, the temperature was made high in the early stages of the process for faster minimisation or learning, then is reduced for greater stability.

That is: The simulated annealing algorithm is a learning process (random or deterministic) that can be applied to a minimum value problem or a basic previous update. In this process, the length of each step of the update process is proportional to the corresponding parameter, which acts as a temperature role. Then, similar to the principle of metal annealing, in the initial phase to minimize or learn faster, the temperature is raised very high, and then (slowly) cooling to stabilize.



The main idea of simulated annealing algorithm

As far as the function minimum problem is concerned, the main idea of simulated annealing is: random walk (i.e. random selection point) in search interval (two-dimensional plane) and metropolis sampling criterion, so that stochastic walk converges to local optimal solution gradually. The temperature is an important control parameter in the Metropolis algorithm, and it can be considered that the size of the parameter controls the speed at which the process moves to local or global optimal solution at any time.

The cooling parameter table, the domain structure and the new solution generator, the acceptance criterion and the random number generator (i.e. the metropolis algorithm) compose the three pillars of the algorithm.



L key sampling and Metroplis algorithm:

Metropolis is an effective key sampling method, the algorithm is as follows: When the system changes from one state of energy to another, the corresponding energy changes from E1 to E2, and the probability is P = exp[-(e2-e1)/kt]. If the E2 < E1, the system receives this state, otherwise it receives this or discards this state in a random probability. The system will gradually tend to a stable distribution state with a constant number of iterations.

When the key sampling, the new state if downward is accepted (local optimal), if up (global search), at a certain probability to accept. Based on the initial solution and the transformation of a large number of solutions, the simulated annealing method can obtain the relative optimal solution of combinatorial optimization problem with the given control parameter value. Then, by reducing the value of the control parameter T and repeating the metropolis algorithm, the whole optimal solution of combinatorial optimization problem can be finally obtained when the control parameter T tends to zero. The value of the control parameter must be slowly attenuated.

The temperature is an important control parameter of Metropolis, and simulated annealing can be regarded as the iteration of Metroplis algorithm for decreasing control parameters. The start T value is large, may accept the poorer solution, with the reduction of T, can only accept a better deterioration of the solution, finally in T to 0 o'clock, no longer accept any deterioration of the solution.

At infinite temperatures, the system is evenly distributed immediately, accepting all proposed transformations. The smaller the attenuation of T, the longer the T reaches the end point, but the smaller the Markov chain, the shorter the time to reach the quasi equilibrium distribution.



L Parameter selection:

We call the cooling schedule a series of important parameters for adjusting the simulated annealing method. It is very important to control the initial value of the parameter T and its attenuation function, the corresponding Markov chain length and the stopping condition.

A cooling schedule should specify the following parameters:

1. The initial value t0 of the control parameter T;

2. The attenuation function of the control parameter T;

3. The length of the Markov chain LK. (that is, each random walk process, how many times to iterate, to tend to a quasi-equilibrium distribution, that is, a local convergence solution location)

4. Selection of end conditions



Effective cooling Schedule Criteria:

A Convergence of the algorithm: mainly depends on the attenuation function and the length of the Markov chain and the selection of the stopping criterion

Two Experimental performance of the algorithm: the Final Solution quality and CPU time



Selection of parameters:

First, the selection of the initial T0 of control parameters

It is generally required that the initial value of the T0 should be sufficiently large, that is, at the beginning of the high temperature state, and the metropolis reception rate is about 1.



Second, the selection of attenuation function

The attenuation function is used to control the annealing speed of the temperature, and a commonly used function is: T (n + 1) = K*t (n), where K is a constant that is very close to 1.





Three) selection of Markov chain length L

The principle is: under the condition that the attenuation function of attenuation parameter T is selected, L should be selected to restore the quasi balance on each value of the control parameter.



IV) Termination conditions

There are many kinds of termination conditions, different conditions have a great influence on the quality of the algorithm's performance and reconciliation, this article only introduces a common termination condition. That is, the difference between the last optimal solution and the newest one is less than that of a certain tolerance, and the Markov chain iteration can be stopped.





L Examples:

The description above may be too abstract, and the next section will illustrate with a practical example, where all the source code has been posted and you can learn a lot of details.



L Reference Documents:

1. Http://www.computer-dictionary-online.org/index.asp?q=simulated+annealing Computer Dictionary

2. Numeric Recipes in C

3. Computational Methods Series non-numerical parallel algorithm (first volume) simulated annealing algorithm