Scientific Computing classical algorithms

Document directory

• 1. The simulated annealing algorithm can be divided into three parts: solution space, target function, and initial solution.
• 2. Basic Ideas:
• Genetic algorithm principle
• Features of Genetic Algorithms
• Working principle of Neural Networks
• Features of Artificial Neural Networks

Simulated Annealing Algorithm

The simulated annealing algorithm is derived from the solid annealing principle. The solid is heated to a sufficiently high level, and then cooled slowly. when heated, the particles inside the solid become unordered with the temperature rise, and the internal energy increases, when slowly cooling, the particles gradually become orderly. When each temperature reaches the equilibrium state, and finally reaches the ground state at normal temperature, the energy can be reduced to a minimum. According to the Metropolis principle, the probability that the particle tends to be balanced at the temperature T is e-△e/(KT), where E is the internal energy when the temperature T, and △e is the amount of change, K is the Boltzmann constant. Using Solid-State annealing to simulate the combination optimization problem, the internal energy E is simulated as the target function value F, and the temperature T is evolved into the control parameter T, that is, the simulated annealing algorithm for solving the combination optimization problem is obtained: start from initial solution I and initial value T of the control parameter, repeat the iteration of the current solution to "generate a new solution → calculate the target function deviation → accept or discard", and gradually decrease the tvalue, the current solution at algorithm termination is the obtained approximate optimal solution. This is a heuristic random search process based on the Monte Carlo iterative solution. The annealing process is controlled by the cooling schedule, including the initial value T of the Control Parameter and its attenuation factor △t, the number of iterations of each tvalue l, and the condition s.

1. The simulated annealing algorithm can be divided into three parts: solution space, target function, and initial solution. A. solution space:

It is a set of all possible (feasible or including unfeasible) solutions of the problem, and it limits the scope of initial and new solutions. For non-constrained optimization problems, any possible solution (possible solution) is a feasible solution. Therefore, the solution space is a set of all feasible solutions. In many combination optimization problems, in addition to meeting the optimal requirements of the target function, a set of constraints must be met. Therefore, the solution set may contain infeasible so1ution ). To this end, we can limit that the solution space is only a set of all feasible solutions, that is, the constraints on the solution are taken into account when constructing the solution. The solution space can also contain infeasible solutions, the so-called penalty function is added to the target function to "Punish" the appearance of infeasible solutions.

B. Target functions:

It is a mathematical description of the optimization goal of the problem. It is usually expressed as a sum of several optimization goals. The selection of the target function must correctly reflect the overall optimization requirements for the problem. For example, as mentioned above, when the solution space contains an infeasible solution, the objective function should contain a penalty function for the infeasible solution, so as to convert a constrained optimization problem into a non-constrained optimization problem. Generally, the target function value is not necessarily the optimization target value of the problem, but its corresponding relationship should be obvious. In addition, the objective function formula should be easy to calculate, which will help simplify the calculation of the objective function difference in the optimization process to improve the efficiency of the algorithm.

C. Initial solution:

It is the starting point of algorithm iteration. Experiments show that the simulated annealing algorithm is robust (robust), that is, the final solution is almost independent from the selection of the initial solution.

2. Basic Ideas:

1. Initialization: the initial temperature T (sufficiently large), the initial solution state s (the starting point of the algorithm iteration), and the number of iterations of each t value L
2. For k = 1 ,......, L perform steps (3) to 6th:
3. Generate new solution s ′
4. Calculates the incremental △t' = C (s ')-C (s), where C (s) is the evaluation function.
5. If △t' <0, then s 'is accepted as the new current solution. Otherwise, probability exp (-△t'/T) is used as the new current solution.
6. If the termination condition is met, the current solution is output as the optimal solution to end the program. The termination condition is usually set to terminate an algorithm when several consecutive new solutions are not accepted.
7. T gradually decreases, and t-> 0, then go to step 1.

Genetic Algorithm

The idea of genetic algorithms is based on Darwin's evolution and Mendel's genetics.

The most important thing about Darwin's evolution is the survival principle of the fittest. It believes that each species is increasingly adapting to the environment in the development. The basic features of each individual in the species are inherited by the offspring, but the offspring will have some new changes different from those of the parent generation. When the environment changes, only the individual characteristics that bear adapts to the environment can be preserved.

The most important thing about Mendel genetics is genetic genetics. It believes that genes exist in cells as passwords and are contained in chromosomes as genes. Each gene has a special location and controls a specific nature; therefore, each individual produced by each gene has a certain degree of adaptability to the environment. Mutation and hybridization can generate offspring that are more adaptable to the environment. The adaptive genetic structure can be preserved after the natural elimination of optimization and removal.

Genetic algorithms (GA) are essentially a direct search method independent of specific problems.

Genetic algorithm principle

Genetic Algorithm GA expresses the solution of the problem as a "chromosome", which is also a binary encoded string. In addition, a group of "chromosomes" is given before the genetic algorithm is executed, that is, the hypothetical solution. Then, place these assumptions in the "Environment" of the problem, and select the "chromosome" that is more adaptable to the environment based on the survival principle of the fittest for replication, and then cross, the mutation process produces a new generation of "chromosomes" that are more adaptable to the environment. In this way, the evolution from generation to generation will eventually converge to a "chromosome" most suitable for the Environment, which is the optimal solution to the problem.

N binary strings (I = 1, 2 ,..., N) constitutes the initial solution group of the genetic algorithm, also known as the initial group. In each string, each binary is the gene of an individual chromosome. According to evolutionary terms, there are three types of group operations:

(1). Selection)

This is to select an individual that is more adaptable to the environment from the group. These selected individuals are used to breed the next generation. This operation is also called reproduction ). When selecting an individual for breeding the next generation, the reproduction volume is determined based on the individual's adaptability to the environment. Therefore, it is sometimes called differential reproduction ).

(2). Crossover (crossover)

This is to exchange genes at the same position of two different individuals in the selection of individuals for breeding the next generation, thus generating new individuals.

(3) mutation)

This is where some genes in the selected individual are converted in an external direction. In a string of Bi, if a gene is 1, it is changed to 0 when the mutation is generated.

Features of Genetic Algorithms

1. Genetic Algorithms start from the centralized solution, rather than from a single solution. This is a big difference between genetic algorithms and traditional optimization algorithms. Traditional optimization algorithms are used to obtain the optimal solution by iteration from a single initial value. It is easy to mistakenly import the local optimal solution. The genetic algorithm starts searching from the string set and has a large coverage, which facilitates global optimization.
2. When genetic algorithms are used to solve specific problems, there is very little information and it is easy to form a general algorithm program. Because the genetic algorithm uses the adaptive value information for search, the problem derivative and other information directly related to the problem are not required. Genetic algorithms only need to adapt to common information such as value and string encoding, so they can handle almost any problem.
3. Genetic algorithms have a strong fault tolerance capability. The initial string set of a genetic algorithm carries a large amount of information far from the optimal solution. Through selection, crossover, and mutation operations, you can quickly exclude strings that differ greatly from the optimal solution; this is a strong filtering process and a parallel filtering mechanism. Therefore, genetic algorithms have high fault tolerance capabilities.
4. Selection, crossover, and Mutation in genetic algorithms are random operations, rather than definite precise rules. This shows that the genetic algorithm uses a random method to search for the optimal solution. The selection reflects the approaching of the optimal solution, the crossover reflects the generation of the optimal solution, and the variation reflects the coverage of the global optimal solution.

Neural network algorithm

Artificial Neural Network (. n. n .) it is an engineering system that simulates its structure and intelligent behavior based on its understanding of the structure and operation wit of human brain. As early as the beginning of the 1940s S, psychologists McCulloch and mathematician Pitts proposed the first mathematical model of artificial neural networks, which created the era of neurology theory research. Later, scholars such as F. Rosenblatt, widrow, and square, J. J. Local, and so on successively proposed sensing models, enabling the vigorous development of artificial neural network technology.

The basic structure of the nervous system is neuron (neural cell), which is the basic unit for processing information transfer between various parts of the human body. According to the results of neurobiologist research, a human brain generally has ???? Neurons. Each neuron is composed of a cell body, an axon connecting other neurons, and some other short branch extending out-of-the-box, called a token. the function of the axon is to transmit the output signal (EXCITED) of the neuron to another neuron. Many nerve ends at its end allow excitement to be transmitted to multiple neurons at the same time. The function of DCS is to accept excitement from other neurons. The neuron cell body simply processes all received signals (such as weighted summation, that is, all input signals are taken into account and each signal is valued differently. The part of the neuron that is connected to the neural peripheral of another neuron is called the syn.

Working principle of Neural Networks

Artificial Neural Networks must learn according to certain learning rules before they can work. The artificial neural network is used as an example to describe how to recognize the handwritten letters "A" and "B", specifying that when "a" is input into the network, "1" should be output ", when the input is "B", the output is "0 ". Therefore, the principle of network learning should be: if the network makes a wrong decision, the network learning should reduce the possibility of making the same mistake next time. First, assign random values in the range (0, 1) to the connection weights of the network, and input the image mode corresponding to "A" to the network, the Network calculates the weighted sum of the input mode, compares it with the threshold, and then performs non-linear operations to obtain the network output. In this case, the probability of the network output being "1" and "0" is 50%, that is, it is completely random. In this case, if the output is "1" (the result is correct), the connection weight is increased so that the network can still make a correct judgment when "A" mode is entered again. If the output is "0" (that is, the result is incorrect), adjust the network connection weight to reduce the weighted value of the integrated input, the goal is to reduce the possibility of making the same mistake when the network Encounters "A" Mode input next time. After inputting several handwritten letters "A" and "B" to the network, the network learns several times based on the above learning methods, the correct rate of network judgment will be greatly improved. This indicates that the network has successfully learned these two modes, and it has stored these two modes in the connection weights of the network. When the network encounters any mode again, it can make rapid and accurate judgment and recognition. Generally, the more neurons contained in the network, the more patterns it can remember and recognize.

Features of Artificial Neural Networks

Artificial Neural Network is a system composed of a large number of neurons that are widely interconnected. Its structural characteristics determine that the artificial neural network has the capability of high-speed information processing. What about every neuron in the human brain ?? A total of about a person's brain is formed by the size of the individual's DCS and corresponding syncs ?? . In terms of neural networks, what does the human brain possess ?? Storage potential for interconnection. Although each neuron has simple operation functions and low signal transmission rate (about 100 times/second), due to its extremely Parallel Interconnection function, in the end, the brain of an ordinary person can complete the tasks that the current computer requires at least 1 billion processing steps in about one second.

The knowledge storage capacity of artificial neural networks is large. In neural networks, knowledge and information are stored as distributed physical connections between neurons. It discretely represents and stores the neurons in the entire network and their connections. Each neuron and its line represent only a part of information, rather than a complete concept. A specific concept and knowledge can be expressed only through the distributed synthesis of each neuron.

Due to the large number of neurons in the artificial neural network and the huge storage capacity of the entire network, it has a strong ability to process uncertain information. Even if the input information is incomplete, inaccurate, or ambiguous, neural networks can still associate the full image of things in memory. As long as the input mode is close to the training sample, the system can give the correct inference conclusion.

It is precisely because of the structural characteristics of the artificial neural network and the distributed features of its information storage that it has another notable advantage over other judgment and recognition systems, such as expert systems. The biological neural network will not lose its memory of the original model due to the loss of individual neurons. The most powerful proof is that when a person's brain is slightly damaged by an accident, it will not lose all the memories of the original things. Artificial Neural Networks are similar. For some reason, whether it is the hardware implementation of the network or the failure of one or some neurons in the software implementation, the entire network can continue to work.

Artificial Neural Networks are non-linear processing units. Only when the Comprehensive processing of all input signals by neurons exceeds a certain threshold value can a single signal be output. Therefore, neural networks are highly nonlinear ultra-large scale continuous time Dynamics Systems. It breaks through the limitations of traditional digital electronic computers based on linear processing, marking a great leap in the smart information processing capability and the ability to simulate the intelligent behavior of the human brain.