Concept of Genetic Algorithm

The concept of genetic algorithm is an iterative adaptive probabilistic search algorithm based on natural selection and natural genetics mechanisms. It was proposed by Professor Holland in 1975.

Biological evolution is a wonderful optimization process. It produces excellent species that adapt to environmental changes by selecting elimination, sudden variations, genetic inheritance, and other patterns. Genetic algorithms are a global optimization algorithm inspired by biological evolution.

The concept of genetic algorithm was first proposed by Bagley J. D in 1967, and the systematic research on the theory and method of genetic algorithm started in 1975. Work It is implemented by J. H. Holland of Michigan University. At that time, the main purpose was to illustrate the adaptive process of natural and artificial systems.

Genetic algorithms (GA) are essentially a direct search method independent of specific problems. Genetic Algorithms in pattern recognition, Neural Networks, image processing, machine learning, industrial optimization control, adaptive control, biological science, and social science are all applied. In AI research, people now think that "genetic algorithms, adaptive systems, cellular automation, and chaos theory, like artificial intelligence, are key technologies that have a significant impact on the computing technology of the next decade ".

3.2.1 basic concepts of Genetic Algorithms

The idea of genetic algorithms is based on Darwin's evolution and Mendel's genetics.

The most important thing about Darwin's evolution is the survival principle of the fittest. It believes that each species is increasingly adapting to the environment in the development. The basic features of each individual in the species are inherited by the offspring, but the offspring will have some new changes different from those of the parent generation. When the environment changes, only the individual characteristics that bear adapts to the environment can be preserved.

The most important thing about Mendel genetics is genetic genetics. It believes that genes exist in cells as passwords and are contained in chromosomes as genes. Each gene has a special location and controls a specific nature; therefore, each individual produced by each gene has a certain degree of adaptability to the environment. Mutation and hybridization can generate offspring that are more adaptable to the environment. The adaptive genetic structure can be preserved after the natural elimination of optimization and removal.

Genetic algorithms are a direct search optimization method based on evolutionary and genetic mechanisms. Therefore, various evolutionary and genetic concepts need to be used in this algorithm. These concepts are as follows:

I. String)

It is an individual form. It is a binary string in the algorithm and corresponds to the chromosome in genetics ).

2. Population)

A set of individuals is called a group, and a string is a group element.

3. Population Size)

The number of individuals in a group is called the size of the group.

Iv. Gene)

A gene is an element in a string that represents individual characteristics. For example, if there is a string of S = 1011, the four elements 1, 0, 1, and 1 are called genes respectively. Their values are called alletes ).

5. Gene position)

The location of a gene in a string is also called a genetic location. The genetic position is calculated from the left to the right of the string. For example, in the string S = 1101, the genetic position of 0 is 3. The genetic Location corresponds to the location in the genetics (loocus ).

Vi. Gene feature)

When a string is used to represent an integer, the feature value of the gene is the same as that of the binary number. For example, in string S = 1011, the feature value of 1 in position 3 is 2; 1 In gene position 1, and its genetic feature value is 8.

VII. String Structure space SS

A set of strings composed of any combination of genes. Genetic operations are performed in structural spaces. String Structure space corresponds to the set of genotype in genetics.

8. Parameter Space sp

This is a ing of string spaces in physical systems. It corresponds to a set of phenotype in genetics.

9. Non-linear

It corresponds to epistasis in genetics)

10. Fitness)

Indicates the adaptability of an individual to the environment.

There are also some other concepts of genetic algorithms, which are further described when introducing the principles and execution processes of genetic algorithms.

3.2.2 principles of Genetic Algorithms

Genetic Algorithm GA expresses the solution of the problem as a "chromosome", which is also a binary encoded string. In addition, a group of "chromosomes" is given before the genetic algorithm is executed, that is, the hypothetical solution. Then, place these assumptions in the "Environment" of the problem, and select the "chromosome" that is more adaptable to the environment based on the survival principle of the fittest for replication, and then cross, the mutation process produces a new generation of "chromosomes" that are more adaptable to the environment. In this way, the evolution from generation to generation will eventually converge to a "chromosome" most suitable for the Environment, which is the optimal solution to the problem.

I. Objective of Genetic Algorithms

The typical genetic algorithm CGA (Canonical Genetic Algorithm) is usually used to solve the following static optimization problems:

Consider the binary encoding of a group of Bi, I = 1, 2 ,..., N; yes

Bi ε {0, 1} l (3-84)

Given the target function F, F (BI), and

0 <F (BI) <∞

At the same time
F (BI) =f (Bi + 1)

Satisfying the following formula

Max {f (BI) | BI ε {0, 1} l} (3-85)

.

Obviously, genetic algorithms are an optimization method. By means of evolution and genetic mechanism, they constantly evolve from the given original solution group to produce new solutions, finally, it converges to a specific string Bi to obtain the optimal solution.

Ii. Basic principles of Genetic Algorithms

N binary strings (I = 1, 2 ,..., N) constitutes the initial solution group of the genetic algorithm, also known as the initial group. In each string, each binary is the gene of an individual chromosome. According to evolutionary terms, there are three types of group operations:

1. Selection)

This is to select an individual that is more adaptable to the environment from the group. These selected individuals are used to breed the next generation. This operation is also called reproduction ). When selecting an individual for breeding the next generation, the reproduction volume is determined based on the individual's adaptability to the environment. Therefore, it is sometimes called differential reproduction ).

2. Crossover)

This is to exchange genes at the same position of two different individuals in the selection of individuals for breeding the next generation, thus generating new individuals.

3. Mutation)

This is where some genes in the selected individual are converted in an external direction. In a string of Bi, if a gene is 1, it is changed to 0 when the mutation is generated.

The principle of genetic algorithm is described as follows:

Choose an intial population

Determine the fitness of each individual

Perform Selection

Repeat

Perform crossover

Perform mutation

Determine the fitness of each individual

Perform Selection

Until some stopping criterion applies

An ending criterion referred to here generally refers to an individual's fitness reaching a given threshold value; or the individual's fitness change rate is zero.

Iii. Steps and significance of Genetic Algorithms

1. Initialization

Select a group, that is, select a string or an individual set of Bi, I =,... n. This initial group is the set of hypothetical solutions. Generally, n = 30-160 is used.

Bi, I = 1, 2,... N, which is a set of strings or individuals, is usually produced in a random way. The optimal solution of the problem will be obtained through the evolution of these initial assumptions.

2. Select

Select the next generation of individuals based on the survival principle of the fittest. When selecting, the adaptability is used as the selection principle. The fitness criteria reflect the survival of the fittest, and the natural law of the incompatibility.

Given the target function F, F (BI) is called the fitness of individual bi. To

... (3-86)

The number of times that Bi is selected as the next generation of individuals.

Obviously, we can see from formula (3-86:

(1) Individuals with higher fitness have a large number of next generations.

(2) Individuals with smaller fitness have fewer breeding generations and are even eliminated.

In this way, there will be future generations with strong adaptability to the environment. To solve the problem, select an intermediate solution that is closer to the optimal solution.

The methods are as follows:

Fitness Ratio Method
Expectation value method
Ranking Method
Essence retention method
3. Crossover
For the selected individuals used to breed the next generation, randomly select the same position of the two individuals, according to the crossover probability P. Exchange is performed at the selected position. This process reflects the random information exchange, which aims to generate a new genetic combination, that is, to generate a new individual. Single Point Crossover or multi-point crossover can be implemented.

For example, an individual

S1 = 100101

S2. = 010111

Select the three places on the left to perform cross operations.

S1 = 010101

S2. = 100111

Generally, cross-Nov slaughter p. The value ranges from 0.25 to 0.75.

4. Variation

According to the principle of genetic variation in biological inheritance, the mutation probability Pm is used to mutate certain bits of some individuals. During the variation, the corresponding bitwise of the string that executes the variation is reversed, that is, 1 is changed to 0, and 0 is changed to 1. The probability of mutation PM is the same as that of biological variation. Therefore, the PM value is smaller, generally between 0.01 and 0.2.

For example, an individual S = 101011.

When the gene at location 1 and 4 is mutated

S' = 001111

Variation alone cannot be used in solving problems. However, it ensures that the algorithm process will not generate a single group that cannot be evolved. Because when all individuals are the same, crossover cannot generate new individuals. In this case, only variations can generate new ones. That is to say, variations increase the global optimization feature.

5. Global Optimal Convergence (convergence to the global optimum)

When the fitness of the optimal individual reaches the given threshold, or the fitness of the optimal individual and the fitness of the Group no longer increase, the iteration process of the algorithm converges and the algorithm ends. Otherwise, replace the previous generation with the new generation group obtained by selection, crossover, and mutation, and return to step 1, that is, select the operation to continue the loop.

Figure 3-7 shows the execution process of the genetic algorithm.

3.2.3 Application of Genetic Algorithms

Genetic algorithms are applied in many fields. Network From the perspective of research, we are most concerned with the application of genetic algorithms in neural networks. In the application of genetic algorithms, the characteristics and key issues should be clarified first to gain an in-depth understanding of this algorithm, flexible application, and further research and development.

I. Features of Genetic Algorithms

1. The genetic algorithm starts from the centralized solution of the problem, rather than from a single solution.

This is a big difference between genetic algorithms and traditional optimization algorithms. Traditional optimization algorithms are used to obtain the optimal solution by iteration from a single initial value. It is easy to mistakenly import the local optimal solution. The genetic algorithm starts searching from the string set and has a large coverage, which facilitates global optimization.

2. There is very little information about specific problems used by Genetic Algorithms for solving, and it is easy to form a general algorithm program.

Because the genetic algorithm uses the adaptive value information for search, the problem derivative and other information directly related to the problem are not required. Genetic algorithms only need to adapt to common information such as value and string encoding, so they can handle almost any problem.

3. Strong Fault Tolerance of Genetic Algorithms

The initial string set of the genetic algorithm itself carries a large amount of information far from the optimal solution. Through the selection, crossover, and mutation operations, it is possible to quickly exclude strings that differ greatly from the optimal solution. This is a strong filtering process; it is also a parallel filtering mechanism. Therefore, genetic algorithms have high fault tolerance capabilities.

4. Selection, crossover, and Mutation in genetic algorithms are random operations, rather than definite precise rules.

This shows that the genetic algorithm uses a random method to search for the optimal solution. The selection reflects the approaching of the optimal solution, the crossover reflects the generation of the optimal solution, and the variation reflects the coverage of the global optimal solution.

5. implicit concurrency of Genetic Algorithms

The basic theory of genetic algorithms is the schema theorem. Its Related content is as follows:

(1) Schema Concept

A genetic string is represented by a symbolic set {0, 1, *}. It is called an expression. * Can be 0 or 1. For example, H = 1X x 0 x is a schema.

(2) Order and length of the schema

The numbers of 0 and 1 in the schema are called the level of the schema and expressed in 0 (h. The distance between the 1st-bit number and the last-digit number in the schema is called the schema length, which is expressed by Delta (h. For schema H = 1X x0x X, 0 (H) = 2, Delta (H) = 4.

(3) Holand schema Theorem

Low level, short-length schemas will be Index Regular Increase. When the Group size is N, the number of schemas processed by each generation is 0 (N3 ).

The processing capability of genetic algorithms is called implicit parallelism (implicit parallelism ). It shows that genetic algorithms have inherent characteristics of parallel processing.

II. Key to the Application of Genetic Algorithms

The following three key problems exist in the application of genetic algorithms:

1. encoding of strings

This is essentially a problem code. Generally, the parameters of the problem are encoded in binary format to form a substring. Then, the substrings are spliced to form a "chromosome" string. String Length and encoding form have a great impact on algorithm convergence.

2. Determine the adaptive Function

Fitness Function, also known as object function, is a measurement function of the problem solving quality. It is often referred to as the "Environment" of the problem ". The problematic model function can be used as an object function, but it must be constructed separately.

3. Genetic Algorithm parameter settings

There are three genetic algorithm parameters, namely group size N, crossover probability Pc, and mutation probability Pm.

The group size N is too small to find the optimal solution. If the group size is too large, the convergence time increases. N = 30-160. The crossover probability Pc is too small to search forward. If it is too large, it is easy to damage the structure of the high adaptive value. Generally, Pc = 0.25-0.75 is used. The mutation probability Pm is too small to generate a new genetic structure, and the genetic algorithm becomes a simple random search. Generally, PM = 0.01-0.2 is used.

Iii. Application of Genetic Algorithms in Neural Networks

The Application of Genetic Algorithms in neural networks is mainly reflected in three aspects: Network Learning Network Structure Design and network analysis.

1. Application of Genetic Algorithms in Network Learning

In neural networks, genetic algorithms can be used for network learning. At this time, it works in two ways

(1) Optimization of learning rules

Genetic algorithms are used to automatically optimize neural network learning rules to increase the learning rate.

(2) network weight coefficient Optimization

The global optimization and implicit parallelism of genetic algorithms are used to increase the speed of Weight Coefficient optimization.

2. Application of Genetic Algorithms in Network Design

To use genetic algorithms to design an excellent neural network structure, we must first solve the encoding problem of the network structure, and then obtain the optimal structure through selection, crossover, and mutation operations. There are three main encoding methods:

(1) Direct Encoding

The neural network structure is directly represented by binary strings. In genetic algorithms, the "chromosome" is essentially a ing relationship with the neural network. The network is optimized by optimizing the chromosome.

(2) parameterized Encoding

Parameterized encoding uses Abstract encoding, including network layers, number of neurons at each layer, interconnection modes at each layer, and other information. Generally, the optimized chromosome after evolution is analyzed, and then the network structure is generated.

(3) reproduction and growth method

This method does not directly encode the structure of the neural network in the chromosome, but encodes some simple growth syntax rules into the chromosome. Then, the growth syntax rules are constantly changed by the genetic algorithm, and finally a neural network suitable for the problem is generated. This method is consistent with natural biological growth and evolution.

3. Application of Genetic Algorithms in Network Analysis

Genetic algorithms can be used to analyze neural networks. Due to its distributed storage and other features, it is generally difficult for a neural network to directly understand its functions from its topological structure. Genetic algorithms can analyze functions, properties, and states of neural networks.

Although genetic algorithms can be applied in a variety of fields, they also show their potential and broad prospects. However, there are still many problems that need to be studied and there are still some shortcomings. First, the convergence speed decreases when there are many variables and the value range is large or there is no given range. Second, the optimal solution can be found, but the location of the most disturbing solution cannot be accurately determined. Finally, there is no quantitative method for selecting parameters of the genetic algorithm. The basic mathematical theory of Genetic Algorithms needs to be further studied, the advantages and disadvantages of genetic algorithms and other optimization technologies need to be proved theoretically, and hardware-based genetic algorithms need to be studied; and general programming and forms of genetic algorithms.