Differential Evolution (de) Differential Evolution Algorithm

Differential Evolution (de) for continuous function optimization (an algorithm by kenth price and Rainer Storn) Table of Contents

• History
• Basics
• Practical advice
• De-Applet
• Java code
• C code
• Matlab code
• SCILAB code
• Psather code
• C ++ code
• FORTRAN90 code
• Python code
• LabVIEW code
• R code
• Pascal code
• Demo-Programs
• Applications of de
• Commercial SW using de
• Other links, books ,...
• Conferences
• Contributors to this page

History

Differential Evolution grew out of Ken Price's attempts to solve the Chebychev polynomial fitting problem that had been posed to him by Rainer Storn. A breakthrough happened, when Ken came up with the idea of using vector differences for perturbing the vector population. since this seminal idea a lively discussion between Ken and Rainer and endless ruminations and computer simulations on both parts yielded into substantial improvements which make de the versatile and robust tool it is today. the "de community" has been growing since the early de years of 1994-1996 and ever more researchers are working on and with de. those scientists who contributed actively to this homepage are listed at the bottom in alphabetical order. it is the strong wish of Ken and Rainer that De will be developed further by scientists around the world and that de may improve to help more users in their daily work. this wish is the reason why de has not been patented in any way.

Basics

De is a very simple population based, stochastic function minimizer which is very powerful at the same time. de managed to finish 3rd at the First International Contest on Evolutionary Computation (1 sticeo) which was held in Nagoya, May 1996. de turned out to be the best genetic type of algorithm for solving the real-valued test function suite of the 1st iceo (the first two places were given to non-ga type algorithms which are not universally applicable but solved the test-problems faster than de ). the crucial idea behind de is a scheme for generating trial parameter vectors. basically, de adds the Weighted Difference between two population vectors to a third vector. this way no separate probability distribution has to be used which makes the scheme completely self-organizing. for further details see the literature page.

Practical advice

If you are going to optimize your own objective function with De, try the following settings for the input file first: choose method e.g. de/Rand/1/exp, set the number of parents np to 10 times the number of parameters, Select Weighting Factor f = 0.8 and crossover constant Cr = 0.9. make sure that you initialize your parameter vectors by exploiting their full numerical range, I. e. if a parameter is allowed to exhibit values in the range [-100,100] it's a good idea to pick the initial values from this range instead of unnecessarily restricting diversity. if you experience misconvergence you usually have to increase the value for NP, but often you only have to adjust F to be a little lower or higher than 0.8. if you increase NP and simultaneously lower f a little, convergence is more likely to occur but generally takes longer, I. e. de is getting more robust (there is always a convergence speed/robustness tradeoff ).

De is much more sensitive to the choice of F than it is to the choice of CR. cr is more like a fine tuning element. high values of Cr like Cr = 1 give faster convergence if convergence occurs. sometimes, however, you have to go down as much as Cr = 0 to make de robust enough for a particle problem. if you choose binomial crossover like, de/Rand/1/bin, Cr is usually higher than in the exponential crossover variant (in this example de/Rand/1/exp ). still, F and Cr are both generally in the range [0.5, 1.] for most problems we 've encountered. keep in mind that different problems usually require different settings for NP, F and Cr (have a look into the different papers to get a feeling for the settings ). if you still get misconvergence you might want to try a different method. we mostly use de/Rand/1 /... or de/best/2 /... (For the latter F = 0.5 is the standard choice ). the crossover method is not so important although Ken price claims that binomial is never worse than exponential. in case of misconvergence also check your choice of objective function. there might be a better one to describe your problem. any knowledge that you have about the problem shocould be worked into the objective function. A good objective function can make all the difference.

De-Applet

De is demonstrated here by example of the polynomial fitting problem (see techreport by Rainer and Ken ). the task is to fit a polynomial of a certain degree into the tolerance Scheme indicated by the red lines. at X-values +/-1.2 The polynomial must be higher than Tk (x) with TK (x) being the Chebychev polynomial of Degree k. the constraint at +/-1.2 can't be seen in the animation as it is fairly L Arge (~ 5.9 for T4 (X) and ~ 72.661 for T8 (x )). the cost function to be minimized is the sum of squared errors. depending on the de-variant the T4 problems takes several thousand function evaluations to be solved. t8 needs about five to ten times as much. the applet was programmed by Mikal Keenan (DE) and Rainer Storn (GUI ). fetch the source code deapplet1.java and deapplet1.html to see what we have done.

Java code

In order to make it easy to do de Optimization on every platform with the support of graphics a Java application of a de optimizer has been written. the current version (1.0.3) runs now both in JDK 1.0.2 as well as JDK 1.1.7. the Code has been enriched with the magnificent plow.capabilities of ptplot, written by Edward. lee and Christopher Hylands from the University of California, Berkeley. now it is possible to zoom in and out of the plots as the optimization is ongoing. a corrected version of the Code documentation is also available.

C code

The latest C code from the book differential evolution-a practical approach to global optimization is available here by courtesy of Springer publisher. the Code has been written with ms visual c ++ V5.0 and also contains some MS Windows based graphics routines (see example plot below ). in order to make the code simple to port to other operating systems it contains the compiler switch do_plow.which has to be turned off in order to turn the code into a console application. the Code uses Hungarian prefix notation to make the data types used more explicit and hence the code hopefully more clear.

 

There are two ways to work with dewin:

1) As a Windows application under Microsoft Windows? For example, in Visual C ++ you have to generate a workspace of type "win32application ". the compiler switch "do_plotid" shocould be defined. then insert all. CPP files into the project and go to build | rebuild all to obtain an executable code. note that the ploadfeatures are working only for the Win32 environment.

2) as a Windows console application under Microsoft Windows? Generate a workspace of type "win32consoleapplication ". make sure the compiler switch "do_plotid" is undefined. then insert all. CPP files into the project and go to build | rebuild all to obtain an executable code. with this source code configuration the code shocould be portable to other operation systems with only minor modifications.

In contrast to older versions of de-C-code (see below) This one has been enhanced to support incorporation of constraints (bounds, equality, inequality ). more details and more sample functions for the code can be found in the above book.

Another C-version provided by Peter Turney updated an older version of the C-code above to make it more compatible to Visual C ++ 6.0 and also to UNIX-based c-compilers.

Matlab code

The latest Matlab code from the book differential evolution-a practical approach to global optimization is available here by courtesy of Springer publisher. see an example plot below.

The Code is designed to in‑ate bounds, in‑ity, and equality constraints. the above book contains a detailed explanation of the Code and some more examples in the CD companion. however, the Code for download here contains the main engine in its full functionality for experimentation. special attention has been given to coding conventions in Hungarian prefix notation to make the programs easier to understand and use.

The script file rundeopt. M (run de optimization) is the main control file in the MATLAB environment. plotting can be turned off by setting the variable I _ploable = 0 in rundeopt. m. per default this variable is set to 1.

This is some older de-code in MATLAB which may still be interesting to some users. the M-files necessary to run the de-demo in Matlab are the demo shell dedemov. m, the actual de-minimizer devec. m and the plot-supporting function xplt. m. if you want to use the de-strategy in MATLAB for your own purposes it is better to use devec3.m which minimizes an externally defined objective function, e.g. rosen. m. use the help function in MATLAB to obtain the details on how to use devec3.m. two helper files, run1.m and run2.m show you how to work with devec3.m more conveniently.

SCILAB code

There is also a SCILAB version of De written by Walter Di Carlo and Helmut jarausch. SCILAB is a public domain clone of MATLAB

Psather code

De is perfectly suited for parallel computation which already has found an implementation in psather, a parallel object oriented language being developed at FLAC. The code was developed by cldio Fleiner.

C ++ code

You can also get a completely object oriented C ++ version written by Lester Godwin in Visual C ++ 5.0. It is available as devcpp.zip.

FORTRAN90 code

Another contribution is a FORTRAN90 version of de developed by prof. Feng-sheng Wang and his students.

Python code

A recent addition is a Python version of de developed by James R. Phillips.

LabVIEW code

The latest contribution is a LABVIEW version of de developed by Franz Joseph F ahlers.

R code

Also available is now the R version of de developed by David ardia.

Pascal code

The Pascal code of De has been contributed by Hubert geldon and Piotr A. gauden.

Demo-Programs

If you happen to have access to an DOS-PC and have the graphics driver egavga. BGI from Borland's c ++ Library (Version 3.1) you have all the requirements to get two little demo programs running which show de at work. these programs are dedemo.exe and dedemo2.exe (fetch them via Shift + "click" in Netscape) along with their data files demo. dat and demo2.dat. after placing these files in the pertinent directory, all you have to do is type

Dedemo demo. dat

Which shows you the polynomial fitting problem, or

Dedemo2 demo2.dat

Which visualizes de working on rosenbrock's saddle.

You can even manipulate. DAT files to experiment with different parameter values. note however, that the demo programs are not crash proof, I. e. if you exceed certain limits for the parameters the programs will crash. so it's best to change e.g. only the number of parents NP <200, weighting constant f ex [] and crossing over factor Cr ex [] but not to change the number of parameters d. all the other parameters might be safely changed if the parameters are> 0. but remember: "Everything that is free comes with no guarantee ".

Applications of de

A selection of scientific or your cial applications of De which are accessible by a URL are listed below. it has to be noted that more than ten years from the inception of De this list is impossible to keep complete. A simple Google search for the keyword "differential evolution" shows that the selection below provides just a few applications of de.

1) multiprocessor synthesis.

2) neural network learning.

3) chrystallographic characterization.

4) Synthesis of modulators.

5) heat transfer parameter estimation in a trickle bed reactor.

6) scenario-Integrated Optimization of dynamic systems.

7) Optimal Design of shell-and-tube heat exchangers.

8) Optimization of an alkylation reaction.

9) Optimization of thermal cracker operation.

10) Optimization of non-linear chemical processes.

11) optimum planning of cropping patterns.

12) Optimization of water pumping system.

13) Optimal Design of Gas Transmission Network.

14) Differential Evolution for multi-objective optimization physiochemistry of carbon materials.

15)

16) radio network design.

17) reflectivity curve simulation.

Commercial SW using de

1) built in optimizer in Mathematica's function nminimize (since version 4.2 ).

2) MATLAB's GA toolbox contains a variant of de.

3) digital filter design.

4) diffraction grating design.

5) atomicity market simulation.

6) auto2fit.

7) LMS Virtual Lab optimization.

8) Optimization of optical systems.

9) finite element design.

Other links, books ,...

Louni lampinen's de-bibliography.

The differential evolution society.

 

 

Available now!

Order the book via or

 

The book "differential evolution-a practical approach to global optimization" by Ken price, Rainer Storn, And Jouni lampinen (Springer, ISBN: 3-540-20950-6) provides the latest findings concerning de. de is a practical approach to global numerical optimization that is easy to understand, simple to implement, reliable, and fast. packed with birth strations, computer code, new insights, and practical advice, this volume has es Differential Evolution in both principle and practice. it is a valuable resource for professionals needing a proven optimizer. A companion CD has des the latest de-based optimization software in several programming languages ages (C, C ++, Matlab, Mathematica, Java, FORTRAN90, SCILAB, LabVIEW ). the C and Matlab versions are enhanced with code for constrained and multiobjective optimization. the C, Matlab, Mathematica, and Java versions come with animated graphics support. see the table of contents for more details.

Order

Another book-"New Optimization Techniques in engineering"-has been published by Springer in 2004 and contains information about the recent developments of differential evolution, especially concerning applications in the specified problem domain.

Order.

The book "New Ideas in optimization" by McGraw-Hill contains a large section about differential evolution. it is demonstrated that De is not only a powerful function optimizer but can also be used for Mixed Integer Programming and game strategy optimization. other topics are Ant Colony Optimization, immune system methods, memetic algorithms, particle swarms (which is similiar to differential evolution), and others.

Conferences

• Special session on the differential evolution algorithm at WCCI- '06 in Vancouver, Canada, July 16-21 by Uday Chakraborty

Contributors to this page

Franz Joseph F ahlers, Walter Di Carlo, cldio Fleiner, Lester Godwin, Mick (Mikal Keenan), rituraj Deb Nath, Arnold noraier, James R. phillips, kenth price, Rainer Storn, Peter Turney, Feng-sheng Wang, Jim Van Zandt, Hubert geldon, Piotr. gauden

.

Rainer Storn's home page