Optimization algorithms everywhere

One: Cause

(0) Optimization algorithm (Optimization algorithm), that is, the optimal value problem of the objective function; How to evaluate the value of your current solution is optimal? This requires the construction of the evaluation function, how to update from the current location (solution) to the new search space? This requires a definition of the form of the transformation.

(1) Optimization algorithm everywhere-- real life of material deployment, a certain means of production how to get the maximum output, a certain investment how to get the best income, etc., can be transformed into the most optimization problem solving; even a travel plan in our normal life can not be separated from the optimization problem: three-day holiday, whether I go to Tibet or to Beijing or to Shanghai, I have to be subject to the duration of the holidays, as well as the tolls, lodging fees, as well as local customs, hobbies, preferences and other factors constraints, In the end we will get a solution in Zhejiang, in fact, is an optimal solution!

(2) optimization algorithm: Quasi-Newton, Newton, gradient, genetic, ant Colony, simulated annealing, particle swarm optimization algorithm, is in a given search space, as fast as possible to search the global optimal value.

II: Optimization of the classification

(0) Simple linear nonlinearity of the ancient optimization algorithm:

Quasi-Newton, Newton, gradient mountain climbing algorithm------easy to cause local optimal solution

(1) solve the local optimal, and begin to combine the biological information theory

Genetic (GA genetic algorithm), simulated annealing (simulated annealing)

(2) Knowledge of swarm intelligence--usually regarded as a kind of cluster intelligence (Swarm Intelligence, SI) . It can be incorporated into multi-agent optimization systems (multiagent optimization system, MAOS)

ant Colony (ant), particle swarm optimization (PSO particle sworm optimization)

(3) According to conditional restriction and divided into constrained optimization problems and unconditional optimization problems

Three: Optimized applications

(0) LR solve the problem: 1 ) Logistic function (or called Sigmoid function) S type function; 2 the establishment of the objective function is transformed into the optimal solution. -- which translates into optimization problems; 3 )

The optimization algorithm (Quasi-Newton, Newton, gradient, genetic, ant Colony, simulated annealing, particle swarm algorithm, etc.) is used to solve the problem.

(1) Similar bpnetwork (using gradient descent method), the solution of the W-weighted parameters of the best weights, but also in the construction of a loss function J, so that it reaches the minimum value, the W weight at this time is the desired parameter.

(3) The material allocation in real life, how to get the maximum output of certain means of production, how to get the best income from the investment, the management of productive life and so on, can be transformed into the optimization problem

Optimization algorithms everywhere