Optimization theory and Method (Shing Sun Wenyu) Notes (i)

I. Overview　

In 1947, Dantzig proposed the simplex method for solving general linear programming problems. Now, the research on the theory of solving linear programming, nonlinear programming, stochastic programming, non-smooth programming, multiobjective programming, geometric programming, integer programming and other optimization problems is developing rapidly.

The general form of the optimization problem is:

　　　　　　　　　　　　　　　

X belongs to RN as constraint set or feasible domain, F (x) is the objective function, X belongs to RN is the decision variable. In particular, the constraint set X=RN, the optimization problem becomes unconstrained optimization problem:

For constrained optimization problems, it is often written as

　　　　　　

Here, E and I are the set of indicator sets of inequality constraints for equality constraints, and CI is a constraint function. When both the target function and the constraint function are linear functions. The problem is called linear programming. When at least one of the objective and constraint functions is a nonlinear function of variable x, the problem is called nonlinear programming.

In addition, the optimization is divided into several branches, such as Integer programming, dynamic programming, network planning, non-smooth planning, stochastic programming, geometric programming, multi-objective programming, according to the different decision variables, objective functions and requirements.

This paper focuses on solving unconstrained optimization problems and constrained optimization problems.

Definition of second, semi-norm and norm

Semi-norm:

Norm:

Three, vector norm and matrix norm

(1) Vector norm

(2) Matrix vector

(i) similar to the definition of vector norm, you can define the matrix norm. Set A is RNXN, and its induction matrix norm is defined as:

Two properties of the induced matrix norm:

(ii) Frobenius norm and other

Optimization theory and Method (Shing Sun Wenyu) Notes (i)