convex sets, convex functions, convex optimization problems, linear programming, two-time planning, two constraints two-time planning, semi-definite planning

There is no systematic study of mathematical optimization, but these tools and techniques are commonly used in machine learning, and the most common optimization in machine learning is convex optimization, which can refer to Ng's teaching materials: http://cs229.stanford.edu/section/ Cs229-cvxopt.pdf, from which we can understand some of the convex optimization concepts, such as convex set, convex function, convex optimization problem, linear programming, two times planning, two constraints two times planning, semi-positive definite planning, so that the convex optimization problem has a preliminary understanding. Here are a few notes of important related concepts.

　　A convex set is defined as:

　　

Its geometric meaning is: if the points on any 2 element lines in Set C are also in Set C, C is a convex set. It is as follows:

　　

Common convex sets are:

n-dimensional real space; sets of norm-constrained forms; affine subspace; intersection of convex sets; n-dimensional semi-definite matrices; these can all be proved by the definition of convex sets.

　　The convex function is defined as:

　　

Its geometric meaning is expressed as the value of the function on any two-point line is greater than the function value at the corresponding independent variable, as follows:

　　

The first order necessary and sufficient conditions for convex functions are:

　　

which requires the first order of F can be micro.

The second order condition is:

　　

which requires F differentiable micro, that the second derivative must be greater than 0 is the convex function.

Common convex functions are: exponential function family, nonnegative logarithm function, affine function, quadratic function, common norm function, convex function nonnegative weighted sum and so on. These can be proved by the above 2 necessary and sufficient conditions or definitions.

　　The convex optimization problem (OPT) is defined as:

　　

　　　　Convex optimization , or called convex optimization, convex minimization. The problem of minimizing convex functions in convex sets is studied. it requires that the objective function is a convex function , and that the set of variables belongs to the optimization problem of convex sets. Or the objective function is a convex function, the constraint function of the variable is the convex function (when inequality is constrained ), or the affine function (equality constraints ).

　　For the convex optimization problem, the local optimal solution is the global optimal solution.

Common convex optimization problems include the following:

　　linear Programming (LP): The problem is to optimize the following equation:

One of the uncommon strange symbols is that the symbol is less than or equal to the element, followed by similar symbols.

　　Two-time planning (QP): The problem is to optimize the following:

　　Two-time constrained two-time planning (QCQP): The problem is to optimize the following equation:

　　semi-positive definite plan (SDP): The problem is to optimize the following equation:

　　

According to the article, SDP is widely used in the field of machine learning and has recently been popular, but I don't seem to have touched it too much.

　　Resources:

Http://cs229.stanford.edu/section/cs229-cvxopt.pdf

Turn from:

Machine Learning & Data Mining Note _15 (some simple concepts on convex optimization)

Convex sets, convex functions, convex optimization problems, linear programming, two-time planning, two constraints two-time planning, semi-definite planning