Numerical algorithm: The Golden section method and Fibonacci sequence method of one dimensional search method with unconstrained optimization

The objective function is a minimization optimization problem of single-valued function f:r->r, which is usually not encountered alone, it is often used as a part of multidimensional optimization problems, such as the estimation of the gradient descent method for each optimal iteration length.

The one-dimensional search method is solved by iterative method, which is different from the direct solution expression method of our human brain. The iterative algorithm starts from the initial search point x (0) and produces an iterative sequence X (1), X (2),...。 In section k=0,1,2, ... iteration, the next iteration point x (k+1) is constructed from the current iteration point X (k) and the target function f. Some algorithms may only need to use the value of the target function at the iteration point, while others may use the derivative F ' (x) of the target function, or even the second derivative F ' (x).

1, Problem Description:

If the unary single-valued function f:r->r is one-peak on the closed interval [a0,b0], the minimum point of f in the interval is solved.

2, Computer solution method:

The essence of the following method is: Interval compression (intercept) method, by shrinking the minimum point of the interval length to enough precision level to approximate the minimum point.

(1) Golden section method (the ratio of each interval interception is fixed value)

First step: initial interval [a0,b0], set intercept ratio R, first pick two intermediate points A1 and B1, they meet the symmetry requirements: a1-a0=b0-b1= R (b0-a0), obviously r<1/2, if F (A1) < F (B1), then the minimum point should be in the interval [A0,B1]; Otherwise, such as F (A1) >= F (B1), the minimum point should be in the interval [a1,b0].

Step two: After the first compression, the minimum interval changes to [A0,B1], or [a1,b0]. Assuming [A0,B1], the second selection of intermediate points A2 and B2 is required below. In order to make full use of the first selection point information and reduce the number of calculations, then the second selected point B2 can take the first pick point A1, so that only the B2 (i.e. A1) in the new zone between the symmetry point A2 and F (A2). To achieve such an idea, the following requirements must be met: R (b1-a0) = B1-b2=b1-a1 <=> r[b0-r (b0-a0)-a0]= (1-2R) (b0-a0) <=>r2-3r+1=0<=>1 -r= (sqrt (5)-1)/2 = 0.618, that is, the retention ratio after each interception is 0.618 so this method is called the Golden Section method. Therefore, after n steps, the interval length is (1-r) n= (0.618) n, which is the total compression ratio.

Pick A1 and B1 as the middle point

Assuming that [b1,b0] is intercepted for the first time, then A2 and B2 are selected on [A0,b1], in order to reduce the calculation to A1 instead of B2, just calculate A2.

(2) Fibonacci sequence method

In the process of the Golden Section compression, the parameter R always remains the same. If the Allow parameter r is constantly adjusted, such as the K-iteration using the parameter RK, the k+1 iteration uses the parameter rk+1, and so on. Also, in order to reduce the number of calculations, it is required to calculate the value of the target function only once per iteration (i.e., reusing the last selected midpoint information each time the intermediate point is selected). Then you can draw the following equation:

Rk+1 (1-RK) =1-2rk <=>rk+1=1-rk/(1-RK)

Obviously many of the sequence {rk},0= <RK<=1/2 meet the requirements of the above, such as the sequence in the Golden-section method. Therefore, it is necessary to find the optimal sequence so as to minimize the total compression ratio (1-R1) (1-R2). (1-RN) is minimal. After mathematical analysis, it is found that using Fibonacci sequence to construct the sequence {rk}={1-fn-k+1/fn-k+2}can meet the above requirements, wherein FK is the Fibonacci sequence K element, K<=n,n step after iteration makes the total compression ratio of 1//fn+1.

However, it can be found that the final iteration of the RN=1/2, which means that the two intermediate points an=bn, they are at the midpoint of the interval, overlapping each other, so that the interval can no longer continue to compress. So we need to artificially adjust RN=1/2-epsion, Epsion is a very small positive real number. This way the final compression ratio can be expressed as (1+2*epsion)/fn+1.

The algorithm requires the pre-set parameter epsion, the final compression ratio, and then the number of iterations N. Note the RN=1/2-epsion! used for the last iteration

[Test Data]:fx.dat Minx.dat

[Algorithmic code]:test.cpp goldseg.a goldseg.so FIBONACCI.A fibonacci.so

Numerical algorithm: The Golden section method and Fibonacci sequence method of one dimensional search method with unconstrained optimization