Newton's Method
Conditions of Use:The objective function has a second derivative, and the sea-slug matrix is definite.
Pros and cons:fast convergence speed, large computational capacity, and very dependent on the initial point selection.
The basic steps of the algorithm:
Algorithm Flowchart:
The Resistance damped method is basically the same as Newton's method,
just added a one-dimensional exact search.:
Pros and cons: Improved local convergence.
we assume that the minimum value of f= (x-1) * (x-1) +y*y is required, the specific algorithm is implemented as follows, only need to run the nttest.m file, and other function files in the same directory:
1. script File nttest.m
Clear ALLCLC syms x yf= (x-1) * (x-1) +y*y;var=[x y];x0=[1 1];eps=0.000001; Disp (' Newton method: ') minnt (f,x0,var,eps) disp (' Drag damped method: ') minmnt (f,x0,var,eps)
2, MINNT.M
function [X,minf]=minnt (f,x0,var,eps)% target functions: f% initial point: x0% argument vector: var% precision: The value of the argument when the eps% target function takes the minimum value: x;% The minimum value of the target function: Minf format long; If nargin==3 eps=1.0e-6;endtol=1;syms l% x0=transpose (x0), while tol>eps% does not meet precision requirements Gradf=jacobian (F,var); % Gradient Direction Jacf=jacobian (Gradf,var); The% Jacobian matrix v=funval (gradf,var,x0); The numerical solution of the% gradient tol=norm (v);% calculates the size of the gradient (i.e., the first-order guide) pv=funval (jacf,var,x0); The numerical solution of the second derivative P=-INV (PV) *transpose (v); % Search Direction x1=x0+p ';% is iterated x0=x1;end x=x1;minf=funval (f,var,x); format short;
3, MINMNT.M
function [X,minf]=minmnt (f,x0,var,eps)% target functions: f% initial point: x0% argument vector: var% precision: The value of the argument when the eps% target function takes the minimum value: x;% The minimum value of the target function: Minf format long If nargin==3 eps=1.0e-6;endtol=1;syms l% x0=transpose (x0), while tol>eps% does not meet accuracy requirements Gradf=jacobian (f,var); % Gradient Direction Jacf=jacobian (Gradf,var); The% Jacobian matrix v=funval (gradf,var,x0); The numerical solution of the% gradient tol=norm (v);% calculates the size of the gradient (i.e., the first-order guide) pv=funval (jacf,var,x0); The numerical solution of the second derivative P=-INV (PV) *transpose (v); % Search Direction %%%% Find the best step%%% y=x0+l*p '; Yf=funval (f,var,y); [A,b]=minjt (yf,0,0.1); XM=MINHJ (yf,a,b); % Golden Section method for one-dimensional search best step x1=x0+xm*p ';% for iteration x0=x1; end x=double (x1); Minf=double (Funval (f,var,x)); format short;
4, MINHJ.M
function [X,minf]=minhj (f,a,b,eps)% target functions: f% extreme interval Left end: a% Extreme Interval Right endpoint: b% precision: eps% The value of the argument when the minimum value of the target function is taken: the minimum value of the X-target function: Minf format long ; if nargin==3 eps=1.0e-6;end l=a+0.382* (b-a); % Test point u=a+0.618* (b-a); % test point k=1;tol=b-a; While tol>eps&&k<100000 fl=subs (F,findsym (f), L); % heuristic point function value fu=subs (F,findsym (f), u); % heuristic Point function value if Fl>fu a=1; % change interval left endpoint l=u; u=a+0.618* (b-a); % shorter search interval else b=u; % change interval right endpoint u=l; l=a+0.382* (b-a); % Shorten search interval end k=k+1; Tol=abs (b-a); endif k==100000 disp (' minimum value not found! '); X=nan; Minf=nan; return;endx= (a+b)/2;minf=subs (F,findsym (f), x); format short;
5, MINJT.M
function [MINX,MAXX]=MINJT (f,x0,h0,eps)% target functions: f% initial point: x0% initial step: h0% accuracy: eps% objective function takes the interval left end of the extremum: minx% objective function takes the interval right end of the Extremum: Maxx Format longif nargin==3 eps=1.0e-6;end x1=x0;k=0;h=h0;while 1 x4=x1+h; % heuristic step k=k+1; F4=subs (F,findsym (f), X4); F1=subs (F,findsym (f), X1); If F4<f1 x2=x1; x1=x4; F2=F1; F1=F4; H=2*h; % increased step size else if k==1 h=-h; % direction search x2=x4; F2=F4; else x3=x2; x2=x1; x1=x4; break; End EndEnd minx=min (x1,x3); Maxx=x1+x3-minx;format Short;
6, FUNVAL.M
function Fv=funval (f,varvec,varval) Var=findsym (f); Varc=findsym (Varvec); S1=length (Var); S2=length (VARC); M=floor (( s1-1)/3+1) Varv=zeros (1,m); if s1~=s2 for i=0: ((s1-1)/3) K=findstr (Varc,var (3*i+1) ); index= (k-1)/3; Varv (i+1) =varval (index+1); End Fv=subs (F,VAR,VARV); else fv=subs (f,varvec,varval); end
running results such as:
The theory of simplex method is still a bit complicated, and this paper mainly aims at the basic realization of the algorithm, therefore, the theoretical part of this is skipped, the details can refer to the relevant information on the Internet.
Here's a concrete implementation:
we illustrate with specific examples:
the specific MATLAB implementation is as follows:
1. script file:
Clear allclc% a=[2 2 1 0 0 0% 1 2 0 1 0 0% 4 0 0 0 1 0% 0 4 0 0 0 1];% c=[-2-3 0 0 0 0];% b=[12 8 [] ';% b Asevector=[3 4 5 6]; A=[1 1-2 1 0 0 2-1 4 0 1 0 -1 2-4 0 0 1];c=[1-2 1 0 0 0];b=[12 8 4] '; basevector=[4 5 6]; [x Y]=MODIFSIMPLEMTHD (A,c,b,basevector)
2. modifsimplemthd.m file
function [X,minf]=modifsimplemthd (a,c,b,basevector)% constraint matrix: a% objective function coefficient vector: c% constrained right-side vector: b% initial base vector: basevector% The value of the argument when the target function takes the minimum value: the minimum value of the X-target function: Minf sz=size (A); Nvia=sz (2); N=sz (1); Xx=1:nvia;nobase=zeros (n.); m=1; If C>=0 vr=find (c~=0,1, ' last '); RGV=INV (A (:, (nvia-n+1): Nvia)) *b; If Rgv>=0 X=zeros (1,VR); minf=0; Else disp (' No optimal solution exists '); X=nan; Minf=nan; Return EndEnd for I=1:nvia% get non-base variable subscript if (IsEmpty (Find (Basevector==xx (i), 1)) Nobase (m) =i; m=m+1; else; EndEnd bcon=1; m=0; B=a (:, Basevector); Invb=inv (B); While BCon nb=a (:, nobase); % non-base variable matrix ncb=c (nobase); % non-base variable coefficient b=a (:, Basevector); % base variable matrix cb=c (Basevector); % base variable coefficient xb=invb*b; F=CB*XB; W=CB*INVB; For I=1:length (nobase)% discriminant Sigma (i) =W*NB (:, i)-NCB (i); End [Maxs,ind]=max (Sigma); %ind for the input variable subscript if maxs<=0% of the maximum value is less than 0, the output solution is optimal minf=cb*xb; Vr=find (c~=0,1, ' last '); For L=1:VR Ele=find (basevector==l,1); if (IsEmpty (Ele)) x (L) = 0; else X (L) =XB (ele); End End bcon=0; else Y=inv (B) *a (:, Nobase (Ind)); If y<=0% there is no optimal solution disp (' There is no optimal solution! '); X=nan; Minf=nan; Return else Minb=inf; chagb=0; For J=1:length (y) if Y (j) >0 Bz=xb (j)/y (j); If Bz<minb minb=bz; Chagb=j; End End End%CHAGB is the base variable subscript tmp=basevector (CHAGB); % Update base matrix and non-base matrix Basevector (CHAGB) =nobase (IND); Nobase (Ind) =tmp; For j=1:chagb-1 the inverse matrix transformation of the base variable matrix if Y (j) ~=0 Invb (J,:) =invb (J,:)-invb (CHAGB,:) *y (j)/y (CHAGB); End End for j=chagb+1:length (y) if Y (j) ~=0 Invb (J,:) =invb (J,:)-invb (CHAGB,:) *y (j)/y (CHAGB); End End INVB (CHAGB,:) =invb (CHAGB,:)/y (CHAGB); End End M=m+1; if (m==1000000)% of the iteration limit disp (' can't find the optimal solution! '); X=nan; Minf=nan; Return EndEnd
running results such as:
For more algorithmic implementations of optimization, please visithttp://download.csdn.net/detail/tengweitw/8434549, which has an index description of each algorithm, and of course includes the above algorithm.
Original: http://blog.csdn.net/tengweitw/article/details/43669185
Nineheadedbird
Optimization algorithm of "engineering optimization"--Newton Method, resistance damped method and Simplex method