Machine learning Week 3-advanced-optimization

The Costfunction code is as follows

Function[jval,gradient]=costfunction (theta) jval= (Theta (1)-5) ^2+ (Theta (2)-5) ^2gradient=zeros (2,1) gradient (1) =2* (Theta (1)-5); gradient (2) =2* (Theta (2)-5);

Run the following code snippet

Options=optimset (' GradObj ', ' on ', ' maxiter ', +) Initialtheta=zeros (2,1) [Opttheta,functionval,exitflag]=fminunc (@ Costfunction,initialtheta,options)

The results are as follows

1Jval =2 3      -4 5 6Gradient =7 8      09      0Ten  One  AJval = -  -    50.0000 the  -  -Gradient = -  +      0 -      0 +  A  atJval = -  -    50.0000 -  -  -Gradient = in  -      0 to      0 +  -  theJval = *  $      0Panax Notoginseng  -  theGradient = +  A      0 the      0 +  -  $Jval = $  -    5.5511e-15 -  the  -Gradient =Wuyi  the      0 -      0 Wu  -  AboutJval = $  -    5.5511e-15 -  -  AGradient = +  the      0 -      0 $  the  the Local minimum found. the  theOptimization completed because the size of the gradient isLess than -Thedefaultvalue of the optimality tolerance. in  the<stopping Criteria Details> the  About  theOpttheta = the  the      5 +      5 -  the BayiFunctionval = the  the      0 -  -  theExitflag = the  the      1

Exitflag helps you understand whether convergence

Fminunc () function

Fminunc finds a local minimum of a function of several variables. X=Fminunc (fun,x0) starts at X0 and attempts to find a local minimizer X of the "function fun."      Fun accepts input X and returns a scalar function value F evaluated at X. X0 can be a scalar, vector or matrix. X= Fminunc (fun,x0,options) minimizes with thedefaultoptimization parameters replaced by valuesinchOPTIONS, an argument created with the Optimoptions function. See Optimoptions fordetails. Use the Specifyobjectivegradient option to specify that also returns a second output argument G that isThePartialderivatives of the function DF/DX, at the point X. With the HESSIANFCN option to specify that also returns a third output argument H that isThe 2ndPartialderivatives of the function (the Hessian) at the point X. The Hessian isOnly used by the trust-Region algorithm. X= Fminunc (problem) finds the minimum forProblem. Problem isa structure with the function funinchproblem.objective, the start pointinchproblem.x0, the options structureinchproblem.options, and Solver name'Fminunc' inchProblem.solver. Use Thissyntax to solve at the command line a problem exported fromOptimtool. [X,fval]=Fminunc (fun,x0,...) returns the value of the objective function fun at the solution x. [X,fval,exitflag]=Fminunc (fun,x0,...) returns an exitflag that describes the exit condition. Possible values of Exitflag and the corresponding exit conditions are listed below. See the documentation fora complete description. 1Magnitude of gradient small enough. 2ChangeinchX too small. 3Changeinchobjective function too small. 5cannot decrease function along search direction. 0Too many function evaluations or iterations. -1Stopped by output/plot function. -3problem seems unbounded. [X,fval,exitflag,output]=Fminunc (fun,x0,...) returns a structure OUTPUT with the number of iterations takeninchOutput.iterations, the number of function evaluationsinchOutput.funccount, the algorithm usedinchOutput.algorithm, the number of CG iterations (ifUsed)inchOutput.cgiterations, the first-order optimality (ifUsed)inchoutput.firstorderopt, and the exit messageinchoutput.message. [X,fval,exitflag,output,grad]=Fminunc (fun,x0,...) returns the value of the gradient of fun at the solution x. [X,fval,exitflag,output,grad,hessian]=Fminunc (fun,x0,...) returns the value of the Hessian of the objective function fun at the solution X. Examples fun can be specifiedusing@: X= Fminunc (@myfun,2)     whereMyfun isA MATLAB function such as: function F=myfun (x) F= sin (x) +3; To minimize Thisfunction with the gradient provided, modify the function myfun so the gradient isThe second output argument:function [f,g]=myfun (x) F= sin (x) +3; G=cos (x); and indicate the gradient value isavailable by creating options with options. SpecifyobjectivegradientSetTotrue(usingoptimoptions): Options= Optimoptions ('Fminunc','specifyobjectivegradient',true); X= Fminunc (@myfun,4, Options); Fun can also is an anonymous function:x= Fminunc (@ (x)5*x (1)^2+ x (2)^2,[5;1]) If Fun isparameterized, you can use anonymous functions to capture the problem-dependent parameters. Suppose want to minimize the objective giveninchThe function myfun, which isparameterized by its second argument C. Here Myfun isA MATLAB file function such asfunction [F,g]=Myfun (x,c) F= C*x (1)^2+2*x (1) *x (2) + x (2)^2; %function G= [2*c*x (1) +2*x (2)               %Gradient2*x (1) +2*x (2)]; To optimize fora specific value of C, first assign the value to C. Then create a one-argument anonymous function, that captures, the value of C and calls Myfun with the arguments. Finally, pass. Thisanonymous function to Fminunc:c=3; %define parameter first options= Optimoptions ('Fminunc','specifyobjectivegradient',true); % indicate gradient isprovided x= Fminunc (@ (x) myfun (x,c), [1;1],options)

Machine learning Week 3-advanced-optimization