5_unconstrained optimization(two)

Example:Gradientet d 3 f ( x3 ) (0.077976 ,0.214434 )T , d 3 0.052062 0.228171 , Accordingto iterationformula: x4 (0.019126 ,0.027816 )T .
Example:Gradient Method
Fifth iteration: Set d 4 f ( x4 ) (0.153008 , 0.055632 )T , d 4 0.026506 0.162807 ε, Accordingto iterationformula: x5 (0.001835 ,0.020195 )T . f ( x5 ) 0.001847 ε, it satisfiesaccuracymeet. So approximat e solution x5 (0.001835 ,0.020195 )T . T heoptimalsolution x (0,0)T .
Descent Iterative Method

There is an important nature of the one-dimensional search: the gradient at the optimal point along search direction and the search direction are orthogonal.
Calculatin g programof steepestdescent method: (1)Giveninitialpoint X 0 E n , permissionerror 0,set k 0; (2)Calcula te f ( X k ); (3)Examinewhethermeetstheconvergenc e criterion: f ( X k ) ε , If it meets, thenstopiteration, X* X k , else turn to(4); (4)Set d k f ( X k ), startingfrom X k do one- dimensiona l search along d k , thatis calculatek thatmakes: min f ( X k d k ) f ( X k k d k ); 0 (5)Set X k 1 X k k d k , k k 1, turn back to(2).

The Concept of Unconstrained Programming
The Concept of Unconstrained Programming
The Concept of Unconstrained Programming
Draw the contour lines（等 高线）: [x1,x2]=meshgrid(6:0.1:6,-6:0.1:6); z=2*x1.^22*x1.*x2+x2.^23*x1+x2; contour(x1,x2,z,10)； We can see that: Have an unique global minimize point


Steepest Descent Method (First-order Gradient method, 最速下降法)
Descent direction
Steepest Descent Method (First-order Gradient method, 最速下降法)
Steepest Descent Method (First-order Gradient method, 最速下降法)
T ry to calculateminimumof f x2 /(1 x1 x2 ) by steepestdescent method,given initialpoint(0,0) . How many times iterationdoes it need? In thisproblem,given initialpoint(1,1),accuracy
T
Example: Gradient Method
By steepestdescent methodsolve theproblem 2 2 min f ( x) 4 x1 x2 , where x ( x1,x2 )T .T heinitialpoint x0 (1,1)T , error 0.1 Solution : f is a positivedefinitequadratic function, and 8 0 0 Q , b , c 0. 0 2 0 T hegradient f ( x) of f at point x ( x1,x2 )T is (8 x1,2 x2 )T
Chapter 5 Unconstrained Optimization
Content
The Concept of Unconstrained Programming Descent Iterative Method （下降迭代法） Steepest Descent Method （最速下降法） Gradient Method for Quadratic Positive Definite Function （二次正定函数的牛顿法） Newton’s Method （牛顿法） Damped-Newton Method （阻尼牛顿法） Quasi-Newton Method （拟牛顿法） Conjugated Gradient Method （共轭梯度法） Solve Unconstrained Optimization by Matlab
Select k thatmakes f (X(k) k d (k) ) min f (X (k) k d (k) )





It is called one- dimensiona l search or line search. T hestep determinedis theoptimalstep.
Example: Gradient Method
Second iteration: Set d1 f ( x1 ) (0.369216 ,1.476924 )T , d1 2.18305 1.522375 , Do one- dimensiona l search from x1 along d1 , according to iterationformula: x2 (0.101537 ,0.147682 )T .
Draw the contour lines （等高线）: [X,Y] = meshgrid(5:.1:5,-2*pi:.1:2*pi); Z=Y./(1+X.^2+Y.^2); contour(X,Y,Z,30) We can see that: Have several extremism point.

That is:
f ( X ( k 1) )T d ( k ) 0 Stated as T heoremas follows: T heorem : Assume objectivefunction f ( X ) has first - order partialderivat ive s. X ( k 1) can be producedas : (k ) (k ) f ( X ( k ) d ( k ) ) f ( X λk d ) min ( k 1) (k ) (k ) X X d k T hen f ( X ( k 1) )T d ( k ) 0
Steepest Descent Method (First-order Gradient method, 最速下降法)

Steepest descent method is a basic algorithm,which plays an important role in the optimal method. Steepest descent method has the advantage of small amount of work, less variable, less demanding about the initial point; Disadvantage of slow convergence. The steepest descent method is appropriate for iterative in optimization process or as a pre-inserted between the steps. When close to the extreme point, it is better to use other kinds of fast convergence algorithm.
Descent Iterative Method(下降迭代法)

In the above iteration,choosing search direction d ( k ) is the key step.The difference of all algothrims is how to determine search direction. There are many algorithms that determine the step length. Step by the objective function value decreased along the search direction most is selected.That is finding the minimum of f(x) alongX ( k ) d ( k ) .