A NEW HYBRID WYL-AMRI CONJUGATE GRADIENT METHOD WITH SUFFICIENT DESCENT CONDITION

FOR UNCONSTRAINED OPTIMIZATION

Ibrahim S. Mohammed*Mustafa Mamat

Abdelrhaman Abashar Kamil Uba Kamfa

Contents :-* Introduction * Objectives of the research* Conjugate Gradient Version* New method and Algorithm* Numerical Results* Conclusion* References

* Introduction

Conjugate gradient method (CG) are designed to solve large scale

unconstrained optimization problems. In general, the method has the

form

𝒎𝒊𝒏𝒙 ∈𝑹𝒏 𝒇(𝒙) (1.1)

where 𝒇 ∶ 𝑹𝒏 → 𝑹 is continuously differentiable . Conjugate gradient

methods are iterative methods of the form

𝒙𝒌+𝟏 = 𝒙𝒌 + 𝜶𝒌𝒅𝒌 (1.2)

where 𝒙𝒌+𝟏 is the current iterate point , 𝜶𝒌 > 𝟎 is step length which is

computed by carrying out a line search , 𝒅𝒌 is the search direction of the

conjugate gradient method define by

𝒅𝒌 −𝒈𝒌 , 𝒊𝒇 𝒌 = 𝟎

−𝒈𝒌 + 𝜷𝒌𝒅𝒌, 𝒊𝒇 𝒌 ≥ 𝟏(1.3)

Some classical formula’s for 𝜷𝒌 are given as follows

𝜷𝒌𝑯𝑺 =

𝒈𝒌𝑻 (𝒈𝒌−𝒈𝒌−𝟏)

(𝒈𝒌−𝒈𝒌−𝟏)𝑻 𝒅𝒌−𝟏

(1.4)

• Cont : Introduction

• 𝜷𝒌𝑭𝑹 =

𝒈𝒌𝑻 𝒈𝒌

𝒈𝒌−𝟏𝑻 𝒈𝒌−𝟏

(1.5)

•

• 𝜷𝒌𝑷𝑹𝑷 =

𝒈𝒌𝑻(𝒈𝒌−𝒈𝒌−𝟏)

𝒈𝒌−𝟏𝑻 𝒈𝒌−𝟏

(1.6)

• 𝜷𝒌𝑪𝑫 = −

𝒈𝒌𝑻 𝒈𝒌

𝒅𝒌−𝟏𝑻 𝒈𝒌−𝟏

(1.7)

• 𝜷𝒌𝑫𝒀 =

𝒈𝒌𝑻 𝒈𝒌

(𝒈𝒌−𝒈𝒌−𝟏)𝑻 𝒅𝒌−𝟏

(1.8)

• 𝜷𝒌𝑳𝑺 = −

𝒈𝒌𝑻(𝒈𝒌−𝒈𝒌−𝟏)

𝒅𝒌−𝟏𝑻 𝒈𝒌−𝟏

(1.9)

Cont : Introduction Zoutendijk proved that FR Method with exact line search is globally

convergent , Al – Baali extended this results to the strong Wolfe -

Powell line search.

Recently, Wei et al. [7], propose a new CG formula

Abashar et al (2014), modified RMIL method to suggest

• Research Objective

* To proposed a new formula for solving unconstrained optimization.

* To analyze the performance these new formulas based on standard optimizations test problem functions.

* To proof the sufficient descent conditions of our new method

• Conjugate Gradient Version

(i) Hybrid CG methods (ii) Scaled CG methods. (iii) Three terms CG methods.

.

An important class of conjugate gradient algorithm is the hybrid CG method, for example Hu and Storey

[18] propose

Dai and Yuan [19] suggested two hybrid methods,

New method and Algorithm

we propose a new hybrid CG method which is a combination of two CG methods

(WYL, AMRI)

ALGORITHM

Step1. Given an initial point nRx 0 , )1,0( , Set 00 gd if |||| 0g , then stop.

Step2. ComputeASW

k

based on (14).

Step3. Compute kd based on (4). If |||| kg , then terminate,

Step4. Compute step size based on (3).

Step5. Update new point based on (2).

Step6. Convergent test and stopping criteria

If )()( 1 kk xfxf and |||| kg , then terminate, else, Set 1 kk and go to Step 2.

Numerical Results

• Test problem functions considered by Andrei.

• Stopping criteria as Hillstrom , 𝑔𝑘 ≤ 𝜀.

• Matlab Subroutine programming was used.

• Using exact line search.

• Performance profile introduced by Dolan and More.

Numerical Results

TABLE 1: A LIST OF PROBLEM FUNCTIONS

No Function Dim Initial Points

1 Six hump 2 (1, 1), (2, 2), (5,5), (10, -10)

2 Three hump 2 (24, 24), (29, 29), (33, 33), (50, 50)

3 Booth 2 (10, -10), (20, 20), (50, 50), (100, 100)

4 Treccani 2 (5, 5), (10,10), (-20, 20), (-50, -50)

5 Matyas 2 (1, 1), (5, 5), (10, 10), (50, 50)

6 Extended Maratos 2, 4 (0,0,0,0), (0.5,5, 0.5, 5), (10, 0.5, 10, 0.5), (70, 70, 70, 70)

7 Ext FREUD & ROTH 2, 4 (13, 13, 13, 13), (21, 21, 21, 21), (25, 25, 25,25), (23, 23, 23, 23)

8 Generalized Trig 2, 4, 10 (0.5, 5, …, 5),(5, 10, …, 10),(7,7, …, 7),(50, 50, …, 50)

9 Fletcher 2, 4, 10 (23, 23, …, 23), (45, 45, …, 45), (50, 5,…, 5), (70, 70, …,70)

10 Extended Penalty 2, 4, 10, 100 (0.5,5, …,5),(10,-0.5…,-0.5), (105,105, …,105), (130,130, …,130)

11 Raydan 1 2, 4, 10, 100 (1, 1, …,1), (3, 3, …, 3), (5, 5, …, 5), (-10, -10, …, -10)

12 Hager 2, 4, 10, 100 (3, -3, …, -3),(21, 21, …, 21), (-23, 23, …, 23), (23, 23, …, 23)

13 Rosenbrock 2, 4, 10, 100, 500, 1000, 10000 (7, 7, …, 7), (13, 13, …, 13), (23, 23, …, 23), (35, 35, …, 35)

14 Shallow 2, 4, 10, 100, 500, 1000, 10000 (21, -21, …, -21), (21, 21, …, 21), (50,50, …, 50),(130, 130, …, 130)

15 Tridiagonal 1 2, 4, 10, 100, 500, 1000, 10000 (0, 0,…, 0), (1, -1, …, -1), (17, -17, …, -17), (30, 30, …, 30)

16 Ext White & Holst 2, 4, 10, 100, 500, 1000, 10000 (-5, -5, …, -5), (2, -2, …, -2), (3, -3, …, -3), (7, -7, …, -7)

17 Ext Denschnb 2, 4, 10, 100, 500, 1000, 10000 (8, 8, …, 8), (11, 11, …,11), (12, 12, …, 12),(13, 13, …, 13)

18 Diagonal 4 2, 4, 10, 100, 500, 1000, 10000 (2, 2, …, 2), (5, 5, …,5), (10, 10, …, 10), (15, 15, …, 15)

Numerical Results

• Performance Profile based on Number iterations

Cont. Numerical Results

• Performance Profile based on CPU time

• Conclusion

* AMRI was able to solve 95% of test problems.

* WYL solve 97% of test problems.

* SW-A solve all test problems.

• References* M.R. Hestenes, E.L. Stiefel, Methods of conjugate gradients for solving linear systems, J. Res. Natl. Bur. Stand. Sec. B 49 1952, pp. 409–432

*Z. Wei, S. Yao, L. Liu, The convergence properties of some new conjugate gradient methods, Appl. Math. Comput. 183 2006, pp. 1341–1350.

*G. Zoutendijk, Nonlinear programming computational methods, in:J.Abadie (Ed.), Integer and Nonlinear Programming, North-Holland, Amsterdam, 1970, pp. 37–86.

*M.Rivaie,M.Mamat,L. June, M.Ismail, a new class of nonlinear conjugate gradient coefficient with global convergence properties, Appl.Math.comput.218 ,2012, pp.11323-11332.

. * Y.H. Dai, Y. Yuan. An efficient hybrid conjugate gradient method for unconstrained optimization. Ann. Oper. Res. 103, 2001, pp. 33–47.

* E. Dolan, J.J.More, Benchmarking optimization software with performance profile, Math. Prog. 91, 2002, pp. 201–213.

*Y. F. Hu, C. Storey . Global convergence result for conjugate gradient methods. J.Optim.Theory.Appl., 71,1991, pp. 399-405.

* N. Andrei, An unconstrained optimization test functions collection, Adv. Modell. Optim. 10, 2008, pp. 147–161.

Thank You