Download - P. Pegon & Ph. on & Ph. CaCapéran - CEA/CEAClub Cast3M, 21th of November 2008 An amazing optimisation problemAn amazing optimisation problem P. Pegon & Ph. on & Ph. CaCapéran European

Club Cast3M, 21th of November 2008Club Cast3M, 21th of November 2008

An amazing optimisation problemAn amazing optimisation problemAn amazing optimisation problemAn amazing optimisation problem

P. Pegon & Ph. P. Pegon & Ph. CapéranCapérangg ppEuropean Laboratory for Structural AssessmentEuropean Laboratory for Structural Assessment

Joint Research CentreJoint Research CentreIspraIspra, Italy, Italy

An amazing An amazing optimisationoptimisation problemproblem

OUTLINE:

• Background: some results from photogrammetry

Th (VERY) i lifi d ti i ti bl• The (VERY) simplified optimisation problem

• EXCE, steepest descent, conjugate gradient & BFGS

• The Gauss-Newton method (+ new operator GANE)The Gauss Newton method (+ new operator GANE)

• Conclusions, further works & bibliography

Background: some results from Background: some results from photogrammetryphotogrammetry

The wall The cameraThe structureESECMASE

20cmm

4096 grey levels, S/N 64 dB, 1536x1024 pixels

Kodak CCD KAF 1602EKodak CCD KAF 1602E

+-1.3mm/pixel

Current tiI iti l t


Current state

i

Set of M interpolation

Reference stateInitial t0

Set of M interpolation functions on M reference

sub-windowsSet of M current sub-windows

under tracking

Given one reference, C3 interpolation function,

( )0W X

Consider its corresponding current discrete function

( )iw x

Optimisation on the Quadratic Error:

( ) ( )( )( )2

∑C3 Analytic Template { } { }1.. ... ..i i

n nP P +

( )With respect to { }nP

( ) ( )( )( )2

01..

( ) ) ;k kk K

E w x W T x=

= −∑P P( )0 ( ; )kW T x P

Tracking MethodCoordinate Transformation

parametrised by jdijd( );T X P

{ }nP


Comparison with Heidenhain measurement (K14 - 0.16g)


Displacement on targets


Displacement on texture


Displacement fields (K12 - 0.12g)


OUTLINE:






The (VERY) simplified The (VERY) simplified optimisationoptimisation problemproblem

In the elastic case…

65 bricks , 1,...,65 0.3iE i ν= =

Joints 66 66

66 ,2 (1 )n s

E EE K K

e e ν→ = =

+

Identification of ??? E


d is imposed

d is measured

1+

=EThe reference solution is obtained with and( )+ +d E ( ) 5%σ + =E1EThe reference solution is obtained with and ( )d E ( ) 5%σ E


{ }* argmin ( )F= EE EFind

( ) ( ) ( )T( )F = =E f E f E f E( ) ( ) ( )( )( )

( ) ( )+ += −f E d E d EWhere

(first) problem !!! ( ) ( )λ=d E d E


Numerical derivation of the gradients

( ) ( )( ) F E E E E F E E E EF δ δ+∂ Ε 1 1( ,..., ,..., ) ( ,..., ,..., )( )2

i n i n

i

F E E E E F E E E EFE E

δ δδ

+ − −∂ Ε ≈∂

And also (later)

1 1( ,..., ,..., ) ( ,..., ,..., )( ) i n i nE E E E E E E Eδ δ+ − −∂ Ε f ff 1 1( , , , , ) ( , , , , )( )2

i n i n

iE Eδ≈

∂


OUTLINE:






EXCE, EXCE, steepeststeepest descentdescent, , conjugateconjugate gradient & BFGSgradient & BFGS

EXCE

The EXCELL operator computes the minimum of a function F(Xi),the usemethod is the well-known MMA (Method of Moving Asymptotes) proposed by K.Svanberg. The purpose is to find the minimum of a function F(Xi) with i=1,N given that :

- the relations Cj(Xi) < Cjmax (j>0 and j=1,M) must be satisfied

- there are relations on each unknown Ximin < Xi < Ximax

The functions F and Cj are defined by the function values and by their derivatives at the starting point X0.

( ) , ( )i j iF E C E ?????


( ) ( ) ( )F + += −E d E d E

EXCE

1st case 1

( ) ( ) ( )

( ) ( ) ( ) 0

( ) 0 ( ) 0

C

C C

σ σ +

+ +

= − <

= − < = − <

E E E

E E E E E E2 3( ) 0 , ( ) 0C C= < = <E E E E E E

( ) ( ) ( )F + += −E d E d E2nd case

21 2

( ) ( ) ( )

( ) ( ) ( ) 0 , ( ) ( ) ( /100) 0

F

C C orσ σ+ ++= − < = − <

E d E d E

E E E E E E E

3rd case … but no4

( ) ( )

( ) ( ) ( ) / ( ) 1 0 10

F

C

σ+ + + + −

=E E

E d E d E d E convincing results41( ) ( ) ( ) / ( ) 1.0 10C + + + + −= − < ×E d E d E d E


Descent method: general format

D t di ti ithiE 0i T F∇ <EDescent direction with

Line search Robust line search???

idE 0id F∇ <

EE

{ }0argmin ( )i i idFα α= +E ELine search Robust line search???

and loop on until convergence

{ }0argmin ( )dFαα α> +E E

i1i i idα+ = +E E E

Meaning of convergence???

Here we concentrate on !!!!!!!iEHere we concentrate on !!!!!!! dE


Steepest descent iist F= −∇

EE

( ) ( ) / ( )+ + + +−d E d E d E

i

The convergenceThe convergence can be very slow!s


Conjugate gradient : the descent directions are mutually orthogonal

(with precise line search)1

121

( )i

i i T ii i i i i igc gc

iFγ γ

−−

−

−= + = −∇ =E

g g gE g E ggg

( ) ( ) / ( )+ + + +−d E d E d E

i

Surprisingly slow convergences

i


BFGS: the Hessian matrix is iteratively constructed

1 ...ii i i ibfgs bfgs bfgs bfgsF −= − ∇ = +

EE H H H

( ) ( ) / ( )+ + + +−d E d E d E

i

BFGS=Broyden-Fletcher-Goldfarb-ShannoSurprisingly slow

convergences


Summary:

• Very slow convergence

• “High” level asymptotic convergence plateau

Conclusion: we need to do better!Conclusion: we need to do better!


OUTLINE:






The GaussThe Gauss--Newton method (+ new operator GANE)Newton method (+ new operator GANE)

Write the problem as a least square problem…

{ }* argmin ( )F= xE EFind

( )( ) ( ) ( ) ( )2 2 T1 1 1( )2 2 2

m

iF f= = =∑E E f E f E f E( )( ) ( ) ( ) ( )

( )12 2 2

( ) ( )i=

+ += −

∑f E d E d E

Where

… and use it!


The Gauss-Newton MethodSame cost as

computing s/ jF E∂ ∂

Using with( ) ( ) ( )δ δ+ +f E E f E J E E ( ) ( )i

ijj

fE

∂⎡ ⎤ =⎣ ⎦ ∂J E E

is given by

j

{ }argmin ( )gn Fδ δ= +EE E E ( )T Tgn F= − = −∇J J E J f

Use as a descent directiongnE

Warning: no indetermination if the columns of are linearly independent

Jindependent


Operator GA(us-)NE(wtonTAB2=GANE TAB1;

CHPO1 RIGI1=GANE TAB1 'MATR';;

Description:____________

The function to minimize is 2F(X)=f(X).f(X), and J(X)=df/dX

The descent direction is H solution of:transpose(J)J H = -transpose(J)f

Contents:_________

TAB1 : Table of type 'VECTEUR' containingTAB1 . 0 = f(X)TAB1 . 1 = df/dX1

.TAB1 . n = df/dXn/

CHPO1 : -transpose(J)fRIGI1 : transpose(J)J

TAB2 : TABLE of type 'VECTEUR' containingTAB2 1 H1TAB2 . 1 = H1

. TAB2 . n = Hn


( ) ( )λ=d E d EThere is a second problem!!! (first problem )( ) ( )λ=d E d EThere is a second problem!!! (first problem )

• is almost always rank 1 deficient (use of RESO ‘ENSE’)( )TJ J

{ }65∂ ∂∑f f

• For instance: ( for )

BUT

1ia =− { }1=E66166

( )ii i

aE E=

∂ ∂=∂ ∂∑f fE

66 65( )E g= E

• GANE can be used to find out the ( )

• Assuming with with

ia 65 65 6566

( )T

E∂=

∂fJ J a J

/ i ig E a∂ ∂ = 65J ia∂ ∂ ∂= +f f f66 65( )gAssuming with with

• Use GANE again for finding and complete with 65

661

gn i gnii

E aE=

=∑i ig 65

66i

i iE E E∂ ∂ ∂

65gnE


And finally!

( ) ( ) / ( )+ + + +−d E d E d E

i

… with

i

7max( ) 10i iE E+ −− ≈( )i i


66E+Substituting with (and still starting from ) 66 , 1,2, 4,8nE n+ = { }1=E

1n 1n =2n =

3n =4n =

WARNING

• Sometimes RESO takes care itself of the indetermination…7E E+ +

• with 7max( ) 10i iE E −+− ≈

E E≠E E


OUTLINE:






Conclusions, further works & bibliographyConclusions, further works & bibliography

Conclusions & further works

• Nothing is given for free!

• Cast3M demonstrates again its ability to help to understand (and solve) problems!(and solve) problems!

• Gauss-Newton is the root for other classical methods

• Damped Gauss-Newton = Levenberg-Marquardt methodT T (already in GANE)

• Powell’s Dog Leg Method

• Use of noisy experimental data

( )T Tlm Fμ+ = − = −∇J J I E J f

• Use of noisy experimental data

• Identification of non-linear models with more refined meshes

• Line search, convergence tests…, g

Conclusions, further works & bibliographyConclusions, further works & bibliography

Short bibliography

• W.H. Press, S.A. Teukolsky, W.T. Vetterling & B.P. Flannery,Numerical Recipes : The Art of Scientific Computing, 1986u e ca ec pes e t o Sc e t c Co put g, 986

• H. Matthis & G. Strang,The Solution of Non-Linear Finite Element EquationsIJNME, 14, 1613-1626, 1979IJNME, 14, 1613 1626, 1979

• K. Madsen, H.B. Nielsen & O. TingleffMethods for Non-Linear Least Squares ProblemsTechnical University of Denmark 2004Technical University of Denmark, 2004

• T. Charras & J. KicheninL’optimisation dans CAST3M