NNLS (Lawson-Hanson) method in linearized

models

LSI & NNLS

• LSI = Least square with linear equality constraints

• NNLS = nonnegative least square

hGxfEx ,min

0,min xfEx

Initial conditions

Stopping condition

yes no

Manipulate indexes

Compute a subproblem

yes

no

finish

nonnegativity conditions for the subproblem

Change the solution sothat it satisfies the

nonegativity conditions

Flowchart

Initial conditions

• Sets Z and P

• Variables indexed in the set Z are held at value zero

• Variables indexed in the set P are free to take values different from zero

• Initially and P:=NULL

Zn,...,1

Initial conditions

Stopping condition

yes no

Manipulate indexes

Compute a subproblem

yes

no

finish

nonnegativity conditions for the subproblem

Change the solution sothat it satisfies the

nonegativity conditions

Flowchart

Stopping condition

• Start of the main loop

• Dual vector

• Stopping condition:

set Z is empty or

)(: ExfEw T

Zjwj

,0

Initial conditions

Stopping condition

yes no

Manipulate indexes

Compute a subproblem

yes

no

finish

nonnegativity conditions for the subproblem

Change the solution sothat it satisfies the

nonegativity conditions

Flowchart

Manipulate indexes

• Based on dual vector, one parameter indexed in Z is chosen to be estimated

• Index of this parameter is moved from set Z to set P

Initial conditions

Stopping condition

yes no

Manipulate indexes

Compute a subproblem

yes

no

finish

nonnegativity conditions for the subproblem

Change the solution sothat it satisfies the

nonegativity conditions

Flowchart

Compute subproblem

• Start of the inner loop

• Subproblem

where column j of Ep

fzEp

Zj

PjEofjcolumn

,0

,

Initial conditions

Stopping condition

yes no

Manipulate indexes

Compute a subproblem

yes

no

finish

nonnegativity conditions for the subproblem

Change the solution sothat it satisfies the

nonegativity conditions

Flowchart

Nonnegativity conditions

• If z satisfies nonnegativity conditions then we set x:=z and jump to stopping condition

• else continue

Initial conditions

Stopping condition

yes no

Manipulate indexes

Compute a subproblem

yes

no

finish

nonnegativity conditions for the subproblem

Change the solution sothat it satisfies the

nonegativity conditions

Flowchart

Manipulating the solution

• x is moved towards z so that every parameter estimate stays positive. Indexes of estimates that are zero are moved from P to Z. The new subproblem is solved.

Testing the algorithm

• Ex. Values of polynomial

are calculated at points x=1,2,3,4 with fixed p1 and p2.

• Columns of E hold the values of polynomial y(x)=x and polynomial at points x=1,2,3,4.

• Values of p1 and p2 are estimated with NNLS.

2

21)( xpxpxy

2)( xxy

25.01.0)( xxxy

nnls_test 0.1 (c) 2003 by Turku PET CentreMatrix E:1 1 2 4 3 9 4 16 Vector f:0.6 2.2 4.8 8.4 Result vector:0.1 0.5

32 13.05.01.0)( xxxxy

nnls_test 0.1 (c) 2003 by Turku PET CentreMatrix E:1 1 1 2 4 8 3 9 27 4 16 64 Vector f:0.73 3.24 8.31 16.72 Result vector:0.1 0.5 0.13

nnls_test 0.1 (c) 2003 by Turku PET CentreMatrix E:1 1 1 1 2 4 8 16 3 9 27 81 4 16 64 256 Vector f:0.73 3.24 8.31 16.72 Result vector:0.1 0.5 0.13 0

432 013.05.01.0)( xxxxxy

32 13.001.0)( xxxxy

nnls_test 0.1 (c) 2003 by Turku PET CentreMatrix E:1 1 1 2 4 8 3 9 27 4 16 64 Vector f:0.23 1.24 3.81 8.72 Result vector:0.1 7.26423e-16 0.13

• Kaisa Sederholm: Turku PET Centre Modelling report TPCMOD0020 2003-05-23