Multi-task Bolasso based aircraft dynamics identification

HAL Id: hal-01643177https://hal.inria.fr/hal-01643177

Submitted on 21 Nov 2017

HAL is a multi-disciplinary open accessarchive for the deposit and dissemination of sci-entific research documents, whether they are pub-lished or not. The documents may come fromteaching and research institutions in France orabroad, or from public or private research centers.

L’archive ouverte pluridisciplinaire HAL, estdestinée au dépôt et à la diffusion de documentsscientifiques de niveau recherche, publiés ou non,émanant des établissements d’enseignement et derecherche français ou étrangers, des laboratoirespublics ou privés.

Multi-task Bolasso based aircraft dynamics identificationCédric Rommel, Joseph Frédéric Bonnans, Baptiste Gregorutti, Pierre

Martinon

To cite this version:Cédric Rommel, Joseph Frédéric Bonnans, Baptiste Gregorutti, Pierre Martinon. Multi-task Bolassobased aircraft dynamics identification. PGMODays, Nov 2017, Paris, France. �hal-01643177�

Multi-task Bolasso based aircraftdynamics identification

C. Rommel1,2, J. F. Bonnans1,B. Gregorutti2 and P. Martinon1

CMAP Ecole Polytechnique - INRIA1

Safety Line2

November 14th 2017

(CMAP, INRIA, Safety Line) Aircraft dynamics identification November 14th 2017 1 / 19

Motivation

20 000 airplanes — 80 000 flights per day,

Should double until 2033,

Responsible for 3% of CO2 emissions,

Accounts for 30% of operational cost for an airline,

Rectilinear climb trajectories at full thrust.

Motivation

Example of optimized trajectory

Reference Optimized1311kg 1141kg

Optimal Control Problem

min(x,u)∈X×U

∫ tf

0C (t,u(t), x(t))dt,

Φ(x(0), x(tf )) ∈ KΦ,u(t) ∈ Uad , for a.e. t ∈ [0, tf ],cj(x(t)) ≤ 0, j = 1, . . . , nc , for all t ∈ [0, tf ],x = g(t,u, x), for a.e. t ∈ [0, tf ].

min(x,u)∈X×U

∫ tf

min(x,u)∈X×U

∫ tf

min(x,u)∈X×U

∫ tf

min(x,u)∈X×U

∫ tf

min(x,u)∈X×U

∫ tf

QAR data

Massive (> 1000 variables recorded every second),

x = g(t,u, x)

QAR data

x = g(t,u, x)

QAR data

x = g(t,u, x)

QAR data

x = g(t,u, x)

QAR data

x = g(t,u, x)

Flight mechanics and state equation

Classic flight mechanics model

h = V sin γ,

V =T cosα− D −mg sin γ

γ =T sinα + L−mg cos γ

m = − T

h = V sin γ,

m = − T

State variables: x = [h,V , γ,m]

h = V sin γ,

m = − T

State variables: x = [h,V , γ,m]Control variables: u = [α,N1]

h = V sin γ,

m = − T

State variables: x = [h,V , γ,m]Control variables: u = [α,N1]Unknown functions of the state and control variables

h = V sin γ,

V =T (x,u) cosα− D(x,u)−mg sin γ

γ =T (x,u) sinα + L(x,u)−mg cos γ

m = − T (x,u)

Isp(x,u).

State variables: x = [h,V , γ,m]Control variables: u = [α,N1]Unknown functions of the state and control variables

Model requirements

T function of (M, ρ,N1),

D function of (M, ρ, q),

L function of (M, ρ, q),

Isp function of (M, h,SAT ),

Need for smooth models,

Need for models which are fast to compute,

Need for interpretable models for safety,

Need for models which are rich enough.

Model requirements

T function of (M, ρ,N1) = ϕT (x,u),

D function of (M, ρ, q) = ϕD(x,u),

L function of (M, ρ, q) = ϕL(x,u),

Isp function of (M, h,SAT ) = ϕIsp(x,u),

Model requirements

T = XT · θT ,D = XD · θD ,L = XL · θL,Isp = XIsp · θIsp.

Model requirements

T = XT · θT , with XT 6= ϕT (x,u),

D = XD · θD , with XD 6= ϕD(x,u),

L = XL · θL, with XL 6= ϕL(x,u),

Isp = XIsp · θIsp, with XIsp 6= ϕIsp(x,u).

Model requirements

T = XT · θT , with XT 6= ϕT (x,u),

D = XD · θD , with XD 6= ϕD(x,u),

L = XL · θL, with XL 6= ϕL(x,u),

Isp = XIsp · θIsp, with XIsp 6= ϕIsp(x,u).

Model requirements

T = XT · θT , with XT = Φd ◦ ϕT (x,u),

D = XD · θD , with XD = Φd ◦ ϕD(x,u),

L = XL · θL, with XL = Φd ◦ ϕL(x,u),

Isp = XIsp · θIsp, with XIsp = Φd ◦ ϕIsp(x,u).

Model requirements

T = XT · θT , with XT = N1(1, ρ,M, ρ2, ρM,M2, ...),

D = XD · θD , with XD = q(1, α,M, α2, αM,M2, ...),

L = XL · θL, with XL = q(1, α,M, α2, αM,M2, ...),

Isp = XIsp · θIsp, with XIsp = (1, h,M, h2, hM,M2, ...).

Regression problems

h = V sin γ

mVr = T cosα− D −mg sin γ

mVr γ = T sinα + L−mg cos γ

m = − TIsp.

Targets to fit Unknown Random error

Regression problems

h = V sin γ

mVr = T cosα− D −mg sin γ

mVr γ = T sinα + L−mg cos γ

m = − TIsp.

Regression problems

h = V sin γ

mVr + mg sin γ = T cosα− D

mVr γ + mg cos γ = T sinα + L

m = − TIsp.

Regression problems

h = V sin γ

0 = T + mIsp.

Regression problems

h = V sin γ

0 = T + mIsp.

⇓Y1 = XT cosα · θT − XD · θD +ε1

Y2 = XT sinα · θT + XL · θL +ε2

0 = XT · θT + mXIsp · θIsp +ε3

Multi-task regression framework

Y1 = XT cosα · θT − XD · θD +ε1

︸︷︷︸

X>T1 −X>D 0 0X>T2 0 X>L 0X>T 0 0 X>Ispm

︸︷︷︸

θTθDθLθIsp

︸︷︷︸

Ensures all components of g to share the same thrust T ,

Better predictive accuracy from tight coupling,

Helps in high correlations setting.

Y1 = XT cosα · θT − XD · θD +ε1

︸︷︷︸

θTθDθLθIsp

︸︷︷︸

Y1 = XT1 · θT − XD · θD +ε1

Y2 = XT2 · θT + XL · θL +ε2

0 = XT · θT + XIspm · θIsp +ε3

︸︷︷︸

θTθDθLθIsp

︸︷︷︸

Y = Xθ + ε,

︸︷︷︸

θTθDθLθIsp

︸︷︷︸

Y = Xθ + ε,

︸︷︷︸

θTθDθLθIsp

︸︷︷︸

Y = Xθ + ε,

︸︷︷︸

θTθDθLθIsp

︸︷︷︸

Y = Xθ + ε,

︸︷︷︸

θTθDθLθIsp

︸︷︷︸

Y = Xθ + ε,

︸︷︷︸

θTθDθLθIsp

︸︷︷︸

Feature selectionLet {(xi ,ui , xi )}Ni=1 set of N observations,

{X i ,Y i}Ni=1,

Maybe not all monomials are relevant for T ,D, L and/or Isp model...

Overfitting...

Polynomial regression high correlations between elements of X i

Unstable selections...

⇒ Bolasso [Bach, 2008]

Feature selectionLet {(xi ,ui , xi )}Ni=1 set of N observations, {X i ,Y i}Ni=1,

Overfitting...

Empirical Risk Minimization

N∑i=1

L(Y i ,X iθ),

Overfitting...

Least Squares Regression

N∑i=1

‖Y i − X iθ‖22,

Feature selectionLet {(xi ,ui , xi )}Ni=1 set of N observations, {X i ,Y i}Ni=1,Maybe not all monomials are relevant for T ,D, L and/or Isp model...

Overfitting...

N∑i=1

‖Y i − X iθ‖22,

Overfitting...

N∑i=1

‖Y i − X iθ‖22,

Overfitting...

L1 penalization

N∑i=1

‖Y i − X iθ‖22 + λ‖θ‖1,

Overfitting...

L1 penalization

N∑i=1

‖Y i − X iθ‖22 + λ‖θ‖1,

' Lasso [Tibshirani, 1994]

Overfitting...

Block sparse Lasso

N∑i=1

‖Y i − X iθ‖22 + λ‖θ‖1,

Overfitting...

Block sparse Lasso

N∑i=1

‖Y i − X iθ‖22 + λ‖θ‖1,

Polynomial regression

high correlations between elements of X i

Overfitting...

Block sparse Lasso

N∑i=1

‖Y i − X iθ‖22 + λ‖θ‖1,

Overfitting...

Block sparse Lasso

N∑i=1

‖Y i − X iθ‖22 + λ‖θ‖1,

Overfitting...

Block sparse Lasso

N∑i=1

‖Y i − X iθ‖22 + λ‖θ‖1,

Bootstrap implementation

Block sparse Bolasso

Require:training data T = {(X i ,Y i )}Ni=1,number of bootstrap replicates m,L1 penalization parameter λ,

1: for k = 1 to m do2: Generate bootstrap sample T k ,3: Compute Block sparse Lasso estimate θk from T k ,4: Compute support Jk = {j , θkj 6= 0},5: end for6: Compute J =

⋂mk=1 Jk ,

7: Compute θJ from TJ = {(X iJ ,Y

i )}Ni=1 using Least-Squares.

Consistency under high correlations proved in [Bach, 2008],

Efficient implementations exists: LARS [Efron et al., 2004].

⋂mk=1 Jk ,

Identifiability issues

Use prior Isp {I isp = Isp(xi ,ui )}.

Y1 = XT1 · θT − XD · θD +ε1

Y2 = XT2 · θT + XL · θL +ε2

Y1 = XT1 · θT − XD · θD +ε1

Y2 = XT2 · θT + XL · θL +ε2

Y1 = XT1 · θT − XD · θD +ε1

Y2 = XT2 · θT + XL · θL +ε2

N∑i=1

‖Y i − X iθ‖22 + λ1‖θ‖1,

Y1 = XT1 · θT − XD · θD +ε1

Y2 = XT2 · θT + XL · θL +ε2

N∑i=1

‖Y i − X iθ‖22 + λ1‖θ‖1,

⇒ θT = θIsp = 0 !

Y1 = XT1 · θT − XD · θD +ε1

Y2 = XT2 · θT + XL · θL +ε2

N∑i=1

‖Y i − X iθ‖22 + λ1‖θ‖1,

⇒ θT = θIsp = 0 !

Use prior Isp {I isp = Isp(xi ,ui )}.(CMAP, INRIA, Safety Line) Aircraft dynamics identification November 14th 2017 13 / 19

Y1 = XT1 · θT − XD · θD +ε1

Y2 = XT2 · θT + XL · θL +ε2

N∑i=1

(‖Y i − X iθ‖2

2 + λ2‖I isp − X iIsp · θIsp‖2

)+ λ1‖θ‖1,

⇒ θT = θIsp = 0 !

Y1 = XT1 · θT − XD · θD +ε1

Y2 = XT2 · θT + XL · θL +ε2

λ2 Isp = λ2XIsp · θIsp +ε4

N∑i=1

(‖Y i − X iθ‖2

2 + λ2‖I isp − X iIsp · θIsp‖2

)+ λ1‖θ‖1,

⇒ θT = θIsp = 0 !

Y1 = XT1 · θT − XD · θD +ε1

Y2 = XT2 · θT + XL · θL +ε2

N∑i=1

‖Y i − X iθ‖22 + λ1‖θ‖1,

⇒ θT = θIsp = 0 !

Y1 = XT1 · θT − XD · θD +ε1

Y2 = XT2 · θT + XL · θL +ε2

N∑i=1

‖Y i − X iθ‖22 + λ1‖θ‖1,

λ2 Iisp

, X i =

T1)> −(X iD)> 0 0

(X iT2)> 0 (X i

(X iT )> 0 0 (X i

Ispm)>

0 0 0 λ2(X iIsp)>

Feature selection results

25 different B737-800,

10 471 flights = 8 261 619 observations,

Block sparse Bolasso used for T ,D, L and Isp,

We expect similar model structures,

Feature selection results for the thrust, drag, lift and specific impulse models.

Effect of λ2 on predicted states derivatives

Effect of λ2 on hidden elements

Identification results assessment

x− g(x,u)

x− x

u− u

minx,u∈X×U

∫ tf

(‖u(t)− um(t)‖2

u + ‖x(t)− xm(t)‖2x

s.t. x = g(x,u)

minx,u∈X×U

∫ tf

(‖u(t)− um(t)‖2

u + ‖x(t)− xm(t)‖2x

s.t. x = g(x,u)

Figure : Obtained using BOCOP [Bonnans et al., 2017]

minx,u∈X×U

∫ tf

(‖u(t)− um(t)‖2

u + ‖x(t)− xm(t)‖2x

s.t. x = g(x,u)

Figure : Obtained using BOCOP [Bonnans et al., 2017]

THANK YOU FOR YOUR ATTENTION !

References

Bach, F. (2008).Bolasso: model consistent Lasso estimation through the bootstrap.pages 33–40. Proceedings of the 25th International Conference onMachine Learning (ICML).

Bonnans, J. F., Giorgi, D., Grelard, V., Heymann, B., Maindrault, S.,Martinon, P., Tissot, O., and Liu, J. (2017).Bocop – A collection of examples.Technical report, INRIA.

Efron, B. et al. (2004).Least angle regression.Annals of Statistics, 32:407–499.

Tibshirani, R. (1994).Regression shrinkage and selection via the Lasso.Journal of the Royal Statistical Society, 58:267–288.

Multi-task Bolasso based aircraft dynamics identification

Documents

Transcript of Multi-task Bolasso based aircraft dynamics identification