Robust Simulation of Hybrid Mechanistic and Machine ...
Transcript of Robust Simulation of Hybrid Mechanistic and Machine ...
![Page 1: Robust Simulation of Hybrid Mechanistic and Machine ...](https://reader034.fdocuments.in/reader034/viewer/2022051606/628088bebd432c27de4a80cb/html5/thumbnails/1.jpg)
1
Robust Simulation of Hybrid Mechanistic and Machine Learning
ModelsOctober 15, 2020
Matthew D. Stuber, PhDAssistant Professor, University of Connecticut
2020 INCOSE New England Fall WorkshopCopyright © 2020 by Matthew D. Stuber. Permission granted to INCOSE New England to publish and use
![Page 2: Robust Simulation of Hybrid Mechanistic and Machine ...](https://reader034.fdocuments.in/reader034/viewer/2022051606/628088bebd432c27de4a80cb/html5/thumbnails/2.jpg)
2
Outline
1. Introduction2. Robust Simulation of Mechanistic Models3. Concepts of Hybrid Models4. Robust Simulation of Hybrid Models5. Conclusion
![Page 3: Robust Simulation of Hybrid Mechanistic and Machine ...](https://reader034.fdocuments.in/reader034/viewer/2022051606/628088bebd432c27de4a80cb/html5/thumbnails/3.jpg)
3
Robust Simulation
• A “Robust System” mitigates the effects of uncertainty to ensure performance/safety constraints are satisfied.
Introduction
![Page 4: Robust Simulation of Hybrid Mechanistic and Machine ...](https://reader034.fdocuments.in/reader034/viewer/2022051606/628088bebd432c27de4a80cb/html5/thumbnails/4.jpg)
4
Robust Simulation
• A “Robust System” mitigates the effects of uncertainty to ensure performance/safety constraints are satisfied.
• “Robust Simulation” refers to the ability to rigorously account for the impacts of uncertainty via a model-based (i.e., simulation) approach– Conclude whether or not a system can meet the desired
performance/safety constraints in the face of uncertainty using mathematical models
Introduction
![Page 5: Robust Simulation of Hybrid Mechanistic and Machine ...](https://reader034.fdocuments.in/reader034/viewer/2022051606/628088bebd432c27de4a80cb/html5/thumbnails/5.jpg)
5
Robust Simulation: Another Perspective
• “Robust Simulation” could also be viewed through the modeler’s lens
Introduction
![Page 6: Robust Simulation of Hybrid Mechanistic and Machine ...](https://reader034.fdocuments.in/reader034/viewer/2022051606/628088bebd432c27de4a80cb/html5/thumbnails/6.jpg)
6
Robust Simulation: Another Perspective
• “Robust Simulation” could also be viewed through the modeler’s lens– Modeling and simulation of systems often requires
changing parameter values and/or model libraries and the solver then fails to converge
Introduction
![Page 7: Robust Simulation of Hybrid Mechanistic and Machine ...](https://reader034.fdocuments.in/reader034/viewer/2022051606/628088bebd432c27de4a80cb/html5/thumbnails/7.jpg)
7
Robust Simulation: Another Perspective
• “Robust Simulation” could also be viewed through the modeler’s lens– Modeling and simulation of systems often requires
changing parameter values and/or model libraries and the solver then fails to converge
Domain violation(numerical issue)
Constraint violation(safety issue)
Introduction
![Page 8: Robust Simulation of Hybrid Mechanistic and Machine ...](https://reader034.fdocuments.in/reader034/viewer/2022051606/628088bebd432c27de4a80cb/html5/thumbnails/8.jpg)
8
Robust Simulation: Another Perspective
• “Robust Simulation” could also be viewed through the modeler’s lens– Modeling and simulation of systems often requires
changing parameter values and/or model libraries and the solver then fails to converge
Domain violation(numerical issue)
Constraint violation(safety issue)
Introduction
![Page 9: Robust Simulation of Hybrid Mechanistic and Machine ...](https://reader034.fdocuments.in/reader034/viewer/2022051606/628088bebd432c27de4a80cb/html5/thumbnails/9.jpg)
9
Robust Simulation: Another Perspective• Numerical infeasibility encountered in algebraic
systems
Domain violation(numerical issue)
Constraint violation(safety issue)
DOI: 10.1002/aic.14447
![Page 10: Robust Simulation of Hybrid Mechanistic and Machine ...](https://reader034.fdocuments.in/reader034/viewer/2022051606/628088bebd432c27de4a80cb/html5/thumbnails/10.jpg)
10
Robust Simulation of Mechanistic Models
![Page 11: Robust Simulation of Hybrid Mechanistic and Machine ...](https://reader034.fdocuments.in/reader034/viewer/2022051606/628088bebd432c27de4a80cb/html5/thumbnails/11.jpg)
11
Accounting for Uncertainty
System Model
Parametric Uncertainty
Design
zState-Space
?For a specific design, how would the system respond to uncertainty?
( , , ) h z u p 0Pp
Uu
Robust Simulation of Mechanistic Models
![Page 12: Robust Simulation of Hybrid Mechanistic and Machine ...](https://reader034.fdocuments.in/reader034/viewer/2022051606/628088bebd432c27de4a80cb/html5/thumbnails/12.jpg)
12
Accounting for Uncertainty
System Model
Parametric Uncertainty
Design
OperatingEnvelope
zState-SpaceFor a specific design, how would the system respond to uncertainty?
( , , ) h z u p 0Pp
Uu
Robust Simulation of Mechanistic Models
![Page 13: Robust Simulation of Hybrid Mechanistic and Machine ...](https://reader034.fdocuments.in/reader034/viewer/2022051606/628088bebd432c27de4a80cb/html5/thumbnails/13.jpg)
13
Accounting for Uncertainty
System Model
Parametric Uncertainty
Design
OperatingEnvelope
zState-SpaceFor a specific design, how would the system respond to uncertainty?
Constraint/Specification
( , , ) h z u p 0Pp
Uu
Robust Simulation of Mechanistic Models
![Page 14: Robust Simulation of Hybrid Mechanistic and Machine ...](https://reader034.fdocuments.in/reader034/viewer/2022051606/628088bebd432c27de4a80cb/html5/thumbnails/14.jpg)
14
Accounting for Uncertainty
System Model
Parametric Uncertainty
Design
OperatingEnvelope
zState-SpaceFor a specific design, how would the system respond to uncertainty?
Constraint/Specification
SYSTEM FAILURE!
( , , ) h z u p 0Pp
Uu
Robust Simulation of Mechanistic Models
![Page 15: Robust Simulation of Hybrid Mechanistic and Machine ...](https://reader034.fdocuments.in/reader034/viewer/2022051606/628088bebd432c27de4a80cb/html5/thumbnails/15.jpg)
15
Accounting for Uncertainty
System Model
Parametric Uncertainty
Design
OperatingEnvelope
zState-SpaceFor a specific design, how would the system respond to uncertainty?
Constraint/Specification
SYSTEM FAILURE!
( , , ) h z u p 0Pp
Uu
Robust Simulation of Mechanistic Models
![Page 16: Robust Simulation of Hybrid Mechanistic and Machine ...](https://reader034.fdocuments.in/reader034/viewer/2022051606/628088bebd432c27de4a80cb/html5/thumbnails/16.jpg)
16
Accounting for Uncertainty
System Model
Parametric Uncertainty
Design
OperatingEnvelope
zState-SpaceFor a specific design, how would the system respond to uncertainty?
Constraint/Specification
ROBUST SYSTEM!
( , , ) h z u p 0Pp
Uu
Robust Simulation of Mechanistic Models
![Page 17: Robust Simulation of Hybrid Mechanistic and Machine ...](https://reader034.fdocuments.in/reader034/viewer/2022051606/628088bebd432c27de4a80cb/html5/thumbnails/17.jpg)
17
Accounting for Uncertainty
System Model
Parametric Uncertainty
Design
OperatingEnvelope
zState-SpaceFor a specific design, how would the system respond to uncertainty?
Constraint/Specification
ROBUST SYSTEM!
( , , ) h z u p 0Pp
Uu
Research Challenge: Verifying a system is not robust is as simple as finding a single realization of uncertainty that violates the constraint.
Verifying a system is robust requires simulating infinitely-many realizations of uncertainty and ensuring the system never violates the constraint.
Robust Simulation of Mechanistic Models
![Page 18: Robust Simulation of Hybrid Mechanistic and Machine ...](https://reader034.fdocuments.in/reader034/viewer/2022051606/628088bebd432c27de4a80cb/html5/thumbnails/18.jpg)
18
Accounting for Uncertainty
• Steady-state vs. dynamical systems models
System Model
( , , ) h z u p 0System Model
( , , ) ( ( , , ), , , )t t tx u p f x u p u p
nonlinear algebraic system nonlinear ODE system
Robust Simulation of Mechanistic Models
![Page 19: Robust Simulation of Hybrid Mechanistic and Machine ...](https://reader034.fdocuments.in/reader034/viewer/2022051606/628088bebd432c27de4a80cb/html5/thumbnails/19.jpg)
19
Accounting for Uncertainty
• Steady-state vs. dynamical systems models
System Model
( , , ) h z u p 0System Model
( , , ) ( ( , , ), , , )t t tx u p f x u p u p
nonlinear algebraic system nonlinear ODE system
Now, we must account for the transient response to uncertainty in our design.
Robust Simulation of Mechanistic Models
![Page 20: Robust Simulation of Hybrid Mechanistic and Machine ...](https://reader034.fdocuments.in/reader034/viewer/2022051606/628088bebd432c27de4a80cb/html5/thumbnails/20.jpg)
20
Accounting for Uncertainty
Dynamic System Model
Parametric Uncertainty
Design
OperatingEnvelope
State-Space
Constraint/Specification
ROBUST SYSTEM!
Pp
Uu
( , , ), , , ) 0(k k
g t t x u p u p
( , , )ktx u p
Robust Simulation of Mechanistic Models
![Page 21: Robust Simulation of Hybrid Mechanistic and Machine ...](https://reader034.fdocuments.in/reader034/viewer/2022051606/628088bebd432c27de4a80cb/html5/thumbnails/21.jpg)
21
OperatingEnvelope
Accounting for Uncertainty
Dynamic System Model
Parametric Uncertainty
Design
State-Space
Constraint/Specification
Pp
Uu
1 1( , , ), , , ) 0(
k kg t t x u p u p
1( , , )
kt x u p
SYSTEM FAILURE!
Robust Simulation of Mechanistic Models
![Page 22: Robust Simulation of Hybrid Mechanistic and Machine ...](https://reader034.fdocuments.in/reader034/viewer/2022051606/628088bebd432c27de4a80cb/html5/thumbnails/22.jpg)
22
Mathematical Preliminaries• From a design perspective, our objective is to verify performance/safety
in the face of (the worst-case) uncertainty over the time horizon.
,
0
( ) max ( ( , , ), , , )
s.t. ( , , ) ( ( , , ), , , )
( , , 0) ( , )
P t Ig t t
t t t
pu x u p u p
x u p f x u p u px u p x u p
Robust Simulation of Mechanistic Models
![Page 23: Robust Simulation of Hybrid Mechanistic and Machine ...](https://reader034.fdocuments.in/reader034/viewer/2022051606/628088bebd432c27de4a80cb/html5/thumbnails/23.jpg)
23
Mathematical Preliminaries• From a design perspective, our objective is to verify performance/safety
in the face of (the worst-case) uncertainty over the time horizon.
If , we have verified the robustness of our design u.
“For a given design, the system does not violate performance/safety at any point in time, even in the face of the worst-case uncertainty”
,
0
( ) max ( ( , , ), , , )
s.t. ( , , ) ( ( , , ), , , )
( , , 0) ( , )
P t Ig t t
t t t
pu x u p u p
x u p f x u p u px u p x u p
( ) 0 u
Robust Simulation of Mechanistic Models
![Page 24: Robust Simulation of Hybrid Mechanistic and Machine ...](https://reader034.fdocuments.in/reader034/viewer/2022051606/628088bebd432c27de4a80cb/html5/thumbnails/24.jpg)
24
Robust Steady-State Simulation• Previous developments: a set-valued mapping theory that
enables the calculation of rigorous bounds on the states over the entire uncertainty space.
System Model
Parametric Uncertainty
Design
OperatingEnvelope
zState-Space
( , , ) h z u p 0Pp
UuStuber, M.D. et al. (2015) DOI: 10.1080/10556788.2014.924514
Robust Simulation of Mechanistic Models
![Page 25: Robust Simulation of Hybrid Mechanistic and Machine ...](https://reader034.fdocuments.in/reader034/viewer/2022051606/628088bebd432c27de4a80cb/html5/thumbnails/25.jpg)
25
Robust Steady-State Simulation
OperatingEnvelope
zState-Space
Convex relaxation of nonconvex operating envelope (without actually simulating the operating envelope)
Robust Simulation of Mechanistic Models
Stuber, M.D. et al. (2015) DOI: 10.1080/10556788.2014.924514
![Page 26: Robust Simulation of Hybrid Mechanistic and Machine ...](https://reader034.fdocuments.in/reader034/viewer/2022051606/628088bebd432c27de4a80cb/html5/thumbnails/26.jpg)
26
Robust Dynamic Simulation
• Our dynamic model is reformulated in the discrete form as a nonlinear algebraic system:
0
1
0
1 0 1 1
( , )
( , , , )( , , )
( , , , )K K K K
h t
h t
y x u py y f y u p
h y u p 0
y y f y u p
( 1) ( 1): px u xnn K n n K h
Robust Simulation of Mechanistic Models
![Page 27: Robust Simulation of Hybrid Mechanistic and Machine ...](https://reader034.fdocuments.in/reader034/viewer/2022051606/628088bebd432c27de4a80cb/html5/thumbnails/27.jpg)
27
Robust Dynamic Simulation
Apply our theory for robust dynamic simulation to our system to calculate rigorous bounds on the state variables over the range of uncertainty variables p and design variables u, forward in time.
Robust Simulation of Mechanistic Models
![Page 28: Robust Simulation of Hybrid Mechanistic and Machine ...](https://reader034.fdocuments.in/reader034/viewer/2022051606/628088bebd432c27de4a80cb/html5/thumbnails/28.jpg)
28
Robust Dynamic Simulation
OperatingEnvelope
zState-Space
OperatingEnvelope
zState-Space
1k kt t
Robust Simulation of Mechanistic Models
![Page 29: Robust Simulation of Hybrid Mechanistic and Machine ...](https://reader034.fdocuments.in/reader034/viewer/2022051606/628088bebd432c27de4a80cb/html5/thumbnails/29.jpg)
29
Robust Dynamic Simulation
OperatingEnvelope
zState-Space
OperatingEnvelope
zState-Space
1k kt t
Robust Simulation of Mechanistic Models
DOI:10.1002/aic.16836
![Page 30: Robust Simulation of Hybrid Mechanistic and Machine ...](https://reader034.fdocuments.in/reader034/viewer/2022051606/628088bebd432c27de4a80cb/html5/thumbnails/30.jpg)
30
Control of a 9-species biological reaction for wastewater treatment.
Robust Dynamic Simulation
Robust Simulation of Mechanistic Models
![Page 31: Robust Simulation of Hybrid Mechanistic and Machine ...](https://reader034.fdocuments.in/reader034/viewer/2022051606/628088bebd432c27de4a80cb/html5/thumbnails/31.jpg)
31
Next steps: verification and validation of robustness
Robust Dynamic Simulation
,
0
min
s.t. max ( ( , , ), , , )
s.t. ( , , ) ( ( , , ), , , ),
( , , 0) ( , )
U
P t Ig t t
t t t t I
u
px u p u p
x u p f x u p u px u p x u p
Robust Design
Robust Simulation of Mechanistic Models
![Page 32: Robust Simulation of Hybrid Mechanistic and Machine ...](https://reader034.fdocuments.in/reader034/viewer/2022051606/628088bebd432c27de4a80cb/html5/thumbnails/32.jpg)
32
Next steps: verification and validation of robustness
Robust Dynamic Simulation
,
0
min
s.t. max ( ( , , ), , , )
s.t. ( , , ) ( ( , , ), , , ),
( , , 0) ( , )
U
P t Ig t t
t t t t I
u
px u p u p
x u p f x u p u px u p x u p
0
max
s.t. min ( ( , , ), , , )
s.t. ( , , ) ( ( , , ), , , ),
( , , 0) ( , )
P
f fUg t t
t t t t I
p
ux u p u p
x u p f x u p u px u p x u p
Robust Design Robust Operation
Robust Simulation of Mechanistic Models
![Page 33: Robust Simulation of Hybrid Mechanistic and Machine ...](https://reader034.fdocuments.in/reader034/viewer/2022051606/628088bebd432c27de4a80cb/html5/thumbnails/33.jpg)
33
Next steps: verification and validation of robustness
Robust Dynamic Simulation
,
0
min
s.t. max ( ( , , ), , , )
s.t. ( , , ) ( ( , , ), , , ),
( , , 0) ( , )
U
P t Ig t t
t t t t I
u
px u p u p
x u p f x u p u px u p x u p
0
max
s.t. min ( ( , , ), , , )
s.t. ( , , ) ( ( , , ), , , ),
( , , 0) ( , )
P
f fUg t t
t t t t I
p
ux u p u p
x u p f x u p u px u p x u p
Robust Design Robust Operation
Find the best design u and seek the worst-case realization of uncertainty p to see if the system violates the performance/safety specifications.
Robust Simulation of Mechanistic Models
![Page 34: Robust Simulation of Hybrid Mechanistic and Machine ...](https://reader034.fdocuments.in/reader034/viewer/2022051606/628088bebd432c27de4a80cb/html5/thumbnails/34.jpg)
34
Next steps: verification and validation of robustness
Robust Dynamic Simulation
,
0
min
s.t. max ( ( , , ), , , )
s.t. ( , , ) ( ( , , ), , , ),
( , , 0) ( , )
U
P t Ig t t
t t t t I
u
px u p u p
x u p f x u p u px u p x u p
0
max
s.t. min ( ( , , ), , , )
s.t. ( , , ) ( ( , , ), , , ),
( , , 0) ( , )
P
f fUg t t
t t t t I
p
ux u p u p
x u p f x u p u px u p x u p
Robust Design Robust Operation
Find the best design u and seek the worst-case realization of uncertainty p to see if the system violates the performance/safety specifications.
Find the worst-case realization of uncertainty p and seek a recourse (control) u and to see if the system violates the performance/safety specifications.
Robust Simulation of Mechanistic Models
![Page 35: Robust Simulation of Hybrid Mechanistic and Machine ...](https://reader034.fdocuments.in/reader034/viewer/2022051606/628088bebd432c27de4a80cb/html5/thumbnails/35.jpg)
35
Next steps: verification and validation of robustness
Robust Dynamic Simulation
,
0
min
s.t. max ( ( , , ), , , )
s.t. ( , , ) ( ( , , ), , , ),
( , , 0) ( , )
U
P t Ig t t
t t t t I
u
px u p u p
x u p f x u p u px u p x u p
0
max
s.t. min ( ( , , ), , , )
s.t. ( , , ) ( ( , , ), , , ),
( , , 0) ( , )
P
f fUg t t
t t t t I
p
ux u p u p
x u p f x u p u px u p x u p
Robust Design Robust Operation
Robust Simulation of Mechanistic Models
![Page 36: Robust Simulation of Hybrid Mechanistic and Machine ...](https://reader034.fdocuments.in/reader034/viewer/2022051606/628088bebd432c27de4a80cb/html5/thumbnails/36.jpg)
36
Software Tools
Robust Simulation of Mechanistic Models
![Page 37: Robust Simulation of Hybrid Mechanistic and Machine ...](https://reader034.fdocuments.in/reader034/viewer/2022051606/628088bebd432c27de4a80cb/html5/thumbnails/37.jpg)
37
Software Tools• EAGO.jl: Easy Advanced Global Optimization in Julia.
– Open-source, competitive with state-of-the-art commercial solvers but much more flexible to account for complicated user-defined functions (UDFs)
Robust Simulation of Mechanistic Models
DOI:10.1080/10556788.2020.1786566
![Page 38: Robust Simulation of Hybrid Mechanistic and Machine ...](https://reader034.fdocuments.in/reader034/viewer/2022051606/628088bebd432c27de4a80cb/html5/thumbnails/38.jpg)
38
Software Tools• EAGO.jl: Easy Advanced Global Optimization in Julia.
– Open-source, competitive with state-of-the-art commercial solvers but much more flexible to account for complicated user-defined functions (UDFs)
Robust Simulation of Mechanistic Models
DOI:10.1080/10556788.2020.1786566
![Page 39: Robust Simulation of Hybrid Mechanistic and Machine ...](https://reader034.fdocuments.in/reader034/viewer/2022051606/628088bebd432c27de4a80cb/html5/thumbnails/39.jpg)
39
Software Tools• EAGO.jl: Easy Advanced Global Optimization in Julia.
Software access: registered Julia package
Github: https://github.com/PSORLab/EAGO.jl
Robust Simulation of Mechanistic Models
DOI:10.1080/10556788.2020.1786566
![Page 40: Robust Simulation of Hybrid Mechanistic and Machine ...](https://reader034.fdocuments.in/reader034/viewer/2022051606/628088bebd432c27de4a80cb/html5/thumbnails/40.jpg)
40
Software Tools• EAGO.jl: Easy Advanced Global Optimization in Julia.
Robust Simulation of Mechanistic Models
DOI:10.1080/10556788.2020.1786566
![Page 41: Robust Simulation of Hybrid Mechanistic and Machine ...](https://reader034.fdocuments.in/reader034/viewer/2022051606/628088bebd432c27de4a80cb/html5/thumbnails/41.jpg)
41
Concepts of Hybrid Models
Concepts of Hybrid Models
![Page 42: Robust Simulation of Hybrid Mechanistic and Machine ...](https://reader034.fdocuments.in/reader034/viewer/2022051606/628088bebd432c27de4a80cb/html5/thumbnails/42.jpg)
42
Dynamic NN Prediction Errors
• Consider the anharmonic oscillator (e.g., pendulum)
(Example from DiffEqFlux.jl)
3 3( ) ( ) ( )
(0) 0
( ) ( )
(0) 1,
x t x t xt t
x
x t kx
x
Concepts of Hybrid Models
![Page 43: Robust Simulation of Hybrid Mechanistic and Machine ...](https://reader034.fdocuments.in/reader034/viewer/2022051606/628088bebd432c27de4a80cb/html5/thumbnails/43.jpg)
43
Dynamic NN Prediction Errors
Concepts of Hybrid Models
(Example from DiffEqFlux.jl)
Tensor product layer with 10th-Order Legendre Basis
![Page 44: Robust Simulation of Hybrid Mechanistic and Machine ...](https://reader034.fdocuments.in/reader034/viewer/2022051606/628088bebd432c27de4a80cb/html5/thumbnails/44.jpg)
44
Dynamic NN Prediction Errors
Concepts of Hybrid Models
![Page 45: Robust Simulation of Hybrid Mechanistic and Machine ...](https://reader034.fdocuments.in/reader034/viewer/2022051606/628088bebd432c27de4a80cb/html5/thumbnails/45.jpg)
45
Dynamic NN Prediction Errors
Concepts of Hybrid Models
![Page 46: Robust Simulation of Hybrid Mechanistic and Machine ...](https://reader034.fdocuments.in/reader034/viewer/2022051606/628088bebd432c27de4a80cb/html5/thumbnails/46.jpg)
46
Dynamic NN Prediction Errors
Actual trajectories exhibit rapid decay
Concepts of Hybrid Models
![Page 47: Robust Simulation of Hybrid Mechanistic and Machine ...](https://reader034.fdocuments.in/reader034/viewer/2022051606/628088bebd432c27de4a80cb/html5/thumbnails/47.jpg)
47
Dynamic NN Prediction Errors
Actual trajectories exhibit rapid decay
ML model fails to capture this behavior
Concepts of Hybrid Models
![Page 48: Robust Simulation of Hybrid Mechanistic and Machine ...](https://reader034.fdocuments.in/reader034/viewer/2022051606/628088bebd432c27de4a80cb/html5/thumbnails/48.jpg)
48
Dynamic NN Prediction Errors
Actual trajectories exhibit rapid decay
ML model fails to capture this behavior
Concepts of Hybrid Models
![Page 49: Robust Simulation of Hybrid Mechanistic and Machine ...](https://reader034.fdocuments.in/reader034/viewer/2022051606/628088bebd432c27de4a80cb/html5/thumbnails/49.jpg)
49
Dynamic NN Prediction Errors
Concepts of Hybrid Models
![Page 50: Robust Simulation of Hybrid Mechanistic and Machine ...](https://reader034.fdocuments.in/reader034/viewer/2022051606/628088bebd432c27de4a80cb/html5/thumbnails/50.jpg)
50
Outstanding ML Challenges
• Absence of theory• Absence of causal models (correlation not causation)• Sensitivity to imperfect data• Computational expense (training)
Begoli, E., Bhattacharya, T., Kusnezov, D. (2019) DOI: 10.1038/s42256-018-0004-1
Concepts of Hybrid Models
![Page 51: Robust Simulation of Hybrid Mechanistic and Machine ...](https://reader034.fdocuments.in/reader034/viewer/2022051606/628088bebd432c27de4a80cb/html5/thumbnails/51.jpg)
51
Outstanding ML Challenges (SE Perspective)
• Lack requirements specification• Lack design specification• Lack interpretability (causal relationships)• Lack robustness
Kuwajima, H., Yasuoka, H., Nakae, T., (2020) DOI: 10.1007/s10994-020-05872-w
Concepts of Hybrid Models
![Page 52: Robust Simulation of Hybrid Mechanistic and Machine ...](https://reader034.fdocuments.in/reader034/viewer/2022051606/628088bebd432c27de4a80cb/html5/thumbnails/52.jpg)
52
• Lack requirements specification• Lack design specification• Lack interpretability• Lack robustness
Kuwajima, H., Yasuoka, H., Nakae, T., (2020) DOI: 10.1007/s10994-020-05872-w
“Greatest impact on conventional system quality models”
Outstanding ML Challenges (SE Perspective)
Concepts of Hybrid Models
![Page 53: Robust Simulation of Hybrid Mechanistic and Machine ...](https://reader034.fdocuments.in/reader034/viewer/2022051606/628088bebd432c27de4a80cb/html5/thumbnails/53.jpg)
53
• Lack requirements specification• Lack design specification• Lack interpretability• Lack robustness
Kuwajima, H., Yasuoka, H., Nakae, T., (2020) DOI: 10.1007/s10994-020-05872-w
Research Challenge: Can we exploit machine learning approaches for safety-critical systems?
Outstanding ML Challenges (SE Perspective)
Concepts of Hybrid Models
![Page 54: Robust Simulation of Hybrid Mechanistic and Machine ...](https://reader034.fdocuments.in/reader034/viewer/2022051606/628088bebd432c27de4a80cb/html5/thumbnails/54.jpg)
54
Hybrid Mechanistic ML Models
• Not a “new” idea (emergence in 1992)• Combine aspects of machine learning and mechanistic
modeling• Black-Box → Gray-Box
Concepts of Hybrid Models
![Page 55: Robust Simulation of Hybrid Mechanistic and Machine ...](https://reader034.fdocuments.in/reader034/viewer/2022051606/628088bebd432c27de4a80cb/html5/thumbnails/55.jpg)
55
Hybrid Mechanistic ML Models
First-Principles
Model
ML
Model
First-Principles
Model
ML
Model
ML
Model
First-Principles
Model
Parallel Series
Concepts of Hybrid Models
![Page 56: Robust Simulation of Hybrid Mechanistic and Machine ...](https://reader034.fdocuments.in/reader034/viewer/2022051606/628088bebd432c27de4a80cb/html5/thumbnails/56.jpg)
56
ThermofluidsReactor
Mass Cons.
Kinetics
Thermo-dynamics
Species Properties
Heat Transfer
Fluid Dynamics
Hybrid Mechanistic ML Models
Concepts of Hybrid Models
![Page 57: Robust Simulation of Hybrid Mechanistic and Machine ...](https://reader034.fdocuments.in/reader034/viewer/2022051606/628088bebd432c27de4a80cb/html5/thumbnails/57.jpg)
57
ThermofluidsReactor
Mass Cons.
Kinetics
Thermo-dynamics
Species Properties
Heat Transfer
Fluid Dynamics
Hybrid Mechanistic ML Models
Concepts of Hybrid Models
Machine Learning Model• Empirical model• Artificial neural net• Dyn. neural net• Support vect. machine
![Page 58: Robust Simulation of Hybrid Mechanistic and Machine ...](https://reader034.fdocuments.in/reader034/viewer/2022051606/628088bebd432c27de4a80cb/html5/thumbnails/58.jpg)
58
Hybrid Mechanistic ML Models
Concepts of Hybrid Models
![Page 59: Robust Simulation of Hybrid Mechanistic and Machine ...](https://reader034.fdocuments.in/reader034/viewer/2022051606/628088bebd432c27de4a80cb/html5/thumbnails/59.jpg)
59
Hybrid Mechanistic ML Models
Concepts of Hybrid Models
“Intermediate” states or calculations. E.g., Complex nonlinear dynamics.
![Page 60: Robust Simulation of Hybrid Mechanistic and Machine ...](https://reader034.fdocuments.in/reader034/viewer/2022051606/628088bebd432c27de4a80cb/html5/thumbnails/60.jpg)
60
Hybrid Mechanistic ML Models
Benefits over pure data-driven models:• Requires less data• Have system insight• Better controller performance• Better performance with nonlinear dynamics• Better performance in extrapolation
– More useful for optimization applications
Concepts of Hybrid Models
![Page 61: Robust Simulation of Hybrid Mechanistic and Machine ...](https://reader034.fdocuments.in/reader034/viewer/2022051606/628088bebd432c27de4a80cb/html5/thumbnails/61.jpg)
61
Robust Simulation of Hybrid Models
![Page 62: Robust Simulation of Hybrid Mechanistic and Machine ...](https://reader034.fdocuments.in/reader034/viewer/2022051606/628088bebd432c27de4a80cb/html5/thumbnails/62.jpg)
62
Physics-Informed Data-Driven Models
PI Controller
MEA
MEA
MEA
MPC/EMPC1/EMPC2
• Want to control wastewater treatment processes to optimize energy consumption and meet discharge requirements
Robust Simulation of Hybrid Models
![Page 63: Robust Simulation of Hybrid Mechanistic and Machine ...](https://reader034.fdocuments.in/reader034/viewer/2022051606/628088bebd432c27de4a80cb/html5/thumbnails/63.jpg)
63
Physics-Informed Data-Driven Models
PI Controller
MEA
MEA
MEA
MPC/EMPC1/EMPC2
• Developed a compartment model with unknownparameters for mass transfer between compartments
• Applied deterministic global optimization for training to obtain guaranteed best-possible fit
Robust Simulation of Hybrid Models
Zone 1
Zone 2
Zone3
Mixing Zone
![Page 64: Robust Simulation of Hybrid Mechanistic and Machine ...](https://reader034.fdocuments.in/reader034/viewer/2022051606/628088bebd432c27de4a80cb/html5/thumbnails/64.jpg)
64
Physics-Informed Data-Driven Models
CFD Pure DD (ML) Model
Experiment 1 108% 329%Experiment 2 1287% 588%Experiment 3 511% 319%
Percent error relative to physics-informed data-driven model
Robust Simulation of Hybrid Models
![Page 65: Robust Simulation of Hybrid Mechanistic and Machine ...](https://reader034.fdocuments.in/reader034/viewer/2022051606/628088bebd432c27de4a80cb/html5/thumbnails/65.jpg)
65
Hybrid Mechanistic ML Models
System Model
Parametric Uncertainty
Design
OperatingEnvelope
zState-Space
Pp
Uu
Robust Simulation of Hybrid Models
0
1
0
1 0 1 1
( , )
( , , , )( , , )
( , , , )K K K K
h t
h t
y x u py y f y u p
h y u p 0
y y f y u p
( , , ) h y u p 0
![Page 66: Robust Simulation of Hybrid Mechanistic and Machine ...](https://reader034.fdocuments.in/reader034/viewer/2022051606/628088bebd432c27de4a80cb/html5/thumbnails/66.jpg)
66
Robust Simulation of Hybrid Models
Robust Simulation of Hybrid Models
![Page 67: Robust Simulation of Hybrid Mechanistic and Machine ...](https://reader034.fdocuments.in/reader034/viewer/2022051606/628088bebd432c27de4a80cb/html5/thumbnails/67.jpg)
67
Robust Simulation of Hybrid Models
k=2k=3
P 1
P 2
Robust Simulation of Hybrid Models
![Page 68: Robust Simulation of Hybrid Mechanistic and Machine ...](https://reader034.fdocuments.in/reader034/viewer/2022051606/628088bebd432c27de4a80cb/html5/thumbnails/68.jpg)
68
Robust Simulation of Hybrid Models
• Use our set-valued bounding theory to rigorously bound the states
k=2k=3
P 1
P 2
OperatingEnvelope
Robust Simulation of Hybrid Models
![Page 69: Robust Simulation of Hybrid Mechanistic and Machine ...](https://reader034.fdocuments.in/reader034/viewer/2022051606/628088bebd432c27de4a80cb/html5/thumbnails/69.jpg)
69
Relaxations of Activation Functions
69
log(1 + exp 𝑥𝑥 ) max(𝑥𝑥, tanh 𝑥𝑥 )max 𝑥𝑥,1
1 + exp(−𝑥𝑥)
Robust Simulation of Hybrid Models
![Page 70: Robust Simulation of Hybrid Mechanistic and Machine ...](https://reader034.fdocuments.in/reader034/viewer/2022051606/628088bebd432c27de4a80cb/html5/thumbnails/70.jpg)
70
Relaxations of Activation Functions
70
𝑥𝑥1 + 𝑥𝑥
11 + exp(−𝑥𝑥) 1 −
21 + exp(−𝑥𝑥)
Robust Simulation of Hybrid Models
![Page 71: Robust Simulation of Hybrid Mechanistic and Machine ...](https://reader034.fdocuments.in/reader034/viewer/2022051606/628088bebd432c27de4a80cb/html5/thumbnails/71.jpg)
71
Conclusion• Developed rigorous bounding theory for steady-state
and dynamical systems for mechanistic models– Formal uncertainty quantification– Extremely powerful open-source deterministic global
optimizer for advanced user-defined models• Want to exploit hybrid modeling approaches to
overcome challenges with pure mechanistic and pure data-driven approaches
![Page 72: Robust Simulation of Hybrid Mechanistic and Machine ...](https://reader034.fdocuments.in/reader034/viewer/2022051606/628088bebd432c27de4a80cb/html5/thumbnails/72.jpg)
72
Conclusion
• Applied global optimization for training a physics-informed data-driven model to demonstrate the tradeoff
• Preliminary work on bounding a library of common basis functions for NN– Enable rigorously bounding hybrid models– Formal uncertainty quantification of hybrid models
![Page 73: Robust Simulation of Hybrid Mechanistic and Machine ...](https://reader034.fdocuments.in/reader034/viewer/2022051606/628088bebd432c27de4a80cb/html5/thumbnails/73.jpg)
73
Any Questions?
This material is based upon work supported by the National Science Foundation under Grant No.: 1706343, 1932723. Any opinions, findings,
and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the National Science
Foundation.
Matthew WilhelmPhD Candidate
Chenyu WangPhD Candidate
THANK YOU