Post on 23-Jun-2020
Verification and SynthesisAlgorithms for Safe Autonomy
Chuchu Fan
Chuchu Fan ⋅ UIUC
2
Pictures from Google Images
Chuchu Fan ⋅ UIUC
Autonomy: moonshot goals and safety challenges
3
Chuchu Fan ⋅ UIUC
4
Chuchu Fan ⋅ UIUC
Actcomputer, engine, steering, brake
Decision and Controlnavigation, path planning, physics, code
Sensing and Perceptioncamera, LIDAR, GPS, computer vision,
machine learning, neural networks, data
5
6Video from Prof. Shashua at World Knowledge Forum’17: Platform for Safe and Scalable AVs
Chuchu Fan ⋅ UIUC
How to assure an autonomous system works correctly
7
Storro
wD
rive
NO
RTH
Ch
amp
aign
57
SO
UTH
Chuchu Fan ⋅ UIUC 8
With perfect information, a design engineer can simulate forward to check safety
But there are sources of uncertainty:o Estimation error in position/speed of other cars o Models of other cars and drivers
In principle, no finite number of simulations or tests can cover all possible corner cases of the system and give absolute correctness assurances
How to assure an autonomous system works correctly
Chuchu Fan ⋅ UIUC
In general, many other sources of uncertainty
o Road frictiono Failures, e.g., flat tireo Distracted drivers
“In practice, 30 billion miles of test driving is needed to achieve acceptable levels of assurance!” [Koopman, CMU] [Shashua, CTO Mobileye]
We need methods to across scenarios
We need principles to smartly direct testing and simulation resources to quickly find any incorrect behavior or “bugs”
9
How to assure an autonomous system works correctly
provide coverage guarantees
10
Formal verification can
System model ARequirement (property) R
Formal verification
provide coverage guarantees
yes: all behaviors of Asatisfy R
11
System model ARequirement (property) R
Formal verification
no: find a behavior of A(“bug”) that violates R
yes: all behaviors of Asatisfy R
Formal verification can provide coverage guarantees
Algorithms with formal guarantees for verification and control synthesis for hybrid systems
Theory & algorithms
Tools
Applications
Dry R
[CAV16][TACAS15] [HSCC18]
C2E2
[IEEE DT16]
My research: design and verification for safeautonomous systems
[CAV18][CAV15][IEEE DT18]
12
[CAV14,17][ATVA15][EMSOFT16][NAHS17][FM18][TECS18][CAV18]
Z3 Yices CVC4
[CDC17]
Chuchu Fan ⋅ UIUC
Physical plant
Autonomous systems verification challenge
13
Dubins’ car model
ሶ𝛿 = 𝑣𝛿
ሶ𝜓 =𝑣
𝑙tan 𝛿
ሶ𝑣 = 𝑎
ሶ𝑠𝑥 = 𝑣 cos 𝜓
ሶ𝑠𝑦 = 𝑣 sin(𝜓)
Steering angle
Heading angle
Speed
Horizontal positionሶ𝑥 = 𝑓 𝑥, 𝑢
𝑥 = [𝑣, 𝑠𝑥, 𝑠𝑦 , 𝛿, 𝜓]
𝑢 = [𝑎, 𝑣𝛿]
State variables
Control inputs
Nonlinear dynamics
Vertical position
System dynamics
Generally, nonlinear ODEs do not have closed-form solutions!
𝑥 𝑡 + 1 = 𝑓 𝑥 𝑡 , 𝑢[𝑡]
Chuchu Fan ⋅ UIUC 14
Nonlinear hybrid dynamics
Autonomous systems verification challenge
merge left
cruise
merge right
speed up
close to front car
slow down
speeding
Decision and control software Physical plant
ሶ𝑥 = 𝑓 𝑥, 𝑢
𝑥 = [𝑣, 𝑠𝑥, 𝑠𝑦 , 𝛿, 𝜓]
𝑢 = [𝑎, 𝑣𝛿]
State variables
Control inputs
System dynamics
Chuchu Fan ⋅ UIUC
Decision and control software
15
Nonlinear hybrid dynamics
merge left
cruise
merge right
speed up
close to front car
slow down
speedingabort
Autonomous systems verification challenge
Physical plant
ሶ𝑥 = 𝑓 𝑥, 𝑢
𝑥 = [𝑣, 𝑠𝑥, 𝑠𝑦 , 𝛿, 𝜓]
𝑢 = [𝑎, 𝑣𝛿]
State variables
Control inputs
System dynamics
Chuchu Fan ⋅ UIUC
Hybrid system model
16
Nonlinear hybrid dynamics
Physical plant
ሶ𝑥 = 𝑓𝑚𝑜𝑑𝑒 𝑥, 𝑢
merge leftሶ𝑥 = 𝑓𝑙𝑒𝑓𝑡(𝑥, 𝑢)
cruiseሶ𝑥 = 𝑓𝑐𝑟𝑠(𝑥, 𝑢)
merge rightሶ𝑥 = 𝑓𝑟𝑖𝑔ℎ𝑡(𝑥, 𝑢)
speed upሶ𝑥 = 𝑓𝑢𝑝(𝑥, 𝑢)
slow downሶ𝑥 = 𝑓𝑑𝑜𝑤𝑛(𝑥, 𝑢)
Autonomous systems verification challenge
Software
merge left
cruise
merge right
speed up
slow down
Interaction between computation and physics can lead to unexpected behaviors
Speed up
Cruise
The engine of the car is stable in each of the modes
Is it possible to switch between them to make the
system unstable?
Interaction between computation and physics can lead to unexpected behaviors
Yes! By switching between them the system becomes unstable
Interaction between computation and physics can lead to unexpected behaviors
Speed up
Cruise
Hybrid system modelPhysical plant
𝑑𝑥
𝑑𝑡= 𝑓 𝑥, 𝑢
merge left𝑑𝑥
𝑑𝑡= 𝑓𝑙𝑒𝑓𝑡(𝑥, 𝑢)
cruise𝑑𝑥
𝑑𝑡= 𝑓𝑐𝑟𝑠(𝑥, 𝑢)
merge right𝑑𝑥
𝑑𝑡= 𝑓𝑟𝑖𝑔ℎ𝑡(𝑥, 𝑢)
speed up𝑑𝑥
𝑑𝑡= 𝑓𝑢𝑝(𝑥, 𝑢)
slow down𝑑𝑥
𝑑𝑡= 𝑓𝑑𝑜𝑤𝑛(𝑥, 𝑢)
Decision and control software
But verifying hybrid systems is very hard!
We can model Autonomous systems as hybrid systems
SynthesisOutline Verification
Theory & algorithms
Tools
Applications
Dry RC2E2
Scalable control synthesis for large dimensional linear systems
?
[EMSOFT16][TECS18]
[CAV16][TACAS15] [HSCC18] [CAV18]
Nonlinear hybrid systems
Verification of incomplete models[CAV17]
[CAV18]
20
[IEEE DT15] [CAV18][CAV15][IEEE DT18]
Chuchu Fan ⋅ UIUC
The bounded safety verification problem
21
𝑥(0)
𝑥(𝑡)
Consider an autonomous system model ሶ𝑥 = 𝑓 𝑥
Trajectory 𝑥(𝑡): state at 𝑡 from 𝑥(0)
Chuchu Fan ⋅ UIUC
The bounded safety verification problem
22
Consider an autonomous system model ሶ𝑥 = 𝑓 𝑥
Trajectory 𝑥(𝑡): state at 𝑡 from 𝑥(0)
Given start set 𝑆, unsafe set 𝑈, time bound 𝑇,
𝑆
𝑈
Chuchu Fan ⋅ UIUC 23
𝑆
𝑅𝑒𝑎𝑐ℎ 𝑆, 𝑇
Consider an autonomous system model ሶ𝑥 = 𝑓 𝑥
Trajectory 𝑥(𝑡): state at 𝑡 from 𝑥(0)
Given start set 𝑆, unsafe set 𝑈, time bound 𝑇, let 𝑅𝑒𝑎𝑐ℎ(𝑆, 𝑇) be the set of reachable states from 𝑆 up to 𝑇
Bounded safety verification problem:
𝑅𝑒𝑎𝑐ℎ(𝑆, 𝑇) ∩ 𝑈 = empty ?
Computing the reach set is hard!
𝑈
The bounded safety verification problem
Chuchu Fan ⋅ UIUC
Brief history of hybrid system verification
24
90’s: modeling frameworks and early theoretical results on computing unbounded time exact reach seto Hybrid I/O Automata framework [Lynch 96]; A unified framework for hybrid control [Mitter 98]
o Decidable for timed automata [Alur and Dill 92]
o Undecidable even for constant dynamics [Henzinger 95]
Late 90’- Now: approximate bounded time reachability for linear hybrid systemso Polytopes [Henzinger 97], ellipsoids [Kurzhanski 97], zonotopes [Girard 05], support functions
[Frehse 08]
o Predicate abstraction [Alur 03], CEGAR [Clarke 03] [Prabhakar 13, 15], Generalized star [Bak 17]
This talk: Data-driven verification for nonlinear and black-box systemso Use simulation data to algorithmically construct formal proofs
o Generalize single simulation using rigorous sensitivity analysis [Fan 15-18]
Chuchu Fan ⋅ UIUC
Basic idea: Generalize single simulation using rigorous sensitivity analysis
Lets say that the distance of the trajectories can be bounded in terms of the distance of the initial states
This is called sensitivity of 𝑥(𝑡) to 𝑥(0)
25
𝑥(0)
Discrepancy formalizes sensitivity𝑥(𝑡)
Chuchu Fan ⋅ UIUC 26
𝑥1(0)
𝑥2(0)
𝑥2 𝑡𝑥1(𝑡) ≤ 𝛽
Definition: Discrepancy is a continuous function 𝛽 that
(1) Bounds the distance between neighboring trajectories
𝑥1 𝑡 − 𝑥2(𝑡) ≤ 𝛽 𝑥1 0 − 𝑥2 0 , 𝑡
Discrepancy formalizes sensitivity
Basic idea: Generalize single simulation using rigorous sensitivity analysis
Chuchu Fan ⋅ UIUC
Discrepancy formalizes sensitivity:
Definition: Discrepancy is a continuous function 𝛽 that
(1) Bounds the distance between neighboring trajectories
𝑥1 𝑡 − 𝑥2(𝑡) ≤ 𝛽 𝑥1 0 − 𝑥2 0 , 𝑡
(2) Error can be driven down:
𝛽 𝑥1 0 − 𝑥2 0 , 𝑡 → 0 as 𝑥1 0 − 𝑥2 0 → 0
27
𝑥1(0)
𝑥2(0)
𝑥2 𝑡𝑥1(𝑡) ≤ 𝛽
Basic idea: Generalize single simulation using rigorous sensitivity analysis
Chuchu Fan ⋅ UIUC
A simple example of a discrepancy function
If 𝑓(𝑥) has a Lipschitz constant 𝐿:
∀𝑥, 𝑦 ∈ ℝ𝑛, 𝑓 𝑥 − 𝑓 𝑦 ≤ 𝐿 𝑥 − 𝑦
then a discrepancy function is:
𝑥1 𝑡 − 𝑥2(𝑡) ≤ 𝑥1(0) − 𝑥2(0) 𝑒𝐿𝑡
28
𝛽 𝑥1(0) − 𝑥2(0) , 𝑡
Chuchu Fan ⋅ UIUC
A simple example of a bad discrepancy function
If 𝑓(𝑥) has a Lipschitz constant 𝐿:
∀𝑥, 𝑦 ∈ ℝ𝑛, 𝑓 𝑥 − 𝑓 𝑦 ≤ 𝐿 𝑥 − 𝑦
then a bad discrepancy function is:
𝑥1 𝑡 − 𝑥2(𝑡) ≤ 𝑥1(0) − 𝑥2(0) 𝑒𝐿𝑡
29
𝛽 𝑥1(0) − 𝑥2(0) , 𝑡
Related approaches: Lyapunov functions [Lyapunov 1892], Lyapunov exponents [Arnold 86], contraction metric [Slotine 96], etc.
Chuchu Fan ⋅ UIUC
What is a good discrepancy?
30
Exact reach set
Over-approximation
General: Applies to general nonlinear 𝑓
Accurate: Smaller error in 𝛽 => Fewer samples for verification
Effective: Computing 𝛽 is fast (in practice)
SynthesisOutline Verification
Theory & algorithms
Tools
Applications
Dry RC2E2
Scalable control synthesis for large dimensional linear systems
?
[EMSOFT16][TECS18]
[CAV16][TACAS15] [HSCC18] [CAV18]
Nonlinear hybrid systems
Verification of incomplete models[CAV17]
[CAV18]
31
[IEEE DT15] [CAV18][CAV15][IEEE DT18]
“Locally optimal reach set over-approximation for nonlinear hybrid systems.” EMSOFT2016. [Best paper finalist]
“Simulation-Driven Reachability Using Matrix Measures.” TECS 2018. [Invited paper]
Chuchu Fan ⋅ UIUC
To find the sensitivity of 𝑥(𝑡) to 𝑥(0)
We have to see how 𝑓 𝑥(𝑡) changes to 𝑓 𝑥 𝑡 + 𝜖
This is 𝜕𝑓 𝑥
𝜕𝑥, the Jacobian matrix 𝐽(𝑥) of 𝑓
We can prove d
dt𝑥2 𝑡 − 𝑥1 𝑡 =
න0
1
𝐽 (1 − 𝑠)𝑥1 + 𝑠𝑥2 𝑑𝑠 𝑥2 𝑡 − 𝑥1 𝑡
For ሶ𝑥 = 𝐴𝑥, matrix measure 𝜇 𝐴 ≜ lim𝑡→0+
𝐼+𝑡𝐴 −1
𝑡is the growth rate of log 𝑥
So 𝜇 𝐽 ⋅ gives the growth rate of log 𝑥1 𝑡 − 𝑥2 𝑡 , which gives the theorem:
32
𝑥1(0)
𝑥2(0)
𝛽
𝒟
𝑥1(𝑡)
𝑥2(𝑡)
Matrix measure of Jacobian can be used to compute discrepancy
Chuchu Fan ⋅ UIUC
Theorem [EMSOFT16]: Over a compact set 𝒟:
𝑥1 𝑡 − 𝑥2 𝑡 ≤ 𝑥1 0 − 𝑥2 0 𝑒𝑐𝑡,
where 𝑐 = max𝑥∈𝒟
𝜇 𝐽 𝑥
𝐿 = max𝑥∈𝒟
𝐽 𝑥 is a Lipschitz constant
Property: 𝜇 𝐽 𝑥 ≤ 𝐽 𝑥
But 𝜇 𝐽 𝑥 has no closed-form solutions!
We bound 𝐽 𝑥 over 𝒟 via interval matrices and estimate 𝜇 𝐽 𝑥 efficiently, while preserving the properties (1) and (2) of discrepancy
33
𝑥1(0)
𝑥2(0)
𝛽
𝒟
𝑥1(𝑡)
𝑥2(𝑡)
𝑐 𝐿
Matrix measure of Jacobian can be used to compute discrepancy
From symbolic 𝐽 𝑥 to numerical matrices
[∗,∗] ⋯ [∗,∗]
⋮ min𝑥∈𝒟
𝐽𝑖𝑗 𝑥 ,max𝑥∈𝒟
𝐽𝑖𝑗 𝑥 ⋮
[∗,∗] ⋯ [∗,∗]
(In practice 𝑐 ≪ 𝐿!)
Chuchu Fan ⋅ UIUC
Theorem [EMSOFT16]: Over a compact set 𝒟:
𝑥1 𝑡 − 𝑥2 𝑡 ≤ 𝑥1 0 − 𝑥2 0 𝑒𝑐𝑡,
where 𝑐 = max𝑥∈𝒟
𝜇 𝐽 𝑥
How should we choose the optimal norm ⋅ and why that matters?
34
𝑥1(0)
𝑥2(0)
𝛽
𝒟
𝑥1(𝑡)
𝑥2(𝑡)
Matrix measure of Jacobian can be used to compute discrepancy
Chuchu Fan ⋅ UIUC
2-norm bounds 𝑅𝑒𝑎𝑐ℎ(𝑆, 𝑇) along all directions uniformly (too conservative)
Instead, with coordinate transformation 𝑷 (ellipsoids) we can bound 𝑅𝑒𝑎𝑐ℎ(𝑆, 𝑇) more accurately
Theorem [EMSOFT16]: we can find the bestellipsoidal bound for 𝑅𝑒𝑎𝑐ℎ(𝑆, 𝑇) by solving a semi-definite program (SDP);
Solution gives the smallest 𝒄 for the discrepancy𝑃 𝑥1 0 − 𝑥2 0
2𝑒𝑐𝑡
35
𝒟𝑥1(0)
𝑥2(0)
𝑥1(𝑡)
𝑥2(𝑡)
How should we choose the optimal norm ⋅ and why that matters?
Chuchu Fan ⋅ UIUC 36
How should we choose the optimal norm ⋅ and why that matters?
Overall locally optimal reachability algorithm
Chuchu Fan ⋅ UIUC 37
How should we choose the optimal norm ⋅ and why that matters?
First locally optimal reachability analysis for nonlinear hybrid system verification
This minimizes the number of simulations or samples needed for verification
Asymptotic convergence: Approximation error goesto 0 as time goes to infinity, for contractive systems.
Chuchu Fan ⋅ UIUC 38
Data-driven verification using discrepancy function
Bounded safety verification problem:
𝑅𝑒𝑎𝑐ℎ(𝑆, 𝑇) ∩ 𝑈 = empty ? 𝑆
𝑈
Chuchu Fan ⋅ UIUC 39
Data-driven verification using discrepancy function
Bounded safety verification problem:
𝑅𝑒𝑎𝑐ℎ(𝑆, 𝑇) ∩ 𝑈 = empty ?
1. Simulate from the center
𝑆
𝑈
Chuchu Fan ⋅ UIUC 40
Data-driven verification using discrepancy function
Bounded safety verification problem:
𝑅𝑒𝑎𝑐ℎ(𝑆, 𝑇) ∩ 𝑈 = empty ?
1. Simulate from the center
2. Generalize simulation using discrepancy
3. Check safety
𝑆
𝑈
Chuchu Fan ⋅ UIUC 41
Data-driven verification using discrepancy function
Bounded safety verification problem:
𝑅𝑒𝑎𝑐ℎ(𝑆, 𝑇) ∩ 𝑈 = empty ?
1. Simulate from the center
2. Generalize simulation using discrepancy
3. Check safety
4. Refine to make over-approximations more accurate
𝑆
𝑈
𝛽 computed using matrix measure, intervals matrices, and optimal ellipsoids, etc., preserve the 2 properties
Chuchu Fan ⋅ UIUC 42
Data-driven verification using discrepancy function
Bounded safety verification problem:
𝑅𝑒𝑎𝑐ℎ(𝑆, 𝑇) ∩ 𝑈 = empty ?
1. Simulate from the center
2. Generalize simulation using discrepancy
3. Check safety
4. Refine to make over-approximations more accurate
𝑆
𝑈
5. Repeat until find an counter-example or all covers are safe
Chuchu Fan ⋅ UIUC
Theoretical guarantees of data-driven verification
43
Soundness: If the data-driven verification algorithm returns safe, then thesystem is indeed safe. If the algorithm returns unsafe, then the counter-exampleis indeed unsafe
Relative completeness: If the system is robustly safe or robustly unsafe, then thedata-driven verification algorithm will terminate with correct answer.
Chuchu Fan ⋅ UIUC
Theoretical guarantees of data-driven verification
44
Soundness: If the data-driven verification algorithm returns safe, then thesystem is indeed safe. If the algorithm returns unsafe, then the counter-exampleis indeed unsafe
Relative completeness: If the system is robustly safe or robustly unsafe, then thedata-driven verification algorithm will terminate with correct answer.
Simulation data + sensitivity (discrepancy) from models => automatic safety proofs!
C2E2
“Automatic reachability analysis for nonlinear hybrid models with C2E2.” CAV 2016.
“Meeting a powertrain verification challenge.” CAV 2015.45
Chuchu Fan ⋅ UIUC 46
Verify scenario with uncertainties like
speeds in [90, 100] km/h and overtake
distance in [20, 30] m
Safety requirement: separation of 1 m
maintained always
Verify the overtake scenario in C2E2
merge left
cruise
merge right
speed up
close to front car
slow down
speedingabort
C2E2 generated safety proof
Unsafe region
47
CarSim simulator connected
to C2E2
C2E2 finds bug: Blue accelerates as white returns to right lane
counter-example visualized
48
CarSim simulator connected
to C2E2
Chuchu Fan ⋅ UIUC
C2E2 application: powertrain control
A suite of powertrain verification benchmarks presented in HSCC 2014 by Toyota [Jin14]
5 dimensional hybrid system with polynomial differential equations Power
Startup
Sensorfail
Normal
ሶ𝜃 = 10(𝜃𝑖𝑛 − 𝜃)
ሶ𝑝 = 𝑐1(2𝜃 𝑐20𝑝2 + 𝑐21𝑝 + 𝑐22 − 𝑐12(𝑐2 + 𝑐3𝜔𝑝 + 𝑐4𝜔𝑝
2 + 𝑐5𝜔𝑝2))
ሶ𝜆 = 𝑐26(𝑐15 + 𝑐16𝑐25𝐹𝑐 + 𝑐17𝑐252 𝐹𝑐
2 + 𝑐18 ሶ𝑚𝑐 + 𝑐19 ሶ𝑚𝑐𝑐25𝐹𝑐 − 𝜆)
ሶ𝑝𝑒 = 𝑐1 2𝑐23𝜃 𝑐20𝑝2 + 𝑐21𝑝 + 𝑐22 − 𝑐2 + 𝑐3𝜔𝑝 + 𝑐4𝜔𝑝
2 + 𝑐5𝜔𝑝2
ሶ𝑖 = 𝑐14(𝑐24𝜆 − 𝑐11)
Normal
49
Controller
Water temp.
Crank Angle
AFR sensor
Sensor inputs
Throttle Angle
Fuel Injection
Spark Timing
HCCatalyst
Chuchu Fan ⋅ UIUC
C2E2 application: powertrain control
A suite of powertrain verification benchmarks presented in HSCC 2014 by Toyota [Jin14]
5 dimensional hybrid system with polynomial differential equations
Verified the model with a set of driver behavior profiles (running time approx. 12 mins) [CAV15]
- First verification of Toyota powertrain model
- Best verification result award at CPSWeek’15
50
Power
Startup
Sensorfail
Normal
SynthesisOutline Verification
Theory & algorithms
Tools
Applications
Dry RC2E2
Scalable control synthesis for large dimensional linear systems
?
[EMSOFT16][TECS18]
[CAV16][TACAS15] [HSCC18] [CAV18]
Nonlinear hybrid systems
Incomplete models [CAV17][IEEE DT18] [CAV18]
51
[IEEE DT15] [CAV18][CAV15][IEEE DT18]
“DryVR: Data-driven verification and compositional reasoning for automotive systems.” CAV 2017.
“Data-driven formal reasoning and their applications in safety analysis of vehicle autonomy features.” IEEE Design & Test, 2018.
Chuchu Fan ⋅ UIUC 52
Mode cruise
Nonlinear hybrid dynamics
Incomplete model?
Challenge: incomplete model
Hybrid model
merge left
cruise
merge right
speed up
close to front car
slow down
speeding
abort
53
Gain serenity, analyze real-world models
“All models are wrong, some are useful”
https://github.com/cfan10/DryVR
54
: A new model semantics for hybrid systems
Control graph
+
Black-box simulator
=
Hybrid system
Complete information of switching structure
Executable access to mode dynamics
DryVR’s Executable hybrid model
Chuchu Fan ⋅ UIUC 55
Global exponential discrepancy
𝑥1 𝑡 − 𝑥2 𝑡 ≤ 𝑥1 0 − 𝑥2 0 𝑒𝑎𝑡+𝑏
Taking log: ∀𝑡, ln𝑥1 𝑡 −𝑥2 𝑡
𝑥1 0 −𝑥2 0≤ 𝑎𝑡 + 𝑏
By setting the initial values 𝑥1 0 , 𝑥2 0 , the black-box simulator can return sampled simulation trajectories 𝑥1 𝑡 , 𝑥2 𝑡
Find 𝑎 and 𝑏 by learning a linear separator
Similar idea works for other formats of the discrepancy function
Also works for other machine learning algorithms
ln𝑥𝑖 𝑡 − 𝑥𝑗 𝑡
𝑥𝑖 0 − 𝑥𝑗 0
𝑡
𝑎𝑡 + 𝑏
: Learn discrepancy from data
Chuchu Fan ⋅ UIUC 56
Theorem [CAV17]: Given the training set, the global exponential discrepancy function that gives the tightest reach set over-approximation can be found by solving a Linear programming (LP) problem
Proposition [CAV17]: ∀𝜖, 𝛿 > 0, if sampling number 𝑛 ≥ 1
𝜖ln
1
𝛿, then with probability 1 − 𝛿, the
algorithm finds (𝑎, 𝑏) such that 𝑒𝑟𝑟𝒟 𝛾, 𝑘 < 𝜖. [PAC learning, Valiant 1984]
o For example: in theory, set 𝜖 = 0.1%, 𝛿 = 1%, 𝑛 ≈ 4600
In practice, we show on >20 systems that it is enough to use 𝑛 = 20 training trajectories
Intuition: the algorithm learned the worst case global exponential sensitivity from the
training trajectories
: Learn discrepancy from data
Chuchu Fan ⋅ UIUC 57
𝐺1 𝐺2≼
𝑅𝑒𝑎𝑐ℎ ⊆ 𝑅𝑒𝑎𝑐ℎ
: Formal reasoning
Find a control graph for given requirements
Simulation relation of control graph for behavior containment
merge left
reach threshold
gain threshold
overtake
merge right
Possible modes:merge left, accelerate, merge right…
Chuchu Fan ⋅ UIUC 58
𝐺1 𝐺2≼
𝑅𝑒𝑎𝑐ℎ ⊆ 𝑅𝑒𝑎𝑐ℎ
: Formal reasoning
Find a control graph for given requirements
Simulation relation of control graph for behavior containment
merge left
reach threshold
gain threshold
overtake
merge right
Possible modes:merge left, accelerate, merge right…
DryVR can verify systems that combine white + black-box modules
by combining formal techniques with learned discrepancy
Chuchu Fan ⋅ UIUC
Verification of spacecraft rendezvous maneuver
Autonomous spacecraft rendezvous, proximity operations, and docking (ARPOD) problem developed by Air Force Research Lab as a challenge problem for verification and controller synthesis [Jewison & Erwin 2016]
Complex, hybrid dynamical system with uncertainties using third-party controllers [Chan 2017][Sanfelice 2017]
3 hour long missions verified in 10 minutes
Learning curve for an undergraduate student was about a month
59
C2E2 result with a LQR controller [Chan 2017]
DryVR result with a robust supervisor controller [Sanfelice 2017]
Chuchu Fan ⋅ UIUC
Risk analysis [IEEE DT17]
Case study: Automatic Emergency Braking
Determining Automotive Safety Integrity Levels (ASIL) mandated by the ISO26262 standard
ASIL risk = probability × severity
60
Verification of safe ADAS features
Chuchu Fan ⋅ UIUC
Risk analysis [IEEE DT17]
Case study: Automatic Emergency Braking
Determining Automotive Safety Integrity Levels (ASIL) mandated by the ISO26262 standard
ASIL risk = probability × severity
61
Verification of safe ADAS features
Chuchu Fan ⋅ UIUC
Compositional verification of networked system
62
ሶ𝑚1 = 𝑔 𝛽1, 𝛾1
ሶ𝑚2 = 𝑔 𝛽2, 𝛾2 …
discrepancy function for ሶ𝑥 = 𝑓(𝑥)
𝑥1 𝑡 − 𝑥2 𝑡 ≤ 𝛽 𝑥1 0 − 𝑥2 0 , 𝑡
Theorem [CAV15]: 𝑚 𝑡 bounds the divergence of original system’s trajectories
Arbitrary topologies and over-approximation error goes to 0 with refinement
This enables us to verify 𝑨 (which can be a network with complex topologies) using simulations of 𝑨 and 𝑩 and only discrepancy computation of each component of 𝑨
ሶ𝑦1 = 𝑓1 𝑦1, 𝑢1
ሶ𝑦2 = 𝑓2 𝑦2, 𝑢2 …
𝑢1 = 𝑦2, 𝑦3𝑢2 = 𝑦1, 𝑦2
…
3𝑛 variables 3 variables𝑥 =
𝑦1𝑦2𝑦3
Input-to-state
+න0
𝑡
𝛾 𝑢1 𝑠 − 𝑢2 𝑠 𝑑𝑠
, 𝑢)
system 𝐴 system 𝐵
Chuchu Fan ⋅ UIUC
Composition for networked autonomous system
63
𝑑𝑚1
𝑑𝑡= 𝑔 𝛽1, 𝛾1
dim(𝑚1) = 1
𝑑𝑚2
𝑑𝑡= 𝑔 𝛽2, 𝛾2
dim(𝑚2) = 1
𝑑𝑚3
𝑑𝑡= 𝑔(𝛽3, 𝛾3)
dim(𝑚3) = 1
With 𝑁 nodes, σ𝑖=1𝑁 𝑛𝑖 vs 𝑁 variables
Theorem [CAV15]: 𝑚 𝑡 bounds the divergence of original system’s trajectories
Arbitrary topologies
𝑚 𝑡 goes to 0 as size of the start set goes to 0
𝑑𝑦1𝑑𝑡
= 𝑓1 𝑦1, 𝑢1
dim(𝑦1) = 𝑛1
𝑑𝑦2𝑑𝑡
= 𝑓2 𝑦2, 𝑢2
dim(𝑦2) = 𝑛2
𝑑𝑦3𝑑𝑡
= 𝑓3(𝑦3, 𝑢3)
dim(𝑦3) = 𝑛3
𝑢1 = 𝑦2, 𝑦3𝑢2 = 𝑦1, 𝑦2
𝑢3 = 𝑦1, 𝑦2
IS-Discrepancy + our composition theorem => scalable automatic proofs for networked systems
What’s ahead?
64
Other works in
verification64
Theory & algorithms
Tools
Applications
Dry RC2E2 First verification tool for
nonlinear hybrid systems with guarantees
A significant step towards real-world verification
Risk analysis of ADAS for ISO 26262
Award-winningverification result for Toyota powertrain system
Verification for AS/CPS is an important but hard problemSimulation + sensitivity (discrepancy) give proofsFirst optimal reachability analysis for nonlinear hybrid dynamicsFirst hybrid model as white + black box with learned discrepancyCompositional analysis with designed or learned components
Uncertain InputsMixed-signal circuits
[ADHS18]Drones with RL[Sibai HSCC18]
Parallel landing[Duggirala EMSOFT 17]
Compositional analysisPacemaker[IEEE DT15]
𝑠𝑥 𝑠𝑦
Drones with RL[Sibai HSCC18]
Theory & algorithms
Tools
Applications
Dry RC2E2
65
First data-driven verification tool for nonlinear hybrid systems with guarantees
A significant step towards real-world verification
Other works in
verification Mixed-signal circuits [ADHS18]
Parallel landing[Duggirala EMSOFT 17]
Pacemaker[IEEE DT15]
Verification for AS/CPS is an important but hard problemSimulation + sensitivity (discrepancy) give proofsFirst optimal reachability analysis for nonlinear hybrid dynamicsFirst hybrid model as white + black box with learned discrepancyCompositional analysis with designed or learned components
��
Chuchu Fan ⋅ UIUC
Many thanks to my collaborators
66
Prof. Mitra’s group
Ulrich SchmidTU Wien
Mahesh ViswanathanUIUC
Marta KwiatkowskaUniversity of Oxford
Xiaoqing JinApple
Jim KapinskiToyota
Umang MathurUIUC
Scott SmolkaStony Brook University