Model-Based Verification of Automotive Controllers -...
Transcript of Model-Based Verification of Automotive Controllers -...
![Page 1: Model-Based Verification of Automotive Controllers - …cmacs.cs.cmu.edu/presentations/april12/Cleaveland_cmacs-2012-04 … · Model-Based Verification of Automotive Controllers Rance](https://reader034.fdocuments.in/reader034/viewer/2022052608/5a791d017f8b9a523d8d2ce9/html5/thumbnails/1.jpg)
© 2012 University of Maryland and Fraunhofer USA
Model-Based Verification of
Automotive Controllers
Rance Cleaveland, PhD
Professor of Computer Science, University of Maryland
and
Executive & Scientific Director, Fraunhofer USA Center for
Experimental Software Engineering (CESE)
![Page 2: Model-Based Verification of Automotive Controllers - …cmacs.cs.cmu.edu/presentations/april12/Cleaveland_cmacs-2012-04 … · Model-Based Verification of Automotive Controllers Rance](https://reader034.fdocuments.in/reader034/viewer/2022052608/5a791d017f8b9a523d8d2ce9/html5/thumbnails/2.jpg)
© 2012 University of Maryland and Fraunhofer USA
This Talk
• Model-based validation
– … of automotive software product lines
– … using instrumentation-based verification
• Talk structure
– Modeling in automotive software development
– Instrumentation-based verification
– Product lines
– An approach to product-line validation
– Conclusions
2
![Page 3: Model-Based Verification of Automotive Controllers - …cmacs.cs.cmu.edu/presentations/april12/Cleaveland_cmacs-2012-04 … · Model-Based Verification of Automotive Controllers Rance](https://reader034.fdocuments.in/reader034/viewer/2022052608/5a791d017f8b9a523d8d2ce9/html5/thumbnails/3.jpg)
© 2012 University of Maryland and Fraunhofer USA
Automotive Software
• Driver of innovation
90% of new feature content based on software [GM]
• Rising cost
20% of vehicle cost [Conti], 50% for hybrids [Toyota]
• Warranty, liability, quality
High-profile recalls in Germany, Japan, US
3
![Page 4: Model-Based Verification of Automotive Controllers - …cmacs.cs.cmu.edu/presentations/april12/Cleaveland_cmacs-2012-04 … · Model-Based Verification of Automotive Controllers Rance](https://reader034.fdocuments.in/reader034/viewer/2022052608/5a791d017f8b9a523d8d2ce9/html5/thumbnails/4.jpg)
© 2012 University of Maryland and Fraunhofer USA
A Grand Challenge
• Ensure high quality of automotive software
– ... preserving time to market
– … at reasonable cost
• Key approach: Model-Based Development (MBD)
4
![Page 5: Model-Based Verification of Automotive Controllers - …cmacs.cs.cmu.edu/presentations/april12/Cleaveland_cmacs-2012-04 … · Model-Based Verification of Automotive Controllers Rance](https://reader034.fdocuments.in/reader034/viewer/2022052608/5a791d017f8b9a523d8d2ce9/html5/thumbnails/5.jpg)
© 2012 University of Maryland and Fraunhofer USA
Traditional Software Development
Requirements / specs /
designs / test plans / etc.
Source code
?
5
![Page 6: Model-Based Verification of Automotive Controllers - …cmacs.cs.cmu.edu/presentations/april12/Cleaveland_cmacs-2012-04 … · Model-Based Verification of Automotive Controllers Rance](https://reader034.fdocuments.in/reader034/viewer/2022052608/5a791d017f8b9a523d8d2ce9/html5/thumbnails/6.jpg)
© 2012 University of Maryland and Fraunhofer USA
Model-Based Development
Use models (MATLAB® / Simulink®) as designs / specs
Requirements /
test plans / etc. Design / spec Source code
? ?
6
![Page 7: Model-Based Verification of Automotive Controllers - …cmacs.cs.cmu.edu/presentations/april12/Cleaveland_cmacs-2012-04 … · Model-Based Verification of Automotive Controllers Rance](https://reader034.fdocuments.in/reader034/viewer/2022052608/5a791d017f8b9a523d8d2ce9/html5/thumbnails/7.jpg)
© 2012 University of Maryland and Fraunhofer USA
Model-Based Development (cont.)
Requirements
System test
Design
Specifications
Unit test
Implementation
Final test
models
models
Main motivation: autocode! Also:• Models support V&V, testing, communication among engineers• Models can be managed electronically
7
![Page 8: Model-Based Verification of Automotive Controllers - …cmacs.cs.cmu.edu/presentations/april12/Cleaveland_cmacs-2012-04 … · Model-Based Verification of Automotive Controllers Rance](https://reader034.fdocuments.in/reader034/viewer/2022052608/5a791d017f8b9a523d8d2ce9/html5/thumbnails/8.jpg)
© 2012 University of Maryland and Fraunhofer USA
Simulink®
• Block-diagram modeling
language / simulator of
The MathWorks, Inc.
• Hierarchical modeling
• Continuous-time and
discrete-time simulation
• Used in MBD of control
software
8
![Page 9: Model-Based Verification of Automotive Controllers - …cmacs.cs.cmu.edu/presentations/april12/Cleaveland_cmacs-2012-04 … · Model-Based Verification of Automotive Controllers Rance](https://reader034.fdocuments.in/reader034/viewer/2022052608/5a791d017f8b9a523d8d2ce9/html5/thumbnails/9.jpg)
© 2012 University of Maryland and Fraunhofer USA
Stateflow®
9
![Page 10: Model-Based Verification of Automotive Controllers - …cmacs.cs.cmu.edu/presentations/april12/Cleaveland_cmacs-2012-04 … · Model-Based Verification of Automotive Controllers Rance](https://reader034.fdocuments.in/reader034/viewer/2022052608/5a791d017f8b9a523d8d2ce9/html5/thumbnails/10.jpg)
© 2012 University of Maryland and Fraunhofer USA
Reactis®
A model-based V&V tool from Reactive Systems, Inc.
Tester Generate tests from models (also C)
Simulator Run, fine-tune tests
Validator Validate models / C
Reactis /
Reactis for C
Simulink /
Stateflow /
CModel / code
10
![Page 11: Model-Based Verification of Automotive Controllers - …cmacs.cs.cmu.edu/presentations/april12/Cleaveland_cmacs-2012-04 … · Model-Based Verification of Automotive Controllers Rance](https://reader034.fdocuments.in/reader034/viewer/2022052608/5a791d017f8b9a523d8d2ce9/html5/thumbnails/11.jpg)
© 2012 University of Maryland and Fraunhofer USA 11
Generating Tests: Guided Simulation
Reactis systematically generates inputs to drive
simulation runs to cover model, produce test suites.
Reactis
Tester
Test Suitemodel
Generate
Extend
![Page 12: Model-Based Verification of Automotive Controllers - …cmacs.cs.cmu.edu/presentations/april12/Cleaveland_cmacs-2012-04 … · Model-Based Verification of Automotive Controllers Rance](https://reader034.fdocuments.in/reader034/viewer/2022052608/5a791d017f8b9a523d8d2ce9/html5/thumbnails/12.jpg)
© 2012 University of Maryland and Fraunhofer USA
Generated Test Data
12
![Page 13: Model-Based Verification of Automotive Controllers - …cmacs.cs.cmu.edu/presentations/april12/Cleaveland_cmacs-2012-04 … · Model-Based Verification of Automotive Controllers Rance](https://reader034.fdocuments.in/reader034/viewer/2022052608/5a791d017f8b9a523d8d2ce9/html5/thumbnails/13.jpg)
© 2012 University of Maryland and Fraunhofer USA
Ongoing Research
Design-time modeling, requirements verification
Requirements
Design
Specificationsmodels
models
13
![Page 14: Model-Based Verification of Automotive Controllers - …cmacs.cs.cmu.edu/presentations/april12/Cleaveland_cmacs-2012-04 … · Model-Based Verification of Automotive Controllers Rance](https://reader034.fdocuments.in/reader034/viewer/2022052608/5a791d017f8b9a523d8d2ce9/html5/thumbnails/14.jpg)
© 2012 University of Maryland and Fraunhofer USA
Instrumentation-Based Verification
• Model-validation technique supported by
Reactis
• Combines assertions in models, testing
14
![Page 15: Model-Based Verification of Automotive Controllers - …cmacs.cs.cmu.edu/presentations/april12/Cleaveland_cmacs-2012-04 … · Model-Based Verification of Automotive Controllers Rance](https://reader034.fdocuments.in/reader034/viewer/2022052608/5a791d017f8b9a523d8d2ce9/html5/thumbnails/15.jpg)
© 2012 University of Maryland and Fraunhofer USA
Instrumentation-Based Verification:
Requirements
• Automatic verification requires formalized requirements
• IBV: formalize requirements as monitor models
• Example“If speed is < 30, cruise control must remain inactive”
15
![Page 16: Model-Based Verification of Automotive Controllers - …cmacs.cs.cmu.edu/presentations/april12/Cleaveland_cmacs-2012-04 … · Model-Based Verification of Automotive Controllers Rance](https://reader034.fdocuments.in/reader034/viewer/2022052608/5a791d017f8b9a523d8d2ce9/html5/thumbnails/16.jpg)
© 2012 University of Maryland and Fraunhofer USA
Instrumentation-Based Verification: Checking
Requirements
• Instrument design model with monitors
• Use coverage testing to check for monitor violations
• Reactis:
– Separates instrumentation, design
– Automates test generation
16
![Page 17: Model-Based Verification of Automotive Controllers - …cmacs.cs.cmu.edu/presentations/april12/Cleaveland_cmacs-2012-04 … · Model-Based Verification of Automotive Controllers Rance](https://reader034.fdocuments.in/reader034/viewer/2022052608/5a791d017f8b9a523d8d2ce9/html5/thumbnails/17.jpg)
© 2012 University of Maryland and Fraunhofer USA
IBV Works
• Three-month case study with Tier-1 automotive supplier on production system
• Artifacts
– 300-page requirements document
– Some source code
• Results (intern)
– 62 requirements for 10 design features formalized as monitor models
– Requirements checked on feature models
– 11 inconsistencies in requirements identified
– Key technical insight: architecture for monitor models
17
![Page 18: Model-Based Verification of Automotive Controllers - …cmacs.cs.cmu.edu/presentations/april12/Cleaveland_cmacs-2012-04 … · Model-Based Verification of Automotive Controllers Rance](https://reader034.fdocuments.in/reader034/viewer/2022052608/5a791d017f8b9a523d8d2ce9/html5/thumbnails/18.jpg)
© 2012 University of Maryland and Fraunhofer USA
From Requirements to Monitors:
A Monitor Model Architecture
“[This] is the complete description of the
control of the CAN output signals can1
and can2 produced by Function A.
Function A can be activated only with in =
1. The activation takes place when either
the CAN bus messages a or b is
present….”
18
![Page 19: Model-Based Verification of Automotive Controllers - …cmacs.cs.cmu.edu/presentations/april12/Cleaveland_cmacs-2012-04 … · Model-Based Verification of Automotive Controllers Rance](https://reader034.fdocuments.in/reader034/viewer/2022052608/5a791d017f8b9a523d8d2ce9/html5/thumbnails/19.jpg)
© 2012 University of Maryland and Fraunhofer USA
Final Monitor Model Architecture
Need for conditional
requirements
– Behavior only
specified for certain
situations
– “If timeout occurs
do something”
19
![Page 20: Model-Based Verification of Automotive Controllers - …cmacs.cs.cmu.edu/presentations/april12/Cleaveland_cmacs-2012-04 … · Model-Based Verification of Automotive Controllers Rance](https://reader034.fdocuments.in/reader034/viewer/2022052608/5a791d017f8b9a523d8d2ce9/html5/thumbnails/20.jpg)
© 2012 University of Maryland and Fraunhofer USA
Software Product Lines
• (From SEI): product line = “a set of software-intensive systems that share a common, managed set of features satisfying the specific needs of a particular market segment or mission and that are developed from a common set of core assets in a prescribed way”
• Key terms
– Common assets
– Variation points
– Variants
20
![Page 21: Model-Based Verification of Automotive Controllers - …cmacs.cs.cmu.edu/presentations/april12/Cleaveland_cmacs-2012-04 … · Model-Based Verification of Automotive Controllers Rance](https://reader034.fdocuments.in/reader034/viewer/2022052608/5a791d017f8b9a523d8d2ce9/html5/thumbnails/21.jpg)
© 2012 University of Maryland and Fraunhofer USA
SPL in Automotive
• Toyota: 1,800 variants for engine control software
– Diesel vs. gas vs. hybrid
– Different emissions regulations
– Performance profiles for different markets
– # of cylinders
– Cruise control?
– Etc.
• Product lines offer a framework for streamlining development, maintenance
• What about V&V?
21
![Page 22: Model-Based Verification of Automotive Controllers - …cmacs.cs.cmu.edu/presentations/april12/Cleaveland_cmacs-2012-04 … · Model-Based Verification of Automotive Controllers Rance](https://reader034.fdocuments.in/reader034/viewer/2022052608/5a791d017f8b9a523d8d2ce9/html5/thumbnails/22.jpg)
© 2012 University of Maryland and Fraunhofer USA
Variants in Monitor Modeling
• Fine-grained product-line info often captured at model level
• How can functionality of product-line models be verified?
– Want to re-use verification effort
– Some requirements are universal (apply to every variant)
– Others are variant-specific
22
![Page 23: Model-Based Verification of Automotive Controllers - …cmacs.cs.cmu.edu/presentations/april12/Cleaveland_cmacs-2012-04 … · Model-Based Verification of Automotive Controllers Rance](https://reader034.fdocuments.in/reader034/viewer/2022052608/5a791d017f8b9a523d8d2ce9/html5/thumbnails/23.jpg)
© 2012 University of Maryland and Fraunhofer USA
Example: Cruise Control
• Product line could include following variants
– Maximum-speed restriction or not
– Adaptive or not
– Manual or automatic transmission
• Sample universal requirement
If the brake pedal is pressed, the cruise control shall become inactive.
• Sample variant-specific requirement
If the transmission is manual, then the cruise control shall become inactive if the desired speed is inconsistent with the current gear.
23
![Page 24: Model-Based Verification of Automotive Controllers - …cmacs.cs.cmu.edu/presentations/april12/Cleaveland_cmacs-2012-04 … · Model-Based Verification of Automotive Controllers Rance](https://reader034.fdocuments.in/reader034/viewer/2022052608/5a791d017f8b9a523d8d2ce9/html5/thumbnails/24.jpg)
© 2012 University of Maryland and Fraunhofer USA
How To Do V&V for Product-Line
Models?
• Use IBV!
• Result of industrial study
– Framework for modeling product lines in Simulink
– Strategy, architecture for variant-specific monitor-
models
– Use of IBV to debug models, find requirements
issues
24
![Page 25: Model-Based Verification of Automotive Controllers - …cmacs.cs.cmu.edu/presentations/april12/Cleaveland_cmacs-2012-04 … · Model-Based Verification of Automotive Controllers Rance](https://reader034.fdocuments.in/reader034/viewer/2022052608/5a791d017f8b9a523d8d2ce9/html5/thumbnails/25.jpg)
© 2012 University of Maryland and Fraunhofer USA
Product-Line Modeling
• Model file defines
control functionality
• Configuration file
defines parameters
• Some parameters
used to define which
variant is intended
Model file
“if num_cyl = 4 …”
Config file
“num_cyl = 6;”
25
![Page 26: Model-Based Verification of Automotive Controllers - …cmacs.cs.cmu.edu/presentations/april12/Cleaveland_cmacs-2012-04 … · Model-Based Verification of Automotive Controllers Rance](https://reader034.fdocuments.in/reader034/viewer/2022052608/5a791d017f8b9a523d8d2ce9/html5/thumbnails/26.jpg)
© 2012 University of Maryland and Fraunhofer USA
Pilot Study: Cruise Control
• Simulink model is in Reactis distribution
• Partner adapted it as sample product-line model
• Variants
– Max-speed limitation
– Adaptive
– Manual vs. automatic transmission
– Output interface
26
![Page 27: Model-Based Verification of Automotive Controllers - …cmacs.cs.cmu.edu/presentations/april12/Cleaveland_cmacs-2012-04 … · Model-Based Verification of Automotive Controllers Rance](https://reader034.fdocuments.in/reader034/viewer/2022052608/5a791d017f8b9a523d8d2ce9/html5/thumbnails/27.jpg)
© 2012 University of Maryland and Fraunhofer USA
Finalizing Product-Line Model in
Simulink / Stateflow
• Program variant selection
– Introduce parameters into model
– Define MATLAB variables for use as parameters
• Product line contained in two files
– cruise_variants.mdl (model)
– cruise_constants.m (MATLAB variables)
27
![Page 28: Model-Based Verification of Automotive Controllers - …cmacs.cs.cmu.edu/presentations/april12/Cleaveland_cmacs-2012-04 … · Model-Based Verification of Automotive Controllers Rance](https://reader034.fdocuments.in/reader034/viewer/2022052608/5a791d017f8b9a523d8d2ce9/html5/thumbnails/28.jpg)
© 2012 University of Maryland and Fraunhofer USA
cruise_variants.mdl
cruise_constants.m
Parameterized constant
MATLAB variable
28
![Page 29: Model-Based Verification of Automotive Controllers - …cmacs.cs.cmu.edu/presentations/april12/Cleaveland_cmacs-2012-04 … · Model-Based Verification of Automotive Controllers Rance](https://reader034.fdocuments.in/reader034/viewer/2022052608/5a791d017f8b9a523d8d2ce9/html5/thumbnails/29.jpg)
© 2012 University of Maryland and Fraunhofer USA
Variant-Specific Monitor Models
• Idea
– Configuration files define variant-selection parameters
– Why not refer to same parameters in monitor models to introduce variant-specificity?
• Pilot study
– Defined six example variant-specific requirements
– Translated each into monitor model
29
![Page 30: Model-Based Verification of Automotive Controllers - …cmacs.cs.cmu.edu/presentations/april12/Cleaveland_cmacs-2012-04 … · Model-Based Verification of Automotive Controllers Rance](https://reader034.fdocuments.in/reader034/viewer/2022052608/5a791d017f8b9a523d8d2ce9/html5/thumbnails/30.jpg)
© 2012 University of Maryland and Fraunhofer USA
Example
[MS1] If the maximum-speed limitation is enabled, the cruise control shall not permit the desired (set) speed to exceed a designated maximum value.
MATLAB variable30
![Page 31: Model-Based Verification of Automotive Controllers - …cmacs.cs.cmu.edu/presentations/april12/Cleaveland_cmacs-2012-04 … · Model-Based Verification of Automotive Controllers Rance](https://reader034.fdocuments.in/reader034/viewer/2022052608/5a791d017f8b9a523d8d2ce9/html5/thumbnails/31.jpg)
© 2012 University of Maryland and Fraunhofer USA
Monitor Model Logistics
• Monitor models
stored in single
Simulink library file
• Monitor models
refer to parameters
cruise_variants.mdl
Product-line model
cruise_variants_monitors.mdl
Monitor models
cruise_constants.m
Parameter file
Include variant-selectors
reads
reads
instrumented by
31
![Page 32: Model-Based Verification of Automotive Controllers - …cmacs.cs.cmu.edu/presentations/april12/Cleaveland_cmacs-2012-04 … · Model-Based Verification of Automotive Controllers Rance](https://reader034.fdocuments.in/reader034/viewer/2022052608/5a791d017f8b9a523d8d2ce9/html5/thumbnails/32.jpg)
© 2012 University of Maryland and Fraunhofer USA
Verification
• Product-line model
instrumented with
monitor models
• Coverage testing
used to check for
violations
• Reactis® used for
both tasks
32
![Page 33: Model-Based Verification of Automotive Controllers - …cmacs.cs.cmu.edu/presentations/april12/Cleaveland_cmacs-2012-04 … · Model-Based Verification of Automotive Controllers Rance](https://reader034.fdocuments.in/reader034/viewer/2022052608/5a791d017f8b9a523d8d2ce9/html5/thumbnails/33.jpg)
© 2012 University of Maryland and Fraunhofer USA
Verification Results
• Bugs found in product-line model (fixed)
• Bugs found in monitor model (fixed)
• Variant-interaction problem discovered
– One variant specified maximum speed
– Other variant specified speed-control by adaptive
mechanism
33
![Page 34: Model-Based Verification of Automotive Controllers - …cmacs.cs.cmu.edu/presentations/april12/Cleaveland_cmacs-2012-04 … · Model-Based Verification of Automotive Controllers Rance](https://reader034.fdocuments.in/reader034/viewer/2022052608/5a791d017f8b9a523d8d2ce9/html5/thumbnails/34.jpg)
© 2012 University of Maryland and Fraunhofer USA
This Talk
Model-based verification of software product
lines
– Model product lines in Simulink / Stateflow
– Variant specificity in monitor models
– Instrumentation-based verification
– Variant interactions!
34
![Page 35: Model-Based Verification of Automotive Controllers - …cmacs.cs.cmu.edu/presentations/april12/Cleaveland_cmacs-2012-04 … · Model-Based Verification of Automotive Controllers Rance](https://reader034.fdocuments.in/reader034/viewer/2022052608/5a791d017f8b9a523d8d2ce9/html5/thumbnails/35.jpg)
© 2012 University of Maryland and Fraunhofer USA
Larger Issues
• Single models vs. parameterized models
– Typical problem: find parameter settings that ensure
satisfaction of requirements
– Here: parameterize requirements, check consistency
of parameterized models vis a vis parameterized
requirements
• Parameter interactions
• Requirements are not the always what’s required
35
![Page 36: Model-Based Verification of Automotive Controllers - …cmacs.cs.cmu.edu/presentations/april12/Cleaveland_cmacs-2012-04 … · Model-Based Verification of Automotive Controllers Rance](https://reader034.fdocuments.in/reader034/viewer/2022052608/5a791d017f8b9a523d8d2ce9/html5/thumbnails/36.jpg)
© 2012 University of Maryland and Fraunhofer USA
Thank You!
Rance Cleaveland
University of Maryland / Fraunhofer USA CESE
+1 301-405-8572
36