Modeling Issues for Validation, Verification, and Certification (VV&C) Paul Miner NASA Langley...
-
Upload
marybeth-franklin -
Category
Documents
-
view
214 -
download
0
Transcript of Modeling Issues for Validation, Verification, and Certification (VV&C) Paul Miner NASA Langley...
Modeling Issues for Validation, Verification, and Certification
(VV&C)
Paul MinerNASA Langley Research Center
22 September 2015
• “Essentially, all models are wrong, but some are useful”– George Box
22 September 2015 Modeling for V&V 2
Example of Useful Models
• Canonical models for designing and analyzing digital hardware introduced in the mid-1950s
– Huffman, D.A., The synthesis of sequential switching circuits, The Journal of the Franklin Institute, 257(3):161-190, 1954
– Mealy, G.H., A method for synthesizing sequential circuits, Bell System Technical Journal, 34:1045-1079, September 1955
– Moore, E.F. Gedanken Experiments on Sequential Machines, in C. Shannon and J. McCarthy, editors, Automata Studies, Princeton University Press, 1956
• These modeling abstractions underpin the digital revolution– But, ..
• “There is no such thing as digital circuitry. There is only analog circuitry driven to extremes.”
– Unknown – quoted by Kevin Driscoll• https://c3.nasa.gov/dashlink/static/media/other/ObservedFailures6.html
• For VV&C, need to consider impact when modeling abstractions no longer hold
22 September 2015 Modeling for V&V 3
Role of Models in VV& C
Benefits• Explore system behavior earlier
in lifecycle• Ability to verify properties that
cannot be effectively demonstrated by test– E.g. Robust partitioning for
Integrated Modular Avionics– No memory leaks, buffer
overflows, etc.
• …
Risks• Invalid assumptions• Unstated assumptions• Tendency to conflate model with
reality• Maintaining consistency between
multiple models (with different underlying abstractions)
• Incompatibility between models– Especially design models vs.
failure models• …
22 September 2015 Modeling for V&V 4
Models for Design vs. VV&C
Design• Focus on functional correctness,
desired properties, and performance
• Emphasis on average case behavior (e.g., for performance)
• Intended interactions between components & environment– Presumption that the only
interaction is through defined interfaces
VV&C• Focus on non-functional
requirements – Safety, Security, etc.
• Emphasis on worst-case behavior• Preclude adverse interaction
between components & environment
– In addition to failure propagation through defined interfaces, must also consider “out-of-band” failure modes
22 September 2015 Modeling for V&V 5
Example “out-of-band” failure mode
https://xkcd.com/538/
22 September 2015
Modeling for V&V 6
Questions?
Downloaded from http://xkcd.com/246/
22 September 2015 Modeling for V&V 7
Backup Slides
22 September 2015 8Modeling for V&V
9
• Assumed importance order- Assumed/known fault hypothesis violated
exhaustion of resources (known fault hypothesis)
- Single point of failure unknown fault hypothesis forgotten failure modeunderestimated probability of occurrence
- Fault propagation = domino effect (fault containment)
• Real occurrence frequency order- Chain or domino effect (missing fault containment)
E.g. TTP membership; shown to be a fault propagation path [Ademaj, Sivencrona]
- Single point of failure (unknown fault hypothesis) E.g. quad-redundant control system (termination of bus)[ 2003]
- Exhaustion of resources (known fault hypothesis)
"How Systems Fail"
An assumption will remain valid only until you come to depend on it*.
22 September 2015 Modeling for V&V 10
* http://www.ece.mtu.edu/faculty/rmkieckh/Kieckhafer-top-ten.htm (version 9.1; law 4.2)