Experiences en analyse statique de logiciels embarqu
Transcript of Experiences en analyse statique de logiciels embarqu
![Page 1: Experiences en analyse statique de logiciels embarqu](https://reader031.fdocuments.in/reader031/viewer/2022020912/6203476024f6b61e9c66264f/html5/thumbnails/1.jpg)
Experiences in the static analysis ofembedded software
Guillaume Brat(Kestrel Technology, Ames Division)
Experiences en analyse statique delogiciels embarqués
![Page 2: Experiences en analyse statique de logiciels embarqu](https://reader031.fdocuments.in/reader031/viewer/2022020912/6203476024f6b61e9c66264f/html5/thumbnails/2.jpg)
Software blowup
8
1700
3
32
160
430
1
10
100
1000
10000
Voyager
(1977)
Galileo
(1989)
Cassini
(1997)
MPF
(1997)
Shuttle
(2000)
ISS
(2000)
Mission
Lin
es
of
Co
de
(T
ho
usa
nd
s)
![Page 3: Experiences en analyse statique de logiciels embarqu](https://reader031.fdocuments.in/reader031/viewer/2022020912/6203476024f6b61e9c66264f/html5/thumbnails/3.jpg)
$165M$125M
4 monthslost
Famous aerospace failures
>$1B
![Page 4: Experiences en analyse statique de logiciels embarqu](https://reader031.fdocuments.in/reader031/viewer/2022020912/6203476024f6b61e9c66264f/html5/thumbnails/4.jpg)
NASA Software Challenges
• Need to develop three systems for each mission:– Flight software– Ground software– Simulation software
• Flight software– Has to fit on radiation-hardened processors– Limited memory resources– Has to provide enough information for diagnosis– Can be patched (or uploaded) during the mission
• Each mission has its own goals, and therefore, each softwaresystem is unique!
• Cannot benefit from opening its source code to the publicbecause of security reasons.– No open-source V&V
• Mission software is getting more complex.– Large source code (~1 MLOC)– The structure of the code is more complex
![Page 5: Experiences en analyse statique de logiciels embarqu](https://reader031.fdocuments.in/reader031/viewer/2022020912/6203476024f6b61e9c66264f/html5/thumbnails/5.jpg)
International Space Station
• International Space Station:– Attitude control system, 1553 bus,
science payloads– International development (interface
issues)– Codes ranging from 10-50 KLOC– A failure in a non critical system can
cause a hazardous situationendangering the whole station
– Enormous maintenance costs– Over 500 defects reported– Over 3 MLOC by now
![Page 6: Experiences en analyse statique de logiciels embarqu](https://reader031.fdocuments.in/reader031/viewer/2022020912/6203476024f6b61e9c66264f/html5/thumbnails/6.jpg)
Mars mission software
• Mars Path Finder:– Code size: 140 KLOC– Famous bug: priority inversion
problem• Deep Space One:
– Code size: 280 KLOC– Famous bug: race condition problem
in the RAX software
• Mars Exploration Rovers:– Code size: > 650 KLOC– Famous bug: Flash memory
problem
![Page 7: Experiences en analyse statique de logiciels embarqu](https://reader031.fdocuments.in/reader031/viewer/2022020912/6203476024f6b61e9c66264f/html5/thumbnails/7.jpg)
How is the Software Verified?
• Testing, testing, testing…• Mars missions: high-fidelity test bench
– Runs 24 hours a day– 8 hour test sessions: lost if a runtime error
occurs• Space Station:
– Critical software: on-ground simulatormaintained at Marshall Space Center
– Payloads:• Independently verified by contractors• NASA test requirement document
![Page 8: Experiences en analyse statique de logiciels embarqu](https://reader031.fdocuments.in/reader031/viewer/2022020912/6203476024f6b61e9c66264f/html5/thumbnails/8.jpg)
How effective is this?
• Badly re-initialized state variable for MPL: caused thecrash of the lander ($150M)
• Unit mismatch for MCO: caused the orbiter to miss itsorbit insertion and burn during re-entry ($85M)
• Thread priority inversion problem for MPF: 24 hours ofscience data lost
• Flash memory problem for MER: rover paralyzedduring several days
• Science mission for the ISS currently under validation:– Passes NASA test requirements– But… 500+ defects reported
![Page 9: Experiences en analyse statique de logiciels embarqu](https://reader031.fdocuments.in/reader031/viewer/2022020912/6203476024f6b61e9c66264f/html5/thumbnails/9.jpg)
Static Analysis
Static analysis offers compile-time techniques for predictingsafe and computable approximations to the set of values
arising dynamically at run-time when executing the program
the analysis is donewithout executing the program
all possible values(and more) are computed
We use abstract interpretation techniques to extract a safe system of semantic equations
which can be resolved using lattice theory techniquesto obtain numerical invariants for each program point
![Page 10: Experiences en analyse statique de logiciels embarqu](https://reader031.fdocuments.in/reader031/viewer/2022020912/6203476024f6b61e9c66264f/html5/thumbnails/10.jpg)
Covered Defect Classes
• Static analysis is well-suited for catching runtime errors,e.g.:– Array-out-bound accesses– Un-initialized variables/pointers– Overflow/Underflow– Invalid arithmetic operations
• Defect classes for Deep Space One:– Misuse: array out-of-bound, pointer mis-assignments– Initialization: no value, incorrect value– Assignment: wrong value, type mismatch– Undefined Ops: FP errors (tan(90)), arithmetic (division by zero)– Omission: case/switch clauses without defaults– Scoping Confusion: global/local, static/dynamic– Argument Mismatches: missing args, too many args, wrong types,
uninitialized args– Finiteness: underflow, overflow
![Page 11: Experiences en analyse statique de logiciels embarqu](https://reader031.fdocuments.in/reader031/viewer/2022020912/6203476024f6b61e9c66264f/html5/thumbnails/11.jpg)
Software Development Process
SoftwareArchitectural Design
SystemIntegration
SystemArchitectural Design
SystemRequirements
SoftwareRequirements Analysis
SoftwareQualification Testing
SoftwareUnit Testing
Software Coding
SoftwareDetailed Design
Software Integration
SystemQualification Testing
STATICANALYSIS
![Page 12: Experiences en analyse statique de logiciels embarqu](https://reader031.fdocuments.in/reader031/viewer/2022020912/6203476024f6b61e9c66264f/html5/thumbnails/12.jpg)
Research Process
Experiments onreal NASA code
Identification oftechnical gaps
Implementation ofresearch prototype
Identification of commercial tools
Our goal was to assess the capabilities of static analysis and identify the technical gaps to make it usable in NASA missions.
PolySpaceC-verifier
MPF DS1 ISSMER
precision scalability usability
CGS: a scalable, precise analyzer
![Page 13: Experiences en analyse statique de logiciels embarqu](https://reader031.fdocuments.in/reader031/viewer/2022020912/6203476024f6b61e9c66264f/html5/thumbnails/13.jpg)
PolySpace C-Verifier
PolySpace C-Verifier finds runtime errors in C programs.
It works like a sophisticated compiler.
Partial Error Coverage
Test cases & drivers
IntegrationTesting
Unit-levelTesting
Conventional Testing
Control & Data Flow Analysis
Source Code CheckingCompiler Front End
Software Safety AnalysisPropagation Algorithm forIdentifying Run-Time Errors
Total Error Coverage
No input cases! No input drivers!
Sophisticated Static Analysis
color-coded reporting:Green always correctRed always incorrectOrange may be incorrectGray never executed
Analysis time ~ e precision
Simple run-time error reporting
![Page 14: Experiences en analyse statique de logiciels embarqu](https://reader031.fdocuments.in/reader031/viewer/2022020912/6203476024f6b61e9c66264f/html5/thumbnails/14.jpg)
MER CVS
C-Verifier
MER TEAM
New error: report it!
VERIFICATION TEAM
Quick analysis: 30 minutesDeep analysis: 2-3 hours
30 KLOCSmodules
analysisreport
studycode
void getData (T* p) { if (flag == TRUE) { p->data = ...; p->status = 1; } else sendEvrMsg(“error”);} …T state;getData(&state);sendData (state->data);/* unreachable */
STATIC ANALYSIS OF MER
![Page 15: Experiences en analyse statique de logiciels embarqu](https://reader031.fdocuments.in/reader031/viewer/2022020912/6203476024f6b61e9c66264f/html5/thumbnails/15.jpg)
Experimental results
NIVOBAIOVFL
NIVErrors3.2KLocs17KLocs25KLocsMax Size
bc, reu, pyro, pwr,dat, adc, pas, imu,mcas, rpdu, bcp, btp,…
HLRCACS+EDLModules
Under-development
UntestedStableMaturity650KLocs40KLocs200KLocsSizeCCCLanguage
MERISSMPFProject
![Page 16: Experiences en analyse statique de logiciels embarqu](https://reader031.fdocuments.in/reader031/viewer/2022020912/6203476024f6b61e9c66264f/html5/thumbnails/16.jpg)
Performance
• Pyro + Pwr modules:– 1st pass: O1, 54 mn, 4610 green, 601 orange– 2nd pass: O1, 44 mn, 4758 green, 409 orange– 2nd pass: O2, 34 mn, 4758 green, 409 orange– No significant red (obvious infinite loops)
• Dat + (adc, pas, imu, mcas, rpdu, pwr, pyro, bcp, btp)
– Quick analysis: 30 mn– Un-initialized variable (not yet fixed)– Returning the address of a local variable (already fixed)– Overflow in constant expression (already fixed)
![Page 17: Experiences en analyse statique de logiciels embarqu](https://reader031.fdocuments.in/reader031/viewer/2022020912/6203476024f6b61e9c66264f/html5/thumbnails/17.jpg)
A Role for Static Analysis
• Extensive experiments with PolySpaceVerifier:– Minors bugs found in MER– Serious out-of-bounds array accesses found in an
ISS Science Payload• Absence of runtime errors (80% precision)• Useful: yes• Effective: no
– It takes 24 hours to analyze 40 KLOC– Difficulty to break down large systems into small
modules
![Page 18: Experiences en analyse statique de logiciels embarqu](https://reader031.fdocuments.in/reader031/viewer/2022020912/6203476024f6b61e9c66264f/html5/thumbnails/18.jpg)
NASA Requirements
• Scalability:– Analyze large systems in less than 24 hours– Analysis time similar to compilation time for
mid-size programs• Precision:
– At least 80%– Informative: the analysis provides enough
information to diagnose a warning
![Page 19: Experiences en analyse statique de logiciels embarqu](https://reader031.fdocuments.in/reader031/viewer/2022020912/6203476024f6b61e9c66264f/html5/thumbnails/19.jpg)
Practical Static Analysis
C Global Surveyor(NASA Ames)
Scalability (KLOC)
Precision
1000
500
50
80% 95%
PolySpaceC-Verifier
DAEDALUS100%
Coverity
Klocwork
dayshours
CERTIFIERS
seconds
DEBUGGERS
minutes
![Page 20: Experiences en analyse statique de logiciels embarqu](https://reader031.fdocuments.in/reader031/viewer/2022020912/6203476024f6b61e9c66264f/html5/thumbnails/20.jpg)
C Global Surveyor
• Prototype analyzer– Based on abstract interpretation– specialized for NASA flight software
• Covers major pointer manipulation errors:– Out-of-bounds array indexing– Un-initialized pointer access– Null pointer access
• Keeps all intermediate results of theanalysis in a human readable form: hugeamount of artifacts
![Page 21: Experiences en analyse statique de logiciels embarqu](https://reader031.fdocuments.in/reader031/viewer/2022020912/6203476024f6b61e9c66264f/html5/thumbnails/21.jpg)
Abstract Interpretation
Programsemantics
AbstractSemantics
ProgrammingLanguageDefinition
Defines operations allowed in the language: assignments, conditionals, loops, functions, …
assigns meaning to a programon a suitable concrete domain
Concretedomain
Abstractdomain
Models some properties of concrete computationsForgets about remaining information
γ concretizationabstraction α
![Page 22: Experiences en analyse statique de logiciels embarqu](https://reader031.fdocuments.in/reader031/viewer/2022020912/6203476024f6b61e9c66264f/html5/thumbnails/22.jpg)
Program Verification
• Check that every operation of a programwill never cause an error (division by zero,buffer overrun, deadlock, etc.)
• Example: int a[1000];
for (i = 0; i < 1000; i++) {
a[i] = … ; // 0 <= i <= 999
}
a[i] = … ; // i = 1000;buffer overrun
safe operation
![Page 23: Experiences en analyse statique de logiciels embarqu](https://reader031.fdocuments.in/reader031/viewer/2022020912/6203476024f6b61e9c66264f/html5/thumbnails/23.jpg)
Simple Example
E5 = E2 ∩ [1000, +∞[
E1 = {n ⇒ Ω}
E4 =〚n = n + 1〛E3
E3 = E2 ∩ ]-∞, 999]
E2 =〚n = 0〛E1 ∪ E4
1
2
3
4
5
n = 0;
while n < 1000 do
n = n + 1;
end
exit
[0,1000]
[0,999]
[1,1000]
1000
]-∞,+∞[
![Page 24: Experiences en analyse statique de logiciels embarqu](https://reader031.fdocuments.in/reader031/viewer/2022020912/6203476024f6b61e9c66264f/html5/thumbnails/24.jpg)
Simple Example
n = 0;
while n < 1000 do
n = n + 1;
end
exit
[0,1000]
[0,999]
[1,1000]
1000
]-∞,+∞[
In effect, the analysishas automaticallycomputed numericalinvariants!
![Page 25: Experiences en analyse statique de logiciels embarqu](https://reader031.fdocuments.in/reader031/viewer/2022020912/6203476024f6b61e9c66264f/html5/thumbnails/25.jpg)
MPF Flight Software Family
Thread Thread Thread
Queue
HeapQueue
Shallow
Large
![Page 26: Experiences en analyse statique de logiciels embarqu](https://reader031.fdocuments.in/reader031/viewer/2022020912/6203476024f6b61e9c66264f/html5/thumbnails/26.jpg)
MPF Flight Software Family
assign (double *p, double *q, int n) {
int i;
for (i = 0; i < n; i++)
p[i] = q[i];
}
assign (A, B, 10) assign (&pS->f, &A[2], m)
10...1000 call sites
Thousands of such functionsAlmost all of them contain loops
![Page 27: Experiences en analyse statique de logiciels embarqu](https://reader031.fdocuments.in/reader031/viewer/2022020912/6203476024f6b61e9c66264f/html5/thumbnails/27.jpg)
The CGS Solution
• Extensive representation using intervals– Some use of DBMs– Adaptive state variable clustering for scalability
• One level of context-sensitivity• Computation of function summaries for
speeding up the interprocedural propagation• Parallel analyses over clusters of processors
![Page 28: Experiences en analyse statique de logiciels embarqu](https://reader031.fdocuments.in/reader031/viewer/2022020912/6203476024f6b61e9c66264f/html5/thumbnails/28.jpg)
Fast Context Sensitivity
• Context-sensitivity is required• We can’t afford performing 1000 fixpoint
iterations with widening and narrowing for eachfunction
• Compute a summary of the function using arelational numerical lattice
access(p[i], 0 <= i < n)
access(q[i], 0 <= i < n)
![Page 29: Experiences en analyse statique de logiciels embarqu](https://reader031.fdocuments.in/reader031/viewer/2022020912/6203476024f6b61e9c66264f/html5/thumbnails/29.jpg)
Implementation of CGS
Database
Equationsfor file1.c
Equationsfor file2.c
Cluster of machines
Analyzefunction f
Analyzefunction g
![Page 30: Experiences en analyse statique de logiciels embarqu](https://reader031.fdocuments.in/reader031/viewer/2022020912/6203476024f6b61e9c66264f/html5/thumbnails/30.jpg)
Working with a Database
• We use PostgreSQL• Mutual exclusion problems are cared for by
the database• Simple interface using SQL queries• Efficient communications require index
structures (B-Trees):– Populating tables is slower– Difficult to manage
• Granularity problems: splitting up largetables into smaller ones
![Page 31: Experiences en analyse statique de logiciels embarqu](https://reader031.fdocuments.in/reader031/viewer/2022020912/6203476024f6b61e9c66264f/html5/thumbnails/31.jpg)
Parallel implementation
• We use the Parallel Virtual Machine (PVM)• High-level interface for process creation
and communication• Allows heterogeneous implementation:
currently a mix of C and OCaml• Remote debugging is extremely difficult• Design is difficult:
– Scheduling policies– Granularity of computations
![Page 32: Experiences en analyse statique de logiciels embarqu](https://reader031.fdocuments.in/reader031/viewer/2022020912/6203476024f6b61e9c66264f/html5/thumbnails/32.jpg)
Effectiveness ofParallelization
Analysis Times
0
2000
4000
6000
8000
10000
12000
1 2 4 6 8
CPUs
Se
co
nd
s
DS1
MPF
![Page 33: Experiences en analyse statique de logiciels embarqu](https://reader031.fdocuments.in/reader031/viewer/2022020912/6203476024f6b61e9c66264f/html5/thumbnails/33.jpg)
The I/O Bottleneck
• The performance curve flattens: overhead ofgoing through the network
• MER takes a bit less than 24 hours to analyze:– 70% of the time is spent in the interprocedural
propagation– I/O times dominate (loading/unloading large tables)
• Under investigation: caching tables on machinesof the cluster and using PVM communicationmechanism (faster than concurrent databaseaccess)
![Page 34: Experiences en analyse statique de logiciels embarqu](https://reader031.fdocuments.in/reader031/viewer/2022020912/6203476024f6b61e9c66264f/html5/thumbnails/34.jpg)
Experimental Results
2080%550550MER
2.580%280280DS1
1.580%140140MPF
8-1280%20140MPF
AnalysisTime
(hours)
PrecisionMax SizeAnalyzed
Size(KLOC)
Commercial tool C Global Surveyor
![Page 35: Experiences en analyse statique de logiciels embarqu](https://reader031.fdocuments.in/reader031/viewer/2022020912/6203476024f6b61e9c66264f/html5/thumbnails/35.jpg)
Conclusion
• NASA a besoin de meilleurs outils de vérification• L’usage d’analyseurs statiques commerciaux s’est révélée
décevante– Problèmes de passage à l’ échelle– Problèmes de précision
• Nous avons donc dévelopé notre propre outil d’analyse statiquepour C– Passe à l’ échelle– Meilleurs temps d’analyse– Précision équivalente
• Prochaine étape: C++