Interpretation of Large C Codebases - Spazio IT · PDF fileInterpretation of Large C Codebases...
Transcript of Interpretation of Large C Codebases - Spazio IT · PDF fileInterpretation of Large C Codebases...
![Page 1: Interpretation of Large C Codebases - Spazio IT · PDF fileInterpretation of Large C Codebases Maurizio Martignano ... CppCheck Post-Processing e.g. CPD, ... out/bugs.html © 2017](https://reader031.fdocuments.in/reader031/viewer/2022030500/5aacb2117f8b9a8d678d34e2/html5/thumbnails/1.jpg)
© 2017 Spazio IT - Soluzioni Informatiche s.a.s.
1June 2017
SpAziO IT – Soluzioni Informatiche
s.a.s.
BoundedModelCheckingandAbstractInterpretationofLarge C Codebases
Maurizio Martignano
Spazio IT – Soluzioni Informatiche s.a.s.
Via Manzoni 40
46030 San Giorgio di Mantova, Mantova
http://www.spazioit.com
![Page 2: Interpretation of Large C Codebases - Spazio IT · PDF fileInterpretation of Large C Codebases Maurizio Martignano ... CppCheck Post-Processing e.g. CPD, ... out/bugs.html © 2017](https://reader031.fdocuments.in/reader031/viewer/2022030500/5aacb2117f8b9a8d678d34e2/html5/thumbnails/2.jpg)
© 2017 Spazio IT - Soluzioni Informatiche s.a.s.
2
Agenda
Code Analyzers
Model Generation and Execution
Staying on phase one (model generation)
Local Analyses and Code Partitioning
Clang Static Analyzer and Facebook Infer
SonarQube Code Quality Platform
Code Inspection (a human activity)
![Page 3: Interpretation of Large C Codebases - Spazio IT · PDF fileInterpretation of Large C Codebases Maurizio Martignano ... CppCheck Post-Processing e.g. CPD, ... out/bugs.html © 2017](https://reader031.fdocuments.in/reader031/viewer/2022030500/5aacb2117f8b9a8d678d34e2/html5/thumbnails/3.jpg)
© 2017 Spazio IT - Soluzioni Informatiche s.a.s.
3
Code Analizers
Why?
– To get metrics / “quality stamps”
– To check compliance with standards/recommendations
– To look for (potential) issues: e.g. bugs, vulnerabilities, code smells (http://sonarsrv.spazioit.com/projects)
– To look for “hot spots” and help/facilitate development, code inspection, ISVV
Two broad categories
– Pattern matcher(s) (e.g. Lint)
– Symbolic / Abstract Executors/Interpreters (e.g. CBMC and Frama-C)
![Page 4: Interpretation of Large C Codebases - Spazio IT · PDF fileInterpretation of Large C Codebases Maurizio Martignano ... CppCheck Post-Processing e.g. CPD, ... out/bugs.html © 2017](https://reader031.fdocuments.in/reader031/viewer/2022030500/5aacb2117f8b9a8d678d34e2/html5/thumbnails/4.jpg)
© 2017 Spazio IT - Soluzioni Informatiche s.a.s.
4
Model Generation and Execution
Program(piece of code)
ModelGenerator
Program Model ModelExecution
ExecutionResults
![Page 5: Interpretation of Large C Codebases - Spazio IT · PDF fileInterpretation of Large C Codebases Maurizio Martignano ... CppCheck Post-Processing e.g. CPD, ... out/bugs.html © 2017](https://reader031.fdocuments.in/reader031/viewer/2022030500/5aacb2117f8b9a8d678d34e2/html5/thumbnails/5.jpg)
© 2017 Spazio IT - Soluzioni Informatiche s.a.s.
5
Model Generation and Execution
CBMC and Frama-C Value Analysis Plugin organize their computation into two phases:
– Generation of a model of the code under analysis
– “Symbolic execution” or “logic verification” of the model itself.
The computation resources required by phase one grow in a polynomial way with the complexity of code under analysis (number of files, packages, classes, functions, parameters, variables, lines of code, loops, constructs and so on…)
![Page 6: Interpretation of Large C Codebases - Spazio IT · PDF fileInterpretation of Large C Codebases Maurizio Martignano ... CppCheck Post-Processing e.g. CPD, ... out/bugs.html © 2017](https://reader031.fdocuments.in/reader031/viewer/2022030500/5aacb2117f8b9a8d678d34e2/html5/thumbnails/6.jpg)
© 2017 Spazio IT - Soluzioni Informatiche s.a.s.
6
Model Generation and Execution
The computation resources required by phase two grow exponentially with the complexity of the code under of analysis.
What can we do about this situation?
![Page 7: Interpretation of Large C Codebases - Spazio IT · PDF fileInterpretation of Large C Codebases Maurizio Martignano ... CppCheck Post-Processing e.g. CPD, ... out/bugs.html © 2017](https://reader031.fdocuments.in/reader031/viewer/2022030500/5aacb2117f8b9a8d678d34e2/html5/thumbnails/7.jpg)
© 2017 Spazio IT - Soluzioni Informatiche s.a.s.
7
Staying on phase one
Infinite Loop
Example
![Page 8: Interpretation of Large C Codebases - Spazio IT · PDF fileInterpretation of Large C Codebases Maurizio Martignano ... CppCheck Post-Processing e.g. CPD, ... out/bugs.html © 2017](https://reader031.fdocuments.in/reader031/viewer/2022030500/5aacb2117f8b9a8d678d34e2/html5/thumbnails/8.jpg)
© 2017 Spazio IT - Soluzioni Informatiche s.a.s.
8
Staying on phase one
A simple never ending C program:
#include <stdio.h>
int main() {
int i = 0;
int n = 10;
for (i = 0; i < n; i++) {
printf("Iteration #% 2d.\n", i + 1);
if (i == 5) i = 0;
}
return 0;
}
![Page 9: Interpretation of Large C Codebases - Spazio IT · PDF fileInterpretation of Large C Codebases Maurizio Martignano ... CppCheck Post-Processing e.g. CPD, ... out/bugs.html © 2017](https://reader031.fdocuments.in/reader031/viewer/2022030500/5aacb2117f8b9a8d678d34e2/html5/thumbnails/9.jpg)
© 2017 Spazio IT - Soluzioni Informatiche s.a.s.
9
Staying on phase one
CBMC analysis results…
Unwinding loop c::main.0 iteration 1205 file loops.c
line 7 function main thread 0
Unwinding loop c::main.0 iteration 1206 file loops.c
line 7 function main thread 0
Unwinding loop c::main.0 iteration 1207 file loops.c
line 7 function main thread 0
Unwinding loop c::main.0 iteration 1208 file loops.c
line 7 function main thread 0
Unwinding loop c::main.0 iteration 1209 file loops.c
line 7 function main thread 0
…
![Page 10: Interpretation of Large C Codebases - Spazio IT · PDF fileInterpretation of Large C Codebases Maurizio Martignano ... CppCheck Post-Processing e.g. CPD, ... out/bugs.html © 2017](https://reader031.fdocuments.in/reader031/viewer/2022030500/5aacb2117f8b9a8d678d34e2/html5/thumbnails/10.jpg)
© 2017 Spazio IT - Soluzioni Informatiche s.a.s.
10
Staying on phase one
Frama-C analysis results…
[value] Done for function printf
[value] computing for function printf <- main.
Called from loops.c:8.
[value] Done for function printf
[value] computing for function printf <- main.
Called from loops.c:8.
[value] Done for function printf
[value] Recording results for main
[value] done for function main
[value] ====== VALUES COMPUTED ======
[value] Values at end of function main:
NON TERMINATING FUNCTION
![Page 11: Interpretation of Large C Codebases - Spazio IT · PDF fileInterpretation of Large C Codebases Maurizio Martignano ... CppCheck Post-Processing e.g. CPD, ... out/bugs.html © 2017](https://reader031.fdocuments.in/reader031/viewer/2022030500/5aacb2117f8b9a8d678d34e2/html5/thumbnails/11.jpg)
© 2017 Spazio IT - Soluzioni Informatiche s.a.s.
11
Local Analyses & Code
Partitioning
![Page 12: Interpretation of Large C Codebases - Spazio IT · PDF fileInterpretation of Large C Codebases Maurizio Martignano ... CppCheck Post-Processing e.g. CPD, ... out/bugs.html © 2017](https://reader031.fdocuments.in/reader031/viewer/2022030500/5aacb2117f8b9a8d678d34e2/html5/thumbnails/12.jpg)
© 2017 Spazio IT - Soluzioni Informatiche s.a.s.
12
Local Analyses & Code
Partitioning
![Page 13: Interpretation of Large C Codebases - Spazio IT · PDF fileInterpretation of Large C Codebases Maurizio Martignano ... CppCheck Post-Processing e.g. CPD, ... out/bugs.html © 2017](https://reader031.fdocuments.in/reader031/viewer/2022030500/5aacb2117f8b9a8d678d34e2/html5/thumbnails/13.jpg)
© 2017 Spazio IT - Soluzioni Informatiche s.a.s.
13
Local Analyses & Code
Partitioning
“Compiler”C Sources ProjectDB
ScriptsGenerator
Scripts
AnalysisTool
(CBMC /Frama-C)
Analysis Results
![Page 14: Interpretation of Large C Codebases - Spazio IT · PDF fileInterpretation of Large C Codebases Maurizio Martignano ... CppCheck Post-Processing e.g. CPD, ... out/bugs.html © 2017](https://reader031.fdocuments.in/reader031/viewer/2022030500/5aacb2117f8b9a8d678d34e2/html5/thumbnails/14.jpg)
© 2017 Spazio IT - Soluzioni Informatiche s.a.s.
14
Clang Static Analyzer
and Facebook Infer
Still too
complicated?
Clang FB Infer
![Page 15: Interpretation of Large C Codebases - Spazio IT · PDF fileInterpretation of Large C Codebases Maurizio Martignano ... CppCheck Post-Processing e.g. CPD, ... out/bugs.html © 2017](https://reader031.fdocuments.in/reader031/viewer/2022030500/5aacb2117f8b9a8d678d34e2/html5/thumbnails/15.jpg)
© 2017 Spazio IT - Soluzioni Informatiche s.a.s.
15
Clang Static Analyzer
and Facebook Infer
Normal build operation
./autogen.sh
./configure
make
Analizers Invocation
./autogen.sh
./configure
scan-build make [clang]
infer – make [fb infer]
![Page 16: Interpretation of Large C Codebases - Spazio IT · PDF fileInterpretation of Large C Codebases Maurizio Martignano ... CppCheck Post-Processing e.g. CPD, ... out/bugs.html © 2017](https://reader031.fdocuments.in/reader031/viewer/2022030500/5aacb2117f8b9a8d678d34e2/html5/thumbnails/16.jpg)
© 2017 Spazio IT - Soluzioni Informatiche s.a.s.
16
Clang Static Analyzer
and Facebook Infer
#include <stdio.h>
#include <stdlib.h>
int main(void) {
int *ip, i;
ip = &i;
ip += 100;
char *ptr = NULL;
/* what will be printed here? */
printf("ip - &i = %d.\n", ip - &i);
printf("(unsigned)ip - (unsigned)&i = %u.\n", (unsigned)ip - (unsigned)&i);
// allocating some memory
ptr = (char *) malloc(100);
// and never releasing it...
return 0;
}
![Page 17: Interpretation of Large C Codebases - Spazio IT · PDF fileInterpretation of Large C Codebases Maurizio Martignano ... CppCheck Post-Processing e.g. CPD, ... out/bugs.html © 2017](https://reader031.fdocuments.in/reader031/viewer/2022030500/5aacb2117f8b9a8d678d34e2/html5/thumbnails/17.jpg)
© 2017 Spazio IT - Soluzioni Informatiche s.a.s.
17
Clang Static Analyzer
and Facebook Infer
[clang]pexa0.c:13:29: warning: format specifies type 'int' but the argument has type 'long' [clang-diagnostic-format]
printf("ip - &i = %d.\n", ip - &i);
^
pexa0.c:17:3: warning: Value stored to 'ptr' is never read [clang-analyzer-deadcode.DeadStores]
ptr = (char *) malloc(100);
^
pexa0.c:17:3: note: Value stored to 'ptr' is never read
ptr = (char *) malloc(100);
^
pexa0.c:21:3: warning: Potential leak of memory pointed to by 'ptr' [clang-analyzer-unix.Malloc]
return 0;
^
pexa0.c:17:18: note: Memory is allocated
ptr = (char *) malloc(100);
^
pexa0.c:21:3: note: Potential leak of memory pointed to by 'ptr'
return 0;
^
![Page 18: Interpretation of Large C Codebases - Spazio IT · PDF fileInterpretation of Large C Codebases Maurizio Martignano ... CppCheck Post-Processing e.g. CPD, ... out/bugs.html © 2017](https://reader031.fdocuments.in/reader031/viewer/2022030500/5aacb2117f8b9a8d678d34e2/html5/thumbnails/18.jpg)
© 2017 Spazio IT - Soluzioni Informatiche s.a.s.
18
Clang Static Analyzer
and Facebook Infer
[fb infer]
pexa0.c:17: error: MEMORY_LEAK
memory dynamically allocated to `ptr` by call to `malloc()` at line 17, column 18 is not reachable after line 17, column 3
15.
16. // allocating some memory
17. > ptr = (char *) malloc(100);
18. // and never releasing it...
19.
Summary of the reports
MEMORY_LEAK: 1
![Page 19: Interpretation of Large C Codebases - Spazio IT · PDF fileInterpretation of Large C Codebases Maurizio Martignano ... CppCheck Post-Processing e.g. CPD, ... out/bugs.html © 2017](https://reader031.fdocuments.in/reader031/viewer/2022030500/5aacb2117f8b9a8d678d34e2/html5/thumbnails/19.jpg)
© 2017 Spazio IT - Soluzioni Informatiche s.a.s.
19
SonarQube Code Quality
Platform
![Page 20: Interpretation of Large C Codebases - Spazio IT · PDF fileInterpretation of Large C Codebases Maurizio Martignano ... CppCheck Post-Processing e.g. CPD, ... out/bugs.html © 2017](https://reader031.fdocuments.in/reader031/viewer/2022030500/5aacb2117f8b9a8d678d34e2/html5/thumbnails/20.jpg)
© 2017 Spazio IT - Soluzioni Informatiche s.a.s.
20
SonarQube Code Quality Platform
SonarQube is an open source Web Application (http://www.sonarqube.org) which
– Takes in input a set of source code files and a set of analyses results (produced by external tools).
– Stores both sources and results in a database.
– Makes available the gathered information via a dynamic website where the results are shown in the context of the code itself.
![Page 21: Interpretation of Large C Codebases - Spazio IT · PDF fileInterpretation of Large C Codebases Maurizio Martignano ... CppCheck Post-Processing e.g. CPD, ... out/bugs.html © 2017](https://reader031.fdocuments.in/reader031/viewer/2022030500/5aacb2117f8b9a8d678d34e2/html5/thumbnails/21.jpg)
© 2017 Spazio IT - Soluzioni Informatiche s.a.s.
21
SonarQube – What is it?
Source Code
Files
SonarQube
Database
SonarQube
Engine
Analyses
Results
![Page 22: Interpretation of Large C Codebases - Spazio IT · PDF fileInterpretation of Large C Codebases Maurizio Martignano ... CppCheck Post-Processing e.g. CPD, ... out/bugs.html © 2017](https://reader031.fdocuments.in/reader031/viewer/2022030500/5aacb2117f8b9a8d678d34e2/html5/thumbnails/22.jpg)
© 2017 Spazio IT - Soluzioni Informatiche s.a.s.
22
SonarQube / Plugins / Sensors
SonarQube
Plugin-Me.g. Java
Plugin-Ie.g. C/C++
Plugin-1e.g. Ada
Sensor-Je.g. PC-Lint
Sensor-Me.g. GCOV
Sensor-1eg. CppCheck
Post-Processinge.g. CPD, Decorators
Pre-Processinge.g. scanning
and parsing
![Page 23: Interpretation of Large C Codebases - Spazio IT · PDF fileInterpretation of Large C Codebases Maurizio Martignano ... CppCheck Post-Processing e.g. CPD, ... out/bugs.html © 2017](https://reader031.fdocuments.in/reader031/viewer/2022030500/5aacb2117f8b9a8d678d34e2/html5/thumbnails/23.jpg)
© 2017 Spazio IT - Soluzioni Informatiche s.a.s.
23
SonarQube – Working with Issues
Analyses on the same code base can be performed at different moments in time and SonarQube keeps track of the changes/evolution.
The problems found during analyses (a.k.a. issues) can be managed directly from within the system itself, e.g.
– Identifying false positives
– Assigning issues to developers
– Checking their status (if they have been solved)
– …
![Page 24: Interpretation of Large C Codebases - Spazio IT · PDF fileInterpretation of Large C Codebases Maurizio Martignano ... CppCheck Post-Processing e.g. CPD, ... out/bugs.html © 2017](https://reader031.fdocuments.in/reader031/viewer/2022030500/5aacb2117f8b9a8d678d34e2/html5/thumbnails/24.jpg)
© 2017 Spazio IT - Soluzioni Informatiche s.a.s.
24
Code Inspection
Code Inspection is a human activity but proper tools
– increase efficiency
– reduce risks.
![Page 25: Interpretation of Large C Codebases - Spazio IT · PDF fileInterpretation of Large C Codebases Maurizio Martignano ... CppCheck Post-Processing e.g. CPD, ... out/bugs.html © 2017](https://reader031.fdocuments.in/reader031/viewer/2022030500/5aacb2117f8b9a8d678d34e2/html5/thumbnails/25.jpg)
© 2017 Spazio IT - Soluzioni Informatiche s.a.s.
25
Further Readings
CBMC - http://www.cprover.org/cbmc/
Frama-C - http://frama-c.com/
Clang Static Analyzer - http://clang-analyzer.llvm.org/
Facebook Infer - http://fbinfer.com/
SonarQube – http://www.sonarqube.org
Spazio IT activities on Code Quality -http://www.spazioit.com/pages_en/sol_inf_en/code_quality_en/
SonarQube Demo - http://sonarsrv.spazioit.com/projects
Clang Static Analyzer Demo -http://www.spazioit.com/software/scan-view-naviserver
Facebook Infer Demo - http://www.spazioit.com/software/infer-out/bugs.html
![Page 26: Interpretation of Large C Codebases - Spazio IT · PDF fileInterpretation of Large C Codebases Maurizio Martignano ... CppCheck Post-Processing e.g. CPD, ... out/bugs.html © 2017](https://reader031.fdocuments.in/reader031/viewer/2022030500/5aacb2117f8b9a8d678d34e2/html5/thumbnails/26.jpg)
© 2017 Spazio IT - Soluzioni Informatiche s.a.s.
26
Questions?