Fault Localization Where is the root cause

Tho T. Quan SAVE research group Faculty of Computer Science and Engineering Hochiminh City University of Technology Hochiminh, Vietnam qttho@cse.hcmut.edu.vn

Outline

Introduction

Fault Localization: An Odyssey

Academic View

Demonstrations

Conclusion

About SAVE

System Analysis and VErification

Founded in 2008, by Dr. Nguyen Hua Phung

Current leader: A/Prof. Quan Thanh Tho

www.cse.hcmut.edu.vn/~save

Research topics

Formal methods to design, verify and secure computer-based systems

Improvement of programming productivity and performance

Education and practical tools to assist teaching and practicing programming and software engineering

Research Directions

Polymorphism Virus

Mobile Security Checking

Petri Nets – PAT – Model Checking

Automatic Composition of Prototype

Software Testing

Fault Localization

Fault Localization: An Odyssey

Concept

Tour

Fault Localization - Concept

Intuition view

Program introduced abnormal behaviors

Test case failed

Where is the bugs

Approaches

Memory dump

Breakpoint setting

Print out intermediate values

Issue: Locate the suspicious code

Fault Localization - Concept

Fault Localization

Locate the code that may cause the bug

Based on testing results

Automatic or Semi-Automatic

Let the tour commence

More insight analysis

SQL Fault localization

Expert-based Localization

SQL inspection

Expert asset

Does it really work? - Testing with Siemens suite

Fault Localization: Under Academic View

Overview

Program

Test suite

Test result: - Passed/failed test cases - Program spectra

Fault localization techniques

The ranked list of elements based on suspicious degrees

The set of suspicious elements

The root cause

spectrum

model

proof

Fault Localization Techniques

Model-based

Proof-based

Spectrum-based

FL techniques: Model-based

Produce a mathematical model of the program

Solve the model to find suspicious elements

FL techniques: Proof-based

Produce an unsatisfied formula for an error

Find the elements in the formula which cause the formula satisfied when removed

FL techniques: Spectrum-based

Based on spectra of testing result

Returned a ranked list of suspicious elements

Some Definitions

S(e): suspicious score of element e

Nef(e): failed executions involving e

Nnf(e): failed executions NOT involving e

Nep(e): passed executions involving e

Nnp(e): passed executions NOT involving e

Taratula, Ochiai and Jaccard

Tarantula Ochiai Jaccard

3 0.5 0.7 0.5

4 1 1 1

6 0 0 0

n = 1 n= -1

Tarantula Ochiai Jaccard

3 0.5 0.58 0.33

4 1 1 1

6 0 0 0

n = 1 n= -1 n = 0

Open Issues

Impact of test-case sets

Missing code problem

Test-case sets

Better strategy of test-case generation yields better fault localization performance

0.0;1.0;2.0 1.0;4.0;3.0 2.0;2.0;3.0

0.0;1.0;2.0 1.0;4.0;3.0 2.0;2.0;3.0 1.0;2.0;1.0

Missing code problem

Fault localization aims at finding suspicious code that causes error

When the logic error caused by missing some code no code

available to be “suspected”

Case Study: The PROVE system

Program Verification system

http://elearning.cse.hcmut.edu.vn/provegroup/index.jsp

Verify student programming works

Produce counter-example

Show execution path

Show suspected code

Conclusion

Fault localization is the next step of testing

Spectrum-based is deemed suitable to be applied in industry

Applied in an online educational tool at HCMUT

Chances for collaboration with industry