Debugging Field Failures by Minimizing Captured Executions (ICSE 2009: NIER example)

ICSE ‘09: New Ideas and Emerging Results

Debugging Field Failures by Minimizing Captured ExecutionsJames Clause and Alessandro OrsoGeorgia Institute of Technology

1


What are field failures?

Failures that occur after application deployment

Difficult to reproduce in-house ➞ difficult to debug

2


Current approaches

Collect limitedinformation

Ask the userhow to reproduce

3


Replay / Debug

Develop Record

✘

Captured

failure

In th

e fi

eldIn

ho

use

Our approach

4


Our approachIn

the

fie

ldIn h

ou

se

Replay / Debug

Develop Record

✘

Minimize✘

Captured

failure

✘

Minimized

failure

5


Recording

Capture high-level events

efficient

amenable to minimization

6


Replaying

Interfere as little as possible

redirect high-level events

maintain relative time of events

7


Minimization

Focus debugging effort

Iteratively remove what is not necessary for reproducing the failure

✘

8


2-phase minimization

Time minimization

✂Data minimization

9


Time minimization

Remove idle times

Reduce event latency

10


Data minimization

Remove data hierarchically

1. complete files

2. pieces of files

3. individual characters ✂11


Prototype implementation

Record / Replay

x86/libc binaries

Minimization

customizable scripts

Limitations

non-determinism

GUI inputs

ADDAAssisting the Debugging of Deployed Applications

12


Evaluation

RQ1: How effective is ADDA at minimizing failing executions?

RQ2: How much overhead does ADDA impose?ADDA

Assisting the Debugging of Deployed Applications

13


Subject and faults

Two real field failures from Pine’s history

address book fault

header-color fault

Pine email and news client

14


RQ1: Minimization results

✘

Around 10% of the recording remains after minimization

We were able to debug the failures using the minimized recording

15


RQ2: Overhead results

Online: negligible overhead while recording

Offline: less than 75 minutes to minimize

16


Future work

Support

GUI events

Non-determinism

Additional studies with more subjects

ADDAAssisting the Debugging of Deployed Applications

17


Utilize passing executions

Captured failures are utilized for debugging

Captured successful executions are discarded

Can we also use them?

Record

✘

Minimize

✘

Captured

failure

✔

Captured

success

Trash

18


Address privacy issues

Captured executions can contain sensitive information

How can we sanitize sensitive information but make sure the failure still occurs?

✘

Captured

failureCredit card numbers

Email addresses

Account names

Passwords

19


Thank [email protected]/~clause

20

mailto:[email protected]

mailto:[email protected]

Debugging Field Failures by Minimizing Captured Executions (ICSE 2009: NIER example)

Technology

Transcript of Debugging Field Failures by Minimizing Captured Executions (ICSE 2009: NIER example)