Author:Alex Groce, Daniel Kroening, and Flavio Lerda

Computer Science Department, Carnegie Mellon UniversityPittsburgh, PA 15213

Source:R. Alur and D.A. Peled (Eds.): CAV 2004, LNCS 3114, pp. 453–456,

2004.©Springer-Verlag Berlin Heidelberg 2004

Presented by Jui-Lung Yao, Master Student of CSIE, CCU

Understanding Counterexamples with explain

1

Outline

IntroductionCBMCDistance metricsPBSexplainCase studyConclusion

2

Outline

IntroductionCBMCDistance metricsPBSexplainCase studyConclusion

3

Outline

IntroductionCBMCDistance metricsPBSexplainCase studyConclusion

4

CBMC

A tool for the formal verification of ANSI-C programs using Bounded Model Checking (BMC).

Two applications of the tool: 1) The tool checks safety properties such as the

correctness of pointer constructs. 2) the tool can compare an ANSI-C program with

another design, such as a circuit given in Verilog.

5

CBMC (cont’)

In BMC, the transition relation for a complex state machine and its specification are jointly unwound to obtain a Boolean formula that is satisfiable if there exists an error trace.

The formula is then checked by using a SAT procedure. If the formula is satisfiable, a counterexample is extracted from the output of the SAT procedure.

6

CBMC (cont’)

The tool comes with a graphical user interface (GUI) that hides the implementation details from the user.

If a counterexample is found, the GUI allows stepping through the trace like a debugger.

7

Outline

IntroductionCBMCDistance metricsPBSexplainCase studyConclusion

8

Distance metrics

A distance metric for program executions is a function d(a, b) (where a and b are executions of the same program) that satisfies certain properties:

9

Distance metrics (cont’)

Let a and b be executions of a program P, represented as sets of assignments,

10

Outline

IntroductionCBMCDistance metricsPBSexplainCase studyConclusion

11

PBS

The term pseudo-Boolean constraints refers to arbitrary linear inequalities 0-1 in terms of variables, however many applications require only integer coefficients.

12

PBS (cont’)

Example representing “ at most 2 out of v1, v2, v3, v4, v5 can be true “ using (a) pure CNF (b) PB form.

13

Outline

IntroductionCBMCDistance metricsPBSexplainCase studyConclusion

14

explain

explain uses distance metrics on program executions, in a manner inspired by the counterfactual theory of causality, to provide a number of automatic analyses: Given a counterexample execution, explain can

automatically produce an execution that is as similar as possible to the failing run but does not violate the specification.

explain can also automatically produce a new counterexample that is as different as possible from the original counterexample.

Finally, explain can determine causal dependencies between predicates in an execution.

15

explain (cont’)

explain is used through the same GUI as CBMC. The interface allows users to step through explanatory traces as they would in a debugger (with the ability to step forwards and backwards).

explain uses the PBS pseudo-Boolean solver to produce the trace, and lists the changes made to the original counterexample.

Portions of the code that explain suggests may be faulty are highlighted for the user.

16

explain (cont’)

The tool assists the user in understanding counterexamples, but knowledge of the program (and the specification) is necessary to guide the tool.

As an example, we will use explain to narrow in on an error in a small but non-trivial C program.

17

Outline

IntroductionCBMCDistance metricsPBSexplainCase studyConclusion

18

Case study: TCAS19

TCAS (Traffic Alert and Collision Avoidance System) is an aircraft conflict detection and resolution system used by all US commercial aircraft.

The Georgia Tech version of the Siemens suite includes 41 buggy versions of ANSI-C code for the Resolution Advisory (RA) component of the TCAS system.

Case study: TCAS (cont’)20

We load tcas.c into the GUI and run the CBMC model checker. After a few seconds, the GUI reports that the assertion on line 257 has been violated.

We run explain on the counterexample to find a successful execution that is as similar as possible to the failing run. explain produces this trace, and lists the changes made to the original counterexample.

Case study: TCAS (cont’)21

Only Down_Separation has changed, causing result to be FALSE instead of TRUE. In the original run, Down Separation was 500, and now it is 1792.

We need to change the > into a >= comparison.

Outline

IntroductionCBMCDistance metricsPBSexplainCase studyConclusion

22

Conclusion

Framework

23

ANSI-C program

CBMC

Trace of runList the changes

Counter-example

Modify source code

PBSexplain

flow order

Conclusion (cont’)

explain is a tool that uses a model checker to assist users in debugging programs (or specifications).

The tool is fully integrated with a model checker that precisely handles a rich variety of the features of the ANSI-C language, and provides a graphical front-end for user interactions.

Case studies have demonstrated that explain can successfully localize errors in a number of programs

24

Thanks for your listening

25