PRACTICAL ANALYSIS OF NON-TERMINATION IN LARGE LOGIC PROGRAMS

Senlin Liang and Michael Kifer

MOTIVATION High-level LP languages, e.g.,

SILK http://silk.semwebcentral.org Flora-2 http://flora.sourceforge.net

are designed to be suitable for knowledge engineers, who are

not programmers KBs created by engineers typically

are complex, large stress the capabilities of the underlying engine=> Non-termination happens often – very hard to debug

To address this, we developed Terminyzer – a non-Termination analyzer (this extends our previous work in PADL-13)

OUTLINE Preliminaries: causes of non-termination,

tabling, and forest logging Adding ids to rules Terminyzer: analyses causes, repairs

problems Experiments Conclusion and future work

CAUSES OF NON-TERMINATION Cause 1: loops in SLD-resolution Example

p(X) :- p(X).?- p(a).

Solution: tabling [SW12] Caches calls to subgoals, which cuts recursive

loops If enough predicates are tabled then

Each subgoal is tabled once Each answer is tabled once Evaluation terminates if there are finitely many

subgoals and answers

[SW12] Terry Swift and David S. Warren. XSB: Extending prolog with tabled logic programming. TPLP’12.

CAUSES OF NON-TERMINATION (CONT’D) Cause 2:

Engine supports tabling But the program generates infinitely many tabled

subgoals Example

p(X) :- p(f(X)).?- p(a).Subgoals to be tabled: p(a), p(f(a)), p(f(f(a))), ...

Solution: subgoal abstraction [RS] Abstracts subgoals that are deeper than a

threshold Assuming threshold = 2, p(f(f(f(a)))) would be

abstracted to p(f(f(X))), X = f(a) Guarantees that there will be only a finite number

of tabled subgoals[RS] F. Riguzzi and T. Swift. Terminating evaluation of logic programs with finite three-valued models. ACM on Computational Logic. To appear.

CAUSES OF NON-TERMINATION (CONT’D) Cause 3:

Engine supports both tabling and subgoal abstraction But infinitely many answers

Examplep(a). p(f(X)) :- p(X).?- p(X).Answers to be derived: p(a), p(f(a)), p(f(f(a))), …

Solution: does not exist Halting problem is undecidable Whether a program has a finite number of answers is

undecidable We can only try to help the user to deal with the issue

If user really intended the program to be the way it is We need a way to limit output. E.g., bounded rationality [BS13]

Our focus: Unexpected non-termination (ie, when it’s a bug)

[BS13] B. Grosof and T. Swift. Radial restraint: A semantically clean approach to bounded rationality for logic programs. AAAI’13

TABLING AND FOREST LOGGING Tabling needs no introduction Forest logging is a new tracing facility in XSB

Events Logs

Calls to tabled subgoalsE.g. parent calls child

tabled_call(child,parent,status,timestamp)neg_tabled_call(child,parent,status,timestamp)

Answer derivationsE.g. ansr is derived for sub

new_answer(ansr,sub,timestamp)new_delayed_answer(ansr,sub,delayed_lits,timestamp)

Return answers to consumersE.g. ansr for child is retuned to parent

answer_return(ansr,child,parent,timestamp)delayed_answer_return(ansr,child,parent,timestamp)

Subgoal completionsE.g. sub is completed

completed(sub,scc_num,timestamp)completed(sub,early_completed,timestamp)

Other events Irrelevant to our discussion

where Timestamp preserves the order of events status = new, complete, incomplete

ADDING IDS TO RULES – KEY INSIGHT Add unique ids to rules s.t. tabled subgoals

remember their host rules:

Each tabling declaration :- table p/n is changed to :- table p/(n+1)

For a query ?- p(x1, …, xn) If p/n is tabled, then

Change it to ?- p(x1, …, xn, Newvar) Chop off the last arguments of returned answers

Otherwise, the query stays the same

TERMINYZER OVERVIEW Two versions Version 1 (most precise) requires:

Tabling Forest logging Subgoal abstraction

Version 2 requires only Tabling Forest logging

Currently only XSB has all three features, but: Several systems have tabling Forest logging info exists internally in all of them

– just needs to be exposed to the user => at least Version 2 is easily portable

TERMINYZER OVERVIEW

Suppose a query does not terminate, Terminyzer then Analyzes an execution forest log Determines the causes of non-termination

The exact sequence of unfinished tabled subgoals, and the host rule id of each subgoal

Subgoals forming recursive cycles Rectifies some causes of misbehavior

(heuristically)

CALL SEQUENCE ANALYSIS Identifies the exact sequence of unfinished calls

and their host rule ids that lead to a non-termination Find unfinished subgoals – whose answers have not

been completely derived – by unfinished(Child,Parent,Timestamp) :-

tabled_call(Child,Parent,new,Timestamp),

not_exists(completed(Child,SCCNum,Timestamp1)).

unfinished(child,parent,timestamp) says that Subgoal parent calls subgoal child, and it happened at timestamp Neither child nor parent have been completely evaluated

Sort unfinished calls by their timestamps Host rule ids are kept in the last arguments of child-

subgoals

not_exists is the XSB well-founded negation operator; existentially quantifies SCCNum and Timestamp1.

CALL SEQUENCE ANALYSIS (CONT’D) Example 1

@!r1 p(a). @!r5 r(X) :- r(X).@!r2 p(f(X)) :- q(X). @!r6 r(X) :- p(X), s(X).@!r3 q(b). @!r7 s(f(b)).@!r4 q(g(X)) :- p(X). ?- r(X).where @!ruleid is the syntax to assign rule ids

Its unfinished calls – the red ones form a non-terminating loopunfinished(r(_h9900,_h9908), root, 0) – root is for intial query

unfinished(r(_h9870,r5), r(_h9870,_h9889), 8)unfinished(r(_h9840,r5), r(_h9840,r5), 11)unfinished(p(_h9810,r6), r(_h9810,r5), 12)unfinished(q(_h9780,r2), p(_h9780,r6), 16)unfinished(p(_h9750,r4), q(_h9750,r2), 20)unfinished(q(_h9720,r2), p(_h9720,r4), 24)

Next, we will find recursive cycles

CALL SEQUENCE ANALYSIS (CONT’D) Unfinished calls can be represented using an unfinished-call

graph UCG = (N,E) N: the set of unfinished subgoals E: {(parent,child) |unfinished(child,parent,ts) is true}

For the above exampleunfinished(r(_h9900,_h9908), root, 0)unfinished(r(_h9870,r5), r(_h9870,_h9889), 8)unfinished(r(_h9840,r5), r(_h9840,r5), 11)unfinished(p(_h9810,r6), r(_h9810,r5), 12)unfinished(q(_h9780,r2), p(_h9780,r6), 16)unfinished(p(_h9750,r4), q(_h9750,r2), 20)unfinished(q(_h9720,r2), p(_h9720,r4), 24)

where: Each node is represented by the timestamp when it is first called -1 represents root Edges are labeled with timestamps of calls

Loops in UCG represent recursive cycles However, not all cycles are causing non-termination

E.g. [8, 8]

Can be represented as

thousandsof subgoals

CALL SEQUENCE ANALYSIS (CONT’D) Assume all predicates are tabled + subgoal abstraction Theorem (Soundness of the call sequence analysis)

If there are unfinished calls in a query’s complete trace, then Call sequence analysis finds the exact sequence of

unfinished calls that caused non-termination, and The ids of the rules that issued these calls

Theorem (Completeness of the call sequence analysis) If the evaluation of a query does not terminate, then There is at least one loop in the UCG for its complete trace, and

the loop’s subgoals are responsible for generating infinite number of answers, and

The last argument of each of these subgoals specifies the rule ids from whose bodies these subgoals were called.

The complete trace is infinite due to non-termination so, practically speaking We work with only a prefix of the trace by limiting term depth/size or

execution time/space It may produce false negatives, but they are also useful for

identifying computational bottlenecks.

ANSWER FLOW ANALYSIS Recall that not all cycles in UCG are causing

non-termination, so we need to refine call sequence analysis

Answer flow analysis does precisely that: it identifies the cycles that actually cause non-termination

Non-termination happens if and only if a subset of subgoals keeps: Receiving answers from producers, Deriving new answers, and Returning answers to callers

Answer flow analysis looks for repeated patterns of answer returns

ANSWER FLOW ANALYSIS (CONT’D) Compute answer-flow patterns (AFP)

Answer-return sequence (ARS): the sequence of (child,parent) pairs where child returns answers to parent

Candidate AFP: a sequence cafp s.t. cafp2+ is a suffix of ARS AFP: the shortest candidate AFP cafp s.t. its repetition forms

the maximal suffix of ARS among all candidate AFP’s In previous Example 1

ARS = [ (p(_h599,r4),q(_h599,r2)), (q(_h619,r2),p(_h619,r4)), (p(_h639,r4),q(_h639,r2)), (q(_h659,r2),p(_h659,r4)), (p(_h679,r4),q(_h679,r2)), (q(_h699,r2),p(_h699,r4)), (p(_h719,r4),q(_h719,r2)), (q(_h739,r2),p(_h739,r4)), (p(_h759,r4),q(_h759,r2)), (q(_h779,r2),p(_h779,r4))].

where (child,parent) indicates child returns answers to parent Candidate AFPs are:

cafp1 = [(p(_h759,r4),q(_h759,r2)), (q(_h779,r2),p(_h779,r4))] cafp2 = cafp1• cafp1

AFP is cafp1 AFP captures information flow pattern without redundancy

@!r2 p(f(X)) :- q(X).@!r4 q(g(X)) :- p(X).

ANSWER FLOW ANALYSIS (CONT’D) An AFP can be represented as an answer-flow

graph AFG = (N,E), where N: the set of subgoals in afp E: {(child,parent) | (child,parent) ∈ afp}

Loops in AFG represent cycles that cause non-termination

ANSWER FLOW ANALYSIS (CONT’D) As before, we assume all predicates are tabled +

subgoal abstraction Theorem (Soundness of the answer flow analysis)

If the complete trace of a query has an AFP then the query does not terminate.

Theorem (Completeness of the answer flow analysis) If the query evaluation does not terminate, then: There is an AFP in its complete trace, AFG = (N, E) contains at least one loop, Every sub ∈ N appears in at least one loop, and Each edge (sub1,sub2)∈E, where sub1=pred(...,

ruleid), tells us that sub2 calls sub1 from the body of a rule whose id is ruleid.

MORE ON UCG AND AFG Theorem (Relationship between UCG and

AFG)Consider the UCG and AFG for a non-terminating forest log, we have: nodes(AFG) ⊂ nodes(UCG) edges(AFG) ⊂ reverse-edges(UCG) loops(AFG) ⊆ loops(UCG)

Theorem (No false results for finite traces) If the evaluation of a query, Q, terminates, then both the UCG and the AFG for Q’s trace are empty.

AUTO-REPAIR OF RULES A query does not terminate if

It has infinitely many answers, or It has a finite number of answers, but one of its

subqueries has an infinite number of them In this case: a different evaluation order may terminate

the query This case is targeted by our auto-repair heuristic

For each unfinished(child,parent,timestamp) We know

the host rule for this call, and the common set of the unbound arguments of parent and

child – the arguments whose bindings are to be derived Thus, to reduce the possibility that parent receives

infinite number of bindings from child, one can delay issuing a child-call from its host rule until these arguments are bound

AUTO-REPAIR OF RULES (CONT’D) Example

@!r1 p(a). @!r5 r(X) :- r(X).@!r2 p(f(X)) :- q(X). @!r6 r(X) :- p(X), s(X).@!r3 q(b). @!r7 s(f(b)).@!r4 q(g(X)) :- p(X). ?- r(X).

Its unfinished calls are:unfinished(r(_h9900,_h9908), root, 0)unfinished(r(_h9870,r5), r(_h9870,_h9889), 8)unfinished(r(_h9840,r5), r(_h9840,r5), 11)unfinished(p(_h9810,r6), r(_h9810,r5), 12)unfinished(q(_h9780,r2), p(_h9780,r6), 16)unfinished(p(_h9750,r4), q(_h9750,r2), 20)unfinished(q(_h9720,r2), p(_h9720,r4), 24)

Applying auto-repair@!r1 p(a). @!r5 r(X) :- wish(ground(X))^r(X).@!r2 p(f(X)) :- wish(ground(X))^q(X). @!r6 r(X) :- wish(ground(X))^p(X), s(X).@!r3 q(b). @!r7 s(f(b)).@!r4 q(g(X)) :- wish(ground(X))^p(X). ?- wish(ground(X))^r(X).

Then the query will terminate with X = f(b)

TABLED ENGINES WITHOUT SUBGOAL ABSTRACTION

Additional cause of non-termination: infinite number of subgoals

Steps Compute the sequence of unfinished subgoals Compute simplified subgoal sequence (SSS) out of

unfinished subgoal sequence Each unfinished subgoal, predicate(…, ruleid), is

simplified to predicate(ruleid) Find the SSS pattern, as in the case of answer flow

pattern SSS pattern contains the predicates and their rule

ids that recursively call one another to form increasingly deep subgoals

TABLED ENGINES WITHOUT SUBGOAL ABSTRACTION (CONT’D)

Example@!r1 p(a). @!r4 r(X) :- r(X).@!r2 p(X) :- q(f1(X)). @!r5 r(X) :- p(X), s(X).@!r3 q(X) :- p(f2(X)). @!r6 s(a).?- r(a).

Its unfinished calls are:unfinished(r(a,_h46), root, 0).unfinished(r(a,r4), r(a,_h27), 8).unfinished(r(a,r4), r(a,r4), 11).unfinished(p(a,r5), r(a,r4), 12).unfinished(q(f1(a),r2), p(a,r5), 16).unfinished(p(f2(f1(a)),r3), q(f1(a),r2), 19).unfinished(q(f1(f2(f1(a))),r2), p(f2(f1(a)),r3), 22).unfinished(p(f2(f1(f2(f1(a)))),r3), q(f1(f2(f1(a))),r2), 25).unfinished(q(f1(f2(f1(f2(f1(a))))),r2), p(f2(f1(f2(f1(a)))),r3), 28).unfinished(p(f2(f1(f2(f1(f2(f1(a)))))),r3), q(f1(f2(f1(f2(f1(a))))),r2), 31).unfinished(q(f1(f2(f1(f2(f1(f2(f1(a)))))),r2), p(f2(f1(f2(f1(f2(f1(a)))))),r3), 34).……

SSS = [root, r(_), r(r4), r(r4), p(r5), q(r2), p(r3), q(r2), p(r3), q(r2), p(r3), q(r2)] SSS pattern = [p(r3), q(r2)] – says that it is predicate p of r3 and predicate q of

r2 that recursively call each other, thus forming increasingly deep nested subgoals

STATUS

Unfinished-call/answer flow implemented in SILK and Flora-2

SILK has a GUI, Flora-2’s underway Rule Ids are crucial for practicality Auto-repair: not implemented yet

EXPERIMENTS System

Dual core 2.4GHz Lenovo X200 with 3GB RAM Ubuntu 11.04 with Linux kernel 2.6.38

Small programs They took a tiny fraction of a second to analyze Correctness of analyses is manually verified

Large programs: one biology ontology from SILK KB size:

Flora-2 program with 4,774 rules and 919 facts Compiled into XSB’s 5,500+ rules and 1,000+ facts

Logs produced until evaluation consumed all memory Size: ~2GB Number of records: ~14M

Took 170 seconds

CONCLUSIONS Terminyzer – a tool for analyzing non-

termination Future work:

Implement auto-repair better auto-repair algorithms

Comparison with others: All other work deals with underpowered logic

engines that are so last Century (Prolog) Or with trying to find sufficient conditions for

termination (different focus)

Thank you!

FOREST LOGGING – EXAMPLE

:- table path/2. edge(1,2). edge(1,3). edge(2,1).path(X,Y) :- edge(X,Y). path(X,Y) :- edge(X,Z), path(Z,Y).?- path(1,Y).