Guided programming and automated error analysis in an intelligent Prolog tutor

Int. J. Human-Computer Studies 61 (2004) 505–534

Guided programming and automated erroranalysis in an intelligent Prolog tutor

Jun Hong*

School of Computing and Mathematics, University of Ulster at Jordanstown, Newtownabbey, Co. Antrim

BT37 0QB, UK

Received 6 October 2002; accepted 5 February 2004

Abstract

We present a Prolog programming technique-based approach to guided programming and

automated error analysis in Prolog tutoring. The concept of Prolog programming technique is

used to characterize and classify programs. Each class of programs use the same programming

technique and share the common pattern of code. A set of programming technique grammar

rules are defined for each class of programs. These rules are used for programming technique

recognition, program construction, and program parsing. A programming technique frame is

used to represent the programming technique-related knowledge for each class of programs. A

program frame is used to represent the coding-related knowledge for the reference program of

each of the most specialized programming techniques. The representation of the programming

technique grammar rules, programming technique-related knowledge, and coding-related

knowledge provides the basis for guided programming and automated error analysis in

tutoring. Our approach to error analysis however does not rely on the representation of buggy

versions of the program. Automated error analysis in our approach is done on the basis of

comparing the parsings of both the student program and the reference program. Our approach

has been implemented in a Prolog tutoring system called the Prolog Tutor, which has been

tested on a collection of 125 programs for list reversal. The Prolog Tutor performs well on

these tests in terms of programming technique recognition, error detection, and error

correction.

r 2004 Elsevier Ltd. All rights reserved.

ARTICLE IN PRESS

*Present address: School of Computer Science, The Queen’s University of Belfast, Belfast BT7 1NN,

UK. Tel.: +44-28-90-974895.

E-mail address: [email protected] (J. Hong).

1071-5819/$ - see front matter r 2004 Elsevier Ltd. All rights reserved.

doi:10.1016/j.ijhcs.2004.02.001

1. Introduction

Prolog has a simple syntax and few program constructs. So the languageknowledge itself does not present any serious problems to Prolog novices. It appearsfairly easy for them to start programming in Prolog. Studies have, however, shownthat they often have many difficulties with Prolog programming due to their lack ofstructured programming knowledge (Taylor and du Boulay, 1987; Taylor, 1999; vanSomeren, 1990). For instance, they might not be quite sure what pattern of codeshould be used. This type of knowledge in contrast to the language knowledge istherefore essential in Prolog programming, Furthermore, even when structuredprogramming knowledge has been correctly applied, various types of programmingerrors often occur in novice programs. The analysis of these errors therefore presentsan effective way of improving a novice’s programming skills.

Many attempts have been made to identify and represent structured programmingknowledge in Prolog programming, and use it for program construction andtransformation. Brna et al. (1991,1999) proposed the concept of Prolog program-ming technique. A Prolog programming technique is a common pattern of code usedby the Prolog programmer in a fairly systematic way. In Bowles and Brna (1999) andBowles et al. (1999), it is shown that knowledge of programming techniques can beuseful in both teaching and supporting novice Prolog programmers.

Bundy et al. (1991) described the use of a Prolog programming technique-basedapproach in the development of a recursive technique editor for Prolog anddemonstrated the utility of such an approach for the construction of recursive Prologprocedures. Like the structure editors for the other programming languages, theireditor ensures the correctness of the syntax of the procedures. It also ensures that therecursive procedure built by the programmer is guaranteed to terminate and to bewell defined.

Gegg-Harrison (1991) discussed the importance of the use of structuredprogramming constructs in Prolog programming. He introduced the notion of basicProlog construct schemata and showed how they could be used to represent the basicconstructs of a structured Prolog for recursive list processing. The basic constructs ofProlog can be treated as common patterns of code and viewed as Prologprogramming techniques. Gegg-Harrison (1992,1993,1999) described how the useof program schemata could be incorporated into an intelligent tutoring system forteaching recursive Prolog programming. Since a basic Prolog construct schemarepresents a Prolog programming technique that is the generalization of a set ofprograms, it is useful for presenting the general idea of the technique to the student.It is also useful for being presented to the student in the form of program templatesfor instantiation and modification. A basic Prolog construct schema is, however, lessuseful for recognizing a student program since it does not provide much grammaticalinformation and relies on pattern matching to match the schema to the studentprogram.

The use of program schemata has been shown to be effective for the optimizationof logic programs. A schema-based approach to program transformation was firstproposed in Fuchs and Fromherz (1992) and later extended in Vasconcelos and

ARTICLE IN PRESSJ. Hong / Int. J. Human-Computer Studies 61 (2004) 505–534506

Fuchs (1996) and Richardson and Fuchs (1998) for the optimization of logicprograms. In this approach, a set of program schemata is first defined.Transformation schemata are further defined as pairs of program schemata. Atransformation schema consisting of a pair of program schemata represents atransformation technique that transforms one program schema in the pair intoanother. Given a program to be optimized, one of the transformation schemata isselected such that one program schema in it covers the given program under aparticular substitution. The same substitution is then applied to another programschema in the transformation schema to create the transformed program.

Schema-based logic program transformation was also studied in Flener andDeville (1996), Buyukyildiz and Flener (1998) and Chasseur and Deville (1998),where the use of transformation schemata for program transformation is similar tothat described in Fuchs and Fromherz (1992), Vasconcelos and Fuchs (1996) andRichardson and Fuchs (1998). The main difference is that instead of searching for atransformation schema in which the covering program schema is discovered, aprogram is synthesized from one program schema in a transformation schema tomatch the given program. Some further work has been carried out on the correctnessof program schemata (Flener et al., 1998) and extensible program schemata (Gegg-Harrison, 1997).

All the above attempts have been mainly focused on facilitating programconstruction and transformation rather than program debugging and tutoring. Looi(1988, 1991) described an algorithm-based approach to the debugging of the studentProlog program in an intelligent Prolog tutoring system. His system detects errors inthe program and proposes the corrections necessary to fix them. Even though Looishowed that his approach to programming debugging is effective, realistic anduseful, his work has some limitations. First, the Prolog algorithms are picked upwithout characterization. Second, all the possible implementations of an algorithmhave to be represented. Third, it relies on the representation of buggy versions ofprograms for error explanation. Finally, it can only give code-related tutoringcommentaries to the student.

In this paper, we propose a Prolog programming technique-based approach to thetutoring of Prolog programming. The concept of Prolog programming technique isused to characterize and classify programs. Each class of programs use the sameprogramming technique and share the common pattern of code. Instead of usingprogram schemata, we propose a Prolog programming technique grammar torepresent Prolog programming techniques. We define a set of programmingtechnique grammar rules for each class of programs. These programming techniquegrammar rules can be used for programming technique recognition, programconstruction and program parsing. The programming technique-related knowledgeis represented for each class of programs in a programming technique frame. Thecoding-related knowledge for the reference program of each most specializedprogramming technique is represented in a program frame.

The representation of programming technique grammar rules, programmingtechnique-related knowledge and coding-related knowledge provides the basis forguided programming and automated error analysis in tutoring. Our approach to

ARTICLE IN PRESSJ. Hong / Int. J. Human-Computer Studies 61 (2004) 505–534 507

error analysis however does not rely on the representation of buggy versions of theprogram. Instead of representing all the possible implementations of a program, ourapproach represents only the standard implementation of the program. The same setof programming technique grammar rules are used to parse both the studentprogram and the standard program. The two parsings are then compared with eachother to identify any errors in the student program.

We have incorporated the above approach to guided programming andautomated error analysis into an intelligent system for tutoring Prolog programmingcalled the Prolog Tutor. The system first poses a programming exercise to the studentand asks him to do the exercise in the form of Prolog program. The system thenconducts tutoring in two different modes: guided programming and automated erroranalysis. The system conducts guided programming when the student is not able toprogram on his own or has failed to use a programming technique in his program.During guided programming, the student is given directive guidance and providedwith program templates in which the relevant programming techniques have beenembedded. Under the directive guidance, the student is able to make use of theprogram templates to program step by step. When the student has shown the correctuse of the programming techniques, the tutoring is shifted to error analysis. Thesystem examines the student program, compares it to the standard program for theexercise question, and identifies any errors in it. The student is then givenappropriate comments, suggestions or corrections so the errors in his program canbe corrected. While the errors are corrected, the student has also learned why theerrors have occurred and how they can be corrected so his programming knowledgecan be gradually improved.

The paper is organized as follows. In Section 2, we describe the use of Prologprogramming techniques to characterize and classify Prolog programs, and therepresentation of Prolog programming techniques. In Section 3, we describe therepresentation of the semantic knowledge including programming technique-relatedand coding-related knowledge, and programming exercises. In Section 4, we presenta new approach to the tutoring of Prolog programming on the basis of Prologprogramming techniques in which three main tasks, programming techniquerecognition, guided programming and automated error analysis are involved. InSection 5, we discuss the experimental results of our tests on a collection of 125program variations. These results show that our Prolog Tutor performs well in termsof programming technique recognition, error detection, and error correction. InSection 6, we compare our work with the related work. In the last section, wesummarize the paper and discuss the further developments.

2. Representing Prolog programming techniques

Many Prolog programs share a common pattern of code. A Prolog programmingtechnique is a common pattern of code used by the Prolog programmer in a fairlysystematic way (Brna et al., 1991,1999). Prolog programs can, therefore, becharacterized and classified into different categories, with each category of programs


corresponding to a particular programming technique. Given a student program, ifwe can recognize its underlying pattern of code we can know the category ofprograms it belongs to and the particular programming technique that has beenintended to implement in the program so that further tutoring decisions can bemade. On the other hand, a programming technique can be used for providingguidance on the generation of the corresponding category of programs. It isimportant to represent the programming technique for recognizing and generating aclass of programs with a common pattern of code.

In this section, we start with a brief description of the schema representation ofProlog programming techniques. We then introduce a Prolog programmingtechnique grammar for the representation of Prolog programming techniques thathas some advantages over the schema representation. We finally discuss how toorganize Prolog programming techniques in a hierarchical structure called theProlog programming technique hierarchy.

2.1. Schematic representation

Gegg-Harrison (1991) proposed a Prolog schema language for defining the basicProlog construct schemata that represent the common syntactic features shared bythe corresponding sets of programs. This schema language is an extension of Prologlanguage itself and supports second-order predicate variables, first-order schemavariables, optional, arbitrary, and permutable arguments and goals. For instance,the following schema represents a basic Prolog construct, which consists of a basecase and a recursive case.

schema A([ ], //&1SS).schema A([H|T], //&2SS) :- /pre pred(//&3SS, H, //&4SS),S

schema A(T, //&5SS)/, post pred(//&6SS, H, //&7SS)S.

In this schema, schema A is a second-order predicate variable that can be replacedby a Prolog predicate. &1, &2, y, and &7 are first-order schema variables. Schemavariables are denoted by &n where n is an arbitrary integer. They allow thegeneralization of arguments and goals. H and T are standard Prolog variables.Optional arguments and goals are denoted by enclosing them with angle brackets.For example, pre pred(//&3SS, H, //&4SS) and post pred(//&6SS, H, //&7SS) are two optional goals that may be instantiated to either the null string or asingle invocation of the goal. The generalization of an optional argument or goal toan arbitrary number of arguments or goals is denoted by enclosing them with doubleangle brackets. For example, any number of arguments can replace schema variables&1, y, &7.

A program schema can effectively capture the structured knowledge of a Prologprogramming technique that is the generalization of programs. It can be presented tothe student in the form of program templates for instantiation and modification. Oneof the shortcomings of a program schema is that it is less useful for recognizing a


student program since it does not provide much grammatical information and relieson pattern matching to match the schema to the student program.

2.2. Grammatical representation

We propose a Prolog programming technique grammar to represent Prologprogramming techniques. The Prolog programming technique grammar is a type ofcontext sensitive grammar and an extension of Prolog grammar with contextvariables, optional or arbitrary arguments and goals. It defines the general formof Prolog clauses. We use a set of programming technique grammar rules todefine the set of programs that correspond to a Prolog programming technique. TheProlog programming technique grammar, therefore, provides a grammaticalrepresentation of Prolog programming techniques. The Prolog programmingtechnique grammar rules are suitable for both program construction and debugging.They can be used to parse both correct and buggy programs. A fundamentaldifference between Gegg-Harrison’s schematic representation (Gegg-Harrison,1991,1993) and our grammatical representation is that the schematic representationis defined at the language level while the grammatical representation is defined at thegrammar level.

Gegg-Harrison (1991,1993) defined a set of Prolog construct schemata torepresent the corresponding Prolog programming techniques. We now describehow to define a set of Prolog programming technique grammar rules for each ofthese programming techniques. Let us look at the following programs, append, sumand reverse, which can be made to instantiate schema A described above in theprevious section.

append([ ], L, L).append([H|T], L, [H|X]) :- append(T, L, X).

sum([ ], 0).sum([X|Y], Sum) :- sum(Y, Sum1), Sum is Sum1 + X.

reverse([ ], [ ]).reverse([X|Y], L) :- reverse(Y,M), append(M, [X], L).

In all three programs, the Prolog programming technique used is to process thelist, by splitting it into the head and the tail, making a recursive call with the tail, andcombining the result of the recursive call with the head in a certain way to producethe overall result. The recursion repeats until the list to be processed becomes empty.We can define a set of Prolog programming technique grammar rules as shown inFig. 1 to represent this programming technique that can be used to generate andparse programs in which this technique has been used.

As shown in Fig. 1, those strings enclosed in a pair of brackets { } such as Pred, Hand T are context variables. Each programming technique grammar rule has thestructure


head ::= body.where the head is a component of the program. The meaning of the rule is that headtakes the form of body. For example, the rule

/list technique programS{Pred} ::= /base caseS{Pred}/recursive caseS{Pred}.

means that a Prolog program of list technique type consists of a base casefollowed by a recursive case, where both cases have the same predicate. The predicatedefined by the program is represented by the context variable Pred in the rule.Another grammar rule /base caseS{Pred} ::= /predS{Pred}(/listS{[ ]},/argumentsS). means that a base case in a Prolog program of list technique typestarts with a predicate followed by a left round bracket, an empty list [ ], a comma,some more arguments, and a right round bracket and closed by a full stop. The rulehas a context variable, Pred, representing the predicate that the base case defines.

Using this set of programming technique grammar rules, the following naivereverse program can be parsed in part as shown in Fig. 2.

ARTICLE IN PRESS

Fig. 1. A partial set of technique grammar rules for a Prolog programming technique.

J. Hong / Int. J. Human-Computer Studies 61 (2004) 505–534 511

reverse([ ], [ ]).reverse([X|Y], L) :- reverse(Y, M), append(M, [X], L).

2.3. Prolog programming technique hierarchies

Prolog programming techniques can be defined at different levels of abstraction.Given a set of Prolog programming techniques, a Prolog programming techniquehierarchy can be created, in which nodes represent programming techniques, linksrepresent abstraction relations between programming techniques. This hierarchy isrooted at the most abstract programming technique, and bottoms out at the mostspecialized programming techniques. For instance, a Prolog programming techniquehierarchy as shown in Fig. 3 can be created to represent a set of Prolog programmingtechniques for recursive list processing in Prolog. Use of a programming techniquehierarchy allows focus shifts along abstraction dimension of programmingtechniques. In association with each node of the programming technique hierarchy,

ARTICLE IN PRESS

<list technique program>

<base_case>{reverse}<reverse_case>{reverse}

<arguments> <rule head>{reverse, X, Y} <rule body>{reverse, X, Y}

<pred>{reverse} <arguments>

<head>{X} <tail>{Y}

<list split>{X, Y}

<argument>

reverse([X|Y], L) :- reverse(Y, M), append(M, [X], L).

<pred>{reverse} <argument>

reverse([], []).

Fig. 2. A partial parsing tree of the naive reverse program using a set of technique grammar rules.

recursive list processing technique

element technique

naive technique

list technique position technique combination technique

accumulator technique railway-shunt technique inverse naive technique

Fig. 3. A partial Prolog programming technique hierarchy based on a viewpoint.

J. Hong / Int. J. Human-Computer Studies 61 (2004) 505–534512

there is a set of programming technique grammar rules. These programmingtechnique grammar rules define what make up the programming techniquerepresented by the node.

McCalla and Greer proposed the notion of granularity hierarchy (McCalla et al.,1992). They argued that the ability in an intelligent tutoring system to be able torecognize student behaviour at varying grain sizes is important both for pedagogicalreasons (in order to respond to the student at various levels of detail) and for reasonsof robustness in diagnosis (obscure student behaviour can be recognized at least at acoarse grain size). They briefly discussed how they used granularity hierarchies in therecognition of novice Lisp programming strategies, and showed how this enhancesrecognition and leads toward planning appropriate feedback for the student. Theyindicated that the level of detail at which a tutor chooses to present a topic,combined with the level at which the student interprets the presentation, will affectthe student’s success at understanding instruction.

McCalla and Greer’s granularity hierarchy (McCalla et al., 1992) is defined alongtwo dimensions: abstraction, corresponding to shifts in focus from general tospecific; and aggregation, corresponding to shifts in focus through part-wholerelationships. In a granularity hierarchy, there is a principal abstraction hierarchythat describes the abstraction dimension of the granularity hierarchy. In associationwith each node of the principal abstraction hierarchy, there is an aggregationhierarchy that describes the aggregation dimension of the granularity hierarchy. OurProlog programming technique hierarchy resembles McCalla and Greer’s notion ofprincipal abstraction hierarchy.

Instead of having an aggregation hierarchy associated with each node of aprincipal abstraction hierarchy, we have a set of programming technique grammarrules associated with each node of our programming technique hierarchy. This set ofgrammar rules defines programs that use the programming technique at variouslevels of aggregation, starting with the most aggregated component of the program,across different levels of aggregation, until the least aggregated component of theprogram. Use of programming technique grammar rules provides a more flexibleway to describe the aggregation dimension of our programming technique hierarchy.The most important advantage of programming technique grammar rules is thatthey can be used for recognizing programming techniques and generating programtemplates at a particular level of abstraction.

Gegg-Harrison (1991) described two major viewpoints, termination and execution,to differentiate Prolog programs along two primary dimensions. The terminationviewpoint groups together Prolog programs which have a similar syntactic structurebetween the termination arguments of their predicates. The execution viewpointgroups together Prolog programs which have a similar syntactic structure betweentheir recursive goal predicates. Since the two viewpoints both group together Prologprograms on the basis of their common patterns of code, they are consistent with theway in which Prolog programs are grouped together by Prolog programmingtechniques. Either of these viewpoints can be used to create a Prolog programmingtechnique hierarchy. The Prolog programming technique hierarchy partially shownin Fig. 3 has been created using the termination viewpoint.


At the top level, we have the most abstract Prolog programming technique,recursive list processing technique, that recursively processes the list. At the secondlevel, we have four Prolog programming techniques: element technique, listtechnique, position technique and combination technique. The element techniquetakes an additional argument (set of arguments) that identifies an element in the listwhich restricts the termination condition to the first (or last) occurrence of theelement in the list. The list technique recursively processes each element in the list.The position technique takes an additional argument that identifies a position in thelist that corresponds to the location of an element, and restricts the terminationcondition to the reaching of the specified position. The combination technique takestwo additional arguments that identifies an element and an index which restrict theterminating condition to the nth (where n is the specified index) occurrence of theelement in the list. At the bottom level, for instance, we have four of the mostspecialized programming techniques, naive technique, accumulator technique,inverse naive technique, and railway-shunt technique that all specialize the listtechnique at the second level.

Though these most specialized programming techniques are still general enough tobe used in different list processing exercises, when they are used in a particular listprocessing exercise, they have often been tailored to it. For instance, when thesemost specialized programming techniques are used in the list reversal exercise, theyhave already tailored to this particular list processing exercise as described below. Allthese techniques are used to define a predicate with its first argument as the list to bereversed and the second argument as the reversed list.

The naive programming technique reverses the list, by splitting it into the head andthe tail and making a recursive call with the tail, joining the reversed tail with the listconsisting of the head to get the reversed list. The recursive process repeats until thelist in the recursive call becomes empty. A standard program, in which this techniquehas been used, is as follows:

reverse([ ], [ ]).reverse([X|Y], L) :- reverse(Y, M), append(M, [X], L).append([ ], L, L).append([H|T], X, [H|L]) :- append(T, X, L).

The accumulator programming technique reverses a list, by defining anotherpredicate in which apart from the original two arguments an extra argument is usedas an accumulator to hold the reversed elements. The accumulator starts with anempty list. Each time a recursive call is made to the newly defined predicate, the headof the current list to be reversed becomes the head of the new accumulator, the listpreviously held in the accumulator becomes the tail of the new accumulator. A newrecursive call is then made with the tail of the current list as the new list to bereversed, the new accumulator, and the resulting argument. The recursive processrepeats until the list to be reversed in a recursive call becomes empty. Theaccumulator is finally passed to the resulting argument as the reversed list. Astandard program, in which this technique has been used, is as follows:


reverse(X, Y) :- reverse(X, [ ], Y).reverse([ ], L, L).reverse([H|T], Y, Z) :- reverse(T, [H|Y], Z).

The railway-shunt programming technique reverses a list, by defining anotherpredicate in which apart from the original argument for the list to be reversed, a listis used with the original resulting argument as its head and an extra variable in its tailas an accumulator to hold the reversed elements. The accumulator starts with anempty list. Each time a recursive call is made to the newly defined predicate, the headof the current list to be reversed becomes the head of the new accumulator, the listpreviously held in the accumulator becomes the tail of the new accumulator. A newrecursive call is then made with the tail of the current list as the new list to bereversed and a new list with the original resulting argument as its head and the newaccumulator as its tail. The recursive process repeats until the list to be reversed in arecursive call becomes empty. The accumulator is finally passed to the resultingargument as the reversed list. A standard program, in which this technique has beenused, is as follows:

reverse(A,B) :- rev(A, [B]).rev([ ], [B|B]).rev([Head|Tail], [B|Temp]) :- rev(Tail, [B, Head|Temp]).

The inverse naive programming technique reverses a list, L, by creating a listconsisting of all but the last element in L, T, and using the last element in L, H, as thehead of the reversed list. A recursive call is made with T to reverse it into RT whichthen becomes the tail of the reversed list. The recursive process repeats until the listin the recursive call becomes empty. A standard program, in which this techniquehas been used, is as follows:

reverse([ ], [ ]).reverse(L, [H|RT]) :- append(T, [H], L), reverse(T, RT).append([ ], A, A).append([A|B], C, [A|D]) :- append(B, C, D).

3. Representing semantic knowledge and programming exercises

Syntactic knowledge in our Prolog tutor is represented by either programmingtechnique grammar rules for programming techniques at various levels ofabstraction, or standard programs at the coding level. Apart from these twotypes of syntactic knowledge, we also represent semantic knowledge includingprogramming technique-related and coding-related knowledge, and the program-ming exercises. In this section, we discuss such knowledge representation.


3.1. Representing programming technique-related and coding-related knowledge

The semantic knowledge associated with each of the nodes representingprogramming techniques in the Prolog programming technique hierarchy isrepresented by either a programming technique frame or a program frame. Thereare also clause frames that represent the semantic knowledge associated withindividual clauses within a program.

Fig. 4 shows the representation of a programming technique frame which consistsof a programming technique name, the description of the programming technique,and child nodes in the programming technique hierarchy. The child nodes are thenames of the programming techniques which are specializations of the programmingtechnique. The child nodes in the programming technique frame are used to link theprogramming technique to the other programming techniques in the programmingtechnique hierarchy. The description of the programming technique is used intutoring when the system advises the student on using this particular programmingtechnique or confirms that this is the programming technique it has recognized fromthe student program.

Fig. 5 shows the representation of a program frame. A program frame consists ofa program name, the format of the invocation, the description of the predicate, thenames of its base cases and recursive cases, and the name of the programmingtechnique that the program has used. The case names refer to the correspondingclause frames. List and Revlist in both the invocation format and the predicatedescription are template variables. These variables are instantiated by the invocationcode in a particular program. Therefore, when the predicate description is used intutoring, it is actually tailored to the particular predicate concerned. This howeveronly occurs when the predicate defined by the program is used as a goal in anotherprogram.

Fig. 6 shows the representation of a base case frame which consists of a case name,the code and description of the head, and the code and description of the body. Insome base cases, there is no body.

Fig. 7 shows the representation of a recursive case frame which consists of a casename, the code and description of the head, and the code and description of the pre-predicates, the recursive calls and the post-predicates in the body. There are templatevariables in both the head and body of the recursive case. These variables areinstantiated by the head and body of the recursive case in a particular program.

ARTICLE IN PRESS

Fig. 4. Representation of a programming technique frame.


Therefore, when the head and body descriptions are used in tutoring, they areactually tailored to the particular program concerned.

3.2. Representing programming exercises

Exercise frames are used to represent the programming exercises given to thestudent.

ARTICLE IN PRESS

Fig. 5. Representation of a program frame.

Fig. 6. Representation of a base case frame.

Fig. 7. Representation of a recursive case frame.


Fig. 8 shows an exercise frame that consists of an exercise name, the description ofthe exercise and the names of the reference programs for the exercise.

4. Programming technique recognition, guided programming and error analysis

The tutoring process involves three major tasks: programming techniquerecognition, guided programming and error analysis. First of all, the system posesa programming exercise to the student by accessing the exercise description in theexercise frame and asks him to write a Prolog program for doing the given task in theexercise. Technique recognition tries to establish whether any programmingtechniques have been intended by the student. Tutoring focuses can then be foundand the system can decide whether the student needs to be tutored at theprogramming technique level, the coding level or both. When the student is tutoredat the programming technique level, the system is involved in the task of guidedprogramming. When the student is tutored at the coding level, the system is involvedin the task of error analysis. In this section, we describe these three tasks,respectively.

4.1. Hierarchy of relevant programming techniques

Given a programming exercise, a hierarchy of relevant programming techniques isfirst created. Each programming exercise has been associated with a list of referenceprograms that are the standard program solutions to the exercise question. Thenames of these reference programs are retrieved from the exercise frame. Differentprogramming techniques have been used in these reference programs. The mostspecialized programming technique that each of these reference programs has used isretrieved from the corresponding program frame. A path can then be found in theprogramming technique hierarchy, that connects the most specialized programmingtechnique to the most abstract programming technique. All such paths arerepresented in the hierarchy of relevant programming techniques. A student, whois doing the programming exercise, is guided by the system using this hierarchy. Thesystem shifts its focus step by step along the abstraction dimension of the hierarchy.

For example, a programming exercise is represented by the exercise frame asshown in Fig. 8. In the exercise, the student is asked to write a program whichreverses the elements of a list into another list. There are four reference programsgiven in the exercise frame, naive, accumulator, inverse reverse, and railway-shunt. A

ARTICLE IN PRESS

Fig. 8. Representation of a programming exercise.


hierarchy of relevant programming techniques as shown in Fig. 9 can be created forthe given programming exercise.

4.2. Recognizing programming techniques

When a programming exercise is given to the student, he is not told about theprogramming techniques to be used in the exercise. The student answers the exercisequestion in the form of a Prolog program that consists of a number of clauses. Thesystem tries to recognize programming techniques in the hierarchy of relevantprogramming techniques, which have been used in the student program. This is doneby parsing the student program using the corresponding sets of programmingtechnique grammar rules for appropriate programming techniques at different levelsof abstraction. The system starts with the most abstract programming technique atthe top of the hierarchy. If the program can be parsed by the corresponding set ofprogramming technique grammar rules, the system moves to the programmingtechniques at the next level of abstraction. If there is more than one programmingtechnique at the current level of abstraction, the system tries to parse the programusing one set of programming technique grammar rules at a time until the program isparsed by a particular set of programming technique grammar rules. The systemthen moves to the next level. This process continues until the most specializedtechnique has been recognized. In this case, a complete path of programmingtechniques at different levels of abstraction has been recognized and the student hasshown no problems with the programming techniques. So the tutoring is shifted tothe program itself.

Guided programming occurs when a complete path of programming techniquescannot be recognized in the hierarchy of relevant programming techniques. In theextreme cases, the student has not been able to use any programming technique at alland no programming technique is recognized. In some other cases, the student is ableto use some more abstract programming techniques so only those programmingtechniques at higher levels of abstraction can be recognized. The student has notbeen able to use those more specialized programming techniques at lower levels ofabstraction in his program. In either case, the system will tutor the student on theappropriate programming techniques so the student can start writing the program orfurther complete the program. The tutoring on programming techniques starts with

ARTICLE IN PRESS

recursive list processing technique

naive technique

list technique

accumulator technique inverse technique railway-shunt technique

Fig. 9. A hierarchy of relevant programming techniques for the list reversal exercise.


the programming technique that is the most specialized programming techniqueamong the ones recognized. If no programming technique has been recognized,the tutoring starts with the most abstract programming technique at the top level ofthe hierarchy. The system continues guiding the student through programmingtechniques at lower level of abstraction until the most specialized programmingtechnique in the hierarchy.

4.3. Guided programming

In this section, we explain how guided programming works in the Prolog Tutorusing an example. The dialogues between the system and the student is in italicsalong with appropriate program templates.

System: Write a program which reverses the elements of a list into another list.Student: Help.The student is given a programming exercise. The system sets the exercise by

accessing the exercise description in the exercise frame, list reversal. The student ishowever not able to start writing the program at all at this stage and asks the systemfor help. The system tries to guide the student through the relevant programmingtechniques at different levels of abstraction. This starts with the most abstractprogramming technique at the top level of the hierarchy of relevant programmingtechniques.

System: You can use a programming technique that recursively processes all elements

in the list. The following program template shows the structure of a program that uses

this programming technique.

reverse(/argumentsS).reverse(/argumentsS) :- /pre-predicatesS,

reverse(/argumentsS),/post-predicateS.

The system first tells the student which programming technique he can use andthen shows a program template for the student to work on. In this case, theprogramming technique is the recursive list programming technique. The descriptionof programming technique is from the corresponding programming technique frame.The program template is generated by parsing one of the reference programs for theprogramming exercise using the set of programming technique grammar rules forthe recursive list programming technique. A list of reference programs is given in theexercise frame. A reference program is chosen from the list if it is shown in thehierarchy of relevant programming techniques that the reference programimplements the programming technique. A reference program implements aprogramming technique at a higher level of abstraction if the program implementsone of the most specialized programming techniques that are connected to thehigher-level programming technique on the hierarchy of relevant programmingtechniques.

Student: More help.


System: You can use a programming technique that processes a list until it is empty

by splitting it into the head and the tail, making a recursive call with the tail.

reverse([ ], /argumentsS).reverse([H|T], /argumentsS) :- /pre-predicateS,

reverse(T,/argumentsS),/post-predicateS.

The student again asks for more help and the system moves to the programmingtechniques at the next level of abstraction. In this case, the student is guided to thelist technique. The program template is generated by parsing the currently chosenreference program for the exercise question using the set of programming techniquegrammar rules for the list technique.

Student: More help.System: You can further complete your program using either a naive technique, an

accumulator technique, an inverse naive technique or a railway-shunt technique.Student: I want to use the naive technique.The student still needs to know more specialized programming techniques and the

system moves to the programming techniques at the next level of abstraction.Because there is more than one programming technique at this particular level, thestudent is asked to choose a programming technique. In this case, there are fouralternative programming techniques, naive, accumulator, inverse naive and railway-shunt techniques. The student chooses the naive technique.

System: The naive technique allows you to process the list until it is empty

by splitting it into the head and the tail, making a recursive call with the tail and

putting the result from the recursive call and the processed head together as

required.

reverse([ ], /argumentS).reverse([H|T], /argumentS) :- reverse(T, /argumentS),

/post-predicateS.

The system tutors the student on the naive programming technique that thestudent has chosen. The system first checks whether the currently chosen referenceprogram is the one that implements the chosen programming technique. If this is notthe case, another reference program will be chosen from the list of referenceprograms, that implements the chosen programming technique. In this case, thecurrently chosen reference program actually implements the chosen programmingtechnique. So the program template is generated by parsing the currently chosenreference program for the programming exercise using the set of programmingtechnique grammar rules for the naive technique.

Student: reverse([ ], Reversed).reverse([H|T], Reversed):- reverse(T, ReversedT),

append(ReversedT, H, Reversed).


System: You seem to have known the correct programming technique for doing this

exercise. You will be given more tutoring on your program itself next.The student is finally able to write the program using the naive programming

technique. The system then moves to program tutoring.

4.4. Error diagnosis and correction

When the student has shown no problems with the programming techniques used,tutoring is shifted to the student program itself at the coding level. By now thesystem has recognized the most specialized programming technique that the studentused in his program. Furthermore, the student program has already been parsed bythe system using the programming technique grammar rules of one of the mostspecialized programming techniques. The system then gets the correspondingreference program for the exercise, in which the most specialized programmingtechnique, the same as the one used in the student program, is used. The referenceprogram is parsed using the same set of programming technique grammar rulesassociated with the most specialized programming technique. Finally, the parsing ofthe student program is compared with the parsing of the reference program. Whencomparing the two parsings, certain rules can be applied that allow the order of thepredicates in the reference program or the order of the clauses of a predicate in thereference program to be permuted. This allows the student program, in which theorder of predicates or the order of clauses within a predicate has been correctlypermuted, to be made to match the reference program.

When tutoring is conducted on the program at the coding level, the tutoringfocus shifts along the aggregation dimension from the most aggregated componentof the program to the least aggregated component of the program. Several types oferrors can be recognized based on the comparison. These include missing clauses,redundant clauses, missing sub-goals, redundant sub-goals, wrong ordering of sub-goals, term type errors, missing arguments, redundant arguments, wrong ordering ofarguments, and inconsistent uses of arguments. When an error is recognized by thesystem, it is explained to the student and he is asked to correct it. The systemconfirms the correction once it is made by the student. The system then moves on tothe next error. This continues until all errors have been corrected by the student.

In the following example, a student program has been recognized to use one of themost specialized programming techniques, naive reversal. The following referenceprogram for doing the given exercise using the programming technique is retrievedand parsed:

reverse([ ], [ ]).reverse([X|Y], L):- reverse(Y,M), append(M, [X], L).append([ ], L, L).append([H|T], X, [H|L]):- append(T, X, L).

The parsing of the reference program is made to match the parsing of the studentprogram without any permutations. By comparing the two parsings, two errors are


found in the base case and the recursive case of the student program, respectively.These errors are explained to the student who is then asked to correct the program.

System: Write a program which reverses the elements of a list into another list.

Student: reverse([ ], List).reverse([H|T], L):- reverse(T,L1), append(L1, H, L).

System: You seem to have used a programming technique that processes a list until it

is empty by splitting it into the head and the tail, making a recursive call with the tail

and putting the result from the recursive call and the processed head together as

required. Is that right?Student: Yes.The system tells the student the most specialized programming technique it has

recognized from the program and asks the student to confirm it. Once the studenthas confirmed it, the system moves on.

System: Great! You have used an appropriate programming technique for this

exercise. The base case in your program is, however, not quite right. It is supposed to

say that reversing an empty list gets the empty list itself. The second argument of your

base case, List, is a variable but it is supposed to be an empty list, [ ]. Now can you

correct your base case?Student: reverse([ ], [ ]).The system indicates an error to the student. In this case, there is an error in the

base case. The system has recognized this error when it compares the secondargument of the base case in the student program with the second argument of thebase case in the reference program in the two corresponding parsings. The systemknows by comparison that the second argument of the base case in the studentprogram, List, is a variable while the corresponding argument in the referenceprogram is an empty list, [ ]. The system then tells the student what he has in thesecond argument of the base case and what is supposed to be there. The student thencorrects the error in the base case.

System: Yes, correct. Your base case says that reversing an empty list gets the empty

list itself.The system confirms the correction made by the student. It then moves to the

second error in the recursive case.System: The recursive call in the body of your recursive case is correct but the post-

predicate isn’t. The second argument in the post-predicate should be a list rather than a

variable because the post-predicate is supposed to say that the list, L, is obtained from

appending the list, [H], and the list, L1. Now can you correct the post-predicate in the

body of your recursive case?Student: reverse([H|T], L) :- reverse(T, L1), append(L1, [H], L).The system confirms that the recursive call in the body of the recursive case is

correct and indicates an error in the post-predicate in the body of the recursive casebecause the second argument of the post-predicate is different from thecorresponding goal in the body of the recursive case of the reference program.The system then tells the student what he has in the post-predicate and what is


supposed to be there. The student again corrects the error in the post-predicate in thebody of the recursive case.

System: That’s correct. Your recursive case says that reversing a non-empty list

consisting of a head, H and a tail, T gets a list, L if the tail, T is reversed to get a list, L1,and the list, L is obtained from appending the list, L1 and the list, [H].

The system confirms the correction made by the student and states how itunderstands the recursive case. The system’s description of the recursive case isretrieved from the clause frame for the recursive case, where the template variableshave been replaced by the variables in the student program.

5. Experimental results

We implemented the Prolog Tutor in Prolog and tested it on a collection of 125programs for the programming exercise, list reversal. In the exercise, the student isasked to write a program, reverse/2, which reverses the elements of a list intoanother list. This collection of programs were used by Gegg-Harrison (1993,1994)for testing his Prolog tutoring system called ADAPT. Among these programs, thefirst 55 programs were originally used by Looi (1988,1991) for testing his Prologtutoring system called APROPOS2, and the remaining 70 programs were added byGegg-Harrison. The whole collection of programs includes 47 correct implementa-tions: variations #1, #4, #8, #17, and variations #83�#125. The remaining 78programs are incorrect implementations.

We tested the Prolog Tutor on this collection of programs in relation to three tasksof programming technique recognition, program correctness recognition, and erroranalysis and error correction.

5.1. Programming technique recognition

Programming technique recognition is concerned with recognizing the program-ming techniques that have been used in the student program. A hierarchy of relevantprogramming techniques as shown in Fig. 9 is first created for the list reversalexercise. In the exercise frame, a list of reference programs that are the standardprogram solutions to the exercise question are given. We have reference programsfor each of the most specialized programming techniques that can be used for theexercise. In the program frame of a reference program, the most specializedprogramming technique that the reference program has used is given. A path canthen be found in the programming technique hierarchy, that connects the mostspecialized programming technique to the most abstract programming technique. Allsuch paths are represented in the hierarchy of relevant programming techniques. Foreach of the programming techniques in the hierarchy, we have a set of programmingtechnique grammar rules.

Given a student program, programming technique recognition is best completed ifthe most specialized programming technique has been recognized being used in theprogram. There is then no need for guided programming and tutoring is shifted to


error analysis and correction straightaway. This is, however, not always possiblewhen the student program is poorly written where only some programmingtechniques at higher levels of abstraction can be recognized. In the worst cases wherethe student programs have been so poorly written, it might even not be possible at allto recognize any programming techniques.

We now discuss how the collection of 125 programs have been recognized in termsof programming techniques that have been used in these programs. Table 1 showsthe experimental results of program technique recognition obtained from the wholecollection of 125 programs. For each program in the collection, a path ofprogramming techniques in the hierarchy of relevant programming techniquesmay have been recognized being used in the program. This path starts with the mostabstract programming technique and ends at a more specialized programmingtechnique. The program, however, counts only once as one of the programs that hasused the programming technique at the end of the path. This is the programmingtechnique at the lowest level of abstraction that has been recognized being used in theprogram. If no programming technique has been recognized, the given programcounts as one of the programs whose techniques are unrecognized.

As shown in Table 1, in the collection of 125 programs, 46 out of 47 correctprograms and 54 out of 78 incorrect programs have been recognized using one of the

ARTICLE IN PRESS

Table 1

Results of programming technique recognition obtained from a collection of 125 programs

No. of correct programs using naive technique 17

No. of incorrect programs using naive technique 47

Total no. of programs using naive technique 64

No. of correct programs using accumulator technique 16

No. of incorrect programs using accumulator technique 4

Total no. of programs using accumulator technique 20

No. of correct programs using railway-shunt technique 5

No. of incorrect programs using railway-shunt technique 0

Total no. of programs using railway-shunt technique 5

No. of correct programs using inverse naive technique 8

No. of incorrect programs using inverse naive technique 3

Total no. of programs using inverse naive technique 11

Total no. of programs whose most specialized techniques recognized 100

No. of correct programs using list technique 0

No. of incorrect programs using list technique 18

Total no. of programs using list technique 18

No. of correct programs using recursive list processing technique 0

No. of incorrect programs using recursive list processing technique 2

Total no. of programs using recursive list processing technique 2

Total no. of programs whose more general techniques recognized 20

Total no. of programs whose techniques recognized 120

No. of correct programs whose techniques unrecognized 1

No. of incorrect programs whose techniques unrecognized 4

Total no. of programs whose techniques unrecognized 5

Total no. of programs for list reversal: 125


most specialized programming techniques. There are only 5 out of 125 programs inthe collection whose programming techniques have not been recognized at all.Among 120 programs in the collection, whose programming techniques have beenrecognized, 100 of them have been recognized using one of the most specializedprogramming techniques, 18 of them have been recognized using the list technique atthe second level of abstraction, and only 2 of them have been recognized using themost abstract programming technique at the top level of abstraction. These resultshave shown that excellent performance has been achieved in the Prolog Tutor interms of programming technique recognition and the Prolog Tutor also significantlyoutperforms both ADAPT and APROPOS2, in terms of programming techniquerecognition.

5.2. Program correctness recognition

Given a student program, in order to recognize its correctness, detect errorsin it and correct the errors, we need to compare the parsing of the student programwith the parsing of the reference program, in which the most specializedprogramming technique, the same as the one recognized being used in the studentprogram, is used. For the naive programming technique, a reference program is asfollows:

reverse([ ], [ ]).reverse([X|Y], L):- reverse(Y, M), append(M, [X], L).append([ ], L, L).append([H|T], X, [H|L]):- append(T, X, L).

In this reference program, a minor predicate, append/3, is also defined, whichsimply joins two lists to get the third one.

Apart from the above reference program, the naive programming technique canalso be implemented by another reference program as follows, in which a predicate,enqueue/3, instead of append/3, is defined.

reverse([ ], [ ]).reserve([X|Y], L):- reverse(Y, M), enqueue (M, X, L).enqueue([ ], X, [X]).enqueue([A|B], Y, [A|Z]):- enqueue(B, Y, Z).

Instead of joining two lists to get the third, enqueue/3 simply adds a singleelement to the end of a list to get another list.

For the accumulator programming technique, a reference program is asfollows:

reverse(X, Y):- reverse(X, [ ], Y).reverse([ ], L, L).reverse([H|T], Y, Z):- reverse(T, [H|Y], Z).


Of course, instead of using the second argument in the reverse/3 as theaccumulator, the third argument can also be used as the accumulator by simplyswitching the second and third arguments in the reverse/3 definition accordingly.So we also have another slightly different reference program as follows to implementthe accumulator programming technique:

reverse(X, Y):- reverse(X, Y, [ ]).reverse([ ], L, L).reverse([H|T], Z, Y):- reverse(T, Z, [H|Y]).

For the railway-shunt programming technique, a reference program is as follows:

reverse(A, B):- rev(A, [B]).rev([ ], [B|B]).rev([Head|Tail], [B|Temp]):- rev(Tail, [B,Head|Temp]).

For the inverse naive programming technique, a reference program is as follows:

reverse([ ], [ ]).reverse(L, [H|RT]):- append(T, [H], L), reverse(T, RT).append([ ], A, A).append([A|B], C, [A|D]):- append(B, C, D).

Similar to the naive programming technique, append/3 in the above referenceprogram can also be replaced by enquequ/3. So we also have another referenceprogram as follows to implement the inverse naive programming technique.

reverse([ ], [ ]).reverse(L, [H|RT]):- enqueue(T, H, L), reverse(T, RT).enqueue([ ], X, [X]).enqueue([A|B], Y, [A|Z]):- enqueue(B, Y, Z).

When comparing the parsing of any of the above reference programs with theparsing of a student program, the order of the predicates in the reference programand the order of the clauses of any predicate in the reference program may bepermuted. These permutations do not affect the correctness of these referenceprograms. A student program is then recognized as correct if it can be made tomatch the parsing or the permuted parsing of the reference program, in which themost specialized programming technique, the same as the one recognized being usedin the student program, is used. Table 2 shows the experimental results of programcorrectness recognition obtained from the whole collection of 125 programs.

As shown in Table 2, 100 out of 125 programs have been accurately recognized interms of program correctness. No correct program has been recognized as incorrect.Neither has any incorrect program recognized as correct. Among 25 programs whosecorrectness has not been recognized, only one program is correct. This shows that


when the correctness of a program could not be recognized, it is most likely that theprogram itself is incorrect and has probably been poorly written.

5.3. Error detection and correction

Error detection and correction is carried out on a program variation on the basisof comparing the parsings of the program variation and the corresponding referenceprogram both of which use the same most specialized programming technique. Sothe errors in those program variations, in which the most specialized programmingtechniques have not been recognized being used, cannot be either detected orcorrected. Where errors in these program variations can still be recognized by humanexperts, they are counted in the total number of undetected errors. Where programvariations are too poorly written and their errors cannot be recognized withoutknowing what have been intended in them, these errors have to be excluded from ourstatistical analysis of error detection and correction. Table 3 shows the experimentalresults of error detection and correction obtained from the whole collection of 125programs.

As shown in Table 3, 115 out of 167 errors can be detected and corrected. Fifty-two out of 167 errors cannot be detected and corrected simply because they haveoccurred in those program variations, in which none of the most specializedprogramming techniques can be recognized as being used. Though these errorscannot be directly detected and corrected in this mode of tutoring, they can be eithercorrected by the student himself when he is tutored in guided programming ordetected and corrected later after guided programming is completed and the most

ARTICLE IN PRESS

Table 2

Results of program correctness recognition obtained from a collection of 125 programs

No. of correct programs recognized as correct 46

No. of correct programs recognized as incorrect 0

No. of correct programs unrecognized 1

Total no. of correct programs 47

No. of incorrect programs recognized as correct 0

No. of incorrect programs recognized as incorrect 54

No. of incorrect programs unrecognized 24

Total no. of incorrect programs 78

Total no. of programs for list reversal 125

Table 3

Results of error detection and correction obtained from a collection of 125 programs

No. of errors detected and corrected 115

No. of errors detected but wrongly corrected 0

No. of errors unrecognized 52

Total no. of errors 167


specialized programming technique used in the program variation has beenrecognized. These results have shown the excellent performance of the Prolog Tutorin terms of error detection and correction.

6. Related work

To a certain extent, our programming technique-based approach to tutoringProlog programming resembles McCalla and Greer’s granularity-based approach totutoring Lisp programming (McCalla et al., 1992). Both approaches intend to tutorthe student on programming along two dimensions: abstraction and aggregation.Our programming technique hierarchy is similar to McCalla and Greer’s principalabstraction hierarchy. In association with each node of a principal abstractionhierarchy, they also have an aggregation hierarchy that describes the aggregationdimension of their granularity hierarchy. We instead have a set of programmingtechnique grammar rules associated with each node of our programming techniquehierarchy. Use of programming technique grammar rules provides a more flexibleway to describe the aggregation dimension of our programming technique hierarchy.The most important advantages of programming technique grammar rules are thatthey can be used for both recognizing programming techniques and generatingprogram templates at a particular level of abstraction.

The way our Prolog Tutor tutors Prolog programming at the coding level issimilar to Looi’s APROPOS2 (Looi, 1988,1991). However there are a number offundamental differences between the two systems. The tutoring of APROPOS2 isrestricted to the coding level only. It cannot tutor the use of appropriate Prologprogramming techniques. So there is basically no guided programming inAPROPOS2. The Prolog Tutor tutors Prolog programming at both the program-ming technique level and the coding level. It provides both guided programming andautomated error analysis. So it teaches not only how to write a program but alsohow to think about writing the program.

The program recognition in APROPOS2 relies on the code matching of thestudent program and a reference program. Any discrepancies between the twoprograms are treated as errors. The student program is then made to match one ofthe buggy versions of the reference program and the pre-generated commentariesassociated with the buggy version are given to the student telling him which parts ofthe program are wrong and how to correct them. The program recognition in ourProlog Tutor is based on the comparison of the parsings of the student program andthe reference program, respectively. The parsings are generated using theprogramming technique grammar rules and they carry grammatical informationexplicitly. So the Prolog Tutor is able to automatically generate the explanation onthe syntactic errors in the student program and suggest the corrections to the errorsbased on the reference program. It is therefore not dependent on the representationof the buggy versions of the reference program.

Gegg-Harrison (1991) emphasized the importance of using structured program-ming constructs in Prolog programming. His structured programming constructs can


be viewed as Prolog programming techniques (Brna et al., 1991,1999). He used a setof basic Prolog construct schemata to group together Prolog programs that sharethe same structured programming construct. A basic Prolog construct schema isthe structural representation of the corresponding group of programs which use thesame programming technique. The underlying knowledge base of Gegg-Harrison’sProlog tutor (Gegg-Harrison, 1993,1994), ADAPT, is a Prolog schema library whichconsists of a selected set of recursive list processing programs and their abstractions(or schemata). The Prolog schemata can be combined to form schema hierarchieswhere each parent node is a generalization of each of its children. The programs inthe Prolog schema library represent normal form programs. These normal formprograms can be systematically transformed into dozens of other transformednormal form programs using a set of general equivalence-preserving transformations.

The correctness assessment of the student program is based on the comparison ofthe student program to the closest matching normal form program or transformednormal form program. If no closest matching normal form program or transformednormal form program can be found for comparison with the student program, thisassessment cannot be carried out. The matching can sometimes be improved byunfolding clauses in the student program and permuting subgoals in the normal formprogram or transformed normal form program to obtain a closer match.

Gegg-Harrison’s schema representation is useful for program construction (Gegg-Harrison, 1992) because the student can be provided with the relevant programtemplates to complete and by doing so, the system has highlighted the generalprogramming technique being used and provided the student with the basic structureof this group of programs. The schema representation is, however, not quite suitablefor program debugging because it does not contain detailed grammaticalinformation. In the sense that the student program can be recognized only at thecoding level if the closest matching normal form program or transformed normalprogram can be found. Much recent work (Fuchs and Fromherz, 1992; Vasconcelosand Fuchs, 1996; Richardson and Fuchs, 1998; Flener and Deville, 1996;Buyukyildiz and Flener, 1998; Chasseur and Deville, 1998; Flener et al., 1998;Gegg-Harrison, 1997, 1995) has further developed schema-based approaches toprogram construction and transformation. These new approaches however inheritthe underlying problem of using schematic representation that relies on patternmatching for program recognition and is therefore less useful for program debuggingand tutoring.

Though we have also emphasized the importance of using programmingtechniques in Prolog programming, instead of schematic representation, we haveused programming technique grammar rules to represent Prolog programmingtechniques. Not only are programming technique grammar rules suitable forprogram construction, they are also suitable for program debugging. Sinceprogramming techniques are defined at different levels of abstraction, the studentprogram can be recognized at different levels of abstraction using different sets ofprogramming technique grammar rules for the corresponding programmingtechniques. Given a student program, the Prolog Tutor tries to parse it using a setof programming technique grammar rules for a programming technique. If it can be


parsed, the corresponding programming technique is recognized as being used inthe program. Even if none of the most specialized programming techniques can berecognized, some programming techniques at higher-level of abstraction could stillbe recognized as being used in the program.

The Prolog Tutor can recognize a much larger class of programs than ADAPT. Inthe collection of 125 program variations used in the tests, the Prolog Tutor canrecognize all but two of those program variations that can be recognized by ADAPTand 41 other program variations that cannot be recognized by ADAPT.

For example, for Variation #18 in the collection of 125 program variations asshown below, ADAPT merely recognizes the variation as not structural recursive butdoes not try to detect and correct any errors in it in detail.

Variation #18

app([ ], L, L).app([H|L1], L2, [H|L3]):- append(L1, L2, L3).reverse([ ], [ ]).reserve([H|T], Res):- app(Sofar, [H], Res), reverse(T, Sofar).

The Prolog Tutor can recognize straightaway that the naive programmingtechnique has been used in the variation and that the two subgoals in the body of therecursive case of reverse/2 are in the wrong order.

For Variation #32 in the collection as shown below, no transformed normal formprogram can be found in ADAPT and it is only recognized that the definition forappend/3 is missing.

Variation #32

reverse([ ], [ ]).reserve([H|T], X):- append(X, H, T), reverse(T, Y).

In the Prolog Tutor, though none of the most specialized programming techniquescan be recognized as being used in the variation, the list technique at a higher-level ofabstraction can however be recognized as being used in the variation. This recogni-tion is very useful in tutoring since there are clearly problems in the use of thesubgoal, append(X, H, T), the student should be guided on the use of the list techniqueand the correct use of this technique should lead to the correct use of this subgoal.

Soloway and his colleagues (Johnson and Soloway, 1985; Spohrer et al., 1985;Soloway, 1985) introduced the notion of programming plan for the detection andexplanation of the semantic errors produced by novice PASCAL programmers. Theplan-based approach may suit the analysis of structured programs written in aprocedural programming language. Its suitability for declarative programs written inProlog is an issue to be addressed.


The Lisp Tutor (Anderson and Reiser, 1985) contains a set of ideal rules torepresent the programming knowledge to be acquired by the student, a large set ofbuggy rules to represent misconceptions often developed by the student and thecorrections to the misconceptions. This allows it to follow the student’sprogramming and try to figure out what rules have led to the student’s program.When any errors are found in the program, it interrupts with immediate feedback.The Lisp Tutor, however, only has specific problem-solving rules that do notexplicitly represent conceptual structures of the programming language. As VanLehn(1996) indicated, it cannot explain or justify any of its rules.

7. Conclusions

In this paper, we discussed how to use Prolog programming techniques tocharacterize and group together Prolog programs that share the common pattern ofcode. We described how to use a set of programming technique grammar rules torepresent a programming technique, and how to organise these programmingtechniques in a hierarchical structure called the programming technique hierarchy.

We presented a new approach to the tutoring of Prolog programming to thenovice. Different types of frames are used to represent programming technique-related, program-related and clause-related programming knowledge, respectively.There is also a set of exercise frames that represent all programming exercises givento the student. These different types of frames provide a knowledge source to ourtutoring system. Our system can give directive guidance on the programmingtechniques to be used, recognize the programming technique used, detect differenttypes of errors in the student program, generate tutorial explanation on the errors tothe student, and suggest corrections to the errors.

We have implemented our Prolog Tutor in LPA WIN-PROLOG. Theprogramming technique grammar rules are written in DCG grammar in the system,which has provided a very efficient way to code these grammar rules. We have testedthe Prolog Tutor on a collection of 125 programs which consists of 47 correct and 78incorrect program implementations. The Prolog Tutor can recognize 46 correct and74 incorrect program implementations. The Prolog Tutor can recognize a muchlarger set of Prolog programs than the existing Prolog tutoring systems, such asAPROPOS2 and ADAPT.

In future work, we intend to investigate the possibility of automatically extractingnew programming techniques and representing them in programming techniquegrammar rules.

Acknowledgements

The author wishes to thank anonymous reviewers for their insights and helpfulcomments. This research was in part supported by the centre for software processTechnologies (CSPT) at the University of Ulster.


References

Anderson, J.R., Reiser, B.J., 1985. The lisp tutor. Byte 10, 159–175.

Bowles, A., Brna, P., 1999. Introductory Prolog: a suitable selection of programming techniques. In: Brna,

P., duBoulay, B., Pain, H. (Eds.), Learning to Build and Comprehend Complex Information

Structures: Prolog as a Case Study. Ablex, pp. 167–178.

Bowles, A., Robertson, D., Brna, P., 1999. A case-based reasoning approach to supporting novice

programmers. In: Brna, P., duBoulay, B., Pain, H. (Eds.), Learning to Build and Comprehend

Complex Information Structures: Prolog as a Case Study. Ablex, pp. 197–216.

Brna, P., Bundy, A., Dodd, T., et al., 1991. Prolog programming techniques. Instructional Science 20,

111–133.

Brna, P., Bundy, A., Dodd, T., et al., 1999. Programming techniques for prolog. In: Brna, P., duBoulay,

B., Pain, H. (Eds.), Learning to Build and Comprehend Complex Information Structures: Prolog as a

Case Study. Ablex, 143–166.

Bundy, A., Grosse, G., Brna, P., 1991. A recursive techniques editor for Prolog. Instructional Science 20,

135–172.

Buyukyildiz, H., Flener, P., 1998. Generalised logic program transformation schemas. In: Fuchs, N.E.

(Ed.), Proceedings of LOPSTR’97—International Symposium on Logic-based Program

Synthesis and Transformation, Lecture Notes on Computer Science, Vol. 1463. Springer, Berlin,

pp. 45–65.

Chasseur, E., Deville, Y., 1998. Logic program schemas, constraints and semi-unification. In: Fuchs, N.E.

(Ed.), Proceedings of LOPSTR’97—International Symposium on Logic-based Program Synthesis and

Transformation, Lecture Notes on Computer Science, Vol. 1463. Springer, Berlin, pp. 69–89.

Flener, P., Deville, Y., 1996. Logic program transformation through generalization schemata. In: Proietti,

M. (Ed.), Proceedings of LOPSTR’95—International Symposium on Logic-based Program Synthesis

and Transformation, Lecture Notes on Computer Science, Vol. 1048. Springer, Utrecht, The

Netherlands, pp. 171–173.

Flener, P., Lau, K.-K., Ornahgi, M., 1998. On correct program schemas. In: Fuchs, N.E. (Ed.),

Proceedings of LOPSTR’97—International Symposium on Logic-based Program Synthesis and

Transformation, Lecture Notes on Computer Science, Vol. 1463. Springer, Berlin, pp. l24–143.

Fuchs, N.E., Fromherz, M.P.J., 1992. Schema-based transformations of logic programs. In: Clement,

T.P., Lau, K. (Eds.), Proceedings of LOPSTR’91—International Symposium on Logic-based Program

Synthesis and Transformation, Lecture Notes on Computer Science. Springer, Berlin, pp. 111–125.

Gegg-Harrison, T.S., 1991. Learning Prolog in a schema-based environment. Instructional Science 20,

173–192.

Gegg-Harrison, T.S., 1992. Adapting instruction to the student’s capabilities. Journal of Artificial

Intelligence in Education 3, 169–181.

Gegg-Harrison, T.S., May 1993. Exploiting program schemata in a prolog tutoring system. Ph.D.

Dissertation, Technical Report CS-1993-11, Department of Computer Science, Duke University,

Durham, North Carolina.

Gegg-Harrison, T.S., 1994. Exploiting program schemata in an automated program debugger. Journal of

Artificial Intelligence in Education 5, 255–278.

Gegg-Harrison, T.S., 1995. Representing logic program schemata in Lambda-Prolog. In: Sterling, L.S.

(Ed.), Proceedings of the 12th International Conference on Logic Programming. MIT Press,

Kanagawa, Japan, pp. 467–481.

Gegg-Harrison, T.S., 1997. Extensible logic program schemata. In: Gallagher, J. (Ed.), Proceedings of

LOPSTR’96—the Sixth International Workshop on Logic Program Synthesis and Transformation,

Lecture Notes on Computer Science, Vol. 1207. Springer, Berlin, pp. 256–274.

Gegg-Harrison, T.S., 1999. Exploiting program schemata to teach recursive programming. In: Brna, P.,

duBoulay, B., Pain, H. (Eds.), Learning to Build and Comprehend Complex Information Structures:

Prolog as a Case Study. Ablex, pp. 347–379.

Johnson, W.L., Soloway, E., 1985. Proust: knowledge-based program understanding. IEEE Transactions

of Software Engineering 11 (3), 267–275.


Looi, C.K., 1988. Automatic program analysis for a prolog intelligent teaching system. Ph.D.

Dissertation, Department of Artificial Intelligence, University of Edinburgh, Edinburgh, Scotland.

Looi, C.K., 1991. Automatic debugging of prolog programs in a prolog tutoring system. Instructional

Science 20, 215–263.

McCalla, G.I., Greer, J.E., Barrie, B., Pospisil, P., 1992. Granularity hierarchies. Computers and

Mathematics with Applications: Special Issue on Semantic Networks 23, 363–376.

Richardson, J., Fuchs, N.E., 1998. Development of correct transformation schemata for Prolog programs.

In: Fuchs, N.E. (Ed.), Proceedings of LOPSTR’97—International Symposium on Logic-based

Program Synthesis and Transformation, Lecture Notes on Computer Science, Vol. 1463. Springer,

Berlin, pp. 263–282.

Soloway, E.M., 1985. From problems to programs via plans: the content and structure of knowledge for

introductory lisp programming. Journal of Educational Computing Research 1, 157–172.

Spohrer, J.C., Soloway, E., Pope, E., 1985. A goal-plan analysis of buggy Pascal programs. Human–

Computer Interaction 1, 163–207.

Taylor, J., 1999. Analyzing novices analyzing Prolog: what stories do novices tell themselves about Prolog?

In: Brna, P., duBoulay, B., Pain, H. (Eds.), Learning to Build and Comprehend Complex Information

Structures: Prolog as a Case Study. Ablex, 43–71.

Taylor, J., du Boulay, B., 1987. Studying novice programmers: why they may find learning Prolog hard.

In: Rutkowska, J.C., Crook, C. (Eds.), Computers, Cognition and Development: Issues for Psychology

and Education. Wiley, New York.

van Someren, M.W., 1990. What’s wrong? Understanding beginners’ problems with Prolog. Instructional

Science 19, 257–282.

VanLehn, K., 1996. Conceptual and meta learning during coached problem solving. In: Intelligent

Tutoring Systems: Proceedings of the Third International Conference, ITS’96. Montreal, Canada,

pp. 29–47.

Vasconcelos, W.W., Fuchs, N.E., 1996. An opportunistic approach for logic program analysis and

optimization using enhanced schema-based transformations. In: Proietti, M. (Ed.), Proceedings of

LOPSTR’95—International Symposium on Logic-based Program Synthesis and Transformation,

Lecture Notes on Computer Science, Vol. 1048. Springer, Utrecht, The Netherlands, pp. 174–188.


Guided programming and automated error analysis in an intelligent Prolog tutor

Documents

Transcript of Guided programming and automated error analysis in an intelligent Prolog tutor