Goteborg University Dialogue Systems Lab Using TrindiKit and GoDiS as OAA resources TALK Edinburgh...

Goteborg University Dialogue Systems Lab

Using TrindiKit and GoDiS as OAA resources

TALKEdinburgh 7/10 2004


Some of our WP5 work

• Improved GUI– logging & monitoring dialogue, incl.

recorded speech

• Further integration with OAA – OAA API for TrindiKit– GoDiS DME as OAA agent– can be used as ”black box”


TrindiKit 3.2 GUI

• Example from GoDIS-VCR corpus • TrindiGui displaying a GoDiS-VCR

session• Audio files linked with utterances• Access to information state at different

stages of the dialogue• (see demo)


TrindiKit 4, GoDiS, and OAA

• TrindiKit 4 can be seen as a tool for building ISU dialogue systems as collectives of OAA agents

• TrindiKit 4 enables building OAA agents containing one or more of– information state (currently only one agent can have IS)– modules (parser etc.)– resources (databases etc.)

• A simple example OAA-based system:– TrindiKit used to build DME– other OAA agents used for SR, TTS, etc– control handled by control agent (either TrindiKit-style, or ”home-

made”)• TrindiKit control agent supports asynchronous control via IS triggers

• The DME can still be run without OAA, as a single Prolog process (useful for debugging)


interpr.agent

ISMIVsRIVsrules

deviceVCR

systemcontrol

update select

GFagent

domainVCR

OAAfac.

TrindiGUI

GoDiS core

Graphicalinput

NuanceASR +

TTS module

GFmodule

Log

VCR

text I/O

module


OAA API for TrindiKit 4

• Specifies solvables for– TIS agent– Module agent– Resource agent– Control agent


TIS agents

• TIS agents include the Total Information State; currently written in prolog

• Solvables:– tkit_check(+C): C is a condition or a list of conditions; returns

true if conditions hold– tkit_apply(+O): O is a operation or a list of operations; returns

true if successful; failure leads to error message– tkit_reset: resets TIS– tkit_print_state: makes TIS agent print the contents of current

TIS– tkit_add_triggers(Ts): Ts is a list of triggering conditions.

Whenever a triggering condition T becomes true, TIS agent calls solvable tkit_trigger(T).

– tkit_clear_triggers: Clears all triggers– tkit_load_resource(ResourceFileName, ResourceType)– tkit_connect_resource_agent(ResourceAgentName,

ResourceType)


Module agents

• Trindikit module agents can be written in any programming language supported by OAA.

• A module agent defines one or several Trindikit modules, and their algorithm.

• A module agent can check contents of TIS (handled by TIS agent) by calling TIS solvable tkit_check/1 and also alter the TIS by calling solvables tkit_apply/1 and tkit_apply_rule/3

• Solvable:– tkit_call_module(+Module,+Algorithm): make

module Module start executing module algorithm Algorithm.


Resource agents

• Resources may be part of TIS, or separate agents– if part of TIS, handled by TIS agent

• Solvables for resource agents include:– tkit_resource_check (+R, +C): C is a

condition or a list of conditions; returns true if conditions hold of resource agent R

– tkit_resource_apply(+R, +U): U is a operation or a list of updates to resource R; returns true if successful; failure leads to error message


Control agent• Standalone SICStus OAA agent used for

controlling the execution of Trindikit module algorithms.

• Reads in a asynchronous control algorithm from a file (either at specified at startup or by calling it's OAA solvable tkit_load_control_algorithm(+File)).

• The asynchronous control is algorithm consists of a prolog list of the form [ Trigger1 => SerialAlgorithm1, ..., TriggerN => SerialAlgorithmN ].


• Should be possible to read in a serial algo too!?

• Should also allow calling arbitrary (non-TK) agents?– oaa_Solve(nuance_agent, getspeech(String,Score)),– input:=String, – score:=Score, – gf_agent:parse,– godis_dme:update, – godis_dme:select,– gf_agent:linearize,– $output = Output,– oaa_Solve(nuance_agent, sayspeech(Output))


• When a trigger fires, the control agent will execute the associated (serial) algorithm, possibly in parallel with other serial algorithms which have been triggered before or at the same time.

• Two types of triggers are currently implemented: – init-triggers which are fired at initialization – condition-triggers which are fired when a

TIS variable changes value.


• Solvables– tkit_control_load_algorithm(+File):loads

parallel control algorithm defined in File– tkit_control_run: starts trindikit by sending

triggers to TIS-agent and starting to execute the algorithm(s) triggered by init.

– tkit_trigger(+Trigger): causes control agent to start executing all algorithms triggered by Trigger. Called by TIS agent, when a triggering condition becomes true.

– tkit_control_stop: stops all serial algorithms and causes control agent to stop responding to tkit_trigger/1 calls.


Sample GoDiS control algorithm% input runs as a separate (active) agent, not controlled by algorithm,% except for when initializing and quitting

% initialisationinit => [ input:init, output:init, generate:init, interpret:init, select,generate].

%quittingcondition( $program_state == quit) => [ input:quit,output:quit, generate:quit, interpret:quit].

% interpret, update,select and generate run controlled by a single% algorithm which triggers on setting of input variable.condition( is_set(input)) => [ interpret, update, select, if not is_empty($latest_moves) then [generate] ].

% output triggers on setting of output variablecondition(not empty($output)) => [ output, update].


Asynchronicity in TrindiKit4

• TrindiKit threaded control agent can be used to set up triggers– if X becomes true of IS, tell agent Y to

do Z• Other OAA Agents may be

– passive, waiting for calls from TrindiKit, or

– active, writing to the information state


Benefits of OAA API

• TrindiKit and GoDiS become more accessible from OAA• TrindiKit becomes less monolithic, more

comprehensible, easier to use & learn• A TrindiKit system such as GoDiS can be used as a

”black box” in an OAA-based system• Allows agents in other languages

– connected via ”shell module” in DME (already possible), or– implemented as TrindiKit agent, i.e. able to communicate

using the TrindiKit/OAA API (in TrindiKit4)• TrindiKit4 will provide Java classes for easy TIS access

• Potential for rewriting TrindiKit in other programming language, without affecting implemented systems– This would be further strengthened by a Prolog-

independent (but equally powerful) TrindiKit language


TrindiKit 4 time plan

• Preliminary release projected Dec 04

• Collect user feedback; maintainence & successive improvements

• Final version at end of project• Contributions from other sites

welcome (new agents etc.)


Differences between TrindiKit 4 and DIPPER

Staffan Larsson 8/10 2004Edinburgh TALK meeting

(draft only, not presented)


Overview

• Rule language• Rule coordination• Overall control• Accessing other OAA agents• Programming language


Rule language

• Prolog variables?– TKIT: yes, more expressive language; easier to write

rules– DIPPER: no, Prolog independent

• OAA calls in rule effects?– TKIT: no

• This should be handled by overall control strategy rather than in rules

• Still possible to connect OAA agents as resources– but resources cannot affect TIS– need to define them as object of a certiain datatype

specifying relations, functions, operations

– DIPPER: yes; convenient & simple


Rule language cont’d

• Prolog calls?– TKIT: no

• takes update rule outside scope of TKIT• decreases utility of TKIT

– e.g. the GUI cannot be used to see what’s happening• however, Prolog predicates can be defined in resources

– need to define resource as object of a certiain datatype specifying relations, functions, operations

– each predicate mus be defined as relation, function, ... in the resource interface (one for each resource type)

– DIPPER: yes• convenient & simple• needed since rule lang does not allow variables (except

in prolog calls, presumably?)


Rule language cont’d

• Explicit evaluation?– TKIT: yes– DIPPER: no


Rule coordination

• Rule coordination algorithms?– TKIT: yes

• When number of rules is large, having an algorithm makes things more manageable, and the rules shorter

– DIPPER: no• Always possible to determine which rule will trigger

next, by inspecting state and rules• In TKIT, coordination algorithm execution state will also

influence which rules are triggered

• Multiple DME modules in DME agent?– TKIT: yes

• DME might not be monolithic; e.g. selection rules only useful after user utterances

– DIPPER: no(?)


Overall control

• TKIT: – control agent– strict separation of agent types

• modules: – cannot be called by rules– can access TIS

• resources:– can be called by rules– cannot access TIS

– pros: clean, object-oriented, modular, decrease risk of asynchronicity problems, easier to debug(?)

– cons: extra work• DIPPER:

– control handled by DME– OAA calls encoded in update rules– pros: simple to set up quickly (?)– cons: less clean, higher risk of asychronicity problems, harder to

debug(?)


Accessing other agents

• DIPPER:– update rules may call other agent using whatever solvables the agent offers– also specifies what to do with the result, e.g. write it to the TIS– so arbitrary OAA agents can be connected without needing modification– passive agents only

• TKIT– each other agent must be defined as module or resource– module agents

• need to specify a set of algorithms• called by control agent, specifying only algorithm name• agent needs to know what to do with the result, i.e. needs to be adapted to TKIT,

and perhaps to a particular DME (e.g. GoDiS)• TKIT offers active module agents, which are initiated and then run on their own,

accessing the TIS– resource agents

• If connected via TIS, and thus accessible to update rules, need to be specified as object of type, which will provide a resource interface

• If accessed directly (not going via TIS), accessed using whatever solvables it defines; not accessible to update rules but stil accessible to OAA community and thus to module agents


Programming language

• TKIT: SICStus prolog– con: expensive

• DIPPER: SICStus prolog, Java

Goteborg University Dialogue Systems Lab Using TrindiKit and GoDiS as OAA resources TALK Edinburgh...

Documents

Transcript of Goteborg University Dialogue Systems Lab Using TrindiKit and GoDiS as OAA resources TALK Edinburgh...