Pattern-Based French-to-IF Analysis Fill in the blanks IF-to-French Generation within NESPOLE!
description
Transcript of Pattern-Based French-to-IF Analysis Fill in the blanks IF-to-French Generation within NESPOLE!
Pattern-Based French-to-IF AnalysisFill in the blanks IF-to-French
Generationwithin NESPOLE!
Hervé BlanchonCLIPS-IMAG
Outline
Introduction Analysis: Pattern-based approach
– Analyzer architecture– Steps in detail
Turn splitting into SDU Topic, sub-domain detection Argument filling DA construction
Generation– Track 1: Fill in the blanks approach– Track 2: Rule-based approach
First results Perspectives
Introduction
Handling the client's utterances
French SLT modulesFrench SLT modules
Speech RecognitionRAPHAEL
French textual output
Pattern-BasedAnalysis
IFs
Introduction
Handling the agent's utterances
French SLT modulesFrench SLT modules
Generation-Fill-in the blanks approach
-Rule-based approach
French textual output
Speech SynthesisEuler (Mons)
Sound file
IFsIFs
Hypothesis– c: c: j aimerais organiserj aimerais organiser une semaineune semaine de de
vacancesvacances dans un parc dans un parc ((I would like to I would like to organizeorganize a weeka week ofof holydaysholydays in a park in a park))
IF– {c: {c:
((
)} )}
IntroductionInterchange FormatInterchange Format
give-informationgive-information+disposition+disposition+reservation+reservation+trip+trip
Pred-partMain-predAttitudeSpeech Act
Introduction
Hypothesis– c: c: j aimerais organiserj aimerais organiser une semaineune semaine de de
vacancesvacances dans un parc dans un parc ((I would like to I would like to organizeorganize a weeka week ofof holydaysholydays in a park in a park))
IF– {c: {c:
((
)} )}
give-informationgive-information+disposition+disposition+reservation+reservation+trip+tripdisposition=( , )disposition=( , ),,
visit-spec=( , )visit-spec=( , ),,location= location= ,,duration=( , )duration=( , )
Top-level arguments
Interchange FormatInterchange Format
Introduction
Hypothesis– c: c: j aimerais organiserj aimerais organiser une semaineune semaine de de
vacancesvacances dans un parc dans un parc ((I would like to I would like to organizeorganize a weeka week ofof holydaysholydays in a park in a park))
IF– {c: {c:
((
)} )}
give-informationgive-information+disposition+disposition+reservation+reservation+trip+tripdisposition=( , )disposition=( , ),,
visit-spec=( , )visit-spec=( , ),,location= location= ,,duration=( , )duration=( , )
who=who=identifiability=identifiability=time-unit=time-unit= quantity=quantity=
Interchange FormatInterchange Format
Embedded arguments
Introduction
Hypothesis– c: c: j aimerais organiserj aimerais organiser une semaineune semaine de de
vacancesvacances dans un parc dans un parc ((I would like to I would like to organizeorganize a weeka week ofof holydaysholydays in a park in a park))
IF– {c: {c:
((
)} )}
give-informationgive-information+disposition+disposition+reservation+reservation+trip+tripdisposition=( , )disposition=( , ),,
visit-spec=( , )visit-spec=( , ),,location= location= ,,duration=( , )duration=( , )
who=who=identifiability=identifiability=time-unit=time-unit= quantity=quantity=
nonovacationvacationii likelike
weekweek 11parkpark
Interchange FormatInterchange Format
Values
Introduction
Hypothesis– c: c: j aimerais organiserj aimerais organiser une semaineune semaine de de
vacancesvacances dans un parc dans un parc ((I would like to I would like to organizeorganize a weeka week ofof holydaysholydays in a park in a park))
IF– {c: {c:
((
)} )}
give-informationgive-information+disposition+disposition+reservation+reservation+trip+tripdisposition=( , )disposition=( , ),,
visit-spec=( , )visit-spec=( , ),,location= location= ,,duration=( , )duration=( , )
who=who=identifiability=identifiability=time-unit=time-unit= quantity=quantity=
nonovacationvacationii likelike
weekweek 11parkpark
Back-translation– J'aimerais organiser des vacancesJ'aimerais organiser des vacances dans un parcdans un parc pendant une pendant une
semainesemaine. . ((I would like to organizeI would like to organize holydaysholydays in a park in a park during a weekduring a week..))
Interchange FormatInterchange Format
Analysis
Necessity to deal with– Insertions– Deletions– Wrong words & agreements
Arguments as the core of an IF– Constrain the possible ones through a topic– Find the realized ones among the possible ones
Phrase spotting mechanism– Regular expressions to express their realization– Minimal linguistic treatments
ChoiceChoice
Analysis
ASR Output Splitting into SDUs
SDU 1 SDU 2 SDU n…
Topic Detection
SDU j Topic i
Topic i handling
IFs
DA Construction
Arguments fillingArg k filling
…
…
ArchitectureArchitecture
Analysis
Simple sentences & phrases as SDU boundaries– Leading to terminal SA or SA with few continuations
AffirmAffirm– oui c'est ça, bien sûr oui c'est ça, bien sûr – yes that's it, of course– yes that's it, of course
NegateNegate – non pas du tout, non pas très bien, nonnon pas du tout, non pas très bien, non – no not at all, no not very – no not at all, no not very
well, nowell, no ExclamationExclamation
– c'est excellent, très bien, ohc'est excellent, très bien, oh – that's excellent, very good, oh – that's excellent, very good, oh GreetingGreeting
– bonjour, au revoir, bonne journéebonjour, au revoir, bonne journée – hello, good bye, have a good – hello, good bye, have a good dayday
Splitting turns into SDUSplitting turns into SDU
Splitting turns into SDUSplitting turns into SDUAnalysis
Articulations as SDU boundaries– Rhetorical + pronoun or question
ExamplesExamples– et donc jeet donc je , et puis il, et j', donc on, est-ce que, quel , et puis il, et j', donc on, est-ce que, quel
est est – and thus I, and then he, and I, so we, is …, what isand thus I, and then he, and I, so we, is …, what is
Treatment of the pronouns– When rhetorical is not explicitly stated
ExamplesExamples– xxx pronoun yyy (the|that|if) zzz pronoun tttxxx pronoun yyy (the|that|if) zzz pronoun ttt
SDU = xxx pronoun yyy (the|that|if) zzz pronoun tttSDU = xxx pronoun yyy (the|that|if) zzz pronoun ttt– xxx pronoun yyy pronoun zzzxxx pronoun yyy pronoun zzz
SDU 1 = xxx pronoun yyySDU 1 = xxx pronoun yyy pronoun zzz may have to be splitpronoun zzz may have to be split
Analysis
Goal– Find a terminal SAterminal SA or a focus conceptfocus concept– Restrict the search for instantiated
arguments Means
– Keywords are associated with each topic– Early matched topic is selected
Some topicsSome topics affirm, greeting, acknowledgment, exclamation, affirm, greeting, acknowledgment, exclamation,
please-wait, …please-wait, … accommodation, activity, attraction, package, room, accommodation, activity, attraction, package, room,
trip, …trip, …
Topic detectionTopic detection
Analysis
A function per topic (Topic2If)– Find the realized (instantiated)
arguments among the possible ones– Calculate the DA– Concatenate
The speaker The DA The arguments
Topic handlingTopic handling
Analysis
A specialized function per argument (Arg2If)– Describing the possible realizations– Capturing the pertinent information
ExempleExemple
Argument fillingArgument filling
proc RoomSpec2If {inString} {proc RoomSpec2If {inString} {
......
} elseif {[regexp } elseif {[regexp "(?:les) (\[0\-9\]+) ($fifservdico::frenchroom)(?:x|s)?""(?:les) (\[0\-9\]+) ($fifservdico::frenchroom)(?:x|s)?" $inString lMatch lQuantity lRoom]!=0} { $inString lMatch lQuantity lRoom]!=0} {
append the_result append the_result "room-spec=(identifiability=yes, quantity=" $lQuantity ", ""room-spec=(identifiability=yes, quantity=" $lQuantity ", " [fifservdico::Room2If $lRoom] ")" [fifservdico::Room2If $lRoom] ")"
} elseif {[regexp } elseif {[regexp "(\[0\-9\]+) ($frenchroom)(?:x|s)?""(\[0\-9\]+) ($frenchroom)(?:x|s)?" $inString lMatch lQuantity lRoom]!=0} { $inString lMatch lQuantity lRoom]!=0} {
append the_result append the_result "room-spec=(identifiability=no, quantity=" $lQuantity ", ""room-spec=(identifiability=no, quantity=" $lQuantity ", " [fifservdico::Room2If $lRoom] ")" [fifservdico::Room2If $lRoom] ")"
} }
......
}}
Analysis
Construct the DA using– The attitudes– The realized arguments
To get the main-predication and the pred-participants
– Other information Subject Verbal construction (give-information, request-
information, request-suggestion, suggest) Negation of the predicate …
DA constructionDA construction
Analysis
Hypothesis– c: c: j aimerais préparerj aimerais préparer une semaineune semaine de de
vacancesvacances dans un parcdans un parc # #– c: c: i would like to preparei would like to prepare a weeka week of of
vacationvacation in a parkin a park ## Domain: Domain:
– visitvisit Arguments: Arguments:
– dispositiondisposition, , visit-specvisit-spec, , durationduration, , locationlocation DA:DA:
– give-give-information+disposition+reservation+triinformation+disposition+reservation+tripp
ExempleExemple
Generation
Fill in the blanks approach (next slides) Rule based approach
– Use (parse) the IF specification files To build dictionary and grammar skeletons To label semantically each link between a terminal
symbol of the IF with its possible continuations
– Steps Associate to each terminal of the IF input the list of its
possible labelled continuations; Built a semantic tree of the IF according to the
continuations; Translate all the terminals and structures of the IF into
French words and structures; Generate a syntactically well-formed sentence in
French.
ApproachesApproaches
Generation
Step 1: Find the normalized DA– Discarding the disposition(s)
Step 2: Find the “fill in the blank sentence”– According to some of the possible
arguments Step 2: Fill in the blanks
– With the phrases generated for the arguments
Fill in the blanksFill in the blanks
Generation
IF c:c:give-information+disposition+reservation+tripgive-information+disposition+reservation+trip ((disposition=(who=i, like)disposition=(who=i, like), , visit-visit-spec=(identifiability=no, vacation)spec=(identifiability=no, vacation), ,
location=parklocation=park, , duration=(time-unit=week, quantity=1)duration=(time-unit=week, quantity=1)))
Fill in the blank sentenceifDisposition2TextifDisposition2Text organiser organiser ifVisitSpec2textifVisitSpec2text ifLocation2TextifLocation2Text ifTime2Text ifTime2Text ifDuration2TextifDuration2Text..
Result– J'aimerais J'aimerais organiserorganiser des vacances des vacances dans un parcdans un parc
pendant une semainependant une semaine..– I would like I would like to organizeto organize vacations vacations in a parkin a park during a during a
weekweek..
Fill in the blanksFill in the blanks
First results
4 unseen dialogues, clients turns– Transcription as References
2 settings– Manually segmented into SDU (all languages)– Automatically segmented into SDU (analysis
module - French only) 5 sets
– ASR alone (WER, Hypos as paraphrase of SDU)– FR-FR on ASR, FR-FR on References– FR-IT on ASR, FR-IT on References
3 graders & 3 grades– p: perfect, k: OK, b: bad
(p+k=acceptable)
EvaluationEvaluation
First results
Fr-Fr and Fr-It are comparable with Xx-Xx and Xx-It It-Fr is performing less than It-Xx
– French generator less oriented towards agent's IFs
EvaluationEvaluationMonolingual (%acceptable)
on Ref//on Rec[on auto SDU]
En-En58%//45%
Ger-Ger31%//25%
Fr-FrFr-Fr
54%/54%/41%41%[62%/[62%/48%]48%]
It-It61%//48%
Bilingual (%acceptable)
on Ref//on Rec[on auto SDU]
En-It55%//43%
Ger-It32%//27%
Fr-ItFr-It
44%/44%/34%34%[58%/[58%/44%]44%]
It-En47%//37%
It-Ger47%//31%
It-FrIt-Fr
40%/40%/27%27%
Perspectives
Pattern-based analysis– Quite promising– Analyzer for second showcase under
development more concepts (actions, attitudes, feature),
arguments (feature, focalizer, modifier, rhetorical)
Fill in the blank generation– Towards a better coverage with greater
factorization of the DA Rule-based generation
– Should be available for second showcase evaluation