Center for PersonKommunikation P.1 CPK NLP Suite for Spoken Language Understanding ( tb/nlpsuite)...
-
date post
20-Dec-2015 -
Category
Documents
-
view
216 -
download
1
Transcript of Center for PersonKommunikation P.1 CPK NLP Suite for Spoken Language Understanding ( tb/nlpsuite)...
P.1
Center for PersonKommunikation
CPK NLP Suite for Spoken Language Understanding
(http://www.cpk.auc.dk/~tb/nlpsuite)
Tom Brøndsted, CPK. Aalborg [email protected]
P.2
Center for PersonKommunikation
Technical Details
• Programs for syntactic/semantic parsing, generation of language models …
• Reads/writes a number of grammar formats
• Free for research/non-commercial use
• Available in source (C++, lex/yacc)
• Compiles and runs on “any” machine under “any” OS having a 32 bit C++ compiler
P.3
Center for PersonKommunikation
25 ProgramsAPS PTR PSG ICM VOC
PARS
CONV
TREC
TSLU
SGEN
e.g. psgtrec(.exe)
(what is done with/to the grammar)
(grammar format)
P.4
Center for PersonKommunikation
Grammar Formats IAPS PTR PSG ICM VOC
PARS
CONV
TREC
TSLU
SGEN
Augmented Phrase Structure Grammer format based on a subset of the “EUROTRA User Language”
•Compound (but non-nested) features and structure sharing (“variables”)
•Semantic frames are created either
•through percolation of features
•with mapping rules creating frames when specified conditions are met
P.5
Center for PersonKommunikation
Grammar Formats IIAPS PTR PSG ICM VOC
PARS
CONV
TREC
TSLU
SGEN
grammar format based on PC-PATR/PATR II (Shieber etc.)
•Compound (including nested) features and structure sharing (“variables”)
•Semantic frames are created
•through percolation of features
•no mapping
P.6
Center for PersonKommunikation
Grammar Formats III-IV-VAPS PTR PSG ICM VOC
PARS
CONV
TREC
TSLU
SGEN
Simple symbol-based (non-unification based) formats including:
•Simple BNFs intended for teaching
•Finite state grammars intended for speech recognition
•[further details in the paper]
P.7
Center for PersonKommunikation
Program Types IAPS PTR PSG ICM VOC
PARS
CONV
TREC
TSLU
SGEN
Natural language parser
Input: 1-best/n-best. Output: semantic frames
•Unification-based left-corner chart parser
•Bottom-up, left-right with top-down filtering
•Exhaustive parsing
•API: allows interface to speech recogniser and dialogue management
P.8
Center for PersonKommunikation
Program Types IIAPS PTR PSG ICM VOC
PARS
CONV
TREC
TSLU
SGEN
Grammar converter
Input: NL-Grammer. Output: equivalent language model
•Types of language models: RTNs, FSNs, Word Pair, “No grams”
•can be written in the HTK standard lattice format etc.
•can substitute the network loading routine of a speech recogniser
P.9
Center for PersonKommunikation
Program Types III-IVAPS PTR PSG ICM VOC
PARS
CONV
TREC
TSLU
SGEN
Grammar constrained typed text recogniser, simulation of speech recognition/understanding
Input: typed sentence. Output: a “similar” sentence covered by the grammar (or corresponding frames)
•standard linear Viterbi pattern-matching algorithm
•uses FSN (“full-gram”) constraints
•supports “garbage models”
P.10
Center for PersonKommunikation
Program Types VAPS PTR PSG ICM VOC
PARS
CONV
TREC
TSLU
SGEN
Random sentence generator, exploring the coverage of grammars
Input: a NL-grammar. Output: a number of sentences covered by the grammar
•Uses a fully equivalent RTN derivation of the grammar
•Corresponds to the HTK-utility HSgen
P.11
Center for PersonKommunikation
ANSI C API (+JAVA)APS PTR PSG ICM VOC
PARS
CONV
TREC
TSLU
SGEN
Functions for
•Loading grammars
•Activating/deactivating subgrammars
•retrieving semantic frames from recognition results
•ANSI C structure implemented for semantic frames
P.12
Center for PersonKommunikation
Adding a NL Grammar FormatAPS PTR PSG ICM VOC XXX
PARS
CONV
TREC
TSLU
SGEN XXXlex/yacc
code
P.13
Center for PersonKommunikation
Adding a Recognition Grammar Format
APS PTR PSG ICM VOC
PARS
CONV
TREC
TSLU
SGEN
RTN
FSN
WPG
NOG
HTK write procedure
VOC write procedures
XXX write procedure (ca. 80 lines of code)
P.14
Center for PersonKommunikation
Conclusion
• Represents work in progress
• Available for non-commercial use “as is”
• Known inconveniences
• Users are welcome to extend the suite
P.15
Center for PersonKommunikation
Basic idea: Controlling speech understanding via one grammar
Speech recogniser
Natural languageparser
“sentence”
semantic frames
grammar
grammar
conversion
speech
P.16
Center for PersonKommunikation
Tutorial Example in Paper
GrapHvite(Entropic)
APSPARS
“sentence”
semantic frames
HTKstandard lattice
EUROTRAAPS Gr.
APSCONV
speech
P.17
Center for PersonKommunikation
Application Example
• Voice driven email application from the GrapHvite tutorial (Entropic)– A. Using the Entropic grammar tools (HTK and
GrapHvite tools) to make HTK standard lattice– B. Using the CPK NLP suite to implement the APS
grammar and the HTK standard lattice.
P.18
Center for PersonKommunikation
Using the HTK Grammar Tools
• An EBNF-like grammar definition language – HTK tools: Hparse, Hbuild, HSGen, HDMan
$digit = ONE | TWO | THREE | FOUR | FIVE | SIX | SEVEN | EIGHT | NINE | OH | ZERO;$name = [ JOOP ] JANSEN | [ JULIAN ] ODELL | [ DAVE ] OLLASON | [ PHIL ] WOODLAND | [ STEVE ] YOUNG;( SENT-START ( DIAL <$digit> | (PHONE|CALL) $name) SENT-END )
P.20
Center for PersonKommunikation
Using the CPK NLP suite
• Implementing the grammar in the APS format using a texteditor
• Testing the grammar
• Simulating speech recognition and understanding
• Converting the APS into HTK standard lattice
P.21
Center for PersonKommunikation
The APS formalism
BNF• Rules consist of symbols,
e.g. BritishCity, DanishCity
• Basic operation:– string comparison
– Boolean (true or false)
APS• Rules consist of feature sets,
e.g. {cat=city,country=UK}, {cat=city,country=DK}
• Basic operation:– unification
– Null or a new feature set
P.22
Center for PersonKommunikation
Semantics in the APS
• Simple: Percolation of values to the top node
• Mapping, postprocessing (see paper)
STRUCTURE BUILDING RULE{cat=person, emailaddr=$A} [{cat=lastname, emailaddr=$A}].
LEXICAL RULE{lex=brondsted,cat=lastname, [email protected]}.
P.23
Center for PersonKommunikation
the msg_spec subnetwork
Examples: delete message number three forward previous message go to first
Semantics: a) an absolute number b) a relative number
P.24
Center for PersonKommunikation
the msg_spec subnetwork
LEXICAL RULES{lex=previous,cat=spec, relpos=-1}.{lex=current, cat=spec, relpos=0}.{lex=this, cat=spec, relpos=0}.{lex=next, cat=spec, relpos=1}.
{lex=first, cat=spec, abspos=0}.{lex=last, cat=spec, abspos=10000}.
{lex=message,cat=msg}.{lex=email,cat=msg}.<number>.
P.25
Center for PersonKommunikation
Testing/running msg_spec
apspars msg_spec.aps msg_spec.snt
apstslu msg_spec.aps 0
apstrec msg_spec.aps 0
apssgen msg_spec.aps
apsconv -f -h msg_spec.aps
testing on a test database
testing random generation
simulating recognition
simulating speech understanding
fullgram HTK standard lattice