MultiLingual Technologies Inc. (MLT) - Multilingual Translation & Localization Service Provider
Mobile Multilingual Maintenance Man
description
Transcript of Mobile Multilingual Maintenance Man
1
MobileMultilingualMaintenanceMan
MobileMultilingualMaintenanceMan
2
MobileMultilingualMaintenanceMan
4M Assists a Service Person
3
MobileMultilingualMaintenanceMan
4M supports Problem Solving
Problem situation Possible solution Problem solved
Consult Do it ReportPlan
Service Center
Support for reporting
Speech recognition Dialogue anagement
Message understanding Situational recognition
Search for information Presentation of instructions
Accumulation of knowledge
Creation of basic knowledge
Multilinguality
4
MobileMultilingualMaintenanceMan
Information Retrieval
INPUT ANALYZER
Model-based Tool
4M Server Interface
Desktop clients(text, speech, graphics)
4M Clients
Mobile clients (menus, speech)
FACTS REASONER
ONTOLOGY SERVICES
Ontologyrepository
OUTPUT GENERATOR
Human Assistance
Case-based Tool
DIALOGUE MANAGER
Speech recognition on either client or
server side
Asynchronic query to tools
Speech synthesis on either client or
server side
1
12
2
113
4 10
9
58
67
Selected tool response
Conceptual system
response
Verbalized system
response
Conceptualized user input
4M Architecture
5
MobileMultilingualMaintenanceMan
Information RetrievalModel-based
DATA SOURCES
Text/XML files
4Mdata
External systems
Externalsystemsand data
4M System
FACT FINDING & REASONING TOOLS ONTOLOGY SERVICES
Ontologyrepository
OUTPUT GENERATOR
RDF/XML Generator
Report Generator
Human AssistanceCase-based…
DIALOGUE MANAGER
Input Module
Dialogue Context
Response Evaluator
Discourse Memory
Reasoner Interface
Dialogue Planner
Text (documentation, emails etc.)
INFORMATION PROCESSING
Segmentation Annotation
ONTOLOGY PROCESSING
Designer Compiler
INPUT ANALYZER
Parser
Concept Analysis
DIALOGUE RDF RULES
TriplEd
4M Knowledge Accumulation
6
MobileMultilingualMaintenanceMan
CoGKS assists a Service Community with 4M
7
MobileMultilingualMaintenanceMan
CoGKS Clients
CoGKS Server
Desktop clients(text, speech, graphics)
Mobile clients (text, speech)
Access manager
Room manager
Room (Job)
ChatSummary Attacheddata
4M System
Summarization of cases
Active or passive assistance mode
Import supporting
material
Establish rooms
Query and copy rooms
Consult participants and solve problems in cooperation
Invite participants
Variety of reasoning: Model, Case, IR,
Human, …Cumulative
knowledge baseConsult 4M
Speech, text
The Cognitive Guidance and Knowledge System (CoGKS)
8
MobileMultilingualMaintenanceMan
4M Ontology: a multilingual ontology matrix
company terms
domain termsdomain lexicon
generic terms
domain ontology
genericontology
generic lexicon
company termscompany lexicon company ontology
company instance bases
9
MobileMultilingualMaintenanceMan
4M Dialogue Manager
Constructive Dialogue Management (CDM)Input Analysis
and Natural Language Generation in 4M Multilingual Agenda Markup Language.
Agent Based Dialogue Model Specifying Query Selection Weighted Rules for Output Responses Dialogue Planning Utilizing Dialogue Objects
Reasoner Data Interfaces in Uniform RDF Blackboard Data Architecture. Reasoner Abstraction Broadcast Messages
10
MobileMultilingualMaintenanceMan
Constructive Dialogue Management theory (CDM)
(Jokinen, 1996)
Human and Machine Participate in Ideal Cooperation (Allwood, 1976)
strive to achieve the same purposes
cognitively and ethically consider each other in trying to achieve these purposes
trust each other to act according to the above principles
4M Dialogue Manager
11
MobileMultilingualMaintenanceMan
Agent Based Dialogue Model Specifying Query Selection Weighted Rules for Output Responses
DM
Question Pool Sorted by Probability:
0.21: [pwr1] [may_be] [off] 0.08: [inet1] [may_be] [down] 0.04: [hub1] [may_be] [broken]
“Is the power switch off?”
Q1
Q2
Q3
4M Dialogue Manager
12
MobileMultilingualMaintenanceMan
DM
R1
R2
R3
Collected facts:
[network] [is_not] [accessible] [eth_cable] [is_not] [loose] [pwr1] [may_be] [off] [hub1] [may_be] [broken]
Model Based Reasoning Case Based Reasoning Information Retrieval
4M Dialogue Manager
Reasoner Data Interfaces in Uniform RDF Blackboard Data Architecture – All current data is in messages. Reasoner Abstraction – New reasoners can be added easily. Broadcast Messages – Messages are sent to all reasoners at once.
13
MobileMultilingualMaintenanceMan
Input Analysis and Natural Language Generation in 4M Multilingual Agenda Markup Language
DM
“Cable is not loose.”
“Is the power switch turned off?”
IAText > RDF
NLGRDF > Text
4M Dialogue Manager
14
MobileMultilingualMaintenanceMan
Code switching with a multilingual grammar
4m5.defbilingual grammar
4m5en.def
English grammar4m5fi.def
Finnish grammar
4m5en_lex.defEnglish terms from ontology
4m5fi_lex.defFinnish terms from ontology
fi4.hFinnish
morphology
15
MobileMultilingualMaintenanceMan
cparse parser
cparse generator
+
multilingualgrammars
+
Jena/JavaAPI
+
<rdf:RDF> <dm:Agenda> <dm:next> <dm:Greet/> <dm:next> </dm:Agenda></rdf:RDF>
Hello
Commutative graph rewriting
16
MobileMultilingualMaintenanceMan
Concept Analyser
User said:“Vertti cannot access his calendar.”
This conversation is about:1. Calendars2. Mobile phones3. Humans4. Poodles5. Computers
Connects dialogue references with the things they refer to:
Keeps track of what the current conversation is about:
Mr. Vertti Hiiri
Vertti'scalendar program
Ontology
17
MobileMultilingualMaintenanceMan
Reasoners
dialogue-manager
baseontology
modelbased
reasoner
diagnosticsplanning
common query-
language
extendable set of
reasoners
extendable data
schema
industry company factory/office project
common protocol
planningconceptanalysis
life-cycle supporting
tasks
case based
reasoner
extendedontology
instancedatabase
casebase
ontology-reasoner
shared database platform
Reasoning methods and techiques
18
MobileMultilingualMaintenanceMan
Ontology based diagnostics
Loads in the conceptual ontology and the instance base
describing the system being diagnosed. Used domain independent search methods for localizing
the faults in the system. Uses the incremental set of observations reported by
users of the functionality of the system.
21
MobileMultilingualMaintenanceMan
Stepwise fault recognition
Observation O1: Mary can't access team calendar.
O2: Jack can't access project directory.
Can Jack access team calendar?
O3: Yes Fault candidates (O1 and
O2 but not O3 )
24
MobileMultilingualMaintenanceMan
Information Retrieval
... User: Internet on rikkiSystem: Toimiiko hiiri?User: En pääse nettiinSystem: Pääseekö hiiri nettiin?User: ......
Identify partslike subject, verb and
object in dialogue turns.Identify known concepts: solve
ambiguities and differencesin terminology, cross the
language barrier
...U: m4:Internet, m4:BrokenS: m4:MouseU: m4:Access, m4:NetS: m4:Mouse, ......
List known concepts and theirrelations to each other and toterms in each language
Build a query out of theconcepts in the dialogue.Retrieve best matching
documents
Sometimes you may encounterproblems with Internet access, or with input devices like keyboards and mice – or with just about anything. By simply following these trouble-shooting instructions,
Mark occurrences of known concepts with
BRIEFS ontology matching
Sometimes you may encounter <cid = "m4:Problem"> problems</> with <c id ="m4:Internet"> Internet </> <cid = "m4:Access"> access </>,or with <c id = "m4:Input">
Unannotated dialogue
Unannotated documents Document analysis
Ontology
Input analysis Annotated dialogue
Annotated documents
Information retrieval
25
MobileMultilingualMaintenanceMan
Case Based Reasoning Tool
Input: a problem description as a set of RDF triplets
Target state
Observations
Case base: example problems with their solutions as xml documents
The problem is matched against the cases in the case base
The cases with similarity measure over a similarity threshold are retrieved, and their solutions are returned in similarity order
If none of the library cases is similar enough compared to the current case, the solutions of the most similar cases are used in creating further questions to the user
The similarity calculation between two cases uses ontology to determine semantic similarity
26
MobileMultilingualMaintenanceMan
Human Assistance Tool
The Human Assistance Tool finds experts, administrators, and contact information according to ontology and instance base
Explicit user queries and queries initiated by Dialogue Manager: "Who is the expert of Windows", "Who is the administrator of server S1"
Passive queries representing the cumulative topic of conversation (the concepts found in the discussion) used for team building
locate all the persons in the instance base with some kind of knowledge in the conversation topic, both experts and administrators are considered
search for the people who are closest matches to the minimum spanning tree representing the topic
29
MobileMultilingualMaintenanceMan
4M: Annotating documents to improve IR
Briefs
RoolToolOntology
Document
Annotateddocument
CPSL
rules