Post on 10-Jun-2018
Multi-modal e-BusinessMulti-modal e-Business
Stéphane H. MaesStéphane H. Maessmaes@us.ibm.com
Human Language Technologies
Mobile Speech Solutions &
Conversational AdTech
OUTLINEOUTLINEMobile e-business and m-Commerce
WAP and WMLi-Mode
Voice e-Business: VoiceXMLA new Middleware for Mobile e-BusinessMulti-channel and Multi-modal applications
ScenariosPain PointsInhibitorsTranscoding
MVC architecture and use of the DOMKey principles: the DOM (and DOM remote control)Steps through a simple application scenarioMulti-modal browser ArchitectureProtocol discussionProgramming models:
tiersAuthoring methodologies
Standard landscapeConclusions: summary of Multi-modal landscape
OBJECTIVESOBJECTIVES
Understand the issues with Mobile e-BusinessUnderstand the directions of Mobile e-BusinessUnderstand the associated time-frame and infrastructure changes
Mobile e-Business and Mobile e-Business and M-CommerceM-Commerce
WAP & WML - The WAP & WML - The birth of M-Commercebirth of M-CommerceSee WAP Forum: http://www.wapforum.org
Web Server
Content
CGIScripts
etc.
WM
L D
ecks
with
WM
L-Sc
ript
WAP Gateway
WML Encoder
WMLScriptCompiler
Protocol Adapters
Client
WML
WML-Script
WTAI
Etc.
HTTPWSP/WTP
Web Server
ContentContent
CGIScripts
etc.
CGIScripts
etc.
WM
L D
ecks
with
WM
L-Sc
ript
WAP Gateway
WML Encoder
WMLScriptCompiler
Protocol Adapters
WAP Gateway
WML Encoder
WMLScriptCompiler
Protocol Adapters
WML Encoder
WMLScriptCompiler
Protocol Adapters
Client
WML
WML-Script
WTAI
Etc.
ClientClient
WML
WML-Script
WTAI
Etc.
WML
WML-Script
WTAI
Etc.
HTTPWSP/WTP
The WAP Architecture
Internet
Comparison between Internet and WAP Technologies
HTMLJavaScript
HTTP
TLS - SSL
TCP/IPUDP/IP
Wireless Application Protocol
Wireless ApplicationEnvironment (WAE)
Session Layer (WSP)
Security Layer (WTLS)
Transport Layer (WDP)
Other Services andApplications
Transaction Layer (WTP)
SMS USSD CSD IS-136 CDMA CDPD PDC-P Etc..Bearers:
HTMLJavaScript
HTTP
TLS - SSLTLS - SSL
TCP/IPUDP/IP
Wireless Application Protocol
Wireless ApplicationEnvironment (WAE)
Session Layer (WSP)
Security Layer (WTLS)
Transport Layer (WDP)
Other Services andApplications
Transaction Layer (WTP)
Wireless ApplicationEnvironment (WAE)
Session Layer (WSP)Session Layer (WSP)
Security Layer (WTLS)Security Layer (WTLS)
Transport Layer (WDP)
Other Services andApplications
Transaction Layer (WTP)Transaction Layer (WTP)
SMS USSD CSD IS-136 CDMA CDPD PDC-P Etc..Bearers:
SMS USSD CSD IS-136 CDMA CDPD PDC-P Etc..Bearers:
WAP specifies…
❚ Wireless Application Environment● WML Microbrowser● WMLScript Virtual Machine● WMLScript Standard Library● Wireless Telephony Application Interface● WAP Content Types
❚ Wireless Protocols● Wireless Session Protocol (WSP)● Wireless Transport Layer Security (WTLS)● Wireless Transaction Protocol (WTP)● Wireless Datagram Protocol (WDP)● Wireless network interface definitions
For a detailed discussion: seehttp://www1.wapforum.org/what/docs/WD99Technical990506.ppt
Other players: iMode Other players: iMode
iMode: http://www.nttdocomo.com/top.shtmlC-HTML
Voice e-BusinessVoice e-Business
Authoring of speech applications in C/C++ (SRAPI, SAPI etc...)Authoring of speech application in Java (JSAPI)Authoring of Speech application in state tables (DirectTalk - IVRs) and other proprietary IVR scriptsUse of prepackaged libraries:
Java beans (IBM DT Java)Speech objects / Dialog components (Nuance Speech Objects)
Conventional MethodologiesConventional Methodologies
Desktop - Enterprise - Telephony - Network - Embedded
Automatic Speech
Recognition(ASR)
Text toSpeech(TTS)
Names etc.Commands
Natural Language
Understanding(NLU)
Natural LanguageGeneration
(NLG)
DialogManagement
LanguageTranslation
InformationRetrieval/
Management
DictationE-mail
etc.
SpeakerIdentification &
Verification
Speakersimulation &voice fonts
Face & GestureAnimation
Audio-visualRecognition
Conversational Engines Core Components
Desktop - Enterprise - Telephony - Network - Embedded
Java Server Pages
Servlets
EnterpriseJava Beans
HTTPServer
Connectors Data
Markup + Java Java
Web Application Server
- Runs as extension of HTTP Server- Emphasizes separation of roles: - JSPs handle presentation logic - Servlets, EJBs handle business logic- Connectors for access to data & content- Load balancing, distributed processing- Cross-platform portability
ApacheVisual Browser - HTML - WML (thorugh gateway) - ...
DB2, SAP, CICS ...
IBM WebSphereBEA WeblogicSun NetDynamics...
Web Programming Application ModelWeb Programming Application Model
Web-based IVR - Voice Access to Web Web-based IVR - Voice Access to Web ApplicationsApplications
HTTPServer
DataWeb App
ServerVisual
Browser
VoiceBrowser
Web IVR- Common application and business logic- Separate presentation logic for visual & speech channels- Common development tools & skills
HTMLPages
VXMLPages
Voice Access- Voice & data channels over IP- IVR integration
Voice Browser- Manages audio, call control, recognition, text-to-speech, dialog with user
MenuFormBrowser
...
JavaSpeech API
HTTP / Distributed Conversational Protocols
SpeechServer
Java ServletEnterprise
Data
CGI ScriptWeb
Server
Speech Engine
JavaSpeech
TranscodingProxy
HTML Web Server
DominoServer Notes
DataCGI Program
Local Application
Local database
Local service
VoiceXML / HTTP
Speech Engine
Audio:
Client
Application Server
Engine Server
Voice Application Server
VoiceXML browsersVoiceXML browsers
VoiceXMLVoiceXML
IBM: SpeechMLMotorola: VoxMLAT&T / Lucent: PMLVoiceXML Forum: http:www.voicexml.orgSpecifications:
VoiceXML 1.0: http://www.w3.org/Submission/2000/04/W3C Voice activity: http://www.w3.org/Voice/
Speech Synthesis Mark-upMark-up for speech grammarsDialog Mark-up language: VoiceXML 2.0Reusable Dialog ComponentsNatural Language Semantics Multi-modal working group
New Middleware for Mobile New Middleware for Mobile e-businesse-business
MAINFRAME/ TEXT-BASED PARADIGM
PC / GUI PARADIGM
Infrastructure Change: Mainframe to PC
WEB / DECLARATIVE GUI PARADIGM
MOBILE INTERNET / CCI PARADIGM
Date Type Amount04/01 1539 $ 19.5004/02 ATM $ 200.0004/03 1548 $ 250.0004/03 1541 $ 12.5004/04 1533 $8500.0004/05 ATM $ 200.0004/10 ATM $ 200.0004/11 Deposit $ 250.0004/12 1540 $ 979.7704/13 1542 $ 98.50
SERVER
CLIENT
Network
Infrastructure Change: Network and Pervasive Clients
Conversational Computing
Multi-channelMulti-modal: Synchronized channelsDistributedConversational / Free Flow / Mixed
initiative
CLIENT
DefinitionsDefinitions
Channel: a particular browser, device, or a particular modality. Multi-channel applications: applications designed for ubiquitous access through different channels, one channel at a time. No particular attention is paid to synchronization or coordination across different channels. Multi-modal applications: multi-channel applications, where multiple channels are simultaneously available and synchronized.
Note that according to this definition, there are no fundamental differences between multiple devices (multi-device browsing) and multiple modalities.
Note: Multi-media players (e.g. Flash) can be considered as a different type of channel just as today GUI is actually multi-modal:
multi-modality is changing over time. For example, large screen graphical output, keyboard and mouse input constitute the GUI user interface while being clearly multi-modal. As platforms and tools are designed to seamless design applications that integrate these different modalities and applications become widely available, we do not distinguish them any more as multi-modal. They rather become a well
understood and well accepted composite modality.
Multi-channel scenario: travel Multi-channel scenario: travel reservations todayreservations today
Multiple access devicesOne interaction mode per device
PC
Standardized rich visual interfaceNot suitable for mobile use
I need a direct flight from New York to San Francisco after
7:30pm today
There are five direct flights from New York's LaGuardia airport to San Francisco after 7:30pm today: Delta flight nnn...
Book me on the United flight
Access from any telephoneOutput is inherently sequential
Voice
Flights Hotels Cars Packages Cruises Maps EXPRESS SEARCHDeparting from: Going to:[__________________] [__________________]
When are you leaving? When are you returning?[Dec] [31] [Noon] [Jan] [ 1] [Noon]
Tip: We have many more flight, hotel, and car options.
WHAT'S NEWSki Travel: Choose from more than 80 ski destinationsCruise Travel: Take a virtual tour of select cruise ships
WAP or i-modeFROM LGATO SFONON-STOPFLIGHT DEP ARR DLnnn 7:30p 10:55pAAnnn 7:55p 10:55pUAnnn 8:30p 11:55pTOnnn 8:45p 0:10pMWnnn 9:15p 0:45p
FROM LGATO SFONON-STOPFLIGHT DEP ARR DLnnn 7:30p 10:55pAAnnn 7:55p 10:55pUAnnn 8:30p 11:55pTOnnn 8:45p 0:10pMWnnn 9:15p 0:45p
FROM LGATO SFONON-STOPFLIGHT DEP ARR DLnnn 7:30p 10:55pAAnnn 7:55p 10:55pUAnnn 8:30p 11:55pTOnnn 8:45p 0:10pMWnnn 9:15p 0:45p
From: LGA__To: ______Date: ______
From: LGA__To: SFO__Date: ______
From: LGA__To: SFO__Date: 00/12/11
Mobile and becoming ubiquitousHard to enter data
Pain points in multi-channel e-businessPain points in multi-channel e-business
Pain pointshard to enter and access data using small devices
tiny keypads and screens
one interaction mode does not suit all circumstanceseach mode has its pros and cons
all-in-one devices are no panaceabulky and expensive
Most mobile device usage today is not for e-business applications
No immediate relief is in sight:Devices are getting smaller, not largerDevices and applications are becoming more complexAdding color, animation, camera, etc. does not simplify or contribute to e-business CRMs / IVRs are mostly not yet web-centric
Different applications maintained by different IT organizations!
TranscodingTranscoding
XSL stylesheets
Transcoder
Developer pain pointsDeveloper pain points
Multiple Authoring: Manual or automated through transcodingTranscodingChallengess with multiple authoring:
M pages for N channels requires M x N stylesheets or final form pagesTo be developedTo be maintained
How to support new modalities?Each supported device characteristics must be known to the application provider
Authoring of the synchronization for multi-modal applications: Merged languages or synchronization tags
How to support different levels of synchronization granularity?Combinatorial problem
Different channel to synchronizeDifferent synchronization levels
Mobile e-Business requires a new programming models
[ ][ ] [**][**] [ ][ ][**][**] [**][**] [**][**][ ][ ] [**][**] [**][**][**][**] [**][**] [**][**][_][_][_] [_][_][_] [_][_][_][_][_][_] [_][_][_] [_][_][_][_][_][_] [_][_][_] [_][_][_][_][_][_] [_][_][_] [_][_][_][_][_][_] [_][_][_] [_][_][_][_][_][_] [_][_][_] [_][_][_][_][_][_] [_][_][_] [_][_][_][_][_][_] [_][_][_] [_][_][_][_][_][_] [_][_][_] [_][_][_]
Select seat 3A or
Multi-modal scenario: travel Multi-modal scenario: travel reservations tomorrowreservations tomorrow
or
Multiple interaction modes per deviceInteraction mode depends on task and situation and can change at any timeMulti-device browsingEarlier versions: transactional persistence and notificationsLater versions: Free flow - Conversational
I need a direct flight from New York to San Francisco
after 7:30pm today
Book me on the United
flight
Example: mobile GUI + voice
We have many more flight, hotel, and car
options.
Additional examples:
display seat selection chart (not simply "window or aisle")
use voice or keys to enter PIN code and perorms speaker verification
use audio or voice for notifications
information can be saved for later use
FROM LGATO SFONON-STOPFLIGHT DEP ARR DLnnn 7:30p 10:55pAAnnn 7:55p 10:55pUAnnn 8:30p 11:55pTOnnn 8:45p 0:10pMWnnn 9:15p 0:45p
Would you like a window or an aisle seat?
If you know your desired itinerary you can use a shortcut...
Express reservations.From which
airport do you wish to depart?
Multi-modal e-business value Multi-modal e-business value propositionproposition
Multi-modal e-businessvalue proposition
easily enter and access data using small devicesby combining multiple input & output modes
choose the interaction mode that suits the task and circumstancesinput: key, touch, stylus, voice...output: display, tactile, audio...
use several devices in combinationby exploiting the resources of multiple devices
Conversational extensionInteracts with and across application as in an every day dialog: free form, mixed initiative and context management
Context:Devices are getting smaller, not largerDevices and applications are becoming more complexAdding color, animation, camera, etc. does not simplify e-business
User can access a given enterprise via
multiple channels
Common DataCommon Business Logic
Common Presentation Design
MULTIMODAL
Multi-modal Space DescriptionMulti-modal Space Description
User perspective: Integration of multiple modes
Developer perspective:Integration of e-business channels
MULTICHANNEL
MULTIPLEAUTHORING
User can access a given application via
multiple channels
Developer uses several modes to implement a
given transaction
User can choose what mode to use for each of
the interactions that make up a transaction
Common Interaction Design SINGLE
AUTHORING
Nature of Nature of Nature of Nature of the the the the
InteractionInteractionInteractionInteraction
EnvironmentalEnvironmentalEnvironmentalEnvironmental
ConsiderationsConsiderationsConsiderationsConsiderations
Free Flow
:
Conve
rsatio
nal
Customer group Customer need Status
In-vehicle information services
Create portal for car buyers that can produce an annuity revenue stream
Early adopters are purchasing "hard-coded" solutions (not based on middleware)
Financial servicesTravel services
Improve mobile access for high-value transactions Requires 3G wireless networks
Mobile information services Build mindshare and customer base for future ubiquitous services Requires 3G wireless networks
Wireless carriers Grow user base and minutes of airtime by improving mobile ease of use Requires 3G wireless networks
PDA buyers Enable use with one hand, hands-free, or eyes-free
Limited by impact on device cost, size and battery life
Logistics, healthcare, insurance, manufacturing
Improve productivity of mobile employees with hands-busy or eyes-busy transaction tasks
Limited by time and cost to develop a solution
Demand DriversDemand Drivers
Numbers of quotes:
We'll figure out a way to integrate speech recognition with [cell phone] displays ... the answer lies in using the modes of interaction together"
Thomas S. Tullis, VP in charge of human interface design, Fidelity, 11/00 (New York Times)
Mobile devices [have] the possibility of having visual as well as aural information. So you might see something that combines the two"
Chan Suh, CEO, Agency.com, 11/00 (Internet World)
Inhibitors
Presentation gateways-Synchronized voice browser-Mode synchronization
Device capabilities-Audio subsystem and SVD / DSR (Distributed Speech
Recognition)-Synchronized GUI browser
Application development-Developer tools-Standards
Distributed architectureenables 'thin' clientrequires network connection
.. or ..
Device capabilties-Audio subsystem -Multimodal browser-CPU/RAM
Application development
-Developer tools-Standards
Client-centric architectureenables disconnected userequires 'fat' client
Carrier infrastructure-Simultaneous voice &
data (SVD) connection or support for DSR
Model View Controller Principle Model View Controller Principle (MVC)(MVC)
Model
View1
View 2
Controller1
Controller2
Controller3
Dialog State
repository
GUI Renderer
SpeechRenderer
GUIInteraction
Multi-modalInteraction
SpeechInteraction
Modality-independentrepresentation
Invertible mappings from model to views
User must be able to switch channel at any unpredictable moment while interacting with the application and seamlessly continue to interact.
Each piece is distributable
GUI Browser
DOM
Wrapper
Voice Browser
DOM
Wrapper
MM Shell
Model View Controller Architecture for Multi-modal Model View Controller Architecture for Multi-modal BrowserBrowser
Steps through a simple Steps through a simple application scenarioapplication scenario
What is your name?
Please select your drink
GUI Browser Speech Browser
What is your name?
Please select your drink
User enters his nameDependingon the granularity (event level or slot level), each character is transmitted to MM shell or buffered until focuus leavs the slot (DOM events)Shell is informed of the new data model valueViews are updated: corresponding VoiceXML guard variable is updated: DOM update (set variable)Speech browser pompts: Please select your drink
Other scenarios can be considered: e.g. name provided by voice.
CommunicationManager
VoiceTransportInterface
Dat
aTr
ansp
ort
Inte
rface
Sync
hron
izat
ion
Inte
rface
GUI Browser DOM Wrapper
Voice Browser
DOM
Wrapper
MM Shell
ConversationalEngines
DSR encoder
GUI drivers
GUI I/O
Audio drivers
Audio I/O
ContentServer
HTTP
SynchronizationInterface
NetworkEdge Server
Audio Codec (s)
DSR decoder
Net
wor
k Tr
ansp
ort L
ayer
Net
wor
k Tr
ansp
ort L
ayer
Gat
eway
and
rout
er w
ith V
oice
tran
spor
t and
Sy
nchr
oniz
atio
n Su
ppor
t
Multi-modal Browser Architecture: Thin Client Multi-modal Browser Architecture: Thin Client
NetworkEdge Server
GUI Browser
DOM
Wrapper
Voice Browser
DOM
Wrapper
MM Shell
Engines
Audio drivers
Audio I/O
GUI drivers
GUI I/O
ContentServer
HTTP
Local Engines
CommunicationManager
VoiceTransportInterface
Dat
aTr
ansp
ort
Inte
rface
Sync
hron
izat
ion
Inte
rface
Audio Codec (s)
DSR decoder
Net
wor
k Tr
ansp
ort L
ayer
Net
wor
k Tr
ansp
ort L
ayer
Gat
eway
and
rout
er w
ith V
oice
tran
spor
t and
Sy
nchr
oniz
atio
n Su
ppor
t
DSR encoder
A Fat Client Variation of the Architecture A Fat Client Variation of the Architecture
Network(s)Edge Server
GUI Browser
DOM
Wrapper
Voice Browser
DOM
Wrapper
MM Shell
EnginesAudio drivers
Audio I/O
GUI drivers
GUI I/O
ContentServerHTTP
Local Engines
CommunicationManager
VoiceTransportInterface
Dat
aTr
ansp
ort
Inte
rface
Sync
hron
izat
ion
Inte
rface
DSR decoder
Voice Browser
WAPWMLGUI
Browser
PSTN
HTTP
Multi-channel /Multi-modal
Configuration
Net
wor
k Tr
ansp
ort L
ayer
Net
wor
k Tr
ansp
ort L
ayer
Gat
eway
and
rout
er w
ith V
oice
tran
spor
t and
Sy
nchr
oniz
atio
n Su
ppor
t
Audio Codec (s)
DSR encoder
Multi-channel and Multi-modal server Multi-channel and Multi-modal server
CommunicationManager
VoiceTransportInterface
Dat
aTr
ansp
ort
Inte
rface
Sync
hron
izat
ion
Inte
rface
GUI Browser DOM Wrapper
Voice Browser
DOM
Wrapper
MM Shell
ConversationalEngines
DSR encoder
GUI drivers
GUI I/O
Audio drivers
Audio I/O
ContentServer
HTTP
SynchronizationInterface
NetworkEdge Server
Audio Codec (s)
DSR decoder
1
2 2
3
4
5
6
7
7
8
Net
wor
k Tr
ansp
ort L
ayer
Net
wor
k Tr
ansp
ort L
ayer
Gat
eway
and
rout
er w
ith V
oice
tran
spor
t and
Syn
chro
niza
tion
Supp
ort
9
Components, protocols and interfaces to specify and / or standardize:1.Voice transport protocol, in particular RT streaming DSR protocols. This includes: Interface between the DSR codec and the communication manager; Protocol between network edge server and DSR decoder; Interface between DSR decoder and conversational engines2. Synchronization protocols between multi-modal shell and DOM wrapper filters (e.g. SOAP, UDDI)3. Interface and protocol for transmission of the synchronization protocols, conversational distributed protocols, data transport protocols and remote conversational engine control protocols between the communication manager and the network edge server.4. Conversational engine remote control protocols5. VoiceXML DOM. Events and interfaces must be specified.6. WML DOM. This exists as XML DOM, but it has never been officially adopted for WML.7. DOM wrappers8. Authoring frameworks, programming models and language specifications: multi-channel and multi-modal9. Multi-modal shell
Standard, Protocols and Interfaces to specifyStandard, Protocols and Interfaces to specify
Synchronization ProtocolsSynchronization Protocols
DOM filtered events:DOM Level 2 UI events (and higher): XHTML generalized UI events, VoiceXML events, Etc…HTTP (or other protocols) requests:URI requests
DOM commands:Page pushOutput eventsSet eventsGet/set variablesDOM tree manipulation
Blocking messagesConfirmation messagesEvents are systematically time stamped. A clock synchronization protocol - see Network Time Protocol (NTP ?)
SOAP- XML RPCIndependent of Transport layer
SOAP v1.1 Spec Defines ..SOAP v1.1 Spec Defines ..
A transport-independent messaging model
Transport bindings for HTTP
An encoding model for a type system
RPC over HTTP
+ Today: SOAP Attachments (MIME type for binaries)
See http://www.w3.org/2000/xp/
Service-Oriented ArchitectureService-Oriented Architecture
Service Instance
Formats & Protocol
Disc
over
y
Service Types
Service QoS
Service FlowsWSFL
WSDL
WSDL
WSEL
SOAP
UDDI
UDDIADS
XML ProtocolsSOAP
GUI Browser
DOM
Wrapper
Voice Browser
DOM
Wrapper
MM Shell
The DOM MVC Multi-modal Browser architecture is an example of web intermediary / service composition
WSFL
WSDL
Service Broker
Service Requester
Service Provider
Other / New ProtocolsOther / New Protocols
Synchronization granularity negotiationDiscovery (UDDI / ADS)Registration of the viewsDescription of the view characteristics / capabilities (WSDL & CC/PP)Spontaneous connection of new views DisconnectHandshake to confirm active connection
Engine remote control protocol:RSTP extensions?etc...
Voice Transport ProtocolsVoice Transport Protocols
Voice and data.Voice over IP: RTP DSR: RT streaming version?
MVC supports all the MVC supports all the authoring authoring methodologiesmethodologies
Multiple authoringMultiple authoring
explicit synchronization tags: indicate when a channel must be synchronized to another and how. This includes co-visit tags that indicate:
to a channel what the other channel should load and present (hence the name co-visit) - Co-browser approaches
Authoring method primarily designed for co-browsers Indicates that a notification must be given to the MM shell to update the other views
HTML VoiceXML
Global Cafe Block Global Cafe Block
Name block Name block
List Sel block List sel block
Submit "Submit"
Multiple authoringMultiple authoringnaming conventions:
In each channel, fields or items that should be synchronized have a same name or portion of name. Modality specific blocks have no corresponding blockMM shell stores a pseudo DOM tree of nodes
HTML VoiceXML
Global Cafe Block Global Cafe Block
Name block Name block
List Sel block List sel block
Submit "Submit"
<name = node 1/>
<name = node 1/>
<name = node 2/>
<name = node 3/>
<name = node 4/>
<name = node 2/>
<name = node 3/>
<name = node 4/>
Multiple authoringMultiple authoringmerged pages:
An application is described by pages that consists of merge portions associated to each target channel; e.g. a series if snippets of HTML and VoiceXML. Also encompasses annotations in one channel page to support other channels or modalities, e.g. to indicate the role of an element or provide associated data file information like the grammar to associated to a form field. (includes traditional command and control)MM shell can produce naming convention or synchronization tags
Global Cafe Block
Global Cafe Block
Name block
Name block
List Sel block
List sel block
Submit
"Submit"
Single authoringSingle authoringOne common modality independent representation of the application (interaction and data model) + specializationTransformation to different views with for example naming conventionMVC
Global Cafe Block
Global Cafe Block
Name block
Name block
List Sel block
List sel block
Submit
"Submit"
Message to user
Message to user& Free form input$name
Message to user
Selection from list$list
Action submit
Data model to transmit
<name = node 1/>
<name = node 2/>
<name = node 3/>
<name = node 4/>
<name = node 5/>
<name = node 6/>
Channel-independent description of the application
Modality/channel Specific Specialization
Modality Specific Presentation
Data
Business Logic
HTML VoiceXML
Global Cafe Block Global Cafe Block
Name block Name block
List Sel block List sel block
Submit "Submit"
<name = node 1/> <name = node 1/>
<name = node 2/>
<name = node 4/>
<name = node 6/>
<name = node 2/>
<name = node 4/>
<name = node 6/>
<name = node 3/> <name = node 3/>
<name = node 5/> <name = node 5/>
Channel-independent description of the application
Modality/channel Specific Specialization
Modality Specific Presentation
Data
Business Logic
Tier 1 & Tier 0
Tier 2
Tier 3
Conventional Factorization
in Tiers
Multi-channel / Multi-modalFactorization
Multi-modalMVC mapping
Views (Acted upon by
controllers)
Model: Channel independent
Representation
Business and Backend
Multi-modalBrowser
MVC Multi-modal
Browser(Local or
Distributed)
Web Application
Server, Web Server and
backend
Tiers positioning with respect to the MVC paradigmTiers positioning with respect to the MVC paradigm
Tiers IllustrationTiers Illustration
What is your name?
Please select your drink
GUI Browser Speech Browser
What is your name?
Please select your drink
Tier 0/1 - Presentation layer: Dialog interaction with the user: What is your name, please select your drink
Tier 2 - Business login:Flow of application: collect user name, provide list of drinks, collect selection, transfer selection to bar for preparation, compute bill, ask the user if he wants to pay or keep a tab; add to restaurant books
Tier 3 - Data base:List of customers, List of drinks, "business ledger/books" etc...
Single Authoring Programming ModelSingle Authoring Programming Model
Data
Business Logic
Data Models
Interaction
Modality Specific Presentation
Specialization
Programming Model for:Multi-channel ApplicationMulti-modal ApplicationMulti-device Applications
Programming Model:1. Programming by interaction (channel/modality independent):
Interaction and data Model layer2. Specialize for class of channel3. Specialize for specific channel4. Support different navigation flows
Single Authoring is the key programming model for Mobile e-Business
Web Application
Server
iML________
Multi-channel Architecture - Single Multi-channel Architecture - Single AuthoringAuthoring
Transcoder
Network Edge
Server:Gateways
Voice Application
Server
HTML
WML
VoiceXML
HTMLBrowser
WMLBrowser
Voice
Application
Backend
XML________
XSLTs
ClientCharacteristics
Data Files
Support Servlets, JSPs, beans etc...
HTTP
WSP
ETSI DSR Application and Protocols WG
W3CDevice-Independent
Authoring Activity
W3CMulti-modal WG
Multi-modalSub-group X
WAP Forum
Multi-modal Web Workshop
Other W3CActivities
Device Indep AuthWeb Workshop
Other W3CActivities
Call forreview
Call forindustry proposal
Multi-modalRequirements& case scenarios
- Authoring (single and multiple authoring)- Multi-modal architecture and interfaces
Single authoring
CC/PP & Other specs
VoiceXMLForum
W3C Voice Activity
~7/01
2/26/01
11/00
10/00
9/00
12/00
1/01
1Q00
Multi-Modal Standard Multi-Modal Standard LandscapeLandscape WAP Forum
3GPP
Other W3CActivities
?
Multi-modal Protocols
specification and recommended
architecture
MC Single authoringmethodologies requirements
andrecommendations
Multi-modal authoring specifications,
Architecture& Interfaces
2Q00 3Q00 4Q00 1Q01 2Q01 3Q01 4Q014Q00 1Q022Q00 3Q00 4Q00 1Q01 2Q01 3Q01 4Q014Q00 1Q02
(Thin client) Multi-modal browser architecture recommendation
Protocols specs+ other TBD
EnablementOpportunity
Summary of multimodal e-business landscape
Customer groupsIn-vehicle information servicesFinancial and travel servicesMobile information servicesWireless carriersPDA buyersLogistics, healthcare, insurance, manufacturing
Pain pointshard to enter and access data using small devicesone interaction mode does not suit all circumstancesall-in-one devices are no panacea
3GStandards
Device capabilities
Multimodal 'thin client'2001 - Japan2002 - Europe2003 - Americas
Multimodal 'fat client'2000 - Cars2001 - PDAs2008? - Feature phones
Key inhibitors
Trends show that all inhibitors will dissapear in the next 2 to 3 years