Multimodal Applications for Mobile Multimedia Broadcasting

Multimodal Applications for Mobile Multimedia

BroadcastingJ.-M. Bouffard, F. Lefebvre, and B.-H. Lee

Presented at the 7th IASTED International Conferences on Wireless and Optical Communications (WOC 2007)

WNET Session 1 – Wireless TechnologiesThursday, Mai 31th 2007

A collaborative research with

2

Agenda

• Introduction• Current Context for Multimodal

Interaction• Issues and Requirements• DMB Voice Applications System

Overview• Open Implementation• Conclusion

3

Introduction

• MMB (Mobile Multimedia Broadcasting) networks such as DAB/DMB are generating a lot of interest

• These networks provide– High capacity down links– Capabilities to transport

digital data

4

Introduction

• Current MMB (Mobile Multimedia Broadcasting) services– User interaction through typical screen and

keypad interfaces– This interaction paradigm is not adapted to

“on the move” use cases– Visually driven applications can even raise

safety issues

5

Introduction

• Goals– Improve the system to make it more user

friendly and more versatile– Add new interfacing and rendering

capabilities• Speech recognition and rendering• Data interfaces extensibility such as time,

location or other types of information• Interactivity using any available return channel

– Create a multimodal browsing experience

6

Current Context for Multimodal Interaction

• Web based standard currently defined for DMB– Broadcast Website (ETSI TS 101-498)

• Visual applicationsonly

• Not extensible• Not well suited for

mobile users

7


• New problems arise when multimedia services are executed on mobile devices– Concurrent user activities

• Driving a car• Crossing the street• Walking in a crowded environment

– Safety issues are raised when user's attention must be oriented to other activities

– Low power and processing capabilities

8


• Technology is not ready for prime time– Speech recognition may be unreliable– Speech rendering can be boring and lengthy– One interaction mode may not suit all

circumstances

• The algorithms are steadily being improved

9

Issues and Requirements

• Key features– Mobility– Backward compatibility– Safety

• Service requirements– Local interactivity– Remote interactivity– Location based services– Optimization for different types of devices

10

Issues and Requirements

• Service requirements (continued)– Automatic switching to a safe mode of

operation– Content hierarchy

11

DMB Voice Applications System Overview

• Built from standard components– W3C / XML

• Voice Extensible Markup Language (VoiceXML)• XML Form of the Speech Recognition Grammar

Specification (SRGS)• BWS (HTML)

– Newly defined XML elements (<isync> <esync> <metasync>)

• Based on XHTML + Voice Profile (X+V)

12


13


• Four main modules– Visual Browser– Voice Browser– Synchronization Manager

• manages interaction and synchronization between the other modules

– Platform Components• optional extensions

14


• Visual Browser– BWS files compliant to ETSI TS 101-498 for

backward compatibility

• Voice Browser– VXML files

• language that managesvocal interaction withthe system

• profiling to removeunnecessary elements:telephony, object, ...

15


• Synchronization Manager– Manages the execution modes

• Voice mode <> Visual mode <> Dual mode• User or application controlled mode switching

– 2 types of sync elements• Internal sync (<isync>)• External sync (<esync>)

– Action occurring in theBWS document must betriggered by an externalsync document

16


• Platform Components– DAB interface

• Receiver control / dgi– /dgi-bin/select_service?service_id="service_id”

• Receiver control / XML control elements– <va:tuning> <va:watch> <va:reservation>

– Location information• New elements

– <va:location>– <va:registerlocation>

17


• Document structure– “.esync” files

• Sync elements

– “.html” files• BWS elements

– “.vxml” files• Voice elements

18


19

Open Implementation

• Software components used– OpenVXI

• Open source VoiceXMLinterpreter

– IExplorer component• HTML interpreter

– Third party ASR and TTS solutions• IBM Viavoice• AT&T Natural Voices• Other SAPI compatible engines

20

Open Implementation

• Test implementation– Hardware:

• Korean T-DMB receiver• Personal computer

– VA applications cannow be developedand tested on afunctional platform

– Next: mobile version

21

Conclusion

• Next steps:– This work was conducted as part of the

“Voice Applications” Task Force under the WorldDMB forum

– The specification is now waiting approval from the WorldDMB Technical Committee

– Next step will be the ETSI standardization process

• This concept could be applied to other broadcast systems

22

Thank you

Questions?

For more info:[email protected]

http://www.crc.ca/mmb

Multimodal Applications for Mobile Multimedia Broadcasting

Technology

Transcript of Multimodal Applications for Mobile Multimedia Broadcasting