A Multimodal Approach To Accessible Web Content On Smartphones
-
Upload
icchp2012 -
Category
Technology
-
view
429 -
download
0
description
Transcript of A Multimodal Approach To Accessible Web Content On Smartphones
![Page 1: A Multimodal Approach To Accessible Web Content On Smartphones](https://reader030.fdocuments.in/reader030/viewer/2022020207/5562e25dd8b42aac778b5603/html5/thumbnails/1.jpg)
A Multimodal Approach To Accessible WebContent On Smartphones
Lars Emil KnudsenHarald Holone
Østfold University College
11.07.2012
1 / 16
![Page 2: A Multimodal Approach To Accessible Web Content On Smartphones](https://reader030.fdocuments.in/reader030/viewer/2022020207/5562e25dd8b42aac778b5603/html5/thumbnails/2.jpg)
Introduction
I Master student at Østfold University College, NorwayI Applied Computer Science
MotivationI ProgrammingI Rewarding to contributeI Research valueI Exciting combination of multimodality and smartphones
2 / 16
![Page 3: A Multimodal Approach To Accessible Web Content On Smartphones](https://reader030.fdocuments.in/reader030/viewer/2022020207/5562e25dd8b42aac778b5603/html5/thumbnails/3.jpg)
Introduction
I I will present our current work in the area of multimodalinterfaces on smartphones.
1. Our implementation of W3C’s multimodal interactionframework on smartphones running the Android OS.
2. The results from user tests and interviews with blind andvisually impaired users
3 / 16
![Page 4: A Multimodal Approach To Accessible Web Content On Smartphones](https://reader030.fdocuments.in/reader030/viewer/2022020207/5562e25dd8b42aac778b5603/html5/thumbnails/4.jpg)
Background
I SMUDI-projectI Norwegian speech recognitionI Multimodal interfaceI Achieve universal designI Run by MediaLT, a norwegian research companyI This project has been created in relation to the SMUDI project
I Rapid development in the mobile marketI Smartphones with new capabilitiesI New opportunities of interface design, with multimodal
interaction being one of them
4 / 16
![Page 5: A Multimodal Approach To Accessible Web Content On Smartphones](https://reader030.fdocuments.in/reader030/viewer/2022020207/5562e25dd8b42aac778b5603/html5/thumbnails/5.jpg)
Designing Robust Multimodal Systems for UniversalAccess, Oviatt 2001
I “Given the right context, temporal disability applies toeveryone.”
5 / 16
![Page 6: A Multimodal Approach To Accessible Web Content On Smartphones](https://reader030.fdocuments.in/reader030/viewer/2022020207/5562e25dd8b42aac778b5603/html5/thumbnails/6.jpg)
Multimodality
I What is multimodality?I Modalities describe the different paths of communication
between a human and the computer.
6 / 16
![Page 7: A Multimodal Approach To Accessible Web Content On Smartphones](https://reader030.fdocuments.in/reader030/viewer/2022020207/5562e25dd8b42aac778b5603/html5/thumbnails/7.jpg)
Multimodality and universal access
OviattI Claims that multimodality can greatly expand the accessibility
of computing for diverse and non-specialist users.I Also claims that multimodality can promote new forms of
computing not previously available.
7 / 16
![Page 8: A Multimodal Approach To Accessible Web Content On Smartphones](https://reader030.fdocuments.in/reader030/viewer/2022020207/5562e25dd8b42aac778b5603/html5/thumbnails/8.jpg)
Multimodality, universal access, visually impaired and blind
I Spoken and multimodal bus timetable systems: design,development and evaluation, Turunen et al., 2005
I Multimodality generally improves performanceI Users need traning to be able to use the system
8 / 16
![Page 9: A Multimodal Approach To Accessible Web Content On Smartphones](https://reader030.fdocuments.in/reader030/viewer/2022020207/5562e25dd8b42aac778b5603/html5/thumbnails/9.jpg)
Voice recognition
I External ServerI Several commercial services:
I Google VoiceI (Siri)I NuanceI Vlingo
9 / 16
![Page 10: A Multimodal Approach To Accessible Web Content On Smartphones](https://reader030.fdocuments.in/reader030/viewer/2022020207/5562e25dd8b42aac778b5603/html5/thumbnails/10.jpg)
Frameworks
I MONA (Niklfield et al.)I Miranda (Paay et al.)I Others (Reithinger et. al)(Nardelli et al.)I W3C Multimodal Interaction Framework
10 / 16
![Page 11: A Multimodal Approach To Accessible Web Content On Smartphones](https://reader030.fdocuments.in/reader030/viewer/2022020207/5562e25dd8b42aac778b5603/html5/thumbnails/11.jpg)
Framework Implementation
I Based on W3C’s specification of their Multimodal InteractionFramework
I Used EMMA.I EMMA - “Extensible MultiModal Annotation markup
language”I Descriptions of XML markup languageI Represents the semantics and meaning of dataI Implemented on a need to have basis
I Consist of a set of componentsI Recognition componentI Interpretation componentI Interaction manager
11 / 16
![Page 12: A Multimodal Approach To Accessible Web Content On Smartphones](https://reader030.fdocuments.in/reader030/viewer/2022020207/5562e25dd8b42aac778b5603/html5/thumbnails/12.jpg)
Method
I Test the framework implementationI Evaluate multimodal mobile interfacesI User testI Four blind and visually impaired usersI Two phases:
1. Practical - Users were given tasks to carry out2. Semi-structural interview
12 / 16
![Page 13: A Multimodal Approach To Accessible Web Content On Smartphones](https://reader030.fdocuments.in/reader030/viewer/2022020207/5562e25dd8b42aac778b5603/html5/thumbnails/13.jpg)
Prototype
I Developed on top of, and in parallel to, the multimodalframework
I For AndroidI Used a norwegian weather service (yr.no)I Used norwegian voice recognition from NuanceI Input modalities:
I SpeechI TouchI Touch gesturesI OrientationI Acceleration gesturesI Touch keyboardI Navigation keys
13 / 16
![Page 14: A Multimodal Approach To Accessible Web Content On Smartphones](https://reader030.fdocuments.in/reader030/viewer/2022020207/5562e25dd8b42aac778b5603/html5/thumbnails/14.jpg)
Results
I Speech input was preferredI Touch gestures and orientation was perceived as more fun
than usefulI Acceleration gestures was seen as simple to useI Touch keyboard, nice but slowI Navigation keys, OK way of navigating
14 / 16
![Page 15: A Multimodal Approach To Accessible Web Content On Smartphones](https://reader030.fdocuments.in/reader030/viewer/2022020207/5562e25dd8b42aac778b5603/html5/thumbnails/15.jpg)
Results
I Feasible to implement W3C’s Multimodal InteractionFramework on Android
I The users were positive towards a multimodal interfaceI Multimodal interface can help support universal access
15 / 16
![Page 16: A Multimodal Approach To Accessible Web Content On Smartphones](https://reader030.fdocuments.in/reader030/viewer/2022020207/5562e25dd8b42aac778b5603/html5/thumbnails/16.jpg)
Thank you!
I Questions?
16 / 16