“Touchscreen Typing Accessibility for the Blind in

India”

Adit Gupta

Research Project Advisor – Dr. Nikhil Balram

Human-Computer Interaction project

Motivation

State of the art in eyes-free typing

tech

Field Study at BPA Future work plan

Broad scope of the talk

HUMAN – COMPUTER INTERACTION

MOTIVATION

Africa15%

Amer-ica8%

Emirates12%

Europe7%

SEAR (India Ex)10%

WPR (China Ex)6%

India21%

China21%

Percentage distribution of blind worldwide

Pascolini D, Mariotti SPM. Global estimates of visual impairment: 2010. British Journal Ophthalmology Online First published December 1, 2011 as 10.1136/bjophthalmol-2011-300539.

Number of people (in thousands)Visually Impaired per million total population

Only 5% of them receive any kind of assistive technology support!

About 8.075 million of 39.365 million in India!

A long term study by accessibility researchers in Portugal found that the blind

spent more time searching for contacts or typing text-messages than actually calling!

Tiago Guerreiro, Hugo Nicolau, Joaquim Jorge, and Daniel Gonsalves. 2009. NavTap: a long term study with excluded blind users. In Proceedings of the 11th international ACM SIGACCESS conference on Computers and accessibility (Assets '09). ACM, New York,NY,USA,99-106.DOI=10.1145/1639642.1639661 http://doi.acm.org/10.1145/1639642.1639661

The problems faced by the blind in India in accessing assistive technologies is much greater!

HUMAN – COMPUTER INTERACTION

Current Assistive Technologies

Technology Braille Display Price $(USD)

Braille Sense Plus Yes 6000

Voice Sense No 2000

VoiceNote BT No 1900

Refreshabraille 18 Yes 1700

PAC Mate BX 400 No 1400

EasyLink & Pocketwrite No 1000

GalaTee No 400

Pictures: (top-right left-right)Braille Sense Plus; VoiceNote BT; GalaTee, Refreshabraille, Easylink respectively

Brian Frey, Caleb Southern, and Mario Romero. 2011. Brailletouch: mobile texting for the visually impaired. In Proceedings of the 6th international conference on Universal access in human-computer interaction: context diversity - Volume Part III (UAHCI'11), Constantine Stephanidis (Ed.), Vol. Part III. Springer-Verlag, Berlin, Heidelberg, 19-25.

Do these technologies solve the problem of Information accessibility for the blind completely?

Tiago Guerreiro, Hugo Nicolau, Joaquim Jorge, and Daniel Gon\&\#231;alves. 2009. NavTap: a long term study with excluded blind users. In Proceedings of the 11th international ACM SIGACCESS conference on Computers and accessibility (Assets '09). ACM, New York, NY, USA, 99-106. DOI=10.1145/1639642.1639661 http://doi.acm.org/10.1145/1639642.1639661

Drawbacks of these Technologies

1. Expensive!!

2. Additional item to carry

3. Batteries, and setup time

4. Procurement in India

5. Truncated features while

customization

6. Reduced computational capabilities

Research Problem

Helping the blind access general information almost as seamlessly as normally sighted people!

Our Approach

Could accessible smartphones be a possible solution?

Why this Form Factor?A simple observation- New York Times applications

Smartphones present information in the most condensed format, hence reduce screen-reading time!

Apart from being Ubiquitous, Affordable, Connected & Mobile

Smartphone App iPad App Website

State of the Art in eyes-free typing

Learning with Braille

Mario Romero, Brian Frey, Caleb Southern, and Gregory D. Abowd. 2011. BrailleTouch: designing a mobile eyes-free soft keyboard. In Proceedings of the 13th International Conference on Human Computer Interaction with Mobile Devices and Services (MobileHCI '11). ACM, New York, NY, USA, 707-709.

The Braille code consists of a 3 by 2 binary matrix en-codes up to 63 characters In English Braille, a single combination encodes one character.

BrailleTouch

Mario Romero, Brian Frey, Caleb Southern, and Gregory D. Abowd. 2011. BrailleTouch: designing a mobile eyes-free soft keyboard. In Proceedings of the 13th International Conference on Human Computer Interaction with Mobile Devices and Services (MobileHCI '11). ACM, New York, NY, USA, 707-709.

Drawbacks:Numeric keypad does not have a fairly easy implementation

Needs both the hands to input text

Feedback for entire sentences along with each character not implemented

Georgia Tech Research

TypeInBraille

Subdivided a braille cell into 3 parts(Derivative of the BrailleTouch project)

Drawbacks:Needs TWO or more gestures for each letter.

Involves splitting each Braille cell into 3 parts mentally which might lead to slower typing speeds.

Numeric keypad does not have a fairly easy implementation

João Oliveira, Tiago Guerreiro, Hugo Nicolau, Joaquim Jorge, and Daniel Gonçalves. 2011. BrailleType: unleashing braille over touch screen mobile phones. In Proceedings of the 13th IFIP TC 13 international conference on Human-computer interaction - Volume Part I (INTERACT'11), Pedro Campos, Nuno Nunes, Nicholas Graham, Joaquim Jorge, and Philippe Palanque (Eds.), Vol. Part I. Springer-Verlag, Berlin, Heidelberg, 100-107

University of Milan

TalkingDialer

Drawbacks:

Alphabetic keypad is not intuitive at all, needs knowledge of the spacial positions of some alphabets

Technique assumes the presence of Hardware call button.

No user interaction studies available for this project although OpenSource

OpenSource code and documentation: https://code.google.com/p/eyes-free/

Google’s Eyes Free project

Field Study in India - Visit to Blind Peoples

Association

Blind Peoples Association (BPA)

• Established in 1947• BPA works out of 15 campuses, has 300 odd professionally

qualified staff• It has a number of innovative programs, such as schools, a

national rehabilitation engineering institute, a computer training centre, an electronics training centre, a bakery and food products division for disabled women, and a number of vocational courses.

www.bpaindia.org

Mr. Tarak LuharPrincipal at BPA School

Copyright: Fynn Studio, Santa Monica, California

“One Braille cell is mapped to THREE different languages we teach here apart from Arithmetic numbers!”

“I want to experience color”

“My salary is Rs. 35,000 per month and I cannot afford those ‘Braillo’ computers that are available!!”

Mr. Rahim KhanBraille Teacher, Toys and puzzle maker

“We try and maintain the teacher-student ratio at about 1:10 so that the children get more attention per class.

Also handling more children would be really tough for blind teachers.”

Mr. R. P. SoniCoordinator BPA Technology Lab

“I dislike the Braille writers because I can’t communicate with the normal sighted people through braille”

“Not only are the specially made devices costly but also reduce the functionality while trying to accommodate the needs of the blind. ”

Unplanned Demo of TalkingDialer, Quick learner!

Copyright: Fynn Studio, Santa Monica, California

JAWS (Job Access With Speech)

Wikipedia.org & the JAWS website. JAWS is produced by the Blind and Low Vision Group of Freedom Scientific, St. Petersburg, Florida, USA

computer screen reader program (text-to-speech output or by a Refreshable Braille display) most popular screen reader worldwide

Usage License cost: $1095!

Drawbacks: On large screens, a lot of unnecessary information is audio

read to the blind user(example- Advertisements on web pages, toolbars in mail, options menu etc.)

Only Audio feedback available for normal desktops

Amitt PatelAlumnus of the BPA Technology Lab

Employed at SBI, Online Banker and Stock Broker!

“I use my smartphone for send email and staying in touch more than my laptop!”

Demo of TalkingDialer and BrailleTouch. He learnt both within a minute of instruction (even after the bug in BrailleTouch’s feedback)!!!

In touch with him via email!

Future Work Plan

Fundamental Scientific Questions

#Q1: What are “intuitive” gestures of text input to the blind?

#Q2 : What feedback systems are the Blind most comfortable with? (Audio/Vibration Hybrid??)

•

•

•

Blind people (congenitally blind or otherwise) with a sound knowledge of typing on mobile phone keypads, preferably smartphones

Target Subjects

FUTURE WORK

Prototyping Experiments

Implementation and Data collection

Analysis and Documentation

New Design proposition in the Indian context

“Touchscreen Typing Accessibility for the Blind in

India”

Adit Gupta

Research Project Advisor – Dr. Nikhil Balram