Accessible Education through Crowd-Powered …...–fix a problem once, reach many students •More...
24
University of Rochester Human-Computer Interaction Jeffrey P. Bigham Accessible Education through Crowd-Powered Systems Jeffrey P. Bigham @jeffbigham University of Rochester
Transcript of Accessible Education through Crowd-Powered …...–fix a problem once, reach many students •More...
University of Rochester Human-Computer Interaction Jeffrey P. Bigham
Accessible Education through Crowd-Powered Systems
Jeffrey P. Bigham @jeffbigham
University of Rochester
University of Rochester Human-Computer Interaction Jeffrey P. Bigham
Accessible Education through Crowd-Powered Systems
Jeffrey P. Bigham @jeffbigham
University of Rochester
University of Rochester Human-Computer Interaction Jeffrey P. Bigham
Accessibility VizWiz Scribe
Dual Goals
1. Advocate for Accessibility
2. Accessibility w/ Crowd-Powered Systems
Accessibility
University of Rochester Human-Computer Interaction Jeffrey P. Bigham
Accessibility VizWiz Scribe
(lower bounds on) Disability in Education
• 15% of the world’s population is disabled (about 1 billion people)
• 9% of the school-age population have disabilities
• 13% of undergraduate IT majors have disabilities. – 5% of graduate IT majors have disabilities.
• 0.8% of IT doctorates have disabilities (53 between 1999 and 2004)
• 40% of students with disabilities complete a bachelor's degree
– Compared with 60% of everyone else
Accessibility
University of Rochester Human-Computer Interaction Jeffrey P. Bigham
Accessibility VizWiz Scribe Accessibility
University of Rochester Human-Computer Interaction Jeffrey P. Bigham
Accessibility VizWiz Scribe
$150 / hour for real-time captioning
hours to convert book to digital text - days to convert its figures
adapting assignments
Accommodation is Expensive and Slow
Accessibility
University of Rochester Human-Computer Interaction Jeffrey P. Bigham
Accessibility VizWiz Scribe
Crowd Education
• Prepared Resources
– fix a problem once, reach many students
• More (distributed) Interaction
– as students support students, less accommodation?
• Student Crowds
– students have the expertise to help
Accessibility
University of Rochester Human-Computer Interaction Jeffrey P. Bigham
Accessibility VizWiz Scribe
University of Rochester Human-Computer Interaction Jeffrey P. Bigham
Accessibility VizWiz Scribe Accessibility
University of Rochester Human-Computer Interaction Jeffrey P. Bigham
VizWiz Nearly Real-Time Answers to Visual Questions Real-Time
University of Rochester Human-Computer Interaction Jeffrey P. Bigham
Accessibility VizWiz Scribe
VizWiz
Bigham et al. Nearly Real-Time Answers to Visual Questions. UIST 2010.
VizWiz
University of Rochester Human-Computer Interaction Jeffrey P. Bigham
Accessibility VizWiz Scribe
Releasing VizWiz
• Released on May 31, 2011
– 5000 users asked more than 50,000 questions
– average time to first answer < a minute
VizWiz
University of Rochester Human-Computer Interaction Jeffrey P. Bigham
Accessibility VizWiz Scribe VizWiz
University of Rochester Human-Computer Interaction Jeffrey P. Bigham
Legion:Scribe Real-Time Captions by Groups of Non-Experts
University of Rochester Human-Computer Interaction Jeffrey P. Bigham
Accessibility VizWiz Scribe
Real-Time Captioning
Stenographers ASR
expensive cheap
difficult to schedule available on demand
lack domain expertise can be trained for new vocab
pretty accurate does not work
Can I help?
NO, you are worse than ASR.
Scribe
University of Rochester Human-Computer Interaction Jeffrey P. Bigham
Accessibility VizWiz Scribe
Real-Time Captioning
Lasecki et al. “Real-Time Captioning by Groups of Non-Experts.” UIST 2012.
Scribe
University of Rochester Human-Computer Interaction Jeffrey P. Bigham
Accessibility VizWiz Scribe
Scribe Algorithm
Worker 1
Worker 2
Worker 3
open the
open java
the java
file up and
Graph
Time
Stage 1 Stage 2 Stage 3
file now
fiel
Baseline open the java file now and
openjava
the
openjava
the
openjava
the
filenow
up
and
W.S. Lasecki, C.D. Miller, D. Borrello and J.P. Bigham. “Online Sequence Alignment for Real-Time Audio Transcription by Non-Experts.” AAAI 2012 (poster).
Multiple Sequence Alignment
Online Version
Scribe
University of Rochester Human-Computer Interaction Jeffrey P. Bigham
Accessibility VizWiz Scribe
Scribe Interface
Encourages: - real-time input - global coverage - short sequences
Co-evolution of Interface and Algorithm
Scribe
University of Rochester Human-Computer Interaction Jeffrey P. Bigham
Accessibility VizWiz Scribe
Coverage Graph
Scribe
University of Rochester Human-Computer Interaction Jeffrey P. Bigham
Accessibility VizWiz Scribe
Tradeoff
Failures: “n-factorial” “in pectoral”
Scribe
University of Rochester Human-Computer Interaction Jeffrey P. Bigham
Accessibility VizWiz Scribe
Interesting Qualities
• Captionists can be experts – not at captioning but in the subject
• Low cost – $30/hour on Mturk (did not optimize)
– or free (impossible before)
• Recruited on demand – for only as long as needed
Scribe
University of Rochester Human-Computer Interaction Jeffrey P. Bigham
Accessibility VizWiz Scribe
Summary
• To be open to all, content must be accessible
• Scale Helps – Produced once, can be reused by many (usually)
– Many students ~= many experts
• Making interactive content accessible – leveraging the student crowd
ç
University of Rochester Human-Computer Interaction Jeffrey P. Bigham
Accessibility VizWiz Scribe
Thanks!
hci.cs.rochester.edu @jeffbigham
Funded by: National Science Foundation Grants (#IIS-1149709, #IIS-1116051, #IIS-1049080 ), and Google.
ç
University of Rochester Human-Computer Interaction Jeffrey P. Bigham
Accessibility VizWiz Scribe
