PLT 2007 CSIS Shorthand Handwriting Recognition for Pen-Centric Interfaces Charles C. Tappert 1 and...
-
date post
20-Dec-2015 -
Category
Documents
-
view
213 -
download
0
Transcript of PLT 2007 CSIS Shorthand Handwriting Recognition for Pen-Centric Interfaces Charles C. Tappert 1 and...
PLT 2007
CSISCSIS
Shorthand Handwriting Recognition for Pen-Centric Interfaces
Charles C. Tappert1 and Jean R. Ward2
1 School of CSIS, Pace University, New York, USA2 Pen Computing Consultant, Massachusetts, USA
PLT 2007
CSISCSIS
Thesis: Pen-Centric, Chatroom-Like Shorthand Interfaces
• Will provide critical infrastructure for many pen-centric applications
• Will provide fast text input
• Will have greatest impact on applications running on small mobile devices
PLT 2007
CSISCSIS
Agenda• Handwriting
– Fundamental Property of Writing– Handwriting Recognition Difficulties
• Historical Shorthand Alphabets• Online (Pen-Centric) Handwriting Recognition
– Online more accurate than Offline Recognition– Online Info Can Complicate Recognition Process– Design Tradeoffs/Decisions
• Pen-Centric Shorthand Alphabets• Pen-Centric Word/Phrase Shorthand• Allegro/Chatroom Experimental Shorthand System
PLT 2007
CSISCSIS
Fundamental Property of Writing
• Differences between different characters are more significant than differences between different drawings of the same character
• This makes handwritten communication possible
PLT 2007
CSISCSIS
Fundamental Property of Writing
• Property holds within subalphabets of uppercase, lowercase, and digits, but not across them
• “I”, “l”, and “1” written with single vertical stroke• “O” and “0” written similarly with an oval
PLT 2007
CSISCSIS
Handwriting Recognition Difficulties
• Shape, size, and slant variation
• Similarly shaped characters – U and V • Careless writing
– in the extreme, almost illegible writing
• Resolving difficult ambiguities requires sophisticated recognition algorithms, syntax/semantics
PLT 2007
CSISCSIS
Historical Shorthand Alphabets(prior to pen computing)
• Famous writings were written in shorthand – Cicero’s orations– Martin Luther’s sermons– Shakespeare’s and George Bernard Shaw’s plays
• We focus on shorthand appropriate for PDAs• Two main types of shorthand
– Non-geometric shorthand– Geometric shorthand
• Small number of basic shapes• Shapes reused in multiple orientations
PLT 2007
CSISCSIS
Other Historical Shorthand Systems
• Phonetic alphabets– Pitman (1837)– Gregg (1885)
• Systems for the blind – Braille (1824)
• Cursive shorthands – Gabelsberger (1834)
PLT 2007
CSISCSIS
Online (Pen-Centric) Handwriting Recognition
• Machine recognizes the writing as the user writes • Digitizer equipment captures the dynamic
information of the writing
– Stroke number, order, direction, speed – A stroke is the writing from pen down to pen up
PLT 2007
CSISCSIS
Online (Pen-Centric) more accurate than Offline (Static) Recognition
• Can use both dynamic and static information• Can often distinguish between similarly
shaped characters
– E.g., 5 versus S where the 5 is usually written with two strokes and the S with one stroke
PLT 2007
CSISCSIS
Online Information Can Complicate Recognition Process
• Segmentation ambiguities– Character-within-character problem – cl versus d
• Large number of possible variations – E can be written with one, two, three, or four strokes, and with various
stroke orders and directions – Four-stroke E has 384 variations (4! stroke orders x 24 stroke directions)
PLT 2007
CSISCSIS
Design Tradeoffs/Decisions
• No constraints on the user– Machine recognizes user's normal writing
• User severely constrained– Must write in particular style such as handprint – Must write strokes in particular order, direction,
and graphical specification• Simplest is one stroke per character, one stroke
direction, one shape
PLT 2007
CSISCSIS
Pen-Centric Shorthand Alphabets
• Some of the earliest were for CAD/CAM
• Others developed for text input on PDAs
• We review geometric and non-geometric shorthands appropriate for small devices
• Historical alphabets presented above could be used for machine recognition
• In addition to shape and orientation, stroke direction can differentiate among symbols
PLT 2007
CSISCSIS
Simplified Design Tradeoffs/Decisions for Graffiti and Allegro PDA Alphabets
• Small alphabet– one case rather than both upper and lowercase
• Small number of writing variations per letter– preferably only one
• One stroke per character (character = stroke)– allows machine to recognize each character upon pen lift
• Separate writing areas for letters and digits– avoids confusion of similarly shaped letters and digits
PLT 2007
CSISCSIS
Graffiti and AllegroCommercially Successful Shorthands
• High correspondence to Roman alphabet– Easier to learn– Graffiti used in Palm OS devices
• notably the Palm Pilot and Handspring models
– Allegro used in Microsoft Windows devices
• Geometric alphabets not successful
PLT 2007
CSISCSIS
Pen-Centric Word/Phrase Shorthande.g., Chatroom Shorthand
• Further increase speed of text entry
• Potential applications – Where input speed important
– Where word/phrase abbreviations occur frequently – e.g., email
PLT 2007
CSISCSIS
Allegro/Chatroom Shorthand System
• Developed for M.S. dissertation– Student was hearing impaired
– Developed as output component of communication system• Handwriting to text to speech
• Two input writing areas– One for Allegro (all-purpose)– One for chatroom-like words/phrases (e.g., CUL, F2F)
PLT 2007
CSISCSIS
Allegro/Chatroom Shorthand SystemStroke acquisition GUI
allegro strokerecognition
alphabet
sentence accumulator
Sentence display and spoken output
allegro strokelibrary
user-defined stroke library
a single stroke
other strokerecognition
word/phrasecharacter
done?no
yes
meaning
is it
PLT 2007
CSISCSIS
Allegro/Chatroom Shorthand System Preliminary Experimental Results
• Allegro/Chatroom pen-centric shorthand input faster than typing text and comparable to typing text and chatroom shorthand characters