UC Berkeley CS294-9 Fall 2000 12- 1
Document Image AnalysisLecture 12: Word Segmentation
Richard J. FatemanHenry S. Baird
University of California – BerkeleyXerox Palo Alto Research Center
UC Berkeley CS294-9 Fall 2000 12- 2
The course, recently….
• We studied symbol recognition, classifiers
and their combinations
• Word recognition as distinct from characters
UC Berkeley CS294-9 Fall 2000 12- 3
A good segmentation method (or several) is handy
• We cannot rely on a lexicon to have all words (names, proper nouns, numbers, acronyms)
• Insisting that words be in the lexicon does not mean they are correct. Powerpoint tries to refuse misspell as mispell since the latter is not in the dictionary!
• Good segmentation means that the symbol based recognition has a better chance of success
UC Berkeley CS294-9 Fall 2000 12- 4
Segmentation/ Naïve or clever
• Numerous papers on the subject• Some without strong models (e.g. cut at
thin parts)• Some with exhaustive search / template
matching• Some with learning/ internal
comparisons
UC Berkeley CS294-9 Fall 2000 12- 5
Naïve connected component analysis can’t come close…
• Characters like “ij:; Ξ â% are separated• Ligatures are not separated: ffl, ŒÆœ ffi
• Vertical cuts between touching characters will not ordinarily work for italics
THIS IS ULTRA CONDENSED ..TZ this is times italic .
(other problems: X2 , )3 22 yx
UC Berkeley CS294-9 Fall 2000 12- 6
Papers of interest on segmentation
• Tsujimoto and Asada• Bayer and Kressel• Tao Hong’s (1995) PhD on Degraded
Text Recognition
UC Berkeley CS294-9 Fall 2000 12- 7
Segmentation + Clustering (Tao Hong)
UC Berkeley CS294-9 Fall 2000 12- 8
Can lead to decoding!
UC Berkeley CS294-9 Fall 2000 12- 9
Sometimes the image itself holds a key to decoding…
UC Berkeley CS294-9 Fall 2000 12- 10
Visual inter-word relations
UC Berkeley CS294-9 Fall 2000 12- 11
An example text block showing visual inter-word relationships
UC Berkeley CS294-9 Fall 2000 12- 12
Pattern matching can lead to identifying a segment
UC Berkeley CS294-9 Fall 2000 12- 13
UC Berkeley CS294-9 Fall 2000 12- 14
Where this fits…
UC Berkeley CS294-9 Fall 2000 12- 15
Example
UC Berkeley CS294-9 Fall 2000 12- 16
Tsujimoto & Asada: Overview
UC Berkeley CS294-9 Fall 2000 12- 17
Resolve the touching characters:
• New metric for finding breaks (find plausible breaks
• Use knowledge about “the usual suspects” rn/m k/lc d/cl … (limits search substantially)
UC Berkeley CS294-9 Fall 2000 12- 18
Metric, pre-processing
ANDing columns for profile removing slant from italics
UC Berkeley CS294-9 Fall 2000 12- 19
Choosing break candidates
UC Berkeley CS294-9 Fall 2000 12- 20
Decision Tree for “The”
UC Berkeley CS294-9 Fall 2000 12- 21
Tree search
• Depth first, looking for solution to the string matching, in sequence.
• Some partitions are penalized (but not eliminated) if the segmentation point is uncertain.
• Segments are matched to omnifont templates (“multiple similarity method..”)
UC Berkeley CS294-9 Fall 2000 12- 22
Reexamined explanations
m rn
q cj
k lc
B 13
H I-I
mm nun
ck dcEtc… 30 confusions
This might be mistaken for This
UC Berkeley CS294-9 Fall 2000 12- 23
Some tough calls…
UC Berkeley CS294-9 Fall 2000 12- 24
Unbelievable accuracy…
UC Berkeley CS294-9 Fall 2000 12- 25
A different, perhaps more general method (Bayer, Kressel)
• Goal: find the column position(s) at which characters are touching– Treat as a systematic classification problem– Learn from a data base containing labelled merged
characters• Collect real life data; get human breakpoints [or could
be synthetic, I suppose]• Find appropriate feature set• Learn the features of touching characters
– Hypothesize column breaks– Application: postal addresses, other stuff too
UC Berkeley CS294-9 Fall 2000 12- 26
Database of touching chars
….2158 patterns
UC Berkeley CS294-9 Fall 2000 12- 27
Big ideaRather than represent the breaks as low points in the projection profile, represent the breaks in the natural context of touching characters by actual example, suitably normalized for size (15-30 pixels high).
These locations are manually marked.
UC Berkeley CS294-9 Fall 2000 12- 28
Local feature set describing cut locations / measures of similarity
• Number of black pixels (= projection profile!)
• Number of white pixels counting from top/bottom
• Number of white-black transitions• Number of identical b or w pixels next to
this column (derivative of pp?)
UC Berkeley CS294-9 Fall 2000 12- 29
Global feature set describing cut locations / measures of similarity
• Width to height ratio of full image (wider suggests touching characters)
• Width to height ratio of the image AFTER cutting(s)
• Number of white-black transitions• Number of identical b or w pixels next to
this column (derivative of pp?)
UC Berkeley CS294-9 Fall 2000 12- 30
Illustration of the strategy
UC Berkeley CS294-9 Fall 2000 12- 31
How accurate, how fast? (cut location)
• Finding cuts: 7.8% error in learning set, 7.2%(!) on test set
• 22% of the no-cut regions had errors• Best results used 50-feature classifier
using 9 column width• Cost for one image cut-analysis one
character analysis• Validates statistics > heuristics..
Top Related