Signboard Optical Character RecognitionIsaac Wu, Hsiao-Chen Chang
Department of Electrical Engineering, Stanford University
Motivation System Pipeline
Results
Having the ability to recognize any store just by taking
a picture of its signboard is a powerful asset for
business reviews and ratings companies such as Yelp
to incorporate into their mobile app.
Phase 1: Training A BC
E
…D
Phase 2:
Segmentation
MSER GrayscaleDetect MSER
RegionsRemove Non-Text Regions
Create Bounding Boxes of Each Region
Merge Boxes and Keep the
Longest
Morphology GrayscaleIncrease Contrast
Adaptive Thresholding
Morphological Opening
Small Region Removal
Region Labeling
Remove Non-Text Regions
Create Bounding Box
Phase 3: RecognitionA B
CE
…D
MCDONALDS
OCRRestrict OCR Matching to
English LettersPerform OCR
Remove short length words
Remove Spaces
Success Rate: 86%
# Testing Images: 113
# Correctly Determined: 97
AlgorithmtrainDatabase()
if (SIFT with MSER has many matches) return result
elseif (SIFT with Morphology has many matches) return result
elseif (OCR with MSER seems valid) return result
elseif (OCR with Morphology seems valid) return result
else return null
* Multi-scale approach
SIFTExtract SIFT Descriptors
Match Descriptors
to Codes
Top 5 Database Matches
Perform SIFT Match with
RANSAC
Return the Most
Matches
Manually Segmented Images
Extract SIFT Descriptors
Construct K-Means Codebook
Match Descriptors to Codes
89.7%
7.2%3.1%
Techniques Used
MSER x SIFT
MORPH x SIFT
MSER x OCR
MORPH xOCR
Top Related