Playing Card RecognizerPlaying Card Recognizer

ECE 4025ECE 4025

February 28, 2002February 28, 2002

Group 5Group 5Robert BarrettRobert Barrett

Jason HodkinJason Hodkin

Chung Tse MarChung Tse Mar

Jay SilverJay Silver

David WinklerDavid Winkler

Yu Ming WuYu Ming Wu

MotivationMotivation

Short-term:Short-term:– Demonstrate effectiveness and usefulness of Demonstrate effectiveness and usefulness of

character recognitioncharacter recognition

Long-term:Long-term:

Design autonomous on-line card playing Design autonomous on-line card playing systemsystem

Project ScopeProject ScopeImage Restrictions:Image Restrictions:– One card per imageOne card per image– No occlusion of cardNo occlusion of card– High contrast solid backgroundHigh contrast solid background– Constant light sourceConstant light source

Image Allowances:Image Allowances:– ScalingScaling– RotationRotation– Font styleFont style

How does it work?How does it work?System diagramSystem diagram

Down Sample & Convert to

Binary

High Resolution

Color Image

Binary Image of Card Corner

Label Connected Objects & Flood Fill Card Area

Cross-correlate

with Templates

Compute Scores

and Compare

Card Value and Suit

Use Major Axis to Find Corners for Scaling and

Rotation

Extract Corner for

Comparison

Color to Gray Scale ConversionColor to Gray Scale Conversion[R G B] YR UR VR

YG UG VG

YB UB VB

= Y U V

We are only interested in intensity (Y)

[R G B] YR

YG

YB

= intensity

YR = 0.299 YG =0.587 YB =0.114

Gray Scale to Binary ConversionGray Scale to Binary ConversionIf grayScaleValue > mean + 2*Pixel = 1; Else, Pixel = 0

25 13 1 50 28 40 247 5 225 13 18 254 190 12 5 7 210 18 255 32 11 5 36 20 13

0 0 0 0 0 0 1 0 1 0 0 1 1 0 0 0 1 0 1 0 0 0 0 0 0

LabelingLabeling

1 1 1 0 0 0 0 0 1 1 1 0 1 1 0 0 1 1 1 0 1 1 0 0 1 1 1 0 0 0 1 0 1 1 1 0 0 0 1 0 1 1 1 0 0 0 1 0 1 1 1 0 0 1 1 0 1 1 1 0 0 0 0 0

1 1 1 0 0 0 0 0 1 1 1 0 2 2 0 0 1 1 1 0 2 2 0 0 1 1 1 0 0 0 3 0 1 1 1 0 0 0 3 0 1 1 1 0 0 0 3 0 1 1 1 0 0 3 3 0 1 1 1 0 0 0 0 0

Identify blobs

Determine characteristics of blobs

Area = 24

Area = 4

Area = 5

Flood FillFlood FillFloodFill ( pixelToEliminate ) FloodFill ( allNeighboringPixels) Erase all neigboring pixels end

Calculate Angle of Major Axis of Calculate Angle of Major Axis of Best Fit Ellipse (orientation)Best Fit Ellipse (orientation)

RotateRotate

25 13 1 50 28 40 247 5 225 13 18 254 190 12 5 7 210 18 255 32 11 5 36 20 13

25 13 1 50 28

40 247 5 225

13 18 254 190 12

5 7 210 18

255 32 11 5 36

20 13

Isolate the Important InformationIsolate the Important Information

Right Side of Card

Bottom of Card

Top of Card

Left Side of Card

Scan each row and column for a white pixel

Locate Identifiable Symbol and isolate itLocate Identifiable Symbol and isolate it

top

left

bottom

right

Choose an Appropriate Template and Choose an Appropriate Template and Scale to Match the TemplateScale to Match the Template

Choose a font that should be consistent with most card fonts, but that is different from the data set.

Template

Image

Freecell (A card game on most windows OS’s)

Take a Normalized Cross CorrelationTake a Normalized Cross Correlation[X(r+m,c+n)*H(r,c)] = Correlation(m,n)n m

Take a Normalized Cross CorrelationTake a Normalized Cross Correlation

Highest “Score” is a MatchHighest “Score” is a MatchMax Corr is 0.5581

Max Corr is 0.3016

Implementation OverviewImplementation Overview

EVM

Correlation Calculation

Templates

Final Result

Color to Binary

HOST

Corner Extraction

ScalingLabel

Flood Fill Rotate

Decision Model

RTDX

Camera

Implementation OverviewImplementation Overview

Image File

Decision Model

HOST

Corner Extraction

ScalingRTDX

EVM

Correlation Calculation

Templates

Final Result

Realtime ConstraintRealtime Constraint

Realtime constraint depends on the Realtime constraint depends on the applicationapplication– How fast the cards changeHow fast the cards change– Bounded by camera frame rateBounded by camera frame rate

Set goal of 1 frame/secondSet goal of 1 frame/second

Computational ComplexityComputational Complexity

1 frame/second1 frame/second

64x64 = 4096 pixels/template64x64 = 4096 pixels/template

100 cross-correlations/template100 cross-correlations/template

13 templates for number, 4 templates for 13 templates for number, 4 templates for suitesuite

100 * 4096 * 17 * 1 = 7 Million MACs/second

Next StepsNext Steps

Implementing processing on EVMImplementing processing on EVM– Cross-correlationCross-correlation

Implementing processing on hostImplementing processing on host– Corner extractionCorner extraction– ScalingScaling– Decision model and displayDecision model and display

Synchronizing host with EVM using RTDX Synchronizing host with EVM using RTDX APIAPI