ART: Augmented Reality Table for Interactive Trading Card Game

ART: Augmented Reality Table for Interactive Trading Card Game

Albert H.T. Lam, Kevin C. H. Chow,Edward H. H. Yau and Michael R. Lyu

Department of Computer Science and EngineeringThe Chinese University of Hong Kong

Presentation OutlinePresentation Outline

• Introduction

• Objective

• Augmented Reality Table

• System Design

• Implementation

• Demonstration

• Conclusion

Trading Card GameTrading Card Game

• Two players play a match.

• Players play cards to fight with each other. It includes summoning monsters, casting spells, or setting traps, etc.

• We choose “YU-GI-OH”as our implementation

Game MatGame Mat

• Game Mat of YU-GI-OH• Different kinds of card must be put inside

particular card zones

Traditional Card Game Computer Card Game Traditional Card Game Computer Card Game

• Traditional Card Game– Interaction with real people– Lack of sound and visual effect– Require players’ own imaginations

• Computer Card Game– 3D animations– Handle complicated rules– Interaction with computer

ObjectiveObjective

• To develop a generic Interactive Trading Card Game Interface which allows:

– Interaction with people and computer– Playing naturally in the original way– Visual and sound effect– Complicated calculation

Augmented Reality Table (ART)Augmented Reality Table (ART)

• ART is a prototype platform– Employ Augmented Reality technology– Users can play trading card games on it

– Provide an interactive environment and interface

– Realize the game with 3D graphics and sound effect

Hardware SetupHardware Setup

• Overhead camera– The only input device

• Plasma monitor– Act as a table– Display 3D-animation– Produce sound effect

• Computer– Recognize valid inputs– Maintain game rules– Generate outputs

System ArchitectureSystem Architecture

• Perception Module– Calibrate– Extract card and input

commands from the camera

• Database Module– Access card information– Access game and card

rules – Identify cards

• Game Enhancement Module– Display game mat– 3D animation– Sound effect

• GameCore Module– Control game flow– Store game state– Rule-based game

engine

System ArchitectureSystem Architecture

ART Card Game

ART Card Game CoreGeneric Card Game Database

Augmented Reality Perception

CommandDetector

Calibration

Card Detector Game Enhancement

Video Decoder

Card Locator

Calibration

Card Recognizer

3D Animation

Image Database

Input Analyzer

Sound Effect

Game CoreRule Database

DisplayCamera

Card Information

Input Information

Environment Information Output Generator

Game Manager

Rule ManagerCard Database

PerceptionPerception

• Read and process the raw video perceived from the camera

• Detect card inputs and commands

• Some assumptions– Fixed camera, fixed table– Camera is approximately right above the table– Cards can only be placed in predefined

regions called card zones

Card LocatorCard Locator

• Locate search window– The screen is partitioned into some fixed small search

windows as card zones identified in calibration– Select those with significant number of changed

pixels as candidates

– Search windows just stop changing will be processed– This minimizes both search space and frequency

Image difference Binary image difference

Card LocatorCard Locator

• Locate four corner card positions given a search window– Apply Canny Algorithm– Extract Contours– Search for contours which

• contain 4 points• contain nearly right

angle corners• have area within a

certain threshold

Card DetectorCard Detector• Detect the card input including:

– Location (in which card zone)– Orientation (vertical or horizontal)– Unique card ID (to query for card information)

• Compute an undistorted version of card image by geometric transformation followed by bilinear interpolation

Card DetectorCard Detector

• Identify card image by 2 steps:– Image retrieval

• Classify and retrieve cards of the same type as candidates

– Image matching• Use color-based matching since higher level features are lost

due to low resolution of query image (about 40 x 40 pixels)• Find the best match card according to pixel difference:

• Reject the image if the pixel difference is larger than the threshold

• Accept the image with minimum pixel difference• Improve the accuracy by using block matching

w

i

h

jDQDQP jiIjiIIID

1 1

2)),(),((),(

DatabaseDatabase

• The Database Module stores all game information• It provide necessary information for the other

three modules• Image Database

– Contain all card images

• Card Database– Contain all card

information

• Rule Database– Contain all game

rules and card rules

Rule-based Game EngineRule-based Game Engine

• The system is driven by rules• Define a rule:

– Premise-conclusion form– e.g.

“If in main phase and player put a card, then summon the monster.”

– The action may be to load another rule, remove a rule, reset a rule, update game states, or show some outputs, etc.

MainPhase PutCard SummonActionLIstActionLIstDisplay Monster

premise 1 premise 2 conclusionaction list

Main Phase Put Card Summon

Destroyer Summon Destroy Card

premise 1 premise 2 conclusion

Rule-based Game EngineRule-based Game Engine• Maintain a pool of rules which loads and stores

rules dynamically during the game

• Receive and raise appropriate events according to the game rules, thus, maintaining the game logic

• Mechanism: forward checking until no more conclusion is reached

Why Rule-based?Why Rule-based?

• Extensible– New cards can be extended by adding new rules and

new card information to the database

• Flexible– Card games are very flexible and some card effect

are unpredictable, of which brute-force programming cannot handle it

• Generic– New card games can be implemented by using a new

set of game rules

Game EnhancementGame Enhancement

• Provide game mat to place cards

• Provide buttons for command input

• Enhance visual and sound effects

• Display game states and hints

Game MatGame Mat

Game statesCard zones Buttons Battle field

Screenshot during BattleScreenshot during Battle

DemonstrationDemonstration

ConclusionConclusion

• We developed a prototype of the Augmented Reality Table for trading card games

• With AR techniques, the system enhances existing trading card games, while remaining much of original playing style

• The system provided an interactive interface for players and performs player’s effort to maintain game rules and calculations

Q&AQ&A

Improved Block Matching Algorithm

Improved Block Matching Algorithm

• Solution, use an Improved Block Matching Algorithm– Divide the image into 9 squares– Apply Block Matching Algorithm to each

squares