WSCG2008, Plzen, 04-07, Febrary 2008WSCG2008, Plzen, 04-07, Febrary 2008

Comparative Evaluation of Random Forest and Fern Comparative Evaluation of Random Forest and Fern classifiers for Real-Time Feature Matchingclassifiers for Real-Time Feature Matching

I. Barandiaran1, C.Cottez1, C.Paloc1 , M.Graña2

1Departamento de Aplicaciones Biomédicas Asociación VICOMTech, San Sebastián, {ibarandiaran,ccottez,cpaloc}@vicomtech.org

2University of Basque Country Computer Science School, Pº. Manuel de Lardizabal, 1 20009, San Sebastián, Spain ccpgrrom@si.ehu.es

VISUAL INTERACTION AND COMMUNICATIONS TECHNOLOGIES

22

SummarySummary

1.Introduction.

2.Random Forest, FERNS

1.Mixed/Augmented Reality Application.

2.Conclusions/Questions.

33

IntroductionIntroduction

Motivation and objectives

• Motivated by the work of Vincent LePetit (CVLab).

• Real-Time Augmented Reality.

• Camera Pose Estimation.

• Markerless tracking.

• Model-based tracking.

• Tracking by detection.

• Test and compare different parameters.

• Scale.

• Size of the Training Set.

• Number of Classes.

• Training Time.

44

Augmented Reality Features:

• Mix Virtual and Real Objects..

• Real-Time.

• Portable Devices (Head Mounted Display, Tablet PC, PDA Device, Movil Phone..)

IntroductionIntroduction

55

Problems:

• Rendering.

• Real-Time(Delay).

• Registration/Pose Estimation.

IntroductionIntroduction

66

Non model-based Tracking

• No a priori knowledge of the object to be tracked.

• Updates/Propagates an estimation over time.

• Partial object occlusions.

• Tend to tracking reinitialization.

Model Based Tracking

• Some a priori knowledge is available.

• May not depend on the past.

• Frame by Frame estimation.

• Robust against partial object occlusion.

• Automatic tracking initialization.

IntroductionIntroduction

77

1. Introduction

2. Random Forest, FERNS

3. Mixed/Augmented Reality

4. Conclusions/Questions

SummarySummary

88

Random Forest, FERNSRandom Forest, FERNS

Tracking of Planar Surfaces.The Classifiers are applied for interest point (feature) matching.Matched Points are used during camera pose estimation Process.

99

Random Forest, FERNSRandom Forest, FERNS

Building the training set.

• Frontal view of the object to be detected.

• Feature Point extraction FAST (Rosten06) and YAPE (CvLab).

• Sub-images (patches) are generated for each class.

Classes to Be recognized by the Classifier

1010

Random Forest, FERNSRandom Forest, FERNS

Building the training set.

• Generate Random Affine transformations.

• Generate new examples of each Class.

…..

Random Affine transformations

Training Set (examples)Training Set (examples)

1111

Random ForestRandom Forest

Multiclassifier based on Randomized Trees.Firstly introduced in 1997 handwritten recognition (Amit, Y.,German, D.)Developed by Leo Breiman (Medical Data Analisys).Recently Applied to tracking by detection (LePetit06).

Main Features

• Fast Training Step, and execution.

• Good Precision.

• Random selection of the independent variables (features).

• Random selection of Examples.

• Easy to Implement and paralelizable.

1212

Random ForestRandom Forest

Classifier Training.

• N Binary-Trees are Grown.

• Pixel intensity tests are executed in any non-terminal node.

• Pixels can be selected at Random.

• Posterior Distributions P(Y=c |T=Tk,n) are stored in leave nodes.

1313

Random ForestRandom Forest

Example Classification.

• Every example is dropped down the trees.

• The Example traverse the tree towards the leaf nodes.

Pixels to be tested

TT11 TT22 TTnn

Random ForestRandom Forest

1414

Random ForestRandom Forest

Combine Results

• The example labeling is obtained as a combination of partial results obtained by every tree in the forest.

T

k

nkitlabelclass tcYPExample

1_ |maxarg

1515

FERNSFERNS

Introduced in 2007 (Mustafa Özuysal).Multiclassifier.Applied to 3D keypoint recognition.Successfully applied to image recognition/retrieval (Zisserman07).

Main Features

• Non hierarchical structure.

• Semi Naive-Bayes Combination Strategy.

• Random selection of the independent variables (features).

• Random selection of Examples.

• Easy to Implement and paralelizable.

1616

FERNSFERNS

Semi-Naive Bayes Combination.

nii fffcCPc ,....,|maxarg 21

n

iinni fffP

cCPcCfffPfffcCP

,....,

)|,....,(,....,|

21

2121

otherwise

tppiff jjj 0

1 2,1, ini cCfffPc |,....,maxarg 21

N

j

ijin cCfPcCfffP1

21 ||,....,

M

j

ikin cCFPcCfffP1

21 ||,...., kF

1717

FERNSFERNS

Classifier Training

x x x

32 Possible Outputs

0 7

.

.

.

ik cCFP |Posterior Distributions

(Look-up Tables)

1818

FERNSFERNS

Classifier Training

1 1 0

0 0 0

0 1 06

0 0 1 1

2

3

Class 1Class 1

Class 2Class 2

Class 1Class 1

Class 2Class 2

..

..

..

..

..

..

..

..

..

..

Fern 1Fern 1

Fern 2Fern 2

Fern Fern nn

Posterior Distributions

(Look-up Tables)

..

..

..

..

1919

FERNSFERNS

Example Classification.

M

ikflabelclass cCFPExample1

_ |maxarg

2 6 1

Posterior Distributions

(Look-up Tables)

Fern 1Fern 1 Fern 2Fern 2 Fern 3Fern 3

2020

Random Forest vs FERNSRandom Forest vs FERNS

Rotation Range

• 20 Trees, 15 Depth.

• 225 Different Clases.

• 400 Images per class.

2121

Random Forest vs FERNSRandom Forest vs FERNS

Scale Range

• 20 Trees, 15 Depth.

• 225 Different Clases.

• 400 Images per class.

2222

Random Forest vs FERNSRandom Forest vs FERNS

Size of the training Set

• 20 Trees, 15 Depth.

• 225 Different Classes.

• [0.5-1.5] Scale Range.

2323

Random Forest vs FERNSRandom Forest vs FERNS

Number of different Classes.

• 20 Trees, 15 Depth.

• [0.8-1.2] Scale Range.

• 1500 Training images per class.

2424

Random Forest vs FERNSRandom Forest vs FERNS

Training time.

• 20 Trees, 15 Depth.

• 225 Different Classes.

• [0.5-1.5] Scale Range.

0

50

100

150

200

250

300

350 550 750 1000 1500 1800 2500 2800 3800

Tra

inin

go

Tim

e (s

)

2525

Pose EstimationPose Estimation

Homography Estimation

• Robust Estimation (RANSAC).

• Non-Linear Minimization (Levenberg-Marquardt).

•

2626

SummarySummary

1. Introduction.

2. Random Forest, FERNS.

3. Mixed/Augmented Reality Application.

4. Conclusions/Questions.

2727

European Project IMPROVE (Improving Display and Rendering Technology for Virtual Environments)

• Develop of new interaction metaphors.

• Develop of new Displays.

• Photo Realistic Rendering.

• Development of Markerless Tracking Techniques.

Augmented Reality ApplicationAugmented Reality Application

2828

Augmented Reality ApplicationAugmented Reality Application

Architectural ScenarioArchitectural Scenario Automotive ScenarioAutomotive Scenario

2929

Marker-Less tracking (InDoor Scenario)Marker-Less tracking (InDoor Scenario)

Feature Points TrackingFeature Points Tracking

Textured planeTextured plane

Image AugmentationImage Augmentation

Augmented Reality ApplicationAugmented Reality Application

3030

Marker-Less tracking (OutDoor Scenario)Marker-Less tracking (OutDoor Scenario)

Image AcquisitionImage Acquisition

Feature points TrackingFeature points Tracking Image AugmentationImage Augmentation

Augmented Reality ApplicationAugmented Reality Application

3131

Performance

• 20 Trees.

• Full Rotation Range and [0.8-1.2] Scale Range.

• 1000 images per Class.

• 250 Different Classes.

Augmented Reality ApplicationAugmented Reality Application

CPU Type

0

5

10

15

20

25

Intel Core2 Duo 1,6 Ghz AMD 2,1+Ghz Pentium4 1,4Ghz Intel Centrino 1Ghz

3232

SummarySummary

1. Introduction.

2. Random Forest, FERNS.

3. Mixed/Augmented Reality Application.

4. Conclusions/Questions.

3333

ConclusionsConclusions

Both Approaches are very Similar.The classifier is more sensitive to variations in scale.The classifier is robust against variations in object orientation.When the classifier converges, increase the number of trees does not improve accuracy.The node test can be selected at random.FERNS Requires more Memmory Than Random Forest.Training and classification Time is Higher in FERNS than in Random Forest.Random Forest are Faster than FERNS (without heuristics)FERNS Supports more classes than Random Forest.The Output of both classifiers must be filtered.The higher the classification accuracy, the better the performance of the tracking.

Thanks For ListeningThanks For Listening

Iñigo Barandiaran Martirena (ibarandiaran@vicomtech.org)

Researcher, VICOMTech

Paseo Mikeletegi 57

20009 San Sebastián

Tfno: +34 943 30 92 30 Fax : +34 943 30 93 93