WSCG2008, Plzen, 04-07, Febrary 2008 Comparative Evaluation of Random Forest and Fern classifiers...

WSCG2008, Plzen, 04-07, Febrary 2008WSCG2008, Plzen, 04-07, Febrary 2008

Comparative Evaluation of Random Forest and Fern Comparative Evaluation of Random Forest and Fern classifiers for Real-Time Feature Matchingclassifiers for Real-Time Feature Matching

I. Barandiaran1, C.Cottez1, C.Paloc1 , M.Graña2

1Departamento de Aplicaciones Biomédicas Asociación VICOMTech, San Sebastián, {ibarandiaran,ccottez,cpaloc}@vicomtech.org

2University of Basque Country Computer Science School, Pº. Manuel de Lardizabal, 1 20009, San Sebastián, Spain [email protected]

VISUAL INTERACTION AND COMMUNICATIONS TECHNOLOGIES

22

SummarySummary

1.Introduction.

2.Random Forest, FERNS

1.Mixed/Augmented Reality Application.

2.Conclusions/Questions.

33

IntroductionIntroduction

Motivation and objectives

• Motivated by the work of Vincent LePetit (CVLab).

• Real-Time Augmented Reality.

• Camera Pose Estimation.

• Markerless tracking.

• Model-based tracking.

• Tracking by detection.

• Test and compare different parameters.

• Scale.

• Size of the Training Set.

• Number of Classes.

• Training Time.

44

Augmented Reality Features:

• Mix Virtual and Real Objects..

• Real-Time.

• Portable Devices (Head Mounted Display, Tablet PC, PDA Device, Movil Phone..)


55

Problems:

• Rendering.

• Real-Time(Delay).

• Registration/Pose Estimation.


66

Non model-based Tracking

• No a priori knowledge of the object to be tracked.

• Updates/Propagates an estimation over time.

• Partial object occlusions.

• Tend to tracking reinitialization.

Model Based Tracking

• Some a priori knowledge is available.

• May not depend on the past.

• Frame by Frame estimation.

• Robust against partial object occlusion.

• Automatic tracking initialization.


77

1. Introduction

2. Random Forest, FERNS

3. Mixed/Augmented Reality

4. Conclusions/Questions

SummarySummary

88

Random Forest, FERNSRandom Forest, FERNS

Tracking of Planar Surfaces.The Classifiers are applied for interest point (feature) matching.Matched Points are used during camera pose estimation Process.

99


Building the training set.

• Frontal view of the object to be detected.

• Feature Point extraction FAST (Rosten06) and YAPE (CvLab).

• Sub-images (patches) are generated for each class.

Classes to Be recognized by the Classifier

1010


Building the training set.

• Generate Random Affine transformations.

• Generate new examples of each Class.

…..

Random Affine transformations

Training Set (examples)Training Set (examples)

1111

Random ForestRandom Forest

Multiclassifier based on Randomized Trees.Firstly introduced in 1997 handwritten recognition (Amit, Y.,German, D.)Developed by Leo Breiman (Medical Data Analisys).Recently Applied to tracking by detection (LePetit06).

Main Features

• Fast Training Step, and execution.

• Good Precision.

• Random selection of the independent variables (features).

• Random selection of Examples.

• Easy to Implement and paralelizable.

1212


Classifier Training.

• N Binary-Trees are Grown.

• Pixel intensity tests are executed in any non-terminal node.

• Pixels can be selected at Random.

• Posterior Distributions P(Y=c |T=Tk,n) are stored in leave nodes.

1313


Example Classification.

• Every example is dropped down the trees.

• The Example traverse the tree towards the leaf nodes.

Pixels to be tested

TT11 TT22 TTnn


1414


Combine Results

• The example labeling is obtained as a combination of partial results obtained by every tree in the forest.

T

k

nkitlabelclass tcYPExample

1_ |maxarg

1515

FERNSFERNS

Introduced in 2007 (Mustafa Özuysal).Multiclassifier.Applied to 3D keypoint recognition.Successfully applied to image recognition/retrieval (Zisserman07).

Main Features

• Non hierarchical structure.

• Semi Naive-Bayes Combination Strategy.

• Random selection of the independent variables (features).

• Random selection of Examples.

• Easy to Implement and paralelizable.

1616

FERNSFERNS

Semi-Naive Bayes Combination.

nii fffcCPc ,....,|maxarg 21

n

iinni fffP

cCPcCfffPfffcCP

,....,

)|,....,(,....,|

21

2121

otherwise

tppiff jjj 0

1 2,1, ini cCfffPc |,....,maxarg 21

N

j

ijin cCfPcCfffP1

21 ||,....,

M

j

ikin cCFPcCfffP1

21 ||,...., kF

1717

FERNSFERNS

Classifier Training

x x x

32 Possible Outputs

0 7

.

.

.

ik cCFP |Posterior Distributions

(Look-up Tables)

1818

FERNSFERNS

Classifier Training

1 1 0

0 0 0

0 1 06

0 0 1 1

2

3

Class 1Class 1

Class 2Class 2

Class 1Class 1

Class 2Class 2

..

..

..

..

..

..

..

..

..

..

Fern 1Fern 1

Fern 2Fern 2

Fern Fern nn

Posterior Distributions

(Look-up Tables)

..

..

..

..

1919

FERNSFERNS

Example Classification.

M

ikflabelclass cCFPExample1

_ |maxarg

2 6 1

Posterior Distributions

(Look-up Tables)

Fern 1Fern 1 Fern 2Fern 2 Fern 3Fern 3

2020

Random Forest vs FERNSRandom Forest vs FERNS

Rotation Range

• 20 Trees, 15 Depth.

• 225 Different Clases.

• 400 Images per class.

2121


Scale Range


• 225 Different Clases.

• 400 Images per class.

2222


Size of the training Set


• 225 Different Classes.

• [0.5-1.5] Scale Range.

2323


Number of different Classes.



• 1500 Training images per class.

2424


Training time.




0

50

100

150

200

250

300

350 550 750 1000 1500 1800 2500 2800 3800

Tra

inin

go

Tim

e (s

)

2525

Pose EstimationPose Estimation

Homography Estimation

• Robust Estimation (RANSAC).

• Non-Linear Minimization (Levenberg-Marquardt).

•

2626

SummarySummary

1. Introduction.

2. Random Forest, FERNS.

3. Mixed/Augmented Reality Application.

4. Conclusions/Questions.

2727

European Project IMPROVE (Improving Display and Rendering Technology for Virtual Environments)

• Develop of new interaction metaphors.

• Develop of new Displays.

• Photo Realistic Rendering.

• Development of Markerless Tracking Techniques.

Augmented Reality ApplicationAugmented Reality Application

2828


Architectural ScenarioArchitectural Scenario Automotive ScenarioAutomotive Scenario

2929

Marker-Less tracking (InDoor Scenario)Marker-Less tracking (InDoor Scenario)

Feature Points TrackingFeature Points Tracking

Textured planeTextured plane

Image AugmentationImage Augmentation


3030

Marker-Less tracking (OutDoor Scenario)Marker-Less tracking (OutDoor Scenario)

Image AcquisitionImage Acquisition

Feature points TrackingFeature points Tracking Image AugmentationImage Augmentation


3131

Performance

• 20 Trees.

• Full Rotation Range and [0.8-1.2] Scale Range.

• 1000 images per Class.



CPU Type

0

5

10

15

20

25

Intel Core2 Duo 1,6 Ghz AMD 2,1+Ghz Pentium4 1,4Ghz Intel Centrino 1Ghz

3232

SummarySummary

1. Introduction.

2. Random Forest, FERNS.

3. Mixed/Augmented Reality Application.

4. Conclusions/Questions.

3333

ConclusionsConclusions

Both Approaches are very Similar.The classifier is more sensitive to variations in scale.The classifier is robust against variations in object orientation.When the classifier converges, increase the number of trees does not improve accuracy.The node test can be selected at random.FERNS Requires more Memmory Than Random Forest.Training and classification Time is Higher in FERNS than in Random Forest.Random Forest are Faster than FERNS (without heuristics)FERNS Supports more classes than Random Forest.The Output of both classifiers must be filtered.The higher the classification accuracy, the better the performance of the tracking.

Thanks For ListeningThanks For Listening

Iñigo Barandiaran Martirena ([email protected])

Researcher, VICOMTech

Paseo Mikeletegi 57

20009 San Sebastián

Tfno: +34 943 30 92 30 Fax : +34 943 30 93 93

WSCG2008, Plzen, 04-07, Febrary 2008 Comparative Evaluation of Random Forest and Fern classifiers...

Documents

Transcript of WSCG2008, Plzen, 04-07, Febrary 2008 Comparative Evaluation of Random Forest and Fern classifiers...