Learning a Fast Emulator of a Binary Decision Process

Center for Machine Perception

Czech Technical University, Prague

ACCV 2007, Tokyo, Japan

Jan Šochman and Jiří Matas

2

Importance of Classification Speed

Time-to-decision vs. precision trade-off is inherent in many detection, recognition and matching problems in computer vision

Often the trade-off is not explicitly stated in the problem formulation, but decision time clearly influences impact of a method

Example: face detection

• Viola-Jones (2001) – real-time performance- 2500 citations

• Schneiderman-Kanade (1998) - smaller error rates, but 1000x slower- 250 citations

Time is implicitly considered as an important characteristic of detection and recognition algorithms

3

Fast Emulation of A Decision Process

The Idea Given a black box algorithm A performing some useful binary decision

task Train a sequential classifier S to (approximately) emulate detection

performance of algorithm A while minimizing time-to-decision Allow user to control quality of the approximation

Contribution A general framework for speeding up existing algorithms by a sequential

classifier learned by the WaldBoost algorithm [1]

Demonstrated on two interest point detectors

Advantages Instead of spending man-months on code optimization, choose relevant

feature class and train sequential classifier S Your (slow) Matlab code can be speeded up this way!

[1] J. Šochman and J. Matas. Waldboost – Learning For Time Constrained Sequential Detection. CVPR 2005

4

Black-box Generated Training Set

Emulator approximates behavior of the black-box algorithm The black-box algorithm can potentially provide almost unlimited

number of training samples

• Efficiency of training is important

• Suitable for incremental or online methods

5

WaldBoost Optimization Task

Basic notions:

Sequential strategy S is characterized by:

Problem formulation:

6

WaldBoost Sequential Classifier Training

Combines AdaBoost training (provides measurements) with Wald’s sequential decision making theory (for sequential decisions)

Sequential WaldBoost classifier

Set of weak classifiers (features) and thresholds are found during training

7

Emulation of Similarity-Invariant Regions

Motivation Hessian-Laplace is close to state of the art Kadir’s detector very slow (100x slower than Difference of Gaussians) Standard testing protocol exists Executables of both methods available at robots.ox.ac.uk Both detectors are scale-invariant which is easily implemented via a

scanning window + a sequential test

Implementation Choice of : various filters computable with integral images - difference

of rectangular regions, variance in a window Positive samples: Patches twice the size of original interest point scale

[1] K. Mikolajczyk, C. Schmid. Scale and Affine Invariant Interest Point Detectors. ICCV 2004.[2] T. Kadir, M. Brady. Saliency, Scale and Image Description. IJCV 2001.

The approach tested on Hessian-Laplace [1] and Kadir-Brady salient [2] interest point detectors

8

Results: Hessian-Laplace – boat sequence

Repeatability comparable (left graph) Matching score almost identical (middle graph) Higher number of correspondences and correct matches in WaldBoost. Speed comparison (850x680 image)

1 1.5 2 2.50

10

20

30

40

50

60

70

80

90

100

scale change

rep

ea

tab

ility

%

WaldBoost

Hessian-Laplace

Original 1.3 sec

WaldBoost 1.3 sec

1 1.5 2 2.50

500

1000

1500

2000

2500

3000

3500

4000

4500

5000

scale change

# c

orr

es

po

nd

en

ce

s

WaldBoost

Hessian-Laplace

1 1.5 2 2.50

10

20

30

40

50

60

70

80

90

100

scale change

ma

tch

ing

sc

ore

WaldBoost

Hessian-Laplace

9

Results: Kadir-Brady – east-south sequence

Repeatability slightly higher (left graph) Matching score slightly higher (middle graph) Higher number of correspondences and correct matches in WaldBoost. Speed comparison (850x680 image)

Original 1 min 44 sec

WaldBoost 1.4 sec

1 1.5 2 2.50

10

20

30

40

50

60

70

80

90

100

scale change

rep

ea

tab

ility

%

WaldBoost

Kadir-Brady

1 1.5 2 2.50

10

20

30

40

50

60

70

80

90

100

scale change

ma

tch

ing

sc

ore

WaldBoost

Kadir-Brady

1 1.5 2 2.50

500

1000

1500

2000

2500

3000

scale change

# c

orr

es

po

nd

en

ce

s

WaldBoost

Kadir-Brady

10

Approximation Quality – Hesian-Laplace

Yellow circlesYellow circles: repeated detections (85% coverage)

Red circlesRed circles: original detections not found by the WaldBoost detector

11

Approximation Quality – Kadir-Brady

Yellow circlesYellow circles: repeated detections (96% coverage)

Red circlesRed circles: original detections not found by the WaldBoost detector

12

Conclusions and Future Work

A general framework for speeding up existing binary decision algorithms has been presented

To optimize the emulator’s time-to-decision a WaldBoost sequential classifier was trained

The approach was demonstrated on (but is not limited to) two interest point detectors emulation

Future work Precise localization of the detections by interpolation on the grid Using real value output of the black-box algorithm (“sequential

regression”) To emulate or to be repeatable?

13

Conclusions and Future Work

A general framework for speeding up existing binary decision algorithms has been presented

To optimize the emulator’s time-to-decision a WaldBoost sequential classifier was trained

The approach was demonstrated (but is not limited to) on two interest point detectors

Future work Precise localization of the detections by interpolation on the grid Using real value output of the black-box algorithm (“sequential

regression”) To emulate or to be repeatable?

Thank you for your attention