Real-Time Recognition of Planar Targets on Mobile Devices A Framework for Fast and Robust Homography...

Real-Time Recognition of Planar Targets

on Mobile Devices A Framework for Fast and Robust Homography Estimation

School of Electrical & Computer Engineering

Faculty of Engineering

Hamid Bazargani

August 2014

/ 26

Outline

• Introduction and motivations

• System overview and background

• Methodology

– Robust target matching framework

– Robust model estimation

• Experimentation

• Conclusion


VIVA Lab

/ 26

Introduction


VIVA Lab

/ Objectives

• Real-time recognition of planar targets in live video.

• Recovering 3D position and orientation of the target w.r.t. the camera coordinates.

• Development of device-friendly algorithms for Augmented Reality (AR) applications.

/ 26

Motivations


VIVA Lab

/ How AR they working?

• The tremendous growth in popularity of mobile phones, has made AR one of the most innovative technologies of the decade.

• AR is something that will happen in our life because we have the tools to make it happen.

• The technology behind several of these thriving AR applications is based on computer vision and object recognition algorithms.

/ 26

Motivations


VIVA Lab

/ Application demo

AR will be, is now becoming the 8th mass medium. Tomi Ahonen

/ 26

System Overview


VIVA Lab

/ Flow diagram

• Detect stable features • FAST-9 feature detector

• Describe local features • Descriptor-base approach • Classification-based approach

• Homography estimation • RANSAC-based strategy

Robust estimation

Feature matching

Feature detection

/ 26

System Overview


VIVA Lab

/ Feature matching

1. Local descriptor-based approach

• Local information is abstracted by descriptors.

• SIFT, SURF, HoG, KAZE, etc.

2. Global classification-based approach

• Train a classifier through an offline process from view synthesis.

• A huge amount of computations is transferred to the offline process.

• Random trees, Ferns, HIP are some classification methods.

• Training BRIEF binary descriptors .

• Uses Hamming distance as a similarity score.

/ 26

Methodology


VIVA Lab

/ Robust Homography estimation

• Real-time estimation of 2D homographic transformation that describes the set of putative correspondences.

• Depending on the quality of target descriptors, the match set will be contaminated by more or less large number of false matches (outliers).

• A robust framework based on RANdom SAmpleing Concensus (RANSAC) is proposed to cope with this uncertainty.

/ 26

Methodology


VIVA Lab

/ What is RANSAC ?

• Randomly select a

minimal set.

• Hypothesize model (by

Gaussian elimination).

• Verify all points.

• Evaluate support.

• Update iteration

bound by the best

support achieved.

Hypothesis Verify

/ 26

Methodology


VIVA Lab

/ PROSAC non-uniform sampling

• Compared to typical RANSAC, PROSAC greatly reduces the number of attempts for the best support hypothesis.

• Uses non-uniform sampling strategy from a smaller subset of data points determined by a growth function g(t).

• Hamming distance is used to sort correspondences based on their similarity score.

/ 26

Methodology


VIVA Lab

/ Termination criterion 1/3

• Maximality: Taking desired level of confidence into account, one can determine maximum number of required iterations to guaranty this level of confidence.

• is level of confidence (typically set to 0.9-0.99). • is the lower bound for inliers’ rate.

0

/ 26

Methodology


VIVA Lab


• Non-Randomness : Non-randomness constraint is a statistical significance test that guaranties the goodness of a solution.

• The probability distribution of evaluating i outliers out of n points all

consistent with the sought model abides by the binomial distribution.

• is the probability of a random point evaluated as inlier given a degenerate model.

• is significance level (5-10 percent).

/ 26

Methodology


VIVA Lab


• Chi-squared approximation: Considering the randomness of a solution as a null hypothesis H0, p-value determines the significance of rejecting H0, given a specific level .

• From the central limit theorem, for sufficiently large n, binomial PDF is approximately a standard normal distribution with and . Thus:

/ 26

Methodology


VIVA Lab

/ Model verification 1/2

• For evaluating hypothesis, the standard RANSAC model verification can be further optimized by quick hypothesis filtering strategies such as the Td,d test or the Sequential Probability Ratio Test (SPRT) .

• The Td,d test

• In the 1st step, a small portion of N data points are verified from a randomly selected subset. The 2nd involves verification of remaining data only if the pre-test passes.

• It is proven that the optimal solution minimizing the average number of verified points leads us to the T1,1 (d = 1) .

/ 26

Methodology


VIVA Lab

/ Model verification 2/2

• The SPRT test:

• SPRT is a statistical test based on Wald’s theory for earlier decision taking on rejecting a model.

• Wald’s likelihood ratio:

• is approximately equal to inliers’ ratio.

• is Bernoulli distributed probability function.

/ 26

Results & Findings


VIVA Lab

/ H estimation performance

1x 1.3x 5.5x

/ 26

Results & Findings


VIVA Lab

/ GE vs. SVD

• GE is experimentally found that produces acceptable results.

• SVD is much slower but numerically more stable than GE. • Degeneracy test avoids geometrically unstable solutions.

• PROSAC complements GE .

Boyd's Law of Iteration:

Speed of iteration beats quality of iteration.

/ 26

Results & Findings


VIVA Lab

/ Recognition rate

/ 26

Results & Findings


VIVA Lab

/ Accuracy

/ 26

Results & Findings


VIVA Lab

/ Verifications

5-10x

/ 26

Results & Findings


VIVA Lab

/ Termination criteria

3-30x 3-30x

/ 26

Results & Findings


VIVA Lab

/Termination criteria speed

/ 26

Results & Findings


VIVA Lab

/ Overall Speed

95182482

76378479 65033425

103329328

17504443

3428717 366915 0

20000000

40000000

60000000

80000000

100000000

120000000

OpenCV 2.4 RANSAC

RANSAC + OpenCV 2.4

SVD

PROSAC + OpenCV 2.4

SVD

PROSAC + LAPAC

DGESVD

PROSAC + GE

PROSAC + optimized

GE

PROSAC + optimized

GE + SPRT

• Recognition process runs at around 40 fps on a smart-phone with Quad-core 1.4 GHz Cortex-A9 processor and at 80 fps on a machine equipped with a 2.26 GHz CPU.

Real-time: 75,333,333 cycles

/ 26

Conclusion


VIVA Lab

• Comprehensive study of several robust estimation algorithms in the context of homography estimation.

• Highly optimized framework for throughput leveraging the state-of-the-art methodologies. Available in :

• Reference C implementation. • SSE-optimized implementation. • https://github.com/VIVAlab/FastGEHomography

• Publication: • Hamid Bazargani, Olexa Bilaniuk, Robert Laganiere: Fast

Target Recognition on Mobile Devices: Revisiting Gaussian Elimination for the Estimation of Planar Homographies. In Computer Vision and Pattern Recognition Workshops, 2014.

/ Contributions

/ 26

Q & A


VIVA Lab

Thank you!

/ 26

Methodology


VIVA Lab


Real-Time Recognition of Planar Targets on Mobile Devices A Framework for Fast and Robust Homography...

Engineering

Transcript of Real-Time Recognition of Planar Targets on Mobile Devices A Framework for Fast and Robust Homography...