Locality-constrained Linear Coding for Image Classification Presenter : Han-Mu Park.

Locality-constrained Linear Coding for Image Classification

Presenter : Han-Mu Park

2

Introduction Coding methods Proposed method Experimental results Conclusion References

Contents

Locality-constrained Linear Coding for Image Classification, CVPR 2010

3

Bag-of-Words (BoW) model– An image is represented as a collection of visual words.– Generally, to represent the collection, histogram of

words form is used.

Introduction


4

General Spatial Pyramid Matching frame-works– Feature extraction

• SIFT• HOG• etc

– Coding• Vector Quantization• Sparse coding• etc

– Pooling• Max pooling• Sum pooling

Introduction


Spatial Pyramid Match-ing framework [J.Wang2010]

5

Spatial Pyramid Match-ing framework [J.Wang2010]

General Spatial Pyramid Matching frame-works

Introduction


6

Coding methods– Vector quantization (VQ)– Sparce coding (SC)– Locality-constrained Linear Coding (LLC)

Coding methods


[J.Wang2010]

7

Vector quantization (VQ)– Hard quantization method

– A set of -dimensional local descriptors •

– Codebook with entries

– Objective function

• Where is the set of codes for X

Coding methods


8

Sparse coding (SC)– Soft quantization method– Relaxed the cardinality constraint


– The roles of sparsity regularization term• Because the codebook is usually over-com-

plete , it is necessary to ensure that the under-determined system has a unique solution.

• Sparsity allows the learned representation to capture salient patterns of local descriptors.

• The sparse coding can achieve much less quantization error than VQ.

Coding methods


9

Locality-constrained Linear Coding (LLC)– Replaced the sparsity regularization term

with new constraint.


• : the element-wise multiplication

Where

Proposed method


10

Properties of LLC1. Better reconstruction

• Because LLC represents each descriptor by using multiple weighted bases (codewords), it has less reconstruction error than VQ.

2. Local smooth sparsity• Because the regularization term of in SC is not smooth, therefore,

SC loses correlations between codes.

3. Analytical solution• The solution of LLC can be derived analytically by

Where

Proposed method


11

Approximated LLC for fast encoding– The LLC selects the local bases for each descriptor to

form a local coordinate system.– To speedup the encoding process, authors used nearest

neighbors of as the local bases , and solve a much smaller linear system to get the codes

– The reduced computation complexity• , where

Proposed method


12

Codebook optimization– To improve the accuracy, authors trained the codebook

to optimize for LLC codes.– The optimal codebook can be obtained by

– This can be solved by using Coordinate Descent method.– However, because the number of training descriptors is

usually very large, the huge memory space is needed to solve that problem.

Proposed method


13

Incremental codebook optimiza-tion

– First, initialize by using K-means clustering.

– Then loop through all the training descriptors to update incrementally.

– In each iteration, we take in a single (or a small set of) examples , and solve original objective function to obtain the corresponding LLC codes.

Proposed method


[J.Wang2010]

14

Incremental codebook optimiza-tion

– Then select bases whose corre-sponding weights are larger than predefined threshold, and refit without the locality constraint.

– The obtained code is used to update the basis in a gradient descent fash-ion.

Proposed method


[J.Wang2010]

15

Performance of codebook

Experimental results


16

Performance under different neighbors



17

Results using Pascal VOC 2007



18

Contribution– In this paper, the Locality-constrained Linear Coding

(LCC) method is proposed• Better reconstruction• Local smooth sparsity• Analytical solution

– For speedup, K-nearest neighbors algorithm is used.– To optimize the accuracy, incremental codebook learning

is proposed for LCC.

Conclusion


19

[1] J. Wang, J. Yang, K. Yu, F. Lv, T. Huang, Y. Gong, “Locality-constrained Linear Coding for Image Classification,” CVPR 2010.

References


20

Thank you!

Locality-constrained Linear Coding for Image Classification Presenter : Han-Mu Park.

Documents

Transcript of Locality-constrained Linear Coding for Image Classification Presenter : Han-Mu Park.