k1 Faster R-CNN Features for Instance Search · 2016 Acknowledgements Paris Buildings TRECVID...
Transcript of k1 Faster R-CNN Features for Instance Search · 2016 Acknowledgements Paris Buildings TRECVID...
Faster R-CNN Features for Instance SearchAmaia Salvador, Xavier Giró, Ferran Marqués, Shin’ichi Satoh
[1] Ren, S., He, K., Girshick, R., & Sun, J. Faster R-CNN: Towards real-time object detection with region proposal networks. NIPS 2015
Motivation
● Both global and local features can be extracted in
a single forward pass from a pre-trained CNN for
object detection.
● Suitable for fast retrieval and spatial reranking.
This work has been developed in the framework of the project BigGraph TEC2013-43935-R, funded by the
Spanish Ministerio de Economía y Competitividad and the European Regional Development Fund (ERDF).
ReferencesWiCV 2016
Acknowledgements
Paris Buildings
TRECVID Instance Search 2013(subset of 23k frames)
Oxford Buildings
Faster R-CNN
Image- and region-wise descriptors are extracted
from the pre-trained Faster R-CNN model [1].
Spatial Reranking
Two different strategies for spatial reranking are explored, using pre-trained and fine-tuned Faster R-CNN models:
Fine-tuning for Query Objects
We fine tune Faster R-CNN to detect query objects, using
query images as training data. We train two models, one
updating all layers (ft#2), and one updating only fully
connected ones (ft#1).
Class-Specific Spatial Reranking (CS-SR)
Class-Agnostic Spatial Reranking (CA-SR)
Results
Spatial reranking improves the retrieval
baseline, and provides object localization:
RepresentationQuery image Matching Ranklist
Image Database
v = (v1
, …, vn)
v1
= (v11
, …, v1n
)
vk = (v
k1, …, v
kn)
Sim
ilari
ty M
etri
c
...
...
Spat
ial R
eran
kin
g
Find code, slides & video at: http://imatge-upc.github.io/retrieval-2016-deepvision
Datasets