Palette Power: Enabling Visual Search through Colors
-
Upload
wei-di -
Category
Technology
-
view
33 -
download
2
Transcript of Palette Power: Enabling Visual Search through Colors
Palette Power: Enabling Visual Search through Colors
Aug 14, 2013
eBay Research Labs
http://labs.ebay.com
Changing Landscape of Search
2
Visual Search for Fashion
eBay
Inventory
Given an item image,
find similar eBay inventory
Query Image
Similar Items
3
Item Similarity
Fre
quency
Color Distributions
Dots Floral Checks
Patterns & Textures
Styles
4
Approach Overview
Large Image Data Speed Requirements
Take Advantage of Context
Our Approach – The Power of Color Distributions
Color Spaces
𝑑 = 𝑓(𝑖1, 𝑖2) Distance Functions
5
6
Challenges (1/3)
Low contrast between background and foreground
7
Challenges (2/3)
Background Clutter
8
Challenges (3/3)
Lighting Variation
9
Insights From Data
Object localization using spatial priors
Choosing the right color space
Why Object Localization?
10
Cluttered background degrades performance.
State-of-the-art segmentation too expensive.
Need a fast and reliable solution!
Spatial Prior to the rescue!
Understanding Spatial Prior
11
12
Choosing Best Color Space
13
Handling Color Confusion
14
Generating Color Histogram
Faster Lookup via k-center
15
Scaling via backend clustering/indexing.
Potential for semantic/intent diversification - e.g. query t-shirt image where you like style but not colors
Achieves 60x speedup close to 70% overlap!
Median speed-up Median %-overlap
16
Architecture
17
Experiment I – Fashion Dataset
Categories: Women’s Dresses, Tops & Blouses, Coats & Jackets,
Skirts, Sweaters and T-Shirts
Data Sets: 1600 Queries & 1 Million Inventory images, 15 users for
30 days
18
Results - Solid Queries
Results - Pattern Queries
19
Experiment II – Generic ecommerce Dataset
Categories: Toys, Sports, Camera
Data Sets: Query & Inventory sets for each category
Ground Truth: ~15 per query
20
Example Inventory Images
21
Toys
Sports
Camera
22
MAP Performance
23
Experiment III – INRIA Holidays Dataset
Categories: Personal Holidays Photos
Data Sets: 500 Queries (1 per group) & 1491 Inventory Images
Ground Truth: Human Annotations
24
MAP Performance
25
Computational Costs
Feature Extraction Time 10 ms
Retrieval Time 80 ms
Feature Vector Size 196 Bytes
Memory Required 190 MB
Machine Stats: 24 GB RAM, 2.53GHz
Index Size: 1M+
26
Summary
Color a fundamental cue
Spatial Prior can eliminate need for expensive
background removal
Future work to focus on efficient descriptors
27
Questions?