Post on 28-Dec-2015
SVCL
Automatic detection of object based Region-of-Interest for image
compression
Sunhyoung Han
SVCL
Transmission in erroneous channel
Basic Motivation
Spatially differentSuper resolution
ConstraintsLimited Resources &Channel Errors
SVCL
Basic motivation
By having information about importance of regionsOne can wisely use the limited resources
SVCL
User-adaptive Coder
v
• visual concepts of interest can
be anything
• main idea:
• let users define a universe
of objects of interest
• train saliency detector for
each object
• e.g. regions of “people”,
“the Capitol”, “trees”, etc.
SVCL
User Adaptive Coder
query providedby user
traindetector
current trainingsets
SVCL
User-adaptive coder
• user-adaptive coder:– detector should be generic enough to handle large
numbers of object categories
– training needs to be reasonably fast (including example preparation time)
“face” “lamp” “car”
SVCL
User-adaptive coder• proposed detector
– top-down object detector (object category specified by user)
– focus on weak supervision instead of highly accurate localization
– composed of saliency detection and saliency validation
– discriminant saliency:
saliencyfilters
training
FIND best features
SVCL
Discriminant Saliency
• start from a universe of classes (e.g. “faces”, “trees”,
“cars”, etc.)
• design a dictionary of features: e.g. linear
combinations of DCT coefficients at multiple scales
• salient features: those that best distinguish the object
class of interest from random background scenes.
• salient regions are the regions of the image where
these detectors have strong response
• see [Gao & Vasconcelos, NIPS, 2004].
SVCL
Top-down Discriminant Saliency Model
Scale Selection
W j
WTA
Faces Discriminant Feature
Selection
Salient Features
Background
Saliency Map
Original Feature Set
Malik-Perona pre-attentive perception model
ZXIk kk
;maxarg*
SVCL
• saliency detector
• salient point sali:– magnitude i
– location li– scale si
• saliency map approximated by a Gaussian mixture
Saliency representation
image saliency map salient points
Probability map
SVCL
Saliency validation• saliency detection:
– due to limited feature dictionary and/or limited training set
– coarse detection of object class of interest
• need to eliminate false positives
• saliency validation:– geometric consistency– reject salient points whose
spatial configuration is inconsistent with training examples
original Image
saliency map for ‘street sign’
example of saliency map
SVCL
Saliency validation• learning a geometric model of salient point con-
figuration• two components:
- image alignment
• model:- classify pointsinto
• true positives
- configuration model
• false positives- model eachas Gaussian
SVCL
Saliency validation
• model: two classes of points Y={0,1}– Y=1 true positive– Y=0 false positive
• saliency map: mixture of true and false positive saliency distributions
• each distribution approximated by aGaussian
SVCL
• this is a two class clustering problem– can be solved by expectation-maximization
• graphical model
• non-standard issues– we start from distributions, not points– alignment does not depend on false
negatives
Saliency validation
E-stepM-step
Y X
L~uniform
Y~Bernoulli (1)
C|Y=i~multinomial (i)
X|Y=i,L=l,S=s,~G(x, l-, )
L,S
C
SVCL
Saliency ValidationFor K training examples (# of saliency point is Nk for kth example) Missing data Y= j,
j {1,0}∈ Parameters j (probability for class j)
∑j (Covariance for class )
k (displacement for kth example)
For robust update
DERIVATION DETAILS
SVCL
Saliency Validation• visualization of EM algorithm
Saliency detection result
Init saliency points overlapped over 40 samples
Visualized variance ∑1Overlapped points classified as ‘’object’’
Overlapped points classified as ‘’noise’’
Visualized variance ∑0
SVCL
Saliency Validation
• examples of classified Points
• in summary, during training we learn– discriminant features– The “right” configuration of salient points
Examples of classified saliency points White if hij1>hij
0 Black otherwise
SVCL
Region of interest detection
• find image window that best matches the learned configuration
• mathematically: - find location p where the posterior probability of the object class is the largest
SVCL
Region of interest detection• by Bayes rule
– Posterior Likelihood x Prior
– likelihood is given by matching saliencies within the window
& the model
- prior measuresthe saliency massinside window ?
?
likelihoodPrior
SVCL
Region of Interest Detection
• given the model– the likelihood, under it, of
a set of points drawn from the observed saliency distribution is
– and the optimal location is given by
Prior for location PWith saliency detector
DERIVATION DETAILS
Measure configuration matching
SVCL
2. Determine scale(shape) of ROI mask Observation(∑*) from data and
prior(∑1) from training data are used
3. Thresholds PY|X,P(1|x,p*) to get binary ROI mask
Region of Interest Detection
** Once the center point is known the assignment of each point is given by
The observed configuration for Y=1 isx
∑1∑*
SVCL
Region of Interest Detection
Saliency detection (for statue of liberty)Probability map (saliency only)
Probability map (with configuration info.) ROI mask
• Example of ROI Detection
SVCL
Evaluation
• Using CalTech “Face” database &UIUC “Car side” database
• Evaluate robustness of learning– Dedicated Training set vs. Web Training set
• Evaluation Metric– ROC area curve
– PSNR gain for ROI coding vs. normal coding
Number of positive example: 550
Number of positive example: 100
SVCL
Evaluation
• ROC area curve
False Positive
Tru
e P
ositi
ve
False PositiveT
rue
Pos
itive
“Car” “Face”
SVCL
Evaluation
• PSNR performance comparison
“Car” “Face”Bit Per Pixel
PS
NR
Bit Per PixelP
SN
R
14.3% bits can be saved even with web train uniform casefor the same image quality
SVCL
Result Examples
SVCL
ResultComparison of needed bits to get the same PSNR (30 dB) for ROI
Maximally, ¼ bits are enough to get the same quality for ROI area
SVCL
Result Examples
Normal coding ROI coding
SVCL
SVCL
EM derivation• Want to fit lower level observation
• For a virtual sample X = {Xik|i=1, …, Nk and k=1, …, K} with the size of Mik=ik*N, likelihood becomes
• For complete set the log likelihood becomes
SVCL
EM derivation
Maximization in the m-step is carried out by maximizing the Lagrangian
SVCL
ROI Detection
For one sample point x1
For samples having distribution of
SVCL
ROI Detection
Therefore,