SVCL Automatic detection of object based Region-of-Interest for image compression Sunhyoung Han.
-
Upload
lauren-quinn -
Category
Documents
-
view
223 -
download
5
Transcript of SVCL Automatic detection of object based Region-of-Interest for image compression Sunhyoung Han.
![Page 1: SVCL Automatic detection of object based Region-of-Interest for image compression Sunhyoung Han.](https://reader035.fdocuments.in/reader035/viewer/2022062718/56649e695503460f94b65771/html5/thumbnails/1.jpg)
SVCL
Automatic detection of object based Region-of-Interest for image
compression
Sunhyoung Han
![Page 2: SVCL Automatic detection of object based Region-of-Interest for image compression Sunhyoung Han.](https://reader035.fdocuments.in/reader035/viewer/2022062718/56649e695503460f94b65771/html5/thumbnails/2.jpg)
SVCL
Transmission in erroneous channel
Basic Motivation
Spatially differentSuper resolution
ConstraintsLimited Resources &Channel Errors
![Page 3: SVCL Automatic detection of object based Region-of-Interest for image compression Sunhyoung Han.](https://reader035.fdocuments.in/reader035/viewer/2022062718/56649e695503460f94b65771/html5/thumbnails/3.jpg)
SVCL
Basic motivation
By having information about importance of regionsOne can wisely use the limited resources
![Page 4: SVCL Automatic detection of object based Region-of-Interest for image compression Sunhyoung Han.](https://reader035.fdocuments.in/reader035/viewer/2022062718/56649e695503460f94b65771/html5/thumbnails/4.jpg)
SVCL
User-adaptive Coder
v
• visual concepts of interest can
be anything
• main idea:
• let users define a universe
of objects of interest
• train saliency detector for
each object
• e.g. regions of “people”,
“the Capitol”, “trees”, etc.
![Page 5: SVCL Automatic detection of object based Region-of-Interest for image compression Sunhyoung Han.](https://reader035.fdocuments.in/reader035/viewer/2022062718/56649e695503460f94b65771/html5/thumbnails/5.jpg)
SVCL
User Adaptive Coder
query providedby user
traindetector
current trainingsets
![Page 6: SVCL Automatic detection of object based Region-of-Interest for image compression Sunhyoung Han.](https://reader035.fdocuments.in/reader035/viewer/2022062718/56649e695503460f94b65771/html5/thumbnails/6.jpg)
SVCL
User-adaptive coder
• user-adaptive coder:– detector should be generic enough to handle large
numbers of object categories
– training needs to be reasonably fast (including example preparation time)
“face” “lamp” “car”
![Page 7: SVCL Automatic detection of object based Region-of-Interest for image compression Sunhyoung Han.](https://reader035.fdocuments.in/reader035/viewer/2022062718/56649e695503460f94b65771/html5/thumbnails/7.jpg)
SVCL
User-adaptive coder• proposed detector
– top-down object detector (object category specified by user)
– focus on weak supervision instead of highly accurate localization
– composed of saliency detection and saliency validation
– discriminant saliency:
saliencyfilters
training
FIND best features
![Page 8: SVCL Automatic detection of object based Region-of-Interest for image compression Sunhyoung Han.](https://reader035.fdocuments.in/reader035/viewer/2022062718/56649e695503460f94b65771/html5/thumbnails/8.jpg)
SVCL
Discriminant Saliency
• start from a universe of classes (e.g. “faces”, “trees”,
“cars”, etc.)
• design a dictionary of features: e.g. linear
combinations of DCT coefficients at multiple scales
• salient features: those that best distinguish the object
class of interest from random background scenes.
• salient regions are the regions of the image where
these detectors have strong response
• see [Gao & Vasconcelos, NIPS, 2004].
![Page 9: SVCL Automatic detection of object based Region-of-Interest for image compression Sunhyoung Han.](https://reader035.fdocuments.in/reader035/viewer/2022062718/56649e695503460f94b65771/html5/thumbnails/9.jpg)
SVCL
Top-down Discriminant Saliency Model
Scale Selection
W j
WTA
Faces Discriminant Feature
Selection
Salient Features
Background
Saliency Map
Original Feature Set
Malik-Perona pre-attentive perception model
ZXIk kk
;maxarg*
![Page 10: SVCL Automatic detection of object based Region-of-Interest for image compression Sunhyoung Han.](https://reader035.fdocuments.in/reader035/viewer/2022062718/56649e695503460f94b65771/html5/thumbnails/10.jpg)
SVCL
• saliency detector
• salient point sali:– magnitude i
– location li– scale si
• saliency map approximated by a Gaussian mixture
Saliency representation
image saliency map salient points
Probability map
![Page 11: SVCL Automatic detection of object based Region-of-Interest for image compression Sunhyoung Han.](https://reader035.fdocuments.in/reader035/viewer/2022062718/56649e695503460f94b65771/html5/thumbnails/11.jpg)
SVCL
Saliency validation• saliency detection:
– due to limited feature dictionary and/or limited training set
– coarse detection of object class of interest
• need to eliminate false positives
• saliency validation:– geometric consistency– reject salient points whose
spatial configuration is inconsistent with training examples
original Image
saliency map for ‘street sign’
example of saliency map
![Page 12: SVCL Automatic detection of object based Region-of-Interest for image compression Sunhyoung Han.](https://reader035.fdocuments.in/reader035/viewer/2022062718/56649e695503460f94b65771/html5/thumbnails/12.jpg)
SVCL
Saliency validation• learning a geometric model of salient point con-
figuration• two components:
- image alignment
• model:- classify pointsinto
• true positives
- configuration model
• false positives- model eachas Gaussian
![Page 13: SVCL Automatic detection of object based Region-of-Interest for image compression Sunhyoung Han.](https://reader035.fdocuments.in/reader035/viewer/2022062718/56649e695503460f94b65771/html5/thumbnails/13.jpg)
SVCL
Saliency validation
• model: two classes of points Y={0,1}– Y=1 true positive– Y=0 false positive
• saliency map: mixture of true and false positive saliency distributions
• each distribution approximated by aGaussian
![Page 14: SVCL Automatic detection of object based Region-of-Interest for image compression Sunhyoung Han.](https://reader035.fdocuments.in/reader035/viewer/2022062718/56649e695503460f94b65771/html5/thumbnails/14.jpg)
SVCL
• this is a two class clustering problem– can be solved by expectation-maximization
• graphical model
• non-standard issues– we start from distributions, not points– alignment does not depend on false
negatives
Saliency validation
E-stepM-step
Y X
L~uniform
Y~Bernoulli (1)
C|Y=i~multinomial (i)
X|Y=i,L=l,S=s,~G(x, l-, )
L,S
C
![Page 15: SVCL Automatic detection of object based Region-of-Interest for image compression Sunhyoung Han.](https://reader035.fdocuments.in/reader035/viewer/2022062718/56649e695503460f94b65771/html5/thumbnails/15.jpg)
SVCL
Saliency ValidationFor K training examples (# of saliency point is Nk for kth example) Missing data Y= j,
j {1,0}∈ Parameters j (probability for class j)
∑j (Covariance for class )
k (displacement for kth example)
For robust update
DERIVATION DETAILS
![Page 16: SVCL Automatic detection of object based Region-of-Interest for image compression Sunhyoung Han.](https://reader035.fdocuments.in/reader035/viewer/2022062718/56649e695503460f94b65771/html5/thumbnails/16.jpg)
SVCL
Saliency Validation• visualization of EM algorithm
Saliency detection result
Init saliency points overlapped over 40 samples
Visualized variance ∑1Overlapped points classified as ‘’object’’
Overlapped points classified as ‘’noise’’
Visualized variance ∑0
![Page 17: SVCL Automatic detection of object based Region-of-Interest for image compression Sunhyoung Han.](https://reader035.fdocuments.in/reader035/viewer/2022062718/56649e695503460f94b65771/html5/thumbnails/17.jpg)
SVCL
Saliency Validation
• examples of classified Points
• in summary, during training we learn– discriminant features– The “right” configuration of salient points
Examples of classified saliency points White if hij1>hij
0 Black otherwise
![Page 18: SVCL Automatic detection of object based Region-of-Interest for image compression Sunhyoung Han.](https://reader035.fdocuments.in/reader035/viewer/2022062718/56649e695503460f94b65771/html5/thumbnails/18.jpg)
SVCL
Region of interest detection
• find image window that best matches the learned configuration
• mathematically: - find location p where the posterior probability of the object class is the largest
![Page 19: SVCL Automatic detection of object based Region-of-Interest for image compression Sunhyoung Han.](https://reader035.fdocuments.in/reader035/viewer/2022062718/56649e695503460f94b65771/html5/thumbnails/19.jpg)
SVCL
Region of interest detection• by Bayes rule
– Posterior Likelihood x Prior
– likelihood is given by matching saliencies within the window
& the model
- prior measuresthe saliency massinside window ?
?
likelihoodPrior
![Page 20: SVCL Automatic detection of object based Region-of-Interest for image compression Sunhyoung Han.](https://reader035.fdocuments.in/reader035/viewer/2022062718/56649e695503460f94b65771/html5/thumbnails/20.jpg)
SVCL
Region of Interest Detection
• given the model– the likelihood, under it, of
a set of points drawn from the observed saliency distribution is
– and the optimal location is given by
Prior for location PWith saliency detector
DERIVATION DETAILS
Measure configuration matching
![Page 21: SVCL Automatic detection of object based Region-of-Interest for image compression Sunhyoung Han.](https://reader035.fdocuments.in/reader035/viewer/2022062718/56649e695503460f94b65771/html5/thumbnails/21.jpg)
SVCL
2. Determine scale(shape) of ROI mask Observation(∑*) from data and
prior(∑1) from training data are used
3. Thresholds PY|X,P(1|x,p*) to get binary ROI mask
Region of Interest Detection
** Once the center point is known the assignment of each point is given by
The observed configuration for Y=1 isx
∑1∑*
![Page 22: SVCL Automatic detection of object based Region-of-Interest for image compression Sunhyoung Han.](https://reader035.fdocuments.in/reader035/viewer/2022062718/56649e695503460f94b65771/html5/thumbnails/22.jpg)
SVCL
Region of Interest Detection
Saliency detection (for statue of liberty)Probability map (saliency only)
Probability map (with configuration info.) ROI mask
• Example of ROI Detection
![Page 23: SVCL Automatic detection of object based Region-of-Interest for image compression Sunhyoung Han.](https://reader035.fdocuments.in/reader035/viewer/2022062718/56649e695503460f94b65771/html5/thumbnails/23.jpg)
SVCL
Evaluation
• Using CalTech “Face” database &UIUC “Car side” database
• Evaluate robustness of learning– Dedicated Training set vs. Web Training set
• Evaluation Metric– ROC area curve
– PSNR gain for ROI coding vs. normal coding
Number of positive example: 550
Number of positive example: 100
![Page 24: SVCL Automatic detection of object based Region-of-Interest for image compression Sunhyoung Han.](https://reader035.fdocuments.in/reader035/viewer/2022062718/56649e695503460f94b65771/html5/thumbnails/24.jpg)
SVCL
Evaluation
• ROC area curve
False Positive
Tru
e P
ositi
ve
False PositiveT
rue
Pos
itive
“Car” “Face”
![Page 25: SVCL Automatic detection of object based Region-of-Interest for image compression Sunhyoung Han.](https://reader035.fdocuments.in/reader035/viewer/2022062718/56649e695503460f94b65771/html5/thumbnails/25.jpg)
SVCL
Evaluation
• PSNR performance comparison
“Car” “Face”Bit Per Pixel
PS
NR
Bit Per PixelP
SN
R
14.3% bits can be saved even with web train uniform casefor the same image quality
![Page 26: SVCL Automatic detection of object based Region-of-Interest for image compression Sunhyoung Han.](https://reader035.fdocuments.in/reader035/viewer/2022062718/56649e695503460f94b65771/html5/thumbnails/26.jpg)
SVCL
Result Examples
![Page 27: SVCL Automatic detection of object based Region-of-Interest for image compression Sunhyoung Han.](https://reader035.fdocuments.in/reader035/viewer/2022062718/56649e695503460f94b65771/html5/thumbnails/27.jpg)
SVCL
ResultComparison of needed bits to get the same PSNR (30 dB) for ROI
Maximally, ¼ bits are enough to get the same quality for ROI area
![Page 28: SVCL Automatic detection of object based Region-of-Interest for image compression Sunhyoung Han.](https://reader035.fdocuments.in/reader035/viewer/2022062718/56649e695503460f94b65771/html5/thumbnails/28.jpg)
SVCL
Result Examples
Normal coding ROI coding
![Page 29: SVCL Automatic detection of object based Region-of-Interest for image compression Sunhyoung Han.](https://reader035.fdocuments.in/reader035/viewer/2022062718/56649e695503460f94b65771/html5/thumbnails/29.jpg)
SVCL
![Page 30: SVCL Automatic detection of object based Region-of-Interest for image compression Sunhyoung Han.](https://reader035.fdocuments.in/reader035/viewer/2022062718/56649e695503460f94b65771/html5/thumbnails/30.jpg)
SVCL
EM derivation• Want to fit lower level observation
• For a virtual sample X = {Xik|i=1, …, Nk and k=1, …, K} with the size of Mik=ik*N, likelihood becomes
• For complete set the log likelihood becomes
![Page 31: SVCL Automatic detection of object based Region-of-Interest for image compression Sunhyoung Han.](https://reader035.fdocuments.in/reader035/viewer/2022062718/56649e695503460f94b65771/html5/thumbnails/31.jpg)
SVCL
EM derivation
Maximization in the m-step is carried out by maximizing the Lagrangian
![Page 32: SVCL Automatic detection of object based Region-of-Interest for image compression Sunhyoung Han.](https://reader035.fdocuments.in/reader035/viewer/2022062718/56649e695503460f94b65771/html5/thumbnails/32.jpg)
SVCL
ROI Detection
For one sample point x1
For samples having distribution of
![Page 33: SVCL Automatic detection of object based Region-of-Interest for image compression Sunhyoung Han.](https://reader035.fdocuments.in/reader035/viewer/2022062718/56649e695503460f94b65771/html5/thumbnails/33.jpg)
SVCL
ROI Detection
Therefore,