600 Powerpoint business diagrams Powerpoint templates Powerpoint maps Powerpoint shapes
PowerPoint Presentation - ICSIstellayu/publication/doc/2001...Title PowerPoint Presentation Author...
Transcript of PowerPoint Presentation - ICSIstellayu/publication/doc/2001...Title PowerPoint Presentation Author...
-
1
Grouping with Bias
Stella X. Yu 1,2 Jianbo Shi 1
Robotics Institute1
Carnegie Mellon UniversityCenter for the Neural Basis of Cognition2
-
2
What Is It About?
Incorporating prior knowledge into groupingUnitary generative modelGlobal configurations: partially labelled data and object modelsAttention
ComputationEfficient solution in a graph partitioning framework
GoalsBridge the gap between generative models and discriminative modelsBridge the gap between formulation and computation
-
3
Grouping with Markov Random Fields
MRF: data structure = data generation model + segmentation model
)(log)|(log);(min XpXfpfXE −−=
Segmentation is to find a partitioning of an image, with generative models explaining each partition.
Generative models constrain the continuous observation data, the segmentation model constrains the discrete states.
The solution sought is the most probable state, or the state of the lowest energy.
f1 f2Image as
observation f
Texture models
X1 X2Grouping as
state X
[Geman & Geman, 84, …]
-
4
Grouping with Spectral Graph Partitioning
SGP: data structure = a weighted graph, weights describing data affinity
)deg()deg(),(
),(min21
2121 VV
VVcutVVNcut
⋅= ∑∑∈ ∈
=1 2
),(),( 21Vp Vq
qpWVVcut
∑∑∈ ∈
=1
),()deg( 1Vp Vq
qpWV
Segmentation is to find a node partitioning of a relational graph, with minimum total cut-off affinity.
Discriminative models are used to evaluate the weights between nodes.
The solution sought is the cuts of the minimum energy.
[Shi & Malik, 97; Perona & Freeman, 98; Malik et al, 01, …]
-
5
Solving MRF by Graph Partitioning
Some simple MRF models can be translated into graph partitioning
∑∑ ∑ +=∈ p
pppp pNq
qpqp fXUXXWfXE ),(),();(min)(
,
Unitary measuresBinary relationships
L1 L2
[Greig et al, 89; Ferrari et al, 95; Boykov et al, 98; Roy & Cox, 98; Ishikawa & Geiger, 98, …]
-
6
Comparison of Two Approaches
Formulation ComputationPros \ Cons
Simulation: e.g. Gibbs samplerParameter estimation is hardDifficult to compute probabilityConvergence is very slowOnly local optimum
Generative modelsBayesian interpretation General local interaction
Sensitive to model mismatch
MarkovRandom
Fields
Spectral decompositionFast and robustGlobal optimum
Discriminative modelsNo models requiredLack prior to guide grouping
GraphPartitioning
-
7
Prior Knowledge in Grouping
Global Configuration ConstraintsLocal Constraints
Unitary generative models
Object models:What to look for
Attention:Where to look for
Red foreground Partial grouping Spatial attention
How to encode them in discriminative models, e.g. SGP?
-
8
Review: Segmentation on Relational Graphs
G = (V, E, A, R)V: each node denotes a pixelE: each edge denotes a pixel-pixel relationship A: each weight measures pairwise similarityR: each weight measures pairwise dissimilarity
Dual criteria on dual measuresMaximize within-group A and between-group RMinimize between-group A and within-group R
Segmentation = node partitioningbreak V into disjoint sets V1 , V2 ; so that cut-off attraction is small cut-off repulsion is large
-
9
Review: Energy Function Formulation
∉∈
=l
ll Vu
VuuX
,0,1
)( Group indicators
Weight matrixRDRAW +−= RDR +−
ADD = RD+ Degree matrix
)deg()deg(,)1( 121 V
VXXy =−−= ααα Change of variables
DyyWyy
DXXWXX
XXNassoc TT
t tTt
tT
t ==∑=
2
121 ),(
Energy function as aRayleigh quotient
yDyWDyyWyy
T
T
1max λ=⇒ Eigenvector as solution
-
10
Review: Eigenvector as a Solution
The derivation holds so long as 121 =+ XX
ααα −=−−= 121)1( XXXy
The eigenvector solution is a linear transformation, scaled and offsetversion of the probabilistic membership indicator for one group.
If y is well separated, then two groups are well defined;otherwise, the separation is ambiguous
Solution ywell separated
Solution yambiguous
stimulus
-
11
Interaction: from Gaussian to Mexican Hat
RDRAW +−= RDR +−
Attraction
Repulsion
Attraction
2
2
12
1
2
21
2
2
)(
2
12
)(
⋅
−−
−−
−= σσ
σσ
σσ
jiji ffff
ij eeW
2
2
12
1
2
2
)(
2
1
⋅
−−
σσ
σ
σσ
ji ff
e
With repulsion, negative correlations in MRF formulations can be translated into graph partitioning formulations directly.
-
12
Encoding Bias: Unitary Preference
Introduce dummy nodesExpand the node setSoft Constraints
Foreground
−
=0000
)1(: T
a
aB
BAA γ
γγPreference of red pixels to be foreground,black pixels tobe background
−
=0
0)1(
:r
rB
BRR T
r
rγγ
γγ
γ controls the relative weighting between data and preference.Background
-
13
Encoding Bias: Partial Grouping
Introduce partial grouping solutionAssign a particular group labelusing dummy nodesHard ConstraintsManual selection:
pixels marked w/ the same color are in one group
Coloring dummy nodes is to assigna particular grouplabel
Background
Foreground
)()(),()( 2211 qXpXqXpX ==
z Qy yM T == or 0
M is the constraint spaceQ is the reduced solution space.
]0,,0,1,0,,0,1,0,0[
0
0)()(
LLL −=
=
=−
T
T
m
ym
qypy
AllQMQM =⊕⊥ ,
Linear:
-
14
Encoding Bias: Constraining Solution Space
Clustering with Partially labelled data
Segmentation withobject models
z Qy =0=yM T
S: translated versions of a shape
Find the eigenshape Q of SConstraining y = Q z allows usto segment out this particularshape in an image.
General form of constraints: 0)( =Ψ y
-
15
Encoding Bias: Attention
ModulationConnections for some nodesare enhanced / weakenedWeights at attentionalhotspot are less distorted
Spatial Attention:center region isanalyzed w/ morediscrimination
-
16
Representations of BiasF
G
1 2 3 4 5 86 7 9
Partial grouping{3, 4}, {6, 7}{9, G}
Preference{F, 1,2,4}{G, 5,7,8}
Attention{5, 6}{8, 9}
Soft constraints ModulationHard constraints
-
17
Constrained Optimization
DyyWyyyNassoc T
T
=)( 0.. =yMts T Constrained optimization
zQyUUIQ
VUMT
T
=−=
Σ= Constraint spaceFeasible solution space
zDQQzzWQQzzNassoc TT
TT
)()()( = Unconstrained optimization
yyWDQ T λ=−1 Eigensolution available
Rank(Q) = # of nodes - # of independent constraintsProblem: Unconstrained affinity matrix becomes denser
-
18
Results: Preference and Partial Grouping
Grouping w/o bias:Each is one group
M: Hard constraintsData: three stripes
O : hard constraints∆ : soft constraintsF/G : Filled / empty
Grouping w/ bias:Left and right are one group
U: Basis of constraint spaceM = U Σ VT
-
19
Results: Preference and Partial Grouping
Hard constraints
Soft constraints
data
prior
-
20
Results: Partial Grouping
1st Eigenvector 2nd Eigenvector 3rd Eigenvector
Image and manually set partial grouping
First row: Grouping w/o bias
Second row:Grouping w/ bias
The pumpkin starts to emerge as a whole from the background regardless of its surface markings.
-
21
Results: Figure Detection with Soft Constraints
Background Foreground Difference
Difference Thresholdedused as soft constraints
Grouping of foreground Grouping of foregroundwith bias
-
22
Results: Spatial Attention
attraction
+repulsion
+bias
Bias for A Bias for R
Grouping with BiasWhat Is It About?Grouping with Markov Random FieldsGrouping with Spectral Graph PartitioningSolving MRF by Graph PartitioningComparison of Two ApproachesPrior Knowledge in GroupingReview: Segmentation on Relational GraphsReview: Energy Function FormulationReview: Eigenvector as a SolutionInteraction: from Gaussian to Mexican HatEncoding Bias: Unitary PreferenceEncoding Bias: Partial GroupingEncoding Bias: Constraining Solution SpaceEncoding Bias: AttentionRepresentations of BiasConstrained OptimizationResults: Preference and Partial GroupingResults: Preference and Partial GroupingResults: Partial GroupingResults: Figure Detection with Soft ConstraintsResults: Spatial Attention