Mestrado em Cincia de Computadores Mestrado Integrado em Engenharia de Redes e Sistemas Informticos...
-
Upload
charles-brooks -
Category
Documents
-
view
217 -
download
0
description
Transcript of Mestrado em Cincia de Computadores Mestrado Integrado em Engenharia de Redes e Sistemas Informticos...
![Page 1: Mestrado em Cincia de Computadores Mestrado Integrado em Engenharia de Redes e Sistemas Informticos VC 15/16 TP10 Advanced Segmentation Miguel Tavares.](https://reader035.fdocuments.in/reader035/viewer/2022062413/5a4d1b887f8b9ab0599bdb0b/html5/thumbnails/1.jpg)
Mestrado em Ciência de ComputadoresMestrado Integrado em Engenharia de Redes e
Sistemas Informáticos
VC 15/16 – TP10Advanced Segmentation
Miguel Tavares Coimbra
![Page 2: Mestrado em Cincia de Computadores Mestrado Integrado em Engenharia de Redes e Sistemas Informticos VC 15/16 TP10 Advanced Segmentation Miguel Tavares.](https://reader035.fdocuments.in/reader035/viewer/2022062413/5a4d1b887f8b9ab0599bdb0b/html5/thumbnails/2.jpg)
VC 15/16 - TP10 - Advanced Segmentation
Outline
• Introduction• Simple clustering• K-means clustering• Graph-theoretic clustering• Fitting lines
Acknowledgements: These slides follow Forsyth and Ponce’s “Computer Vision: A Modern Approach”, Chapters 14 and 15.
![Page 3: Mestrado em Cincia de Computadores Mestrado Integrado em Engenharia de Redes e Sistemas Informticos VC 15/16 TP10 Advanced Segmentation Miguel Tavares.](https://reader035.fdocuments.in/reader035/viewer/2022062413/5a4d1b887f8b9ab0599bdb0b/html5/thumbnails/3.jpg)
VC 15/16 - TP10 - Advanced Segmentation
Topic: Introduction
• Introduction• Simple clustering• K-means clustering• Graph-theoretic clustering• Fitting lines
![Page 4: Mestrado em Cincia de Computadores Mestrado Integrado em Engenharia de Redes e Sistemas Informticos VC 15/16 TP10 Advanced Segmentation Miguel Tavares.](https://reader035.fdocuments.in/reader035/viewer/2022062413/5a4d1b887f8b9ab0599bdb0b/html5/thumbnails/4.jpg)
What is ‘Segmentation’? (again?)
• Traditional definition:“Separation of the image in different areas“• Decompose an image into “superpixels”.• Colour and texture coherence.
• Aren’t there other ways to look at the ‘Segmentation’ concept?
VC 15/16 - TP10 - Advanced Segmentation
![Page 5: Mestrado em Cincia de Computadores Mestrado Integrado em Engenharia de Redes e Sistemas Informticos VC 15/16 TP10 Advanced Segmentation Miguel Tavares.](https://reader035.fdocuments.in/reader035/viewer/2022062413/5a4d1b887f8b9ab0599bdb0b/html5/thumbnails/5.jpg)
Other ‘Segmentation’ problems
• Fitting lines to edge points.
• Fitting a fundamental matrix to a set of feature points.
VC 15/16 - TP10 - Advanced Segmentation
This one is complicated!Check Forsyth and Ponce, chap.14
We can’t see this as
‘separating an image in
different areas’!
![Page 6: Mestrado em Cincia de Computadores Mestrado Integrado em Engenharia de Redes e Sistemas Informticos VC 15/16 TP10 Advanced Segmentation Miguel Tavares.](https://reader035.fdocuments.in/reader035/viewer/2022062413/5a4d1b887f8b9ab0599bdb0b/html5/thumbnails/6.jpg)
Segmentation as Clustering
• Tries to answer the question:“Which components of the data set naturally
belong together?”• Two approaches:
– Partitioning• Decompose a large data set into pieces that are
‘good’ according to our model.– Grouping
• Collect sets of data items that ‘make sense’ according to our model.
VC 15/16 - TP10 - Advanced Segmentation
![Page 7: Mestrado em Cincia de Computadores Mestrado Integrado em Engenharia de Redes e Sistemas Informticos VC 15/16 TP10 Advanced Segmentation Miguel Tavares.](https://reader035.fdocuments.in/reader035/viewer/2022062413/5a4d1b887f8b9ab0599bdb0b/html5/thumbnails/7.jpg)
Human Clustering
• How de we humans cluster images?– Well… we don’t really know...
• Gestalt school of psychologists– Attempts to study this problem.– Key ideas:
• Context affects perception. So…• Responses to stimuli are not important.• Grouping is the key to understanding visual
perception.
VC 15/16 - TP10 - Advanced Segmentation
![Page 8: Mestrado em Cincia de Computadores Mestrado Integrado em Engenharia de Redes e Sistemas Informticos VC 15/16 TP10 Advanced Segmentation Miguel Tavares.](https://reader035.fdocuments.in/reader035/viewer/2022062413/5a4d1b887f8b9ab0599bdb0b/html5/thumbnails/8.jpg)
E
Examples of Gestalt factors that lead to groupingVC 15/16 - TP10 - Advanced Segmentation
![Page 9: Mestrado em Cincia de Computadores Mestrado Integrado em Engenharia de Redes e Sistemas Informticos VC 15/16 TP10 Advanced Segmentation Miguel Tavares.](https://reader035.fdocuments.in/reader035/viewer/2022062413/5a4d1b887f8b9ab0599bdb0b/html5/thumbnails/9.jpg)
Examples of Gestalt factors that lead to groupingVC 15/16 - TP10 - Advanced Segmentation
![Page 10: Mestrado em Cincia de Computadores Mestrado Integrado em Engenharia de Redes e Sistemas Informticos VC 15/16 TP10 Advanced Segmentation Miguel Tavares.](https://reader035.fdocuments.in/reader035/viewer/2022062413/5a4d1b887f8b9ab0599bdb0b/html5/thumbnails/10.jpg)
Gestalt in Practice
• Rules function fairly well as explanations.• However, they are insufficient to form an
algorithm.• So, how is Gestalt useful?
– Gives us ‘hints’ on where to go.– Shatters the traditional definition of
segmentation, clearly showing us that we need something better.
– Context is vital! Grouping is vital!
VC 15/16 - TP10 - Advanced Segmentation
![Page 11: Mestrado em Cincia de Computadores Mestrado Integrado em Engenharia de Redes e Sistemas Informticos VC 15/16 TP10 Advanced Segmentation Miguel Tavares.](https://reader035.fdocuments.in/reader035/viewer/2022062413/5a4d1b887f8b9ab0599bdb0b/html5/thumbnails/11.jpg)
VC 15/16 - TP10 - Advanced Segmentation
Topic: Simple clustering
• Introduction• Simple clustering• K-means clustering• Graph-theoretic clustering• Fitting lines
![Page 12: Mestrado em Cincia de Computadores Mestrado Integrado em Engenharia de Redes e Sistemas Informticos VC 15/16 TP10 Advanced Segmentation Miguel Tavares.](https://reader035.fdocuments.in/reader035/viewer/2022062413/5a4d1b887f8b9ab0599bdb0b/html5/thumbnails/12.jpg)
What do we mean by ‘clustering’?
• “Clustering is a process whereby a data set is replaced by clusters, which are collections of data points that belong together”
Forsyth and Ponce, “Computer Vision: A modern approach”
• Why do points “belong together”?– Same colour.– Same texture.– Same… something!
VC 15/16 - TP10 - Advanced Segmentation
![Page 13: Mestrado em Cincia de Computadores Mestrado Integrado em Engenharia de Redes e Sistemas Informticos VC 15/16 TP10 Advanced Segmentation Miguel Tavares.](https://reader035.fdocuments.in/reader035/viewer/2022062413/5a4d1b887f8b9ab0599bdb0b/html5/thumbnails/13.jpg)
Simple clustering
• Two natural types of clustering:– Divisive clustering
• Entire data set is regarded as a cluster.• Clusters are recursively split.
– Agglomerative clustering• Each data item is a cluster.• Clusters are recursively merged.
• Where have I seen this before?
VC 15/16 - TP10 - Advanced Segmentation
![Page 14: Mestrado em Cincia de Computadores Mestrado Integrado em Engenharia de Redes e Sistemas Informticos VC 15/16 TP10 Advanced Segmentation Miguel Tavares.](https://reader035.fdocuments.in/reader035/viewer/2022062413/5a4d1b887f8b9ab0599bdb0b/html5/thumbnails/14.jpg)
VC 15/16 - TP10 - Advanced Segmentation
Split and Merge (Region-based
segmentation) is in fact a clustering
algorithm.
![Page 15: Mestrado em Cincia de Computadores Mestrado Integrado em Engenharia de Redes e Sistemas Informticos VC 15/16 TP10 Advanced Segmentation Miguel Tavares.](https://reader035.fdocuments.in/reader035/viewer/2022062413/5a4d1b887f8b9ab0599bdb0b/html5/thumbnails/15.jpg)
Generic simple clustering algorithms
VC 15/16 - TP10 - Advanced Segmentation
Which inter-cluster distance?
What does this mean?
• Divisive Clustering– Construct a single cluster containing all points– While the clustering is not satisfactory
• Split the cluster that yields the two components with the largest inter-cluster distance
– end• Agglomerative Clustering
– Make each point a separate cluster– Until the clustering is satisfactory
• Merge the two clusters with smallest inter-cluster distance
– end
![Page 16: Mestrado em Cincia de Computadores Mestrado Integrado em Engenharia de Redes e Sistemas Informticos VC 15/16 TP10 Advanced Segmentation Miguel Tavares.](https://reader035.fdocuments.in/reader035/viewer/2022062413/5a4d1b887f8b9ab0599bdb0b/html5/thumbnails/16.jpg)
VC 15/16 - TP10 - Advanced Segmentation
![Page 17: Mestrado em Cincia de Computadores Mestrado Integrado em Engenharia de Redes e Sistemas Informticos VC 15/16 TP10 Advanced Segmentation Miguel Tavares.](https://reader035.fdocuments.in/reader035/viewer/2022062413/5a4d1b887f8b9ab0599bdb0b/html5/thumbnails/17.jpg)
Simple clustering with images
• Some specific problems arise:– Lots of pixels! Graphical representations are
harder to read.– Segmentation: It is desirable that certain
objects are connected. How to enforce this?– When do we stop splitting/merging process?
• Complex situations require more complex clustering solutions!
VC 15/16 - TP10 - Advanced Segmentation
![Page 18: Mestrado em Cincia de Computadores Mestrado Integrado em Engenharia de Redes e Sistemas Informticos VC 15/16 TP10 Advanced Segmentation Miguel Tavares.](https://reader035.fdocuments.in/reader035/viewer/2022062413/5a4d1b887f8b9ab0599bdb0b/html5/thumbnails/18.jpg)
VC 15/16 - TP10 - Advanced Segmentation
Topic: K-means clustering
• Introduction• Simple clustering• K-means clustering• Graph-theoretic clustering• Fitting lines
![Page 19: Mestrado em Cincia de Computadores Mestrado Integrado em Engenharia de Redes e Sistemas Informticos VC 15/16 TP10 Advanced Segmentation Miguel Tavares.](https://reader035.fdocuments.in/reader035/viewer/2022062413/5a4d1b887f8b9ab0599bdb0b/html5/thumbnails/19.jpg)
Objective function
• What if we know that there are k clusters in the image?
• We can define an objective function!– Expresses how good my representation is.
• We can now build an algorithm to obtain the best representation.
VC 15/16 - TP10 - Advanced Segmentation
Caution! “Best” given my objective function!
![Page 20: Mestrado em Cincia de Computadores Mestrado Integrado em Engenharia de Redes e Sistemas Informticos VC 15/16 TP10 Advanced Segmentation Miguel Tavares.](https://reader035.fdocuments.in/reader035/viewer/2022062413/5a4d1b887f8b9ab0599bdb0b/html5/thumbnails/20.jpg)
K-means Clustering
• Assume:– We have k clusters.– Each cluster i has a centre ci.– Element j to be clustered is described by a
feature vector xj.
• Our objective function is thus:
VC 15/16 - TP10 - Advanced Segmentation
clustersi iclusterj
ijT
ij cxcxdataclusters)(
)()(),(
What does this mean?
![Page 21: Mestrado em Cincia de Computadores Mestrado Integrado em Engenharia de Redes e Sistemas Informticos VC 15/16 TP10 Advanced Segmentation Miguel Tavares.](https://reader035.fdocuments.in/reader035/viewer/2022062413/5a4d1b887f8b9ab0599bdb0b/html5/thumbnails/21.jpg)
Iteration step
• Too many possible allocations of points to clusters to search this space for a minimum.
• Iterate!– Assume cluster centres are known and
allocate each point to the closest cluster centre.
– Assume the allocation is known and choose a new set of cluster centres. Each centre is the mean of the points allocated to that cluster.
VC 15/16 - TP10 - Advanced Segmentation
![Page 22: Mestrado em Cincia de Computadores Mestrado Integrado em Engenharia de Redes e Sistemas Informticos VC 15/16 TP10 Advanced Segmentation Miguel Tavares.](https://reader035.fdocuments.in/reader035/viewer/2022062413/5a4d1b887f8b9ab0599bdb0b/html5/thumbnails/22.jpg)
[Wikipedia]VC 15/16 - TP10 - Advanced Segmentation
![Page 23: Mestrado em Cincia de Computadores Mestrado Integrado em Engenharia de Redes e Sistemas Informticos VC 15/16 TP10 Advanced Segmentation Miguel Tavares.](https://reader035.fdocuments.in/reader035/viewer/2022062413/5a4d1b887f8b9ab0599bdb0b/html5/thumbnails/23.jpg)
Interactive Java Tutorial
VC 15/16 - TP10 - Advanced Segmentation
http://home.dei.polimi.it/matteucc/Clustering/tutorial_html/AppletKM.html
![Page 24: Mestrado em Cincia de Computadores Mestrado Integrado em Engenharia de Redes e Sistemas Informticos VC 15/16 TP10 Advanced Segmentation Miguel Tavares.](https://reader035.fdocuments.in/reader035/viewer/2022062413/5a4d1b887f8b9ab0599bdb0b/html5/thumbnails/24.jpg)
VC 15/16 - TP10 - Advanced Segmentation
Topic: Graph-theoretic clustering
• Introduction• Simple clustering• K-means clustering• Graph-theoretic clustering• Fitting lines
![Page 25: Mestrado em Cincia de Computadores Mestrado Integrado em Engenharia de Redes e Sistemas Informticos VC 15/16 TP10 Advanced Segmentation Miguel Tavares.](https://reader035.fdocuments.in/reader035/viewer/2022062413/5a4d1b887f8b9ab0599bdb0b/html5/thumbnails/25.jpg)
Using graphs
• Clustering can be seen as a problem of “cutting graphs into good pieces”.
• Data Items– Vertex in a weighted graph.– Weights are large if elements are similar.
• Cut edges– Cut edges with small weights.– Keep connected components with large
interior weights.
VC 15/16 - TP10 - Advanced Segmentation
Regions!
![Page 26: Mestrado em Cincia de Computadores Mestrado Integrado em Engenharia de Redes e Sistemas Informticos VC 15/16 TP10 Advanced Segmentation Miguel Tavares.](https://reader035.fdocuments.in/reader035/viewer/2022062413/5a4d1b887f8b9ab0599bdb0b/html5/thumbnails/26.jpg)
VC 15/16 - TP10 - Advanced Segmentation
![Page 27: Mestrado em Cincia de Computadores Mestrado Integrado em Engenharia de Redes e Sistemas Informticos VC 15/16 TP10 Advanced Segmentation Miguel Tavares.](https://reader035.fdocuments.in/reader035/viewer/2022062413/5a4d1b887f8b9ab0599bdb0b/html5/thumbnails/27.jpg)
Graphs and Clustering
• Associate each element to be clustered with a vertex on a graph.
• Construct an edge from every element to every other.
• Associate a weight with each edge based on a similarity measure.
• Cut the edges in the graph to form a good set of connected components.
VC 15/16 - TP10 - Advanced Segmentation
![Page 28: Mestrado em Cincia de Computadores Mestrado Integrado em Engenharia de Redes e Sistemas Informticos VC 15/16 TP10 Advanced Segmentation Miguel Tavares.](https://reader035.fdocuments.in/reader035/viewer/2022062413/5a4d1b887f8b9ab0599bdb0b/html5/thumbnails/28.jpg)
Weight Matrices
• Typically look like block diagonal matrices.• Why?
– Interclusters similarities are strong.– Intracluster similarities are weak.
• Split a matrix into smaller matrices, each of which is a block.
• Define Affinity Measures.
VC 15/16 - TP10 - Advanced Segmentation
![Page 29: Mestrado em Cincia de Computadores Mestrado Integrado em Engenharia de Redes e Sistemas Informticos VC 15/16 TP10 Advanced Segmentation Miguel Tavares.](https://reader035.fdocuments.in/reader035/viewer/2022062413/5a4d1b887f8b9ab0599bdb0b/html5/thumbnails/29.jpg)
• Affinity measures– Affinity by Distance– Affinity by Intensity– Affinity by Colour– Affinity by Texture
• Popular method: Normalized cutsJianbo Shi and Jitendra Malik, “Normalized Cuts and Image
Segmentation”, IEEE Transactions on Pattern Analysis And Machine Intelligence, Vol. 22, No. 8, August 2000
VC 15/16 - TP10 - Advanced Segmentation
Want to know more?Check out: Forsyth and Ponce, Section
14.5
![Page 30: Mestrado em Cincia de Computadores Mestrado Integrado em Engenharia de Redes e Sistemas Informticos VC 15/16 TP10 Advanced Segmentation Miguel Tavares.](https://reader035.fdocuments.in/reader035/viewer/2022062413/5a4d1b887f8b9ab0599bdb0b/html5/thumbnails/30.jpg)
VC 15/16 - TP10 - Advanced Segmentation
Topic: Fitting lines
• Introduction• Simple clustering• K-means clustering• Graph-theoretic clustering• Fitting lines
![Page 31: Mestrado em Cincia de Computadores Mestrado Integrado em Engenharia de Redes e Sistemas Informticos VC 15/16 TP10 Advanced Segmentation Miguel Tavares.](https://reader035.fdocuments.in/reader035/viewer/2022062413/5a4d1b887f8b9ab0599bdb0b/html5/thumbnails/31.jpg)
Fitting and Clustering
• Another definition for segmentation:– Pixels belong together because they conform
to some model.• Sounds like “Segmentation by
Clustering”...• Key difference:
– The model is now explicit.
We have a mathematical model for the
object we want to segment.E.g. A line
VC 15/16 - TP10 - Advanced Segmentation
![Page 32: Mestrado em Cincia de Computadores Mestrado Integrado em Engenharia de Redes e Sistemas Informticos VC 15/16 TP10 Advanced Segmentation Miguel Tavares.](https://reader035.fdocuments.in/reader035/viewer/2022062413/5a4d1b887f8b9ab0599bdb0b/html5/thumbnails/32.jpg)
VC 15/16 - TP10 - Advanced Segmentation
Hough Transform
• Elegant method for direct object recognition
• Edges need not be connected• Complete object need not be visible• Key Idea: Edges VOTE for the possible
model
![Page 33: Mestrado em Cincia de Computadores Mestrado Integrado em Engenharia de Redes e Sistemas Informticos VC 15/16 TP10 Advanced Segmentation Miguel Tavares.](https://reader035.fdocuments.in/reader035/viewer/2022062413/5a4d1b887f8b9ab0599bdb0b/html5/thumbnails/33.jpg)
Least Squares
• Popular fitting procedure.• Simple but biased (why?).• Consider a line:
y = ax + b• What is the line that best predicts all
observations (xi,yi)?– Minimize:
iii baxy 2)(
VC 15/16 - TP10 - Advanced Segmentation
![Page 34: Mestrado em Cincia de Computadores Mestrado Integrado em Engenharia de Redes e Sistemas Informticos VC 15/16 TP10 Advanced Segmentation Miguel Tavares.](https://reader035.fdocuments.in/reader035/viewer/2022062413/5a4d1b887f8b9ab0599bdb0b/html5/thumbnails/34.jpg)
What happens if the line is vertical?
VC 15/16 - TP10 - Advanced Segmentation
![Page 35: Mestrado em Cincia de Computadores Mestrado Integrado em Engenharia de Redes e Sistemas Informticos VC 15/16 TP10 Advanced Segmentation Miguel Tavares.](https://reader035.fdocuments.in/reader035/viewer/2022062413/5a4d1b887f8b9ab0599bdb0b/html5/thumbnails/35.jpg)
This looks better…
VC 15/16 - TP10 - Advanced Segmentation
![Page 36: Mestrado em Cincia de Computadores Mestrado Integrado em Engenharia de Redes e Sistemas Informticos VC 15/16 TP10 Advanced Segmentation Miguel Tavares.](https://reader035.fdocuments.in/reader035/viewer/2022062413/5a4d1b887f8b9ab0599bdb0b/html5/thumbnails/36.jpg)
Total Least Squares
• Works with the actual distance between the point and the line (rather than the vertical distance).
• Lines are represented as a collection of points where:
• And:122 ba
Again... Minimize the error, obtain the line with the ‘best
fit’.
0 cbyax
VC 15/16 - TP10 - Advanced Segmentation
![Page 37: Mestrado em Cincia de Computadores Mestrado Integrado em Engenharia de Redes e Sistemas Informticos VC 15/16 TP10 Advanced Segmentation Miguel Tavares.](https://reader035.fdocuments.in/reader035/viewer/2022062413/5a4d1b887f8b9ab0599bdb0b/html5/thumbnails/37.jpg)
Point correspondence
• We can estimate a line but, which points are on which line?
• Usually:– We are fitting lines to edge points, so…– Edge directions can give us hints!
• What if I only have isolated points?• Let’s look at two options:
– Incremental fitting.– Allocating points to lines with K-means
VC 15/16 - TP10 - Advanced Segmentation
![Page 38: Mestrado em Cincia de Computadores Mestrado Integrado em Engenharia de Redes e Sistemas Informticos VC 15/16 TP10 Advanced Segmentation Miguel Tavares.](https://reader035.fdocuments.in/reader035/viewer/2022062413/5a4d1b887f8b9ab0599bdb0b/html5/thumbnails/38.jpg)
Incremental Fitting
• Start with connected curves of edge points• Fit lines to those points in that curve.• Incremental fitting:
– Start at one end of the curve.– Keep fitting all points in that curve to a line.– Begin another line when the fitting
deteriorates too much.• Great for closed curves!
VC 15/16 - TP10 - Advanced Segmentation
![Page 39: Mestrado em Cincia de Computadores Mestrado Integrado em Engenharia de Redes e Sistemas Informticos VC 15/16 TP10 Advanced Segmentation Miguel Tavares.](https://reader035.fdocuments.in/reader035/viewer/2022062413/5a4d1b887f8b9ab0599bdb0b/html5/thumbnails/39.jpg)
VC 15/16 - TP10 - Advanced Segmentation
![Page 40: Mestrado em Cincia de Computadores Mestrado Integrado em Engenharia de Redes e Sistemas Informticos VC 15/16 TP10 Advanced Segmentation Miguel Tavares.](https://reader035.fdocuments.in/reader035/viewer/2022062413/5a4d1b887f8b9ab0599bdb0b/html5/thumbnails/40.jpg)
K-means allocation
• What if points carry no hints about which line they lie on?
• Assume there are k lines for the x points.• Minimize:
• Iteration:– Allocate each point to the closest line.– Fir the best line to the points allocated to each
line.VC 15/16 - TP10 - Advanced Segmentation
2
lines points
)point,line( dist
![Page 41: Mestrado em Cincia de Computadores Mestrado Integrado em Engenharia de Redes e Sistemas Informticos VC 15/16 TP10 Advanced Segmentation Miguel Tavares.](https://reader035.fdocuments.in/reader035/viewer/2022062413/5a4d1b887f8b9ab0599bdb0b/html5/thumbnails/41.jpg)
VC 15/16 - TP10 - Advanced Segmentation
![Page 42: Mestrado em Cincia de Computadores Mestrado Integrado em Engenharia de Redes e Sistemas Informticos VC 15/16 TP10 Advanced Segmentation Miguel Tavares.](https://reader035.fdocuments.in/reader035/viewer/2022062413/5a4d1b887f8b9ab0599bdb0b/html5/thumbnails/42.jpg)
Resources
• Forsyth and Ponce, Chapter 14• Forsyth and Ponce, Chapter 15
VC 15/16 - TP10 - Advanced Segmentation