Data Clustering for Image Segmentation

7/28/2019 Data Clustering for Image Segmentation

1/51

DataData clusteringclustering forfor imageimage

segmentationsegmentation

Domenico [email protected]


2/51

27/10/2008Data clustering for image segmentation Page 2

Summary

ARGOS Project overview

Image Segmentation and Optical Flow (OF) K-means based OF data clustering

Rek-means vs k-means Examples and experimental evaluation


3/51


ARGOS project overview

TheARGOS system controls a waterway of about 6

km length, 80 to 150 meters width, through 14observation posts (Survey Cells).

Automatic Remote Grand Canal Observation System


4/51


Rai Tg1 video


5/51


ARGOS objectives

management and evaluation of

navigation rulestraffic statistics and analysis

securitypreservation of historical heritage

(reduction of wave motion)


6/51


Motivation

Human Surveillance

Until 2006 the Municipal Administration of Venice paid people forboats counting:

9 Error prone

9 Una tantum

Automatic Surveillance

FromNovember 2007 Venice has an automatic video surveillancesystem

9 24 hours/day 7 days/week

9 recorded traffic violations

9 Certified error


7/51


ARGOS functions

optical detection and tracking of moving targets

computing position, speed and heading of targetsevent detection (speed limits, access control, )recording 24/7 video and track information (post-

analysis)rectifying camera frames and stitching them into acomposite view

automatic PTZ trackingon line traffic information http://www.argos.venezia.it


8/51


www.argos.venezia.it


9/51


Examples

Detect boats docking inthe highlighted area

Speed limit control


10/51


ARGOS features

Dynamic background (water)

Multi object tracking(up to 10 boats in the same view)

Multi camera for large areas(56 cameras x 6 km length)

Third party extensive evaluation(2.2107 frames analyzed)


11/51


General architecture


12/51


Survey cell

3 high resolution network cameras, a PTZ camera for zoomand tracking of the selected target, and 2 computers running

the image processing and tracking software.

The survey cells are installedon the top of severalbuildings leaning over theGrand Canal


13/51


Survey cells


14/51


Segmentation module

Rek-meansclustering algorithm

S t ti


15/51


Segmentation(detecting objects of interest)

Background

Formation

CurrentFrame

I

BackgroundModel

BForegroundComputation

ListofDetected Objects

continuously

updatedF


16/51


Background formation

Problems:

- gradual illumination changes and sudden ones (clouds)- motion changes (camera oscillations)- high frequency noise (waves in our case)- changes in the background geometry (parked boats).

Approach:- computation of color distribution of a set of frames

- highest component formthe background


17/51


Background modeling

Background Image

computed fromS(the image displayonly the higher

gaussian values)

SetS ofn images froma camera

Natural images

Artificial image

Foreground computation


18/51


Foreground computation(background subtraction technique)

current frame

background image

foreground image

THRESHOLD T(based on illumination conditions)

blobs (Binay Large OBjectS)

>

T


19/51


Tracking module

Single-hypothesis Tracking

We use a set of Kalman Filters (one for each tracked boat).

Data Association: Nearest Neighbor ruleTrack formation: unassociated observationsTrack deletion: high covariance in the filter

Multi-hypotheses Tracking

Track splitting: in ambiguous cases (data association has

multiple solutions)Track merging: high correlation between tracks


20/51


Multi hypothesis tracking

3 tracks (240, 247, 285)only 1 actual observation (285)

240285247

R tifi ti


21/51


Rectification


22/51


Unified Views


23/51


Panoramic view

PTZ Camera

DENSITA DI TRAFFICO TEMPO REALE


24/51


DENSITA DI TRAFFICO TEMPO REALE

1

2

4

3

8

5

7

6

10

9

12

11

13

88Bacino S.MarcoSalute13

1814SaluteAccademia12

98AccademiaCa' Rezzonico11

2521Ca' RezzonicoSan Tom10

2614San TomS.Silvestro9

85San SilvestroRialto8

1612RialtoCa D'oro7

44Ca D'oroSanta Fosca6

186Santa FoscaCannaregio5

1810CannaregioFerrovia4

108FerroviaPonte Calatrava3

126Ponte CalatravaScomensera2

85ScomenseraPonte Libert1

Densit

maxDensit mediaADaTratto

Numero Totale Imbarcazioni in Canal Grande: 121

02/11/2006 o re 11,30

DENSITA' MEDIE E MASSIME DEL TRAFFICO

E l


25/51


Example

B k d bt ti bl


26/51


Background subtraction problems

Background subtraction is a fast and effectivetechnique, but it presents a series of problems:

How to compute a correct background?

How to manage gradual and sudden illumination

changes? How to manage high-frequencies background objects(such as tree branches, sea waves, and similar)?

How to manage partially occluded objects (for example,two cars crossing in a street)?

How to manage partially occluded


27/51


How to manage partially occluded

objects?

UNDER SEGMENTATION1 blob 2 boats

OVER SEGMENTATION3 blobs 2 boats

P d l ti


28/51


Proposed solution

Background Subtraction

+Optical Flow+

Rek-means

[Bloisi and Iocchi 2008]

O ti l fl


29/51


Optical flow

We use a sparse iterative version of Lucas-Kanade optical flow inpyramids [Bouget2000]. It calculates coordinates of the featurepoints on the currentvideo frame given their coordinates on the

previous frame. The function finds the coordinates with sub-pixelaccuracy. Every feature point is classiefied into one of four principaldirections.

previous frame current frameoptical flow image

(a particular)

NW direction

Under segm solution using OF


30/51


Under segm. solution using OF

Solution (non trivial):clustering the OF sparse map

outliersnoisek=?

Over segm solution using OF


31/51


Over segm. solution using OF

K means clustering


32/51


K-means clustering

+k-means centroidsx truecentroids

randominitialization

K means limitations


33/51


K-means limitations

the user may knowthe exact number of clusters (k)beforehand

k-means is notguaranteed to return a global optimum:The quality of the final solution depends largely on theinitial set of centroids, and may, in practice, be much

poorer than the global optimum.

If we choose the k initial clusters at random, k-means

can converge to the wrong answer (in the sense thatadifferent and optimal solution to the minimization functionexists).

Rek means features


34/51


Rek-means features

Improved clustering using Rek-means


35/51


Improved clustering using Rek-means

Rek means algorithm


36/51


Rek-means algorithm

Rek means steps


37/51


Rek-means steps

a) Initial data setb) Over-clustering

c) Rectangles mergingd) Final centroids

Rectangle distance


38/51


Rectangle distance

Validating step


39/51


Validating step

Anderson Darling test


40/51


Anderson Darling test

if A2 > CV the test is negative

else if A2 < CV the test is positivewhere CV is the critical value

Validating step example


41/51


Validating step example

The associatingstep finds 1

cluster

The validatingstep correctsthe error and

finds 2 clusters

d >>

Rek-means real data example


42/51


Rek means real data example

Current frame and foreground image Rek-means output

Rek-means evaluation


43/51


Rek means evaluation

Time complexity O(nlogn)

100 images (with resolution 2000x2000)

ARGOS experimental evaluation


44/51


ARGOS experimental evaluation

on-line, evaluation is performed during the actual operationof the system;recorded on-line evaluation is performed on a video

recording the output of the system running on-line;off-line evaluation is performed on the system running off-line on recorded input videos.

Online evaluation


45/51


Online evaluation

FN: False negatives, i.e. boats not tracked

FP-R: False positives due to reflections (wrong track with a random direction)FP-W: False positives due to wakes (wrong track following the correctone)

Counting evaluation


46/51


Counting evaluation

COUNTING EVALUATION TEST

A virtual line has been put across the Canal in the field of viewof a surveycell, the number of boats passing this line has been counted automatically

by the systemnSys, and the same value is manually calculated by visuallyinspection n, the average percentage error is then computed as

= | nSys n | / n

An additional error measure is calculated byconsidering the probability of makingan error in counting a single boat passing the line

where () is 0 when the argument is 0 and 1 otherwise.

Counting evaluation (2)


47/51


Cou g e a ua o ( )

New features


48/51


It is crucial to know who does what

Behavior analysis based on theidentity of boats observed is verydifficult through Computer Vision only

(plate recognition impossible withocclusion)

Add sensors such as GPS or RadioFrequency Identifiers (RFID)

References


49/51


- D. Bloisi, L. Iocchi, G.R. Leone, R. Pigliacampo, L. Tombolini, L. Novelli. ADistributed Vision System for Boat Traffic Monitoring in the Venice Grand Canal

(VISAPP), 2007.

- D. Bloisi and L. Iocchi. Rek-Means: A k-Means Based Clustering Algorithm. In Proc.of 6th Int. Conf. on Computer Vision Systems (ICVS). 2008. pp. 109 - 118

- J .Y. Bouguet. Pyramidal Implementation of the Lucas Kanade Feature Tracker.2000

Homework


50/51


Generate a data set froma distribution formedby a mixture of k Gaussians with the same

circular variance (i.e., = diag {, }) andwith | i - j | > 3 for each pair i,j

Apply k-means on this data (with the correctvalue of k) several times starting fromdifferent

initial values and compute the percentage ofcorrect clustering.



51/51


segmentationsegmentation

Domenico [email protected]

Data Clustering for Image Segmentation

Documents

Transcript of Data Clustering for Image Segmentation