Monitoring Camera Video Analysis Vera Kettnaker. Motivation Cameras are everywhere...…but how can...
-
Upload
claud-arnold -
Category
Documents
-
view
213 -
download
0
Transcript of Monitoring Camera Video Analysis Vera Kettnaker. Motivation Cameras are everywhere...…but how can...
Motivation
Cameras are everywhere...…but how can we make sense of all this information ?
NYT 3/19/98: > 80 cams on short route through Manhattan
Current use of monitoring cameras (1)
Video from most cameras is analyzed by users
On the web: Traffic cams, weather cams, Seismo-cam….
Current use of monitoring cameras (2)
Computer Vision event detection
Recording and preventing law violations: running red lights, accidents, theft & robbery, planting bombs in airports…
Current use of monitoring cameras (3)
Single camera content analysis systems
Commercial example: compute traffic statistics for traffic planning
Volume/counts (# of vehicles) Lane Occupancy (% time lane is occupied) Speed (average speed in mph or kph) Density (average density = volume/speed) Headway (average headway in seconds) Length (average vehicle length in ft/meters) Vehicle Classification by user-selectable lengths (5 bins) Delay (average delay in seconds) Queue Length
Current use of monitoring cameras (4)
Systems of multiple, overlapping cameras
Construct multi-view mosaics or project into common coordinate system
large area can be monitored even if there are obstaclesCool application: One company sells a small, fast, submersible remote vehicle as a “3 camera, 240 degree view, 6,000' depth underwater survey system”
Quote: “You can even seem to swim with sharks, hundredsof them ! All in comfort, safety, and with friends or family.”
System in this talk
Several non-overlapping cameras monitor environmentin which objects move along pathways, e.g. corridors, streets, highways
Open environment : objects enter and leave, it is not known how many different people were visible.
All single camera observations are collected into globaldatabase.
Task
Task: reconstruct the paths of all objects through the environment = link successive occurrences of the same object
Application: trajectory statistics are useful for traffic planning
Preprocessing
Single-camera tracker extracts moving object info that is stored in database.
database
Single-camera information extraction
1. Use combination of background subtraction and Boykov-Veksler-Zabih motion algorithm to segment moving objects.
Main problem: strong reflections and shadows
2. Represent people by the color of their clothing: How much area of the person is covered in which color ? = coarse histograms; choice of color space important ! Varies over time: use probabilistic model
Main problems: • shadows • different lighting conditions• different viewing angles • clothing colors difficult: beige, grey, jeans • inclusion of background
Visual matching alone won’t be sufficient to solve the task !
Single-camera information extraction (2)
Exploiting additional information
Observation: the pathway structure imposes constraints on when and where moving objects can reappear.
Input
C1
C3
C4
C2
1/32/3
1/2
1/2
2.Traffic statistics:• transition probabilities
• transition time probabilities
• frequency of new objects.
1. Database of observation intervals
Output
Requirements:• most likely according to - usual traffic behavior - similarity of visual appearance • chains must be mutually exclusive “Quality measure”
Solution = set of chains
Difficult to efficiently compute a solution that satisfies both requirements ! If we only have a black-box quality measure, we can’t do much better than brute force search. (Bad !! Huge search space….)
Efficient computation
1. Step of decomposition: quality of set of chains =combination of quality measure of each chain separately
Key for efficient computation: decompose quality measure so that we can make decisionsmore independent of each other.
Decomposing the quality measure - Step 2
Transformation such thatquality of a chain =combination of quality measure of each link
What does the decomposition mean?
First decomposition step (chains) Main obstacle: # chains is a global property of a solution Relation between #chains & quality is not simple:
Second decomposition step (links) Requires additional assumptions -- Markov transition probabilities -- transitivity of visual similarity
Adding a chain can increase or decrease the qualityof the solution !
Transformation into a Linear Program
Weighted Assignment Problem
& exclusivity constraints
Decomposition,
Transformation
max quality
Special case of a Linear Program
1. Mutual exclusiveness of chains:• in-degree of successor = 1 (predecessor = real o.i. or ‘NEW’)
• out-degree of predecessor <=1
2. Minimize total link costs
Weighted Assignment Problem
Two roles for each observation interval:predecessor and successor
Can be solved in cubic time via Munkres algorithm. see OR courses...
NEW
... ...
1
N
N
2
3
1
2
3
N
Matrix elements = link terms
Matrix representation
?
Very unlikely matches:occurred long time ago
# comparison pairsgrows very big with time
Pruning # comparisons: Focus Sets
Observation: links that are less likely than will notbe part of the optimal solution Conservative estimate: which links are plausible candidates ? Use upper bound on visual match quality relative time window for each pair of cameras
NEW
Focus set =set of plausiblematch candidates
So far: batch algorithm
Not appropriate for monitoring systems !• problem size becomes very big• continuous output of answers desirable
Problem: we can prove that there is no online algorithm that returns the correct result for all inputs.
Proposal: Approximate online algorithm that uses limited, semantically meaningful lookahead.
Online Algorithm
3. Solve this subproblem.
Online Algorithm
1. Determine possible match candidates for A.V
AFocus set A
W
X
B
C
Y
ZFocus set B
Focus set C
2. Which other focus setsoverlap with A’s focus set ?
W
X
4. Fix A’s assignment
• Inclusion of A in previous lookaheads consistency with past• variant: include also the contestants of the contestants of ... trade off waiting time and correctness
Results
• 2 out of 28 observation intervals are linked to wrong predecessor• Additionally: 2 links are missing for reoccurences after unnaturally long disappearance time• small focus sets
Summary
• Introduced Multi-camera Path Reconstruction Task
• visual appearance matching aided by knowledge of traffic statistics
• Transformation into a Weighted Assignment Problem: efficient computation
• Focus sets, online version
• Preliminary experimental results are promising