Video Trails: Representing and Visualizing Structure in Video Sequences
description
Transcript of Video Trails: Representing and Visualizing Structure in Video Sequences
![Page 1: Video Trails: Representing and Visualizing Structure in Video Sequences](https://reader035.fdocuments.in/reader035/viewer/2022062500/56815928550346895dc650b4/html5/thumbnails/1.jpg)
Video Trails: Representing and Visualizing Structure in Video Sequences
Vikrant Kobla
David Doermann
Christos Faloutsos
![Page 2: Video Trails: Representing and Visualizing Structure in Video Sequences](https://reader035.fdocuments.in/reader035/viewer/2022062500/56815928550346895dc650b4/html5/thumbnails/2.jpg)
Outline
Background and Motivation Overview Video Trails Trail Segmentation Trail Classification Gradual Transition Detection Experiments and Results Conclusion
![Page 3: Video Trails: Representing and Visualizing Structure in Video Sequences](https://reader035.fdocuments.in/reader035/viewer/2022062500/56815928550346895dc650b4/html5/thumbnails/3.jpg)
Background and Motivation
Video is a valuable information resource There are still few efficient ways to provide
access to the information the video contains Early work on indexing video treated video
sequence as collections of still images, ignored the temporal structure
Efficient analysis and representation of the temporal structure of a video is necessary
![Page 4: Video Trails: Representing and Visualizing Structure in Video Sequences](https://reader035.fdocuments.in/reader035/viewer/2022062500/56815928550346895dc650b4/html5/thumbnails/4.jpg)
Overview
1. Generate a trail of points (Video Trails) in a low-dimensional space
2. Segment the video trails
3. Classify each of those segmented trails into two types:
Stationary (low activity) VS Transitional (high activity)
4. Detect gradual transition
![Page 5: Video Trails: Representing and Visualizing Structure in Video Sequences](https://reader035.fdocuments.in/reader035/viewer/2022062500/56815928550346895dc650b4/html5/thumbnails/5.jpg)
Video Trails
Definition: A trail of points in a low-dimensional space where each point is derived from physical features of a single frame in the video clip
Features: DC coefficients of the luminance and chrominance components of an MPEG frame
Dimensionality Reduction (FastMap) initial feature vector
a vector in that dimensional
target dimension spaceFastMap
![Page 6: Video Trails: Representing and Visualizing Structure in Video Sequences](https://reader035.fdocuments.in/reader035/viewer/2022062500/56815928550346895dc650b4/html5/thumbnails/6.jpg)
Example
Consider a video clip with a 320x240 frame size Each frame has 20x15 MBs( Macroblock) Each MB contains 6 DC coefficients ( 4
luminance and 2 chrominance) Totally, 20x15x6=1800 coefficients (initial vector) 1800-by-1 vector
(X1,X2,X3)
3 (target dimension) FastMap
![Page 7: Video Trails: Representing and Visualizing Structure in Video Sequences](https://reader035.fdocuments.in/reader035/viewer/2022062500/56815928550346895dc650b4/html5/thumbnails/7.jpg)
Example
![Page 8: Video Trails: Representing and Visualizing Structure in Video Sequences](https://reader035.fdocuments.in/reader035/viewer/2022062500/56815928550346895dc650b4/html5/thumbnails/8.jpg)
Example
![Page 9: Video Trails: Representing and Visualizing Structure in Video Sequences](https://reader035.fdocuments.in/reader035/viewer/2022062500/56815928550346895dc650b4/html5/thumbnails/9.jpg)
Trail Segmentation
Segment the video in order to determine regions of high activity corresponding to transitions and low activity corresponding to individual shots
The problem of segmenting the video into sets of frames is transformed into the problem of splitting the video trails into smaller trails corresponding to segments of video
![Page 10: Video Trails: Representing and Visualizing Structure in Video Sequences](https://reader035.fdocuments.in/reader035/viewer/2022062500/56815928550346895dc650b4/html5/thumbnails/10.jpg)
Splitting Algorithm
1. Start by placing the first point in a new trail
2. Consider each successive point in the sequence in order
3. Perform a test for “inclusion” of this point in the current trail
4. if (the test pass)
5. Include the point in the current trail
6. Move to the next point
7. Goto 2
8. else
9. Close the current trail with the previous point as the last one
10. Start a new trail with only the current point
11. Goto 2
![Page 11: Video Trails: Representing and Visualizing Structure in Video Sequences](https://reader035.fdocuments.in/reader035/viewer/2022062500/56815928550346895dc650b4/html5/thumbnails/11.jpg)
“Inclusion Test”
Marginal Cost:Total cost per point in the trail Consider a clip with N frames Assume there are m points in the current
trail, denoted by set , and be the point being considered for inclusion
Define ,d is the dimensionality So the new marginal cost is
new marginal cost > previous one : not include
new marginal cost < previous one : include
![Page 12: Video Trails: Representing and Visualizing Structure in Video Sequences](https://reader035.fdocuments.in/reader035/viewer/2022062500/56815928550346895dc650b4/html5/thumbnails/12.jpg)
Example
![Page 13: Video Trails: Representing and Visualizing Structure in Video Sequences](https://reader035.fdocuments.in/reader035/viewer/2022062500/56815928550346895dc650b4/html5/thumbnails/13.jpg)
Example (close-up)
![Page 14: Video Trails: Representing and Visualizing Structure in Video Sequences](https://reader035.fdocuments.in/reader035/viewer/2022062500/56815928550346895dc650b4/html5/thumbnails/14.jpg)
The sequence of frames that yield the sparse transition between the two dense clusters
![Page 15: Video Trails: Representing and Visualizing Structure in Video Sequences](https://reader035.fdocuments.in/reader035/viewer/2022062500/56815928550346895dc650b4/html5/thumbnails/15.jpg)
Trail Classification
Classify each of those segmented trails into: Stationary (low activity) or Transitional (high activity)
Classification Criteria– Monotonicity W1=0.4 – Sparsity W2=0.3– Convex Hull Volume Ratio W3=0.2– MBR Shape W4=0.1
![Page 16: Video Trails: Representing and Visualizing Structure in Video Sequences](https://reader035.fdocuments.in/reader035/viewer/2022062500/56815928550346895dc650b4/html5/thumbnails/16.jpg)
Monotonicity
If a trail is (close to) monotonic, in some direction,it’s likely transitional projection of distance along k
projected distance ratio
the length of MBR dimension k Minimum projected distance ratio
![Page 17: Video Trails: Representing and Visualizing Structure in Video Sequences](https://reader035.fdocuments.in/reader035/viewer/2022062500/56815928550346895dc650b4/html5/thumbnails/17.jpg)
Monotonicity (Normalization)
Recall: W1 is the weight of monotonicity Tlow is the lower bound=1.1 Tup is the upper bound=2.0
![Page 18: Video Trails: Representing and Visualizing Structure in Video Sequences](https://reader035.fdocuments.in/reader035/viewer/2022062500/56815928550346895dc650b4/html5/thumbnails/18.jpg)
Sparsity
Sparsity: total MBR volume per point
Average Sparsity
Sparsity Ratio
Normalize
![Page 19: Video Trails: Representing and Visualizing Structure in Video Sequences](https://reader035.fdocuments.in/reader035/viewer/2022062500/56815928550346895dc650b4/html5/thumbnails/19.jpg)
Convex Hull Volume Ratio
The ratio of volume of the convex hull of points in a trail to the volume of MBR
Normalize
![Page 20: Video Trails: Representing and Visualizing Structure in Video Sequences](https://reader035.fdocuments.in/reader035/viewer/2022062500/56815928550346895dc650b4/html5/thumbnails/20.jpg)
MBR Shape
Cuboidal Planar Elongated
![Page 21: Video Trails: Representing and Visualizing Structure in Video Sequences](https://reader035.fdocuments.in/reader035/viewer/2022062500/56815928550346895dc650b4/html5/thumbnails/21.jpg)
Classification
![Page 22: Video Trails: Representing and Visualizing Structure in Video Sequences](https://reader035.fdocuments.in/reader035/viewer/2022062500/56815928550346895dc650b4/html5/thumbnails/22.jpg)
Gradual Transition Detection
Dissolves, Fades, Wipes Difficulty: activity arising from camera or
large object motion also yields trails similar to trails resulting from gradual edits
Filter out any kind of global motion leading to a transitional trail, Analysis global motion
![Page 23: Video Trails: Representing and Visualizing Structure in Video Sequences](https://reader035.fdocuments.in/reader035/viewer/2022062500/56815928550346895dc650b4/html5/thumbnails/23.jpg)
Results
![Page 24: Video Trails: Representing and Visualizing Structure in Video Sequences](https://reader035.fdocuments.in/reader035/viewer/2022062500/56815928550346895dc650b4/html5/thumbnails/24.jpg)
Conclusion
Provide a compact representation of a video sequence structure
Reduce a sequence MPEG frames to a trail of points in a low dimensional space
Segment trails and classify each segment as either stationary or transitional
Detect gradual edits