LOWER ATTENTIVE REGION DETECTION FOR VIRTUAL CONTENT INSERTION IN BROADCAST VIDEO
-
Upload
erasmus-hall -
Category
Documents
-
view
34 -
download
0
description
Transcript of LOWER ATTENTIVE REGION DETECTION FOR VIRTUAL CONTENT INSERTION IN BROADCAST VIDEO
LOWER ATTENTIVE REGION DETECTION FOR VIRTUAL CONTENT INSERTION IN BROADCAST VIDEO
ICME 2008Huiying Liu, Shuqiang Jiang, Qingming Huang, Changsheng Xu
Outline
Introduction LAR detection framework LAR detection for sports video VCI methods Experiments Conclusion
Introduction
Introduction
Virtual Content Insertion (VCI) is an emerging application of video analysis and has been studied for several years.
The task of VCI is to make the inserted content attractive to the viewers and meanwhile not intrusive.
Compared with time point, spatial position is even more important as improper placement will make the insertion intrusive.
LAR detection
Framework In our work we perform LAR detection with
shot as the spatio-temporal context.
LAR detection
Attention analysis : static saliency map + motion saliency map
A region contains more perceptive information than a pixel or a block and can be obtained by image segmentation▪ Image segmentation DENSITY-BASED CLUSTERING [12]
[12] Q. Ye, W. Gao, W. Zeng, Color Image Segmentation Using Density-Based Clustering, ICME 2003.
LAR detection
Attention analysis Static saliency
[5] H. Liu, S. Jiang, Q. Huang, C. Xu, and W. Gao. “Region-Based Visual Attention Analysis with Its Application in Image Browsing on Small Displays”. ACM Multimedia,2007
feature contrast
area factor
spatial distance between the two regions
adjacency degree
central effect
LAR detection
Attention analysis Motion saliency [8]
▪ The MVS visualizes the MVF and provides us with a visual aid to understand.
[8] L-Y. Duan, M. Xu, Q. Tian, C-S. Xu, J. S. Jin, “A Unified Framework for Semantic Shot Classification in Sports Video” IEEE Multimedia,2005
Angle MagnitudeTexture
Hue SaturationValue
LAR detection
Information analysis : information map + entropy map The necessity of a region is evaluated by its
information and entropy in our work. An LAR must supplies less information and
contains less entropy at the same time. In this paper a simple and effective method
is adopted to calculate the information and entropy of each region and the result is used to evaluate the region’s necessity.
LAR detection
Information can be calculated as:
Entropy of the region can be calculated as:
The final LAR should be of less information and less entropy.
H :the normalized accumulative histogram of a shoth :the normalized histogram of a RegionA : the region’s area
Example of bottom-up LAR Detection
Fusing the 4 maps generates the frame attention map.
(a): The previous frame(b): Current frame(c): Motion field(d):Segmentation result(e): Static saliency map (f ): Motion intensity(g): Texture by DCT AC energy(h): HSV image of motion vector(i): Motion saliency map(j): Information map.(k): Entropy map(l): Final attentive map
LAR detection for sports video
Method Overview The shots here are classified into field view,
player view (coach , referee) and audience view.
Filed view and player view shots take up more than 90% of the video. So we perform LAR detection and VCI only on filed view and player view shots.
For player view shots LAR can be detected by using the bottom up method.
LAR detection for sports video
Field View LAR Detection Playfield detection, line and curve
detection, as well as object detection [2] can be combined to detect the LAR.
The GMM based method [9] is adopted to detect the playfield.
[2]C. Xu, K. W. Wan, S. H. Bui, Q. Tian, “Implanting Virtual Advertisement into Broadcast Soccer Video”, PCM, 2004[9] S. Jiang, Q. Ye, W. Gao, T. Huang. “A New Method to segment Playfield and Its Applications in Match Analysis in Sports Video”. ACM
Multimedia, 2004
Virtual content insertion
The methods for VCI include Static insertion▪ The content floats over the original video.
Dynamic insertion▪ Merge the content into the background.▪ Camera parameter must be reconstructed and
the content is adapted to the camera metric [10].
input Static insertion
dynamic insertion[10] X. Yu, X. Yan, T. T. P. Chi and L. F. Cheong, “Inserting 3D projected virtual content into broadcast tennis
video”, ACM Multimedia, 2006
Experiments
Test sequences (MPEG format) Broadcast TV play series (22 minutes) Sports1 soccer video and (45 minutes)
Sports2 tennis videos (26minutes)
For the field view shots, we insert the virtual content using dynamic method. The VCI method presented in before
estimates the camera motion and adapts the VC to the camera metric.
Experiments
Static insertion
dynamic insertion
Conclusion
In this paper, an LAR detection method is proposed using visual attention analysis and information theory.
In the future we will construct an integrated system of VCI and the temporal position for VCI.