1 Video Classification By: Maryam S. Mirian For: Multimedia & Pattern Recognition Joint Courses...
-
Upload
philip-walters -
Category
Documents
-
view
221 -
download
1
Transcript of 1 Video Classification By: Maryam S. Mirian For: Multimedia & Pattern Recognition Joint Courses...
![Page 1: 1 Video Classification By: Maryam S. Mirian For: Multimedia & Pattern Recognition Joint Courses Project.](https://reader036.fdocuments.in/reader036/viewer/2022062407/56649e315503460f94b2279b/html5/thumbnails/1.jpg)
1
Video Classification
By: Maryam S. MirianFor: Multimedia & Pattern Recognition Joint Courses Project
![Page 2: 1 Video Classification By: Maryam S. Mirian For: Multimedia & Pattern Recognition Joint Courses Project.](https://reader036.fdocuments.in/reader036/viewer/2022062407/56649e315503460f94b2279b/html5/thumbnails/2.jpg)
/249
Outline
What is Video Classification? Straightforward or Difficult? What is its Applications? What are its methods? Review of Video Classification
Methods What is my own Project, exactly?
![Page 3: 1 Video Classification By: Maryam S. Mirian For: Multimedia & Pattern Recognition Joint Courses Project.](https://reader036.fdocuments.in/reader036/viewer/2022062407/56649e315503460f94b2279b/html5/thumbnails/3.jpg)
/349
What is Video Classification?
Classify a Video (Shot) into one of Nc predefined Classes:
Indoor / outdoor News / Sports …
![Page 4: 1 Video Classification By: Maryam S. Mirian For: Multimedia & Pattern Recognition Joint Courses Project.](https://reader036.fdocuments.in/reader036/viewer/2022062407/56649e315503460f94b2279b/html5/thumbnails/4.jpg)
/449
Is Video Classification Difficult? Why?
YES, Because: Data Stream is a Multi-dimensional
signal. It has a subjective nature.
![Page 5: 1 Video Classification By: Maryam S. Mirian For: Multimedia & Pattern Recognition Joint Courses Project.](https://reader036.fdocuments.in/reader036/viewer/2022062407/56649e315503460f94b2279b/html5/thumbnails/5.jpg)
5
Classification
![Page 6: 1 Video Classification By: Maryam S. Mirian For: Multimedia & Pattern Recognition Joint Courses Project.](https://reader036.fdocuments.in/reader036/viewer/2022062407/56649e315503460f94b2279b/html5/thumbnails/6.jpg)
/649
Required Steps for Classification
ClassificationFeature
ExtractionFeature
Reduction
The most Important and the most difficult
part
Using Methods
like: PCA, LDA
Object
Observations
ClassLabels
![Page 7: 1 Video Classification By: Maryam S. Mirian For: Multimedia & Pattern Recognition Joint Courses Project.](https://reader036.fdocuments.in/reader036/viewer/2022062407/56649e315503460f94b2279b/html5/thumbnails/7.jpg)
/749
Methods of Classification
Bayesian Classification kNN Classification Neural Classification
MLP RBF
Classification based on Support Vector Machines
Rule-based Classification
![Page 8: 1 Video Classification By: Maryam S. Mirian For: Multimedia & Pattern Recognition Joint Courses Project.](https://reader036.fdocuments.in/reader036/viewer/2022062407/56649e315503460f94b2279b/html5/thumbnails/8.jpg)
/849
Bayesian Decision Making So, x belongs to w2
![Page 9: 1 Video Classification By: Maryam S. Mirian For: Multimedia & Pattern Recognition Joint Courses Project.](https://reader036.fdocuments.in/reader036/viewer/2022062407/56649e315503460f94b2279b/html5/thumbnails/9.jpg)
/949
Methods of Classification
Bayesian Classification kNN Classification Neural Classification
MLP RBF
Classification based on Support Vector Machines
Rule-based Classification
![Page 10: 1 Video Classification By: Maryam S. Mirian For: Multimedia & Pattern Recognition Joint Courses Project.](https://reader036.fdocuments.in/reader036/viewer/2022062407/56649e315503460f94b2279b/html5/thumbnails/10.jpg)
/1049
kNN Decision Making k = 5 ,2 Red
NeighborWhile 3 Black
Neighbor, so X should be
Black!
![Page 11: 1 Video Classification By: Maryam S. Mirian For: Multimedia & Pattern Recognition Joint Courses Project.](https://reader036.fdocuments.in/reader036/viewer/2022062407/56649e315503460f94b2279b/html5/thumbnails/11.jpg)
/1149
Methods of Classification
Bayesian Classification kNN Classification Neural Classification
MLP RBF
Classification based on Support Vector Machines
Rule-based Classification
![Page 12: 1 Video Classification By: Maryam S. Mirian For: Multimedia & Pattern Recognition Joint Courses Project.](https://reader036.fdocuments.in/reader036/viewer/2022062407/56649e315503460f94b2279b/html5/thumbnails/12.jpg)
/1249
MLP Classifier
![Page 13: 1 Video Classification By: Maryam S. Mirian For: Multimedia & Pattern Recognition Joint Courses Project.](https://reader036.fdocuments.in/reader036/viewer/2022062407/56649e315503460f94b2279b/html5/thumbnails/13.jpg)
/1349
Video Content Analysis
![Page 14: 1 Video Classification By: Maryam S. Mirian For: Multimedia & Pattern Recognition Joint Courses Project.](https://reader036.fdocuments.in/reader036/viewer/2022062407/56649e315503460f94b2279b/html5/thumbnails/14.jpg)
/1449
Applications of Automatic video classification
Automatic Video segmentation content based retrieval browsing and retrieving digitized video identifying close-up video frames before
running a computationally expensive face recognizer.
effective management of ever-increasing amount of broadcast news video: personalization of news video.
![Page 15: 1 Video Classification By: Maryam S. Mirian For: Multimedia & Pattern Recognition Joint Courses Project.](https://reader036.fdocuments.in/reader036/viewer/2022062407/56649e315503460f94b2279b/html5/thumbnails/15.jpg)
/1549
Classify Shot or Video?
One effective way to organize the video is to segment the video into small, single-story units and classify these units according to their semantics.
A shot represents a contiguous sequence of visually similar frames. It is a syntactical representation and does not usually convey any coherent semantics to the users.
![Page 16: 1 Video Classification By: Maryam S. Mirian For: Multimedia & Pattern Recognition Joint Courses Project.](https://reader036.fdocuments.in/reader036/viewer/2022062407/56649e315503460f94b2279b/html5/thumbnails/16.jpg)
16
Looking @Video Classification
![Page 17: 1 Video Classification By: Maryam S. Mirian For: Multimedia & Pattern Recognition Joint Courses Project.](https://reader036.fdocuments.in/reader036/viewer/2022062407/56649e315503460f94b2279b/html5/thumbnails/17.jpg)
/1749
Ide et al. [1998] Problem Domain: News video Features:
Videotext motion face
segmented the video into shots used clustering techniques classify each shot into 1 of 5 classes: Speech/report,
Anchor, Walking, Gathering, and Computer graphics shots.
Quite simple but seems effective for this restricted class of problems.
![Page 18: 1 Video Classification By: Maryam S. Mirian For: Multimedia & Pattern Recognition Joint Courses Project.](https://reader036.fdocuments.in/reader036/viewer/2022062407/56649e315503460f94b2279b/html5/thumbnails/18.jpg)
/1849
Huang et al. [1999] Problem Domain: TV Programs
news report weather forecast Commercials basketball games football games
Features: Audio Color motion
![Page 19: 1 Video Classification By: Maryam S. Mirian For: Multimedia & Pattern Recognition Joint Courses Project.](https://reader036.fdocuments.in/reader036/viewer/2022062407/56649e315503460f94b2279b/html5/thumbnails/19.jpg)
/1949
Chen and Wong [2001] Problem Domain:
news video: News Weather Reporting Commercials Basketball Football
Features: Motion Color text caption cut rate
used a rule-based approach
![Page 20: 1 Video Classification By: Maryam S. Mirian For: Multimedia & Pattern Recognition Joint Courses Project.](https://reader036.fdocuments.in/reader036/viewer/2022062407/56649e315503460f94b2279b/html5/thumbnails/20.jpg)
/2049
Looking @ Lekha Chaisorn et.al [2002] in More Details
![Page 21: 1 Video Classification By: Maryam S. Mirian For: Multimedia & Pattern Recognition Joint Courses Project.](https://reader036.fdocuments.in/reader036/viewer/2022062407/56649e315503460f94b2279b/html5/thumbnails/21.jpg)
/2149
Basic Ideas Proposes a two-level, multi-modal framework. The video is analyzed at the shot and story unit (or scene)
levels. At the shot level, a Decision Tree to classify the shot into
one of 13 pre-defined categories is employed. At the scene level, the HMM (Hidden Markov Models)
analysis is used to eliminate shot classification errors Results indicate that a high accuracy of over 95 % for shot
classification can be achieved. The use of HMM analysis helps to improve the accuracy of
the shot classification and achieve over 89% accuracy on story segmentation.
![Page 22: 1 Video Classification By: Maryam S. Mirian For: Multimedia & Pattern Recognition Joint Courses Project.](https://reader036.fdocuments.in/reader036/viewer/2022062407/56649e315503460f94b2279b/html5/thumbnails/22.jpg)
/2249
Predefined Classes
![Page 23: 1 Video Classification By: Maryam S. Mirian For: Multimedia & Pattern Recognition Joint Courses Project.](https://reader036.fdocuments.in/reader036/viewer/2022062407/56649e315503460f94b2279b/html5/thumbnails/23.jpg)
/2349
Features in Shot Level Low-level Visual Content Feature
Color Histogram Temporal Features
Background scene change Speaker change Audio Motion activity Shot duration
High-level Object-based features Face Shot type Videotext Centralized Videotext
![Page 24: 1 Video Classification By: Maryam S. Mirian For: Multimedia & Pattern Recognition Joint Courses Project.](https://reader036.fdocuments.in/reader036/viewer/2022062407/56649e315503460f94b2279b/html5/thumbnails/24.jpg)
/2449
Feature vector of a shot
Si = (a, m, d, f, s, t, c) a the class of audio, a ∈{ t=speech, m=music, s=silence,
n =noise, tn = speech + noise, tm= speech + music, mn=music+noise}
m the motion activity, m ∈{l=low, m=medium, h=high} d the shot duration, d ∈{s=short, m=medium, l=long} f the number of faces, Ν ∈ f s the shot type, s ∈{c= closed-up, m=medium, l=long,
u=unknown} t the number of lines of text in the scene, Ν ∈ t c set to “true” if the videotexts present are centralized, c
∈{t=true, f=false}
![Page 25: 1 Video Classification By: Maryam S. Mirian For: Multimedia & Pattern Recognition Joint Courses Project.](https://reader036.fdocuments.in/reader036/viewer/2022062407/56649e315503460f94b2279b/html5/thumbnails/25.jpg)
/2549
Decision Tree for Shot Classification
![Page 26: 1 Video Classification By: Maryam S. Mirian For: Multimedia & Pattern Recognition Joint Courses Project.](https://reader036.fdocuments.in/reader036/viewer/2022062407/56649e315503460f94b2279b/html5/thumbnails/26.jpg)
26
Reading these papers, I decided about My own Project….
![Page 27: 1 Video Classification By: Maryam S. Mirian For: Multimedia & Pattern Recognition Joint Courses Project.](https://reader036.fdocuments.in/reader036/viewer/2022062407/56649e315503460f94b2279b/html5/thumbnails/27.jpg)
/2749
About Problem Domain…
Sport Classification seems OK Interesting Enough It is helpful for Sports-Lovers
![Page 28: 1 Video Classification By: Maryam S. Mirian For: Multimedia & Pattern Recognition Joint Courses Project.](https://reader036.fdocuments.in/reader036/viewer/2022062407/56649e315503460f94b2279b/html5/thumbnails/28.jpg)
/2849
About Extracting features…. Features used in video analysis: color,texture,shape,motion vector… Criteria of choosing features : they should
have similar statistical behavior across time
Color histogram: simple and robust Motion vectors:invariance to color and
light
![Page 29: 1 Video Classification By: Maryam S. Mirian For: Multimedia & Pattern Recognition Joint Courses Project.](https://reader036.fdocuments.in/reader036/viewer/2022062407/56649e315503460f94b2279b/html5/thumbnails/29.jpg)
/2949
So, My Own Project is Sports Video Classifications : Football, Basketball, ….
(Those Well-defined sports, I can find Video On!) Steps I should take:
Finding or Gathering a Video Collection Shot Detection Feature Extraction :
Key Frame (s) Extraction: Selecting Middle Shot I-Frame Use of Clustering …
Motion Vector–based Features Straight Lines Detection
Design a Classifier Test the Approach
![Page 30: 1 Video Classification By: Maryam S. Mirian For: Multimedia & Pattern Recognition Joint Courses Project.](https://reader036.fdocuments.in/reader036/viewer/2022062407/56649e315503460f94b2279b/html5/thumbnails/30.jpg)
/3049
Looking @Ekin,Tekalp[2003]
one Research on Football Video Classification
![Page 31: 1 Video Classification By: Maryam S. Mirian For: Multimedia & Pattern Recognition Joint Courses Project.](https://reader036.fdocuments.in/reader036/viewer/2022062407/56649e315503460f94b2279b/html5/thumbnails/31.jpg)
/3149
Cinematic result from common video composition
and production rules. shot types, camera motions and replays.
Object-based Described by their spatial, e.g., color,
texture, and shape, and spatio-temporal features, such as object motions and interactions
Features
![Page 32: 1 Video Classification By: Maryam S. Mirian For: Multimedia & Pattern Recognition Joint Courses Project.](https://reader036.fdocuments.in/reader036/viewer/2022062407/56649e315503460f94b2279b/html5/thumbnails/32.jpg)
/3249
Robust Dominant Color Region Detection
A soccer field has one distinct dominant color (a tone of green) that may vary from stadium to stadium, and also due to weather and lighting conditions within the same stadium.
The statistics of this dominant color, in the HSI space, are learned by the system at start-up, and then automatically updated to adapt to temporal variations.
![Page 33: 1 Video Classification By: Maryam S. Mirian For: Multimedia & Pattern Recognition Joint Courses Project.](https://reader036.fdocuments.in/reader036/viewer/2022062407/56649e315503460f94b2279b/html5/thumbnails/33.jpg)
/3349
Shot classification Long Shot
A long shot displays the global view of the field. In-Field Medium Shot
a whole human body is usually visible. Close-Up Shot
shows the above-waist view of one person Out of Field Shot
The audience, coach, and other shots
![Page 34: 1 Video Classification By: Maryam S. Mirian For: Multimedia & Pattern Recognition Joint Courses Project.](https://reader036.fdocuments.in/reader036/viewer/2022062407/56649e315503460f94b2279b/html5/thumbnails/34.jpg)
/3449
![Page 35: 1 Video Classification By: Maryam S. Mirian For: Multimedia & Pattern Recognition Joint Courses Project.](https://reader036.fdocuments.in/reader036/viewer/2022062407/56649e315503460f94b2279b/html5/thumbnails/35.jpg)
/3549
How Extend to Shot from a Frame?
Due to the computational simplicity they find the class of every frame in a shot and assign the shot class to the label of the majority of frames.
![Page 36: 1 Video Classification By: Maryam S. Mirian For: Multimedia & Pattern Recognition Joint Courses Project.](https://reader036.fdocuments.in/reader036/viewer/2022062407/56649e315503460f94b2279b/html5/thumbnails/36.jpg)
/3649
Decision Schema based on G
The first stage uses G value and two thresholds, TcloseUp and Tmedium to determine the frame view label.
![Page 37: 1 Video Classification By: Maryam S. Mirian For: Multimedia & Pattern Recognition Joint Courses Project.](https://reader036.fdocuments.in/reader036/viewer/2022062407/56649e315503460f94b2279b/html5/thumbnails/37.jpg)
/3849
Soccer Eevent Detection
Goal Detection Referee Detection
Controversial calls, such as red-yellow cards and penalties
Penalty Box Detection
![Page 38: 1 Video Classification By: Maryam S. Mirian For: Multimedia & Pattern Recognition Joint Courses Project.](https://reader036.fdocuments.in/reader036/viewer/2022062407/56649e315503460f94b2279b/html5/thumbnails/38.jpg)
/3949
Goal Detection Occurrence of a goal is generally
followed by a special pattern of cinematic features. A goal event leads to a break in the
game. one or more close-up views of the actors
of the goal event. show one or more replay(s) the restart of the game is usually
captured by a long shot.
![Page 39: 1 Video Classification By: Maryam S. Mirian For: Multimedia & Pattern Recognition Joint Courses Project.](https://reader036.fdocuments.in/reader036/viewer/2022062407/56649e315503460f94b2279b/html5/thumbnails/39.jpg)
/4049
![Page 40: 1 Video Classification By: Maryam S. Mirian For: Multimedia & Pattern Recognition Joint Courses Project.](https://reader036.fdocuments.in/reader036/viewer/2022062407/56649e315503460f94b2279b/html5/thumbnails/40.jpg)
/4149
Referee Detection
Assumed that there is, a single referee in a: medium out of field close-up shot So no search for a referee in a long shot
![Page 41: 1 Video Classification By: Maryam S. Mirian For: Multimedia & Pattern Recognition Joint Courses Project.](https://reader036.fdocuments.in/reader036/viewer/2022062407/56649e315503460f94b2279b/html5/thumbnails/41.jpg)
/4249
Penalty Box Detection
Field lines in a long view can be used to localize the view and/or register the current frame on the standard field model
![Page 42: 1 Video Classification By: Maryam S. Mirian For: Multimedia & Pattern Recognition Joint Courses Project.](https://reader036.fdocuments.in/reader036/viewer/2022062407/56649e315503460f94b2279b/html5/thumbnails/42.jpg)
/4349
Interesting Summaries
Goal summaries summaries with Referee and
Penalty box objects
![Page 43: 1 Video Classification By: Maryam S. Mirian For: Multimedia & Pattern Recognition Joint Courses Project.](https://reader036.fdocuments.in/reader036/viewer/2022062407/56649e315503460f94b2279b/html5/thumbnails/43.jpg)
/4449
Adaptation of Parameters
Parameters Tcolor in dominant color region detection TcloseUp and Tmedium in shot classification referee color statistics
The training stage can be performed in a very short time to find Mean and Variance of a Normal pdf.
![Page 44: 1 Video Classification By: Maryam S. Mirian For: Multimedia & Pattern Recognition Joint Courses Project.](https://reader036.fdocuments.in/reader036/viewer/2022062407/56649e315503460f94b2279b/html5/thumbnails/44.jpg)
/4549
Results for High-Level Analysis and Summarization
Goal detection results
![Page 45: 1 Video Classification By: Maryam S. Mirian For: Multimedia & Pattern Recognition Joint Courses Project.](https://reader036.fdocuments.in/reader036/viewer/2022062407/56649e315503460f94b2279b/html5/thumbnails/45.jpg)
/4649
Results for High-Level Analysis and Summarization(2)
Referee detection results
![Page 46: 1 Video Classification By: Maryam S. Mirian For: Multimedia & Pattern Recognition Joint Courses Project.](https://reader036.fdocuments.in/reader036/viewer/2022062407/56649e315503460f94b2279b/html5/thumbnails/46.jpg)
/4749
Results for High-Level Analysis and Summarization(3)
Penalty box detection results
![Page 47: 1 Video Classification By: Maryam S. Mirian For: Multimedia & Pattern Recognition Joint Courses Project.](https://reader036.fdocuments.in/reader036/viewer/2022062407/56649e315503460f94b2279b/html5/thumbnails/47.jpg)
/4849
References Automatic soccer video analysis and summarization, in
Symp. Electronic Imaging: Science and Technology: Storage and Retrieval for Image and Video Databases IV, IS&T/SPI03, Jan. 2003, CA.
“The Segmentation and Classification of Story Boundaries In News Video”, Proceeding of 6th IFIP working conference on Visual Database Systems-
VDB6 2002, Australia 2002 Pattern Classification, by Duda, Hart, and Stork,
2000
![Page 48: 1 Video Classification By: Maryam S. Mirian For: Multimedia & Pattern Recognition Joint Courses Project.](https://reader036.fdocuments.in/reader036/viewer/2022062407/56649e315503460f94b2279b/html5/thumbnails/48.jpg)
49
Thanks for Your Attention
Any Question or Comment?