Looking for a Needle in Video Haystack #appsummit14
-
Upload
us-ignite -
Category
Technology
-
view
306 -
download
1
description
Transcript of Looking for a Needle in Video Haystack #appsummit14
![Page 1: Looking for a Needle in Video Haystack #appsummit14](https://reader034.fdocuments.in/reader034/viewer/2022051609/5473f104b4af9fa90a8b54df/html5/thumbnails/1.jpg)
Finding the Needle in the Video Haystack
Dr. Gerald Friedland Director Audio and Multimedia Research
International Computer Science Institute Berkeley, CA [email protected]
![Page 2: Looking for a Needle in Video Haystack #appsummit14](https://reader034.fdocuments.in/reader034/viewer/2022051609/5473f104b4af9fa90a8b54df/html5/thumbnails/2.jpg)
The Internet is Multimedia
2
![Page 3: Looking for a Needle in Video Haystack #appsummit14](https://reader034.fdocuments.in/reader034/viewer/2022051609/5473f104b4af9fa90a8b54df/html5/thumbnails/3.jpg)
Multimedia in the Internet is Growing
3
![Page 4: Looking for a Needle in Video Haystack #appsummit14](https://reader034.fdocuments.in/reader034/viewer/2022051609/5473f104b4af9fa90a8b54df/html5/thumbnails/4.jpg)
Multimedia in the Internet is Growing
3
• YouTube alone claims 48 72 100 hours video uploads every minute.
![Page 5: Looking for a Needle in Video Haystack #appsummit14](https://reader034.fdocuments.in/reader034/viewer/2022051609/5473f104b4af9fa90a8b54df/html5/thumbnails/5.jpg)
Multimedia in the Internet is Growing
3
• YouTube alone claims 48 72 100 hours video uploads every minute.
• Youku (Chinese YouTube) claims 80k video uploads per day
![Page 6: Looking for a Needle in Video Haystack #appsummit14](https://reader034.fdocuments.in/reader034/viewer/2022051609/5473f104b4af9fa90a8b54df/html5/thumbnails/6.jpg)
Multimedia in the Internet is Growing
3
• YouTube alone claims 48 72 100 hours video uploads every minute.
• Youku (Chinese YouTube) claims 80k video uploads per day
• Flickr, Instagram, Liveleak, Vimeo...
![Page 7: Looking for a Needle in Video Haystack #appsummit14](https://reader034.fdocuments.in/reader034/viewer/2022051609/5473f104b4af9fa90a8b54df/html5/thumbnails/7.jpg)
4
![Page 8: Looking for a Needle in Video Haystack #appsummit14](https://reader034.fdocuments.in/reader034/viewer/2022051609/5473f104b4af9fa90a8b54df/html5/thumbnails/8.jpg)
The Opportunity
5
![Page 9: Looking for a Needle in Video Haystack #appsummit14](https://reader034.fdocuments.in/reader034/viewer/2022051609/5473f104b4af9fa90a8b54df/html5/thumbnails/9.jpg)
The Opportunity
• Consumer-Produced Multimedia allows empirical studies at never-before-seen scale:
5
![Page 10: Looking for a Needle in Video Haystack #appsummit14](https://reader034.fdocuments.in/reader034/viewer/2022051609/5473f104b4af9fa90a8b54df/html5/thumbnails/10.jpg)
The Opportunity
• Consumer-Produced Multimedia allows empirical studies at never-before-seen scale:– sociology,
5
![Page 11: Looking for a Needle in Video Haystack #appsummit14](https://reader034.fdocuments.in/reader034/viewer/2022051609/5473f104b4af9fa90a8b54df/html5/thumbnails/11.jpg)
The Opportunity
• Consumer-Produced Multimedia allows empirical studies at never-before-seen scale:– sociology, – medicine,
5
![Page 12: Looking for a Needle in Video Haystack #appsummit14](https://reader034.fdocuments.in/reader034/viewer/2022051609/5473f104b4af9fa90a8b54df/html5/thumbnails/12.jpg)
The Opportunity
• Consumer-Produced Multimedia allows empirical studies at never-before-seen scale:– sociology, – medicine,– economics,
5
![Page 13: Looking for a Needle in Video Haystack #appsummit14](https://reader034.fdocuments.in/reader034/viewer/2022051609/5473f104b4af9fa90a8b54df/html5/thumbnails/13.jpg)
The Opportunity
• Consumer-Produced Multimedia allows empirical studies at never-before-seen scale:– sociology, – medicine,– economics, – …
5
![Page 14: Looking for a Needle in Video Haystack #appsummit14](https://reader034.fdocuments.in/reader034/viewer/2022051609/5473f104b4af9fa90a8b54df/html5/thumbnails/14.jpg)
The Opportunity
• Consumer-Produced Multimedia allows empirical studies at never-before-seen scale:– sociology, – medicine,– economics, – …
• Problem: Videos need to be searchable beyond keywords.
5
![Page 15: Looking for a Needle in Video Haystack #appsummit14](https://reader034.fdocuments.in/reader034/viewer/2022051609/5473f104b4af9fa90a8b54df/html5/thumbnails/15.jpg)
The Opportunity
• Consumer-Produced Multimedia allows empirical studies at never-before-seen scale:– sociology, – medicine,– economics, – …
• Problem: Videos need to be searchable beyond keywords.
• 5
![Page 16: Looking for a Needle in Video Haystack #appsummit14](https://reader034.fdocuments.in/reader034/viewer/2022051609/5473f104b4af9fa90a8b54df/html5/thumbnails/16.jpg)
Our Approach
6
Ball soundMale voice (near)
Child’s voice (distant)Child’s whoop (distant)
Room tone
Cameron learns to catch (http://www.youtube.com/watch?v=o6QXcP3Xvus)
![Page 17: Looking for a Needle in Video Haystack #appsummit14](https://reader034.fdocuments.in/reader034/viewer/2022051609/5473f104b4af9fa90a8b54df/html5/thumbnails/17.jpg)
Our Approach
Multimodal exploitation of video content, including audio and temporal information.
6
Ball soundMale voice (near)
Child’s voice (distant)Child’s whoop (distant)
Room tone
Cameron learns to catch (http://www.youtube.com/watch?v=o6QXcP3Xvus)
![Page 18: Looking for a Needle in Video Haystack #appsummit14](https://reader034.fdocuments.in/reader034/viewer/2022051609/5473f104b4af9fa90a8b54df/html5/thumbnails/18.jpg)
Location Estimation
7
J. Choi, G. Friedland, V. Ekambaram, K. Ramchandran: "Multimodal Location Estimation of Consumer Media: Dealing with Sparse Training Data," in Proceedings of IEEE ICME 2012, Melbourne, Australia, July 2012.
![Page 19: Looking for a Needle in Video Haystack #appsummit14](https://reader034.fdocuments.in/reader034/viewer/2022051609/5473f104b4af9fa90a8b54df/html5/thumbnails/19.jpg)
Bayesian graphical framework
8
{berkeley, sathergate, campanile}
{berkeley, haas}
{campanile} {campanile, haas}
Node: Geoloca7on of the image
Edge: Correlated loca7ons (e.g. common tag)
Edge Poten,al: Strength of an edge, (e.g. posterior distribu7on of loca7ons given common tags)
p(xi, xj |{tki } � {tkj })
p(xj |{tkj })p(xi|{tki })