Video Analysis in Autonomous Systems: Data Analytics Challenges
-
Upload
krishnadubba -
Category
Data & Analytics
-
view
68 -
download
1
description
Transcript of Video Analysis in Autonomous Systems: Data Analytics Challenges
![Page 1: Video Analysis in Autonomous Systems: Data Analytics Challenges](https://reader033.fdocuments.in/reader033/viewer/2022052909/55990d351a28abc95b8b47d2/html5/thumbnails/1.jpg)
School of somethingFACULTY OF OTHERSchool of ComputingFACULTY OF ENGINEERING
Video Analysis in Autonomous Systems: Data Analytics Challenges
Krishna Dubba
Institute for Artificial Intelligence and Biological Systems
![Page 2: Video Analysis in Autonomous Systems: Data Analytics Challenges](https://reader033.fdocuments.in/reader033/viewer/2022052909/55990d351a28abc95b8b47d2/html5/thumbnails/2.jpg)
*
School of ComputingFACULTY OF ENGINEERING
Leeds Activity Analysis Group
Computer Vision (Prof. David Hogg)
Knowledge Representation and Reasoning (Prof. Tony Cohn)
![Page 3: Video Analysis in Autonomous Systems: Data Analytics Challenges](https://reader033.fdocuments.in/reader033/viewer/2022052909/55990d351a28abc95b8b47d2/html5/thumbnails/3.jpg)
*
School of ComputingFACULTY OF ENGINEERING
Motivation:
“We are drowning in data yet starving for knowledge” ~ John Naisbitt
![Page 4: Video Analysis in Autonomous Systems: Data Analytics Challenges](https://reader033.fdocuments.in/reader033/viewer/2022052909/55990d351a28abc95b8b47d2/html5/thumbnails/4.jpg)
*
School of ComputingFACULTY OF ENGINEERING
Motivation:● Are computers drowning in (video) data?
○ CCTV cameras○ Personal digital video cameras○ Video content on TV and Internet○ In future: Google glass, autonomous cars, personal
robots
![Page 5: Video Analysis in Autonomous Systems: Data Analytics Challenges](https://reader033.fdocuments.in/reader033/viewer/2022052909/55990d351a28abc95b8b47d2/html5/thumbnails/5.jpg)
*
School of ComputingFACULTY OF ENGINEERING
TrixiUniversity of Hamburg
LUCIELeeds University Cognitive Intelligent Entity
![Page 6: Video Analysis in Autonomous Systems: Data Analytics Challenges](https://reader033.fdocuments.in/reader033/viewer/2022052909/55990d351a28abc95b8b47d2/html5/thumbnails/6.jpg)
*
School of ComputingFACULTY OF ENGINEERING
Motivation:● Are computers starving for knowledge?
![Page 7: Video Analysis in Autonomous Systems: Data Analytics Challenges](https://reader033.fdocuments.in/reader033/viewer/2022052909/55990d351a28abc95b8b47d2/html5/thumbnails/7.jpg)
*
School of ComputingFACULTY OF ENGINEERING
Motivation:● Applications:
○ Security and Surveillance○ Intelligent autonomous systems (robots, cars etc.)○ Content based video retrieval (instead of text tags)○ Automatic script and commentary generation for videos
![Page 8: Video Analysis in Autonomous Systems: Data Analytics Challenges](https://reader033.fdocuments.in/reader033/viewer/2022052909/55990d351a28abc95b8b47d2/html5/thumbnails/8.jpg)
*
School of ComputingFACULTY OF ENGINEERING
Nature of Data:● Images ● Each pixel in image is a tuple (R,G,B)
![Page 9: Video Analysis in Autonomous Systems: Data Analytics Challenges](https://reader033.fdocuments.in/reader033/viewer/2022052909/55990d351a28abc95b8b47d2/html5/thumbnails/9.jpg)
*
School of ComputingFACULTY OF ENGINEERING
Nature of Data:● Videos (series of images)
![Page 10: Video Analysis in Autonomous Systems: Data Analytics Challenges](https://reader033.fdocuments.in/reader033/viewer/2022052909/55990d351a28abc95b8b47d2/html5/thumbnails/10.jpg)
*
School of ComputingFACULTY OF ENGINEERING
Nature of Data:● Videos (series of images)
Third Person View
![Page 11: Video Analysis in Autonomous Systems: Data Analytics Challenges](https://reader033.fdocuments.in/reader033/viewer/2022052909/55990d351a28abc95b8b47d2/html5/thumbnails/11.jpg)
*
School of ComputingFACULTY OF ENGINEERING
Nature of Data:● Videos (series of images)
Third Person View Ego-Centric View
![Page 12: Video Analysis in Autonomous Systems: Data Analytics Challenges](https://reader033.fdocuments.in/reader033/viewer/2022052909/55990d351a28abc95b8b47d2/html5/thumbnails/12.jpg)
*
School of ComputingFACULTY OF ENGINEERING
Nature of Data● Sensor data such as laser, depth data etc (Kinect).
![Page 13: Video Analysis in Autonomous Systems: Data Analytics Challenges](https://reader033.fdocuments.in/reader033/viewer/2022052909/55990d351a28abc95b8b47d2/html5/thumbnails/13.jpg)
*
School of ComputingFACULTY OF ENGINEERING
Nature of Data● Sensor data such as laser, depth data etc (Kinect).
![Page 14: Video Analysis in Autonomous Systems: Data Analytics Challenges](https://reader033.fdocuments.in/reader033/viewer/2022052909/55990d351a28abc95b8b47d2/html5/thumbnails/14.jpg)
*
School of ComputingFACULTY OF ENGINEERING
Nature of Data:
● Text (annotations, additional information from web)
● Verbal instructions
![Page 15: Video Analysis in Autonomous Systems: Data Analytics Challenges](https://reader033.fdocuments.in/reader033/viewer/2022052909/55990d351a28abc95b8b47d2/html5/thumbnails/15.jpg)
*
School of ComputingFACULTY OF ENGINEERING
Challenges:● Supervised, unsupervised and semi-supervised learning
![Page 16: Video Analysis in Autonomous Systems: Data Analytics Challenges](https://reader033.fdocuments.in/reader033/viewer/2022052909/55990d351a28abc95b8b47d2/html5/thumbnails/16.jpg)
*
School of ComputingFACULTY OF ENGINEERING
Challenges:● Supervised, unsupervised and semi-supervised learning● Data comes from multiple sources and mainly aimed at
humans - Multidisciplinary approach
![Page 17: Video Analysis in Autonomous Systems: Data Analytics Challenges](https://reader033.fdocuments.in/reader033/viewer/2022052909/55990d351a28abc95b8b47d2/html5/thumbnails/17.jpg)
*
School of ComputingFACULTY OF ENGINEERING
Challenges:● Supervised, unsupervised and semi-supervised learning● Data comes from multiple sources and mainly aimed at
humans - Multidisciplinary approach● Real time analysis: GPU processing
○ LUCIE has three kinects attached and needs a separate computer for each kinect.
![Page 18: Video Analysis in Autonomous Systems: Data Analytics Challenges](https://reader033.fdocuments.in/reader033/viewer/2022052909/55990d351a28abc95b8b47d2/html5/thumbnails/18.jpg)
*
School of ComputingFACULTY OF ENGINEERING
Challenges:● Supervised, unsupervised and semi-supervised learning● Data comes from multiple sources and mainly aimed at
humans - Multidisciplinary approach● Real time analysis: GPU processing
○ LUCIE has three kinects attached and needs a separate computer for each kinect.
● Integrating low-level representation and high level reasoning: Statistical Relational Models like Markov Logic Networks
![Page 19: Video Analysis in Autonomous Systems: Data Analytics Challenges](https://reader033.fdocuments.in/reader033/viewer/2022052909/55990d351a28abc95b8b47d2/html5/thumbnails/19.jpg)
*
School of ComputingFACULTY OF ENGINEERING
Challenges:● Supervised, unsupervised and semi-supervised learning● Data comes from multiple sources and mainly aimed at
humans - Multidisciplinary approach● Real time analysis: GPU processing
○ LUCIE has three kinects attached and needs a separate computer for each kinect.
● Integrating low-level representation and high level reasoning: Statistical Relational Models like Markov Logic Networks
● Online learning and how learning affects the state of the system.
![Page 20: Video Analysis in Autonomous Systems: Data Analytics Challenges](https://reader033.fdocuments.in/reader033/viewer/2022052909/55990d351a28abc95b8b47d2/html5/thumbnails/20.jpg)
*
Thank You
School of ComputingFACULTY OF ENGINEERING