Foundations & Core in Computer Vision: A System Perspective
description
Transcript of Foundations & Core in Computer Vision: A System Perspective
![Page 1: Foundations & Core in Computer Vision: A System Perspective](https://reader036.fdocuments.in/reader036/viewer/2022062302/5681665b550346895dd9dc7e/html5/thumbnails/1.jpg)
Foundations & Core in Computer Vision: A System Perspective
Ce Liu
Microsoft Research New England
![Page 2: Foundations & Core in Computer Vision: A System Perspective](https://reader036.fdocuments.in/reader036/viewer/2022062302/5681665b550346895dd9dc7e/html5/thumbnails/2.jpg)
Vision vs. Learning
• Computer vision: visual application of machine learning?
• Data features algorithms data
• ML: design algorithms given input and output data
• CV: find the best input and output data given available algorithms
![Page 3: Foundations & Core in Computer Vision: A System Perspective](https://reader036.fdocuments.in/reader036/viewer/2022062302/5681665b550346895dd9dc7e/html5/thumbnails/3.jpg)
Theoretical vs. Experimental
• Theoretical analysis of a visual system– Best & worst cases – Average performance
• Theoretical analysis is challenging as many visual distributions are hard to model (signal processing: 2nd order processes, machine learning: exponential families)
• Experimental approach: full spectrum of system performance as a function of the amount of data, annotation, number of categories, noise, and other conditions
![Page 4: Foundations & Core in Computer Vision: A System Perspective](https://reader036.fdocuments.in/reader036/viewer/2022062302/5681665b550346895dd9dc7e/html5/thumbnails/4.jpg)
Quality vs. Speed
• HD videos, billions of images to index• Real time & 90% vs. one hour per frame & 95%?• Mechanism to balance quality and speed in modeling
![Page 5: Foundations & Core in Computer Vision: A System Perspective](https://reader036.fdocuments.in/reader036/viewer/2022062302/5681665b550346895dd9dc7e/html5/thumbnails/5.jpg)
Automatic vs. semi-automatic
• Common review feedback: parameters are hand-tuned; not clear how to set the parameters
• Vision system user feedback: I don’t know how to tweak parameters!
• Computer-oriented vs. human-oriented representations
• Human-in-the-loop (collaborative) vision– How to optimally use humans (what, which and how
accurate) beyond traditional active learning– Model design by crowd-sourcing– Learning by subtraction
![Page 6: Foundations & Core in Computer Vision: A System Perspective](https://reader036.fdocuments.in/reader036/viewer/2022062302/5681665b550346895dd9dc7e/html5/thumbnails/6.jpg)
Algorithms vs. Sensors
• Two approaches to solving a vision problem– Look at images, design algorithms, experiment, improve…– Look at cameras, design new/better sensors, …
• Cameras for full-spectrum, high res, low noise, depth, motion, occluding boundary, object, …
• What’s the optimal sensor/device for solving a vision problem?
• What’s the limit of sensors?
![Page 7: Foundations & Core in Computer Vision: A System Perspective](https://reader036.fdocuments.in/reader036/viewer/2022062302/5681665b550346895dd9dc7e/html5/thumbnails/7.jpg)
Thank you!
Ce Liu
Microsoft Research New England