Applications in Computer Vision Machine Learning for · 2016. 12. 2. · FlowNet: Learning Optical...
Transcript of Applications in Computer Vision Machine Learning for · 2016. 12. 2. · FlowNet: Learning Optical...
![Page 1: Applications in Computer Vision Machine Learning for · 2016. 12. 2. · FlowNet: Learning Optical Flow with Convolutional Networks Philipp Fischer, Alexey Dosovitskiy, Eddy Ilg,](https://reader036.fdocuments.in/reader036/viewer/2022071401/60ec716a7a18fe32d96fbf45/html5/thumbnails/1.jpg)
Computer Vision Group Prof. Daniel CremersComputer Vision Group Prof. Daniel Cremers
Machine Learning for Applications in Computer Vision
Neural Networks andDeep Learning
Philip Häusser
![Page 2: Applications in Computer Vision Machine Learning for · 2016. 12. 2. · FlowNet: Learning Optical Flow with Convolutional Networks Philipp Fischer, Alexey Dosovitskiy, Eddy Ilg,](https://reader036.fdocuments.in/reader036/viewer/2022071401/60ec716a7a18fe32d96fbf45/html5/thumbnails/2.jpg)
Philip HaeusserComputer Vision Group
Machine Learning for Computer Vision
Philip HaeusserComputer Vision Group
Machine Learning for Computer Vision
Recap: What is Learning?• problem (“classification”)
• data (images + labels)• model (linear regression)• cost function• optimization algorithm
2
![Page 3: Applications in Computer Vision Machine Learning for · 2016. 12. 2. · FlowNet: Learning Optical Flow with Convolutional Networks Philipp Fischer, Alexey Dosovitskiy, Eddy Ilg,](https://reader036.fdocuments.in/reader036/viewer/2022071401/60ec716a7a18fe32d96fbf45/html5/thumbnails/3.jpg)
Philip HaeusserComputer Vision Group
Machine Learning for Computer Vision
Philip HaeusserComputer Vision Group
Machine Learning for Computer Vision
Recap: What is Learning?• problem (“classification”)
• data (images + labels)• model (linear regression)• cost function• optimization algorithm
3
What is “Deep”?
![Page 4: Applications in Computer Vision Machine Learning for · 2016. 12. 2. · FlowNet: Learning Optical Flow with Convolutional Networks Philipp Fischer, Alexey Dosovitskiy, Eddy Ilg,](https://reader036.fdocuments.in/reader036/viewer/2022071401/60ec716a7a18fe32d96fbf45/html5/thumbnails/4.jpg)
Philip HaeusserComputer Vision Group
Machine Learning for Computer Vision
Philip HaeusserComputer Vision Group
Machine Learning for Computer Vision
Recap: What is Learning?• problem (“classification”)
• data (images + labels)• model (linear regression)• cost function• optimization algorithm
4
What is “Deep”?
![Page 5: Applications in Computer Vision Machine Learning for · 2016. 12. 2. · FlowNet: Learning Optical Flow with Convolutional Networks Philipp Fischer, Alexey Dosovitskiy, Eddy Ilg,](https://reader036.fdocuments.in/reader036/viewer/2022071401/60ec716a7a18fe32d96fbf45/html5/thumbnails/5.jpg)
Philip HaeusserComputer Vision Group
Machine Learning for Computer Vision
Perceptron• The human brain (1010 cells) is the archetype of neural networks
http://home.agh.edu.pl/~vlsi/AI/intro/
5
![Page 6: Applications in Computer Vision Machine Learning for · 2016. 12. 2. · FlowNet: Learning Optical Flow with Convolutional Networks Philipp Fischer, Alexey Dosovitskiy, Eddy Ilg,](https://reader036.fdocuments.in/reader036/viewer/2022071401/60ec716a7a18fe32d96fbf45/html5/thumbnails/6.jpg)
Philip HaeusserComputer Vision Group
Machine Learning for Computer Vision
Neuron Activations• Different type of activation functions
Threshold activation (binary classifier)
6
![Page 7: Applications in Computer Vision Machine Learning for · 2016. 12. 2. · FlowNet: Learning Optical Flow with Convolutional Networks Philipp Fischer, Alexey Dosovitskiy, Eddy Ilg,](https://reader036.fdocuments.in/reader036/viewer/2022071401/60ec716a7a18fe32d96fbf45/html5/thumbnails/7.jpg)
Philip HaeusserComputer Vision Group
Machine Learning for Computer Vision
Neuron Activations• Different type of activation functions
Sigmoid activation
7
![Page 8: Applications in Computer Vision Machine Learning for · 2016. 12. 2. · FlowNet: Learning Optical Flow with Convolutional Networks Philipp Fischer, Alexey Dosovitskiy, Eddy Ilg,](https://reader036.fdocuments.in/reader036/viewer/2022071401/60ec716a7a18fe32d96fbf45/html5/thumbnails/8.jpg)
Philip HaeusserComputer Vision Group
Machine Learning for Computer Vision
Neuron Activations• Different type of activation functions
rectified Linear Unit activation
8
![Page 9: Applications in Computer Vision Machine Learning for · 2016. 12. 2. · FlowNet: Learning Optical Flow with Convolutional Networks Philipp Fischer, Alexey Dosovitskiy, Eddy Ilg,](https://reader036.fdocuments.in/reader036/viewer/2022071401/60ec716a7a18fe32d96fbf45/html5/thumbnails/9.jpg)
Philip HaeusserComputer Vision Group
Machine Learning for Computer Vision
Simple Neural Network
• good reference: neuralnetworksanddeeplearning.com
9
![Page 10: Applications in Computer Vision Machine Learning for · 2016. 12. 2. · FlowNet: Learning Optical Flow with Convolutional Networks Philipp Fischer, Alexey Dosovitskiy, Eddy Ilg,](https://reader036.fdocuments.in/reader036/viewer/2022071401/60ec716a7a18fe32d96fbf45/html5/thumbnails/10.jpg)
Philip HaeusserComputer Vision Group
Machine Learning for Computer Vision
Simple Neural Network
Why is it good?
• inspired by brain• highly non-linear → function approximator• shown to solve complicated tasks
successfully
• “self-organizing map” / parameters are learned
• data driven
10
![Page 11: Applications in Computer Vision Machine Learning for · 2016. 12. 2. · FlowNet: Learning Optical Flow with Convolutional Networks Philipp Fischer, Alexey Dosovitskiy, Eddy Ilg,](https://reader036.fdocuments.in/reader036/viewer/2022071401/60ec716a7a18fe32d96fbf45/html5/thumbnails/11.jpg)
Philip HaeusserComputer Vision Group
Machine Learning for Computer Vision
Simple Neural Network
Why is it bad?
• inspired by brain• highly non-linear• high degree of freedom• shown to solve complicated tasks
successfully
• “self-organizing map”→ parameters are learned
• data driven• not much theory yet
11
![Page 12: Applications in Computer Vision Machine Learning for · 2016. 12. 2. · FlowNet: Learning Optical Flow with Convolutional Networks Philipp Fischer, Alexey Dosovitskiy, Eddy Ilg,](https://reader036.fdocuments.in/reader036/viewer/2022071401/60ec716a7a18fe32d96fbf45/html5/thumbnails/12.jpg)
Philip HaeusserComputer Vision Group
Machine Learning for Computer Vision
Back Propagation• Define a cost function
• Goal: Find global minimum
12
predictionground truth
![Page 13: Applications in Computer Vision Machine Learning for · 2016. 12. 2. · FlowNet: Learning Optical Flow with Convolutional Networks Philipp Fischer, Alexey Dosovitskiy, Eddy Ilg,](https://reader036.fdocuments.in/reader036/viewer/2022071401/60ec716a7a18fe32d96fbf45/html5/thumbnails/13.jpg)
Philip HaeusserComputer Vision Group
Machine Learning for Computer Vision
Back Propagation
13
Gradient Descent
predictionground truth
• Define a cost function
• Goal: Find global minimum
![Page 14: Applications in Computer Vision Machine Learning for · 2016. 12. 2. · FlowNet: Learning Optical Flow with Convolutional Networks Philipp Fischer, Alexey Dosovitskiy, Eddy Ilg,](https://reader036.fdocuments.in/reader036/viewer/2022071401/60ec716a7a18fe32d96fbf45/html5/thumbnails/14.jpg)
Philip HaeusserComputer Vision Group
Machine Learning for Computer Vision
Back Propagation
14
Learning rate
predictionground truth
• Define a cost function
• Minimize the error (derivative of the cost) with gradient descentGradient descent update rule:
![Page 15: Applications in Computer Vision Machine Learning for · 2016. 12. 2. · FlowNet: Learning Optical Flow with Convolutional Networks Philipp Fischer, Alexey Dosovitskiy, Eddy Ilg,](https://reader036.fdocuments.in/reader036/viewer/2022071401/60ec716a7a18fe32d96fbf45/html5/thumbnails/15.jpg)
Philip HaeusserComputer Vision Group
Machine Learning for Computer Vision
Back Propagation• Define a cost function
• Propagate the error through layers•Take the derivative of the output of the layer w.r.t the input of the
layer•Apply update rule for the parameters
15
predictionground truth
![Page 16: Applications in Computer Vision Machine Learning for · 2016. 12. 2. · FlowNet: Learning Optical Flow with Convolutional Networks Philipp Fischer, Alexey Dosovitskiy, Eddy Ilg,](https://reader036.fdocuments.in/reader036/viewer/2022071401/60ec716a7a18fe32d96fbf45/html5/thumbnails/16.jpg)
Philip HaeusserComputer Vision Group
Machine Learning for Computer Vision
MNIST Digit Classification with NN
16
![Page 17: Applications in Computer Vision Machine Learning for · 2016. 12. 2. · FlowNet: Learning Optical Flow with Convolutional Networks Philipp Fischer, Alexey Dosovitskiy, Eddy Ilg,](https://reader036.fdocuments.in/reader036/viewer/2022071401/60ec716a7a18fe32d96fbf45/html5/thumbnails/17.jpg)
Philip HaeusserComputer Vision Group
Machine Learning for Computer Vision
Philip HaeusserComputer Vision Group
Machine Learning for Computer Vision
Convolutional Neural Networks
17
![Page 18: Applications in Computer Vision Machine Learning for · 2016. 12. 2. · FlowNet: Learning Optical Flow with Convolutional Networks Philipp Fischer, Alexey Dosovitskiy, Eddy Ilg,](https://reader036.fdocuments.in/reader036/viewer/2022071401/60ec716a7a18fe32d96fbf45/html5/thumbnails/18.jpg)
Philip HaeusserComputer Vision Group
Machine Learning for Computer Vision
Philip HaeusserComputer Vision Group
Machine Learning for Computer Vision
Convolutional Neural Networks
18
![Page 19: Applications in Computer Vision Machine Learning for · 2016. 12. 2. · FlowNet: Learning Optical Flow with Convolutional Networks Philipp Fischer, Alexey Dosovitskiy, Eddy Ilg,](https://reader036.fdocuments.in/reader036/viewer/2022071401/60ec716a7a18fe32d96fbf45/html5/thumbnails/19.jpg)
Philip HaeusserComputer Vision Group
Machine Learning for Computer Vision
Philip HaeusserComputer Vision Group
Machine Learning for Computer Vision
Convolutional Neural Networks
19
![Page 20: Applications in Computer Vision Machine Learning for · 2016. 12. 2. · FlowNet: Learning Optical Flow with Convolutional Networks Philipp Fischer, Alexey Dosovitskiy, Eddy Ilg,](https://reader036.fdocuments.in/reader036/viewer/2022071401/60ec716a7a18fe32d96fbf45/html5/thumbnails/20.jpg)
Philip HaeusserComputer Vision Group
Machine Learning for Computer Vision
Philip HaeusserComputer Vision Group
Machine Learning for Computer Vision
Convolutional Neural Networks
20
![Page 21: Applications in Computer Vision Machine Learning for · 2016. 12. 2. · FlowNet: Learning Optical Flow with Convolutional Networks Philipp Fischer, Alexey Dosovitskiy, Eddy Ilg,](https://reader036.fdocuments.in/reader036/viewer/2022071401/60ec716a7a18fe32d96fbf45/html5/thumbnails/21.jpg)
Philip HaeusserComputer Vision Group
Machine Learning for Computer Vision
Philip HaeusserComputer Vision Group
Machine Learning for Computer Vision
Convolutional Neural Networks
21
AlexNetKrizhevsky, Alex, Ilya Sutskever, and Geoffrey E. Hinton."Imagenet classification with deep convolutional neural networks.”Advances in neural information processing systems. 2012.
![Page 22: Applications in Computer Vision Machine Learning for · 2016. 12. 2. · FlowNet: Learning Optical Flow with Convolutional Networks Philipp Fischer, Alexey Dosovitskiy, Eddy Ilg,](https://reader036.fdocuments.in/reader036/viewer/2022071401/60ec716a7a18fe32d96fbf45/html5/thumbnails/22.jpg)
Philip HaeusserComputer Vision Group
Machine Learning for Computer Vision
Philip HaeusserComputer Vision Group
Machine Learning for Computer Vision
Convolutional Neural Networks
22
Convolutional networks take advantage of the properties of natural signals:
• local connections • shared weights
• pooling • the use of many layers
Person
![Page 23: Applications in Computer Vision Machine Learning for · 2016. 12. 2. · FlowNet: Learning Optical Flow with Convolutional Networks Philipp Fischer, Alexey Dosovitskiy, Eddy Ilg,](https://reader036.fdocuments.in/reader036/viewer/2022071401/60ec716a7a18fe32d96fbf45/html5/thumbnails/23.jpg)
FlowNet: Learning Optical Flowwith Convolutional Networks
Philipp Fischer, Alexey Dosovitskiy, Eddy Ilg, Thomas BroxPhilip Häusser, Caner Hazırbaş, Vladimir Golkov, Daniel Cremers, Patrick van der Smagt
![Page 24: Applications in Computer Vision Machine Learning for · 2016. 12. 2. · FlowNet: Learning Optical Flow with Convolutional Networks Philipp Fischer, Alexey Dosovitskiy, Eddy Ilg,](https://reader036.fdocuments.in/reader036/viewer/2022071401/60ec716a7a18fe32d96fbf45/html5/thumbnails/24.jpg)
Philip HaeusserComputer Vision Group
Machine Learning for Computer Vision
Flying Chairs
24
![Page 25: Applications in Computer Vision Machine Learning for · 2016. 12. 2. · FlowNet: Learning Optical Flow with Convolutional Networks Philipp Fischer, Alexey Dosovitskiy, Eddy Ilg,](https://reader036.fdocuments.in/reader036/viewer/2022071401/60ec716a7a18fe32d96fbf45/html5/thumbnails/25.jpg)
Philip HaeusserComputer Vision Group
Machine Learning for Computer Vision
Data Augmentation
25
Gen
erat
edA
ugm
ente
d
• translation, rotation, scaling, additive Gaussian noise• changes in brightness, contrast, gamma and colour
25
![Page 26: Applications in Computer Vision Machine Learning for · 2016. 12. 2. · FlowNet: Learning Optical Flow with Convolutional Networks Philipp Fischer, Alexey Dosovitskiy, Eddy Ilg,](https://reader036.fdocuments.in/reader036/viewer/2022071401/60ec716a7a18fe32d96fbf45/html5/thumbnails/26.jpg)
Philip HaeusserComputer Vision Group
Machine Learning for Computer Vision
FlowNetSimple
26
![Page 27: Applications in Computer Vision Machine Learning for · 2016. 12. 2. · FlowNet: Learning Optical Flow with Convolutional Networks Philipp Fischer, Alexey Dosovitskiy, Eddy Ilg,](https://reader036.fdocuments.in/reader036/viewer/2022071401/60ec716a7a18fe32d96fbf45/html5/thumbnails/27.jpg)
Philip HaeusserComputer Vision Group
Machine Learning for Computer Vision
FlowNetSimple - Flying Chairs
27
![Page 28: Applications in Computer Vision Machine Learning for · 2016. 12. 2. · FlowNet: Learning Optical Flow with Convolutional Networks Philipp Fischer, Alexey Dosovitskiy, Eddy Ilg,](https://reader036.fdocuments.in/reader036/viewer/2022071401/60ec716a7a18fe32d96fbf45/html5/thumbnails/28.jpg)
Philip HaeusserComputer Vision Group
Machine Learning for Computer Vision
FlowNetSimple - Sintel
28
![Page 29: Applications in Computer Vision Machine Learning for · 2016. 12. 2. · FlowNet: Learning Optical Flow with Convolutional Networks Philipp Fischer, Alexey Dosovitskiy, Eddy Ilg,](https://reader036.fdocuments.in/reader036/viewer/2022071401/60ec716a7a18fe32d96fbf45/html5/thumbnails/29.jpg)
Philip HaeusserComputer Vision Group
Machine Learning for Computer Vision
FlowNetSimple + Variational Smoothing
29
![Page 30: Applications in Computer Vision Machine Learning for · 2016. 12. 2. · FlowNet: Learning Optical Flow with Convolutional Networks Philipp Fischer, Alexey Dosovitskiy, Eddy Ilg,](https://reader036.fdocuments.in/reader036/viewer/2022071401/60ec716a7a18fe32d96fbf45/html5/thumbnails/30.jpg)
Philip HaeusserComputer Vision Group
Machine Learning for Computer Vision
FlowNet: Learning Optical Flowwith Convolutional Networks
30