Deep Learning – Part II - Anyline · Deep Learning – Part II . Title: 20160224 Computer Vision...

23
Visual Computing – Data Analysis – Consulting [email protected] René Donner Deep Learning – Part II

Transcript of Deep Learning – Part II - Anyline · Deep Learning – Part II . Title: 20160224 Computer Vision...

Page 1: Deep Learning – Part II - Anyline · Deep Learning – Part II . Title: 20160224 Computer Vision Meetup Deep Learning 2 Created Date: 2/26/2016 3:41:33 PM ...

Visual Computing – Data Analysis – Consulting [email protected]

René Donner

Deep Learning – Part II

Page 2: Deep Learning – Part II - Anyline · Deep Learning – Part II . Title: 20160224 Computer Vision Meetup Deep Learning 2 Created Date: 2/26/2016 3:41:33 PM ...

René Donner Deep Learning

Overview

2

Deep Learning recap

How to classify images

Convolutional Neural Networks (CNNs)

Classifying MNIST with CNNs

Page 3: Deep Learning – Part II - Anyline · Deep Learning – Part II . Title: 20160224 Computer Vision Meetup Deep Learning 2 Created Date: 2/26/2016 3:41:33 PM ...

René Donner Deep Learning

Image recognition

3

Google Research Blog

Page 4: Deep Learning – Part II - Anyline · Deep Learning – Part II . Title: 20160224 Computer Vision Meetup Deep Learning 2 Created Date: 2/26/2016 3:41:33 PM ...

René Donner Deep Learning

Scene labeling

4

http://www.purdue.edu/newsroom/releases/2014/Q1/smartphone-to-become-smarter-with-deep-learning-innovation.html

Page 5: Deep Learning – Part II - Anyline · Deep Learning – Part II . Title: 20160224 Computer Vision Meetup Deep Learning 2 Created Date: 2/26/2016 3:41:33 PM ...

René Donner Deep Learning

Text recognition

5

http://www.pyimagesearch.com/2014/09/22/getting-started-deep-learning-python/

Large-Scale Deep Learning for Intelligent Computer Systems, Jeff Dean, Google, BayLearn 2015

Page 6: Deep Learning – Part II - Anyline · Deep Learning – Part II . Title: 20160224 Computer Vision Meetup Deep Learning 2 Created Date: 2/26/2016 3:41:33 PM ...

How does it work?

Page 7: Deep Learning – Part II - Anyline · Deep Learning – Part II . Title: 20160224 Computer Vision Meetup Deep Learning 2 Created Date: 2/26/2016 3:41:33 PM ...

René Donner Deep Learning

Deep learning

7

http://theanalyticsstore.ie/deep-learning/

Page 8: Deep Learning – Part II - Anyline · Deep Learning – Part II . Title: 20160224 Computer Vision Meetup Deep Learning 2 Created Date: 2/26/2016 3:41:33 PM ...

René Donner Deep Learning

Visualization

8

1. Layer

higher Layers

Emergence of Object-Selective Features in Unsupervised Feature Learning, Adam Coates, NIPS 2012

Page 9: Deep Learning – Part II - Anyline · Deep Learning – Part II . Title: 20160224 Computer Vision Meetup Deep Learning 2 Created Date: 2/26/2016 3:41:33 PM ...

René Donner Deep Learning

Deep learning

9

How does it work?

http://theanalyticsstore.ie/deep-learning/ http://stats.stackexchange.com/questions/114385/what-is-the-difference-between-convolutional-neural-networks-restricted-boltzma

Page 10: Deep Learning – Part II - Anyline · Deep Learning – Part II . Title: 20160224 Computer Vision Meetup Deep Learning 2 Created Date: 2/26/2016 3:41:33 PM ...

Getting started with DL

Page 11: Deep Learning – Part II - Anyline · Deep Learning – Part II . Title: 20160224 Computer Vision Meetup Deep Learning 2 Created Date: 2/26/2016 3:41:33 PM ...

René Donner Deep Learning

Frameworks

11

Caffehttp://caffe.berkeleyvision.org/Plain text filesFastest CNN, GPU

Kerashttps://github.com/fchollet/kerasPython, on top of Theano

TensorFlowhttp://tensorflow.org/Python, by Google

MXNethttps://github.com/dmlc/mxnetPython, R, Julia

Slid

e fro

m c

affe

tuto

rial

Page 12: Deep Learning – Part II - Anyline · Deep Learning – Part II . Title: 20160224 Computer Vision Meetup Deep Learning 2 Created Date: 2/26/2016 3:41:33 PM ...

René Donner Deep Learning

Tensorflow

12

General optimization framework

C++ / PythonCPU / GPU / (Multi-GPU)

Construct / Execution

Page 13: Deep Learning – Part II - Anyline · Deep Learning – Part II . Title: 20160224 Computer Vision Meetup Deep Learning 2 Created Date: 2/26/2016 3:41:33 PM ...

René Donner Deep Learning

MNIST

13

Page 14: Deep Learning – Part II - Anyline · Deep Learning – Part II . Title: 20160224 Computer Vision Meetup Deep Learning 2 Created Date: 2/26/2016 3:41:33 PM ...

René Donner Deep Learning

MNIST

14

https://www.tensorflow.org/versions/r0.7/tutorials/mnist/beginners/index.html#mnist-for-ml-beginners

Page 15: Deep Learning – Part II - Anyline · Deep Learning – Part II . Title: 20160224 Computer Vision Meetup Deep Learning 2 Created Date: 2/26/2016 3:41:33 PM ...

René Donner Deep Learning

Activation functions

15

http://gforge.se/2015/06/benchmarking-relu-and-prelu/

Rectified linear unit

Tanh

http://blog.otoro.net/2015/03/28/neural-slime-volleyball/

Page 16: Deep Learning – Part II - Anyline · Deep Learning – Part II . Title: 20160224 Computer Vision Meetup Deep Learning 2 Created Date: 2/26/2016 3:41:33 PM ...

René Donner Deep Learning

MNIST - Demo

16

Page 17: Deep Learning – Part II - Anyline · Deep Learning – Part II . Title: 20160224 Computer Vision Meetup Deep Learning 2 Created Date: 2/26/2016 3:41:33 PM ...

René Donner Deep Learning

MNIST

17

http://deeplearning4j.org/rbm-mnist-tutorial.html

Page 18: Deep Learning – Part II - Anyline · Deep Learning – Part II . Title: 20160224 Computer Vision Meetup Deep Learning 2 Created Date: 2/26/2016 3:41:33 PM ...

Accuracy: 91%

Page 19: Deep Learning – Part II - Anyline · Deep Learning – Part II . Title: 20160224 Computer Vision Meetup Deep Learning 2 Created Date: 2/26/2016 3:41:33 PM ...

Convolutional NN

Page 20: Deep Learning – Part II - Anyline · Deep Learning – Part II . Title: 20160224 Computer Vision Meetup Deep Learning 2 Created Date: 2/26/2016 3:41:33 PM ...

René Donner Deep Learning

Convolution

20

https://devblogs.nvidia.com/parallelforall/accelerate-machine-learning-cudnn-deep-neural-network-library/https://developer.apple.com/library/ios/documentation/Performance/Conceptual/vImage/ConvolutionOperations/ConvolutionOperations.html

Page 21: Deep Learning – Part II - Anyline · Deep Learning – Part II . Title: 20160224 Computer Vision Meetup Deep Learning 2 Created Date: 2/26/2016 3:41:33 PM ...

René Donner Deep Learning

Convolution

21

https://sites.google.com/site/shahriarinia/home/ai/machine-learning/neural-networks/deep-learning/theano-mnist/3-convolutional-neural-network-lenet

Page 22: Deep Learning – Part II - Anyline · Deep Learning – Part II . Title: 20160224 Computer Vision Meetup Deep Learning 2 Created Date: 2/26/2016 3:41:33 PM ...

Accuracy: ~99.2%

Page 23: Deep Learning – Part II - Anyline · Deep Learning – Part II . Title: 20160224 Computer Vision Meetup Deep Learning 2 Created Date: 2/26/2016 3:41:33 PM ...

Visual Computing – Data Analysis – Consulting [email protected]

René Donner

Deep Learning – Part II


From Reinforcement Learning to Deep Reinforcement …fagostin/assets/files/...Keywords: Machine learning · Reinforcement learning Deep learning · Deep reinforcement learning 1 Introduction

From Reinforcement Learning to Deep Reinforcement …fagostin/assets/files/...Keywords: Machine learning · Reinforcement learning Deep learning · Deep reinforcement learning 1 Introduction

Deep Learning with H2O - University of Idahostevel/504/Deep Learning with...Deep Learning with H2O. . 4Deep Learning Overview Unlike the neural networks of the past, modern Deep Learning

Deep Learning with H2O - University of Idahostevel/504/Deep Learning with...Deep Learning with H2O. . 4Deep Learning Overview Unlike the neural networks of the past, modern Deep Learning

Deep Learning for Depth Learning CS 229 Course Project ...cs229.stanford.edu/...DepthLearning_FinalReport.pdf · Deep Learning for Depth Learning CS 229 Course Project, ... Deep Learning

Deep Learning for Depth Learning CS 229 Course Project ...cs229.stanford.edu/...DepthLearning_FinalReport.pdf · Deep Learning for Depth Learning CS 229 Course Project, ... Deep Learning

Deep Learning I - Korea Universitydsba.korea.ac.kr/wp/wp-content/seminar/Deep learning/Deep Learning... · Deep Learning I Deep learning is research on learning models with multilayer

Deep Learning I - Korea Universitydsba.korea.ac.kr/wp/wp-content/seminar/Deep learning/Deep Learning... · Deep Learning I Deep learning is research on learning models with multilayer

Deep Learning: An Introduction from the NLP Perspective Deep Learning - An...Deep Learning Approach 1 Deep Learning Approach 2 Disclaimer I am not (yet) an expert in Deep Learning.

Deep Learning: An Introduction from the NLP Perspective Deep Learning - An...Deep Learning Approach 1 Deep Learning Approach 2 Disclaimer I am not (yet) an expert in Deep Learning.

Binary Deep Learning - TAUweb.eng.tau.ac.il/.../wp-content/uploads/2017/03/Binary-Deep-Learning.pdf · Binary Deep Learning Deep Learning Seminar, School of Electrical Engineering,

Binary Deep Learning - TAUweb.eng.tau.ac.il/.../wp-content/uploads/2017/03/Binary-Deep-Learning.pdf · Binary Deep Learning Deep Learning Seminar, School of Electrical Engineering,

Deep Learning in real world @Deep Learning Tokyo

Deep Learning in real world @Deep Learning Tokyo

CS 678 – Deep Learning1 Deep Learning Early Work Why Deep Learning Stacked Auto Encoders Deep Belief Networks.

CS 678 – Deep Learning1 Deep Learning Early Work Why Deep Learning Stacked Auto Encoders Deep Belief Networks.

Lecture 1: Introduction to Deep Learning - uvadlc.github.io · Lecture 1: Introduction to Deep Learning Efstratios Gavves. UVA DEEP LEARNING COURSE UVA DEEP LEARNING COURSE –EFSTRATIOS

Lecture 1: Introduction to Deep Learning - uvadlc.github.io · Lecture 1: Introduction to Deep Learning Efstratios Gavves. UVA DEEP LEARNING COURSE UVA DEEP LEARNING COURSE –EFSTRATIOS

ANYLINE® SDK PRODUCT SPECIFICATIONS - MRZ SKIP THE LINES AND SCAN … · 2016. 10. 27. · current passport scanning solution to a mobile device: this Anyline SDK configuration is

ANYLINE® SDK PRODUCT SPECIFICATIONS - MRZ SKIP THE LINES AND SCAN … · 2016. 10. 27. · current passport scanning solution to a mobile device: this Anyline SDK configuration is

Deep Learning Early Work Why Deep Learning Stacked Auto Encoders Deep Belief Networks CS 678 – Deep Learning1.

Deep Learning Early Work Why Deep Learning Stacked Auto Encoders Deep Belief Networks CS 678 – Deep Learning1.

An Introduction to Deep Learning · Deep Learning An Introduction to ... •Deep Learning •Feed Forward Network •Problems of Deep Learning •AutoEncoders •Restricted Boltzmann

An Introduction to Deep Learning · Deep Learning An Introduction to ... •Deep Learning •Feed Forward Network •Problems of Deep Learning •AutoEncoders •Restricted Boltzmann

Deep Learning for Reinforcement Learning in Pacman · Deep Learning for Reinforcement Learning in Pacman Deep Learning für Reinforcement Learning in Pacman Vorgelegte Bachelor-Thesis

Deep Learning for Reinforcement Learning in Pacman · Deep Learning for Reinforcement Learning in Pacman Deep Learning für Reinforcement Learning in Pacman Vorgelegte Bachelor-Thesis

UNSUPERVISED DEEP LEARNING - TAUweb.eng.tau.ac.il/.../uploads/2016/12/Unsupervised-Deep-Learning.pdf · UNSUPERVISED DEEP LEARNING Erez Aharonov Noam Eilon Deep Learning Seminar School

UNSUPERVISED DEEP LEARNING - TAUweb.eng.tau.ac.il/.../uploads/2016/12/Unsupervised-Deep-Learning.pdf · UNSUPERVISED DEEP LEARNING Erez Aharonov Noam Eilon Deep Learning Seminar School

Deep Learning for Laser Based Odometry Estimation - …juxi.net/workshop/deep-learning-rss-2016/papers/Nicolai - Deep... · Deep Learning for Laser Based Odometry Estimation ... deep

Deep Learning for Laser Based Odometry Estimation - …juxi.net/workshop/deep-learning-rss-2016/papers/Nicolai - Deep... · Deep Learning for Laser Based Odometry Estimation ... deep

Introduction to Deep Learning - Joyce Hojoyceho.github.io/cs534_s17/slide/25-deep-learning.pdf · Deep Learning: Overview ... Review: Backpropogation https: ... • Deep Learning

Introduction to Deep Learning - Joyce Hojoyceho.github.io/cs534_s17/slide/25-deep-learning.pdf · Deep Learning: Overview ... Review: Backpropogation https: ... • Deep Learning

Deep Learning - South Dakota State UniversityDeep Learning Deep learning (also known as deep structured learning, hierarchical learning or deep machine learning) is a branch of machine

Deep Learning - South Dakota State UniversityDeep Learning Deep learning (also known as deep structured learning, hierarchical learning or deep machine learning) is a branch of machine

Introduction to Deep Learning - Computer Graphics€¦ · Introduction to Deep Learning CS468 Spring 2017 Charles Qi. What is Deep Learning? Deep learning allows computational models

Introduction to Deep Learning - Computer Graphics€¦ · Introduction to Deep Learning CS468 Spring 2017 Charles Qi. What is Deep Learning? Deep learning allows computational models

Deep Learning for Reinforcement Learning in · PDF fileDeep Learning for Reinforcement Learning in ... Deep Learning for Reinforcement Learning in Pacman Deep Learning für ... Während

Deep Learning for Reinforcement Learning in · PDF fileDeep Learning for Reinforcement Learning in ... Deep Learning for Reinforcement Learning in Pacman Deep Learning für ... Während

THE DEEP LEARNING REVOLUTION - Nvidia...THE DEEP LEARNING REVOLUTION Superhuman Breakthroughs in Modern Artificial Intelligence Powered by GPUs JOIN THE DEEP LEARNING ERA DEEP LEARNING

THE DEEP LEARNING REVOLUTION - Nvidia...THE DEEP LEARNING REVOLUTION Superhuman Breakthroughs in Modern Artificial Intelligence Powered by GPUs JOIN THE DEEP LEARNING ERA DEEP LEARNING