Introduction to Deep LearningIntroduction to Deep Learning Optimization CNN Introduction to NN...

68
Introduction to Deep Learning 1 I2DL: Prof. Niessner, Prof. Leal-Taixé

Transcript of Introduction to Deep LearningIntroduction to Deep Learning Optimization CNN Introduction to NN...

Page 1: Introduction to Deep LearningIntroduction to Deep Learning Optimization CNN Introduction to NN Machine Learning basics Back-propagation RNN I2DL: Prof. Niessner, Prof. Leal-Taix é

Introduction to Deep Learning

1I2DL: Prof. Niessner, Prof. Leal-Taixé

Page 2: Introduction to Deep LearningIntroduction to Deep Learning Optimization CNN Introduction to NN Machine Learning basics Back-propagation RNN I2DL: Prof. Niessner, Prof. Leal-Taix é

2

Lecturers

PhDs

I2DL: Prof. Niessner, Prof. Leal-Taixé

Prof. Dr. Laura Leal-Taixé

Prof. Dr. Matthias Niessner

Patrick Dendorfer

AndreasRössler

The Team

Page 3: Introduction to Deep LearningIntroduction to Deep Learning Optimization CNN Introduction to NN Machine Learning basics Back-propagation RNN I2DL: Prof. Niessner, Prof. Leal-Taix é

What is Computer Vision?

• First defined in the 60s in artificial intelligence groups

• “Mimic the human visual system”

• Center block of robotic intelligence

3I2DL: Prof. Niessner, Prof. Leal-Taixé

Page 4: Introduction to Deep LearningIntroduction to Deep Learning Optimization CNN Introduction to NN Machine Learning basics Back-propagation RNN I2DL: Prof. Niessner, Prof. Leal-Taix é

Hubel and Wiesel• David Hubel and Torsten Wiesel

were neurobiologists from Harvard Medical School

• Experiment revealed several secrets of the human vision system

• Won 2 Nobel prizes

4I2DL: Prof. Niessner, Prof. Leal-Taixé

Page 5: Introduction to Deep LearningIntroduction to Deep Learning Optimization CNN Introduction to NN Machine Learning basics Back-propagation RNN I2DL: Prof. Niessner, Prof. Leal-Taix é

Hubel and Wiesel Experiment• Recorded electrical activity from

individual neurons in the brains of cats.

• Slide projector to show specific patterns to the cats noted specific patterns stimulated activity in specific parts of the brain.

• Results: Visual cortex cells are sensitive to the orientation of edges but insensitive to their position

5I2DL: Prof. Niessner, Prof. Leal-Taixé

Page 6: Introduction to Deep LearningIntroduction to Deep Learning Optimization CNN Introduction to NN Machine Learning basics Back-propagation RNN I2DL: Prof. Niessner, Prof. Leal-Taix é

6I2DL: Prof. Niessner, Prof. Leal-Taixé

Page 7: Introduction to Deep LearningIntroduction to Deep Learning Optimization CNN Introduction to NN Machine Learning basics Back-propagation RNN I2DL: Prof. Niessner, Prof. Leal-Taix é

Computer Vision

Few Decades Later…

7I2DL: Prof. Niessner, Prof. Leal-Taixé

Page 8: Introduction to Deep LearningIntroduction to Deep Learning Optimization CNN Introduction to NN Machine Learning basics Back-propagation RNN I2DL: Prof. Niessner, Prof. Leal-Taix é

Computer Vision

Physics PsychologyBiology

MathematicsEngineering Computer

scienceArtificial Intelligence

ML

Neuroscience

AlgorithmsOptimization

NLPSpeech

Robotics

OpticsImage

processing

8I2DL: Prof. Niessner, Prof. Leal-Taixé

Page 9: Introduction to Deep LearningIntroduction to Deep Learning Optimization CNN Introduction to NN Machine Learning basics Back-propagation RNN I2DL: Prof. Niessner, Prof. Leal-Taix é

Pre 2012

Image Classification

9I2DL: Prof. Niessner, Prof. Leal-Taixé

Page 10: Introduction to Deep LearningIntroduction to Deep Learning Optimization CNN Introduction to NN Machine Learning basics Back-propagation RNN I2DL: Prof. Niessner, Prof. Leal-Taix é

AAwesome magic box

Become magicians Post 2012Open the box

Image Classification

10I2DL: Prof. Niessner, Prof. Leal-Taixé

Page 11: Introduction to Deep LearningIntroduction to Deep Learning Optimization CNN Introduction to NN Machine Learning basics Back-propagation RNN I2DL: Prof. Niessner, Prof. Leal-Taix é

Why Deep Learning?

11I2DL: Prof. Niessner, Prof. Leal-Taixé

Page 12: Introduction to Deep LearningIntroduction to Deep Learning Optimization CNN Introduction to NN Machine Learning basics Back-propagation RNN I2DL: Prof. Niessner, Prof. Leal-Taix é

Deep Learning History

12I2DL: Prof. Niessner, Prof. Leal-Taixé

Page 13: Introduction to Deep LearningIntroduction to Deep Learning Optimization CNN Introduction to NN Machine Learning basics Back-propagation RNN I2DL: Prof. Niessner, Prof. Leal-Taix é

The Empire Strikes Back

13

0

5

10

15

20

25

30

2010 2011 2012AlexNet

2013 2014VGGNet

Human 2015ResNet

2016Ensemble

2017SENet

ILSVRC top-5 error on ImageNet

Deep Learning Approaches

I2DL: Prof. Niessner, Prof. Leal-Taixé

Page 14: Introduction to Deep LearningIntroduction to Deep Learning Optimization CNN Introduction to NN Machine Learning basics Back-propagation RNN I2DL: Prof. Niessner, Prof. Leal-Taix é

• MNIST digit recognition dataset

• 107 pixels used in training

• ImageNet image recognition dataset

• 1014 pixels used in training

1998LeCunet al.

2012Krizhevskyet al.

What Has Changed?

14I2DL: Prof. Niessner, Prof. Leal-Taixé

Page 15: Introduction to Deep LearningIntroduction to Deep Learning Optimization CNN Introduction to NN Machine Learning basics Back-propagation RNN I2DL: Prof. Niessner, Prof. Leal-Taix é

Big Data

Models know where to learn from

Hardware

Models are trainable

Deep

Models are complex

What Made this Possible?

15I2DL: Prof. Niessner, Prof. Leal-Taixé

Page 16: Introduction to Deep LearningIntroduction to Deep Learning Optimization CNN Introduction to NN Machine Learning basics Back-propagation RNN I2DL: Prof. Niessner, Prof. Leal-Taix é

Deep Learning Recognition

16

ACM Turing Award 2019 (Nobel Prize of Computing)Yann LeCun, Geoffrey Hinton, and Yoshua Bengio

I2DL: Prof. Niessner, Prof. Leal-Taixé

Page 17: Introduction to Deep LearningIntroduction to Deep Learning Optimization CNN Introduction to NN Machine Learning basics Back-propagation RNN I2DL: Prof. Niessner, Prof. Leal-Taix é

Credits: Dr. Pont-Tuset, ETH Zurich

Deep Learning and Computer Vision

17I2DL: Prof. Niessner, Prof. Leal-Taixé

Page 18: Introduction to Deep LearningIntroduction to Deep Learning Optimization CNN Introduction to NN Machine Learning basics Back-propagation RNN I2DL: Prof. Niessner, Prof. Leal-Taix é

Credits: Dr. Pont-Tuset, ETH Zurich

Deep Learning and Computer Vision

18I2DL: Prof. Niessner, Prof. Leal-Taixé

Page 19: Introduction to Deep LearningIntroduction to Deep Learning Optimization CNN Introduction to NN Machine Learning basics Back-propagation RNN I2DL: Prof. Niessner, Prof. Leal-Taix é

Deep Learning Today

19Object Detection

I2DL: Prof. Niessner, Prof. Leal-Taixé

Page 20: Introduction to Deep LearningIntroduction to Deep Learning Optimization CNN Introduction to NN Machine Learning basics Back-propagation RNN I2DL: Prof. Niessner, Prof. Leal-Taix é

Self-driving cars

Deep Learning Today

20I2DL: Prof. Niessner, Prof. Leal-Taixé

Page 21: Introduction to Deep LearningIntroduction to Deep Learning Optimization CNN Introduction to NN Machine Learning basics Back-propagation RNN I2DL: Prof. Niessner, Prof. Leal-Taix é

AlphaGo

Machine translation

Emoticon suggestion

Deep Learning Today

21I2DL: Prof. Niessner, Prof. Leal-Taixé

Page 22: Introduction to Deep LearningIntroduction to Deep Learning Optimization CNN Introduction to NN Machine Learning basics Back-propagation RNN I2DL: Prof. Niessner, Prof. Leal-Taix é

Deep Learning Today

22Alpha StarI2DL: Prof. Niessner, Prof. Leal-Taixé

Page 23: Introduction to Deep LearningIntroduction to Deep Learning Optimization CNN Introduction to NN Machine Learning basics Back-propagation RNN I2DL: Prof. Niessner, Prof. Leal-Taix é

Deep Learning Today

23

Automated Text Generation [Karpathy et al.]

I2DL: Prof. Niessner, Prof. Leal-Taixé

Page 24: Introduction to Deep LearningIntroduction to Deep Learning Optimization CNN Introduction to NN Machine Learning basics Back-propagation RNN I2DL: Prof. Niessner, Prof. Leal-Taix é

Deep Learning Today

24

Google Assistant (Google IO’19)I2DL: Prof. Niessner, Prof. Leal-Taixé

Page 25: Introduction to Deep LearningIntroduction to Deep Learning Optimization CNN Introduction to NN Machine Learning basics Back-propagation RNN I2DL: Prof. Niessner, Prof. Leal-Taix é

Healthcare, cancer detection

Deep Learning Today

25I2DL: Prof. Niessner, Prof. Leal-Taixé

Page 26: Introduction to Deep LearningIntroduction to Deep Learning Optimization CNN Introduction to NN Machine Learning basics Back-propagation RNN I2DL: Prof. Niessner, Prof. Leal-Taix é

Deep Learning Market

[…] market research report Deep Learning Market […] “ the deep learning market is expected to be worth USD 1,722.9 Million by 2022.

26I2DL: Prof. Niessner, Prof. Leal-Taixé

Page 27: Introduction to Deep LearningIntroduction to Deep Learning Optimization CNN Introduction to NN Machine Learning basics Back-propagation RNN I2DL: Prof. Niessner, Prof. Leal-Taix é

Deep Learning Job Perspective• Excellent Job Perspectives!

– Automation requires ML/DL -> growth!– Top-notch companies will gladly hire you!

• Many industries now:– IT-Companies– Cars, Logistic, Health Care, etc…– Manufacturing / Robotics, etc…

27I2DL: Prof. Niessner, Prof. Leal-Taixé

Page 28: Introduction to Deep LearningIntroduction to Deep Learning Optimization CNN Introduction to NN Machine Learning basics Back-propagation RNN I2DL: Prof. Niessner, Prof. Leal-Taix é

But: Also Challenging!• High-level understanding is not enough

– Need proper theory background– Need proper practical skillsets

• Can be competitive!– Many good people– Downloading scripts / running code not enough – Deeper understanding often requires PhDs

28I2DL: Prof. Niessner, Prof. Leal-Taixé

Page 29: Introduction to Deep LearningIntroduction to Deep Learning Optimization CNN Introduction to NN Machine Learning basics Back-propagation RNN I2DL: Prof. Niessner, Prof. Leal-Taix é

Deep Learning Culture

29I2DL: Prof. Niessner, Prof. Leal-Taixé

Page 30: Introduction to Deep LearningIntroduction to Deep Learning Optimization CNN Introduction to NN Machine Learning basics Back-propagation RNN I2DL: Prof. Niessner, Prof. Leal-Taix é

Deep Learning Memes

30I2DL: Prof. Niessner, Prof. Leal-Taixé

Page 31: Introduction to Deep LearningIntroduction to Deep Learning Optimization CNN Introduction to NN Machine Learning basics Back-propagation RNN I2DL: Prof. Niessner, Prof. Leal-Taix é

Deep Learning Memes

31I2DL: Prof. Niessner, Prof. Leal-Taixé

Page 32: Introduction to Deep LearningIntroduction to Deep Learning Optimization CNN Introduction to NN Machine Learning basics Back-propagation RNN I2DL: Prof. Niessner, Prof. Leal-Taix é

Deep Learning Memes

32I2DL: Prof. Niessner, Prof. Leal-Taixé

Page 33: Introduction to Deep LearningIntroduction to Deep Learning Optimization CNN Introduction to NN Machine Learning basics Back-propagation RNN I2DL: Prof. Niessner, Prof. Leal-Taix é

Deep Learning Memes

33I2DL: Prof. Niessner, Prof. Leal-Taixé

Page 34: Introduction to Deep LearningIntroduction to Deep Learning Optimization CNN Introduction to NN Machine Learning basics Back-propagation RNN I2DL: Prof. Niessner, Prof. Leal-Taix é

Deep Learning Memes

34I2DL: Prof. Niessner, Prof. Leal-Taixé

Page 35: Introduction to Deep LearningIntroduction to Deep Learning Optimization CNN Introduction to NN Machine Learning basics Back-propagation RNN I2DL: Prof. Niessner, Prof. Leal-Taix é

Deep Learning Memes

35I2DL: Prof. Niessner, Prof. Leal-Taixé

Page 36: Introduction to Deep LearningIntroduction to Deep Learning Optimization CNN Introduction to NN Machine Learning basics Back-propagation RNN I2DL: Prof. Niessner, Prof. Leal-Taix é

Deep Learning at TUM

36I2DL: Prof. Niessner, Prof. Leal-Taixé

Page 37: Introduction to Deep LearningIntroduction to Deep Learning Optimization CNN Introduction to NN Machine Learning basics Back-propagation RNN I2DL: Prof. Niessner, Prof. Leal-Taix é

Many TUM Research Labs use DL• Visual Computing Lab (Prof. Niessner):

– Research in computer vision, graphics, and machine learning• Dynamic Vision and Learning Group (Prof. Leal-Taixe)

– Research on Computer Vision; e.g., video editing/segmentation etc.• 3D Understanding Lab (Dr. Dai):

– Research in 3D scenes and its semantics.• Computer Vision Group (Prof. Cremers)

– Research in computer vision and pattern recognition• Data Mining and Analytics Lab (Prof. Günnemann)

– Research methods for robust machine learning• Computer Aided Medical Procedures (Prof. Navab)

– Research in machine learning for medical applications• And probably many more

37I2DL: Prof. Niessner, Prof. Leal-Taixé

Page 38: Introduction to Deep LearningIntroduction to Deep Learning Optimization CNN Introduction to NN Machine Learning basics Back-propagation RNN I2DL: Prof. Niessner, Prof. Leal-Taix é

Our Research Labs• Visual Computing Lab (Prof. Niessner):

https://niessnerlab.org/publications.html– Twitter: https://twitter.com/MattNiessner– Youtube: https://www.youtube.com/channel/UCXN2nYjVT0cR9G61RPEzK5Q– Facebook: https://www.facebook.com/matthias.niessner

• Dynamic Vision and Learning Lab (Prof. Leal-Taixé):https://dvl.in.tum.de/publications.html– Twitter: https://twitter.com/lealtaixe– Youtube: https://www.youtube.com/channel/UCQVCsX1CcZQr0oUMZg6szIQ

38I2DL: Prof. Niessner, Prof. Leal-Taixé

https://niessnerlab.org/publications.htmlYoutube
https://twitter.com/MattNiessner
https://www.youtube.com/channel/UCXN2nYjVT0cR9G61RPEzK5Q
https://www.facebook.com/matthias.niessner
https://dvl.in.tum.de/publications.html
https://twitter.com/lealtaixe
https://www.youtube.com/channel/UCQVCsX1CcZQr0oUMZg6szIQ
Page 39: Introduction to Deep LearningIntroduction to Deep Learning Optimization CNN Introduction to NN Machine Learning basics Back-propagation RNN I2DL: Prof. Niessner, Prof. Leal-Taix é

[Caelles et al., CVPR’ 17] One-Shot Video Object Segmentation

Deep Learning at TUM

39I2DL: Prof. Niessner, Prof. Leal-Taixé

Page 40: Introduction to Deep LearningIntroduction to Deep Learning Optimization CNN Introduction to NN Machine Learning basics Back-propagation RNN I2DL: Prof. Niessner, Prof. Leal-Taix é

Deep Learning at TUM

40[Dosovitskiy et al., ICCV’ 15] FlowNetI2DL: Prof. Niessner, Prof. Leal-Taixé

Page 41: Introduction to Deep LearningIntroduction to Deep Learning Optimization CNN Introduction to NN Machine Learning basics Back-propagation RNN I2DL: Prof. Niessner, Prof. Leal-Taix é

Deep Learning at TUM

41

CC3

CC2

CC1

Reshape Conv+BN+ReLU Pooling Upsample Concat Score

DDFF

[Hazirbas et al., IJCV’18] Deep Depth From Focus.I2DL: Prof. Niessner, Prof. Leal-Taixé

Page 42: Introduction to Deep LearningIntroduction to Deep Learning Optimization CNN Introduction to NN Machine Learning basics Back-propagation RNN I2DL: Prof. Niessner, Prof. Leal-Taix é

Deep Learning at TUM• Video anonymization

42

[Maximov et al., CVPR 2020] CIAGAN: Conditional identity anonymization generative adversarial networks.I2DL: Prof. Niessner, Prof. Leal-Taixé

So

urce

Control identity

Page 43: Introduction to Deep LearningIntroduction to Deep Learning Optimization CNN Introduction to NN Machine Learning basics Back-propagation RNN I2DL: Prof. Niessner, Prof. Leal-Taix é

Deep Learning at TUM• Multiple object tracking with graph neural networks

43

[Brasó and Leal-Taixé, CVPR 2020] Learning a Neural Solver forMultiple Object Tracking.I2DL: Prof. Niessner, Prof. Leal-Taixé

Page 44: Introduction to Deep LearningIntroduction to Deep Learning Optimization CNN Introduction to NN Machine Learning basics Back-propagation RNN I2DL: Prof. Niessner, Prof. Leal-Taix é

Deep Learning at TUM

44[Yang et al., ECCV’ 18] Deep Virtual Stereo Odometry I2DL: Prof. Niessner, Prof. Leal-Taixé

Page 45: Introduction to Deep LearningIntroduction to Deep Learning Optimization CNN Introduction to NN Machine Learning basics Back-propagation RNN I2DL: Prof. Niessner, Prof. Leal-Taix é

Deep Learning at TUM

45[Xie et al. Siggraph’ 18] tempoGANI2DL: Prof. Niessner, Prof. Leal-Taixé

Page 46: Introduction to Deep LearningIntroduction to Deep Learning Optimization CNN Introduction to NN Machine Learning basics Back-propagation RNN I2DL: Prof. Niessner, Prof. Leal-Taix é

Deep Learning at TUM

47[Dai et al., CVPR’17] ScanNet

ScanNet Stats:-Kinect-style RGB-D sensors-1513 scans of 3D environments-2.5 Mio RGB-D frames-Dense 3D, crowd-source MTurk labels-Annotations projected to 2D frames

I2DL: Prof. Niessner, Prof. Leal-Taixé

Page 47: Introduction to Deep LearningIntroduction to Deep Learning Optimization CNN Introduction to NN Machine Learning basics Back-propagation RNN I2DL: Prof. Niessner, Prof. Leal-Taix é

Deep Learning at TUM

48[Hou et al., CVPR’19] 3D Semantic Instance Segmentation

I2DL: Prof. Niessner, Prof. Leal-Taixé

Page 48: Introduction to Deep LearningIntroduction to Deep Learning Optimization CNN Introduction to NN Machine Learning basics Back-propagation RNN I2DL: Prof. Niessner, Prof. Leal-Taix é

Deep Learning at TUM

49[Thies et al., Siggraph’19 ]: Neural TexturesI2DL: Prof. Niessner, Prof. Leal-Taixé

Page 49: Introduction to Deep LearningIntroduction to Deep Learning Optimization CNN Introduction to NN Machine Learning basics Back-propagation RNN I2DL: Prof. Niessner, Prof. Leal-Taix é

Deep Learning at TUM

50[Thies et al., Siggraph’19 ]: Neural TexturesI2DL: Prof. Niessner, Prof. Leal-Taixé

Page 50: Introduction to Deep LearningIntroduction to Deep Learning Optimization CNN Introduction to NN Machine Learning basics Back-propagation RNN I2DL: Prof. Niessner, Prof. Leal-Taix é

Deep Learning at TUM

51[Thies et al., Siggraph’19 ]: Neural TexturesI2DL: Prof. Niessner, Prof. Leal-Taixé

Page 51: Introduction to Deep LearningIntroduction to Deep Learning Optimization CNN Introduction to NN Machine Learning basics Back-propagation RNN I2DL: Prof. Niessner, Prof. Leal-Taix é

Deep Learning at TUM

52[Thies et al., Siggraph’19 ]: Neural TexturesI2DL: Prof. Niessner, Prof. Leal-Taixé

Page 52: Introduction to Deep LearningIntroduction to Deep Learning Optimization CNN Introduction to NN Machine Learning basics Back-propagation RNN I2DL: Prof. Niessner, Prof. Leal-Taix é

Deep Learning at TUM

53

[Roessler et al., ICCV’19 ]: Face Forensics++

I2DL: Prof. Niessner, Prof. Leal-Taixé

Page 53: Introduction to Deep LearningIntroduction to Deep Learning Optimization CNN Introduction to NN Machine Learning basics Back-propagation RNN I2DL: Prof. Niessner, Prof. Leal-Taix é

Context of Other Lectures at TUM

54I2DL: Prof. Niessner, Prof. Leal-Taixé

Page 54: Introduction to Deep LearningIntroduction to Deep Learning Optimization CNN Introduction to NN Machine Learning basics Back-propagation RNN I2DL: Prof. Niessner, Prof. Leal-Taix é

Introduction to Deep Learning

Optimization

CNN

Introduction to NN

Machine Learning

basics

Back-propagation RNN

55I2DL: Prof. Niessner, Prof. Leal-Taixé

Page 55: Introduction to Deep LearningIntroduction to Deep Learning Optimization CNN Introduction to NN Machine Learning basics Back-propagation RNN I2DL: Prof. Niessner, Prof. Leal-Taix é

Deep Learning at TUM

56

Intro to Deep

Learning(Niessner,

Leal-Taixe)DL for Physics(Thuerey)

ADL for Vision (Niessner,

Leal-Taixe)

DL for Medical Applicat.

(Menze)

DL in Robotics

(Bäuml)

Machine Learning(Günnemann)

I2DL: Prof. Niessner, Prof. Leal-Taixé

Page 56: Introduction to Deep LearningIntroduction to Deep Learning Optimization CNN Introduction to NN Machine Learning basics Back-propagation RNN I2DL: Prof. Niessner, Prof. Leal-Taix é

Why is I2DL so Important?• Many of the other lectures / practical require it!

– Often only limited spots are available (e.g., in the prestigious Advanced Deep Learning for Computer Vision Class)

• Always preparation for guided research / IDP– Most topics require it – For career in AI/DL these are the best ways to get into

57I2DL: Prof. Niessner, Prof. Leal-Taixé

Page 57: Introduction to Deep LearningIntroduction to Deep Learning Optimization CNN Introduction to NN Machine Learning basics Back-propagation RNN I2DL: Prof. Niessner, Prof. Leal-Taix é

What’s New?

• Last semester: – Re-vamped content of slides / improved design– Re-organization of lecture structure

• This semester:– Re-vamping of the exercises (incl. sessions)– Earlier intro to pyTorch, more but smaller exercises

58I2DL: Prof. Niessner, Prof. Leal-Taixé

Page 58: Introduction to Deep LearningIntroduction to Deep Learning Optimization CNN Introduction to NN Machine Learning basics Back-propagation RNN I2DL: Prof. Niessner, Prof. Leal-Taix é

Introduction to Deep Learning

59I2DL: Prof. Niessner, Prof. Leal-Taixé

Page 59: Introduction to Deep LearningIntroduction to Deep Learning Optimization CNN Introduction to NN Machine Learning basics Back-propagation RNN I2DL: Prof. Niessner, Prof. Leal-Taix é

Moodle• Announcements via Moodle - IMPORTANT!

– Sign up in TUM online for access: https://www.moodle.tum.de/

– We will share common information (e.g., regarding exam)– Ask content questions online so others benefit– Don’t post solutions– TAs will monitor Moodle (expect answers within hours)

• We have a team of 11 TAs

60I2DL: Prof. Niessner, Prof. Leal-Taixé

https://www.moodle.tum.de/
Page 60: Introduction to Deep LearningIntroduction to Deep Learning Optimization CNN Introduction to NN Machine Learning basics Back-propagation RNN I2DL: Prof. Niessner, Prof. Leal-Taix é

Email• Email list:

• Do NOT email us personally!– Cannot handle so many emails / hence will be ignored

• Email list for organizational questions only!– Content questions -> Moodle or Office Hours

61

[email protected]

I2DL: Prof. Niessner, Prof. Leal-Taixé

Page 61: Introduction to Deep LearningIntroduction to Deep Learning Optimization CNN Introduction to NN Machine Learning basics Back-propagation RNN I2DL: Prof. Niessner, Prof. Leal-Taix é

Website• Links and slides will be shared on website

62

https://niessner.github.io/I2DL/I2DL: Prof. Niessner, Prof. Leal-Taixé

https://niessner.github.io/I2DL/
Page 62: Introduction to Deep LearningIntroduction to Deep Learning Optimization CNN Introduction to NN Machine Learning basics Back-propagation RNN I2DL: Prof. Niessner, Prof. Leal-Taix é

About the Lecture• Theory lectures• Every Monday 14:15-15:45

– Lectures are virtual and will be uploaded to youtube:https://www.youtube.com/channel/UCXN2nYjVT0cR9G61RPEzK5Q

• Practical sessions– Every Thursday 8:30-10:00 – There will be practical exercises

• Guest lecture TBD

63I2DL: Prof. Niessner, Prof. Leal-Taixé

https://www.youtube.com/channel/UCXN2nYjVT0cR9G61RPEzK5Q
Page 63: Introduction to Deep LearningIntroduction to Deep Learning Optimization CNN Introduction to NN Machine Learning basics Back-propagation RNN I2DL: Prof. Niessner, Prof. Leal-Taix é

Grading System• Exam: TBA

• Review: TUM Exam

• Important: no retake exam

• Practice: More information on Thursday!

• Bonus 0.3 + questions in the final exam

• Do not miss the first practical session !!!

64I2DL: Prof. Niessner, Prof. Leal-Taixé

Page 64: Introduction to Deep LearningIntroduction to Deep Learning Optimization CNN Introduction to NN Machine Learning basics Back-propagation RNN I2DL: Prof. Niessner, Prof. Leal-Taix é

Preliminary Syllabus

Lecture 1: Introduction to the lecture, Deep Learning, Machine Learning.Lecture 2: Machine Learning Basics, Linear regression, Maximum LikelihoodLecture 3: Introduction to Neural Networks, Computational GraphsLecture 4: Optimization and BackpropagationLecture 5: Scaling Optimization to large Data, Stochastic Gradient DescentLecture 6: Training Neural Networks ILecture 7: Training Neural Networks IILecture 8: Training Neural Networks IIILecture 9: Introduction to CNNsLecture 10: CNNs architectures; CNNs for object detectionLecture 11: Recurrent Neural Networks (RNNs) Lecture 12: Autoencoders, VAEs and GANsLecture 13: Reinforcement LearningLecture 14: Guest Lecture (TBD)

66I2DL: Prof. Niessner, Prof. Leal-Taixé

Page 65: Introduction to Deep LearningIntroduction to Deep Learning Optimization CNN Introduction to NN Machine Learning basics Back-propagation RNN I2DL: Prof. Niessner, Prof. Leal-Taix é

Other Administrative• External students welcome

– Will get Certificate / Schein at the end– Send email to list and we will add you to moodle

• Lectures will be recorded and put on youtube– https://www.youtube.com/channel/UCXN2nYjVT0cR9G

61RPEzK5Q

• Again: check announcements on moodle

67I2DL: Prof. Niessner, Prof. Leal-Taixé

https://www.youtube.com/channel/UCXN2nYjVT0cR9G61RPEzK5Q
Page 66: Introduction to Deep LearningIntroduction to Deep Learning Optimization CNN Introduction to NN Machine Learning basics Back-propagation RNN I2DL: Prof. Niessner, Prof. Leal-Taix é

Office Hours

• We will have dedicated office hours regarding– Theoretical help (e.g., specific lecture questions)– Help on exercises

• Will be announced with first tutorial session and on Moodle

68I2DL: Prof. Niessner, Prof. Leal-Taixé

Page 67: Introduction to Deep LearningIntroduction to Deep Learning Optimization CNN Introduction to NN Machine Learning basics Back-propagation RNN I2DL: Prof. Niessner, Prof. Leal-Taix é

Upcoming Lecture

• Next Lecture: Lecture 2: Machine Learning basics

• This Thursday: 1st Practical Session– if you want bonus, do not miss it!

69I2DL: Prof. Niessner, Prof. Leal-Taixé

Page 68: Introduction to Deep LearningIntroduction to Deep Learning Optimization CNN Introduction to NN Machine Learning basics Back-propagation RNN I2DL: Prof. Niessner, Prof. Leal-Taix é

See you next week

70I2DL: Prof. Niessner, Prof. Leal-Taixé


Incremental Dense Semantic Stereo Fusion for Large-Scale …niessner/papers/2015/3... · 2017-03-29 · Victor A. Prisacariu Olaf K¨ahler David W. Murray Shahram Izadi Patrick P

Incremental Dense Semantic Stereo Fusion for Large-Scale …niessner/papers/2015/3... · 2017-03-29 · Victor A. Prisacariu Olaf K¨ahler David W. Murray Shahram Izadi Patrick P

Introduction to Deep Learning (I2DL)...I2DL: Prof. Niessner 1 Today’s Outline • Neural Networks – Mathematical Motivation – Modularization • Exercise 5 – Implementation

Introduction to Deep Learning (I2DL)...I2DL: Prof. Niessner 1 Today’s Outline • Neural Networks – Mathematical Motivation – Modularization • Exercise 5 – Implementation

Real-Time Deformation of Subdivision Surfaces …graphics.stanford.edu/~niessner/papers/2014/4subddef/schaefer2014...Schäfer et al. / Real-Time Deformation of Subdivision Surfaces

Real-Time Deformation of Subdivision Surfaces …graphics.stanford.edu/~niessner/papers/2014/4subddef/schaefer2014...Schäfer et al. / Real-Time Deformation of Subdivision Surfaces

Dynamic Feature-Adaptive Subdivisiongraphics.stanford.edu/~niessner/papers/2015/0dynamic/...Dynamic Feature-Adaptive Subdivision H. Schafer¨ 1J. Raab B. Keinert1 M. Meyer2 M. Stamminger1

Dynamic Feature-Adaptive Subdivisiongraphics.stanford.edu/~niessner/papers/2015/0dynamic/...Dynamic Feature-Adaptive Subdivision H. Schafer¨ 1J. Raab B. Keinert1 M. Meyer2 M. Stamminger1

Efficient GPU Rendering of Subdivision Surfaces using ...niessner/papers/2016/4...quadtree for each post-tessellation vertex, we are able to accurately and efficiently evaluate the

Efficient GPU Rendering of Subdivision Surfaces using ...niessner/papers/2016/4...quadtree for each post-tessellation vertex, we are able to accurately and efficiently evaluate the

One-Shot Video Object Segmentation€¦ · One-Shot Video Object Segmentation S. Caelles1,* K.-K. Maninis1,∗ J. Pont-Tuset1 L. Leal-Taix´e2 D. Cremers2 L. Van Gool1 1ETH Zurich¨

One-Shot Video Object Segmentation€¦ · One-Shot Video Object Segmentation S. Caelles1,* K.-K. Maninis1,∗ J. Pont-Tuset1 L. Leal-Taix´e2 D. Cremers2 L. Van Gool1 1ETH Zurich¨

Supplemental Material for ”Face2Face: Real-time Face ...niessner/papers/2016/1facetoface/thies... · Supplemental Material for ”Face2Face: Real-time Face Capture and Reenactment

Supplemental Material for ”Face2Face: Real-time Face ...niessner/papers/2016/1facetoface/thies... · Supplemental Material for ”Face2Face: Real-time Face Capture and Reenactment

Supplemental Material for ”Face2Face: Real-time Face ...niessner/papers/... · 1. Optimization Framework Our Gauss-Newton optimization framework is based on the work of Thies et

Supplemental Material for ”Face2Face: Real-time Face ...niessner/papers/... · 1. Optimization Framework Our Gauss-Newton optimization framework is based on the work of Thies et

Introduction to Deep Learning (I2DL) · 2020-06-04 · Python “Introduction” •Why python: –Very easy to write development code thanks to an intuitive syntax –A plethora

Introduction to Deep Learning (I2DL) · 2020-06-04 · Python “Introduction” •Why python: –Very easy to write development code thanks to an intuitive syntax –A plethora

Data-driven Manifolds for Outdoor Motion · PDF fileData-driven Manifolds for Outdoor Motion Capture Gerard Pons-Moll 1, Laura Leal-Taix´e , Juergen Gall2;3, Bodo Rosenhahn1 1 Leibniz

Data-driven Manifolds for Outdoor Motion · PDF fileData-driven Manifolds for Outdoor Motion Capture Gerard Pons-Moll 1, Laura Leal-Taix´e , Juergen Gall2;3, Bodo Rosenhahn1 1 Leibniz

Learning by tracking: Siamese CNN for robust target association · 2016-05-03 · Learning by tracking: Siamese CNN for robust target association Laura Leal-Taix´e TU Munchen¨ Munich,

Learning by tracking: Siamese CNN for robust target association · 2016-05-03 · Learning by tracking: Siamese CNN for robust target association Laura Leal-Taix´e TU Munchen¨ Munich,

IN THE DIGITAL AGE - venice.coe.int Interference_Oct... · • Matthias Niessner, Professor and Head, Visual Computing Lab, Technical University of Munich • Guido Caldarelli, Professor

IN THE DIGITAL AGE - venice.coe.int Interference_Oct... · • Matthias Niessner, Professor and Head, Visual Computing Lab, Technical University of Munich • Guido Caldarelli, Professor

Cayman Islands: Assessment of the Supervision and ... · Nigel Davies, and Ms. Nancy Rawlings (all MFD); Messrs. Stefan Niessner (Banking Supervision, Deutsche Bundesbank), Timothy

Cayman Islands: Assessment of the Supervision and ... · Nigel Davies, and Ms. Nancy Rawlings (all MFD); Messrs. Stefan Niessner (Banking Supervision, Deutsche Bundesbank), Timothy

Taxonomy of generative models - TUMTaxonomy of generative models Prof. Leal-Taixé and Prof. Niessner Figure from Ian Goodfellow, Tutorial on Generative Adversarial /networks, 2017

Taxonomy of generative models - TUMTaxonomy of generative models Prof. Leal-Taixé and Prof. Niessner Figure from Ian Goodfellow, Tutorial on Generative Adversarial /networks, 2017

Deep Appearance MapsDeep Appearance Maps Maxim Maximov 1, Tobias Ritschel2, Laura Leal-Taix e , and Mario Fritz3 1Technical University of Munich 2University College London 3CISPA Helmholtz

Deep Appearance MapsDeep Appearance Maps Maxim Maximov 1, Tobias Ritschel2, Laura Leal-Taix e , and Mario Fritz3 1Technical University of Munich 2University College London 3CISPA Helmholtz

Advanced Deep Learning for Computer Vision · –Chat after the lecture –Post it on Moodle 15. Prof. Leal-Taixé and Prof. Niessner Project Ideas / Directions 16. Prof. Leal-Taixé

Advanced Deep Learning for Computer Vision · –Chat after the lecture –Post it on Moodle 15. Prof. Leal-Taixé and Prof. Niessner Project Ideas / Directions 16. Prof. Leal-Taixé

Lecture 11 Recap - GitHub Pages · Figure copyright and adapted from Ian Goodfellow, Tutorial on Generative Adversarial Networks, 2017. 44 Variational Autoencoders I2DL: Prof. Niessner,

Lecture 11 Recap - GitHub Pages · Figure copyright and adapted from Ian Goodfellow, Tutorial on Generative Adversarial Networks, 2017. 44 Variational Autoencoders I2DL: Prof. Niessner,

Real-Time Deformation of Subdivision Surfaces from Object ...niessner/papers/... · Schäfer et al. / Real-Time Deformation of Subdivision Surfaces from Object Collisions 3 bump maps.

Real-Time Deformation of Subdivision Surfaces from Object ...niessner/papers/... · Schäfer et al. / Real-Time Deformation of Subdivision Surfaces from Object Collisions 3 bump maps.

Volumetric and Multi-View CNNs for Object Classification on ...niessner/papers/... · available for object classification on 3D data. 1. Introduction Understanding 3D environments

Volumetric and Multi-View CNNs for Object Classification on ...niessner/papers/... · available for object classification on 3D data. 1. Introduction Understanding 3D environments

PlaneMatch: Patch Coplanarity Prediction for …PlaneMatch: Patch Coplanarity Prediction for Robust RGB-D Reconstruction Yifei Shi, Kai Xu, Matthias Niessner, Szymon Rusinkiewicz,

PlaneMatch: Patch Coplanarity Prediction for …PlaneMatch: Patch Coplanarity Prediction for Robust RGB-D Reconstruction Yifei Shi, Kai Xu, Matthias Niessner, Szymon Rusinkiewicz,