Seminar Rita Osadchycs.haifa.ac.il/~rita/courses_moodle/object_recognition_seminar/Intro.pdf ·...
Transcript of Seminar Rita Osadchycs.haifa.ac.il/~rita/courses_moodle/object_recognition_seminar/Intro.pdf ·...
![Page 1: Seminar Rita Osadchycs.haifa.ac.il/~rita/courses_moodle/object_recognition_seminar/Intro.pdf · Object Recognition Seminar Rita Osadchy So what does object recognition involve? Verification:](https://reader034.fdocuments.in/reader034/viewer/2022042709/5f4e1d0dbbfe595a094d057e/html5/thumbnails/1.jpg)
Object Recognition
Seminar
Rita Osadchy
![Page 2: Seminar Rita Osadchycs.haifa.ac.il/~rita/courses_moodle/object_recognition_seminar/Intro.pdf · Object Recognition Seminar Rita Osadchy So what does object recognition involve? Verification:](https://reader034.fdocuments.in/reader034/viewer/2022042709/5f4e1d0dbbfe595a094d057e/html5/thumbnails/2.jpg)
So what does object recognition involve?
![Page 3: Seminar Rita Osadchycs.haifa.ac.il/~rita/courses_moodle/object_recognition_seminar/Intro.pdf · Object Recognition Seminar Rita Osadchy So what does object recognition involve? Verification:](https://reader034.fdocuments.in/reader034/viewer/2022042709/5f4e1d0dbbfe595a094d057e/html5/thumbnails/3.jpg)
Verification: is that a bus?
![Page 4: Seminar Rita Osadchycs.haifa.ac.il/~rita/courses_moodle/object_recognition_seminar/Intro.pdf · Object Recognition Seminar Rita Osadchy So what does object recognition involve? Verification:](https://reader034.fdocuments.in/reader034/viewer/2022042709/5f4e1d0dbbfe595a094d057e/html5/thumbnails/4.jpg)
Detection: locate the cars in the image
![Page 5: Seminar Rita Osadchycs.haifa.ac.il/~rita/courses_moodle/object_recognition_seminar/Intro.pdf · Object Recognition Seminar Rita Osadchy So what does object recognition involve? Verification:](https://reader034.fdocuments.in/reader034/viewer/2022042709/5f4e1d0dbbfe595a094d057e/html5/thumbnails/5.jpg)
Identification: is that a picture of Mao?
![Page 6: Seminar Rita Osadchycs.haifa.ac.il/~rita/courses_moodle/object_recognition_seminar/Intro.pdf · Object Recognition Seminar Rita Osadchy So what does object recognition involve? Verification:](https://reader034.fdocuments.in/reader034/viewer/2022042709/5f4e1d0dbbfe595a094d057e/html5/thumbnails/6.jpg)
Object categorization
sky
building
flag
wallbanner
bus
cars
bus
face
street lamp
![Page 7: Seminar Rita Osadchycs.haifa.ac.il/~rita/courses_moodle/object_recognition_seminar/Intro.pdf · Object Recognition Seminar Rita Osadchy So what does object recognition involve? Verification:](https://reader034.fdocuments.in/reader034/viewer/2022042709/5f4e1d0dbbfe595a094d057e/html5/thumbnails/7.jpg)
Challenges 1: view point variation
Michelangelo 1475-1564
![Page 8: Seminar Rita Osadchycs.haifa.ac.il/~rita/courses_moodle/object_recognition_seminar/Intro.pdf · Object Recognition Seminar Rita Osadchy So what does object recognition involve? Verification:](https://reader034.fdocuments.in/reader034/viewer/2022042709/5f4e1d0dbbfe595a094d057e/html5/thumbnails/8.jpg)
Challenges 2: illumination
slide credit: S. Ullman
![Page 9: Seminar Rita Osadchycs.haifa.ac.il/~rita/courses_moodle/object_recognition_seminar/Intro.pdf · Object Recognition Seminar Rita Osadchy So what does object recognition involve? Verification:](https://reader034.fdocuments.in/reader034/viewer/2022042709/5f4e1d0dbbfe595a094d057e/html5/thumbnails/9.jpg)
Challenges 3: occlusion
Magritte, 1957
![Page 10: Seminar Rita Osadchycs.haifa.ac.il/~rita/courses_moodle/object_recognition_seminar/Intro.pdf · Object Recognition Seminar Rita Osadchy So what does object recognition involve? Verification:](https://reader034.fdocuments.in/reader034/viewer/2022042709/5f4e1d0dbbfe595a094d057e/html5/thumbnails/10.jpg)
Challenges 4: scale
![Page 11: Seminar Rita Osadchycs.haifa.ac.il/~rita/courses_moodle/object_recognition_seminar/Intro.pdf · Object Recognition Seminar Rita Osadchy So what does object recognition involve? Verification:](https://reader034.fdocuments.in/reader034/viewer/2022042709/5f4e1d0dbbfe595a094d057e/html5/thumbnails/11.jpg)
Challenges 5: deformation
Xu, Beihong 1943
![Page 12: Seminar Rita Osadchycs.haifa.ac.il/~rita/courses_moodle/object_recognition_seminar/Intro.pdf · Object Recognition Seminar Rita Osadchy So what does object recognition involve? Verification:](https://reader034.fdocuments.in/reader034/viewer/2022042709/5f4e1d0dbbfe595a094d057e/html5/thumbnails/12.jpg)
Challenges 7: intra-class variation
![Page 13: Seminar Rita Osadchycs.haifa.ac.il/~rita/courses_moodle/object_recognition_seminar/Intro.pdf · Object Recognition Seminar Rita Osadchy So what does object recognition involve? Verification:](https://reader034.fdocuments.in/reader034/viewer/2022042709/5f4e1d0dbbfe595a094d057e/html5/thumbnails/13.jpg)
Recognition Steps
Features Image
representation
Train Images
Preprocessing
Features Image
representation
Test Images
Recognition
classifier
labels
classifier output
![Page 14: Seminar Rita Osadchycs.haifa.ac.il/~rita/courses_moodle/object_recognition_seminar/Intro.pdf · Object Recognition Seminar Rita Osadchy So what does object recognition involve? Verification:](https://reader034.fdocuments.in/reader034/viewer/2022042709/5f4e1d0dbbfe595a094d057e/html5/thumbnails/14.jpg)
Object Recognition System
conveyer belt
sorting
chamber
salmon
sea bass
camera
classifier
![Page 15: Seminar Rita Osadchycs.haifa.ac.il/~rita/courses_moodle/object_recognition_seminar/Intro.pdf · Object Recognition Seminar Rita Osadchy So what does object recognition involve? Verification:](https://reader034.fdocuments.in/reader034/viewer/2022042709/5f4e1d0dbbfe595a094d057e/html5/thumbnails/15.jpg)
How to design a PR system? Collect data and classify by hand
salmon salmon salmonsea bass sea bass sea bass
Preprocess by segmenting fish from background
Extract possibly discriminating features length, lightness,width,number of fins,etc.
Classifier design
Choose model
Train classifier on part of collected data (training data)
Test classifier on the rest of collected data (test data) i.e. the data not used for training Should classify new data (new fish images) well
![Page 16: Seminar Rita Osadchycs.haifa.ac.il/~rita/courses_moodle/object_recognition_seminar/Intro.pdf · Object Recognition Seminar Rita Osadchy So what does object recognition involve? Verification:](https://reader034.fdocuments.in/reader034/viewer/2022042709/5f4e1d0dbbfe595a094d057e/html5/thumbnails/16.jpg)
Interest Point Detectors
• Basic requirements:
– Sparse
– Informative
– Repeatable
• Invariance
– Rotation
– Scale (Similarity)
– Affine
Lecture 3
![Page 17: Seminar Rita Osadchycs.haifa.ac.il/~rita/courses_moodle/object_recognition_seminar/Intro.pdf · Object Recognition Seminar Rita Osadchy So what does object recognition involve? Verification:](https://reader034.fdocuments.in/reader034/viewer/2022042709/5f4e1d0dbbfe595a094d057e/html5/thumbnails/17.jpg)
17
Learned models of local features, and got object outline from
background subtraction:
Objects may then be found under occlusion and 3D rotation
Images fro
m: D
avid
Low
e,
Ob
ject
Reco
gn
itio
n f
rom
Lo
cal S
cale
-In
vari
an
t F
eatu
res P
roc. of
the Inte
rnational C
onfe
rence o
n C
om
pute
r V
isio
n,C
orf
u (
Sept. 1
999)
Lecture 3
Recognizing Specific Objects
![Page 18: Seminar Rita Osadchycs.haifa.ac.il/~rita/courses_moodle/object_recognition_seminar/Intro.pdf · Object Recognition Seminar Rita Osadchy So what does object recognition involve? Verification:](https://reader034.fdocuments.in/reader034/viewer/2022042709/5f4e1d0dbbfe595a094d057e/html5/thumbnails/18.jpg)
Bag of Features
=
Lecture 3
![Page 19: Seminar Rita Osadchycs.haifa.ac.il/~rita/courses_moodle/object_recognition_seminar/Intro.pdf · Object Recognition Seminar Rita Osadchy So what does object recognition involve? Verification:](https://reader034.fdocuments.in/reader034/viewer/2022042709/5f4e1d0dbbfe595a094d057e/html5/thumbnails/19.jpg)
Pros: fairly flexible and computationally efficient
Cons: problems with large clutter
Different objects, but Similar
representations;Similar objects, different
representations;
Bag of FeaturesLecture 3
![Page 20: Seminar Rita Osadchycs.haifa.ac.il/~rita/courses_moodle/object_recognition_seminar/Intro.pdf · Object Recognition Seminar Rita Osadchy So what does object recognition involve? Verification:](https://reader034.fdocuments.in/reader034/viewer/2022042709/5f4e1d0dbbfe595a094d057e/html5/thumbnails/20.jpg)
• Computing bags of features on sub-
windows of the whole image.
level 2
Lazebnik, Schmid & Ponce (CVPR 2006)
Beyond Bags of FeaturesLecture 3
![Page 21: Seminar Rita Osadchycs.haifa.ac.il/~rita/courses_moodle/object_recognition_seminar/Intro.pdf · Object Recognition Seminar Rita Osadchy So what does object recognition involve? Verification:](https://reader034.fdocuments.in/reader034/viewer/2022042709/5f4e1d0dbbfe595a094d057e/html5/thumbnails/21.jpg)
Convolutional Neural Networks
• Learn all in one deep architecture:
– low level features
– high level representations
– context
– classifiers
• Efficient Classification
• Efficient Detection
• Scalable to very large sets and large number of
categories
Lecture 4
Krizhevsky, I. Sutskever, and G. Hinton. ImageNet classification with deep convolutional neural networks
![Page 22: Seminar Rita Osadchycs.haifa.ac.il/~rita/courses_moodle/object_recognition_seminar/Intro.pdf · Object Recognition Seminar Rita Osadchy So what does object recognition involve? Verification:](https://reader034.fdocuments.in/reader034/viewer/2022042709/5f4e1d0dbbfe595a094d057e/html5/thumbnails/22.jpg)
Convolutional Neural Networks
Lecture 4
![Page 23: Seminar Rita Osadchycs.haifa.ac.il/~rita/courses_moodle/object_recognition_seminar/Intro.pdf · Object Recognition Seminar Rita Osadchy So what does object recognition involve? Verification:](https://reader034.fdocuments.in/reader034/viewer/2022042709/5f4e1d0dbbfe595a094d057e/html5/thumbnails/23.jpg)
Training the Network
p1=0.03
p0=0.01
pN-1=0.02
pN=0.015
cat
banana
mug
dog
0
1
0
0
𝑋 𝑌
Compute the gradient with respect to the parameters of the
network.
Lecture 4
![Page 24: Seminar Rita Osadchycs.haifa.ac.il/~rita/courses_moodle/object_recognition_seminar/Intro.pdf · Object Recognition Seminar Rita Osadchy So what does object recognition involve? Verification:](https://reader034.fdocuments.in/reader034/viewer/2022042709/5f4e1d0dbbfe595a094d057e/html5/thumbnails/24.jpg)
Very Deep NetworksLecture 4
![Page 25: Seminar Rita Osadchycs.haifa.ac.il/~rita/courses_moodle/object_recognition_seminar/Intro.pdf · Object Recognition Seminar Rita Osadchy So what does object recognition involve? Verification:](https://reader034.fdocuments.in/reader034/viewer/2022042709/5f4e1d0dbbfe595a094d057e/html5/thumbnails/25.jpg)
Very Deep NetworksLecture 4
• K. Simonyan and A. Zisserman Very Deep Convolutional Networks for Large-Scale Image Recognition
• K. He, X. Zhang, S. Ren, and J. Sun: Deep Residual Learning for Image Recognition
![Page 26: Seminar Rita Osadchycs.haifa.ac.il/~rita/courses_moodle/object_recognition_seminar/Intro.pdf · Object Recognition Seminar Rita Osadchy So what does object recognition involve? Verification:](https://reader034.fdocuments.in/reader034/viewer/2022042709/5f4e1d0dbbfe595a094d057e/html5/thumbnails/26.jpg)
Apply classifier at Scale / position range to search over
DetectionLecture 5
![Page 27: Seminar Rita Osadchycs.haifa.ac.il/~rita/courses_moodle/object_recognition_seminar/Intro.pdf · Object Recognition Seminar Rita Osadchy So what does object recognition involve? Verification:](https://reader034.fdocuments.in/reader034/viewer/2022042709/5f4e1d0dbbfe595a094d057e/html5/thumbnails/27.jpg)
DetectionLecture 5
![Page 28: Seminar Rita Osadchycs.haifa.ac.il/~rita/courses_moodle/object_recognition_seminar/Intro.pdf · Object Recognition Seminar Rita Osadchy So what does object recognition involve? Verification:](https://reader034.fdocuments.in/reader034/viewer/2022042709/5f4e1d0dbbfe595a094d057e/html5/thumbnails/28.jpg)
DetectionLecture 5
![Page 29: Seminar Rita Osadchycs.haifa.ac.il/~rita/courses_moodle/object_recognition_seminar/Intro.pdf · Object Recognition Seminar Rita Osadchy So what does object recognition involve? Verification:](https://reader034.fdocuments.in/reader034/viewer/2022042709/5f4e1d0dbbfe595a094d057e/html5/thumbnails/29.jpg)
– Combine detection over space and scale.
DetectionLecture 5
![Page 30: Seminar Rita Osadchycs.haifa.ac.il/~rita/courses_moodle/object_recognition_seminar/Intro.pdf · Object Recognition Seminar Rita Osadchy So what does object recognition involve? Verification:](https://reader034.fdocuments.in/reader034/viewer/2022042709/5f4e1d0dbbfe595a094d057e/html5/thumbnails/30.jpg)
Deep Learning in Object
Detection
SVM classifier
Lecture 5
Rich feature hierarchies for accurate object detection and semantic segmentation. R. Girshick et al.
![Page 31: Seminar Rita Osadchycs.haifa.ac.il/~rita/courses_moodle/object_recognition_seminar/Intro.pdf · Object Recognition Seminar Rita Osadchy So what does object recognition involve? Verification:](https://reader034.fdocuments.in/reader034/viewer/2022042709/5f4e1d0dbbfe595a094d057e/html5/thumbnails/31.jpg)
Faster R-CNN
Faster R-CNN is a single,
unified network for object
detection. The RPN module
serves as the ‘attention’ of this
unified network.
Lecture 5
![Page 32: Seminar Rita Osadchycs.haifa.ac.il/~rita/courses_moodle/object_recognition_seminar/Intro.pdf · Object Recognition Seminar Rita Osadchy So what does object recognition involve? Verification:](https://reader034.fdocuments.in/reader034/viewer/2022042709/5f4e1d0dbbfe595a094d057e/html5/thumbnails/32.jpg)
Transfer LearningLecture 6
J. Donahue et.al. DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognition.
![Page 33: Seminar Rita Osadchycs.haifa.ac.il/~rita/courses_moodle/object_recognition_seminar/Intro.pdf · Object Recognition Seminar Rita Osadchy So what does object recognition involve? Verification:](https://reader034.fdocuments.in/reader034/viewer/2022042709/5f4e1d0dbbfe595a094d057e/html5/thumbnails/33.jpg)
One Shot LearningLecture 6
G. Koch, R. Zemel, and R. Salakhutdinov Siamese Neural Networks for One-shot Image Recognition
![Page 34: Seminar Rita Osadchycs.haifa.ac.il/~rita/courses_moodle/object_recognition_seminar/Intro.pdf · Object Recognition Seminar Rita Osadchy So what does object recognition involve? Verification:](https://reader034.fdocuments.in/reader034/viewer/2022042709/5f4e1d0dbbfe595a094d057e/html5/thumbnails/34.jpg)
Few-Shot Learning
• Prototypical Networks
• Maps examples to embedding such that examples of a given class
are close together
• Calculates a prototype (mean vector) for every class
• Maps test instances to the same embedding
• Uses softmax over distance to prototype for label prediction
test
Lecture 6
Prototypical Networks for Few-shot Learning (2017)Jake Snell, Kevin Swersky, Richard S. Zemel
![Page 35: Seminar Rita Osadchycs.haifa.ac.il/~rita/courses_moodle/object_recognition_seminar/Intro.pdf · Object Recognition Seminar Rita Osadchy So what does object recognition involve? Verification:](https://reader034.fdocuments.in/reader034/viewer/2022042709/5f4e1d0dbbfe595a094d057e/html5/thumbnails/35.jpg)
Describing Objects with Attributes
• Shift the goal of recognition from naming
to describing
Discover/detect new
categories
Lecture 7
Describing Objects by Their Attributes, A. Farhadi, I. Endres, D. Hoiem, and D. Forsyth
![Page 36: Seminar Rita Osadchycs.haifa.ac.il/~rita/courses_moodle/object_recognition_seminar/Intro.pdf · Object Recognition Seminar Rita Osadchy So what does object recognition involve? Verification:](https://reader034.fdocuments.in/reader034/viewer/2022042709/5f4e1d0dbbfe595a094d057e/html5/thumbnails/36.jpg)
Improvement
Lecture 7
Learning To Detect Unseen Object Classes by Between-Class Attribute Transfer, C. Lampert, H. Nickisch, and S. Harmeling
![Page 37: Seminar Rita Osadchycs.haifa.ac.il/~rita/courses_moodle/object_recognition_seminar/Intro.pdf · Object Recognition Seminar Rita Osadchy So what does object recognition involve? Verification:](https://reader034.fdocuments.in/reader034/viewer/2022042709/5f4e1d0dbbfe595a094d057e/html5/thumbnails/37.jpg)
Deep Learning for Faces
Survey
Lecture 8
![Page 38: Seminar Rita Osadchycs.haifa.ac.il/~rita/courses_moodle/object_recognition_seminar/Intro.pdf · Object Recognition Seminar Rita Osadchy So what does object recognition involve? Verification:](https://reader034.fdocuments.in/reader034/viewer/2022042709/5f4e1d0dbbfe595a094d057e/html5/thumbnails/38.jpg)
Image Descriptions (Captioning)
Uses CNNs and RNNs
Lecture 9
Deep Visual-Semantic Alignments for Generating Image Descriptions, A. Karpathy, Li Fei-Fei.
![Page 39: Seminar Rita Osadchycs.haifa.ac.il/~rita/courses_moodle/object_recognition_seminar/Intro.pdf · Object Recognition Seminar Rita Osadchy So what does object recognition involve? Verification:](https://reader034.fdocuments.in/reader034/viewer/2022042709/5f4e1d0dbbfe595a094d057e/html5/thumbnails/39.jpg)
VQA: Visual Question
Answering
Lecture 9
VQA: Visual Question Answering. Antol et al.
![Page 40: Seminar Rita Osadchycs.haifa.ac.il/~rita/courses_moodle/object_recognition_seminar/Intro.pdf · Object Recognition Seminar Rita Osadchy So what does object recognition involve? Verification:](https://reader034.fdocuments.in/reader034/viewer/2022042709/5f4e1d0dbbfe595a094d057e/html5/thumbnails/40.jpg)
ResultsLecture 9
![Page 41: Seminar Rita Osadchycs.haifa.ac.il/~rita/courses_moodle/object_recognition_seminar/Intro.pdf · Object Recognition Seminar Rita Osadchy So what does object recognition involve? Verification:](https://reader034.fdocuments.in/reader034/viewer/2022042709/5f4e1d0dbbfe595a094d057e/html5/thumbnails/41.jpg)
Generative Adversarial
Networks (GAN)• GAN takes game-theoretic approach: learn to generate from training distribution
through 2-player game.
𝐺 𝑧tries to fool the
discriminator by
generating real-
looking images
𝐷 𝑥tries to
distinguish
between real and
fake images
Lecture 10
![Page 42: Seminar Rita Osadchycs.haifa.ac.il/~rita/courses_moodle/object_recognition_seminar/Intro.pdf · Object Recognition Seminar Rita Osadchy So what does object recognition involve? Verification:](https://reader034.fdocuments.in/reader034/viewer/2022042709/5f4e1d0dbbfe595a094d057e/html5/thumbnails/42.jpg)
GAN: Convolutional Architecture
Radford et al, “Unsupervised Representation Learning with Deep Convolutional Generative Adversarial
Networks”, 2016
Lecture 10
![Page 43: Seminar Rita Osadchycs.haifa.ac.il/~rita/courses_moodle/object_recognition_seminar/Intro.pdf · Object Recognition Seminar Rita Osadchy So what does object recognition involve? Verification:](https://reader034.fdocuments.in/reader034/viewer/2022042709/5f4e1d0dbbfe595a094d057e/html5/thumbnails/43.jpg)
Image Transfer
Lecture 10
![Page 44: Seminar Rita Osadchycs.haifa.ac.il/~rita/courses_moodle/object_recognition_seminar/Intro.pdf · Object Recognition Seminar Rita Osadchy So what does object recognition involve? Verification:](https://reader034.fdocuments.in/reader034/viewer/2022042709/5f4e1d0dbbfe595a094d057e/html5/thumbnails/44.jpg)
CycleGAN
Discriminator 𝐷𝑌 pushes Generator 𝐺 to
translate X into outputs indistinguishable
from domain Y , and vice versa for 𝐷𝑋and F.
forward cycle-
consistency loss:
x → G(x) → F(G(x)) ≈ x
backward cycle-
consistency loss:
y → F(y) → G(F(y)) ≈ y
Lecture 10
![Page 45: Seminar Rita Osadchycs.haifa.ac.il/~rita/courses_moodle/object_recognition_seminar/Intro.pdf · Object Recognition Seminar Rita Osadchy So what does object recognition involve? Verification:](https://reader034.fdocuments.in/reader034/viewer/2022042709/5f4e1d0dbbfe595a094d057e/html5/thumbnails/45.jpg)
Data Augmentation Generative
Adversarial Network (DAGAN):
• Generator: – an encoder maps the
input image into latent space
– a random vector is concatenated with the latent vector
– concatenated vector is passed to the decoder to generate an image
• Discriminator: similar to GAN
A. Antoniou, et al., “Data augmentation generative adversarial
Lecture 10
![Page 46: Seminar Rita Osadchycs.haifa.ac.il/~rita/courses_moodle/object_recognition_seminar/Intro.pdf · Object Recognition Seminar Rita Osadchy So what does object recognition involve? Verification:](https://reader034.fdocuments.in/reader034/viewer/2022042709/5f4e1d0dbbfe595a094d057e/html5/thumbnails/46.jpg)
Lecture 10
![Page 47: Seminar Rita Osadchycs.haifa.ac.il/~rita/courses_moodle/object_recognition_seminar/Intro.pdf · Object Recognition Seminar Rita Osadchy So what does object recognition involve? Verification:](https://reader034.fdocuments.in/reader034/viewer/2022042709/5f4e1d0dbbfe595a094d057e/html5/thumbnails/47.jpg)
Video Classification
• Using CNN – Naïve Approach
Lecture 11
Kapathy et al.: Large‐scale Video Classification with Convolutional Neural Networks
![Page 48: Seminar Rita Osadchycs.haifa.ac.il/~rita/courses_moodle/object_recognition_seminar/Intro.pdf · Object Recognition Seminar Rita Osadchy So what does object recognition involve? Verification:](https://reader034.fdocuments.in/reader034/viewer/2022042709/5f4e1d0dbbfe595a094d057e/html5/thumbnails/48.jpg)
Video Classification
• Using CNN – Naïve Approach
Temporal fusion
Lecture 11
Kapathy et al.: Large‐scale Video Classification with Convolutional Neural Networks
![Page 49: Seminar Rita Osadchycs.haifa.ac.il/~rita/courses_moodle/object_recognition_seminar/Intro.pdf · Object Recognition Seminar Rita Osadchy So what does object recognition involve? Verification:](https://reader034.fdocuments.in/reader034/viewer/2022042709/5f4e1d0dbbfe595a094d057e/html5/thumbnails/49.jpg)
Video Classification
• Modern Approaches
Lecture 11
K. Simonyan, A. Zissermann: Two‐Stream Convolutional Networks for Action Recognition in Videos.
![Page 50: Seminar Rita Osadchycs.haifa.ac.il/~rita/courses_moodle/object_recognition_seminar/Intro.pdf · Object Recognition Seminar Rita Osadchy So what does object recognition involve? Verification:](https://reader034.fdocuments.in/reader034/viewer/2022042709/5f4e1d0dbbfe595a094d057e/html5/thumbnails/50.jpg)
Multi-task training
Training SetRandom
Batch
Lecture 12
![Page 51: Seminar Rita Osadchycs.haifa.ac.il/~rita/courses_moodle/object_recognition_seminar/Intro.pdf · Object Recognition Seminar Rita Osadchy So what does object recognition involve? Verification:](https://reader034.fdocuments.in/reader034/viewer/2022042709/5f4e1d0dbbfe595a094d057e/html5/thumbnails/51.jpg)
Continual Learning
Training Set
at Time T1
Training Set at
Time T2
Training Set
at Time T3
…
Lecture 12
![Page 52: Seminar Rita Osadchycs.haifa.ac.il/~rita/courses_moodle/object_recognition_seminar/Intro.pdf · Object Recognition Seminar Rita Osadchy So what does object recognition involve? Verification:](https://reader034.fdocuments.in/reader034/viewer/2022042709/5f4e1d0dbbfe595a094d057e/html5/thumbnails/52.jpg)
Continual Learning
• Tasks are learned sequentially over time.
• At time Ti, the data of the tasks T1…Ti-1 is no longer available.
• Leads to forgetting of previously learned tasks.
• Termed “Catastrophic Forgetting”.
Possible Solution:
• Measure the importance of learnable parameters in NN for the
task, constrain their change in learning a new task.
• This could work because deep networks are highly
overparametrized
James Kirkpatrick et al., “Overcoming catastrophic forgetting in neural networks”
Lecture 12
![Page 53: Seminar Rita Osadchycs.haifa.ac.il/~rita/courses_moodle/object_recognition_seminar/Intro.pdf · Object Recognition Seminar Rita Osadchy So what does object recognition involve? Verification:](https://reader034.fdocuments.in/reader034/viewer/2022042709/5f4e1d0dbbfe595a094d057e/html5/thumbnails/53.jpg)
Adversarial Examples
Lecture 12
Explaining and Harnessing Adversarial Examples by Goodfellow et al.
![Page 54: Seminar Rita Osadchycs.haifa.ac.il/~rita/courses_moodle/object_recognition_seminar/Intro.pdf · Object Recognition Seminar Rita Osadchy So what does object recognition involve? Verification:](https://reader034.fdocuments.in/reader034/viewer/2022042709/5f4e1d0dbbfe595a094d057e/html5/thumbnails/54.jpg)
Adversarial Examples: Imperceptible Noise
Intriguing properties of neural networks by Szegedy et al.