Beyond Convolutional Neural Networks...

15
Beyond Convolutional Neural Networks (CNN) Industrial AI Lab. Prof. Seungchul Lee

Transcript of Beyond Convolutional Neural Networks...

Page 1: Beyond Convolutional Neural Networks (CNN)i-systems.github.io/HSE545/iAI/DL/topics/06_Modern_CNN/02_Beyond_CNN.pdf · Beyond Convolutional Neural Networks (CNN) Industrial AI Lab.

Beyond Convolutional Neural Networks(CNN)

Industrial AI Lab.

Prof. Seungchul Lee

Page 2: Beyond Convolutional Neural Networks (CNN)i-systems.github.io/HSE545/iAI/DL/topics/06_Modern_CNN/02_Beyond_CNN.pdf · Beyond Convolutional Neural Networks (CNN) Industrial AI Lab.

Beyond Classification

2

Page 3: Beyond Convolutional Neural Networks (CNN)i-systems.github.io/HSE545/iAI/DL/topics/06_Modern_CNN/02_Beyond_CNN.pdf · Beyond Convolutional Neural Networks (CNN) Industrial AI Lab.

An Architecture for Many Applications

3

Page 4: Beyond Convolutional Neural Networks (CNN)i-systems.github.io/HSE545/iAI/DL/topics/06_Modern_CNN/02_Beyond_CNN.pdf · Beyond Convolutional Neural Networks (CNN) Industrial AI Lab.

An Architecture for Many Applications

4

Page 5: Beyond Convolutional Neural Networks (CNN)i-systems.github.io/HSE545/iAI/DL/topics/06_Modern_CNN/02_Beyond_CNN.pdf · Beyond Convolutional Neural Networks (CNN) Industrial AI Lab.

Beyond Classification

5

Page 6: Beyond Convolutional Neural Networks (CNN)i-systems.github.io/HSE545/iAI/DL/topics/06_Modern_CNN/02_Beyond_CNN.pdf · Beyond Convolutional Neural Networks (CNN) Industrial AI Lab.

Localization

• Single object per image

• Predict coordinates of a bounding box (x, y, w, h)

• Evaluate via Intersection over Union (IoU)

6

Page 7: Beyond Convolutional Neural Networks (CNN)i-systems.github.io/HSE545/iAI/DL/topics/06_Modern_CNN/02_Beyond_CNN.pdf · Beyond Convolutional Neural Networks (CNN) Industrial AI Lab.

Localization as Regression

7

Page 8: Beyond Convolutional Neural Networks (CNN)i-systems.github.io/HSE545/iAI/DL/topics/06_Modern_CNN/02_Beyond_CNN.pdf · Beyond Convolutional Neural Networks (CNN) Industrial AI Lab.

Localization as Regression

8

Page 9: Beyond Convolutional Neural Networks (CNN)i-systems.github.io/HSE545/iAI/DL/topics/06_Modern_CNN/02_Beyond_CNN.pdf · Beyond Convolutional Neural Networks (CNN) Industrial AI Lab.

Classification + Localization

• Use a pre-trained CNN on ImageNet (ex. ResNet)

• The "localization head" is trained separately with regression

• Possible end-to-end fine-tuning of both tasks

9

Page 10: Beyond Convolutional Neural Networks (CNN)i-systems.github.io/HSE545/iAI/DL/topics/06_Modern_CNN/02_Beyond_CNN.pdf · Beyond Convolutional Neural Networks (CNN) Industrial AI Lab.

Object Detection

• We don't know in advance the number of objects in the image.

• Object detection relies on object proposal and object classification– Object proposal: find regions of interest (ROIs) in the image

– Object classification: classify the object in these regions

• Two main families:– A grid in the image where each cell is a proposal (SSD, YOLO)

– Region proposal (SPP, MultiBox, Faster RCNN, ⋯)

10

Page 11: Beyond Convolutional Neural Networks (CNN)i-systems.github.io/HSE545/iAI/DL/topics/06_Modern_CNN/02_Beyond_CNN.pdf · Beyond Convolutional Neural Networks (CNN) Industrial AI Lab.

Object Detection with R-CNNs

• R-CNN: – Find regions that we think have objects.

– Use CNN to classify

11

Page 12: Beyond Convolutional Neural Networks (CNN)i-systems.github.io/HSE545/iAI/DL/topics/06_Modern_CNN/02_Beyond_CNN.pdf · Beyond Convolutional Neural Networks (CNN) Industrial AI Lab.

Segmentation

• Output a class map for each pixel (here: dog vs background)

• Instance segmentation: specify each object instance as well (two dogs have different instances)

• This can be done through object detection + segmentation

12

Page 13: Beyond Convolutional Neural Networks (CNN)i-systems.github.io/HSE545/iAI/DL/topics/06_Modern_CNN/02_Beyond_CNN.pdf · Beyond Convolutional Neural Networks (CNN) Industrial AI Lab.

Semantic Segmentation: FCNs

• FCN: Fully Convolutional Network– Network designed with all convolutional layers, with down-sampling and up-sampling operations

13

Page 14: Beyond Convolutional Neural Networks (CNN)i-systems.github.io/HSE545/iAI/DL/topics/06_Modern_CNN/02_Beyond_CNN.pdf · Beyond Convolutional Neural Networks (CNN) Industrial AI Lab.

Image Captioning using RNNs

14

Page 15: Beyond Convolutional Neural Networks (CNN)i-systems.github.io/HSE545/iAI/DL/topics/06_Modern_CNN/02_Beyond_CNN.pdf · Beyond Convolutional Neural Networks (CNN) Industrial AI Lab.

Image Captioning using RNNs

15


Convolutional Neural Network - CNN - UFPR

Convolutional Neural Network - CNN - UFPR

Convolutional Neural Nets · Convolutional Neural Network (CNN) Convolutional layers localize connections to a small window of input For example, the input node x 3 connects to the

Convolutional Neural Nets · Convolutional Neural Network (CNN) Convolutional layers localize connections to a small window of input For example, the input node x 3 connects to the

CNN-MGP: Convolutional Neural Networks for Metagenomics ... › content › pdf › 10.1007 › s12539-018-0313-4.pdfA convolutional neural network (CNN) is a special type of neural

CNN-MGP: Convolutional Neural Networks for Metagenomics ... › content › pdf › 10.1007 › s12539-018-0313-4.pdfA convolutional neural network (CNN) is a special type of neural

COMPLEX INPUT CONVOLUTIONAL NEURAL NETWORKS FOR …hero.engin.umich.edu/wp-content/uploads/sites/335/... · convolutional neural network (CNN) that accepts fully complex input features.

COMPLEX INPUT CONVOLUTIONAL NEURAL NETWORKS FOR …hero.engin.umich.edu/wp-content/uploads/sites/335/... · convolutional neural network (CNN) that accepts fully complex input features.

Convolutional Neural Networks with Alternately Updated … · the convolutional neural network (CNN) ... also inspires a series of ResNet variations, including ... competitive results

Convolutional Neural Networks with Alternately Updated … · the convolutional neural network (CNN) ... also inspires a series of ResNet variations, including ... competitive results

Convolutional Neural Networks for Document Image …Classifying and grouping document images into known ... image classification using Convolutional Neural Networks (CNNs). CNN is

Convolutional Neural Networks for Document Image …Classifying and grouping document images into known ... image classification using Convolutional Neural Networks (CNNs). CNN is

Convolutional Neural Network Hyper-Parameters Optimization ...€¦ · applications, Convolutional Neural Network (CNN) is the most widely used technique for image classification.

Convolutional Neural Network Hyper-Parameters Optimization ...€¦ · applications, Convolutional Neural Network (CNN) is the most widely used technique for image classification.

Convolutional Neural Network - CNN · Introduction CNN Layers CNN Models Popular Frameworks Papers References Convolutional Neural Network - CNN Eduardo Todt, Bruno Alexandre Krinski

Convolutional Neural Network - CNN · Introduction CNN Layers CNN Models Popular Frameworks Papers References Convolutional Neural Network - CNN Eduardo Todt, Bruno Alexandre Krinski

Neural Network Convolutional with recognition Face · 2015-06-03 · Using Convolutional Neural Network (CNN) to recognize person on the image Face recognition with CNN Face recognition

Neural Network Convolutional with recognition Face · 2015-06-03 · Using Convolutional Neural Network (CNN) to recognize person on the image Face recognition with CNN Face recognition

AnApproachtoSemanticandStructuralFeaturesLearningfor ...downloads.hindawi.com/journals/mpe/2020/6038619.pdf3.2. Convolutional Neural Network. Convolutional neural network(CNN)isone

AnApproachtoSemanticandStructuralFeaturesLearningfor ...downloads.hindawi.com/journals/mpe/2020/6038619.pdf3.2. Convolutional Neural Network. Convolutional neural network(CNN)isone

Convolutional Neural Networks (CNN)www.深度学习.top/Talks/CNN1.pdf · Convolutional Neural Networks (CNN) 2016.11.3 SCHOOL OF MATHEMATICAL SCIENCE, PEKING UNIVERSITY

Convolutional Neural Networks (CNN)www.深度学习.top/Talks/CNN1.pdf · Convolutional Neural Networks (CNN) 2016.11.3 SCHOOL OF MATHEMATICAL SCIENCE, PEKING UNIVERSITY

Deep Convolutional Neural Networks for Forensic Age ...blog.hakzone.info/.../uploads/2020/05/CNN-for-Forensic-Age-Estimation.… · 3. The Convolutional Neural Network (CNN) The most

Deep Convolutional Neural Networks for Forensic Age ...blog.hakzone.info/.../uploads/2020/05/CNN-for-Forensic-Age-Estimation.… · 3. The Convolutional Neural Network (CNN) The most

Convolutional Neural Networks - University of Pennsylvaniacis581/Lectures/Fall... · 2017-12-05 · Input: Convolutional Neural Network (CNN) Early layers learn to detect low level

Convolutional Neural Networks - University of Pennsylvaniacis581/Lectures/Fall... · 2017-12-05 · Input: Convolutional Neural Network (CNN) Early layers learn to detect low level

Convolutional Networks - Université de Montréal · Convolutional Neural Networks •Object detection systems based on the deep convolutional neural network (CNN) have recently made

Convolutional Networks - Université de Montréal · Convolutional Neural Networks •Object detection systems based on the deep convolutional neural network (CNN) have recently made

A-CNN: Annularly Convolutional Neural Networks on Point Clouds · Figure 1: The proposed annularly convolutional neural networks (A-CNN) model on point clouds to perform classification,

A-CNN: Annularly Convolutional Neural Networks on Point Clouds · Figure 1: The proposed annularly convolutional neural networks (A-CNN) model on point clouds to perform classification,

ViP-CNN: Visual Phrase Guided Convolutional Neural ......IEEE 2017 Conference on Computer Vision and Pattern Recognition ViP-CNN: Visual Phrase Guided Convolutional Neural Network

ViP-CNN: Visual Phrase Guided Convolutional Neural ......IEEE 2017 Conference on Computer Vision and Pattern Recognition ViP-CNN: Visual Phrase Guided Convolutional Neural Network

T-CNN: Tubelets with Convolutional Neural Networks for ...1 T-CNN: Tubelets with Convolutional Neural Networks for Object Detection from Videos Kai Kang, Hongsheng Li, Junjie Yan,

T-CNN: Tubelets with Convolutional Neural Networks for ...1 T-CNN: Tubelets with Convolutional Neural Networks for Object Detection from Videos Kai Kang, Hongsheng Li, Junjie Yan,

Performance Analysis of GPU-Based Convolutional Neural ...howie/publications/GPU-CNN-ICPP16.pdfPerformance Analysis of GPU-based Convolutional Neural Networks Xiaqing Li †‡§,

Performance Analysis of GPU-Based Convolutional Neural ...howie/publications/GPU-CNN-ICPP16.pdfPerformance Analysis of GPU-based Convolutional Neural Networks Xiaqing Li †‡§,

Convolutional Neural Networkslvelho.impa.br/ip16/proj/slides/ImageNet_DeepCNN.pdf · Convolutional Neural Networks 7 Motivation It was one of the largest CNN trained with the ImageNet

Convolutional Neural Networkslvelho.impa.br/ip16/proj/slides/ImageNet_DeepCNN.pdf · Convolutional Neural Networks 7 Motivation It was one of the largest CNN trained with the ImageNet

O-CNN: Octree-based Convolutional Neural Networks for … · O-CNN: Octree-based Convolutional Neural Networks for 3D ... object classification, ... Octree-based Convolutional Neural

O-CNN: Octree-based Convolutional Neural Networks for … · O-CNN: Octree-based Convolutional Neural Networks for 3D ... object classification, ... Octree-based Convolutional Neural