CNN RNN Autoencoder - UNISTisystems.unist.ac.kr/wp-content/uploads/sites/209/2017/... · 2018. 12....

48
Today 1. Machine Learning 2. Deep Learning CNN RNN Autoencoder 3. Deep Learning of Things (DoT) 4. Epilog 1

Transcript of CNN RNN Autoencoder - UNISTisystems.unist.ac.kr/wp-content/uploads/sites/209/2017/... · 2018. 12....

Page 1: CNN RNN Autoencoder - UNISTisystems.unist.ac.kr/wp-content/uploads/sites/209/2017/... · 2018. 12. 28. · CNN RNN Autoencoder. Today 1. Machine*Learning 2. Deep*Learning –CNN –RNN

Today

1. Machine  Learning

2. Deep  Learning– CNN– RNN– Autoencoder

3. Deep  Learning  of  Things  (DoT)

4. Epilog

1

Page 2: CNN RNN Autoencoder - UNISTisystems.unist.ac.kr/wp-content/uploads/sites/209/2017/... · 2018. 12. 28. · CNN RNN Autoencoder. Today 1. Machine*Learning 2. Deep*Learning –CNN –RNN

Machine  Learning

2

• Learns from  data  

Page 3: CNN RNN Autoencoder - UNISTisystems.unist.ac.kr/wp-content/uploads/sites/209/2017/... · 2018. 12. 28. · CNN RNN Autoencoder. Today 1. Machine*Learning 2. Deep*Learning –CNN –RNN

Machine  Learning

3

• Learns from  data• predicts on  data

Page 4: CNN RNN Autoencoder - UNISTisystems.unist.ac.kr/wp-content/uploads/sites/209/2017/... · 2018. 12. 28. · CNN RNN Autoencoder. Today 1. Machine*Learning 2. Deep*Learning –CNN –RNN

Framework  of  Machine  Learning

4

Sensor  Data

Data  Window

Features

Decision

Data  acquisition  and  pre-­‐processing

Windowing

Feature  extraction

Model  building  and  Classification  (Inference)

Classification  AlgorithmsSupport  Vector  Machine Logistic  Regression

K-­NN  Algorithm Artificial  Neural  Networks

Page 5: CNN RNN Autoencoder - UNISTisystems.unist.ac.kr/wp-content/uploads/sites/209/2017/... · 2018. 12. 28. · CNN RNN Autoencoder. Today 1. Machine*Learning 2. Deep*Learning –CNN –RNN

5

이사?

간다

온다

Page 6: CNN RNN Autoencoder - UNISTisystems.unist.ac.kr/wp-content/uploads/sites/209/2017/... · 2018. 12. 28. · CNN RNN Autoencoder. Today 1. Machine*Learning 2. Deep*Learning –CNN –RNN

6

이사?

간다

온다

“의외의 정보가 문제를 해결하는

좋은 Feature����������� ������������������  (특성인자)����������� ������������������  가 될 수 있다.”����������� ������������������  

Page 7: CNN RNN Autoencoder - UNISTisystems.unist.ac.kr/wp-content/uploads/sites/209/2017/... · 2018. 12. 28. · CNN RNN Autoencoder. Today 1. Machine*Learning 2. Deep*Learning –CNN –RNN

Weather  Station  

Temperature

Humidity

Brightness

Temperature

Humidity

Brightness

Page 8: CNN RNN Autoencoder - UNISTisystems.unist.ac.kr/wp-content/uploads/sites/209/2017/... · 2018. 12. 28. · CNN RNN Autoencoder. Today 1. Machine*Learning 2. Deep*Learning –CNN –RNN

(Unexpected)  Hidden  Information  

Brightness

Jul  31  05:27일출

Jul  30  23:26  학생퇴근

Jul  31  10:02학생출근

Temperature

Humidity

Brightness

Page 9: CNN RNN Autoencoder - UNISTisystems.unist.ac.kr/wp-content/uploads/sites/209/2017/... · 2018. 12. 28. · CNN RNN Autoencoder. Today 1. Machine*Learning 2. Deep*Learning –CNN –RNN

(Unexpected)  Hidden  Information  

Brightness

Jul  31  05:27일출

Jul  30  23:26  학생퇴근

Jul  31  10:02학생출근

Temperature

Humidity

Brightness

“학생들의 출석 (정보) 를 알기 위해서 조도 데이터

과연 생각해 낼 수 있었을까 ?”����������� ������������������  

Page 10: CNN RNN Autoencoder - UNISTisystems.unist.ac.kr/wp-content/uploads/sites/209/2017/... · 2018. 12. 28. · CNN RNN Autoencoder. Today 1. Machine*Learning 2. Deep*Learning –CNN –RNN

Today

1. Machine  Learning

2. Deep  Learning– CNN– RNN– Autoencoder

3. Deep  Learning  of  Things  (DoT)

4. Epilog

10

Page 11: CNN RNN Autoencoder - UNISTisystems.unist.ac.kr/wp-content/uploads/sites/209/2017/... · 2018. 12. 28. · CNN RNN Autoencoder. Today 1. Machine*Learning 2. Deep*Learning –CNN –RNN

Machine  Learning  and  Deep  Learning

11

Data  Acquisition Feature  Extraction Classification

-­ Time  domain-­ Frequency  domain

[  Machine  Learning  ]

대부분지도학습 현장전문가지식

Page 12: CNN RNN Autoencoder - UNISTisystems.unist.ac.kr/wp-content/uploads/sites/209/2017/... · 2018. 12. 28. · CNN RNN Autoencoder. Today 1. Machine*Learning 2. Deep*Learning –CNN –RNN

Machine  Learning  and  Deep  Learning

12

대부분지도학습 현장전문가지식

Data  Acquisition Feature  Extraction Classification

-­ Time  domain-­ Frequency  domain

[  Machine  Learning  ]

“����������� ������������������  딥러닝은기계학습보다는

Domain  Knowledge의존도가 낮다.����������� ������������������  ”����������� ������������������  

Page 13: CNN RNN Autoencoder - UNISTisystems.unist.ac.kr/wp-content/uploads/sites/209/2017/... · 2018. 12. 28. · CNN RNN Autoencoder. Today 1. Machine*Learning 2. Deep*Learning –CNN –RNN

Deep  Artificial  Neural  Networks  (심층인공신경망)• Complex/Nonlinear  function  approximator– Linearly  connected  networks  – Simple  nonlinear  neurons

• Hidden  layers– Autonomous  feature  learning

13

Classification

Class  2Class  1

Feature  learning

nonlinear

linear

Input

Page 14: CNN RNN Autoencoder - UNISTisystems.unist.ac.kr/wp-content/uploads/sites/209/2017/... · 2018. 12. 28. · CNN RNN Autoencoder. Today 1. Machine*Learning 2. Deep*Learning –CNN –RNN

Deep  Artificial  Neural  Networks  (심층인공신경망)• Complex/Nonlinear  function  approximator– Linearly  connected  networks  – Simple  nonlinear  neurons

• Hidden  layers– Autonomous  feature  learning

14

Class  2Class  1

nonlinear

linear

Feature  learningClassification

Page 15: CNN RNN Autoencoder - UNISTisystems.unist.ac.kr/wp-content/uploads/sites/209/2017/... · 2018. 12. 28. · CNN RNN Autoencoder. Today 1. Machine*Learning 2. Deep*Learning –CNN –RNN

Deep  Artificial  Neural  Networks  (심층인공신경망)• Complex/Nonlinear  function  approximator– Linearly  connected  networks  – Simple  nonlinear  neurons

• Hidden  layers– Autonomous  feature  learning

15

Class  2Class  1

nonlinear

linear

Feature  learningClassification

“����������� ������������������  은닉층의 개수만 늘어난 것이 아닌

독특한 구조의 딥러닝 모델 개발.����������� ������������������  ”����������� ������������������  

CNN RNN Autoencoder

Page 16: CNN RNN Autoencoder - UNISTisystems.unist.ac.kr/wp-content/uploads/sites/209/2017/... · 2018. 12. 28. · CNN RNN Autoencoder. Today 1. Machine*Learning 2. Deep*Learning –CNN –RNN

Today

1. Machine  Learning

2. Deep  Learning– CNN– RNN– Autoencoder

3. Deep  Learning  of  Things  (DoT)

4. Epilog

16

Page 17: CNN RNN Autoencoder - UNISTisystems.unist.ac.kr/wp-content/uploads/sites/209/2017/... · 2018. 12. 28. · CNN RNN Autoencoder. Today 1. Machine*Learning 2. Deep*Learning –CNN –RNN

Convolutional  Neural  Networks  (CNN)• Image  pattern  recognition  problems  (spatial)– Individual  cortical  neurons  respond  to  restricted  region  of  space– Perception  like  humans  – Convolutional Neural  Networks  (CNN)

17

1  Pixel  cannot  explainany  information

Small  area  can  explain  context  of  image

Image Kernel Output

Page 18: CNN RNN Autoencoder - UNISTisystems.unist.ac.kr/wp-content/uploads/sites/209/2017/... · 2018. 12. 28. · CNN RNN Autoencoder. Today 1. Machine*Learning 2. Deep*Learning –CNN –RNN

Convolutional  Neural  Networks  (CNN)• Image  pattern  recognition  problems  (spatial)– Individual  cortical  neurons  respond  to  restricted  region  of  space– Perception  like  humans  – Convolutional Neural  Networks  (CNN)

18

Image Kernel Output

Page 19: CNN RNN Autoencoder - UNISTisystems.unist.ac.kr/wp-content/uploads/sites/209/2017/... · 2018. 12. 28. · CNN RNN Autoencoder. Today 1. Machine*Learning 2. Deep*Learning –CNN –RNN

Convolutional  Neural  Networks  (CNN)• Image  pattern  recognition  problems  (spatial)– Individual  cortical  neurons  respond  to  restricted  region  of  space– Perception  like  humans  – Convolutional Neural  Networks  (CNN)

19

Image Kernel Output

1 1 1 0 0 0

1 1 1

é ùê úê ú- - -ê úë û

Page 20: CNN RNN Autoencoder - UNISTisystems.unist.ac.kr/wp-content/uploads/sites/209/2017/... · 2018. 12. 28. · CNN RNN Autoencoder. Today 1. Machine*Learning 2. Deep*Learning –CNN –RNN

Convolutional  Neural  Networks  (CNN)• Image  pattern  recognition  problems  (spatial)– Individual  cortical  neurons  respond  to  restricted  region  of  space– Perception  like  humans  – Convolutional Neural  Networks  (CNN)

• NN:  feature  extraction  and  transformation

20

Image

Convolution  and  pooling  layers

Convolution  and  nonlinearity Max  pooling

0

1

Fully  connected  layers Label

Convolutional  Neural  Networks

9

Feature  Extraction Classification

Page 21: CNN RNN Autoencoder - UNISTisystems.unist.ac.kr/wp-content/uploads/sites/209/2017/... · 2018. 12. 28. · CNN RNN Autoencoder. Today 1. Machine*Learning 2. Deep*Learning –CNN –RNN

Deep  Artificial  Neural  Networks  (심층인공신경망)

• Complex/Nonlinear  function  approximator– Linearly  connected  networks  – Simple  nonlinear  neurons

• Hidden  layers– Autonomous  feature  learning

21

Class  2Class  1

Page 22: CNN RNN Autoencoder - UNISTisystems.unist.ac.kr/wp-content/uploads/sites/209/2017/... · 2018. 12. 28. · CNN RNN Autoencoder. Today 1. Machine*Learning 2. Deep*Learning –CNN –RNN

Convolutional  Neural  Networks  (심층인공신경망)

• Structure– Weight  sharing– Local  connectivity

• Optimization– Smaller  searching  space

22

Class  2Class  1

Page 23: CNN RNN Autoencoder - UNISTisystems.unist.ac.kr/wp-content/uploads/sites/209/2017/... · 2018. 12. 28. · CNN RNN Autoencoder. Today 1. Machine*Learning 2. Deep*Learning –CNN –RNN

Today

1. Machine  Learning

2. Deep  Learning– CNN– RNN– Autoencoder

3. Deep  Learning  of  Things  (DoT)

4. Epilog

23

Page 24: CNN RNN Autoencoder - UNISTisystems.unist.ac.kr/wp-content/uploads/sites/209/2017/... · 2018. 12. 28. · CNN RNN Autoencoder. Today 1. Machine*Learning 2. Deep*Learning –CNN –RNN

Robocup 2011  Final:  Team  DARwIn -­‐ CIT  Brains

24

Page 25: CNN RNN Autoencoder - UNISTisystems.unist.ac.kr/wp-content/uploads/sites/209/2017/... · 2018. 12. 28. · CNN RNN Autoencoder. Today 1. Machine*Learning 2. Deep*Learning –CNN –RNN

25

Page 26: CNN RNN Autoencoder - UNISTisystems.unist.ac.kr/wp-content/uploads/sites/209/2017/... · 2018. 12. 28. · CNN RNN Autoencoder. Today 1. Machine*Learning 2. Deep*Learning –CNN –RNN

Recurrent  NN  (RNN)

• Hidden  state  extraction  and  transformation  

26

Yn-­‐1 Yn Yn+1

On+1 Classification

Page 27: CNN RNN Autoencoder - UNISTisystems.unist.ac.kr/wp-content/uploads/sites/209/2017/... · 2018. 12. 28. · CNN RNN Autoencoder. Today 1. Machine*Learning 2. Deep*Learning –CNN –RNN

Recurrent  NN  (RNN)

• Hidden  state  extraction  and  transformation  

27

Yn-­‐1 Yn Yn+1

Xn+1XnXn-­1

On+1

Learned  latent  state

Classification  based  on  states

U U U

Page 28: CNN RNN Autoencoder - UNISTisystems.unist.ac.kr/wp-content/uploads/sites/209/2017/... · 2018. 12. 28. · CNN RNN Autoencoder. Today 1. Machine*Learning 2. Deep*Learning –CNN –RNN

Recurrent  NN  (RNN)

• Hidden  state  extraction  and  transformation  • Good  for  sequential  data  (dynamic  behavior)

28

Yn-­‐1 Yn Yn+1

Xn+1XnXn-­1

On+1

… Learned  latent  state  and  its  dynamics

Classification  based  on  states

W

U

W W

U U

Page 29: CNN RNN Autoencoder - UNISTisystems.unist.ac.kr/wp-content/uploads/sites/209/2017/... · 2018. 12. 28. · CNN RNN Autoencoder. Today 1. Machine*Learning 2. Deep*Learning –CNN –RNN

Time  Series  Data  and RNN

29

Page 30: CNN RNN Autoencoder - UNISTisystems.unist.ac.kr/wp-content/uploads/sites/209/2017/... · 2018. 12. 28. · CNN RNN Autoencoder. Today 1. Machine*Learning 2. Deep*Learning –CNN –RNN

Today

1. Machine  Learning

2. Deep  Learning– CNN– RNN– Autoencoder

3. Deep  Learning  of  Things  (DoT)

4. Epilog

30

Page 31: CNN RNN Autoencoder - UNISTisystems.unist.ac.kr/wp-content/uploads/sites/209/2017/... · 2018. 12. 28. · CNN RNN Autoencoder. Today 1. Machine*Learning 2. Deep*Learning –CNN –RNN

Dimension  Reduction

Principal  Component  Analysis  (PCA)  in  time  signals– not  easily  seen

31

Page 32: CNN RNN Autoencoder - UNISTisystems.unist.ac.kr/wp-content/uploads/sites/209/2017/... · 2018. 12. 28. · CNN RNN Autoencoder. Today 1. Machine*Learning 2. Deep*Learning –CNN –RNN

Autoencoder

• Recover  the  input  data

32

Page 33: CNN RNN Autoencoder - UNISTisystems.unist.ac.kr/wp-content/uploads/sites/209/2017/... · 2018. 12. 28. · CNN RNN Autoencoder. Today 1. Machine*Learning 2. Deep*Learning –CNN –RNN

Autoencoder

• Recover  the  input  data  • Data  compression  to  lower  dimension  → Latent  variable• Latent  variables  ≈ features• Realistic  ← unsupervised  learning• Nonlinear

33

Page 34: CNN RNN Autoencoder - UNISTisystems.unist.ac.kr/wp-content/uploads/sites/209/2017/... · 2018. 12. 28. · CNN RNN Autoencoder. Today 1. Machine*Learning 2. Deep*Learning –CNN –RNN

Artistic  Style  Transfer

34

Page 35: CNN RNN Autoencoder - UNISTisystems.unist.ac.kr/wp-content/uploads/sites/209/2017/... · 2018. 12. 28. · CNN RNN Autoencoder. Today 1. Machine*Learning 2. Deep*Learning –CNN –RNN

Imbalanced  Data

• Not  enough  data  from  faulty  status

• Data  Imbalance  – Under  sampling– Over  sampling– Re-­‐weighting– (Ada)Boosting

• Crazy  idea:  – Can  we  generate  phantom  (fake)  data?– Then  use  them  for  further  data  analysis  (ML  or  DL)  

35

1( ) ˆ( , , ) ( , )

N

i ii

iL x y l y yywq=

= ×åOK

NG

Labe

led  data

Page 36: CNN RNN Autoencoder - UNISTisystems.unist.ac.kr/wp-content/uploads/sites/209/2017/... · 2018. 12. 28. · CNN RNN Autoencoder. Today 1. Machine*Learning 2. Deep*Learning –CNN –RNN

Data  Generation

36

Page 37: CNN RNN Autoencoder - UNISTisystems.unist.ac.kr/wp-content/uploads/sites/209/2017/... · 2018. 12. 28. · CNN RNN Autoencoder. Today 1. Machine*Learning 2. Deep*Learning –CNN –RNN

Latent  Space

37

Page 38: CNN RNN Autoencoder - UNISTisystems.unist.ac.kr/wp-content/uploads/sites/209/2017/... · 2018. 12. 28. · CNN RNN Autoencoder. Today 1. Machine*Learning 2. Deep*Learning –CNN –RNN

Generative  Adversarial  Networks  (GAN)Analogous  to  Turing  Test

38

Page 39: CNN RNN Autoencoder - UNISTisystems.unist.ac.kr/wp-content/uploads/sites/209/2017/... · 2018. 12. 28. · CNN RNN Autoencoder. Today 1. Machine*Learning 2. Deep*Learning –CNN –RNN

Generative  Adversarial  Networks  (GAN)Analogous  to  Turing  Test

39

Generated

Real

RealFake

Generator Discriminator

Page 40: CNN RNN Autoencoder - UNISTisystems.unist.ac.kr/wp-content/uploads/sites/209/2017/... · 2018. 12. 28. · CNN RNN Autoencoder. Today 1. Machine*Learning 2. Deep*Learning –CNN –RNN

Today

1. Machine  Learning

2. Deep  Learning– CNN– RNN– Autoencoder

3. Deep  Learning  of  Things  (DoT)

4. Epilog

40

Page 41: CNN RNN Autoencoder - UNISTisystems.unist.ac.kr/wp-content/uploads/sites/209/2017/... · 2018. 12. 28. · CNN RNN Autoencoder. Today 1. Machine*Learning 2. Deep*Learning –CNN –RNN

Computation  Environment  for  Model  Learning

• Development  environment (open  source)– Ubuntu  14.04– Python3– TensorFlow

• Machine  (약 1,500만원)– GPU:  GeForce  GTX  TITAN  X  (PASCAL)– CPU:  Intel  i7-­‐5930k  6  Core  3.5GHz  processor

• Parallel  computing– Multi  GPUs

41

Page 42: CNN RNN Autoencoder - UNISTisystems.unist.ac.kr/wp-content/uploads/sites/209/2017/... · 2018. 12. 28. · CNN RNN Autoencoder. Today 1. Machine*Learning 2. Deep*Learning –CNN –RNN

Implementation  of  Deep  Learning  Model

42

Server

Model  Training  at  Server

학습

Module Internet  of  Things

Embedded  Systems  or  Internet  of  Things

Load  Model

실행

Save  Model

Trained  Model

w학습된모델

학습과실행은다르다- 학습은비싸고오래걸릴수있지만- 실행은대부분싸고빠르다

Page 43: CNN RNN Autoencoder - UNISTisystems.unist.ac.kr/wp-content/uploads/sites/209/2017/... · 2018. 12. 28. · CNN RNN Autoencoder. Today 1. Machine*Learning 2. Deep*Learning –CNN –RNN

Deep  Learning  of  Things  (DoT)

43

Handwritten  Digits  Recognition

Page 44: CNN RNN Autoencoder - UNISTisystems.unist.ac.kr/wp-content/uploads/sites/209/2017/... · 2018. 12. 28. · CNN RNN Autoencoder. Today 1. Machine*Learning 2. Deep*Learning –CNN –RNN

Today

1. Machine  Learning

2. Deep  Learning– CNN– RNN– Autoencoder

3. Deep  Learning  of  Things  (DoT)

4. Epilog

44

Page 45: CNN RNN Autoencoder - UNISTisystems.unist.ac.kr/wp-content/uploads/sites/209/2017/... · 2018. 12. 28. · CNN RNN Autoencoder. Today 1. Machine*Learning 2. Deep*Learning –CNN –RNN

인공지능으로이런문제도해결할수있나요?

(최소한)인간이구별할수있는문제면딥러닝으로도해결할수있다.

(이론적으로)인간이구별할수없는문제도딥러닝으로해결할수있다.– 커제:알파고 2.0 “바둑의신에가까워지고있다.– 알파고의수를인간이배우려고하고있다.

45

Page 46: CNN RNN Autoencoder - UNISTisystems.unist.ac.kr/wp-content/uploads/sites/209/2017/... · 2018. 12. 28. · CNN RNN Autoencoder. Today 1. Machine*Learning 2. Deep*Learning –CNN –RNN

인공지능을성공적으로적용하기위한필요조건?

• 기본적으로데이터가많아야한다.– 특히불량또는비정상데이터 (현실적으로어렵다)– Data-­‐driven  방식에대한단점이해필요

• 필요한데이터를가지고올수있는자동화팀역량필요– 하드웨어프로그래밍

• Data  Analytics  역량필요– 소프트웨어프로그래밍– 컴퓨터공학,산업공학,통계– 제조분야에서해당인력을구하기가쉽지않다 (인공지능인재영입전쟁)

• 시작은조각모음방식 (작은성공사례부터만들자)

46

Page 47: CNN RNN Autoencoder - UNISTisystems.unist.ac.kr/wp-content/uploads/sites/209/2017/... · 2018. 12. 28. · CNN RNN Autoencoder. Today 1. Machine*Learning 2. Deep*Learning –CNN –RNN

딥러닝장·∙단점

• 기존의모든 function  approximator  를대체하는분위기

• 기계학습보다는 domain  knowledge  에대한의존도가낮다→범용성

• 개발속도가빨라진다.→ Fast  deploy

• Lack  of  interpretability and explainability– Still  acting  as  a  black  box

47

Page 48: CNN RNN Autoencoder - UNISTisystems.unist.ac.kr/wp-content/uploads/sites/209/2017/... · 2018. 12. 28. · CNN RNN Autoencoder. Today 1. Machine*Learning 2. Deep*Learning –CNN –RNN

http://isystems.unist.ac.kr/

All  materials  (codes  +  hardware  design)are  available

48