소개• Since  2013  July:  UNIST

• 2010,  Ph.D.  from  the  University  of  Michigan,  Ann  Arbor– S.  M.  Wu  Manufacturing  Research  Center– The  Center  of  Intelligent  Maintenance  Systems  (IMS)

• 2007,  M.S.  from  the  University  of  Michigan,  Ann  Arbor

• 2005,  B.S.  of  Electrical  Engineering  from  Seoul  National  University

• 2001,  B.S.  of  Mechanical  Engineering  from  Seoul  National  University

1

iSystems  Design  Lab• immune engineering  for  self-­‐sustainable  system  and  maintenance-­‐free  machine  design• informatics for  visualization  and  machine  health  monitoring• internet  of  things  for  smart  factories

2http://isystems.unist.ac.kr/

이상진단 플랫폼

3

Internet  of  Things  &  PHM

4

IoT  Sensor• System

– Wi-­‐fi Micro-­‐controller– IMU  Accelerometer– Lithium-­‐ion  battery

• Training  Data  Acquisition– Rotor  Testbed

5

Image Spec

ParticlePhoton

Broadcom  BCM43362  Wi-­‐Fi  chipSTM32F205  120Mhz  ARM  Cortex  M3

1MB  flash,  128KB  RAMhttps://store.particle.io/

IMU  Sensor

3  acceleration  channels16-­‐bit  data  output1 kHz  Sample  Rate

https://www.sparkfun.com

Rotor  Testbed

RPM 1500

Fault Mode Normal Unbalance Misalignment

Sensor  Position Bearing  Housing

Sensor X  axis accelerometer

Sample  Rate 1  kHz

*  Wi-­fi Communication  Maximum  Speed  :  11  MBit/s

IoT  Sensor  with  Machine  Learning• Algorithm  Embedded  (C++)– Feature  Extraction  Function– Trained  classification  model

• Data  processing  process

6

-­ Data  Acquisition

-­ FFT

-­ RBF  Kernel

-­ Logistic  Regression

0 100 200 300 400 500 600 700 800 900 1000-0.3

-0.2

-0.1

0

0.1

0.2

0.3

0.4

Data Number

Ampl

itude

Time Signal

10 20 30 40 50 60 70 80 90 1000

0.1

0.2

0.3

0.4

0.5

0.6

0.7FFT

Frequency

Amplitude

Machinery • Feature  Vector§ 1X  Amplitude§ 2X  Amplitude

• Probability  of  Machine  State

Web-­‐based  Dashboard• Web-­‐based  service– Cloud  Server– Various  devices  can  access

7

Machine  status  in  Feature  Space

Probability  of  ClassificationFull  Spectrum

Dashboard  on  Mobile  Devices

8

PHM  with  IoT  and  Cloud  Platform• Prognostic  Health  Management  (PHM)– Short-­‐term  Analysis

• IoT  Sensors• Local• 현재설비상태분석• 고장상태분류

– Long-­‐term  Analysis• Cloud  Computing• Integral• 누적된정보활용을통한트랜드분석• 시계열분석및인과관계분석

• Monitoring  Systems– Data  Visualization

• Intuitive  Information• Interactive  Information

– Web-­‐based  Service

9

IoT  Sensor

Machinery

Machine  Learning-­‐ Classification-­‐ Pattern  Recognition

State  Estimation

Cloud  Platform Machine  Learning-­‐ Time  Series  Analysis-­‐ Probabilistic  Graph  Model

Data  Visualization-­‐ Web  Service-­‐ Interactive

SensorsFeature3

Short-­‐term  Analysis

Long-­‐term  Analysis Monitoring

PHM

Diagnostics

Prognostics

Data  Flow

Estimation

Maintenance

Deep  Learning  &  PHM

10

• Prognostics  and  Health  Management  (PHM)  approach• Prevent/Predict  system  failures

Monitoring  Systems

11

Time  Domain Frequency  Domain Time-­‐frequency Orbit  Analysis

• Input  :  8×1  Vector• 350  orbit  images  are  used  for  validation• Total  misclassification  for  the  given  test  set  is  overall  6.0%

Gaussian  Discriminant  Analysis  (GDA)

12

True  Shape

Classified

C E H 8 T

C 69 5 0 0 0

E 1 64 0 0 0

H 0 1 68 12 0

8 0 0 2 58 0

T 0 0 0 0 70

Full  SpectrumOrbit  Decomposition(Radius  &  Phase)

z a bj= +( ), ( )z zreal imag

Feature  Extraction  (Real  and  imaginary  value,  8-­‐dim)

GDAClassification  in  8-­‐dim  

Artificial  Neural  Network

13

• Input  :  8×1  Vector• 350  orbit  images  are  used  for  validation• Total  misclassification  for  the  given  test  set  is  overall 8.5%

True  Shape

Classified

C E H 8 T

C 68 0 0 0 0

E 0 69 6 0 0

H 0 0 54 8 0

8 2 1 10 62 3

T 0 0 0 0 67

Structure

• 1  Input  Layer§ 8 neurons

• 1  Hidden  Layer§ 100  neurons

• 1  Output  Layer§ 5  neurons

Orbit  Analysis  for  Rotating  Machinery• Visualize  shaft  movement– Vibration  information

• Integrated  analysis  possible

14

Orbit

Axes  Signal2  Sensors

Normal Unbalance Misalignment Rubbing

Proposed  Idea

Orbit  Shape

Machine  Learning

Fault  Detection

15

-­ Signal  to  image   -­ Image  pattern  recognition-­ Deep  learning(Google,  Facebook,  …)

-­ Use  known-­fault  modes

Deep  Learning• Automatic  discovery  of  the  representation  for  classification• Abstraction  from  combination  of  non-­‐linear  method• Image  pattern  recognition  problems  – hand-­‐written  digit  recognition  and  face  recognition– Convolutional  Neural  Networks  (CNN)

• Hierarchical  structure

16

RGB  =  [  0  0  0  ]1  Pixel  cannot  explainany  information

Small  area  can  explain  context  of  image

Structure  of  Convolutional  Neural  Networks Key  idea  of  Convolutional  Neural  Networks

Convolutional  Neural  Networks  (CNN)• 5  Output  neurons– Max  pooling– Value  of  neuron  means  the  degree  of  activation• Probability  of  classification

– Each  neuron  represent  each  class• 1  0  0  0  0 =  Class  1• 0.3  0.9  0  0  0 =  Class  2

17

0.10.10.90.20.1

é ùê úê úê úê úê úê úë û

Convolution  layer  and  Subsampling  layer

• Autonomous  orbit  image  pattern  recognition• Training  and  classification  process

Deep  Learning  on  Orbit  Images

18

Training  Data Pre-­‐processing

Pre-­‐processing

Deep  Learning  Structure

Input  Data ClassificationStructure

Training  Process

Classification  Process

2

1

min

ˆ ( )z

T T

z b

z b-

F -

= F F F

2

1

min

ˆ ( )z

T T

z b

z b-

F -

= F F F

wTrained

Kalman  Filter  &  PHM(모델 기반 진단)

19

Model-­‐based  Diagnostics• Possible  only  for  simple  systems– Analytical– Computer  simulation– But,  expensive

20

Model-­‐based  Diagnostics• Assumptions  – system  can  be  approximated  as  state  space  representation

• If  system  dynamics are  changed (due  to  fault)

• Real-­‐time  diagnostics  via  estimating  matrix  A– From  xn and  yn– From  A

21

1n n n

n n n

x Axy Cx

wu

+ = +

= +

: state: observation (measurement): system matrix: measurement matrix: system noise: obervation noise

n

n

n

xyACwu

1n n n

n n n

x Axy Cx

wu

+ = += +

1n n n

n n n

Ax xy Cx

wu

+ +=

¢=+

Rotating  Machinery: Misalignment• Vibration  measurement• Induce  misalignment  during  operation

• Continuous  • No  training  step  required  

22

Data-­driven  ML  classification  (SVM)

Kalman  Filter  Estimation  error  

Data  Visualization

23

Data  Visualization

• 데이터분석결과를쉽게이해할수있도록시각적으로

표현하고전달하는과정

• 빅데이터à기계가추론à정보

24

시각화 (visualization)

사람이추론 (reasoning)

도서관 데이터 시각화• 유니스트도서관도서데이터– 106,331권의책을 4단계로분류

• Tree  시각화방법사용– 구조가아래로길게나열되어한눈에보이지않음

25

도서관 데이터 시각화

26

도서관 데이터 시각화

27

대전 타슈 자전거대여 시스템 시각화

28

29

한국 수출입 시각화

시각화 적용 사례 – PCA  (주성분분석)

31

1 ( )x temperature

2(

)xvibration

1x

2x

1l

2l

PCA-­ Eigenvalue

PCA-­ Eigenvector

1u2u

1x

2x

2c

1c3c

4c

-­ Correlated-­ Linear  decomposition

• Linear  data  dimension  reduction

고유치와 고유벡터

• Eigenvalues– 새로운축이원래데이터의정보를얼마나많이가지고있는지에대한지표

• Eigenvectors– 계수 c를 통해새로운축에대한각특성인자들의중요도를유추가능

32

[ ] [ ] 1 31 2 1 2

2 4

ˆ ˆ ˆ ˆc c

u u x xc cé ù

= ê úë û

2l1l

2x

1x

2u

1u

PC 1 PC 2

주성분 분석 결과

• 기기진단모니터링에사용되는 PCA의 한계점– Eigenvectors와 Eigenvalues 값에 대한 분석이부족• 선정된축과원래데이터축사이의관계성파악에중요지표

– PCA에 대한 배경지식이없다면이해가어려움

33

PCA  시각화

34

2l1l

1x

2x

1u

2u

link

신호 기반과 시각화의 연계• 신호기반진단모듈– 학습된추론엔진을통해기기고장진단

• 현장근무자의의사결정– 데이터시각화기법을통한직관적인판단

• 진단결과조합을통해정밀한진단가능

35

Data Acquisition Feature Selection

-0.02 0 0.02-0.04

-0.03

-0.02

-0.01

0

0.01

0.02

0.03

0.04Mode1

X axis

Y ax

is

-100 0 1000

5

10

15

20

25

30

35

40FFT of Mode1

Frequency

Ampl

itude

-0.02 0 0.02

-0.04

-0.03

-0.02

-0.01

0

0.01

0.02

0.03

0.04

Mode2

X axisY

axis

-100 0 1000

5

10

15

20

25

30

35

40FFT of Mode2

Frequency

Ampl

itude

-0.01 0 0.01

-0.03

-0.02

-0.01

0

0.01

0.02

0.03

Mode3

X axis

Y ax

is

-100 0 1000

5

10

15

20

25

30

35

40FFT of Mode3

Frequency

Ampl

itude

-0.02 0 0.02-0.05

0

0.05Mode4

X axis

Y ax

is

-100 0 1000

5

10

15

20

25

30

35

40FFT of Mode4

Frequency

Ampl

itude

시각화

최종 기기 진단

현장 근무자 결정

신호 기반

시각화 기반

warning

warning

warning

신호 기반 진단

1: 11:

1: 1

(Z | ) ( | )( | Z )(Z | )

k k k kk k

k k

P X P X ZP XP Z

-

-

=