Deep Anomaly Detection - AiFrenzAI Friends]Deep Anomaly... · Deep Anomaly Detection Kang, Min-Guk...
Transcript of Deep Anomaly Detection - AiFrenzAI Friends]Deep Anomaly... · Deep Anomaly Detection Kang, Min-Guk...
Contents
1. Introduction
2. Conventional Anomaly Detection
3. Deep Anomaly Detection
4. What is important?
2/47
3/47
Introduction of Anomaly Detection(Vision data only)
1. Introduction
Anomaly Detection is the process of identifying the new or unexplained set of data to determine
if they are within the norm or outside of it.
4/47
1. Introduction
OODD: Out Of Distribution Detection!
5/47
Patient’s Heart rate data
1. Introduction(Time Series Data)
QTUM Coin
6/47
Assembly Inspection
1. Introduction(Vision Data)
Welding Defect
7/47
1. Introduction(Vision Data)
8/47
Welding Defect
1. Introduction
9/47
Wasserstein GAN
VAE
Conventional Anomaly Detection(Vision data only)
10/47
Before dealing with Anomaly Detection, It is essential to identify the definition of the problem.
→ Domain Knowledge
2. Conventional Anomaly Detection
11/47
PCA
2. Conventional Anomaly Detection
https://www.tableau.com/resource/top-10-big-data-trends-2017
outlier
12/47
KDE
2. Conventional Anomaly Detection
https://www.tableau.com/resource/top-10-big-data-trends-2017
Normal
outlier
𝒇𝒉 𝒙 =𝟏
𝒏
𝒊=𝟏
𝒏
𝑲𝒉(𝑿 − 𝑿𝒊) =𝟏
𝒏𝒉
𝒊=𝟏
𝒏
𝑲(𝑿 − 𝑿𝒊
𝒉)
13/47
2. Conventional Anomaly Detection
Support Vector Data Description(SVDD)
https://link.springer.com/article/10.1023/B:MACH.0000008084.60811.49
Isolation forests(IF)
https://cs.nju.edu.cn/zhouzh/zhouzh.files/publication/icdm08b.pdf
14/47
https://en.wikipedia.org/wiki/Anomaly_detection#cite_note-12
15/47
Deep Anomaly Detection(Vision data only)
NeuralNetworks
𝑍1
𝑍2
Scatter
3. Deep Anomaly Detection(Representation Learning)
16/47
NeuralNetworks
𝑍1
𝑍2
Scatter
17/47
3. Deep Anomaly Detection(Representation Learning)
NeuralNetworks
𝑍1
𝑍2
NeuralNetworks
𝑍1
𝑍2
18/47
3. Deep Anomaly Detection(Representation Learning)
normal
abnormal
① Non linear Classifier
Deep One-Class Classification
http://proceedings.mlr.press/v80/ruff18a.html
DCAE(Deep Convolutional Autoencoder)
Deep Anomaly Detection Using Geometric Transformation
https://arxiv.org/pdf/1805.10917.pdf
3. Deep Anomaly Detection(Representation Learning)
19/47
NeuralNetworks
𝑍1
𝑍2
NeuralNetworks
L1, L2 distance
3. Deep Anomaly Detection(Reconstruction Error)
20/47
Unsupervsed Anomaly Detection with Generative Adversarial Networks to Guide marker discovery(IPML, 2017)
Adversarially Learned One-Class Classifier forNovelty Detection(CVPR, 2018)
NeuralNetworks
𝑍1
𝑍2
NeuralNetworks
High L1, L2 distance
3. Deep Anomaly Detection(Reconstruction Error)
21/47
Generator Discriminator 0(fake) or 1(real)
𝑃𝑑𝑎𝑡𝑎(𝑥)
𝐺(𝑧)
3. Deep Anomaly Detection(GANs)
22/47
3. Deep Anomaly Detection(GANs)
23/47
Generator
Generator𝜃
Discriminator∅ 1
𝐺(𝑧)
𝐺𝑙𝑜𝑠𝑠 = 𝑎𝑟𝑔𝑚𝑖𝑛𝜃𝔼𝑧 ~ 𝑝(𝑧)[log(1 − 𝐷(𝐺(𝑧)))]
3. Deep Anomaly Detection(GANs)
24/47
Generator𝜃
Discriminator∅ 1
𝐺(𝑧)
𝐺𝑙𝑜𝑠𝑠 = 𝑎𝑟𝑔𝑚𝑖𝑛𝜃𝔼𝑧 ~ 𝑝(𝑧)[log(1 − 𝐷(𝐺(𝑧)))]
3. Deep Anomaly Detection(GANs)
25/47
3. Deep Anomaly Detection(GANs)
26/47
Discriminator
D(G(z)) = 0
𝑃𝑑𝑎𝑡𝑎(𝑥)
𝐺(𝑧)
𝐷𝑙𝑜𝑠𝑠 = 𝑎𝑟𝑔𝑚𝑎𝑥∅𝔼𝑥 ~ 𝑝𝑑𝑎𝑡𝑎 𝑙𝑜𝑔𝐷 𝑥 + 𝔼𝑧 ~ 𝑝(𝑧)[log(1 − 𝐷(𝐺(𝑥)))]
D(X) = 1
Generator𝜃
Discriminator∅
3. Deep Anomaly Detection(GANs)
27/47
3. Deep Anomaly Detection(GANs)
28/47
Repeat Over and Over
D(G(z)) = 0.5
𝑃𝑑𝑎𝑡𝑎(𝑥)
𝐺(𝑧)
D(X) = 0.5
Generator𝜃
Discriminator∅
3. Deep Anomaly Detection(GANs)
29/47
DCGenerator
DCDiscriminator 0(fake) or 1(real)
𝑃𝑑𝑎𝑡𝑎(𝑥)
𝐺(𝑧)
3. Deep Anomaly Detection(AnoGAN)Unsupervsed Anomaly Detection with Generative Adversarial Networks to Guide marker discovery (IPML, 2017)
30/47
Sampler(Normal instances)
3. Deep Anomaly Detection(AnoGAN)
Generator
Unsupervsed Anomaly Detection with Generative Adversarial Networks to Guide marker discovery (IPML, 2017)
31/47
Generator
𝑮 𝒁𝟏
Sampler(Normal instances)
3. Deep Anomaly Detection(AnoGAN)Unsupervsed Anomaly Detection with Generative Adversarial Networks to Guide marker discovery (IPML, 2017)
32/47
normal Instance
𝑮 𝒁𝟏
𝑿𝒕𝒂𝒓𝒈𝒆𝒕
3. Deep Anomaly Detection(AnoGAN)Unsupervsed Anomaly Detection with Generative Adversarial Networks to Guide marker discovery (IPML, 2017)
Generator
33/47
𝑳𝒂𝒏𝒐_𝟏 = | 𝑿𝒕𝒂𝒓𝒈𝒆𝒕 − 𝑮 𝒁𝟏 |
normal Instance
𝑮 𝒁𝟏
𝑿𝒕𝒂𝒓𝒈𝒆𝒕
Generator
3. Deep Anomaly Detection(AnoGAN)Unsupervsed Anomaly Detection with Generative Adversarial Networks to Guide marker discovery (IPML, 2017)
34/47
3. Deep Anomaly Detection(AnoGAN)Unsupervsed Anomaly Detection with Generative Adversarial Networks to Guide marker discovery (IPML, 2017)
Discriminator 𝑳𝒂𝒏𝒐_𝟐 = |𝑫( 𝑿𝒕𝒂𝒓𝒈𝒆𝒕) − 𝑫(𝑮 𝒁𝟏 )|
𝐿𝑎𝑛𝑜𝑚𝑎𝑙𝑦 = (1 − 𝜆) × 𝐿𝑎𝑛𝑜_1 + 𝜆 × 𝐿𝑎𝑛𝑜_2
𝑳𝒂𝒏𝒐_𝟏 = | 𝑿𝒕𝒂𝒓𝒈𝒆𝒕 − 𝑮 𝒁𝟏 |
normal Instance
𝑮 𝒁𝟏
𝑿𝒕𝒂𝒓𝒈𝒆𝒕
Generator
35/47
Fix
Fix
Update latent vector to reduce 𝑳𝒂𝒏𝒐𝒎𝒂𝒍𝒚!
Discriminator 𝑳𝒂𝒏𝒐_𝟐 = |𝑫( 𝑿𝒕𝒂𝒓𝒈𝒆𝒕) − 𝑫(𝑮 𝒁𝟏 )|
𝐿𝑎𝑛𝑜𝑚𝑎𝑙𝑦 = (1 − 𝜆) × 𝐿𝑎𝑛𝑜_1 + 𝜆 × 𝐿𝑎𝑛𝑜_2
𝑳𝒂𝒏𝒐_𝟏 = | 𝑿𝒕𝒂𝒓𝒈𝒆𝒕 − 𝑮 𝒁𝟏 |
normal Instance
𝑮 𝒁𝟏
𝑿𝒕𝒂𝒓𝒈𝒆𝒕
Generator
3. Deep Anomaly Detection(AnoGAN)Unsupervsed Anomaly Detection with Generative Adversarial Networks to Guide marker discovery (IPML, 2017)
36/47
Discriminator 𝑳𝒂𝒏𝒐_𝟐 = |𝑫( 𝑿𝒕𝒂𝒓𝒈𝒆𝒕) − 𝑫(𝑮 𝒁𝟏 )|
𝐿𝑎𝑛𝑜𝑚𝑎𝑙𝑦 = (1 − 𝜆) × 𝐿𝑎𝑛𝑜_1 + 𝜆 × 𝐿𝑎𝑛𝑜_2 ↓↓
𝑳𝒂𝒏𝒐_𝟏 = | 𝑿𝒕𝒂𝒓𝒈𝒆𝒕 − 𝑮 𝒁𝟏 |
normal Instance
𝑮 𝒁𝟏
𝑿𝒕𝒂𝒓𝒈𝒆𝒕
Generator
3. Deep Anomaly Detection(AnoGAN)Unsupervsed Anomaly Detection with Generative Adversarial Networks to Guide marker discovery (IPML, 2017)
37/47
Discriminator 𝑳𝒂𝒏𝒐_𝟐 = |𝑫( 𝑿𝒕𝒂𝒓𝒈𝒆𝒕) − 𝑫(𝑮 𝒁𝟏 )|
𝐿𝑎𝑛𝑜𝑚𝑎𝑙𝑦 = (1 − 𝜆) × 𝐿𝑎𝑛𝑜_1 + 𝜆 × 𝐿𝑎𝑛𝑜_2 ↑↑
𝑳𝒂𝒏𝒐_𝟏 = | 𝑿𝒕𝒂𝒓𝒈𝒆𝒕 − 𝑮 𝒁𝟏 |
Abnormal Instance
𝑮 𝒁
𝑿𝒕𝒂𝒓𝒈𝒆𝒕
Generator
3. Deep Anomaly Detection(AnoGAN)Unsupervsed Anomaly Detection with Generative Adversarial Networks to Guide marker discovery (IPML, 2017)
38/47
Generated Images are similar
to the normal images.
3. Deep Anomaly Detection(AnoGAN)Unsupervsed Anomaly Detection with Generative Adversarial Networks to Guide marker discovery (IPML, 2017)
39/47
Anomaly Detection Feature Extraction How do we extract representation of data well?
3. Deep Anomaly Detection(DSVDD)Deep One-Class Classification(ICML, 2018)
40/47
Anomaly Detection Feature Extraction How do we extract representation of data well?
3. Deep Anomaly Detection(DSVDD)Deep One-Class Classification(ICML, 2018)
𝑆𝑜𝑓𝑡 𝑆𝑉𝐷𝐷 = 𝑚𝑖𝑛𝑅,𝑊 𝑅2 +1
𝑣𝑛σ𝑖=1𝑛 max{0, | 𝜙 𝑥𝑖;𝑊 − 𝑐||2 − 𝑅2 +
𝜆
2σ𝑙=1𝐿 ||𝑊𝑙||𝐹
2
𝑆𝑉𝐷𝐷 = 𝑚𝑖𝑛𝑊
𝑖=1
𝑛
| 𝜙 𝑥𝑖;𝑊 − 𝑐2 | +𝜆
2
𝑙=1
𝐿
||𝑊𝑙||𝐹2
41/47
Anomaly Detection Feature Extraction How do we extract representation of data well?
3. Deep Anomaly Detection(DSVDD)Deep One-Class Classification(ICML, 2018)
𝑆𝑜𝑓𝑡 𝑆𝑉𝐷𝐷 = 𝑚𝑖𝑛𝑅,𝑊 𝑅2 +1
𝑣𝑛σ𝑖=1𝑛 max{0, | 𝜙 𝑥𝑖;𝑊 − 𝑐||2 − 𝑅2 +
𝜆
2σ𝑙=1𝐿 ||𝑊𝑙||𝐹
2
𝑆𝑉𝐷𝐷 = 𝑚𝑖𝑛𝑊
𝑖=1
𝑛
| 𝜙 𝑥𝑖;𝑊 − 𝑐2 | +𝜆
2
𝑙=1
𝐿
||𝑊𝑙||𝐹2
42/47
3. Deep Anomaly Detection(DSVDD)Deep One-Class Classification(ICML, 2018)
43/47
What is important?
44/47
4. What is important?
1. Domain Knowledge
2. GAN을 이용한 Anomaly Detection은 아직은 별루다.
But Mode Problem을 잘 이용하면 재미있는 것을 할 수 있을지도…
3. Anomaly Detection은 Representation을 잘 추출하는게 관건이다.
(Reconstruction, Classification)
4. 추출한 잠재변수를 잘 사용해서 비정상 점수를 잘 뽑아내야한다.
45/47
We are Hiring
• Selected as Deep Learning Best Practice at NVIDIA AI Conference 2018 Keynote.
• Use Cases for “Accelerate AI with Synthetic data using generative adversarial networks” at Strata Data Conference 2018 NY.
• ICPR 2018 Contests on Object Detection in Aerial Images 2위, 국방부 주관 제 20차 M&S 발전 세미나 우수 논문상 수상
• CVPR 2017 NTIRE Challenge Task 2,3,4,5 위 수상, Nips 2017 DeeoArt.io Poster Contest 1등상 수상
• Kaggle “DSTL Satellite Imagery Feature Detection” Silver medal 수상
• 이외에도 SIA의 연구 내용은 CVPR, ICPR, ACML, ICML, ACM SIGSPATIAL 등 학회에서 발표되었습니다.
46/47
https://github.com/MINGUKKANG/SIA_
Thank you!
47/47