Anomaly Detection and Predictive Maintenance · PDF fileWrite to PMML files Time Series...
Transcript of Anomaly Detection and Predictive Maintenance · PDF fileWrite to PMML files Time Series...
![Page 1: Anomaly Detection and Predictive Maintenance · PDF fileWrite to PMML files Time Series Training: Learn what is ⍈normal](https://reader031.fdocuments.in/reader031/viewer/2022030414/5a9f9d327f8b9a71178cf5b3/html5/thumbnails/1.jpg)
Copyright © 20145KNIME.com AG
Anomaly Detection and Predictive Maintenance
Rosaria Silipo [email protected]
Iris Adae [email protected]
Christian Dietz [email protected]
Phil Winters [email protected]
KNIME.com AG, Zurich, Switzerland
www.knime.com
@KNIME
![Page 2: Anomaly Detection and Predictive Maintenance · PDF fileWrite to PMML files Time Series Training: Learn what is ⍈normal](https://reader031.fdocuments.in/reader031/viewer/2022030414/5a9f9d327f8b9a71178cf5b3/html5/thumbnails/2.jpg)
Copyright © 2015 KNIME.com AG
KNIME UGM 2012…
Text Mining
Meets
NetworkMining
2
Dr. Killian ThielDr. Tobias KötterPhil Winters
Text Mining for Sentiment
Network Mining for Relevance+ =
![Page 3: Anomaly Detection and Predictive Maintenance · PDF fileWrite to PMML files Time Series Training: Learn what is ⍈normal](https://reader031.fdocuments.in/reader031/viewer/2022030414/5a9f9d327f8b9a71178cf5b3/html5/thumbnails/3.jpg)
Copyright © 2015 KNIME.com AG
KNIME UGM 2013
Time Series + Machine Learning + Big Data
• Rosaria Silipo + Phil Winters
– Manufacturing
– Chemical
– Life Science
– Transportation
– Utilities
– Automotive
– Cyber Security
• The Irish Energy Trials
3
![Page 4: Anomaly Detection and Predictive Maintenance · PDF fileWrite to PMML files Time Series Training: Learn what is ⍈normal](https://reader031.fdocuments.in/reader031/viewer/2022030414/5a9f9d327f8b9a71178cf5b3/html5/thumbnails/4.jpg)
Copyright © 2015 KNIME.com AG
KNIME UGM 2014:
The Internet of Things
Street Maps
Weather
Holiday Schedules
Commuters
Tourists
Topology / Elevations
Bike sources
Bike Sinks
16th and Harvard
Union Station
Rosaria SilipoAaron HartPhil Winters
![Page 5: Anomaly Detection and Predictive Maintenance · PDF fileWrite to PMML files Time Series Training: Learn what is ⍈normal](https://reader031.fdocuments.in/reader031/viewer/2022030414/5a9f9d327f8b9a71178cf5b3/html5/thumbnails/5.jpg)
Copyright © 2015 KNIME.com AG
White Papers, Public Data and the KNIME Workflows!
5
![Page 6: Anomaly Detection and Predictive Maintenance · PDF fileWrite to PMML files Time Series Training: Learn what is ⍈normal](https://reader031.fdocuments.in/reader031/viewer/2022030414/5a9f9d327f8b9a71178cf5b3/html5/thumbnails/6.jpg)
Copyright © 2015 KNIME.com AG
KNIME UGM 2015: The Hot Topics….
• Intrusion Detection / Prevention• Fraud Detection / Prevention• Fault Detection / Prevention• Failure Detection / Prevention• Health Monitoring / Early Warning• Eco-System Disturbances / Prediction• Preventive Maintenance• Predictive Maintenance• Reliability Analysis• Anticipatory Failure Determination• Guided Error Prevention
6
![Page 7: Anomaly Detection and Predictive Maintenance · PDF fileWrite to PMML files Time Series Training: Learn what is ⍈normal](https://reader031.fdocuments.in/reader031/viewer/2022030414/5a9f9d327f8b9a71178cf5b3/html5/thumbnails/7.jpg)
Copyright © 2015 KNIME.com AG
Anomaly Detection
7
PreventionMaintenance
Failu
reWarningReliability
Preventive
Predictive
Pre
dic
tion
Monito
ring
Intru
sio
nHealth
Guid
ed
Fra
ud
Fault
Error
EcoSystem
Early
Dis
turb
ances
Dete
rmin
atio
n
Anticipatory
Analysis“The ideal prevention
would predict all failure
before it occurs”
![Page 8: Anomaly Detection and Predictive Maintenance · PDF fileWrite to PMML files Time Series Training: Learn what is ⍈normal](https://reader031.fdocuments.in/reader031/viewer/2022030414/5a9f9d327f8b9a71178cf5b3/html5/thumbnails/8.jpg)
Copyright © 2015 KNIME.com AG
Anomaly Detection: The Approaches
1. Supervised Anomaly Detection.
A group of patterns are labelled as anomalies and we need to find them. This is just a classification problem where one of the classes is named ”anomaly”.
2. Static Unsupervised Anomaly Detection.
There are a number of labelled pattern classes and suddenly a weird unrecognized outlier pattern shape shows up. Like an unknown heartbeat during an ECG. Either benign or worrisome, an alarm must be triggered.
![Page 9: Anomaly Detection and Predictive Maintenance · PDF fileWrite to PMML files Time Series Training: Learn what is ⍈normal](https://reader031.fdocuments.in/reader031/viewer/2022030414/5a9f9d327f8b9a71178cf5b3/html5/thumbnails/9.jpg)
Copyright © 2015 KNIME.com AG
3. Dynamic Unsupervised Anomaly Detection.
Here some measures change over time till their values are not normal anymore. For example, while a motor is slowly deteriorating, one of the measurements might change till it gets out of control and the motor breaks. We want to stop the motor before it completely breaks producing even more damages.
This problem is similar to number 2 but slightly more challenging because it is not pattern based and it changes slowly over time.
Anomaly Detection: The Approach
![Page 10: Anomaly Detection and Predictive Maintenance · PDF fileWrite to PMML files Time Series Training: Learn what is ⍈normal](https://reader031.fdocuments.in/reader031/viewer/2022030414/5a9f9d327f8b9a71178cf5b3/html5/thumbnails/10.jpg)
Copyright © 2015 KNIME.com AG
Dynamic Unsupervised Anomaly Detection Applications
10
![Page 11: Anomaly Detection and Predictive Maintenance · PDF fileWrite to PMML files Time Series Training: Learn what is ⍈normal](https://reader031.fdocuments.in/reader031/viewer/2022030414/5a9f9d327f8b9a71178cf5b3/html5/thumbnails/11.jpg)
Copyright © 2015 KNIME.com AG
The Fourier Transform
11
Every signal can be filtered into a series of circular paths:• How Big? (amplitude)• How Fast? (Frequency)• Where to Start (Phase Angle)
![Page 12: Anomaly Detection and Predictive Maintenance · PDF fileWrite to PMML files Time Series Training: Learn what is ⍈normal](https://reader031.fdocuments.in/reader031/viewer/2022030414/5a9f9d327f8b9a71178cf5b3/html5/thumbnails/12.jpg)
Copyright © 2015 KNIME.com AG
The Real Challenge…….
![Page 13: Anomaly Detection and Predictive Maintenance · PDF fileWrite to PMML files Time Series Training: Learn what is ⍈normal](https://reader031.fdocuments.in/reader031/viewer/2022030414/5a9f9d327f8b9a71178cf5b3/html5/thumbnails/13.jpg)
Copyright © 2015 KNIME.com AG
Until Now! A Motor and its Sensors
28 time series from 28 sensors on 8 different parts of a mechanical engine.
A1 (input shaft vertical)
A2 (second shaft horizontal upper bearing)
A3 (third shaft horizontal lower bearing)
A4 (internal gear 275 degrees)
A5 (internal gear 190,5 degree)
A6 (input shaft bearing 150)
A7 (input shaft bearing 151)
M1 (torque KnM)
![Page 14: Anomaly Detection and Predictive Maintenance · PDF fileWrite to PMML files Time Series Training: Learn what is ⍈normal](https://reader031.fdocuments.in/reader031/viewer/2022030414/5a9f9d327f8b9a71178cf5b3/html5/thumbnails/14.jpg)
Copyright © 2015 KNIME.com AG
KNIME in Action
14
![Page 15: Anomaly Detection and Predictive Maintenance · PDF fileWrite to PMML files Time Series Training: Learn what is ⍈normal](https://reader031.fdocuments.in/reader031/viewer/2022030414/5a9f9d327f8b9a71178cf5b3/html5/thumbnails/15.jpg)
Copyright © 2015 KNIME.com AG
The Data
• Time Series are FFT-derived Spectral Amplitudes
• There is only one motor breakdown episodes on July 21, 2008
• The breakdown is visible only from some sensors and only in some frequency bands
• The engine was substituted with a new one after the breakdown
![Page 16: Anomaly Detection and Predictive Maintenance · PDF fileWrite to PMML files Time Series Training: Learn what is ⍈normal](https://reader031.fdocuments.in/reader031/viewer/2022030414/5a9f9d327f8b9a71178cf5b3/html5/thumbnails/16.jpg)
Copyright © 2014 KNIME.com AG 16
Align Time Series
![Page 17: Anomaly Detection and Predictive Maintenance · PDF fileWrite to PMML files Time Series Training: Learn what is ⍈normal](https://reader031.fdocuments.in/reader031/viewer/2022030414/5a9f9d327f8b9a71178cf5b3/html5/thumbnails/17.jpg)
Copyright © 2015 KNIME.com AG
Combine all Spectral Time Series
• Data Cleaning and DateTime Conversion
• Frequency Binning
• Average Spectral Amplitude on Frequency Bins vs. Date
![Page 18: Anomaly Detection and Predictive Maintenance · PDF fileWrite to PMML files Time Series Training: Learn what is ⍈normal](https://reader031.fdocuments.in/reader031/viewer/2022030414/5a9f9d327f8b9a71178cf5b3/html5/thumbnails/18.jpg)
Copyright © 2015 KNIME.com AG
Pivoting
Average Spectral Amplitude by Day and Frequency Bin
18
[0-100] [100-200] [200-300] … Hz[1100-1200] [> 1200]
time
01.01.2007
02.01.2007
![Page 19: Anomaly Detection and Predictive Maintenance · PDF fileWrite to PMML files Time Series Training: Learn what is ⍈normal](https://reader031.fdocuments.in/reader031/viewer/2022030414/5a9f9d327f8b9a71178cf5b3/html5/thumbnails/19.jpg)
Copyright © 2015 KNIME.com AG
Time Alignment: Column Merger Node
• Outer join of all column values
• Union of datetime values in one single column
![Page 20: Anomaly Detection and Predictive Maintenance · PDF fileWrite to PMML files Time Series Training: Learn what is ⍈normal](https://reader031.fdocuments.in/reader031/viewer/2022030414/5a9f9d327f8b9a71178cf5b3/html5/thumbnails/20.jpg)
Copyright © 2014 KNIME.com AG 20
Data Visualization
![Page 21: Anomaly Detection and Predictive Maintenance · PDF fileWrite to PMML files Time Series Training: Learn what is ⍈normal](https://reader031.fdocuments.in/reader031/viewer/2022030414/5a9f9d327f8b9a71178cf5b3/html5/thumbnails/21.jpg)
Copyright © 2015 KNIME.com AG
Data Visualization: Time Plots by Frequency Bands
A1-SV3 [500, 600] Hz
New motor piece
Old motor piece
Breaking pointJuly 21, 2008
Only some Spectral Time Series shows the break down
A1-SV3 [0, 100] Hz
![Page 22: Anomaly Detection and Predictive Maintenance · PDF fileWrite to PMML files Time Series Training: Learn what is ⍈normal](https://reader031.fdocuments.in/reader031/viewer/2022030414/5a9f9d327f8b9a71178cf5b3/html5/thumbnails/22.jpg)
Copyright © 2015 KNIME.com AG
Data Visualization: HeatMap
Heatmap of spectrum amplitudes by frequency bands for A1 – SV3 sensor signal
Frequency bands [200,300] Hz and [500,600] Hz show the breaking point in red
![Page 23: Anomaly Detection and Predictive Maintenance · PDF fileWrite to PMML files Time Series Training: Learn what is ⍈normal](https://reader031.fdocuments.in/reader031/viewer/2022030414/5a9f9d327f8b9a71178cf5b3/html5/thumbnails/23.jpg)
Copyright © 2015 KNIME.com AG
Scatter Matrix
23
Jan.01.2007 – Aug.31.2007
Jul.22.2008 – Apr.20.2009
Sep.01.2008 – Jul.21.2008
![Page 24: Anomaly Detection and Predictive Maintenance · PDF fileWrite to PMML files Time Series Training: Learn what is ⍈normal](https://reader031.fdocuments.in/reader031/viewer/2022030414/5a9f9d327f8b9a71178cf5b3/html5/thumbnails/24.jpg)
Copyright © 2015 KNIME.com AG
Data Visualization: Auto-Correlation Map
A1-SV3 [300-400]Hz signal01.Jan.2007 – 31.Aug.2007
A1-SV3 [300-400]Hz signal01.Sep.2007 – 20.Jul.2008
Correlation with past values changes in Time!
![Page 25: Anomaly Detection and Predictive Maintenance · PDF fileWrite to PMML files Time Series Training: Learn what is ⍈normal](https://reader031.fdocuments.in/reader031/viewer/2022030414/5a9f9d327f8b9a71178cf5b3/html5/thumbnails/25.jpg)
Copyright © 2015 KNIME.com AG
Data Visualization: Correlation Map
A1-SV3 signal01.Jan.2007 – 31.Aug.2007
A1-SV3 signal01.Sep.2007 – 20.Jul.2008
Correlation across Frequency Bands changes in Time!
![Page 26: Anomaly Detection and Predictive Maintenance · PDF fileWrite to PMML files Time Series Training: Learn what is ⍈normal](https://reader031.fdocuments.in/reader031/viewer/2022030414/5a9f9d327f8b9a71178cf5b3/html5/thumbnails/26.jpg)
Copyright © 2014 KNIME.com AG 26
Data Analytics
![Page 27: Anomaly Detection and Predictive Maintenance · PDF fileWrite to PMML files Time Series Training: Learn what is ⍈normal](https://reader031.fdocuments.in/reader031/viewer/2022030414/5a9f9d327f8b9a71178cf5b3/html5/thumbnails/27.jpg)
Copyright © 2015 KNIME.com AG
The Approach
• Read all data
• Spectral amplitudes on frequency bins vs. time
• Define “normal”
• Learn “normal” values and patterns
• Detect alarming situations
• Export models as PMML
![Page 28: Anomaly Detection and Predictive Maintenance · PDF fileWrite to PMML files Time Series Training: Learn what is ⍈normal](https://reader031.fdocuments.in/reader031/viewer/2022030414/5a9f9d327f8b9a71178cf5b3/html5/thumbnails/28.jpg)
Copyright © 2015 KNIME.com AG
Learn “normal” to recognize what is not
• We only have normal measures/patterns available till the system starts failing
• We need to train a machine learning algorithm to predict/classify those “normal” patterns/measures
• We then use the machine learning algorithm’s uncertainty to fire an alarm on what might not be “normal”
![Page 29: Anomaly Detection and Predictive Maintenance · PDF fileWrite to PMML files Time Series Training: Learn what is ⍈normal](https://reader031.fdocuments.in/reader031/viewer/2022030414/5a9f9d327f8b9a71178cf5b3/html5/thumbnails/29.jpg)
Copyright © 2015 KNIME.com AG
Learn “normal”: Training Set
A1-SV3 [0, 100] Hz
A1-SV3 [500, 600] HzBreaking pointJuly 21, 2008
Only some Spectral Time Series shows the break down
31 August 2007
Training Set
Predictive Maintenance
![Page 30: Anomaly Detection and Predictive Maintenance · PDF fileWrite to PMML files Time Series Training: Learn what is ⍈normal](https://reader031.fdocuments.in/reader031/viewer/2022030414/5a9f9d327f8b9a71178cf5b3/html5/thumbnails/30.jpg)
Copyright © 2014 KNIME.com AG 30
Time Series Prediction
![Page 31: Anomaly Detection and Predictive Maintenance · PDF fileWrite to PMML files Time Series Training: Learn what is ⍈normal](https://reader031.fdocuments.in/reader031/viewer/2022030414/5a9f9d327f8b9a71178cf5b3/html5/thumbnails/31.jpg)
Copyright © 2015 KNIME.com AG
Learn “normal”: Input and Target Features
inp
ut
target
Time series prediction
![Page 32: Anomaly Detection and Predictive Maintenance · PDF fileWrite to PMML files Time Series Training: Learn what is ⍈normal](https://reader031.fdocuments.in/reader031/viewer/2022030414/5a9f9d327f8b9a71178cf5b3/html5/thumbnails/32.jpg)
Copyright © 2015 KNIME.com AG
Time Series Training: Learn what is “normal”
Train a Linear Auto-Regressive Model
(Lag = 10, no seasonality)
595 AR Models
On each Frequency Band, on each Sensor Signal, on each Motor Part:
![Page 33: Anomaly Detection and Predictive Maintenance · PDF fileWrite to PMML files Time Series Training: Learn what is ⍈normal](https://reader031.fdocuments.in/reader031/viewer/2022030414/5a9f9d327f8b9a71178cf5b3/html5/thumbnails/33.jpg)
Copyright © 2015 KNIME.com AG 33
Last value for missing values
10 past values
Train 595 AR ModelsAnd
Write to PMML files
Time Series Training: Learn what is “normal”
![Page 34: Anomaly Detection and Predictive Maintenance · PDF fileWrite to PMML files Time Series Training: Learn what is ⍈normal](https://reader031.fdocuments.in/reader031/viewer/2022030414/5a9f9d327f8b9a71178cf5b3/html5/thumbnails/34.jpg)
Copyright © 2015 KNIME.com AG 34
A7, SA1 [200,300]Hz
Sep 2007Apr 2009
Apr 15 2008 Jul 21 2008
Prediction
Original signal
Time Series Production: Linear Auto-Regression Prediction
Last value for missing values
10 past values
Prediction,Prediction errors,First level alarms
![Page 35: Anomaly Detection and Predictive Maintenance · PDF fileWrite to PMML files Time Series Training: Learn what is ⍈normal](https://reader031.fdocuments.in/reader031/viewer/2022030414/5a9f9d327f8b9a71178cf5b3/html5/thumbnails/35.jpg)
Copyright © 2015 KNIME.com AG
KNIME in Action
35
![Page 36: Anomaly Detection and Predictive Maintenance · PDF fileWrite to PMML files Time Series Training: Learn what is ⍈normal](https://reader031.fdocuments.in/reader031/viewer/2022030414/5a9f9d327f8b9a71178cf5b3/html5/thumbnails/36.jpg)
Copyright © 2014 KNIME.com AG 36
Signal Execution: Part 1
![Page 37: Anomaly Detection and Predictive Maintenance · PDF fileWrite to PMML files Time Series Training: Learn what is ⍈normal](https://reader031.fdocuments.in/reader031/viewer/2022030414/5a9f9d327f8b9a71178cf5b3/html5/thumbnails/37.jpg)
Copyright © 2015 KNIME.com AG
Time Series Production: Recognize what is not “normal”
Prediction Error PE(t) as:PE(t) = abs(Prediction(t) – x(t))
First Level Alarm
IF PE(t) > Mean(PE(t)) + K*stddev(MA(t)) => alm(t) = PE(t)ELSE alm(t) = 0
MA(t) = Moving Average(alm(t), N=21, backward window)
Second Level AlarmIF MA(t) > 0.01 => Alarm
On each Frequency Band, Sensor Motor part:
![Page 38: Anomaly Detection and Predictive Maintenance · PDF fileWrite to PMML files Time Series Training: Learn what is ⍈normal](https://reader031.fdocuments.in/reader031/viewer/2022030414/5a9f9d327f8b9a71178cf5b3/html5/thumbnails/38.jpg)
Copyright © 2015 KNIME.com AG
Time Series Production: Second Level Alarms
Sep 2007
Mar 06 2008
May 05 2008
MA
(alm
(t))
Jul 2008
A7, SA1 [200,300]Hz
A7, SA1 [400,500]Hz
A7, SA1 [300,400]Hz
If alarm => send email
![Page 39: Anomaly Detection and Predictive Maintenance · PDF fileWrite to PMML files Time Series Training: Learn what is ⍈normal](https://reader031.fdocuments.in/reader031/viewer/2022030414/5a9f9d327f8b9a71178cf5b3/html5/thumbnails/39.jpg)
Copyright © 2015 KNIME.com AG
Time Series Production: Second Level Alarms R Stacked Plot
Sep 2007
Mar 06 2008
May 05 2008
MA
(alm
(t))
Jul 2008
A7, SA1 [200,300]Hz
![Page 40: Anomaly Detection and Predictive Maintenance · PDF fileWrite to PMML files Time Series Training: Learn what is ⍈normal](https://reader031.fdocuments.in/reader031/viewer/2022030414/5a9f9d327f8b9a71178cf5b3/html5/thumbnails/40.jpg)
Copyright © 2014 KNIME.com AG 40
Signal Execution: Part 2
![Page 41: Anomaly Detection and Predictive Maintenance · PDF fileWrite to PMML files Time Series Training: Learn what is ⍈normal](https://reader031.fdocuments.in/reader031/viewer/2022030414/5a9f9d327f8b9a71178cf5b3/html5/thumbnails/41.jpg)
Copyright © 2015 KNIME.com AG 41
Time Series Production: Use KNIME to Send Alarms !!!
Using RESTful Services:
![Page 42: Anomaly Detection and Predictive Maintenance · PDF fileWrite to PMML files Time Series Training: Learn what is ⍈normal](https://reader031.fdocuments.in/reader031/viewer/2022030414/5a9f9d327f8b9a71178cf5b3/html5/thumbnails/42.jpg)
Copyright © 2015 KNIME.com AG
KNIME in Action
42
![Page 43: Anomaly Detection and Predictive Maintenance · PDF fileWrite to PMML files Time Series Training: Learn what is ⍈normal](https://reader031.fdocuments.in/reader031/viewer/2022030414/5a9f9d327f8b9a71178cf5b3/html5/thumbnails/43.jpg)
Copyright © 2015 KNIME.com AG
But we can go Further! Pattern Recognition
inp
ut
target
Input/target
Time series prediction
Pattern Recognition
![Page 44: Anomaly Detection and Predictive Maintenance · PDF fileWrite to PMML files Time Series Training: Learn what is ⍈normal](https://reader031.fdocuments.in/reader031/viewer/2022030414/5a9f9d327f8b9a71178cf5b3/html5/thumbnails/44.jpg)
Copyright © 2015 KNIME.com AG
KNIME
44
![Page 45: Anomaly Detection and Predictive Maintenance · PDF fileWrite to PMML files Time Series Training: Learn what is ⍈normal](https://reader031.fdocuments.in/reader031/viewer/2022030414/5a9f9d327f8b9a71178cf5b3/html5/thumbnails/45.jpg)
Copyright © 2014 KNIME.com AG 45
Pattern Recognition
Iris Adae
![Page 46: Anomaly Detection and Predictive Maintenance · PDF fileWrite to PMML files Time Series Training: Learn what is ⍈normal](https://reader031.fdocuments.in/reader031/viewer/2022030414/5a9f9d327f8b9a71178cf5b3/html5/thumbnails/46.jpg)
Copyright © 2015 KNIME.com AG
Recognize what is not “normal”
Pattern Recognition
• Output Probability / Confidence < threshold => Alarm
• Auto-associator pattern reproduction error > threshold => Alarm
![Page 47: Anomaly Detection and Predictive Maintenance · PDF fileWrite to PMML files Time Series Training: Learn what is ⍈normal](https://reader031.fdocuments.in/reader031/viewer/2022030414/5a9f9d327f8b9a71178cf5b3/html5/thumbnails/47.jpg)
Copyright © 2015 KNIME.com AG
Pattern Recognition: General Windowing Scheme
• Model the system with data from the past
• Use the system with most recent data point
47
Time
01.Jan.2007 – 31.Aug.2007(as normality)
01.Sep.2007 – 20.Jul.2008(check for anomaly)
Single observationon multiple
frequency bands
![Page 48: Anomaly Detection and Predictive Maintenance · PDF fileWrite to PMML files Time Series Training: Learn what is ⍈normal](https://reader031.fdocuments.in/reader031/viewer/2022030414/5a9f9d327f8b9a71178cf5b3/html5/thumbnails/48.jpg)
Copyright © 2015 KNIME.com AG
Pattern Recognition: Cluster Based
• Fuzzy c-means -> clusters
• Fuzzy c-means also generates an outlier cluster
• If recent data point is in outlier cluster -> Anomaly
48
![Page 49: Anomaly Detection and Predictive Maintenance · PDF fileWrite to PMML files Time Series Training: Learn what is ⍈normal](https://reader031.fdocuments.in/reader031/viewer/2022030414/5a9f9d327f8b9a71178cf5b3/html5/thumbnails/49.jpg)
Copyright © 2015 KNIME.com AG
Pattern Recognition: Cluster Based Results
49
Apr 14 2008
Jul 21 2008
• Results: Fuzzy Cluster
![Page 50: Anomaly Detection and Predictive Maintenance · PDF fileWrite to PMML files Time Series Training: Learn what is ⍈normal](https://reader031.fdocuments.in/reader031/viewer/2022030414/5a9f9d327f8b9a71178cf5b3/html5/thumbnails/50.jpg)
Copyright © 2015 KNIME.com AG
Pattern Recognition: Similarity Based
• Similarity Search -> nearest neighbor to current pattern
• If no nearest neighbor -> Anomaly
50
![Page 51: Anomaly Detection and Predictive Maintenance · PDF fileWrite to PMML files Time Series Training: Learn what is ⍈normal](https://reader031.fdocuments.in/reader031/viewer/2022030414/5a9f9d327f8b9a71178cf5b3/html5/thumbnails/51.jpg)
Copyright © 2015 KNIME.com AG
Pattern Recognition: Similarity Based
51
Apr 14 2008
Jul 21 2008
![Page 52: Anomaly Detection and Predictive Maintenance · PDF fileWrite to PMML files Time Series Training: Learn what is ⍈normal](https://reader031.fdocuments.in/reader031/viewer/2022030414/5a9f9d327f8b9a71178cf5b3/html5/thumbnails/52.jpg)
Copyright © 2015 KNIME.com AG
KNIME in Action
52
![Page 53: Anomaly Detection and Predictive Maintenance · PDF fileWrite to PMML files Time Series Training: Learn what is ⍈normal](https://reader031.fdocuments.in/reader031/viewer/2022030414/5a9f9d327f8b9a71178cf5b3/html5/thumbnails/53.jpg)
Copyright © 2015 KNIME.com AG
But KNIME can do even more !!!
53
Slow motion cameras can now record 4,400,000,000,000 frames per Second
We don’t have images of the motor turning…… but what if we did ?
![Page 54: Anomaly Detection and Predictive Maintenance · PDF fileWrite to PMML files Time Series Training: Learn what is ⍈normal](https://reader031.fdocuments.in/reader031/viewer/2022030414/5a9f9d327f8b9a71178cf5b3/html5/thumbnails/54.jpg)
Copyright © 2014 KNIME.com AG 54
Image Processing
Christian Dietz
![Page 55: Anomaly Detection and Predictive Maintenance · PDF fileWrite to PMML files Time Series Training: Learn what is ⍈normal](https://reader031.fdocuments.in/reader031/viewer/2022030414/5a9f9d327f8b9a71178cf5b3/html5/thumbnails/55.jpg)
Copyright © 2015 KNIME.com AG
The Dataset
Data from DAGM 2007 Competition(data sponsored by Robert Bosch Corporate Research department, Schwieberdingen, Germany)
Goal: Find defects in images withminimal user-interaction
Dataset contains about 1100 images.
55
![Page 56: Anomaly Detection and Predictive Maintenance · PDF fileWrite to PMML files Time Series Training: Learn what is ⍈normal](https://reader031.fdocuments.in/reader031/viewer/2022030414/5a9f9d327f8b9a71178cf5b3/html5/thumbnails/56.jpg)
Copyright © 2015 KNIME.com AG
What‘s a Defect?
56
![Page 57: Anomaly Detection and Predictive Maintenance · PDF fileWrite to PMML files Time Series Training: Learn what is ⍈normal](https://reader031.fdocuments.in/reader031/viewer/2022030414/5a9f9d327f8b9a71178cf5b3/html5/thumbnails/57.jpg)
Copyright © 2015 KNIME.com AG
How to detect them in KNIME?
57
![Page 58: Anomaly Detection and Predictive Maintenance · PDF fileWrite to PMML files Time Series Training: Learn what is ⍈normal](https://reader031.fdocuments.in/reader031/viewer/2022030414/5a9f9d327f8b9a71178cf5b3/html5/thumbnails/58.jpg)
Copyright © 2015 KNIME.com AG
Reading Images
58
![Page 59: Anomaly Detection and Predictive Maintenance · PDF fileWrite to PMML files Time Series Training: Learn what is ⍈normal](https://reader031.fdocuments.in/reader031/viewer/2022030414/5a9f9d327f8b9a71178cf5b3/html5/thumbnails/59.jpg)
Copyright © 2015 KNIME.com AG
Calculating Features
For each pixel we calculate features in the neighborhood of the pixel.
Examples: Intensity Statistics (Mean, Variance, StdDev, etc), Texture, …
Computational expensive step: We are working on a faster version.
59
![Page 60: Anomaly Detection and Predictive Maintenance · PDF fileWrite to PMML files Time Series Training: Learn what is ⍈normal](https://reader031.fdocuments.in/reader031/viewer/2022030414/5a9f9d327f8b9a71178cf5b3/html5/thumbnails/60.jpg)
Copyright © 2015 KNIME.com AG
Annotate Training Data
Defect
Not a defect
60
![Page 61: Anomaly Detection and Predictive Maintenance · PDF fileWrite to PMML files Time Series Training: Learn what is ⍈normal](https://reader031.fdocuments.in/reader031/viewer/2022030414/5a9f9d327f8b9a71178cf5b3/html5/thumbnails/61.jpg)
Copyright © 2015 KNIME.com AG
Learn a Model
Given the training data we can train a model to distinguish defect pixels fromnot defect pixels.
Model: Random Forest
Additional: We determine the average size of the defects for pruning too smalland too big defects
61
![Page 62: Anomaly Detection and Predictive Maintenance · PDF fileWrite to PMML files Time Series Training: Learn what is ⍈normal](https://reader031.fdocuments.in/reader031/viewer/2022030414/5a9f9d327f8b9a71178cf5b3/html5/thumbnails/62.jpg)
Copyright © 2015 KNIME.com AG
Detecting Defects
Defect
62
Defect
Defect
![Page 63: Anomaly Detection and Predictive Maintenance · PDF fileWrite to PMML files Time Series Training: Learn what is ⍈normal](https://reader031.fdocuments.in/reader031/viewer/2022030414/5a9f9d327f8b9a71178cf5b3/html5/thumbnails/63.jpg)
Copyright © 2015 KNIME.com AG
Detecting Defects: pass the new column back to learning!
![Page 64: Anomaly Detection and Predictive Maintenance · PDF fileWrite to PMML files Time Series Training: Learn what is ⍈normal](https://reader031.fdocuments.in/reader031/viewer/2022030414/5a9f9d327f8b9a71178cf5b3/html5/thumbnails/64.jpg)
Copyright © 2015 KNIME.com AG
KNIME in Action
64
![Page 65: Anomaly Detection and Predictive Maintenance · PDF fileWrite to PMML files Time Series Training: Learn what is ⍈normal](https://reader031.fdocuments.in/reader031/viewer/2022030414/5a9f9d327f8b9a71178cf5b3/html5/thumbnails/65.jpg)
Copyright © 2015 KNIME.com AG
Possible Next Steps…..
65
Random Forrest and Threshold DetectionCombining the ModelsText ProcessingBig Data
![Page 66: Anomaly Detection and Predictive Maintenance · PDF fileWrite to PMML files Time Series Training: Learn what is ⍈normal](https://reader031.fdocuments.in/reader031/viewer/2022030414/5a9f9d327f8b9a71178cf5b3/html5/thumbnails/66.jpg)
Copyright © 2015 KNIME.com AG
Thank You
40k
60k
20k
66
We will announce White Paper availability!
Questions:• http://tech.knime.org/forum• [email protected]• [email protected]
Follow us:• Twitter: @KNIME• LinkedIn: https://www.linkedin.com/groups?gid=2212172
• KNIME Blog: http://www.knime.org/blog