TB 6/22/06 1 Anomaly Detection for Prognostic and Health Management System Development Tom...
-
Upload
anna-jordan -
Category
Documents
-
view
219 -
download
0
Transcript of TB 6/22/06 1 Anomaly Detection for Prognostic and Health Management System Development Tom...
TB 6/22/06 1
Anomaly Detection for Prognostic Anomaly Detection for Prognostic and Health Management System and Health Management System
DevelopmentDevelopment
Tom BrothertonTom Brotherton
TB 6/22/06 2
New Stealth TechnologyNew Stealth Technology
TB 6/22/06 3
OutlineOutline
• What is Anomaly Detection– Different types of anomaly detectors
• Radial Basis Function Neural Net Anomaly Detector– The basics– Comparison with other neural net approaches– Feature ‘off-nominal’ distance measures– Training
• Implementations– Continuous = Gas turbine engine monitoring– Snap shot = Web server helicopter vibration condition indicators
• RBF NN & Boxplots• Application to detection of helicopter bearing fault• Application to monitoring fish behavior for water quality monitoring
TB 6/22/06 4
What is Anomaly Detection?What is Anomaly Detection?
• Anomaly Detection = The Detection of Any Off-Nominal Event Data– Known fault conditions– Novel event = New - never seen before data
• New type of fault• New variation of ‘known’ nominal or fault data
• What is ‘Nominal’– Sets of parameters that behave as expected
• Physics models• Statistical models
TB 6/22/06 5
ApproachesApproaches
Applicability
Acc
urac
y &
Cos
t Physics
Parametric- Estimate of physics
Empirical- Derived from collected data
•State Variable Models (derived from physics)
•JPL: BEAM (coherence = model of linear relationships)
•Neural nets (non-linear relationships)
•Academic: Support Vector
•Ex: Gas Turbine Engine Deck: Component level physics model
•Simple statistics
•Hybrid Model: Combine Physics + Empirical
•Fused empirical: BEAM + NN
TB 6/22/06 6
Empirical ModelingEmpirical Modeling
Collected ‘Nominal’ Data
IdeaIdea: Theoretical boundary (multi-dimensional ‘tube’) that data should lie within: - Nominal data is inside the boundary - Anomaly data is outside
Problem: How to estimate / approximate the boundary?Problem: How to estimate / approximate the boundary?
An anomaly
Problem: What measurement(s) caused the
anomaly?
Problem: What measurement(s) caused the
anomaly?
Problem: How far off-nominal is the anomaly / feature?
Problem: How far off-nominal is the anomaly / feature?
TB 6/22/06 7
RBF Neural Net Anomaly Detection: The IdeaRBF Neural Net Anomaly Detection: The Idea
• Dynamic data = Lots of NN basis units to model
• Piecewise stationary approximation
• Distance measure = Function of the signal set
• Individual signal distances from nominal = distance from “closest” basis unit
– Detection can be for set of signals when no single signal is anomalous
• The model can be adaptively updated to include additional data / known fault classes
• Trajectories of features relative to basis unit = Prognosis
= Sample of nominal data
= Sample of anomalous data
NN = Model for Nominal Data
‘Distance’ fromNominal Model
Yes
?
Radial Basis Function Radial Basis Function (RBF) Neural Net Model(RBF) Neural Net ModelRadial Basis Function Radial Basis Function
(RBF) Neural Net Model(RBF) Neural Net Model
TB 6/22/06 8
MLP NN
?
Why Use Radial Basis Function Neural Nets?Why Use Radial Basis Function Neural Nets?
• Radial Basis Function Neural Net– Nearest neighbor classifier– Distance metric : Measure “nominal”– Multi-layer perceptron (MLP) does not have these properties
RBF NN
?
TB 6/22/06 9
Support Vector MachineSupport Vector Machine
RBF Model
Support Vector Machine Model
• In some sense, much better model of ‘truth’ …. but- Automated selection of number of
basis units• Lots!
• Trade off between fidelity vs smoothness
• Not practical for on-wing• How to compute individual signal
distances• Loss of intuition
Training data
TB 6/22/06 10
NN = Model for Nominal Data
Distance s1
Distance s2
Feature Distance CalculationFeature Distance Calculation
?
MahalanobisMahalanobis
Nearest Neighbor Distance
TB 6/22/06 11
NN = Model for Nominal Data
Alternative Distance CalculationAlternative Distance Calculation
Truth
Closest Basis Unit
- Truth: Single Feature X = ‘Bad’-Report: Feature X = ‘OK’ & Feature Y = ‘Bad’
-Alternative Distance = Which Basis Unit gives the smallest number of individual off-nominal features -> Hamming Distance (from digital communications decoding)
TB 6/22/06 12
‘‘RBF’ NN ArchitecturesRBF’ NN Architectures
Is output for Nominal?
= 1 Yes> 1- Likely< 1- ?< 1- No 0< < <1
•••
Inpu
t fea
ture
s
Basis Units
Weights
Gaussian elliptical basis function : Fuzzy membership basis function :Rayleigh basis function :
DetectorOutput
= Gaussian Mixture ModelGood for magnitude spectral data
* Basis function is ‘matched’ to the data distributionFor those who like things fuzzy
TB 6/22/06 13
- Small number of clusters Small number of basis units Low False Alarms
Very general Missed detections
Too General ?
- Large number of clusters Good ‘tracking’ of data dynamics Large number of basis units
More sensitive to outliers More false alarms
Over Trained ?
Don’t know a-priori what are the ‘best’ settings
Training : Neural Net Architectures – How to Training : Neural Net Architectures – How to select parametersselect parameters
TB 6/22/06 15
M of N DetectionM of N Detection
Detection?False alarm?
Large scale factor
Small scale factor
• Trade off single point detection capability vs false alarm rate
Large Scale Factor / Small N- Short – high SNR anomalies
Small Scale Factor / Large N- Long – persistent – low SNR
anomalies
• Trade off single point detection capability vs false alarm rate
Large Scale Factor / Small N- Short – high SNR anomalies
Small Scale Factor / Large N- Long – persistent – low SNR
anomalies
4 points persist over time = detection
Only 2 points = false alarm
False alarms?
Idea: M of N detection allows one sample high false alarm rate – Then integrate over time to remove
Idea: M of N detection allows one sample high false alarm rate – Then integrate over time to remove
TB 6/22/06 16
AlternativesAlternatives
• This technique works well– Demonstrated by Pratt & Whitney for C-17 F117
applications• Transient engine operations
– Long time to train – lots of different types of transients– Model can become very complex
• Engine control system• On-wing memory and timing constraints
• Alternative– Combine equipment operating regime recognition with
anomaly detector– Ex: Identify steady operation and then take a snapshot of
the data• Simple statistics may suffice
TB 6/22/06 17
Example Gas Turbine OperationsExample Gas Turbine Operations
Regime recognition- Regimes:
• Transient Throttle up• Transient Throttle down• Steady state – B14 open• Steady state – B14 closed
Neural NetDetection
Neural NetDetection
Scale Signal
Off-NominalSignal
Distance
RegimeRecognition
Trained NNs
Neural NetDetection
Input Signal Vector
DetectionFlag
Neural NetSelect
Median Filter
Break the big problem in to a set of small problems
Break the big problem in to a set of small problems
TB 6/22/06 18
Anomaly Detection of Stationary Regime Anomaly Detection of Stationary Regime Detected DataDetected Data
• Web Server Implementation for Helicopter Vibration Data– Condition Indicators (CIs) = Features derived
from on-board vibration measurements
• Two types of problems:– Single CI for a component
• Simple statistics solution = Boxplot– Intuitive = Army user’s like it
• RBF neural net implementation as well
– Multi-CIs for a component• RBF neural net implementation
TB 6/22/06 19
On Board SystemOn Board System
Absorbers Hanger Bearings
Tail Gearbox
Intermediate Gearbox
TransmissionsEngines
Cockpit VMU
Advanced Rotor Smoothing / Engine
Diagnostics
• 18 Sensors Installed – Vibration• Automated Exceedance Monitoring using HUD data• Automated engine HIT, Max Power Check and exceedances• Complete aircraft vibration survey in under 30 seconds
Accelerometer
Tach Sensor
Other Connections
FWDLAT
FWDVRT
Main Rotor
MainD/S
IAC-1209Modern Signal Processing Unit
(MSPU)
USB Memory Drive
Cockpit Control Head
USB Download
Ethernet
+28VDC Power
Parameter Data
FWDSP
CPITVRT
CPITLAT
FWDXMSNVRT
FWDXMSNLAT
HB2
HB3
HB4
HB5
HB6
HB7
XSHAFT1
XSHAFT2
ENG1COMP
ENG1NOSE
ENG1AXIAL
ENG1LAT
ENG2COMP
ENG2NOSE
ENG2AXIAL
ENG2LAT
AFTLAT
AFTVRT
AFTSPCBOXOCFA
CBOXOCLAT
APU
AFTFANLAT
AFTXMSNVRT
AFTXMSNLAT
Configuration• 36 Vibration Sensors• 2 Speed Sensors• 1553 connection to HUD
CVR-FDRCVR-FDR
TB 6/22/06 20
Aircraft / Server Physical ConnectivityAircraft / Server Physical Connectivity
SCARNG
Deployed Unit
AARNG
INTERNET
AIRCRAFT OEMs
VMEPPARTNER
PC-GBS Facility
PC-GBS Facility
PC-GBS Remote
PC-GBS Remote
PC-GBS Remote
USB Memory Stick Data Download
Browser
Wireless link
TB 6/22/06 21
Aircraft / Server Logical ConnectivityAircraft / Server Logical Connectivity
-Army P-GBS-Army P-GBS
Support Team- e-mail notification- Fleet level reports
- Automated s/w upgrades
Aircraft Maintenance-Electronic help desk
- Automated data archive- Automated s/w upgrades
Fleet Statistics& Reports
Fleet Statistics& Reports
Help DeskHelp Desk
Data ArchiveA/C config filesData Archive
A/C config files
MDS ServerMDS Server
Help Training BaseElectronic Manuals
FAQs
Help Training BaseElectronic Manuals
FAQs
PrognosticsPrognosticsDiagnosticsDiagnostics
NetworkSecurityNetworkSecurity
AutomatedData ArchiveAutomated
Data Archive
AnomalyDetectionAnomalyDetection
Portable SystemPortable System
- Army F-GBS- Army F-GBSWeb ClientWeb Client
BrowserBrowser
Facility SystemsFacility Systems
AnomalyDetection
TB 6/22/06 22
Advanced Engineering on the WebAdvanced Engineering on the Web
The role of anomaly detection on the website is to detect and bring to engineering’s attention the MOST INTERESTING data = Something that has NOT been encountered before
- More normal data not really of interest
TB 6/22/06 23
Default based on boxplot statistics
User set
Single Feature Anomaly DetectionSingle Feature Anomaly Detection
BoxplotsBoxplots = Simple statistics - single feature anomaly detector. No Gaussian assumption, just counting points. They seem to work very well!
BoxplotsBoxplots = Simple statistics - single feature anomaly detector. No Gaussian assumption, just counting points. They seem to work very well!
TB 6/22/06 24
Threshold SettingThreshold Setting
TB 6/22/06 25
Anomaly AnalysisAnomaly Analysis
Summary of all aircraft
TB 6/22/06 26
The Raw DataThe Raw Data
TB 6/22/06 27
Gaussian Transformation DataGaussian Transformation Data
• Problem: How to select a “matched” basis function– Gaussian assumption? Usually violated!
• Statistical Model Fit– Transform data to be Gaussian
• Transformation stored and is part of the model– Almost always only a single basis unit is required!
• Works on single feature data• All processing “behind the scenes” done on transformed data
Original Transformed
TB 6/22/06 28
RBF Anomaly DetectionRBF Anomaly Detection
TB 6/22/06 29
RBF Anomaly DetectionRBF Anomaly Detection
TB 6/22/06 30
Case Study: Apache Swashplate Bearing Spectral Case Study: Apache Swashplate Bearing Spectral Server DataServer Data
• Anomalous data identified with RBF NN AD running on the Server– Aircraft was in Iraq
– Automatic email alert sent to users• “Evidence” sent as well
– Data reviewed by AED-Aeromechanics and IAC via iMDS website• Large peak in spectral data at 1250 Hz for tail #460
• Sidebands spaced at intervals corresponding to bearing fault frequencies
• Suspected bad swashplate bearing
Tail 460
0
1
2
3
4
5
0 2000 4000 6000
Frequency (Hz)
Ma
gn
itu
de
(g
)
Tail 460
Tail 986
Tail 460OtherA/C
Other A/C
Main SP Spectra
TB 6/22/06 31
Case Study Case Study Apache Swashplate BearingApache Swashplate Bearing
• AED-Aeromechanics acquired raw vibe data Apr 04 and received swashplate May 04 before aircraft was turned-in for D model conversion
• Swashplate disassembled by PIF per DMWR Aug 04
• Minor spalling, corrosion and broken cage discovered
• Additional algorithms developed from raw data and implemented into VMEP for release Sep 04
Broken Cage
Spalling/Corrosion
TB 6/22/06 32
Follow UpFollow Up
• Specific algorithms to identify this fault now included with the on-board system
• US Army now uses ‘on-condition’ information from the system to perform maintenance– True condition-based maintenance (CBM)
TB 6/22/06 33
Other ApplicationsOther Applications
IAC 1090 is a mobile, web-enabled automated biomonitoring system that utilizing the ventilatory and body movement patterns of the bluegill fish as a bio-sensor, much like a canary in a coal mine.
Sixteen Bluegills are placed in individual flow-through Plexiglas chambers. Each chamber is equipped with an individual water input and drainage system. By utilizing sixteen different Bluegills, the IAC 1090 samples more biosensors than any other system on the market resulting in lower false alarm rates.
All fish generate a micro volt level electric field. Each individual fish is monitored by non-contact electrodes suspended above and below each fish in a Plexiglas chamber.
The electrical signals generated by the fish’s normal movement is amplified, filtered and passed on via the internet to IAC’s Bio-Monitoring Expert (BME) software system for automated analysis.
Water Quality Bio-MonitorWater Quality Bio-Monitor
TB 6/22/06 34
BME is a neural network based expert system that provides for rapid, real time assessment of water toxicity based on the ventilatory behavior of fish. BME has shown excellent detection capabilities for toxic compounds with a low false alarm rate. False alarms, common in other similar systems, are typically generated by normal, non-toxic variations in the environment.
Automated data collection and management tools, user interfaces, and real-time data interpretation employing advanced (artificial intelligence) models of fish ventilatory behavior make BME easy to use.
Remote (Internet) access to IAC 1090 is provided through an easy-to-use graphical user interface. BME’s modular design provides users with the ability to reconfigure the system for different biomonitoring applications and biosensors
Water Quality Bio-MonitorWater Quality Bio-Monitor
TB 6/22/06 35
Questions?Questions?
Conference papers / case studies available at:
www.iac-online.comwww.iac-online.com