Advanced Diagnostics Algorithms in Online Field Device Monitoring Vagan Terziyan (editor) ...

Advanced Diagnostics Algorithms in Online Field Device Monitoring

Vagan Terziyan (editor)

http://www.cs.jyu.fi/ai/Metso_Diagnostics.ppt

“Industrial Ontologies” Group: http://www.cs.jyu.fi/ai/OntoGroup/index.html

“Industrial Ontologies” Group, Agora Center, University of Jyväskylä, 2003

Contents

Introduction: OntoServ.NetOntoServ.Net – Global “Health-Care” Environment for Industrial Devices;

Bayesian MetanetworksBayesian Metanetworks for Context-Sensitive Industrial Diagnostics;

Temporal Industrial DiagnosticsTemporal Industrial Diagnostics with Uncertainty;

Dynamic IntegrationDynamic Integration of Classification Algorithms for Industrial Diagnostics;

Industrial Diagnostics with Real-Time Neuro-Real-Time Neuro-Fuzzy SystemsFuzzy Systems;

Conclusion.

Vagan Terziyan

Oleksiy Khriyenko

Oleksandr Kononenko

Andriy Zharko

Web Services for Smart DevicesWeb Services for Smart Devices

Smart industrial devices can be also Web Service “users”. Their embedded agents are able to monitor the state of appropriate device, to communicate and exchange data with another agents. There is a good reason to launch special Web Services for such smart industrial devices to provide necessary online condition monitoring, diagnostics, maintenance support, etc.

OntoServ.Net: “Semantic Web Enabled Network of Maintenance Services for Smart Devices”, Industrial Ontologies Group, Tekes Project Proposal, March 2003,

Global Network of Maintenance ServicesGlobal Network of Maintenance Services

OntoServ.Net: “Semantic Web Enabled Network of Maintenance Services for Smart Devices”, Industrial Ontologies Group, Tekes Project Proposal, March 2003,

http://images.google.com/imgres?imgurl=www.sun.de/Produkte/Promo/b4b/images/server.jpg&imgrefurl=http://www.sun.de/Produkte/Promo/b4b/voraussetzungen.html&h=195&w=140&prev=/images%3Fq%3Dserver%26start%3D60%26svnum%3D100%26hl%3Dru%26lr%3D%26ie%3DUTF-8%26oe%3DUTF-8%26newwindow%3D1%26sa%3DN

http://images.google.com/imgres?imgurl=www.mincom.cz/images/maintenance.jpg&imgrefurl=http://www.mincom.cz/maintenance.html&h=189&w=200&prev=/images%3Fq%3Dmaintenance%26start%3D40%26svnum%3D100%26hl%3Dru%26lr%3D%26ie%3DUTF-8%26oe%3DUTF-8%26newwindow%3D1%26sa%3DN









Embedded Maintenance PlatformsEmbedded Maintenance Platforms

Service Agents

Host Agent

Embedded Platform

Based on the online diagnostics, a service agent, selected for the

specific emergency situation, moves to the embedded platform to help the host agent to

manage it and to carry out the predictive

maintenance activities

Maintenance Service

OntoServ.NetOntoServ.Net Challenges Challenges

New group of Web service users – smart industrial smart industrial devicesdevices.

InternalInternal (embedded) and externalexternal (Web-based) agent enabled service platformsservice platforms.

“Mobile Service ComponentMobile Service Component” concept supposes that any service component can move, be executed and learn at any platform from the Service Network, including service requestor side.

Semantic Peer-to-PeerSemantic Peer-to-Peer concept for service network management assumes ontology-based decentralized service network management.

Agents in Semantic WebAgents in Semantic Web

1. “I feel bad, pressure more than 200,

headache, … Who can advise what to do ? “

4. “Never had such experience. No

idea what to do”

3. “Wait a bit, I will give you some pills”

2. “ I think you should stop drink beer for a while “

Agents in Semantic Web supposed to understand each other because they will share common standard, platform, ontology and language

The Challenge: The Challenge: GGlobal lobal UUnderstanding nderstanding eeNNvironmentvironment ( (GUNGUN))

How to make entities from our physical world to understand

each other when necessary ?..

… Its elementary ! But not easy !! Just to make agents from them !!!

GUN ConceptGUN Concept

Entities will interoperate through OntoShells, which are “supplements” of these

entities up to Semantic Web

enabled agents

1. “I feel bad, temperature 40, pain in stomach, … Who can advise what to do ? “

2. “I have some pills for you”

Semantic Web: Before GUNSemantic Web: Before GUN

Semantic Web Resources

Semantic Web Applications

Semantic Web applications “understand”, (re)use, share, integrate, etc. Semantic Web

resources

GUN Concept:GUN Concept: All GUN resources “understand” each otherAll GUN resources “understand” each other

Real World objects

OntoAdapters

Real World Object ++ OntoAdapter +

+ OntoShell == GUN ResourceGUN Resource

GUNGUN

OntoShells

Real World objects of new generation (OntoAdapter inside)

Read Our Recent ReportsRead Our Recent Reports

Semantic Web: The Future Starts TodaySemantic Web: The Future Starts Today (collection of research papers and presentations of Industrial Ontologies

Group for the Period November 2002-April 2003)

Semantic Web and Peer-to-Peer: Semantic Web and Peer-to-Peer: Integration and Interoperability in IndustryIntegration and Interoperability in Industry

Semantic Web Enabled Web Services: Semantic Web Enabled Web Services: State-of-Art and ChallengesState-of-Art and Challenges

Distributed Mobile Web Services Based on Semantic Web: Distributed Mobile Web Services Based on Semantic Web: Distributed Industrial Product Maintenance SystemDistributed Industrial Product Maintenance System

Available online in: http://www.cs.jyu.fi/ai/OntoGroup/index.html

Industrial Ontologies GroupIndustrial Ontologies Group

V. Terziyan

A. Zharko

O. Kononenko

O. Khriyenko

Vagan Terziyan

Oleksandra Vitko

Example of Simple Bayesian Network

X

Y

P(X)

P(Y)-?

P(Y|X)

n

iiin XParentsXPXXXP

121 ))(|(),...,,(

)|()(),( ijiij xXyYPxXPxXyYP

i

ijij xXyYPxXPyYP )|()()(

)(

)|()()|(

j

ijiji yYP

xXyYPxXPyYxXP

Conditional (in)dependence rule

Joint probability rule

Marginalization rule

Bayesian rule

Contextual and Predictive Attributes

Machine

Environment

Sensors

XX x1 x2 x3 x4 x5 x6 x7

predictive attributes contextual attributes

air pressure

dust

humidity

temperature

emission

Contextual Effect on Conditional Probability

XX x1 x2 x3 x4 x5 x6 x7


xk xr

Assume conditional dependence between predictive attributes

(causal relation between physical quantities)…

xt

… some contextual attribute may effect

directly the conditional dependence between

predictive attributes but not the attributes itself

Contextual Effect on Conditional Probability

X

Y

P(X)

P(Y)-? P(P(Y|X)|Z)

Z

P(Z) P(Y|X)

pk(Y|X)

P(P(Y|X))

•X ={x1, x2, …, xn} – predictive attribute with

n values;•Z ={z1, z2, …, zq} – contextual attribute with q

values;•P(Y|X) = {p1(Y|X), p2(Y|X), …, p r(Y|X)} –

conditional dependence attribute (random variable) between X and Y with r possible values;•P(P(Y|X)|Z) – conditional dependence between attribute Z and attribute P(Y|X);

})]|)|()|(()([

)()|({)(

1

1 1

q

mmkm

r

k

n

iiijkj

zZXYpXYPPzZP

xXPxXyYpyYP

Contextual Effect on Unconditional Probability

XX x1 x2 x3 x4 x5 x6 x7


xk

Assume some predictive attribute is a random

variable with appropriate probability distribution

for its values…

xt

… some contextual attribute may effect

directly the probability distribution of the predictive attribute

x1 x2 x3x4

XX

P(X)P(X)

Contextual Effect on Unconditional Probability

X

Y

P(Y)-? P(P(X)|Z)

Z

P(Z)

P(X)

pk(X)

P(P(X))

P(Y|X)

X ={x1, x2, …, xn} – predictive attribute with n

values;

· Z ={z1, z2, …, zq} – contextual attribute with q values

and P(Z) – probability distribution for values of Z;

• P(X) = {p1(X), p2(X), …, pr(X)} – probability

distribution attribute for X (random variable) with r possible values (different possible probability distributions for X) and P(P(X)) is probability distribution for values of attribute P(X);

· P(Y|X) is a conditional probability distribution of Y given X;

· P(P(X)|Z) is a conditional probability distribution

for attribute P(X) given Z

})]|)()(()([

)()|({)(

1

1 1

q

mmkm

r

k

n

iikijj

zZXpXPPzZP

xXpxXyYPyYP

Bayesian Metanetworks for Advanced Diagnostics

3-level Bayesian Metanetwork forManaging Feature Relevance

X

Y

A

BQ

RSX

Y

A

B

Q

RS

2 -lev e l B ay esian M etan e tw o rk fo rm o d e llin g re lev an t fea tu res’ se lec tio n

C o n te x tu a l le ve l

P re d ic tiv e le v e l

Two-level Bayesian Metanetwork formanaging conditional dependencies

X

Y

A

BQ

RS

X

Y

A

B

Q

RS

T w o -lev e l B ay esian M etan e tw o rk fo rm an ag in g co n d itio n a l d ep en d en c ies

C o n te x tu a l le ve l

P re d ic tiv e le v e l

Terziyan V., Vitko O., Probabilistic Metanetworks for Intelligent Data Analysis, Artificial Intelligence, Donetsk Institute of Artificial Intelligence, Vol. 3, 2002, pp. 188-197.

Terziyan V., Vitko O., Bayesian Metanetwork for Modelling User Preferences in Mobile Environment, In: German Conference on Artificial Intelligence (KI-2003), Hamburg, Germany, September 15-18, 2003.

Two-level Bayesian Metanetwork for managing conditional dependencies

Contextual level

Predictive level A

B

X

Y

P(B|A) P(Y|X)

Example of Bayesian Metanetwork

The nodes of the 2nd-level network correspond to the conditional probabilities of the 1st-level network P(B|A) and P(Y|X). The arc in the 2nd-level network corresponds to the conditional probability P(P(Y|X)|P(B|A))

X

Y

P(X)

P(Y)-?

P(P(Y|X)|P(B|A))

A

B

P(A)

pr(B|A)

P(P(B|A)) P(B|A) P(Y|X)

pk(Y|X)

P(P(Y|X))

))]}.|()|(())|()|((|)|()|(([

)()|({)(

ABpABPPXYpABPPXYpXYPP

xXPxXyYpyYP

rr

rk

i kiijkj

Other Cases of Bayesian Metanetwork (1)

P(A) P(X)

X

A

Contextual level

Predictive level

a)

P(P(X)|P(A))

A

pr(A)

P(P(A))

P(A) X

pk(X)

P(P(X))

P(X)

b)

Unconditional probability distributions associated with nodes of the predictive level network depend on probability distributions associated with nodes of the contextual level network


The metanetwork on the contextual level models conditional dependence particularly between unconditional and conditional probabilities of the predictive level

P(A) P(Y|X)

X A

Contextual level

Predictive level

Y

c)

X

Y

P(X)

P(Y)-? P(P(Y|X)|P(A))

A

pr(A)

P(P(A))

P(A) P(Y|X)

pk(Y|X)

P(P(Y|X))

d)

Contextual level

Predictive level

2-level RelevanceRelevance Bayesian Metanetwork (for modelling relevant features’ selection)

Simple Relevance Bayesian MetanetworkWe consider relevance as a probability of importance of the variable to the inference of target attribute in the given context. In such definition relevance inherits all properties of a probability.

X

Y

Probability

P(X)

P(Y)-? P(Y|X)

Relevance

Ψ(X)

X

Y

P(X)

P(Y|X)

Probability to have this model is:

P((X)=”yes”)= X

Y

P0(Y) Probability to have this model is:

P((X)=”no”)= 1-X

.)]1()([)|(1

)( X

XX XPnxXYPnx

YP

Example of 2-level Relevance Bayesian Metanetwork

In a relevance network the relevancies are considered as random variables between which the conditional dependencies can be learned.

X

Y

P(X)

P(Y)-?

P(Y|X) P(Ψ(X)|Ψ(A))

A

P(A) Ψ(A) Ψ(X)

)]}.1()()|()([)|({1

)( XAAXX A

PPXPnxXYPnx

YP

More Complicated Case of Managing Relevance (1)

X

Y

Probability

P(X)

P(Y)-?

P(Y|X,Z)

Relevance

Ψ(X)

Z

Probability

P(Z) Relevance

Ψ(Z)

X

Y

Probability

P(X)

P(Y|X,Z)

Z

Probability

P(Z)

Probability of this case is equal to:

P((X)=”yes”)×P((Z)=”yes”) = = X·Z

11

X

Y

Probability

P(X)

P(Y|X)


P((X)=”yes”)×P((Z)=”no”) = = X·(1-Z)

Y

P(Y|Z)

Z

Probability

P(Z)


P((X)=”no”)×P((Z)=”yes”) = = (1-X)·Z

Y

Probability

P0(Y)


P((X)=”no”)×P((Z)=”no”) = = (1-X)·(1-Z)

22 33 44

More Complicated Case of Managing Relevance (2)

X

Y

Probability

P(X)

P(Y)-?

P(Y|X,Z)

Relevance

Ψ(X)

Z

Probability

P(Z) Relevance

Ψ(Z)

,),|(1

)1()1(

)(),|(1

)1(

)(),|(1

)1(

)()(),|()(

1

1

1

1

nx

i

nz

ikkiZX

nx

i

nz

ikkkiZX

nx

i

nz

ikikiZX

nx

i

nz

ikkikiZX

zZxXYPnznx

zZPzZxXYPnx

xXPzZxXYPnz

zZPxXPzZxXYPYP

General Case of Managing Relevance (1)

X1

Y

Probability

P(X1)

P(Y)-?

P(Y|X1,X2,…,XN)

Relevance

Ψ(X1)

XN

Probability

P(XN) Relevance

Ψ(XN) X2

Probability

P(X2) Relevance

Ψ(X2)

…

Predictive attributes:

X1 with values {x11,x12,…,x1nx1};

X2 with values {x21,x22,…,x2nx2};

…XN with values {xn1,xn2,…,xnnxn};

Target attribute:

Y with values {y1,y2,…,yny}.

Probabilities:

P(X1), P(X2),…, P(XN);P(Y|X1,X2,…,XN).

Relevancies:X1 = P((X1) = “yes”);

X2 = P((X2) = “yes”);

…XN = P((XN) = “yes”);

Goal: to estimate P(Y).

General Case of Managing Relevance (2)

X1

Y

Probability

P(X1)

P(Y)-?

P(Y|X1,X2,…,XN)

Relevance

Ψ(X1)

XN

Probability

P(XN) Relevance

Ψ(XN) X2

Probability

P(X2) Relevance

Ψ(X2)

…

1 2 )"")(()"")((

1

])1()(),...2,1|([...1

)(X X XN noXqq

XqyesXrr

XrN

s

XrPnxrXNXXYPnxs

YP

Example of Relevance Metanetwork

X

Y

A

BQ

RS

a)

X

Y

A

B

Q

RS

b)c)

Relevance level

Predictive level

Combined Bayesian Metanetwork

In a combined Metanetwork two controlling

(contextual) levels will effect the basic level

Contextual level A

Predictive level

Contextual level B

Learning Bayesian Metanetworks from Data

Learning Bayesian Metanetwork structure (conditional, contextual and relevance (in)dependencies at each level);

Learning Bayesian Metanetwork parameters (conditional and unconditional probabilities and relevancies at each level).

Vitko O., Multilevel Probabilistic Networks for Modelling Complex Information Systems under Uncertainty, Ph.D. Thesis, Kharkov National University of Radioelectronics, June 2003. Supervisor: Terziyan V.

When Bayesian Metanetworks ?

1. Bayesian Metanetwork can be considered as very powerful tool in cases where structure (or strengths) of causal relationships between observed parameters of an object essentially depends on context (e.g. external environment parameters);

2. Also it can be considered as a useful model for such an object, which diagnosis depends on different set of observed parameters depending on the context.

Vagan Terziyan

Vladimir Ryabov

Temporal Diagnostics of Field Devices

• The approach to temporal diagnostics uses the algebra of uncertain temporal relations*.

• Uncertain temporal relations are formalized using probabilistic representation.

• Relational networks are composed of uncertain relations between some events (set of symptoms)

• A number of relational networks can be combined into a temporal scenario describing some particular course of events (diagnosis).

• In future, a newly composed relational network can be compared with existing temporal scenarios, and the probabilities of belonging to each particular scenario are derived.

* Ryabov V., Puuronen S., Terziyan V., Representation and Reasoning with Uncertain Temporal Relations, In: A. Kumar and I. Russel (Eds.), Proceedings of the Twelfth International Florida AI Research Society Conference - FLAIRS-99, AAAI Press, California, 1999, pp. 449-453.

Conceptual Schema for Temporal Diagnostics

N

S1 S2 … Sn

Temporal scenarios

1,SND2,SND

nSND ,

Recognition of temporal scenarios

• We estimate the probability of belonging of the particular relational network to known temporal scenarios.

Generating temporal scenarios

• We compose a temporal scenario combining a number of relational networks consisting of the same set of symptoms and possibly different temporal relations between them.

N1

N2

N3

N4N5

S

Terziyan V., Ryabov V., Abstract Diagnostics Based on Uncertain Temporal Scenarios, International Conference on Computational Intelligence for Modelling Control and Automation CIMCA’2003, Vienna, Austria, 12-14 February 2003, 6 pp.

Industrial Temporal Diagnostics (conceptual schema)

Industrial object

Temporal data

Relational network

DB ofscenarios

Estimation Recognition Diagnosis

Learning

Ryabov V., Terziyan V., Industrial Diagnostics Using Algebra of Uncertain Temporal Relations, IASTED International Conference on Artificial Intelligence and Applications, Innsbruck, Austria, 10-13 February 2003, 6 pp.

Event 2

< a1; a2; a3 > - imperfect temporal relation

between temporal points (Event 1 and Event 2):

P(event 1, before, event 2) = a1;

P(event 1, same time, event 2) = a2;

P(event 1, after, event 2) = a3.

Event 1

< a1; a2; a3 >

Imperfect Relation Between Temporal Point Events: Definition

Ryabov V., Handling Imperfect Temporal Relations, Ph.D. Thesis, University of Jyvaskyla, December 2002. Supervisors: Puuronen S., Terziyan V.

Example of Imperfect Relation

Event 2

< 0.5; 0.2; 0.3 > - imperfect temporal relation between temporal points:

P(event 1, before, event 2) = 0.5;

P(event 1, same time, event 2) = 0.2;

P(event 1, after, event 2) = 0.3.

Event 1

< 0.5; 0.2; 0.3 >

1

<= >

R(Event 1,Event 2)

Operations for Reasoning with Temporal Relations

rb,a = bar,~

ra,b

a b

ra,b rb,c

ra,c = ra,b rb,c

a

b

c

r r ra b a b a b, , , 1 2

r 1 a , br 2 a , b

a b

Inversion

Sum

Composition

Temporal Interval Relations

The basic interval relations are the thirteen Allen’s relations:

A before (b) B B after (bi) A

A meets (m) B B met-by (mi) A

A overlaps (o) B B overlapped-by (oi) A

A starts (s) B B started-by (si) A

A during (d) B B contains (di) A

A finishes (f) B B finished-by (fi) A

A equals (eq) B B equals A

A B

AB

AB

BA

AB

AB

BA

Imperfect Relation Between Temporal Intervals: Definition

interval 2

< a1; a2;… ; a13 > - imperfect temporal relation between

temporal intervals (interval 1 and interval 2):

P(interval 1, before, interval 2) = a1;

P(interval , meets, interval 2) = a2;

P(interval 1, overlaps, interval 2) = a3;

…

P(interval 1, equals, interval 2) = a13;

interval 1

< a1; a2 ;… ; a13 >

Industrial Temporal Diagnostics (composing a network of relations)

Sensor 3Sensor 2

Relational network representing the particular caseIndustrial object

Sensor 1

Estimation of temporal relations between

symptoms

Industrial Temporal Diagnostics (generating temporal scenarios)

N1

Scenario S

N3N2

Object A Object B Object C

Generating the temporal scenario

for “Failure X”DB of

scenarios

1. for i=1 to n do

2. for j=i+1 to n do

3. if (R1) or…or (Rk) then

4. begin

5. for g=1 to n do

6. if not (Rg) then Reasoning(, Rg)

7. // if “Reasoning” = False then (Rg)=TUR

8. ( R) = Å ( Rt), where t=1,..k

9. end

10. else go to line 2

Recognition of Temporal Scenario

m

ii

m

iii

w

dwD

1

1SN,

)Bal()Bal(,, , DC,BA,DCBA

RRd RR

12

0,

1

12

1

i

iei BABal(RA,B) =

Industrial object

Temporal data

Relational network

DB ofscenarios

Estimation Recognition Diagnosis

Learning

bm

ofi

disi eq

sd

foi

mi

bi

wbi =1

weq

=0.5

wb =0 wf =0.75

Balance point for RA,B

Balance point for RC,D

Probability value

When Temporal Diagnostics ?1. Temporal diagnostics considers not only a static set of symptoms, but

also the time during which they were monitored. This often allows having a broader view on the situation, and sometimes only considering temporal relations between different symptoms can give us a hint to precise diagnostics;

2. This approach might be useful for example in cases when appropriate causal relationships between events (symptoms) are not yet known and the only available for study are temporal relationships;

3. Combination of Bayesian (based on probabilistic causal knowledge) and Temporal Diagnostics would be quite powerful diagnostic tool.

Terziyan V., Dynamic Integration of Virtual Predictors, In: L.I. Kuncheva, F. Steimann, C. Haefke, M. Aladjem, V. Novak (Eds), Proceedings of the International ICSC Congress on Computational Intelligence: Methods and Applications - CIMA'2001, Bangor, Wales, UK, June 19 - 22, 2001, ICSC Academic Press, Canada/The Netherlands, pp. 463-469.

VaganTerziyan

The Problem

During the past several years, in a variety of application domains, researchers in machine learning, computational learning theory, pattern recognition and statistics have tried to combine

efforts to learn how to create and combine an ensemble of classifiers.

The primary goal of combining several classifiers is to obtain a more accurate prediction than can be obtained from any single classifier alone.

Approaches to Integrate Multiple Classifiers

Integrating Multiple Classifiers

Selection Combination

Global (Static)

Local (Dynamic)

Local (“Virtual” Classifier)

Global (Voting-Type)

Decontextualization

Inductive learning with integration of predictors

rrmrr yxxx ,...,, 21

Sample Instances

tmtt xxx ,...,, 21

yt

Learning Environment

P1 P2 ... Pn

Predictors/Classifiers

Virtual Classifier

TeamtionClassificasInstructorTeam

Members Team Elective

Members TeamConstant

, CLDE,FS,TITP,TM,TC,

TC - Team Collector

TM - Training Manager

TP - Team Predictor

TI - Team Integrator

FS - Feature Selector

DE - Distance Evaluator

CL - Classification Processor

Virtual Classifier is a group of seven cooperative agents:

Classification Team: Feature Selector




, CL DE, ,TI TP, TM, TC, FS

FS - Feature Selector

Feature Selector:

finds the minimally sized feature subset that is sufficient for correct classification of the instance

Fea

ture

Sel

ecto

r

Sample InstancesSample Instances

rr yΧrr

' ΧΧΧ ,'rr y

Classification Team: Distance Evaluator




, CL , FS,TI TP, TM, TC, DE

DE - Distance Evaluator

Distance between Two Instances with Heterogeneous Attributes (example)

YyXxi

iii

ii

yxdYXD,,

2),(),(

i

ii

ii

ii

range

yx

yxi

yxd

:else

otherwise ,1

if ,0 - nominal is attributeth if

),(

where:

d (“red”, “yellow”) = 1 d (15°, 25°) = 10°/((+50°)-(-50°)) = 0.1

Distance Evaluator:

measures distance between instances based on their numerical or nominal attribute values

Distance Evaluator

imii xxx ,...,, 21 jmjj xxx ,...,, 21

ijd

Classification Team: Classification Processor




, CL DE, FS,TI TP, TM, TC,

CL - Classification Processor

Classification Processor:

predicts class for a new instance based on its selected features and its location relatively to sample instances

Classification Processor

imii xxx ,...,, 21

iy

Sample Instances

Feature Selector

Distance Evaluator

Team Instructors:Team Collector




, CL DE, FS,TI TP, TM, TC,

TC - Team Collector completes Classification Teams for training

Team Collector

completes classification teams for future training

Team Collector FSi DEj CLk

Feature Selection methods

Distance Evaluation functions

Classification rules

Team Instructors:Training Manager




, CL DE, FS,TI TP, , TC, TM

TM - Training Manager trains allcompleted teams on sample instances

Training Manager

trains all completed teams on sample instances

Training Manager

FSi1 DEj1CLk1

FSi2 DEj2CLk2

FSin DEjnCLkn

rrmrr yxxx ,...,, 21

Sample Instances

rnrrrmrr wwwxxx ,...,,,...,, 2121

Sample Metadata

Classification Teams

Team Instructors:Team Predictor




, CL DE, FS,TI , TM, TC, TP

TP - Team Predictor predicts weights forevery classification team in certain location

Team Predictor

predicts weights for every classification team in certain location

Team Predictor:

e.g. WNN algorithm

rnrrrmrr wwwxxx ,...,,,...,, 2121

Sample Metadata

imii xxx ,...,, 21 inii www ,...,, 21

Predicted weightsof classification teamsLocation

Team Prediction:Locality assumption

Each team has certain subdomains in the space of instance attributes, where it is more reliable than the others;

This assumption is supported by the experiences, that classifiers usually work well not only in certain points of the domain space, but in certain subareas of the domain space [Quinlan, 1993];

If a team does not work well with the instances near a new instance, then it is quite probable that it will not work well with this new instance also.

Team Instructors:Team Integrator




, CL DE, FS, , TP TM, TC, TI

TI - Team Integrator produces classificationresult for a new instance by integratingappropriate outcomes of learned teams

Team integrator

produces classification result for a new instance by integrating appropriate outcomes of learned teams

Tea

m In

teg

rato

r

FSi1 DEj1CLk1

FSi2 DEj2CLk2

FSin DEjnCLkn

tmtt xxx ,...,, 21

New instance

tntt www ,...,, 21

yt1

yt2

yt1

yt

Weights of classification teamsin the location of a new instance

Classification teams

Static Selection of a Classifier

Static selection means that we try all teams on a sample set and for further classification select one, which achieved the best classification accuracy among others for the whole sample set. Thus we select a team only once and then use it to classify all new domain instances.

Dynamic Selection of a Classifier

Dynamic selection means that the team is being selected for every new instance separately depending on where this instance is located. If it has been predicted that certain team can better classify this new instance than other teams, then this team is used to classify this new instance. In such case we say that the new instance belongs to the “competence area” of that classification team.

Conclusion

Knowledge discovery with an ensemble of classifiers is known to be more accurate than with any classifier alone [e.g. Dietterich, 1997].

If a classifier somehow consists of certain feature selection algorithm, distance evaluation function and classification rule, then why not to consider these parts also as ensembles making a classifier itself more flexible?

We expect that classification teams completed from different feature selection, distance evaluation, and classification methods will be more accurate than any ensemble of known classifiers alone, and we focus our research and implementation on this assumption.

Yevgeniy Bodyanskiy

Volodymyr Kushnaryov

Online Stochastic Faults’ PredictionControl Systems Research Laboratory, AI Department, Kharkov National University of Radioelectronics. Head: Prof. E. Bodyanskiy. Carries out research on development of mathematical and algorithmic support of systems for control, diagnostics, forecasting and emulation:

1. Neural network architectures and real-time algorithms for observation and sensor data processing (smoothing, filtering, prediction) under substantial uncertainty conditions;

2. Neural networks in polyharmonic sequence analysis with unknown non-stationary parameters;

3. Analysis of chaotic time series; adaptive algorithms and neural network architectures for early fault detection and diagnostics of stochastic processes;

4. Adaptive multivariable predictive control algorithms for stochastic systems under various types of constraints;

5. Adaptive neuro-fuzzy control of non-stationary nonlinear systems;

6. Adaptive forecasting of non-stationary nonlinear time series by means of neuro-fuzzy networks;

7. Fast real-time adaptive learning procedures for various types of neural and neuro-fuzzy networks.

Bodyanskiy Y., Vorobyov S, Recurrent Neural Network Detecting Changes in the Properties of Non-Linear Stochastic Sequences, Automation and Remote Control, V. 1, No. 7, 2000, pp. 1113-1124.

Bodyanskiy Y., Vorobyov S., Cichocki A., Adaptive Noise Cancellation for Multi-Sensory Signals, Fluctuation and Noise Letters, V. 1, No. 1, 2001, pp. 12-23.

Bodyanskiy Y., Kolodyazhniy V., Stephan A. An Adaptive Learning Algorithm for a Neuro-Fuzzy Network, In: B. Reusch (ed.), Computational Intelligence. Theory and Applications, Berlin-Heidelberg-New York: Springer, 2001, pp. 68-75.

Existing Tools

Most existing (neuro-) fuzzy systems used for fault diagnosis or classification are based on offline learning with the use of genetic algorithms or modifications of the error back propagation. When the number of features and possible fault situations is large, tuning of the classifying system becomes very time consuming. Moreover, such systems perform very poorly in high dimensions of the input space, so special modifications of the known architectures are required.

Neuro-Fuzzy Fault Diagnostics

Successful application of the neuro-fuzzy synergism to fault diagnosis of complex systems demands development of an online diagnosing system that quickly learns from examples even with a large amount of data, and maintains high processing speed and high classification accuracy when the number of features is large as well.

Challenge: Growing (Learning) Probabilistic Neuro-Fuzzy Network (1)

input layer,n inputs

1-st hidden layer,N neurons

2-nd hidden layer,(m+1) elements

output layer,m divisors

Bodyanskiy Ye., Gorshkov Ye., Kolodyazhniy V., Wernstedt J., Probabilistic Neuro-Fuzzy Network with Non-Conventional Activation Functions, In: Knowledge-Based Intelligent Information & Engineering Systems, Proceedings of Seventh International Conference KES’2003, 3–5 September, Oxford, United Kingdom, LNAI, Springer-Verlag, 2003.

Bodyanskiy Ye., Gorshkov Ye., Kolodyazhniy V. Resource-Allocating Probabilistic Neuro-Fuzzy Network, In: Proceedings of International Conference on Fuzzy Logic and Technology, 10–12 September, Zittau, Germany, 2003.

Challenge: Growing (Learning) Probabilistic Neuro-Fuzzy Network (2)

Implements fuzzy reasoning and classification (fuzzy classification fuzzy classification networknetwork);

Creates automatically neurons based on training set (growing growing networknetwork);

Learns free parameters of the network based on training set (learning networklearning network);

Guarantees high precision of classification based on fast learning (high- performance networkhigh- performance network);

Able to perform with huge volumes of data with limited computational resources (powerful and economical networkpowerful and economical network);

Able to work in real-time (real-time networkreal-time network).

Tested on real data in comparison with classical probabilistic neural network

Unique combination of features

Tests for Neuro-Fuzzy Algorithms

Industrial Ontologies Group (Kharkov’s Branch), Data Mining Research Group and Control Systems Research Laboratory of the Artificial Intelligence Department of Kharkov National University of Radioelectronics have essential theoretical and practical experience in implementing neuro-fuzzy approach and specifically Real-Time Probabilistic Neuro-Fuzzy Systems for Simulation, Modeling, Forecasting, Diagnostics, Clustering, Control .

We are interested in cooperation with Metso in that area and we are ready to present the performance of our algorithms on real data taken from any of Metso’s products to compare our algorithms with existing in Metso algorithms.

Inventions we can offer (1)

Method of intelligent preventive or predictive diagnostics and forecasting of technical condition of industrial equipment, machines, devices, systems, etc. in real time based on analysis of non-stationary stochastic signals (e.g. from sensors of temperature, pressure, current, shifting, frequency, energy consumption, and other parameters with threshold values).

The method is based on advanced data mining techniques, which utilize fuzzy-neuro technologies, and differs from existing tools by flexible self-organizing network structure and by optimization of computational resources while learning.


Method of intelligent real-time preventive or predictive diagnostics and forecasting of technical condition of industrial equipment, machines, devices, systems, etc. based on analysis of signals with non-stationary and non-multiplied periodical components (e.g. from sensors of vibration, noise, frequencies of rotation, current, voltage, etc.).

The method is based on optimization of computational resources while learning because of intelligent reducing of the number of signal components being analyzed.


Method and mechanism of optimal control of dosage and real-time infusion of anti-wear oil additives into industrial machines based on its real-time condition monitoring.

Summary of problems we can solve

Rather global system for condition monitoring and preventive maintenance based on OntoServ.Net (global, agent-based, ontology-based, Semantic Web services-based, semantic P2P search-based) technologies, modern and advanced data-mining methods and tools with knowledge creation, warehousing, and updating during not only device’s lifetime, but also utilizing (for various maintenance needs) knowledge obtained afterwards (various testing and investigations techniques other than information taken from “living” device’s sensors) from broken-down, worn out or aged components of the same type.

Recently Performed Case Studies (1)

Ontology Development for Gas Compressing Equipment Diagnostics Realized by Neural Networks

Available in: http://www.cs.jyu.fi/ai/OntoGroup/docs/July2003.pdf

VolodymyrKushnaryov

SemenSimkin

1212

NN and Ontology using for DiagnosticNN and Ontology using for Diagnostic

SENSOR

SIGNAL

Neural NetworkDiagnostic out

Training

Diagnosing

1515

The creating ontology classes The creating ontology classes instance programinstance program

The subclasses and their slots forming and The subclasses and their slots forming and instances filling by the information is instances filling by the information is carried out automatically with the program carried out automatically with the program on Java. The filling occurs from RDBMS on Java. The filling occurs from RDBMS Oracle, which contains in the Oracle, which contains in the actualizedactualizedbase using in ”base using in ”UkrTransGasUkrTransGas”.”.

OracleJava

Program Ontology

Recently Performed Case Studies (2)

The use of Ontologies for Faults and State Description of Gas-Transfer Units

Available in: http://www.cs.jyu.fi/ai/OntoGroup/docs/July2003.pdf

Agent

SCADA

Agent

SCADA

SCADA SCADA

Diagnosist Diagnosist

GTUGTU GTUGTU

Ontologyfor agent communication

VolodymyrKushnaryov

KonstantinTatarnikov

GTU-

MAINTENANCE

GTU

Control-type

Subsystem

GTU-State

Support-History

Period

Signal-Types

Repair-Reason

PARAMETER

ACTIONS

Shutdown

Launch

REPAIRMid-life Repair

Major Repair

Current Repair Planned Repair

GTU-Node

Compressorstation

SITUATIONS

Oil-temperaturedeviation

Axle-shear

Vibration

Rise-of-temperature

AnalogSignal

ComputeVariable

Trend

Conclusion

Industrial Ontologies Research GroupIndustrial Ontologies Research Group (University of Jyvaskyla), which is piloting the OntoServ.NetOntoServ.Net concept of the Global Semantic Web - Based System for Industrial Maintenance, has also powerful branches in Kharkovbranches in Kharkov (e.g. IOG-Kharkov’s Branch, Control Systems Research Laboratory, Data Mining Research Group, etc.) with experts and experiencesexperts and experiences in various and challenging data mining and knowledge discovery, online diagnostics, forecasting and control, models learning and integration, etc. methods, which can be and reasonable to be successfully utilized within going-on cooperation between MetsoMetso and Industrial Ontologies Group.

Advanced Diagnostics Algorithms in Online Field Device Monitoring Vagan Terziyan (editor) ...

Documents

Transcript of Advanced Diagnostics Algorithms in Online Field Device Monitoring Vagan Terziyan (editor) ...