1 DocFlow - kick off - 2007 - Monitoring 1 Distributed Monitoring in P2P Systems Serge Abiteboul,...

1

DocFlow - kick off - 2007 - Monitoring 1

Distributed Monitoring in P2P Systems

Serge Abiteboul, Bogdan MarinoiuINRIA-Futurs and Univ. Paris 11

2


Organization

Motivation

An algebra over XML streams

The P2PMonitor system

Optimization

Conclusion

3

Motivation

4


P2P systems

Peer-to-peer: A large and varying number of computers cooperate to solve some particular task without any centralized authority

seti@home; kazaa; cabal

Interactivity between the peers (Web 2.0)

Content sharing communities

Distributed applications (e.g., games)

New services

5


P2P systems

Benefit from technologies like XML and Web services

Complex and hard to control systems• Many peers

• Peers are distributed

• Peers are autonomous

• Peers are often unreliable and selfish

• Peers come and leave

• Intense communications between peers

• Content changes rapidly

Goal: monitor such systems

6


Why monitor P2P systems?

P2P Monitoring, e.g., supply chain or manufacturing system

Web Intelligence: e.g., business intelligence and surveillance of competing companies

P2P Security: e.g., against intrusion, spamming

P2P optimization and tuning: e.g., gather statistics, control indexing

P2P system administration: e.g., choose where to log or control replication

P2P error management: e.g., error detection, error, diagnosis

Business: e.g., billing for using resources, procurement

7

An algebra over XML streams

8


XM

L

strea

m

Monitoring one peer - Alerters

Detect events at the peer level

Data changes• Database updates• Events in RSS feed• Web page changes

Web service calls• Out call• In call

Each event is represented as an XML document

XML Stream

9


An algebra over streams

Efficient Filter (can provide many selections simultaneously)

Binary operators: union, join, etc.

Operator with memory: duplicate elimination

Same plans meaningful for query processing or monitoring

= outer join & = s=“Lhasa”

s1 s2 s3 s4

s5

10


ActiveXML algebra in short

Used both for query and monitoring plans

Local computation - Algebra for queries over trees:• Pick your favorite

• Has to work on streams

Manage replication• Generic data and services

Glue• Send, receive, eval

Essential• Include service calls – typically query calls

• Exchange streams of ActiveXML Algebraic Expressions

11


AX

ML

strea

mAXML stream operator

r1

t s

r1

t s

r1

t s

rr1@p1Music@p3Music@p2

a

Peer p1

Q

“Easy case”Monotone in-streamMonotone query

Stream of ActiveXML

12

P2P Monitor system

13


Stream specification language

for $c on local: outCOM

let $timeCall := $c.call.time

and $duration := $c.response.time - $timeCall

where

$c.call.method = "GetTemp“and $duration > 300

and $c.call.site = "http://meteofrance.fr"

return <longGetTemp>

<callTime>{$timeCall}</callTime>

<duration>{$duration}</duration>

</longGetTemp>

14


Compiled into a monitoring plan

Compile the specification into a monitoring plan• Some alerters produce input streams

• Stream operators

Evaluate it

The monitoring plan is distributed

15


What to do with the result streams

Publish a stream• RSS, email, web site

Turn a stream into a channel• Users can subscribe to it

Store stream – distributed tracing• Provide traces of the computation

• Basis for querying a run

• Basis for analyzing a run: diagnosis

16


Example: monitoring involving many peers

For $com1 on http://peer.a.com :outCOM | http://peer.b.com :outCOM,

$com2 on http://meteo.fr : inCOM

Let $duration := $com1.response.timestamp - $com1.call.timestamp

Where

$duration > 10 and

$com1.call.method = « getTemp » and Filter

$com1.call.site = http://meteo.fr and

$com1.call.id = $com2.call.id Join

Return Report($com1, $com2)

By email to [email protected];

A

B

M

17


Monitoring Task Global Plan

alerter@A

Filter

alerter@B

Filter

∪

Join

alerter@M

Reporter

Publisher

18


Distributed Monitoring Task

alerter@A

Filter@Aalerter@B

Filter@B

∪@B

Join@M

alerter@M

Reporter@M

Publisher@M

Publisher@A

Publisher@B

Peer A:For $c on local:outCOMLet $duration = … Where .Return $cBy publish as channel X

Peer B:For $c on local:outCOM | XLet $duration = … Where ….Return $cBy publish as channel Y

Peer M:For $c1 on local:inCOM , $c2 on YWhere $c1.call.id = $c2.call.id Return Report($c1, $c2)By email to [email protected]

Y

X

19

Optimization

20


Optimize the plan

Distribute the work among the peers

Try to place computation close to data if possible

Try to reduce redundancy

21


= outer join & = s=“Lhasa”

r1[t,s] r2[s,at] r3[t,s] r4[t,s]

r1[t,s] r3[t,s] r4[t,s]

r1[t,s] r3[t,s] r4[t,s]

r5[t,s]

r5[t,s]

r5[t,s]

monitoring plan (a)

monitoring plan (b)

monitoring plan (c)

Optimization based on ActiveXML algebra

22

Conclusion

23


The plan

streams

Specify monitoringGenerate data streams Store tracesQuery tracesMine traces (e.g., diagnosis)

24


Merci

Merci

1 DocFlow - kick off - 2007 - Monitoring 1 Distributed Monitoring in P2P Systems Serge Abiteboul,...

Documents

Transcript of 1 DocFlow - kick off - 2007 - Monitoring 1 Distributed Monitoring in P2P Systems Serge Abiteboul,...