1

DocFlow - kick off - 2007 - Monitoring 1

Distributed Monitoring in P2P Systems

Serge Abiteboul, Bogdan MarinoiuINRIA-Futurs and Univ. Paris 11

2

DocFlow - kick off - 2007 - Monitoring 2

Organization

Motivation

An algebra over XML streams

The P2PMonitor system

Optimization

Conclusion

3

Motivation

4

DocFlow - kick off - 2007 - Monitoring 4

P2P systems

Peer-to-peer: A large and varying number of computers cooperate to solve some particular task without any centralized authority

seti@home; kazaa; cabal

Interactivity between the peers (Web 2.0)

Content sharing communities

Distributed applications (e.g., games)

New services

5

DocFlow - kick off - 2007 - Monitoring 5

P2P systems

Benefit from technologies like XML and Web services

Complex and hard to control systems• Many peers

• Peers are distributed

• Peers are autonomous

• Peers are often unreliable and selfish

• Peers come and leave

• Intense communications between peers

• Content changes rapidly

Goal: monitor such systems

6

DocFlow - kick off - 2007 - Monitoring 6

Why monitor P2P systems?

P2P Monitoring, e.g., supply chain or manufacturing system

Web Intelligence: e.g., business intelligence and surveillance of competing companies

P2P Security: e.g., against intrusion, spamming

P2P optimization and tuning: e.g., gather statistics, control indexing

P2P system administration: e.g., choose where to log or control replication

P2P error management: e.g., error detection, error, diagnosis

Business: e.g., billing for using resources, procurement

7

An algebra over XML streams

8

DocFlow - kick off - 2007 - Monitoring 8

XM

L

strea

m

Monitoring one peer - Alerters

Detect events at the peer level

Data changes• Database updates• Events in RSS feed• Web page changes

Web service calls• Out call• In call

Each event is represented as an XML document

XML Stream

9

DocFlow - kick off - 2007 - Monitoring 9

An algebra over streams

Efficient Filter (can provide many selections simultaneously)

Binary operators: union, join, etc.

Operator with memory: duplicate elimination

Same plans meaningful for query processing or monitoring

= outer join & = s=“Lhasa”

s1 s2 s3 s4

s5

10

DocFlow - kick off - 2007 - Monitoring 10

ActiveXML algebra in short

Used both for query and monitoring plans

Local computation - Algebra for queries over trees:• Pick your favorite

• Has to work on streams

Manage replication• Generic data and services

Glue• Send, receive, eval

Essential• Include service calls – typically query calls

• Exchange streams of ActiveXML Algebraic Expressions

11

DocFlow - kick off - 2007 - Monitoring 11

AX

ML

strea

mAXML stream operator

r1

t s

r1

t s

r1

t s

rr1@p1Music@p3Music@p2

a

Peer p1

Q

“Easy case”Monotone in-streamMonotone query

Stream of ActiveXML

12

P2P Monitor system

13

DocFlow - kick off - 2007 - Monitoring 13

Stream specification language

for $c on local: outCOM

let $timeCall := $c.call.time

and $duration := $c.response.time - $timeCall

where

$c.call.method = "GetTemp“and $duration > 300

and $c.call.site = "http://meteofrance.fr"

return <longGetTemp>

<callTime>{$timeCall}</callTime>

<duration>{$duration}</duration>

</longGetTemp>

14

DocFlow - kick off - 2007 - Monitoring 14

Compiled into a monitoring plan

Compile the specification into a monitoring plan• Some alerters produce input streams

• Stream operators

Evaluate it

The monitoring plan is distributed

15

DocFlow - kick off - 2007 - Monitoring 15

What to do with the result streams

Publish a stream• RSS, email, web site

Turn a stream into a channel• Users can subscribe to it

Store stream – distributed tracing• Provide traces of the computation

• Basis for querying a run

• Basis for analyzing a run: diagnosis

16

DocFlow - kick off - 2007 - Monitoring 16

Example: monitoring involving many peers

For $com1 on http://peer.a.com :outCOM | http://peer.b.com :outCOM,

$com2 on http://meteo.fr : inCOM

Let $duration := $com1.response.timestamp - $com1.call.timestamp

Where

$duration > 10 and

$com1.call.method = « getTemp »  and Filter

$com1.call.site = http://meteo.fr and

$com1.call.id = $com2.call.id Join

Return Report($com1, $com2)

By email to service@meteo.fr;  

A

B

M

17

DocFlow - kick off - 2007 - Monitoring 17

Monitoring Task Global Plan

alerter@A

Filter

alerter@B

Filter

∪

Join

alerter@M

Reporter

Publisher

18

DocFlow - kick off - 2007 - Monitoring 18

Distributed Monitoring Task

alerter@A

Filter@Aalerter@B

Filter@B

∪@B

Join@M

alerter@M

Reporter@M

Publisher@M

Publisher@A

Publisher@B

Peer A:For $c on local:outCOMLet $duration = … Where .Return $cBy publish as channel X

Peer B:For $c on local:outCOM | XLet $duration = … Where ….Return $cBy publish as channel Y

Peer M:For $c1 on local:inCOM , $c2 on YWhere $c1.call.id = $c2.call.id Return Report($c1, $c2)By email to service@meteo.fr

Y

X

19

Optimization

20

DocFlow - kick off - 2007 - Monitoring 20

Optimize the plan

Distribute the work among the peers

Try to place computation close to data if possible

Try to reduce redundancy

21

DocFlow - kick off - 2007 - Monitoring 21

= outer join & = s=“Lhasa”

r1[t,s] r2[s,at] r3[t,s] r4[t,s]

r1[t,s] r3[t,s] r4[t,s]

r1[t,s] r3[t,s] r4[t,s]

r5[t,s]

r5[t,s]

r5[t,s]

monitoring plan (a)

monitoring plan (b)

monitoring plan (c)

Optimization based on ActiveXML algebra

22

Conclusion

23

DocFlow - kick off - 2007 - Monitoring 23

The plan

streams

Specify monitoringGenerate data streams Store tracesQuery tracesMine traces (e.g., diagnosis)

24

DocFlow - kick off - 2007 - Monitoring 24

Merci

Merci