Supporting Disconnected Operations in Publish/Subscribe Systems
description
Transcript of Supporting Disconnected Operations in Publish/Subscribe Systems
Supporting Disconnected Operations in Publish/Subscribe Systems
Vinod Muthusamy
Joint work withMilenko Petrovic, Ioana Burcea, H.-Arno Jacobsen, Eyal de Lara
University of Toronto
2004 IEEE International Conference on Mobile Data Management
Agenda• Part I – Publish/Subscribe
– Centralized/distributed models– Benefits– Applications
• Part II – Mobile Publish/Subscribe– Disconnected operation in distributed publish/subscribe
systems• Motivation and Problem• Solutions
– Evaluation• Factors• Results
• Conclusions
The Publish/Subscribe Model
Broker
Publisher Publisher
Subscriber Subscriber
Subscriptions
Publications
NotificationNotification
IBM=84
MSFT=27 INTC=19 JNJ=58ORCL=12
HON=24
AMGN=58
Stock marketsNYSE
NASDAQTSX
Subscriptions:IBM > 85ORCL < 10JNJ > 60
Distributed Publish/Subscribe
• Hierarchy of brokers [Siena, JEDI]
• Subscriptions to root
• Publications to root and multicast down to interested subscribers
Broker
Broker
stocks/ibm/*
stocks/*
Publisher
Broker
Broker
stocks/oracle
Broker Broker
Publish/Subscribe Benefits• Simple interface
• Decoupling of producers and consumers of data– Address
• Content-based routing• Anonymity
– Platform– Space– Time– Representation (semantic)
• Efficient data dissemination (scalability)– Push model– Multicast
Publish/Subscribe Applications
• News dissemination• Location based services• Workflow management• E-commerce• Auctions• Distributed gaming• Software updates delivery• Sensor networks
• Existing systems– Research: ToPSS, Siena, JEDI, Gryphon, Hermes– Industry: IBM, Precache, TIBCO, Talarian
Part II
• How does disconnected operation work in distributed publish/subscribe systems?
• Why is disconnected operation a problem?
• What are some solutions?
• How do these solutions perform?
Disconnected Operation: Motivation
• User’s laptop connected at work• Disconnect at the end of the day• Reconnect at home
• Events published while a subscriber is disconnected should be stored somewhere and replayed upon reconnection
Broker Broker
Broker
Subscriber
Disconnected Operation [JEDI]
• Subscriber receives publications• Subscriber disconnects • Broker stores publications• Subscriber connects• Transfer state to new broker &
replay old publications• New publications go directly to
subscriber Broker Broker
Broker
Publisher
Subscriber
t1
At Old Broker
t2
Disconnected At New Broker
t4t3
Receivenew events
Connect(movein)
Disconnect(moveout)
Disconnected Operation [JEDI]
• Subscriber receives publications• Subscriber disconnects • Broker stores publications• Subscriber connects• Transfer state to new broker &
replay old publications• New publications go directly to
subscriber Broker Broker
Broker
Publisher
Subscriber
t1
At Old Broker
t2
Disconnected At New Broker
t4t3
Receivenew events
Connect(movein)
Disconnect(moveout)
Disconnected Operation [JEDI]
• Subscriber receives publications• Subscriber disconnects • Broker stores publications• Subscriber connects• Transfer state to new broker &
replay old publications• New publications go directly to
subscriber Broker Broker
Broker
Publisher
Subscriber
moveout
t1
At Old Broker
t2
Disconnected At New Broker
t4t3
Receivenew events
Connect(movein)
Disconnect(moveout)
Disconnected Operation [JEDI]
• Subscriber receives publications• Subscriber disconnects • Broker stores publications• Subscriber connects• Transfer state to new broker &
replay old publications• New publications go directly to
subscriber Broker Broker
Broker
Publisher
Subscriber
t1
At Old Broker
t2
Disconnected At New Broker
t4t3
Receivenew events
Connect(movein)
Disconnect(moveout)
Disconnected Operation [JEDI]
• Subscriber receives publications• Subscriber disconnects • Broker stores publications• Subscriber connects• Transfer state to new broker &
replay old publications• New publications go directly to
subscriber Broker Broker
Broker
Publisher
movein
t1
At Old Broker
t2
Disconnected At New Broker
t4t3
Receivenew events
Connect(movein)
Disconnect(moveout)
Disconnected Operation [JEDI]
• Subscriber receives publications• Subscriber disconnects • Broker stores publications• Subscriber connects• Transfer state to new broker &
replay old publications• New publications go directly to
subscriber Broker Broker
Broker
Publisher
t1
At Old Broker
t2
Disconnected At New Broker
t4t3
Receivenew events
Connect(movein)
Disconnect(moveout)
Disconnected Operation [JEDI]
• Subscriber receives publications• Subscriber disconnects • Broker stores publications• Subscriber connects• Transfer state to new broker &
replay old publications• New publications go directly to
subscriber Broker Broker
Broker
Publisher
t1
At Old Broker
t2
Disconnected At New Broker
t4t3
Receivenew events
Connect(movein)
Disconnect(moveout)
Problem: Unicast State Transfer
Broker
Publish/subscribe is efficient [multicast]
State transfer is inefficient [unicast]
What’s the bandwidth overhead of unicast state transfer?
Broker Broker
BrokerBroker
Broker
State Transfer Optimizations:Prefetching
• Eagerly migrate state while the user is disconnected
• Exploits knowledge of future mobility patterns
• Shortens perceived length of disconnection to the system
• Less state to transfer• State transfer period hidden from user Broker Broker
Broker
moveout
t1
At Old Broker
t2
Disconnected At New Broker
t4t3
Receivenew events
Connect(movein)
tp
Disconnect(moveout)
State Transfer Optimizations:Logging
• Brokers cache recent publications
• Exploits locality
• Only partial state transfer needed if locality exists
• Overhead could be worse if no locality exists
• Requires cache space at brokers
Broker Broker
Broker
State Transfer Optimizations:Home-Broker
• Mobile client logically reconnects to “home” broker
• Eliminates state transfer overhead
• Increases perceived disconnection period– Unicast transfer even when
connected movein
Broker Broker
Broker
Evaluation:Factors Affecting Mobility
Network
Bandwidth and latency of links
Broker placement
Broker topology Number of
brokers
Mobility
Connection and disconnection periods
Predictive or repetitive mobility
Group mobility
Application
No. of publishers and subscribers
Publication rate Subscription
specificity Subscription
(interest) locality Message size
Evaluation: Setup• Simulation Environment
– NS2 network simulator– Implemented state transfer optimizations
• Parameters– Topology
• Metropolitan Area Network• 4 levels of degree 4 64 leaf brokers
– Subscribers: 200, 400, 600, 800– Publishers: 100– Locality: random, 30%, 60%, 90%
• Metric– Total message traffic– State transfer overhead (unicast/multicast)
• • •
64• • •
1
Commute Scenario• Simulate evening commute home
– Subscribers work downtown (20 inner brokers)– Start commute between 4:00 pm and 6:00 pm– 40% live in city center (30 inner brokers)
• Take 15 min to 45 min to commute– 60% live in outskirts (34 outer brokers)
• Take 45 min to 90 min to commute
• Few total disconnections• Long disconnection periods
Downtown
City center OutskirtsOutskirts
1 64
Commute Scenario:Average Overhead
• Standard: almost 100% overhead
• Logging: Slight improvement
• Prefetching: negligible overhead
• Home broker: poor due to sustained unicast
• Peak overhead results
– Up to 3x for Standard
– Up to 27x for Home-broker
Commute Scenario:Total Message Cost
• 800 subscribers
• Same relative performance
• Message cost tracks # of state transfers
• Home-broker overhead even after reconnection
• Max 10% concurrent state transfers
Random Scenario
• More ad-hoc mobility– Subscriber starts at random broker– Disconnects for 10 min to 30 min– Walk (5 km/h), city driving (50 km/h), highway
driving (100 km/h)– Reconnects to a random broker within range– Remains connected for 10 min to 30 min– (repeat)
• More disconnections• Shorter disconnection periods
Random Scenario:Average Overhead
• Standard: 100% overhead
• Prefetching overhead now noticeable
– Due to shorter disconnections
• Logging diverges more from Standard
– Due to hidden increase in locality with population
– Due to more disconnections
Random Scenario:Effect of Locality
• 800 subscribers
• Adjust subscription locality by varying % of subscribers with identical subscriptions
• Logging can approach “ideal” Prefetching
• Others’ increasing overhead is due to decreasing multicast, and constant unicast
Random Scenario:Effect of Log Size
• For Logging approach
• Log size 0, 400, 800, 1200 events
• Diminishing returns of increasing log size
Pervasive Scenario
• Persistent connectivity– Similar mobility patterns as Random scenario– Disconnections are few seconds long
• E.g. Cellular handoffs between cells
• Many disconnections• Short disconnection periods
Pervasive Scenario:Average Overhead
• Event migration is small portion of state transfer overhead
• Similar overhead (except home-broker)
– Due to very short disconnec-tions
Conclusions• The publish/subscribe model lends itself well to mobile
applications
• Mobility can break a distributed pub/sub system– Network overhead can double with only 10% concurrent movein’s– Stored events typically dominate overhead, not subscriptions– With sufficient locality, Logging approaches Prefetching
• Future Work– Other scenarios: realistic traces, mobile publishers– Investigate effects of state transfer traffic on latency– Develop more state transfer optimizations
• Multicast state transfer• Logging at intermediate brokers• Smarter cache replacement policies
Thank You
Pub/Sub Optimizations
Broker
Reduce subscription traffic
Broker
Covering
stocks/ibm/*stocks/*
stocks/*
Broker
Reduce publication traffic to the root
Broker
Advertisements
stocks/*
Publisher
Subscriptions
Advertisements
Publications
Broker Broker
Covering and Advertisements• Our scenarios did not benefit significantly
from covering and ads
• Forwarding of events dominate overhead – 98% of overhead in Commute and Random
scenarios– Covering only helps to quench subscriptions
• There is little subscriber locality (in our scenarios)– Most events must propagate to the root broker– Ads only help to quench upstream publications
Toronto Publish/Subscribe System (ToPSS) Research Areas
S-ToPSS(semantic)
X-ToPSS(semi-structured data; XML)
A-ToPSS(approximate)
M-ToPSS(mobile)
Ad hoc-ToPSS(ad hoc networking)
Federated-ToPSS(federation of ToPSS brokers)
persistent-ToPSS(Subject Spaces)
Rb-ToPSS(rule-based)
ToPSS(matching algorithms)
L-ToPSS(location-based correlation)
p2p-ToPSS(peer-to-peer)ToPSS
Information consumers subscribe to information of interest.Information producers publish information. ToPSS-broker(s) match and route relevant information to interested subscribers.