Data-centric Networking Through Adaptive Content-based Routing

Data-centric Networking Through Adaptive Content-based Routing

Hans-Arno Jacobsen

Bell University Laboratory Chair

Middleware Systems Research Group

University of Toronto

MIDDLEWARE SYSTEMSRESEARCH GROUP

http://www.padres.msrg.utoronto.ca

University of Oslo, February 2009


Copyright © 2008 - Hans-Arno Jacobsen University of Oslo, February 2009

Querying the Future



Amazon to Chapters to You ....

Monday, October 10th in Cyberspace

Your book “...”is available

at .... $10 off

Thursday, November 15th,in Toronto



elseelse

Business Process Example

Checkscore

Creditcheck

Creditcheck 2

Checkscore 2

Reject

Approve

Send toofficer

Store inDB

…

…

Loan Application Processing



Large-scale Business ProcessesVendorVendor

SaleSale

ManufactoryManufactory

FinanceFinance

Dispatch B

Out-stock B

Pick-up goods

Packaging

MarketingMarketing

Design

Out-stock B

Target price

Prototype

Out Take

Control

Assign

Confirm

Determinateplan

Check stock

Rawmaterials

Audit

Raw

Determinateplan

Executeplan

Processcontrol

Monitor

Process

Pay

Check

Signature

Print receipt

WarehouseWarehouse

Delivery

FedEx

Pick up

Monitoring

Statistic

Chart

StrategyStrategy DesignDesign MarketingMarketing OrderOrderManufactoryManufactory PaymentPayment

Requirementcollection

Feature selection

Goods selection

Confirmfeatures

Material

Make plan

Feedback

Check order

Fill order

Check dealer Check credit

Approval Approval

Validate

Affirm order

Sale prediction

SignContract

CCCadministrate

Goods delivery

Fill dispatch bill

Fill out-stock bill

Credit card



What is the Common Denominator?

Many applications are driven by asynchronous state transitions. Something happens, … an appropriate reaction is expected

and required.

Asynchronous state transitions represent events. A process is triggered, a request submitted, …

Many applications require event management and processing capabilities to run effectively.



In Terms of the Examples These applications are driven by events

Information matching the query is found and indexed Person walks by a bookstore Loan request is submitted online

Abstractly speaking events are disseminated and filtered against queries

events queries



What Event Processing Support is Required ?

De-coupling and loose coupling

Fine-grained event filtering

In-network event processing

Composite event detection

Event correlation



Many Applications are Event-based

A

B C

D

E

F

RFID and sensor networksService oriented architectures

Workflows, business processes and job scheduling

Supply chain and logistics

Event-Based

Light

CallbackRazor SKU

Transform

Fault

Temperature

Invoke Loan

Order

Delivered

In flightJob A done

Trigger


Copyright © 2008 - Hans-Arno Jacobsen

Agenda

What is the right abstractions? My point of view The PADRES project Some details & results




What Abstractions Do Not Work? Databases

Great for managing historic data But what about future data

Data streams Great for managing structured streams of tuples But what about un-structured, multi-typed, sporadic events

from many sources Rule-based expert systems

Great for inference and reasoning But what about managing large numbers of fined-grained

filters in distributed envrionments

Take this cum gran salis



What Abstractions Enable Event Processing?

The afore-mentioned points can best be addressed by The content-based publish/subscribe model Realized by content-based message routing

Events are conveyed as publications.

Event listening, filtering and correlating is based on content-based subscriptions managed by the pub/sub system.



Publish/Subscribe 101

Not all publish/subscribe is equal

Publish/Subscribe models and evolution Channel-based

OMG CORBA Event Service, … Topic-based

WS Notifications, OMG Data Dissemination Service … Type-based

OMG Data Dissemination Service (partially), … Content-based

The PADRES ESB (see below), … State-based Subject Spaces



Content-based Publish/Subscribe

Publisher Publisher

Subscriber Subscriber

Subscriptions

Publications

NotificationNotification

IBM=84

MSFT=27 INTC=19 JNJ=58ORCL=12

HON=24

AMGN=58

Stock marketsNYSE

NASDAQTSX

Subscriptions:IBM > 85ORCL < 10JNJ > 60

Broker(s)



The Content-based Pub/Sub Model

Language and data model Boolean functions over predicates Subscriptions are conjunctions of predicates Publications are sets of attribute-value pairs

Matching semantic A subscription matches if all its predicates match

Example Tree-structured data

Graph-structured data

Un-structured data

Regular languages

Relational model

Subscription XPath RDF Query Keywords Regular expressions

SQL

Publication XML RSS feeds Text, documents Sentences over some alphabet

DBs, i.e., tables



data tuples subscriptions

query publication

Query and subscription are very similar.Data tuples and publication are very similar.

However, the two problem statements are inverse.

That’s Like Data Base Querying !!

sets of tuples

Abo

ut p

ast

Abo

ut f

utur

e

sets of tuples



Content-based Message Routing

Event-BasedEvent-Based

DecoupledDecoupled

FlexibleFlexible

ResponsiveResponsive

Content RoutingContent Routing

DeclarativeDeclarative

A1

S2

P1 S1

P2S2

[class,=,stock],[symbol,=,YHOO],[price,>,20.0]

[class,=,stock],[symbol,=,MSFT],[price,>,50.0]

[class,=,stock],[symbol,=,YHOO]

[class,=,stock],[price,>,40.0]A2

S1

[class, stock],[symbol, YHOO],[price,25.0][class, stock],[symbol, YHOO],[price,45.0]

[class, stock],[symbol, MSFT],[price, 55.0]




Publication Space

weight

height

Adv: [height > 70],[weight > 25]Sub: [height > 75],[weight > 20]

7075

Sub intersecting Adv

20 25

weight

height

Sub: [height > 75],[weight > 20]

70

90

Pub matching Sub

20 32

Pub: [height , 90],[weight , 32]




ToPSS - The Toronto Publish/Subscribe System Family [2000 – present]

Matching algorithms Language expressiveness

vs. efficient matching

Routing protocols Network architectures &

scalability

Higher level abstractions Workflow execution Monitoring

S-ToPSS(semantic)

X-ToPSS(XML matching)

A-ToPSS(approximate)

persistent-ToPSS(subject spaces)

L-ToPSS(location-based)

ToPSS(matching)

M-ToPSS(mobile)

Ad hoc-ToPSS(ad hoc networking)

Federated-ToPSS(federation of ToPSS brokers)

Rb-ToPSS(rule-based)

P2P-ToPSS(peer-to-peer)

LB-ToPSS(load balancing)

FT-ToPSS(fault tolerance)

Historic-ToPSS(historic data)

CS-ToPSS(composite subs)

BPEL-ToPSS(BPEL execution)

JS-ToPSS(job scheduling)



PADRES Data-centric Event Bus

First generation of students, when I looked away Peng Alex David aRno Eli Serge

PADRES is Publish/subscribe Applied to Distributed Resource Scheduling

PAdres is Distributed REsource Scheduling

http://www.padres.msrg.utoronto.ca

Acknowledgements

http://padres.msrg.utoronto.ca



Copyright © 2008 - Hans-Arno Jacobsen University of Oslo, February 2009 23/4/2121

PADRES Architecture

Server Farm

Computers

ComputersDatabase

Laptops

Computers

Workstation

Laptops

DatabaseServer

ServerSwitch

Server

Switch

Computing, Storage, and Networking Resources

Content-based Routing (Publish/Subscribe)

Content-based RouterClients (publisher/subscriber)

PADRES ESB

Business Process Execution

Deploy Control UpdateVisualize

Monitor ...

6

43

7start halt

Business Process Management and Business Activity Monitoring

Redirectresume

addremove

Event Management Framework

Application Events

Business Events

Complex Events

Network and System Events



PADRES Event Bus

Consists of pub/sub message brokers Content-based publish/subscribe interface Content-based message routing Store-and-forward message queuing

Comprised of a federation of brokers deployed as overlay

Offers a slim client library for applications

Soon available under an open (source) license model and as Apache Poloka incubation project



B

B

B

S

S

P

B

P

B

input queue

output queue dest2

output queue dest3

dest1

dest2

dest3

P

S

= publisher

= subscriber

subscription dest

Matching Engine

Routing Table

+

temperature > 37 dest2

temperature > 40 dest3

Publications

temperature = 38temperature = 42temperature = 36

PADRES Event Broker



Queue HandlerQueue Handler

Pre

Pro

cess

or

Pos

t Pro

cess

or

For

war

der

PADRES Broker

MatcherInput Queue Output Queues

PRT

SRT

Pub

/Sub

Mes

sage

s Event Broker Architecture

If sub intersects Send to

[class,=,foo],[attr,>,17] B1

If pub matches Send to

[class,=,foo],[attr,>,20] B2

AdvSub

SubPub

Sub

Pub




Innovative PADRES Features

A

B C

D

E

F

CompositeEvents

HistoricAccess

Management

Robustness

LoadBalancing

Security



Limitations of Acyclic Overlays Sensitive to

Congestion Imbalanced workloads Broker failures Overlay changes

Broker

Publisher

Subscriber

P




PPPP

RobustRobust

Self-healingSelf-healing AdaptiveAdaptive

FlexibleFlexible

Publisher

Subscriber

Congested Link

General Overlay Network




Challenges with General Overlays Subscriptions route

in loops Brokers receive

duplicate subscriptions

Multiple copies of message maybe created

Same problem for publications

2

3 4

5

1

6

Adv 1

Adv 2

S X

S

S




Number of Redundant Messages

0

2000

4000

6000

8000

10000

12000

14000

16000

50 100 500

Number of Nodes

Num

ber

of D

ropp

ed M

essa

ges

Average Connect Degree = 5

Average Connect Degree = 10




Content-based Routing in General Overlays

Maintain the same interface to pub/sub clients

Develop content-based routing protocols for Advertisement Subscription Publication




Advertisement Routing

Each advertisement forms a spanning advertisement tree

Duplicate advertisements are discarded by brokers

Each advertisement is assigned a unique tree identifier (TID) e.g., A: [class,=,stock]……[TID,=,adv_msg_id]

SRT (Subscription Routing Table) A set of [advertisement, last hop] pairs




Subscription Routing I

Each subscription is augmented with a TID-predicate with a variable e.g., S: [class,=,stock] … [TID,=,$X]

The variable is bound to the TID of matching advertisements

PRT (Publication Routing Table) A set of

[subscription, { (TID, last hop of subscription), … } ] pairs




S: [class,=,stock],[name,=,*],[price,>,50], [TID,=,$Z]

At Broker 1:

Adv1: [class,=,stock],[name,=,IBM],

[price,>,60],[TID,=,Adv1]

Adv2: [class,=,stock],[name,=,HP],


S matching Adv1:

[class,=,stock],[name,=,*],


S matching Adv2:

[class,=,stock],[name,=,*],[price,>,50],

[TID,=,Adv2]

Subscription Routing II

2

3 4

5

1

6

Adv 1

Adv 2

S X

S




Publication Routing

Each publication is assigned the TID of its inducing advertisement e.g., P [class, stock]……[TID, adv_msg_id]

Publication routing protocols: Fixed TID routing: a publication is routed to

subscribers along its advertisement tree. Dynamic publication routing: a publication may

be routed to subscribers across branches of different advertisement trees.




Fixed TID Routing

2

3 4

5

1

6

Adv 1

Adv 2

XP

Sub

P

Property 1: No broker receives duplicate publication messages.




Dynamic Publication Routing Publication’s TID can be

changed in transit. ``Best`` path algorithms

Property 2: Changing a publication P’s TID while in transit will not change the set of subscribers notified of P.

2

3 4

5

1

6

Adv 1

Adv 2

X

Sub

P




Faster Matching with TIDs Subscriptions are augmented with TIDs only

once at the first broker. Other brokers can route the subscription

based on the TID alone. Similar argument applies to publication

routing.




Advantages

Simple and powerful concept Retain the publish/subscribe client interface Speed up subscription and publication

propagation Generate duplicated messages only at

advertisement level Build multiple subscription routing paths for

publications Route publications dynamically




Composite Subscription

AND

OR

S1 S2

OR

S3 S4

AND

S5

CS={ {S1 OR S2} AND {S3 OR S4} AND S5 }

A composite event is the constellation ofevents being detected by the compositesubscription.

S are atomic subscriptions. I.e., they are satisfied by a single, multi-attribute event.

Composite subscriptions (CS) are used for event correlation, in network filtering, and thedetection of composite events (complex event).

Applications: Business process management, Business activity monitoring



Topology-based CS Routing

2

3 4

5

1

6Adv 1

Adv 2

7

8

9

CS={ {S1 AND S2} AND S3 }

CS

S2

S3

S1

CS’

Adv 3




Adaptive CS Routing

CSs may be split according to potential publication traffic, bandwidth, latency etc.

2

1 3

Adv 1

Adv 2

CS={S1 AND S2}

2

1 3

Adv 1

Adv 2

CS={S1 AND S2}

(a) (b)




Adaptive CS Routing

2

3 4

5

1

6Adv 1

Adv 2

7

8

9

CS={{S1 AND S2} AND S3}CS

S2

S3

S1

Adv 3

5

CS’

CS’




Evaluation

32 overlay brokers, 20 publishers, 30 subscribers, initially

20 machine vs. PlanetLab Workload

http://research.msrg.utoronto.ca/Padres/DataSets Yahoo!Finance stock quote traces




Dense Topology




On PlanetLab




Increased Publication Rate




With Broker Failures




Composite Event Detection




Conclusions

The right abstraction for event processing is content-based publish/subscribe.

Event processing & publish/subscribe are interesting research areas.

ToPSS and PADRES explore many aspects of these areas.


http://padres.msrg.utoronto.ca



Acknowledgements Graduate students, visitors, and PDFs currently working on PADRES.

Alex Cheung Chen Chen Amer Farroukh Patrick Lee Guoli Li Bala Maniymaran Vinod Muthusamy Reza Sherafat Naweed Tajuddin Chunyang Ye Young Yoon

Partners from CA Serge Mankovskii & Kirk Wilson

Partners from IBM Phil Coultard & Allen Chan

Partners from Bell Bell Systems & Technology

Plus many PADRES alumniPlus many PADRES alumni



References The PADRES ESB project home

http://padres.msrg.utoronto.ca An eQoSystem for declarative distributed applications with SLAs

http://research.msrg.utoronto.ca/Eqosystem/ The Micro-ToPSS event processing middleware for sensor networks

http://microToPSS.msrg.utoronto.ca/ Mobile-ToPSS – publish/subscribe for mobile and location-based applications

http://research.msrg.utoronto.ca/Mobile/ ToPSS - the Toronto Publish/Subscribe System Family Portal

http://www.ToPSS.biz (coming soon ) Quantifying events in software to increase modularity & customization in C-

based systems and software-based product lines http://www.AspeCtC.net (ACC - the AspeCt-oriented C compiler)

The Middleware Systems Research Group http://www.msrg.utoronto.ca

My web site http://www.eecg.toronto.edu/~jacobsen

@ the University of Toronto

The DEBS Conferencehttp://www.debs.org

July 2009 at Vanderbilt U.

Everything events Portalhttp://www.event-based.org



Questions?

A D R E SP



Benefits of Content-based Publish/Subscribe

Simplifies IT development and maintenance by decoupling enterprise components

Supports sophisticated interactions among components using expressive subscription languages – going beyond the limits of topics

Allows fine-grained queries and event management

Achieves scalability with in-network filtering and processing



Faster Matching with TIDs

Subscriptions are augmented with TIDs only once at the first broker.

Other brokers can route the subscription based on the TID alone.

Similar argument applies to publication routing.

2

3 4

5

1

6

Adv 1

Adv 2

S X

SAdv1

SAdv2




Sparser Topology




Effect of Subscriber Distance

Distance Fixed(ms) Dynamic(ms) Improvement

6 Hops 47.202 47.568 -0.78%

10 Hops 64.477 52.895 17.96%

12 Hops 74.416 60.598 18.57%

Max Diff 57.65% 27.39%




More Publishers




Publication Burst




More Results

Faster matching 1926 publications cause 16997 times of matching

operations in the “Dense Topology” experiment. About 89% of the matching operations can be saved if we

apply the TID-based faster matching.

Overhead of dynamic publication routing Average CPU usage: 6.3% more than the Fixed routing Average memory usage: 8.9% more than the Fixed routing




CS Routing Traffic




Summary

Minimize redundant traffic induced by cycles Improve scalability and robustness of

pub/sub systems by offering routing path alternatives

Enable flexible composite subscription routing

Simplify solutions for failure recovery and load balancing




Cost Model Routing cost of CS

RCB(CS)) = Σi Tin*|P(CSBi)| + Σi Tmatching*|P(CSBi)| + Σi Tout_i*|P(CS)|

Selection factor (SF) SFA(S) = |σS P(A)| / |P(A)| e.g., SFA(a = v) = 1 / (max - min)

Subscription cardinality |P(S)| = Σi ri * SFAi (S)

|P(Sl)| + |P(Sr)| if op = or

|P(CS)| = min (|P(Sl)| , |P(Sr)|) if op = and

Reteinput

outputs




Fast Matching

1926 publications cause 16997 matching operations in the “Dense Topology” experiment.

89% of the matching operations can be saved with TID-based fast matching.


Data-centric Networking Through Adaptive Content-based Routing

Documents

Transcript of Data-centric Networking Through Adaptive Content-based Routing