Demaq: A Foundation for Declarative XML Message Processing fileCarl-Christian Kanne, January 8, 2007...
Transcript of Demaq: A Foundation for Declarative XML Message Processing fileCarl-Christian Kanne, January 8, 2007...
Carl-Christian Kanne, January 8, 2007 Demaq - p. 1/20
Demaq: A Foundation for Declarative XMLMessage Processing
Alexander Böhm Carl-Christian Kanne
Guido MoerkotteUniversity of Mannheim
Introduction
● XML Messaging
● Networks of XML Queues
● Messaging Rules
● ∞-tier Architectures
● State of the Art
Demaq
Appendix
Carl-Christian Kanne, January 8, 2007 Demaq - p. 2/20
XML Messaging
XML
XML
XML
XML
XML
XML
XML
XML
XML
XML
XML
Manufacturer
Supplier C
Supplier B
Supplier A
Customer
Customer
SOA, Web Services, AJAX, RSS/Atom...
Introduction
● XML Messaging
● Networks of XML Queues
● Messaging Rules
● ∞-tier Architectures
● State of the Art
Demaq
Appendix
Carl-Christian Kanne, January 8, 2007 Demaq - p. 3/20
Networks of XML Queues
XML
XML
XML
XML
XML
XML
XML
XML
XML
XML
XML
Introduction
● XML Messaging
● Networks of XML Queues
● Messaging Rules
● ∞-tier Architectures
● State of the Art
Demaq
Appendix
Carl-Christian Kanne, January 8, 2007 Demaq - p. 4/20
Messaging Rules
■ "If a request for an offer comes in, forward it to thelegal, finance, and planning departments"
■ "If the delivery of all items has been confirmed,send a completion message to the customer"
Introduction
● XML Messaging
● Networks of XML Queues
● Messaging Rules
● ∞-tier Architectures
● State of the Art
Demaq
Appendix
Carl-Christian Kanne, January 8, 2007 Demaq - p. 5/20
∞-tier Architectures
XML
XML
JavaJava
C#Java
C#
Network
Messaging System
DBMS
App Code
App Server
TP Monitor
Introduction
● XML Messaging
● Networks of XML Queues
● Messaging Rules
● ∞-tier Architectures
● State of the Art
Demaq
Appendix
Carl-Christian Kanne, January 8, 2007 Demaq - p. 6/20
State of the Art
t o p i c = ( ( QIjmsSession ) t_sess ) . getTopic ( " strmadmin " , " oe_queue " ) ;
t_pub = t_sess . c rea tePub l i she r ( t o p i c ) ;
db_conn = ( ( QIjmsSession ) t_sess ) . getDBConnection ( ) ;
agent = new QIjmsAgent ( " e x p l i c i t _ e n q " , n u l l ) ;
adt_msg = ( ( QIjmsSession ) t_sess ) . createAdtMessage ( ) ;
l c r _ d a t a = new S t r i n g B u f f e r ( ) ;
l c r _ d a t a . append (" <ROW_LCR " ) ;
l c r _ d a t a . append ( " xmlns= ’ h t t p : / / xmlns . t e n t a c l e . com/ streams / schemas / l c r ’ " ) ;
l c r _ d a t a . append ( " xmlns : xs i = ’ h t t p : / / www. w3 . org /2001/XMLSchema−instance ’ " ) ;
l c r _ d a t a . append ( " xs i : schemaLocation = ’ h t t p : / / xmlns . t e n t a c l e . com / streams / schemas / l c r " ) ;
l c r _ d a t a . append ( " h t t p : / / xmlns . t e n t a c l e . com / streams / schemas / l c r / s t reams lc r . xsd ’ > " ) ;
l c r _ d a t a . append (" < source_database_name>source_dbname</ source_database_name > " ) ;
. . . MORE DOCUMENT CONSTRUCTION HERE . . .
xm l_ lc r = t e n t a c l e . xdb . XMLType . createXML ( db_conn , l c r _d a t a . t o S t r i n g ( ) ) ;
adt_msg . setAdtPayload ( xm l_ lc r ) ;
( ( QIjmsMessage) adt_msg ) . setSenderID ( agent ) ;
System . out . p r i n t l n ( " Pub l ish message 3 − XMLType con ta in ing LCR ROW" ) ;
r e c i p L i s t = new QIjmsAgent [ 1 ] ;
r e c i p L i s t [ 0 ] = new QIjmsAgent ( " e x p l i c i t _ d q " , n u l l ) ;
( ( QI jmsTopicPubl isher ) t_pub ) . pub l i sh ( top ic , adt_msg , r e c i p L i s t ) ;
t_sess . commit ( ) ;
Introduction
Demaq
● Demaq Application
● Demaq Language(s)
● Demaq QML Rules
● Demaq Sample Rule
● Demaq Server
● Messages all the way
● Demaq Project
● Thank you
Appendix
Carl-Christian Kanne, January 8, 2007 Demaq - p. 7/20
Demaq Application
NetworkQueue
Queue
Queue
Queue
Queue
Gateway
Gateway
Demaq Application
■ Complete■ Declarative■ Executable
Introduction
Demaq
● Demaq Application
● Demaq Language(s)
● Demaq QML Rules
● Demaq Sample Rule
● Demaq Server
● Messages all the way
● Demaq Project
● Thank you
Appendix
Carl-Christian Kanne, January 8, 2007 Demaq - p. 8/20
Demaq Language(s)
Network
Gateway
GatewayQueue
Queue
QueueQueue
Queue
Queue
Queue
Queue
Queue
Queue
Gateway
Gateway
Demaq Application
Application Rules (QML)
Message Queues (QDL)
Demaq Language
Introduction
Demaq
● Demaq Application
● Demaq Language(s)
● Demaq QML Rules
● Demaq Sample Rule
● Demaq Server
● Messages all the way
● Demaq Project
● Thank you
Appendix
Carl-Christian Kanne, January 8, 2007 Demaq - p. 9/20
Demaq QML Rules
■ "If the delivery of all items has been confirmed,send a completion message to the customer"
Introduction
Demaq
● Demaq Application
● Demaq Language(s)
● Demaq QML Rules
● Demaq Sample Rule
● Demaq Server
● Messages all the way
● Demaq Project
● Thank you
Appendix
Carl-Christian Kanne, January 8, 2007 Demaq - p. 9/20
Demaq QML Rules
■ "If the delivery of all items has been confirmed,send a completion message to the customer"
XML messages −→ new XML messages
Introduction
Demaq
● Demaq Application
● Demaq Language(s)
● Demaq QML Rules
● Demaq Sample Rule
● Demaq Server
● Messages all the way
● Demaq Project
● Thank you
Appendix
Carl-Christian Kanne, January 8, 2007 Demaq - p. 9/20
Demaq QML Rules
■ "If the delivery of all items has been confirmed,send a completion message to the customer"
XQuery Update Facility+ Queuing Primitives
XML messages −→ new XML messages
Introduction
Demaq
● Demaq Application
● Demaq Language(s)
● Demaq QML Rules
● Demaq Sample Rule
● Demaq Server
● Messages all the way
● Demaq Project
● Thank you
Appendix
Carl-Christian Kanne, January 8, 2007 Demaq - p. 10/20
Demaq Sample Rule
create rule sendComplete for orderMsgs
i f ( / / del iverymsg / @type = " conf i rm " ) then
l e t $ordered := fn : count ( qs : s l ice ( ) / / ordermsg / / i tem )
l e t $de l i ve red := fn : count ( qs : s l ice ( ) / / del iverymsg / i tem )
where $ordered eq $de l i ve red
return do enqueue <done> { / / order ID } </done>
in to customerReply
Introduction
Demaq
● Demaq Application
● Demaq Language(s)
● Demaq QML Rules
● Demaq Sample Rule
● Demaq Server
● Messages all the way
● Demaq Project
● Thank you
Appendix
Carl-Christian Kanne, January 8, 2007 Demaq - p. 11/20
Demaq Server
Network
Gateway
GatewayQueue
Queue
QueueQueue
Queue
Queue
Queue
Queue
Queue
Queue
Gateway
Gateway
Demaq Application
Application Rules (QML)
Message Queues (QDL)
Transport
Engine
Demaq Server
Rule
Transactional
Store
Demaq Language
XML
Introduction
Demaq
● Demaq Application
● Demaq Language(s)
● Demaq QML Rules
● Demaq Sample Rule
● Demaq Server
● Messages all the way
● Demaq Project
● Thank you
Appendix
Carl-Christian Kanne, January 8, 2007 Demaq - p. 12/20
Messages all the way
■ Everything is an XML message◆ Rule Input◆ Rule Output◆ Errors◆ Timeouts
■ Messages are processed once, but kept "forever"■ Message History
◆ captures process state◆ organized into slices (virtual queues)◆ declarative expiration
Introduction
Demaq
● Demaq Application
● Demaq Language(s)
● Demaq QML Rules
● Demaq Sample Rule
● Demaq Server
● Messages all the way
● Demaq Project
● Thank you
Appendix
Carl-Christian Kanne, January 8, 2007 Demaq - p. 13/20
Demaq Project
Network
Gateway
GatewayQueue
Queue
QueueQueue
Queue
Queue
Queue
Queue
Queue
Queue
Gateway
Gateway
Demaq Application
Application Rules (QML)
Message Queues (QDL)
QEP
QEP
QEP Transport
Engine
Demaq Language Demaq Compiler Demaq Server
Rule
Transactional
Store
XML
Introduction
Demaq
● Demaq Application
● Demaq Language(s)
● Demaq QML Rules
● Demaq Sample Rule
● Demaq Server
● Messages all the way
● Demaq Project
● Thank you
Appendix
Carl-Christian Kanne, January 8, 2007 Demaq - p. 14/20
Thank you
http://demaq.net
http://db.informatik.uni-mannheim.de
Introduction
Demaq
Appendix
● Slices
● Slice usage
● Error Handling
● A Demaq Rule
● Work in progress
● Demaq Goals
Carl-Christian Kanne, January 8, 2007 Demaq - p. 15/20
Slices
Orders Queue
...
...
Queues
Slices
Slice for customer 23
...
Slice for customer 42
... ...
...
Delivery Notifications Queue
Requests Queue
23 47
742
9 42
23 23
424223232323
234715
create property customerID f ixed
queue requests , orders ,
d e l i v e r y N o t i f i c a t i o n s value / / customerID
create s l ic ing customers on customerID
Introduction
Demaq
Appendix
● Slices
● Slice usage
● Error Handling
● A Demaq Rule
● Work in progress
● Demaq Goals
Carl-Christian Kanne, January 8, 2007 Demaq - p. 16/20
Slice usage
■ Merge parallel control flow
Rating
Check SetupCosts
Check PlantCapacity
Check ExportRestrictions
Check Credit
Send/RefuseOffer
create property c o r r e l a t i o n I D f ixed
queue credi tCheck , exportCheck ,
plantCheck , setupCheck value / / c o r r e l a t i o n I D
create s l ic ing checkResults on c o r r e l a t i o n I D
create rule merge for checkResults
i f ( count ( qs : s l ice ( ) ) eq 4) then . . .
Introduction
Demaq
Appendix
● Slices
● Slice usage
● Error Handling
● A Demaq Rule
● Work in progress
● Demaq Goals
Carl-Christian Kanne, January 8, 2007 Demaq - p. 17/20
Error Handling
■ Plenty of error sources in distributed applications◆ Application-related (dynamic errors)◆ Message-related (invalid XML, wrong schema)◆ Network-related (disconnects, routing, . . . )◆ . . .
■ Message-based error handling■ Error queues, e.g. for rules
create rule errorSource for foo errorqueue e r r o r s
Introduction
Demaq
Appendix
● Slices
● Slice usage
● Error Handling
● A Demaq Rule
● Work in progress
● Demaq Goals
Carl-Christian Kanne, January 8, 2007 Demaq - p. 18/20
A Demaq Rule
create rule newOfferRequest for customerMsgs
i f ( / / o f ferRequest ) then
l e t $customerInfo :=
<requestCustomerInfo reqID = " { / / request ID }" >
<customer > { / / customerID } < / customer>
</ requestCustomerInfo >
l e t $ e x p o r t R e s t r i c t i o n I n f o := . . .
l e t $p lan tCapac i t y In fo := . . .
return do enqueue $customerInfo in to f inance ,
do enqueue $ e x p o r t R e s t r i c t i o n s I n f o in to l ega l ,
do enqueue $p lan tCapac i t y In fo in to su p p l i e r
Introduction
Demaq
Appendix
● Slices
● Slice usage
● Error Handling
● A Demaq Rule
● Work in progress
● Demaq Goals
Carl-Christian Kanne, January 8, 2007 Demaq - p. 19/20
Work in progress
■ Optimization across rules■ Optimization/verification across sites■ Template Folding [XIMEP06]■ Rules driven by XML Schema validation
Introduction
Demaq
Appendix
● Slices
● Slice usage
● Error Handling
● A Demaq Rule
● Work in progress
● Demaq Goals
Carl-Christian Kanne, January 8, 2007 Demaq - p. 20/20
Demaq Goals
■ Declarative XML message processing language◆ Move work from programmer to system◆ Data independence◆ Optimizable
■ Execution Engine◆ Reliability◆ Scalability◆ Reuse DB system knowledge