DCEP-Sim: An Open Simulation Framework for Distributed CEPComplex Event Processing - Ubi Sumus? Quo...
Transcript of DCEP-Sim: An Open Simulation Framework for Distributed CEPComplex Event Processing - Ubi Sumus? Quo...
![Page 1: DCEP-Sim: An Open Simulation Framework for Distributed CEPComplex Event Processing - Ubi Sumus? Quo Vadimus?, In Mobile Big Data: A Roadmap from Models to Technologies. Springer •](https://reader035.fdocuments.in/reader035/viewer/2022071509/612a034fee272872d013da63/html5/thumbnails/1.jpg)
DCEP-Sim: An Open Simulation Framework for Distributed CEPIntroduction for Users and Prospective Developers
Fabrice Starks Stein Kristiansen, Thomas Plagemann
![Page 2: DCEP-Sim: An Open Simulation Framework for Distributed CEPComplex Event Processing - Ubi Sumus? Quo Vadimus?, In Mobile Big Data: A Roadmap from Models to Technologies. Springer •](https://reader035.fdocuments.in/reader035/viewer/2022071509/612a034fee272872d013da63/html5/thumbnails/2.jpg)
Introduction and Motivation
• Data streams and information flow processing
– Financial tickers
– Traffic management
– Internet of Things
– eHealth
• Real-time processing:
– Data Stream Management Systems
– Complex Event Processing
26.06.2018 3
![Page 3: DCEP-Sim: An Open Simulation Framework for Distributed CEPComplex Event Processing - Ubi Sumus? Quo Vadimus?, In Mobile Big Data: A Roadmap from Models to Technologies. Springer •](https://reader035.fdocuments.in/reader035/viewer/2022071509/612a034fee272872d013da63/html5/thumbnails/3.jpg)
Distributed CEP
• CEP instances communicate via a network– End to end delay– Error rate
– Available bandwidth
• How deterministic are the network properties– Guaranteed QoS vs. best effort
– Private vs. public networks
– Static vs. mobile networks
26.06.2018 4
[Source: https://www.pcsteps.com/10751-mobile-internet-e-3g-h-plus-4g-mobile-network/]
![Page 4: DCEP-Sim: An Open Simulation Framework for Distributed CEPComplex Event Processing - Ubi Sumus? Quo Vadimus?, In Mobile Big Data: A Roadmap from Models to Technologies. Springer •](https://reader035.fdocuments.in/reader035/viewer/2022071509/612a034fee272872d013da63/html5/thumbnails/4.jpg)
Distributed CEP - Challenges
• Test and evaluate• Real world vs. emulation vs. simulation• What are realistic, representative network properties?
26.06.2018 5Network dynamics
Effort to doproper testingand evaluation
Data centerISP with QoS
Best effort Internet
3G 4G
VANETs
MANET, WSN
The more dynamics,the harder
![Page 5: DCEP-Sim: An Open Simulation Framework for Distributed CEPComplex Event Processing - Ubi Sumus? Quo Vadimus?, In Mobile Big Data: A Roadmap from Models to Technologies. Springer •](https://reader035.fdocuments.in/reader035/viewer/2022071509/612a034fee272872d013da63/html5/thumbnails/5.jpg)
Some insights from a recent survey• Starks, F., Plagemann, T., Goebel, V., Kristiansen, S. (2018). Mobile Distributed
Complex Event Processing - Ubi Sumus? Quo Vadimus?, In Mobile Big Data: A Roadmap from Models to Technologies. Springer
• 13 publications on mobile Distributed CEP with 19 evaluation reports
– 2 based on mathematical modeling– 3 based on PlanetLab experiments
– 3 based on emulation– 11 based on simulation
• 7 based on simulators created for the specific experiments
• 4 based on popular network simulators (J-Sim, OMNet and PeerSim)
• The missing consensus on evaluation approaches motivated our development of DCEP-Sim (presented at DEBS 2017)6/26/18 6
![Page 6: DCEP-Sim: An Open Simulation Framework for Distributed CEPComplex Event Processing - Ubi Sumus? Quo Vadimus?, In Mobile Big Data: A Roadmap from Models to Technologies. Springer •](https://reader035.fdocuments.in/reader035/viewer/2022071509/612a034fee272872d013da63/html5/thumbnails/6.jpg)
Aim of this tutorial
• For us:– Start an open source project with DCEP-Sim
• For you (assuming 3 types of attendees):– Explain what you could do with DCEP-Sim in your work– How to get started with DCEP-Sim– How to use DCEP-Sim in your research and contribute to the code
base
6/26/18 7
![Page 7: DCEP-Sim: An Open Simulation Framework for Distributed CEPComplex Event Processing - Ubi Sumus? Quo Vadimus?, In Mobile Big Data: A Roadmap from Models to Technologies. Springer •](https://reader035.fdocuments.in/reader035/viewer/2022071509/612a034fee272872d013da63/html5/thumbnails/7.jpg)
Disclaimer
• DCEP-Sim is – not a commercial product,– but an outcome of the PhD thesis from Fabrice Starks– and is now open to contributions from the community
• DCEP-Sim inherits strength and weaknesses of ns-3– many high quality network models– high flexibility– powerful tracing and data collection– efficient– software execution time is not considered26.06.2018 8
![Page 8: DCEP-Sim: An Open Simulation Framework for Distributed CEPComplex Event Processing - Ubi Sumus? Quo Vadimus?, In Mobile Big Data: A Roadmap from Models to Technologies. Springer •](https://reader035.fdocuments.in/reader035/viewer/2022071509/612a034fee272872d013da63/html5/thumbnails/8.jpg)
26.06.2018 9
Outline
• Introduction and motivation• Concepts and architecture of the distributed CEP engine in DCEP-Sim
– Requirements
– Design principles
– CEP engine
– Placement
– Overall architecture
• Introduction to the network simulator ns-3– Principles of discrete event simulation
– ns-3 Overview
– Key ns-3 modeling and simulation concepts
– Fundamental ns-3 models
– ns-3 simulation via example
![Page 9: DCEP-Sim: An Open Simulation Framework for Distributed CEPComplex Event Processing - Ubi Sumus? Quo Vadimus?, In Mobile Big Data: A Roadmap from Models to Technologies. Springer •](https://reader035.fdocuments.in/reader035/viewer/2022071509/612a034fee272872d013da63/html5/thumbnails/9.jpg)
Outline (cont.)
• DCEP-Sim use and extensions– Overview code structure– How do I run DCEP-Sim & how works a «script»– Changing the workload– How are placement policies implemented -> adding new placement – How are operators implemented -> adding new operators
• Conclusions
• Hands-on if you want to install ns-3 and run DCEP-Sim on you Linux laptop
26.06.2018 10
![Page 10: DCEP-Sim: An Open Simulation Framework for Distributed CEPComplex Event Processing - Ubi Sumus? Quo Vadimus?, In Mobile Big Data: A Roadmap from Models to Technologies. Springer •](https://reader035.fdocuments.in/reader035/viewer/2022071509/612a034fee272872d013da63/html5/thumbnails/10.jpg)
Placement the Main Challenge of Distributed CEP
26.06.2018 11
Query: (A ∨ B) ∧ C
A∨ B∧C
Operator graph
C B
A
Network node
∧∨
Operator overlay
Event Broker
Application
![Page 11: DCEP-Sim: An Open Simulation Framework for Distributed CEPComplex Event Processing - Ubi Sumus? Quo Vadimus?, In Mobile Big Data: A Roadmap from Models to Technologies. Springer •](https://reader035.fdocuments.in/reader035/viewer/2022071509/612a034fee272872d013da63/html5/thumbnails/11.jpg)
Placement the Main Challenge of Distributed CEP
26.06.2018 12
C B
A
Network node
∧∨
Operator overlay
Event Broker
Application
Where to place the operators?
Network link properties & overlay link properties:Latency, available bandwidth, loss
Traffic properties:High event rate vs. low event rate from sourcesSelectivity of operators
Other concerns: node resources, constraints, security
![Page 12: DCEP-Sim: An Open Simulation Framework for Distributed CEPComplex Event Processing - Ubi Sumus? Quo Vadimus?, In Mobile Big Data: A Roadmap from Models to Technologies. Springer •](https://reader035.fdocuments.in/reader035/viewer/2022071509/612a034fee272872d013da63/html5/thumbnails/12.jpg)
Placement the Main Challenge of Distributed CEP
26.06.2018 13
C B
A
Network node
∧∨
Operator overlay
Event Broker
Application
What do you do if you have some cool new ideas for placement?
Model, design, implement
Test & implement – but how? → DCEPSim
![Page 13: DCEP-Sim: An Open Simulation Framework for Distributed CEPComplex Event Processing - Ubi Sumus? Quo Vadimus?, In Mobile Big Data: A Roadmap from Models to Technologies. Springer •](https://reader035.fdocuments.in/reader035/viewer/2022071509/612a034fee272872d013da63/html5/thumbnails/13.jpg)
DCEP-Sim Goals
• Tool for experimentation with Distributed CEP solutions• Realistic models of various network types and conditions• Ability to create arbitrary traffic patterns• Support CEP query and query processing concepts
– Operators, windows, selection policy, consumption policy– without the need to implement a »full CQL»
• Extensibility and flexibility• Easy to use
26.06.2018 14
![Page 14: DCEP-Sim: An Open Simulation Framework for Distributed CEPComplex Event Processing - Ubi Sumus? Quo Vadimus?, In Mobile Big Data: A Roadmap from Models to Technologies. Springer •](https://reader035.fdocuments.in/reader035/viewer/2022071509/612a034fee272872d013da63/html5/thumbnails/14.jpg)
Major Design Decisions
• Use the well established network simulator ns-3– Benefit from many years effort– Many existing models for link, network, transport level protocols,
++– High degree of realism– Tools for debugging, tracing, data collection, ++
• Simulation instead of emulation– Scalability
26.06.2018 15
![Page 15: DCEP-Sim: An Open Simulation Framework for Distributed CEPComplex Event Processing - Ubi Sumus? Quo Vadimus?, In Mobile Big Data: A Roadmap from Models to Technologies. Springer •](https://reader035.fdocuments.in/reader035/viewer/2022071509/612a034fee272872d013da63/html5/thumbnails/15.jpg)
Engineering Principles
• Separation of concerns
• Separation of mechanisms and policies
26.06.2018 16
![Page 16: DCEP-Sim: An Open Simulation Framework for Distributed CEPComplex Event Processing - Ubi Sumus? Quo Vadimus?, In Mobile Big Data: A Roadmap from Models to Technologies. Springer •](https://reader035.fdocuments.in/reader035/viewer/2022071509/612a034fee272872d013da63/html5/thumbnails/16.jpg)
Design & Implementation Approach
• Start:– Gianpaolo Cugola and Alessandro Margara. 2012. Processing
Flows of Information: From Data Stream to Complex Event Processing. ACM Computing Surveys 44, June 2012
• Apply the engineering principles to develop the architecture• Components & sub-components are good candidates to be
implemented as objects• Leverage the ns-3 features for the implementation of an
extensible and flexible solution
26.06.2018 17
![Page 17: DCEP-Sim: An Open Simulation Framework for Distributed CEPComplex Event Processing - Ubi Sumus? Quo Vadimus?, In Mobile Big Data: A Roadmap from Models to Technologies. Springer •](https://reader035.fdocuments.in/reader035/viewer/2022071509/612a034fee272872d013da63/html5/thumbnails/17.jpg)
Functional Architecture of an IFP System
26.06.2018 18[Cugola et al. 2012]
![Page 18: DCEP-Sim: An Open Simulation Framework for Distributed CEPComplex Event Processing - Ubi Sumus? Quo Vadimus?, In Mobile Big Data: A Roadmap from Models to Technologies. Springer •](https://reader035.fdocuments.in/reader035/viewer/2022071509/612a034fee272872d013da63/html5/thumbnails/18.jpg)
DCEP-Sim Components
26.06.2018 19
Communication
CEP Engine
Source Sink
DispatcherPlacement
![Page 19: DCEP-Sim: An Open Simulation Framework for Distributed CEPComplex Event Processing - Ubi Sumus? Quo Vadimus?, In Mobile Big Data: A Roadmap from Models to Technologies. Springer •](https://reader035.fdocuments.in/reader035/viewer/2022071509/612a034fee272872d013da63/html5/thumbnails/19.jpg)
CEP-Engine
26.06.2018
20
Detector Producer Forwarder
Dispatcher CommunicationPlacement
CEP Engine
Source Sink
![Page 20: DCEP-Sim: An Open Simulation Framework for Distributed CEPComplex Event Processing - Ubi Sumus? Quo Vadimus?, In Mobile Big Data: A Roadmap from Models to Technologies. Springer •](https://reader035.fdocuments.in/reader035/viewer/2022071509/612a034fee272872d013da63/html5/thumbnails/20.jpg)
DCEP-Sim Components
26.06.2018 21
Dispatcher CommunicationPlacement
CEP Engine
Source Sink
![Page 21: DCEP-Sim: An Open Simulation Framework for Distributed CEPComplex Event Processing - Ubi Sumus? Quo Vadimus?, In Mobile Big Data: A Roadmap from Models to Technologies. Springer •](https://reader035.fdocuments.in/reader035/viewer/2022071509/612a034fee272872d013da63/html5/thumbnails/21.jpg)
Forwarder vs. Communication
26.06.2018
22
Detector Producer Forwarder
Dispatcher CommunicationPlacement
CEP Engine
Source Sink
How to send
We do not want to change the CEPengine to use different protocols, etc.!
Where to send
![Page 22: DCEP-Sim: An Open Simulation Framework for Distributed CEPComplex Event Processing - Ubi Sumus? Quo Vadimus?, In Mobile Big Data: A Roadmap from Models to Technologies. Springer •](https://reader035.fdocuments.in/reader035/viewer/2022071509/612a034fee272872d013da63/html5/thumbnails/22.jpg)
Forwarder & Placement
23
Detector Producer Forwarder
Dispatcher CommunicationPlacement
CEP Engine
Source Sink
Forwarder passes events to Placement
Placement uses Event Routing Table todetermine destination of event
Event Routing Table
![Page 23: DCEP-Sim: An Open Simulation Framework for Distributed CEPComplex Event Processing - Ubi Sumus? Quo Vadimus?, In Mobile Big Data: A Roadmap from Models to Technologies. Springer •](https://reader035.fdocuments.in/reader035/viewer/2022071509/612a034fee272872d013da63/html5/thumbnails/23.jpg)
Operator in Detector
26.06.2018 24
Buffer Manager
Operator
Sel
ectio
nP
olic
y
Con
sum
ptio
nP
olic
y
Policy
Mechanism
26.06.2018Detector Producer Forwarder
CEP Engine
![Page 24: DCEP-Sim: An Open Simulation Framework for Distributed CEPComplex Event Processing - Ubi Sumus? Quo Vadimus?, In Mobile Big Data: A Roadmap from Models to Technologies. Springer •](https://reader035.fdocuments.in/reader035/viewer/2022071509/612a034fee272872d013da63/html5/thumbnails/24.jpg)
Placement
• Assign operators to event brokers– Initial– Adaptation– Challenging optimization problems
• Network utilization• Energy consumption• Event delivery latency
• (security) constraints
• Result of placement: Operator overlay resp. operator tree• Further tasks: event routing & forwarding26.06.2018 25
[Koldehofe et al. 2012]
![Page 25: DCEP-Sim: An Open Simulation Framework for Distributed CEPComplex Event Processing - Ubi Sumus? Quo Vadimus?, In Mobile Big Data: A Roadmap from Models to Technologies. Springer •](https://reader035.fdocuments.in/reader035/viewer/2022071509/612a034fee272872d013da63/html5/thumbnails/25.jpg)
Example: Centralized Placement
• Sink node knows network topology• Could calculate optimal placement for (A ∨ B) ∧ C• Sends the operators to the selected brokers• Sends routing information to all overlay nodes
26.06.2018 26
C B
A
Application∧
∨
![Page 26: DCEP-Sim: An Open Simulation Framework for Distributed CEPComplex Event Processing - Ubi Sumus? Quo Vadimus?, In Mobile Big Data: A Roadmap from Models to Technologies. Springer •](https://reader035.fdocuments.in/reader035/viewer/2022071509/612a034fee272872d013da63/html5/thumbnails/26.jpg)
Example: Centralized Placement as it is in the Code
• Places the entire query on one node• Sends the operators to the selected broker• Sends routing information to all overlay nodes
26.06.2018 27
CB
A
Application∧
∨
![Page 27: DCEP-Sim: An Open Simulation Framework for Distributed CEPComplex Event Processing - Ubi Sumus? Quo Vadimus?, In Mobile Big Data: A Roadmap from Models to Technologies. Springer •](https://reader035.fdocuments.in/reader035/viewer/2022071509/612a034fee272872d013da63/html5/thumbnails/27.jpg)
Example: Distributed Placement
• Sink (CCC) forwards operator graph onthe shortes path towards sources
• On each following node:– can all sources reached through a
single link?• Yes: forward entire (sub-)graph• No: split operator graph, place operator
locally forward sub-graphs,update event routing table
26.06.2018 28Starks, F., Plagemann, T.: Operator placement for efficient distributedcomplex event processing in MANETs, WiMOB 2015
![Page 28: DCEP-Sim: An Open Simulation Framework for Distributed CEPComplex Event Processing - Ubi Sumus? Quo Vadimus?, In Mobile Big Data: A Roadmap from Models to Technologies. Springer •](https://reader035.fdocuments.in/reader035/viewer/2022071509/612a034fee272872d013da63/html5/thumbnails/28.jpg)
Placement
26.06.2018 29
Placement Mechanism
Placement Policy Event routing table
![Page 29: DCEP-Sim: An Open Simulation Framework for Distributed CEPComplex Event Processing - Ubi Sumus? Quo Vadimus?, In Mobile Big Data: A Roadmap from Models to Technologies. Springer •](https://reader035.fdocuments.in/reader035/viewer/2022071509/612a034fee272872d013da63/html5/thumbnails/29.jpg)
Source and Sink
26.06.2018 30
Source SinkProduce atomicevents
Pose a query
Receive event stream
![Page 30: DCEP-Sim: An Open Simulation Framework for Distributed CEPComplex Event Processing - Ubi Sumus? Quo Vadimus?, In Mobile Big Data: A Roadmap from Models to Technologies. Springer •](https://reader035.fdocuments.in/reader035/viewer/2022071509/612a034fee272872d013da63/html5/thumbnails/30.jpg)
Communication
• Responsible for transport of messages– Placement messages
• Forwarding of (parts of) operator graph• Coordination of placement adaptation
– Event notifications• Current implementation uses UDP
26.06.2018 31
Dispatcher Comm.
CEP Engine
Placement
Src. Sink
![Page 31: DCEP-Sim: An Open Simulation Framework for Distributed CEPComplex Event Processing - Ubi Sumus? Quo Vadimus?, In Mobile Big Data: A Roadmap from Models to Technologies. Springer •](https://reader035.fdocuments.in/reader035/viewer/2022071509/612a034fee272872d013da63/html5/thumbnails/31.jpg)
Dispatcher
• Facade component• Dispatches
26.06.2018 32
Dispatcher Comm.
CEP Engine
Placement
Src. Sink
![Page 32: DCEP-Sim: An Open Simulation Framework for Distributed CEPComplex Event Processing - Ubi Sumus? Quo Vadimus?, In Mobile Big Data: A Roadmap from Models to Technologies. Springer •](https://reader035.fdocuments.in/reader035/viewer/2022071509/612a034fee272872d013da63/html5/thumbnails/32.jpg)
Component InteractionsCentralized Placement
26.06.2018 33
Dispatcher Comm.
CEP Engine
Placement
Dispatcher Comm.
CEP Engine
Placement
Dispatcher Comm.
CEP Engine
Placement
Src.
Dispatcher Comm.
CEP Engine
Placement
Src.
Sink
Query
![Page 33: DCEP-Sim: An Open Simulation Framework for Distributed CEPComplex Event Processing - Ubi Sumus? Quo Vadimus?, In Mobile Big Data: A Roadmap from Models to Technologies. Springer •](https://reader035.fdocuments.in/reader035/viewer/2022071509/612a034fee272872d013da63/html5/thumbnails/33.jpg)
Component InteractionsDistributed Placement
26.06.2018 34
Dispatcher Comm.
CEP Engine
Placement
Dispatcher Comm.
CEP Engine
Placement
Dispatcher Comm.
CEP Engine
Placement
Src.
Dispatcher Comm.
CEP Engine
Placement
Src.
Sink
Query
![Page 34: DCEP-Sim: An Open Simulation Framework for Distributed CEPComplex Event Processing - Ubi Sumus? Quo Vadimus?, In Mobile Big Data: A Roadmap from Models to Technologies. Springer •](https://reader035.fdocuments.in/reader035/viewer/2022071509/612a034fee272872d013da63/html5/thumbnails/34.jpg)
Component InteractionsEvent Processing
26.06.2018 35
Dispatcher Comm.
CEP Engine
Placement
Dispatcher Comm.
CEP Engine
Placement
Dispatcher Comm.
CEP Engine
Placement
Src.
Dispatcher Comm.
CEP Engine
Placement
Src.
Sink
![Page 35: DCEP-Sim: An Open Simulation Framework for Distributed CEPComplex Event Processing - Ubi Sumus? Quo Vadimus?, In Mobile Big Data: A Roadmap from Models to Technologies. Springer •](https://reader035.fdocuments.in/reader035/viewer/2022071509/612a034fee272872d013da63/html5/thumbnails/35.jpg)
Scalability: number of brokers
26.06.2018 36
Number of brokersbrokers events operators500 500 11000200030004000
![Page 36: DCEP-Sim: An Open Simulation Framework for Distributed CEPComplex Event Processing - Ubi Sumus? Quo Vadimus?, In Mobile Big Data: A Roadmap from Models to Technologies. Springer •](https://reader035.fdocuments.in/reader035/viewer/2022071509/612a034fee272872d013da63/html5/thumbnails/36.jpg)
Scalability: number of events
26.06.2018 37
Number of events
events brokers operators2000 1 14000600080001000012000
![Page 37: DCEP-Sim: An Open Simulation Framework for Distributed CEPComplex Event Processing - Ubi Sumus? Quo Vadimus?, In Mobile Big Data: A Roadmap from Models to Technologies. Springer •](https://reader035.fdocuments.in/reader035/viewer/2022071509/612a034fee272872d013da63/html5/thumbnails/37.jpg)
End of Part 1
• Very short motivation for distributed CEP• Design approach• Components and their responsibility• Component interaction
• Components correspond to objects in the code (part 3)• To understand the implementation it is very important to
understand ns-3 (next part)
26.06.2018 38
![Page 38: DCEP-Sim: An Open Simulation Framework for Distributed CEPComplex Event Processing - Ubi Sumus? Quo Vadimus?, In Mobile Big Data: A Roadmap from Models to Technologies. Springer •](https://reader035.fdocuments.in/reader035/viewer/2022071509/612a034fee272872d013da63/html5/thumbnails/38.jpg)
Outline (cont.)
• DCEP-Sim use and extensions– Overview code structure– How do I run DCEP-Sim & how works a «script»– Changing the workload– How are placement policies implemented -> adding new placement – How are operators implemented -> adding new operators
• Conclusions
• Hands-on if you want to install ns-3 and run DCEP-Sim on you Linux laptop
26.06.2018 39
![Page 39: DCEP-Sim: An Open Simulation Framework for Distributed CEPComplex Event Processing - Ubi Sumus? Quo Vadimus?, In Mobile Big Data: A Roadmap from Models to Technologies. Springer •](https://reader035.fdocuments.in/reader035/viewer/2022071509/612a034fee272872d013da63/html5/thumbnails/39.jpg)
DCEP-Sim on github
• https://github.com/fabricesb/DCEP-Sim• GNU GPLv2 license (to be in line with ns-3)
26.06.2018 40
![Page 40: DCEP-Sim: An Open Simulation Framework for Distributed CEPComplex Event Processing - Ubi Sumus? Quo Vadimus?, In Mobile Big Data: A Roadmap from Models to Technologies. Springer •](https://reader035.fdocuments.in/reader035/viewer/2022071509/612a034fee272872d013da63/html5/thumbnails/40.jpg)
26.06.2018 41
DCEP-Sim code
![Page 41: DCEP-Sim: An Open Simulation Framework for Distributed CEPComplex Event Processing - Ubi Sumus? Quo Vadimus?, In Mobile Big Data: A Roadmap from Models to Technologies. Springer •](https://reader035.fdocuments.in/reader035/viewer/2022071509/612a034fee272872d013da63/html5/thumbnails/41.jpg)
26.06.2018 42
DCEP-Sim code
![Page 42: DCEP-Sim: An Open Simulation Framework for Distributed CEPComplex Event Processing - Ubi Sumus? Quo Vadimus?, In Mobile Big Data: A Roadmap from Models to Technologies. Springer •](https://reader035.fdocuments.in/reader035/viewer/2022071509/612a034fee272872d013da63/html5/thumbnails/42.jpg)
26.06.2018 43
Here you find the example script we will walk throughnamed decep-example.cc
Here you find all models, i.e., the core of DCEP-Sim
![Page 43: DCEP-Sim: An Open Simulation Framework for Distributed CEPComplex Event Processing - Ubi Sumus? Quo Vadimus?, In Mobile Big Data: A Roadmap from Models to Technologies. Springer •](https://reader035.fdocuments.in/reader035/viewer/2022071509/612a034fee272872d013da63/html5/thumbnails/43.jpg)
Components, objects, and aggregation
26.06.2018 44
DCEP application
Data source
CEP engine
Communication
Placement
Sink
dcep.cc
communication.cc
placement.cc
cep-engine.cc
inside dcep.cc
![Page 44: DCEP-Sim: An Open Simulation Framework for Distributed CEPComplex Event Processing - Ubi Sumus? Quo Vadimus?, In Mobile Big Data: A Roadmap from Models to Technologies. Springer •](https://reader035.fdocuments.in/reader035/viewer/2022071509/612a034fee272872d013da63/html5/thumbnails/44.jpg)
Components, objects, and aggregation (cont.)
26.06.2018 45
All in cep-engine.cc
CEPEngine
Detector
ProducerForwarder
![Page 45: DCEP-Sim: An Open Simulation Framework for Distributed CEPComplex Event Processing - Ubi Sumus? Quo Vadimus?, In Mobile Big Data: A Roadmap from Models to Technologies. Springer •](https://reader035.fdocuments.in/reader035/viewer/2022071509/612a034fee272872d013da63/html5/thumbnails/45.jpg)
26.06.2018 46
![Page 46: DCEP-Sim: An Open Simulation Framework for Distributed CEPComplex Event Processing - Ubi Sumus? Quo Vadimus?, In Mobile Big Data: A Roadmap from Models to Technologies. Springer •](https://reader035.fdocuments.in/reader035/viewer/2022071509/612a034fee272872d013da63/html5/thumbnails/46.jpg)
26.06.2018 47
Right now it contains one helper to set up dcep-appdcep-app-helper.cc
![Page 47: DCEP-Sim: An Open Simulation Framework for Distributed CEPComplex Event Processing - Ubi Sumus? Quo Vadimus?, In Mobile Big Data: A Roadmap from Models to Technologies. Springer •](https://reader035.fdocuments.in/reader035/viewer/2022071509/612a034fee272872d013da63/html5/thumbnails/47.jpg)
Typical elements of a script
26.06.2018 48
Create and installnetwork topology
Configuration of logging level
Setting up network, and transport layer
Setting up the Distributed CEP overlay
Running the simulation
![Page 48: DCEP-Sim: An Open Simulation Framework for Distributed CEPComplex Event Processing - Ubi Sumus? Quo Vadimus?, In Mobile Big Data: A Roadmap from Models to Technologies. Springer •](https://reader035.fdocuments.in/reader035/viewer/2022071509/612a034fee272872d013da63/html5/thumbnails/48.jpg)
26.06.2018 49
Create and installnetwork topology
Configuration of logging level
Setting up network, and transport layer
Setting up the Distributed CEP overlay
Running the simulation
LogComponentEnable ("Placement", LOG_LEVEL_INFO);LogComponentEnable ("Dcep", LOG_LEVEL_INFO);LogComponentEnable ("Detector", LOG_LEVEL_INFO);LogComponentEnable ("Communication",
LOG_LEVEL_INFO);
![Page 49: DCEP-Sim: An Open Simulation Framework for Distributed CEPComplex Event Processing - Ubi Sumus? Quo Vadimus?, In Mobile Big Data: A Roadmap from Models to Technologies. Springer •](https://reader035.fdocuments.in/reader035/viewer/2022071509/612a034fee272872d013da63/html5/thumbnails/49.jpg)
26.06.2018 50
Create and installnetwork topology
Configuration of logging level
Setting up network, and transport layer
Setting up the Distributed CEP overlay
Running the simulation
uint32_t numNodes = gridWidth*gridWidth;NodeContainer n;n.Create (numNodes);
NetDeviceContainer devices = SetupWirelessNetwork(n);
MobilityHelper mobility;
mobility.SetPositionAllocator("ns3::GridPositionAllocator", "MinX",
DoubleValue (0.0), "MinY", DoubleValue (0.0), "DeltaX", DoubleValue (distance), "DeltaY", DoubleValue (distance), "GridWidth", UintegerValue (gridWidth), "LayoutType", StringValue ("RowFirst"));
mobility.SetMobilityModel("ns3::ConstantPositionMobilityModel");
mobility.Install (n);
![Page 50: DCEP-Sim: An Open Simulation Framework for Distributed CEPComplex Event Processing - Ubi Sumus? Quo Vadimus?, In Mobile Big Data: A Roadmap from Models to Technologies. Springer •](https://reader035.fdocuments.in/reader035/viewer/2022071509/612a034fee272872d013da63/html5/thumbnails/50.jpg)
Typical elements of a script
26.06.2018 51
Create and installnetwork topology
Configuration of logging level
Setting up network, and transport layer
Setting up the Distributed CEP overlay
Running the simulation
OlsrHelper olsr;InternetStackHelper internet;
internet.SetRoutingHelper (olsr); internet.Install (n);Ipv4AddressHelper ipv4;
ipv4.SetBase ("10.1.1.0", "255.255.255.0");Ipv4InterfaceContainer iface =
ipv4.Assign (devices);
![Page 51: DCEP-Sim: An Open Simulation Framework for Distributed CEPComplex Event Processing - Ubi Sumus? Quo Vadimus?, In Mobile Big Data: A Roadmap from Models to Technologies. Springer •](https://reader035.fdocuments.in/reader035/viewer/2022071509/612a034fee272872d013da63/html5/thumbnails/51.jpg)
26.06.2018 52
Create and installnetwork topology
Configuration of logging level
Setting up network, and transport layer
Setting up the Distributed CEP overlay
Running the simulation
sinkAddress = Address(iface.GetAddress (0));DcepAppHelper dcepApphelper;ApplicationContainer dcepApps =
dcepApphelper.Install (n);uint32_t eventCode = 1;
for(uint32_t i = 0; i <= numNodes; i++) {dcepApps.Get(i)->SetAttribute
("SinkAddress", AddressValue(sinkAddress));dcepApps.Get(i)->SetAttribute("placementpolicy", StringValue(placementPolicy));
if(i == 0) { /* sink node*/dcepApps.Get(i)->SetAttribute("IsSink", BooleanValue(true));
}else if ((i == (numNodes-1)) || (i == (numNodes-2))){
dcepApps.Get(i)->SetAttribute("IsGenerator", BooleanValue(true));
dcepApps.Get(i)->SetAttribute("event code",UintegerValue (eventCode++));
dcepApps.Get(i)->SetAttribute("number ofevents", UintegerValue (numberOfEvents));
}}
![Page 52: DCEP-Sim: An Open Simulation Framework for Distributed CEPComplex Event Processing - Ubi Sumus? Quo Vadimus?, In Mobile Big Data: A Roadmap from Models to Technologies. Springer •](https://reader035.fdocuments.in/reader035/viewer/2022071509/612a034fee272872d013da63/html5/thumbnails/52.jpg)
Typical elements of a script
26.06.2018 53
Create and installnetwork topology
Configuration of logging level
Setting up network, and transport layer
Setting up the Distributed CEP overlay
Running the simulation
dcepApps.Start (Seconds (1.0));dcepApps.Stop (Seconds (30.0));Simulator::Stop(Seconds(35.0));Simulator::Run ();Simulator::Destroy ();
![Page 53: DCEP-Sim: An Open Simulation Framework for Distributed CEPComplex Event Processing - Ubi Sumus? Quo Vadimus?, In Mobile Big Data: A Roadmap from Models to Technologies. Springer •](https://reader035.fdocuments.in/reader035/viewer/2022071509/612a034fee272872d013da63/html5/thumbnails/53.jpg)
Change the workload
• Current event sources produce uniform traffic • Configure Distributed CEP instances as data sources in the
script, e.g.,
• Set number of events in the script
26.06.2018 54
dcepApps.Get(0)->SetAttribute("IsGenerator", BooleanValue(true));dcepApps.Get(0)->SetAttribute("event code",UintegerValue (eventCode++));
dcepApps.Get(i)->SetAttribute("number of events", UintegerValue (numberOfEvents));
![Page 54: DCEP-Sim: An Open Simulation Framework for Distributed CEPComplex Event Processing - Ubi Sumus? Quo Vadimus?, In Mobile Big Data: A Roadmap from Models to Technologies. Springer •](https://reader035.fdocuments.in/reader035/viewer/2022071509/612a034fee272872d013da63/html5/thumbnails/54.jpg)
Change the workload (cont.)
• Currently, the event rate is set in the DataSource::GenerateAtomicEvents()implementation in dcep.cc
26.06.2018 55
if(counter < numEvents){
Simulator::Schedule (MilliSeconds (100), &DataSource::GenerateAtomicEvents, this);
}
Good example of scheduling discrete ns-3 events….….. to generate at a fixed rate atomic events!
![Page 55: DCEP-Sim: An Open Simulation Framework for Distributed CEPComplex Event Processing - Ubi Sumus? Quo Vadimus?, In Mobile Big Data: A Roadmap from Models to Technologies. Springer •](https://reader035.fdocuments.in/reader035/viewer/2022071509/612a034fee272872d013da63/html5/thumbnails/55.jpg)
Change workload (cont.)
• For more complex event patterns extend the data source model or create a new data source model– Get inspired by ns-3 traffic models
• Statistical distributions• Trace files• .......
– Extend/modify the function GenerateCEPEvents()which can be found in the file dcep.cc
26.06.2018 56
![Page 56: DCEP-Sim: An Open Simulation Framework for Distributed CEPComplex Event Processing - Ubi Sumus? Quo Vadimus?, In Mobile Big Data: A Roadmap from Models to Technologies. Springer •](https://reader035.fdocuments.in/reader035/viewer/2022071509/612a034fee272872d013da63/html5/thumbnails/56.jpg)
Creating Your Own Placement Policy
• Main responsabilities of placement:1. Operator assignment2. Event routing and forwarding
• Approach:– High + low level views– Creating a new placement policy– Example: centralized placement
26.06.2018 57
Placement
PlacementMechanism
PlacementPolicy
Makes decisions(typically subjectof research)
Executes decisions(usually invariantamong policies)
![Page 57: DCEP-Sim: An Open Simulation Framework for Distributed CEPComplex Event Processing - Ubi Sumus? Quo Vadimus?, In Mobile Big Data: A Roadmap from Models to Technologies. Springer •](https://reader035.fdocuments.in/reader035/viewer/2022071509/612a034fee272872d013da63/html5/thumbnails/57.jpg)
Placement Assignment: High-Level Overview
26.06.2018 58
Placement
Placement Mechanism
Placement Policy
Sink
E vent rou ting tab le
CEP Engine Communication
Operator graphA∨∧ CB
∧∨ABC
Locally placedoperators
Remotely placedoperators
Operator flow
• Next hop• Ouput
destination
Control flow
![Page 58: DCEP-Sim: An Open Simulation Framework for Distributed CEPComplex Event Processing - Ubi Sumus? Quo Vadimus?, In Mobile Big Data: A Roadmap from Models to Technologies. Springer •](https://reader035.fdocuments.in/reader035/viewer/2022071509/612a034fee272872d013da63/html5/thumbnails/58.jpg)
Placement Node Y
Placement Mechanism
Placement Policy
Placement Node X
Placement Mechanism
Placement Policy
Placement Assignment: High-Level Overview
26.06.2018 59
Communication
V
Remotely placedoperators
Placement Node Z
Placement Mechanism
Placement PolicyA
Operator flow
Control flow
Network
![Page 59: DCEP-Sim: An Open Simulation Framework for Distributed CEPComplex Event Processing - Ubi Sumus? Quo Vadimus?, In Mobile Big Data: A Roadmap from Models to Technologies. Springer •](https://reader035.fdocuments.in/reader035/viewer/2022071509/612a034fee272872d013da63/html5/thumbnails/59.jpg)
Placement Assignment: High-Level Overview
26.06.2018 60
Placement
Placement Mechanism
Placement Policy
Sink
E vent rou ting tab le
CEP Engine Communication
Operator graphA∨∧ CB
∧∨ABC
Locally placedoperators
Remotely placedoperators
Operator flow
• Next hop• Ouput
destination
Control flowMain responsabilityof Placement Policy!
![Page 60: DCEP-Sim: An Open Simulation Framework for Distributed CEPComplex Event Processing - Ubi Sumus? Quo Vadimus?, In Mobile Big Data: A Roadmap from Models to Technologies. Springer •](https://reader035.fdocuments.in/reader035/viewer/2022071509/612a034fee272872d013da63/html5/thumbnails/60.jpg)
The Event Routing Table (ERT)• Accessed via interface called DcepState• Important fields in entries:
– Destination of event (output destination)– Destination of the query (next hop)– Data sources
• Additional fields mostly for adaptation and monitoring– Operator state (active or not)– Freeze acknowledgement counter– Freeze queue– Monitoring– Current processor
26.06.2018 61
Event Routing Table(ERT)
![Page 61: DCEP-Sim: An Open Simulation Framework for Distributed CEPComplex Event Processing - Ubi Sumus? Quo Vadimus?, In Mobile Big Data: A Roadmap from Models to Technologies. Springer •](https://reader035.fdocuments.in/reader035/viewer/2022071509/612a034fee272872d013da63/html5/thumbnails/61.jpg)
26.06.2018 62
S ink D cep P lacem ent X P lacem entP o licy C om m un ica tion
D cepE ng ine
D cepS ta te
Placement
Placement Mechanism
Placement Policy
CEP Engine Communication
Sink
E vent rou ting tab le
StartApplication()
Data object flow
Schedule execution
Function call
q = queryo = output addressn = next hop addresse = event
![Page 62: DCEP-Sim: An Open Simulation Framework for Distributed CEPComplex Event Processing - Ubi Sumus? Quo Vadimus?, In Mobile Big Data: A Roadmap from Models to Technologies. Springer •](https://reader035.fdocuments.in/reader035/viewer/2022071509/612a034fee272872d013da63/html5/thumbnails/62.jpg)
26.06.2018 63
S ink D cep P lacem ent X P lacem entP o licy C om m un ica tion
D cepE ng ine
D cepS ta te
Placement
Placement Mechanism
Placement Policy
CEP Engine Communication
Sink
E vent rou ting tab le
StartApplication()
BuildAndSendQuery()
DispatchQuery(q)RecvQuery(q)
DoPlacement()
Repeated for every operator q
Data object flowSchedule executionFunction call
q = queryo = output addressn = next hop addresse = event
if query q
![Page 63: DCEP-Sim: An Open Simulation Framework for Distributed CEPComplex Event Processing - Ubi Sumus? Quo Vadimus?, In Mobile Big Data: A Roadmap from Models to Technologies. Springer •](https://reader035.fdocuments.in/reader035/viewer/2022071509/612a034fee272872d013da63/html5/thumbnails/63.jpg)
26.06.2018 64
S ink D cep P lacem ent X P lacem entP o licy C om m un ica tion
D cepE ng ine
D cepS ta te
Placement
Placement Mechanism
Placement Policy
CEP Engine Communication
Sink
E vent rou ting tab le
StartApplication()
BuildAndSendQuery()
DispatchQuery(q)RecvQuery(q)
DoPlacement()
Repeated for every operator q
E R T
q o,n
Data object flow
Schedule execution
Function call
q = queryo = output addressn = next hop addresse = event
if query q
CreateEventRoutingTableEntry(q)
SetOutDest(q,o)
SetNextHop(q,n)
q
ForwardQuery(q)
![Page 64: DCEP-Sim: An Open Simulation Framework for Distributed CEPComplex Event Processing - Ubi Sumus? Quo Vadimus?, In Mobile Big Data: A Roadmap from Models to Technologies. Springer •](https://reader035.fdocuments.in/reader035/viewer/2022071509/612a034fee272872d013da63/html5/thumbnails/64.jpg)
26.06.2018 65
S ink D cep P lacem ent X P lacem entP o licy C om m un ica tion
D cepE ng ine
D cepS ta te
Placement
Placement Mechanism
Placement Policy
CEP Engine Communication
Sink
E vent rou ting tab le
StartApplication()
BuildAndSendQuery()
DispatchQuery(q)RecvQuery(q)
DoPlacement()
Repeated for every operator q
E R T
q o,n
ActivateDataSource(q)
RecvQuery (q)SendPacket (q)
ScheduleSend (q)
Action chosen based on decisionmade by placement policy, which is obtained from the ERT if localif remote
if atomic and local
Data object flowSchedule executionFunction call
q = queryo = output addressn = next hop addresse = event
if query q
CreateEventRoutingTableEntry(q)
SetOutDest(q,o)
SetNextHop(q,n)
q
q
ForwardQuery(q)
![Page 65: DCEP-Sim: An Open Simulation Framework for Distributed CEPComplex Event Processing - Ubi Sumus? Quo Vadimus?, In Mobile Big Data: A Roadmap from Models to Technologies. Springer •](https://reader035.fdocuments.in/reader035/viewer/2022071509/612a034fee272872d013da63/html5/thumbnails/65.jpg)
26.06.2018 66
S ink D cep P lacem ent X P lacem entP o licy C om m un ica tion
D cepE ng ine
D cepS ta te
RecvQuery(q)
DoPlacement()
q
CreateEventRoutingTableEntry(q)
E R T
qSetOutDest(q,o)
o,n
ForwardQuery(q)ActivateDataSource(q)
RecvQuery (q)SendPacket (q)
ScheduleSend (q)
if localif remote
if atomic and local
Data object flow
Schedule execution
Function call
q = queryo = output addressn = next hop addresse = event
q
Placement Node X
Placement Mechanism
Placement Policy
Communication
V
A
Ne
two
rk
Placement Node Z
Placement Mechanism
Placement Policy
Placement Node Z
Placement Mechanism
Placement Policy
HandleRead(socket)
RecvRemoveMsg(buffer, …)
if query
q
SetNextHop(q,n)
![Page 66: DCEP-Sim: An Open Simulation Framework for Distributed CEPComplex Event Processing - Ubi Sumus? Quo Vadimus?, In Mobile Big Data: A Roadmap from Models to Technologies. Springer •](https://reader035.fdocuments.in/reader035/viewer/2022071509/612a034fee272872d013da63/html5/thumbnails/66.jpg)
X P lacem entP o licy
26.06.2018 67
S ink D cep P lacem ent
RecvCepEvent(e)
HandleRead(socket)
Event Routing and Forwarding
if event
Data object flow
Schedule execution
Function call
q = query
o = output address
n = next hop address
e = event
C om m un ica tion
D cepE ng ine
D cepS ta te
RecvRemoteMsg(buffer, …)
![Page 67: DCEP-Sim: An Open Simulation Framework for Distributed CEPComplex Event Processing - Ubi Sumus? Quo Vadimus?, In Mobile Big Data: A Roadmap from Models to Technologies. Springer •](https://reader035.fdocuments.in/reader035/viewer/2022071509/612a034fee272872d013da63/html5/thumbnails/67.jpg)
X P lacem entP o licy
26.06.2018 68
S ink D cep P lacem ent
RecvCepEvent(e)
HandleRead(socket)
RecvRemoteMsg(buffer, …)
Event Routing and Forwarding
if event
ProcessCepEvent(e)
SendFinalEventToSink(e)
if finalif not finalreceiveFinalEvent(e)
Detector
Producer
CepOperator
Forwarder
ForwardProducedEvent(e)
Data object flow
Schedule execution
Function call
q = query
o = output address
n = next hop address
e = event
C om m un ica tion
D cepE ng ine
D cepS ta te
![Page 68: DCEP-Sim: An Open Simulation Framework for Distributed CEPComplex Event Processing - Ubi Sumus? Quo Vadimus?, In Mobile Big Data: A Roadmap from Models to Technologies. Springer •](https://reader035.fdocuments.in/reader035/viewer/2022071509/612a034fee272872d013da63/html5/thumbnails/68.jpg)
X P lacem entP o licy
26.06.2018 69
S ink D cep P lacem ent
RecvCepEvent(e)
HandleRead(socket)
Event Routing and Forwarding
if event
ProcessCepEvent(e)
SendFinalEventToSink(e)
if finalif not finalreceiveFinalEvent(e)
Detector
Producer
CepOperator
Forwarder
ForwardProducedEvent(e)
SendFinalEventToSink(e)
if finalreceiveFinalEvent(e)
ProcessCepEvent(e)
if not final and local
GetOutputDest(e)
o
SendPacket(e,o)
if remote
ScheduleSend (e,o)
Data object flow
Schedule execution
Function call
q = query
o = output address
n = next hop address
e = event
C om m un ica tion
D cepE ng ine
E R T
q o,n
D cepS ta te
RecvRemoteMsg(buffer, …)
![Page 69: DCEP-Sim: An Open Simulation Framework for Distributed CEPComplex Event Processing - Ubi Sumus? Quo Vadimus?, In Mobile Big Data: A Roadmap from Models to Technologies. Springer •](https://reader035.fdocuments.in/reader035/viewer/2022071509/612a034fee272872d013da63/html5/thumbnails/69.jpg)
Adding a New Placement Policy• Create a sub-class of PlacementPolicy• Must be defined:
– Ns-3-specific functions, attributes and trace-sources:
• Mandatory: GetTypeID() ->– configure()
• Initialisation– DoPlacement()
• Mandatory• Manipulate ERT via aggregated DcepState-object ->
• Call placement mechanism once per operator ->
26.06.2018 70
class DcepState : public Object {…void SetNextHop (std::string eventType, Ipv4Address adr);void SetOutDest (std::string eventType, Ipv4Address adr);
dcep-state.h
class Placement : public Object {…void ForwardQuery(std::string eType);
placement.h
TypeId CentralizedPlacementPolicy::GetTypeId(void) {static TypeId tid = TypeId("ns3::CentralizedPlacementPolicy")
.SetParent<PlacementPolicy> ()
.AddConstructor<CentralizedPlacementPolicy> ();return tid;
}
placement.cc
![Page 70: DCEP-Sim: An Open Simulation Framework for Distributed CEPComplex Event Processing - Ubi Sumus? Quo Vadimus?, In Mobile Big Data: A Roadmap from Models to Technologies. Springer •](https://reader035.fdocuments.in/reader035/viewer/2022071509/612a034fee272872d013da63/html5/thumbnails/70.jpg)
Example Placement Policy: Centralized Placement
26.06.2018 71
B
A
v
Sink Sources
Query: A ∨ B
A∨ BOperator graph
![Page 71: DCEP-Sim: An Open Simulation Framework for Distributed CEPComplex Event Processing - Ubi Sumus? Quo Vadimus?, In Mobile Big Data: A Roadmap from Models to Technologies. Springer •](https://reader035.fdocuments.in/reader035/viewer/2022071509/612a034fee272872d013da63/html5/thumbnails/71.jpg)
Example Placement Policy: Centralized Placement
26.06.2018 72
Query: A ∨ B
A∨ BOperator graph
B
A
v
Sink Sources
Atomic events from B
Atomic events from A
Final, composite eventsfrom v
![Page 72: DCEP-Sim: An Open Simulation Framework for Distributed CEPComplex Event Processing - Ubi Sumus? Quo Vadimus?, In Mobile Big Data: A Roadmap from Models to Technologies. Springer •](https://reader035.fdocuments.in/reader035/viewer/2022071509/612a034fee272872d013da63/html5/thumbnails/72.jpg)
26.06.2018 73
void Sink::BuildAndSendQuery() {Ptr<Dcep> dcep = GetObject<Dcep> ();
Ptr<Query> q1 = CreateObject<Query> ();q1->actionType = NOTIFICATION;q1->id = query_counter++;q1->isFinal = false;q1->isAtomic = true;q1->eventType = "A";q1->output_dest = Ipv4Address::GetAny();q1->inevent1 = "A";q1->inevent2 = "";q1->op = "true";q1->assigned = false;q1->currentHost.Set("0.0.0.0");q1->parent_output = "AorB";NS_LOG_INFO ("Setup query " << q1->eventType);dcep->DispatchQuery(q1);…q2->eventType = "B";q2->inevent1 = "B";dcep->DispatchQuery(q2);…q3->isFinal = true;q3->isAtomic = false;q3->eventType = "AorB";q3->inevent1 = "A";q3->inevent2 = "B";q3->op = "or";NS_LOG_INFO ("Setup query " << q3->eventType);dcep->DispatchQuery(q3);
}
SinkOperator graph
A∨ B
model/dcep.cc
StartApplication()BuildAndSendQuery()
DispatchQuery(q)RecvQuery(q)Repeated for
every operator q if query q
S ink D cep P lacem ent
![Page 73: DCEP-Sim: An Open Simulation Framework for Distributed CEPComplex Event Processing - Ubi Sumus? Quo Vadimus?, In Mobile Big Data: A Roadmap from Models to Technologies. Springer •](https://reader035.fdocuments.in/reader035/viewer/2022071509/612a034fee272872d013da63/html5/thumbnails/73.jpg)
26.06.2018 74
Placement
Placement Mechanism
Placement Policy
Locally placedoperators
Remotely placedoperators
voidCentralizedPlacementPolicy::configure() {
}
voidCentralizedPlacementPolicy::DoPlacement() {
NS_LOG_INFO ("Doing centralized placement");Ptr<Placement> p = GetObject<Placement>();
std::vector<Ptr < Query>>::iterator it;std::vector<Ptr < Query>> qs = p->q_queue;
for (it = qs.begin(); it != qs.end(); ++it) {
Ptr<Query> q = *it;if (!PlaceQuery(q)) {
Simulator::Schedule(Seconds(3.0), &CentralizedPlacementPolicy::DoPlacement, this);
} elsep->RemoveQuery(q);
}
}
model/placement.cc
DoPlacement()
q q
P lacem ent X P lacem entP o licy
![Page 74: DCEP-Sim: An Open Simulation Framework for Distributed CEPComplex Event Processing - Ubi Sumus? Quo Vadimus?, In Mobile Big Data: A Roadmap from Models to Technologies. Springer •](https://reader035.fdocuments.in/reader035/viewer/2022071509/612a034fee272872d013da63/html5/thumbnails/74.jpg)
D cepS ta te
Placement
Placement Mechanism
Placement Policy
Locally placedoperators
Remotely placedoperators
26.06.2018
75
bool CentralizedPlacementPolicy::PlaceQuery(Ptr<Query> q) {Ptr<Placement> p = GetObject<Placement>();Ptr<DcepState> dstate = GetObject<DcepState>();dstate->CreateEventRoutingTableEntry(q);Ptr<Communication> cm = GetObject<Communication>();bool placed = false;
if (!q->isAtomic) dstate->SetNextHop(q->eventType, cm->GetLocalAddress());placed = true;
} else if (q->isAtomic) {if(q->eventType == "A") { …
dstate->SetNextHop(q->eventType, Ipv4Address("10.0.0.2"));placed = true;
} else if(q->eventType == "B") {
dstate->SetNextHop(q->eventType, Ipv4Address("10.0.0.3"));placed = true;
} …
}
if (placed) {NS_LOG_INFO ("QUERY PLACED");newLocalPlacement(q->eventType);if(dstate->GetNextHop(q->eventType).IsEqual(cm->GetLocalAddress())) {
NS_LOG_INFO ("QUERY PLACED ON LOCAL NODE");if (!q->isAtomic)
dstate->SetOutDest(q->eventType, cm->GetLocalAddress());else
dstate->SetOutDest(q->eventType, cm->GetSinkAddress());}
p->ForwardQuery(q->eventType);}
return placed;}
model/placement.cc
X P lacem entP o licy
E R T
q o,n
SetOutDest(q,o)
SetNextHop(q,n)
ForwardQuery(q)
P lacem ent
CreateEventRoutingTableEntry(q)
![Page 75: DCEP-Sim: An Open Simulation Framework for Distributed CEPComplex Event Processing - Ubi Sumus? Quo Vadimus?, In Mobile Big Data: A Roadmap from Models to Technologies. Springer •](https://reader035.fdocuments.in/reader035/viewer/2022071509/612a034fee272872d013da63/html5/thumbnails/75.jpg)
Add new operators
• Operator implementation based on• CEP engine wrappers class -> detector class
26.06.2018 76
As mentioned earlier:
the focus for DCEPSim until now was placement →simple event model and few operators implemented
![Page 76: DCEP-Sim: An Open Simulation Framework for Distributed CEPComplex Event Processing - Ubi Sumus? Quo Vadimus?, In Mobile Big Data: A Roadmap from Models to Technologies. Springer •](https://reader035.fdocuments.in/reader035/viewer/2022071509/612a034fee272872d013da63/html5/thumbnails/76.jpg)
Query vs. Operator • Query used for placement• Operator used for event
processing
Info managedduring placement
The query- Values to be matched- Operator
These values are copied into an AndOperator instance
![Page 77: DCEP-Sim: An Open Simulation Framework for Distributed CEPComplex Event Processing - Ubi Sumus? Quo Vadimus?, In Mobile Big Data: A Roadmap from Models to Technologies. Springer •](https://reader035.fdocuments.in/reader035/viewer/2022071509/612a034fee272872d013da63/html5/thumbnails/77.jpg)
Conceptual structure of operator
26.06.2018 78
operatorquery
buffered events
incoming event
selection consumption
producer
![Page 78: DCEP-Sim: An Open Simulation Framework for Distributed CEPComplex Event Processing - Ubi Sumus? Quo Vadimus?, In Mobile Big Data: A Roadmap from Models to Technologies. Springer •](https://reader035.fdocuments.in/reader035/viewer/2022071509/612a034fee272872d013da63/html5/thumbnails/78.jpg)
Operator class in cep-engine.cc
26.06.2018 79
CepOperator::
Configure() Evaluate() ExpectingEvent()()
AndOperator::Configure()AndOperator::Evaluate()AndOparator::ExpectingEvent()
![Page 79: DCEP-Sim: An Open Simulation Framework for Distributed CEPComplex Event Processing - Ubi Sumus? Quo Vadimus?, In Mobile Big Data: A Roadmap from Models to Technologies. Springer •](https://reader035.fdocuments.in/reader035/viewer/2022071509/612a034fee272872d013da63/html5/thumbnails/79.jpg)
Event class in cep-engine.h
26.06.2018 80
![Page 80: DCEP-Sim: An Open Simulation Framework for Distributed CEPComplex Event Processing - Ubi Sumus? Quo Vadimus?, In Mobile Big Data: A Roadmap from Models to Technologies. Springer •](https://reader035.fdocuments.in/reader035/viewer/2022071509/612a034fee272872d013da63/html5/thumbnails/80.jpg)
26.06.2018 81
Copy info from query objectduring placement
Create a buffer manager for the operator
Set consumption and selection policies
![Page 81: DCEP-Sim: An Open Simulation Framework for Distributed CEPComplex Event Processing - Ubi Sumus? Quo Vadimus?, In Mobile Big Data: A Roadmap from Models to Technologies. Springer •](https://reader035.fdocuments.in/reader035/viewer/2022071509/612a034fee272872d013da63/html5/thumbnails/81.jpg)
26.06.2018 82
Make sure both buffers are not empty
Check which buffer theevent belongs to
Find the event from the other buffer which matches the sequencenumber of the current event
![Page 82: DCEP-Sim: An Open Simulation Framework for Distributed CEPComplex Event Processing - Ubi Sumus? Quo Vadimus?, In Mobile Big Data: A Roadmap from Models to Technologies. Springer •](https://reader035.fdocuments.in/reader035/viewer/2022071509/612a034fee272872d013da63/html5/thumbnails/82.jpg)
26.06.2018 83
Determines whether this operator is expecting events of the type provided as parameter.
![Page 83: DCEP-Sim: An Open Simulation Framework for Distributed CEPComplex Event Processing - Ubi Sumus? Quo Vadimus?, In Mobile Big Data: A Roadmap from Models to Technologies. Springer •](https://reader035.fdocuments.in/reader035/viewer/2022071509/612a034fee272872d013da63/html5/thumbnails/83.jpg)
Conclusions
• DCEPSim is – a tool for our research in operator placement for mobile distributed
CEP– not perfect– but «easily» extensible (especially if one gets aquainted with ns-3)
• In case you have any questions/ideas/comments – Talk with us here @ DEBS 2018– Email us: [email protected], [email protected],