QoS Supported Clustered Query Processing in Large Collaboration of Heterogeneous Sensor Networks...
-
Upload
asher-derek-grant -
Category
Documents
-
view
216 -
download
0
Transcript of QoS Supported Clustered Query Processing in Large Collaboration of Heterogeneous Sensor Networks...
![Page 1: QoS Supported Clustered Query Processing in Large Collaboration of Heterogeneous Sensor Networks Debraj De and Lifeng Sang Ohio State University Workshop.](https://reader036.fdocuments.in/reader036/viewer/2022062804/5697bf911a28abf838c8e8d5/html5/thumbnails/1.jpg)
QoS Supported Clustered Query Processing in Large Collaboration of
Heterogeneous Sensor Networks
Debraj De and Lifeng Sang
Ohio State University
Workshop on Distributed Collaborative Sensors Networks DCSN’09 (part of CTS’09)
May 18 – May 22, 2009
![Page 2: QoS Supported Clustered Query Processing in Large Collaboration of Heterogeneous Sensor Networks Debraj De and Lifeng Sang Ohio State University Workshop.](https://reader036.fdocuments.in/reader036/viewer/2022062804/5697bf911a28abf838c8e8d5/html5/thumbnails/2.jpg)
Outline
Collaborative Heterogeneous Sensor Networks (CHSN)
Problem definition Existing work Contribution Simulation results Discussion
![Page 3: QoS Supported Clustered Query Processing in Large Collaboration of Heterogeneous Sensor Networks Debraj De and Lifeng Sang Ohio State University Workshop.](https://reader036.fdocuments.in/reader036/viewer/2022062804/5697bf911a28abf838c8e8d5/html5/thumbnails/3.jpg)
Collaborative Heterogeneous Sensor Networks (CHSN) System
CHSN: collaboration of deployed sensor networks worldwide. Networks in CHSN are reachable among themselves. NASA’s Sensor Web, Sensor Web Enablement (SWE).
![Page 4: QoS Supported Clustered Query Processing in Large Collaboration of Heterogeneous Sensor Networks Debraj De and Lifeng Sang Ohio State University Workshop.](https://reader036.fdocuments.in/reader036/viewer/2022062804/5697bf911a28abf838c8e8d5/html5/thumbnails/4.jpg)
Problem Definition
User, Portal, Central Manager (CM), Sensor Network Fabrics. How to process streams of different kinds of queries efficiently?
![Page 5: QoS Supported Clustered Query Processing in Large Collaboration of Heterogeneous Sensor Networks Debraj De and Lifeng Sang Ohio State University Workshop.](https://reader036.fdocuments.in/reader036/viewer/2022062804/5697bf911a28abf838c8e8d5/html5/thumbnails/5.jpg)
Implication Factor (IF) between two sensor networks If SN1 is queried then need to query SN2? Indicates correlation between phenomena measured
by two sensor networks
Normal detection Less need to query Camera Sensor
Fire detected More need to query sensor for wind speed/direction
Examples of higher correlation and lower valued Implication Factor
![Page 6: QoS Supported Clustered Query Processing in Large Collaboration of Heterogeneous Sensor Networks Debraj De and Lifeng Sang Ohio State University Workshop.](https://reader036.fdocuments.in/reader036/viewer/2022062804/5697bf911a28abf838c8e8d5/html5/thumbnails/6.jpg)
Cost of Query Processing Implication Factor and Cost of Query Processing
![Page 7: QoS Supported Clustered Query Processing in Large Collaboration of Heterogeneous Sensor Networks Debraj De and Lifeng Sang Ohio State University Workshop.](https://reader036.fdocuments.in/reader036/viewer/2022062804/5697bf911a28abf838c8e8d5/html5/thumbnails/7.jpg)
Existing Work
Pipelined processing of Query Cost of Query Processing (CQP) Implication Factor (IF) IAP or Implication Aware Processing: Greedy
Algorithm for finding sub-optimal solution for order of networks to be queried.
![Page 8: QoS Supported Clustered Query Processing in Large Collaboration of Heterogeneous Sensor Networks Debraj De and Lifeng Sang Ohio State University Workshop.](https://reader036.fdocuments.in/reader036/viewer/2022062804/5697bf911a28abf838c8e8d5/html5/thumbnails/8.jpg)
Existing Work (Contd.)
Disadvantage Simplistic Cost Model Large delay for pipelined query processing No special feature for query streams
Improvement Query specific support: delay/energy/reliability Notion of QoS How to process streams of so different kinds of
queries efficiently? Efficiency – Concurrency – Fairness – Scalability
![Page 9: QoS Supported Clustered Query Processing in Large Collaboration of Heterogeneous Sensor Networks Debraj De and Lifeng Sang Ohio State University Workshop.](https://reader036.fdocuments.in/reader036/viewer/2022062804/5697bf911a28abf838c8e8d5/html5/thumbnails/9.jpg)
Contribution
A. QoS specific model for Cost of Query Processing
B. Clustered Query Processing algorithm
C. Handling stream of queries
![Page 10: QoS Supported Clustered Query Processing in Large Collaboration of Heterogeneous Sensor Networks Debraj De and Lifeng Sang Ohio State University Workshop.](https://reader036.fdocuments.in/reader036/viewer/2022062804/5697bf911a28abf838c8e8d5/html5/thumbnails/10.jpg)
A. Proposed QoS based Query Cost Model
Cost model of processing a single sensor network Notion of Quality of Service (QoS)
– Minimum energy– Maximum reliability– Minimum delay
Query is classified in CM for QoS support
![Page 11: QoS Supported Clustered Query Processing in Large Collaboration of Heterogeneous Sensor Networks Debraj De and Lifeng Sang Ohio State University Workshop.](https://reader036.fdocuments.in/reader036/viewer/2022062804/5697bf911a28abf838c8e8d5/html5/thumbnails/11.jpg)
A. Proposed QoS based Query Cost Model (contd.) Energy Consumption based cost model:
Reliability based cost model:
Delay based cost model:
![Page 12: QoS Supported Clustered Query Processing in Large Collaboration of Heterogeneous Sensor Networks Debraj De and Lifeng Sang Ohio State University Workshop.](https://reader036.fdocuments.in/reader036/viewer/2022062804/5697bf911a28abf838c8e8d5/html5/thumbnails/12.jpg)
B. Proposed Clustered Query Processing algorithm
Fully Pipelined query: large delay cost Fully Parallel query: large energy cost Compromise: something between fully pipelined and
fully parallel CHSN Graph G = (V, E, W, Y): directed graph G with
edge weight and node weight each node in V is a network fabric each node has some weight weight of each directed edge in E
is (1-Implication Factor)
![Page 13: QoS Supported Clustered Query Processing in Large Collaboration of Heterogeneous Sensor Networks Debraj De and Lifeng Sang Ohio State University Workshop.](https://reader036.fdocuments.in/reader036/viewer/2022062804/5697bf911a28abf838c8e8d5/html5/thumbnails/13.jpg)
B. Proposed Clustered Query Processing algorithm (contd.)
CHSN Graph Partitioning problem:
To determine the best partition V = P1 U P2 U P3..... U Pn, such that:
a. The sum of the weights of the edges that connect any two different partitions is
minimized.
b. For all 1 ≤ i ≤ n, |Pi| ≤ K for some fixed K.
(|Pi| = sum of weights of the vertices in the partition Pi,
K = a defined upper bound.)
![Page 14: QoS Supported Clustered Query Processing in Large Collaboration of Heterogeneous Sensor Networks Debraj De and Lifeng Sang Ohio State University Workshop.](https://reader036.fdocuments.in/reader036/viewer/2022062804/5697bf911a28abf838c8e8d5/html5/thumbnails/14.jpg)
B. Clustered Query Processing algorithm (contd.) Solution:
Constrained Graph Partitioning algorithm by Karypis and Kumar.
Advantage:
a. efficient partitioning
b. support for partition size constraint
c. fast
![Page 15: QoS Supported Clustered Query Processing in Large Collaboration of Heterogeneous Sensor Networks Debraj De and Lifeng Sang Ohio State University Workshop.](https://reader036.fdocuments.in/reader036/viewer/2022062804/5697bf911a28abf838c8e8d5/html5/thumbnails/15.jpg)
How the proposed system works
1. Query forward to CM.
2. Query classification to choose cost model.
3. Clustered query processing
4. Response aggregation
![Page 16: QoS Supported Clustered Query Processing in Large Collaboration of Heterogeneous Sensor Networks Debraj De and Lifeng Sang Ohio State University Workshop.](https://reader036.fdocuments.in/reader036/viewer/2022062804/5697bf911a28abf838c8e8d5/html5/thumbnails/16.jpg)
C. Handling stream of queries Highly efficient for processing stream of queries in
real world.
a. Clustering in soft state different concurrent clustering for independent queries.
b. Reduce cluster size less latency for processing.
c. To solve Fairness issue:
(i) blocking
(ii) popular networks in different clusters. Proposed system is flexible enough to support varied
real world issues and requirements.
![Page 17: QoS Supported Clustered Query Processing in Large Collaboration of Heterogeneous Sensor Networks Debraj De and Lifeng Sang Ohio State University Workshop.](https://reader036.fdocuments.in/reader036/viewer/2022062804/5697bf911a28abf838c8e8d5/html5/thumbnails/17.jpg)
Simulation result Total 50 sensor networks, 5 different queries. Three sets of simulation study with different
implication profiles: Normal scenario (0 ≤ IF ≤ 1) Networks with high correlation (0 ≤ IF ≤ 0.3) Networks with low correlation (0.7 ≤ IF ≤ 1)
Comparison algorithms: Parallel : completely parallel querying. Random: random order of querying. IAP: Greedy based algorithm. Cluster: proposed algorithm – implication aware
clustering and then IAP in each cluster.
![Page 18: QoS Supported Clustered Query Processing in Large Collaboration of Heterogeneous Sensor Networks Debraj De and Lifeng Sang Ohio State University Workshop.](https://reader036.fdocuments.in/reader036/viewer/2022062804/5697bf911a28abf838c8e8d5/html5/thumbnails/18.jpg)
Simulation result (contd.)
IAP is best, Parallel is worst. Cluster is worse than IAP by some margin, but way better than
Parallel.
IF [0, 1] IF [0, 0.3] IF [0.7, 1]
Energy Cost of total query
![Page 19: QoS Supported Clustered Query Processing in Large Collaboration of Heterogeneous Sensor Networks Debraj De and Lifeng Sang Ohio State University Workshop.](https://reader036.fdocuments.in/reader036/viewer/2022062804/5697bf911a28abf838c8e8d5/html5/thumbnails/19.jpg)
Simulation result (contd.)
Parallel is best, random is worst. Cluster is worse than Parallel by some margin, but way
better than others. Overall, performance of Cluster is a good compromise.
Delay Cost of total query
IF [0, 1] IF [0, 0.3] IF [0.7, 1]
![Page 20: QoS Supported Clustered Query Processing in Large Collaboration of Heterogeneous Sensor Networks Debraj De and Lifeng Sang Ohio State University Workshop.](https://reader036.fdocuments.in/reader036/viewer/2022062804/5697bf911a28abf838c8e8d5/html5/thumbnails/20.jpg)
Discussion A Hybrid Push-Pull Model will Improving performance
of query processing in each individual Network Efficient Query Injection or query diffusion across
networks for improving query processing performance
Addressing Context Awareness of Query structure: Local and Global
Wide Area Human Centric Search using Clustered Query Processing (locality based clustering)
![Page 21: QoS Supported Clustered Query Processing in Large Collaboration of Heterogeneous Sensor Networks Debraj De and Lifeng Sang Ohio State University Workshop.](https://reader036.fdocuments.in/reader036/viewer/2022062804/5697bf911a28abf838c8e8d5/html5/thumbnails/21.jpg)
Conclusion
A QoS supported Cost Model Implication-Aware clustered query processing
algorithm Flexibility and support for stream of queries
![Page 22: QoS Supported Clustered Query Processing in Large Collaboration of Heterogeneous Sensor Networks Debraj De and Lifeng Sang Ohio State University Workshop.](https://reader036.fdocuments.in/reader036/viewer/2022062804/5697bf911a28abf838c8e8d5/html5/thumbnails/22.jpg)
Questions……
THANK YOU !