Optimizing heterogeneous networks David Chu Computer Science Division EECS Department UC Berkeley...
-
date post
19-Dec-2015 -
Category
Documents
-
view
217 -
download
0
Transcript of Optimizing heterogeneous networks David Chu Computer Science Division EECS Department UC Berkeley...
![Page 1: Optimizing heterogeneous networks David Chu Computer Science Division EECS Department UC Berkeley Qualifying Exam UC Berkeley 16 October 2007.](https://reader036.fdocuments.in/reader036/viewer/2022062516/56649d2e5503460f94a06290/html5/thumbnails/1.jpg)
optimizing heterogeneous networks
David ChuComputer Science DivisionEECS DepartmentUC Berkeley
Qualifying ExamUC Berkeley
16 October 2007
![Page 2: Optimizing heterogeneous networks David Chu Computer Science Division EECS Department UC Berkeley Qualifying Exam UC Berkeley 16 October 2007.](https://reader036.fdocuments.in/reader036/viewer/2022062516/56649d2e5503460f94a06290/html5/thumbnails/2.jpg)
can a declarative approach help us cope with heterogeneous networks?
![Page 3: Optimizing heterogeneous networks David Chu Computer Science Division EECS Department UC Berkeley Qualifying Exam UC Berkeley 16 October 2007.](https://reader036.fdocuments.in/reader036/viewer/2022062516/56649d2e5503460f94a06290/html5/thumbnails/3.jpg)
3
ken: approximate data collection(chu-icde06)
sdlib: sensor data library(chu-spots06)
dsn: declarative sensor networks(chu-sensys07)
optimizing heterogeneous networks
in the field: does it help?
optimizing for adaptability and competing incentives
completed futureongoing
X
X
![Page 4: Optimizing heterogeneous networks David Chu Computer Science Division EECS Department UC Berkeley Qualifying Exam UC Berkeley 16 October 2007.](https://reader036.fdocuments.in/reader036/viewer/2022062516/56649d2e5503460f94a06290/html5/thumbnails/4.jpg)
4
outline
• declarative sensor networks
• motivation
• language & example programs
• feasibility assessment
• optimizing heterogeneous networks
• optimizing for adaptability and competing incentives
• deployment application
![Page 5: Optimizing heterogeneous networks David Chu Computer Science Division EECS Department UC Berkeley Qualifying Exam UC Berkeley 16 October 2007.](https://reader036.fdocuments.in/reader036/viewer/2022062516/56649d2e5503460f94a06290/html5/thumbnails/5.jpg)
5
heterogeneous networks
• … are proliferating
• more demands on expert engineers
• sensornets as one example…
![Page 6: Optimizing heterogeneous networks David Chu Computer Science Division EECS Department UC Berkeley Qualifying Exam UC Berkeley 16 October 2007.](https://reader036.fdocuments.in/reader036/viewer/2022062516/56649d2e5503460f94a06290/html5/thumbnails/6.jpg)
6
contextSensor Networks early experiences
![Page 7: Optimizing heterogeneous networks David Chu Computer Science Division EECS Department UC Berkeley Qualifying Exam UC Berkeley 16 October 2007.](https://reader036.fdocuments.in/reader036/viewer/2022062516/56649d2e5503460f94a06290/html5/thumbnails/7.jpg)
7
motivation
programming sensor networks is difficult
building entire sensor systems is even harder
![Page 8: Optimizing heterogeneous networks David Chu Computer Science Division EECS Department UC Berkeley Qualifying Exam UC Berkeley 16 October 2007.](https://reader036.fdocuments.in/reader036/viewer/2022062516/56649d2e5503460f94a06290/html5/thumbnails/8.jpg)
8
inspiration
data management network design
s e n s o r n e t w o r k s
![Page 9: Optimizing heterogeneous networks David Chu Computer Science Division EECS Department UC Berkeley Qualifying Exam UC Berkeley 16 October 2007.](https://reader036.fdocuments.in/reader036/viewer/2022062516/56649d2e5503460f94a06290/html5/thumbnails/9.jpg)
9
inspiration : data management• declarative is widely used in data
management• relational databases• spreadsheets• abstract “what” from “how”
• (Sensor-Network-As-Database)• [MFH05], [YG02]
![Page 10: Optimizing heterogeneous networks David Chu Computer Science Division EECS Department UC Berkeley Qualifying Exam UC Berkeley 16 October 2007.](https://reader036.fdocuments.in/reader036/viewer/2022062516/56649d2e5503460f94a06290/html5/thumbnails/10.jpg)
10
inspiration : network design• declarative is new idea in networking• compact• flexible• analyzable, optimizable• Internet Routing, Overlays built declaratively
• (the P2 project)
![Page 11: Optimizing heterogeneous networks David Chu Computer Science Division EECS Department UC Berkeley Qualifying Exam UC Berkeley 16 October 2007.](https://reader036.fdocuments.in/reader036/viewer/2022062516/56649d2e5503460f94a06290/html5/thumbnails/11.jpg)
11
inspiration
data management network design
s e n s o r n e t w o r k s
( DSN )
![Page 12: Optimizing heterogeneous networks David Chu Computer Science Division EECS Department UC Berkeley Qualifying Exam UC Berkeley 16 October 2007.](https://reader036.fdocuments.in/reader036/viewer/2022062516/56649d2e5503460f94a06290/html5/thumbnails/12.jpg)
12
what we did
• adapted declarative language
• wrote declarative examples
• built compiler & runtime for sensornets
![Page 13: Optimizing heterogeneous networks David Chu Computer Science Division EECS Department UC Berkeley Qualifying Exam UC Berkeley 16 October 2007.](https://reader036.fdocuments.in/reader036/viewer/2022062516/56649d2e5503460f94a06290/html5/thumbnails/13.jpg)
13
brief language overview
Rule2:
Fact:
Rule1:
join don’t care
Built-ins:
implies
builtin(temperature, ‘TemperatureImplementingModule.c’).
bodyTwohead bodyOne
![Page 14: Optimizing heterogeneous networks David Chu Computer Science Division EECS Department UC Berkeley Qualifying Exam UC Berkeley 16 October 2007.](https://reader036.fdocuments.in/reader036/viewer/2022062516/56649d2e5503460f94a06290/html5/thumbnails/14.jpg)
14
a full example : tree
D
S
C
D
Z
C2
SC1
![Page 15: Optimizing heterogeneous networks David Chu Computer Science Division EECS Department UC Berkeley Qualifying Exam UC Berkeley 16 October 2007.](https://reader036.fdocuments.in/reader036/viewer/2022062516/56649d2e5503460f94a06290/html5/thumbnails/15.jpg)
15
working programs
geographic routing
tracking
localization
link estimator
multi-hop collection
tree routing
trickle, in-network data aggregation, beacon vector routing,pathDCS, convex hull fallback routing, right-hand rule fallback routing, …
![Page 16: Optimizing heterogeneous networks David Chu Computer Science Division EECS Department UC Berkeley Qualifying Exam UC Berkeley 16 October 2007.](https://reader036.fdocuments.in/reader036/viewer/2022062516/56649d2e5503460f94a06290/html5/thumbnails/16.jpg)
evaluating gossip dissemination
source*borrowed picture
![Page 17: Optimizing heterogeneous networks David Chu Computer Science Division EECS Department UC Berkeley Qualifying Exam UC Berkeley 16 October 2007.](https://reader036.fdocuments.in/reader036/viewer/2022062516/56649d2e5503460f94a06290/html5/thumbnails/17.jpg)
19
… from designer’s paper [LPC04] … DSN specification
10x6 topology
30x2 topology
![Page 18: Optimizing heterogeneous networks David Chu Computer Science Division EECS Department UC Berkeley Qualifying Exam UC Berkeley 16 October 2007.](https://reader036.fdocuments.in/reader036/viewer/2022062516/56649d2e5503460f94a06290/html5/thumbnails/18.jpg)
evaluating tree-collection
2. collection messages flowtoward root
1. treeconstruction
*borrowed picture
![Page 19: Optimizing heterogeneous networks David Chu Computer Science Division EECS Department UC Berkeley Qualifying Exam UC Berkeley 16 October 2007.](https://reader036.fdocuments.in/reader036/viewer/2022062516/56649d2e5503460f94a06290/html5/thumbnails/19.jpg)
21
evaluating tree-collection
messages sent
hop-counts
(similar performance)
![Page 20: Optimizing heterogeneous networks David Chu Computer Science Division EECS Department UC Berkeley Qualifying Exam UC Berkeley 16 October 2007.](https://reader036.fdocuments.in/reader036/viewer/2022062516/56649d2e5503460f94a06290/html5/thumbnails/20.jpg)
22
lines of code
![Page 21: Optimizing heterogeneous networks David Chu Computer Science Division EECS Department UC Berkeley Qualifying Exam UC Berkeley 16 October 2007.](https://reader036.fdocuments.in/reader036/viewer/2022062516/56649d2e5503460f94a06290/html5/thumbnails/21.jpg)
23
compiled size
TelosB mote code space = 48KB, data space = 10KB
![Page 22: Optimizing heterogeneous networks David Chu Computer Science Division EECS Department UC Berkeley Qualifying Exam UC Berkeley 16 October 2007.](https://reader036.fdocuments.in/reader036/viewer/2022062516/56649d2e5503460f94a06290/html5/thumbnails/22.jpg)
24
live tracking demo (vldb06)
![Page 23: Optimizing heterogeneous networks David Chu Computer Science Division EECS Department UC Berkeley Qualifying Exam UC Berkeley 16 October 2007.](https://reader036.fdocuments.in/reader036/viewer/2022062516/56649d2e5503460f94a06290/html5/thumbnails/23.jpg)
25
architectural flexibility
• dsn can…
• describe entire system stack• application + network + mac layers
• naturally expose abstractions
• freely mix and match with outside libraries
![Page 24: Optimizing heterogeneous networks David Chu Computer Science Division EECS Department UC Berkeley Qualifying Exam UC Berkeley 16 October 2007.](https://reader036.fdocuments.in/reader036/viewer/2022062516/56649d2e5503460f94a06290/html5/thumbnails/24.jpg)
26
thoughts up to now…
• sensor networks→ data + communication
• several examples of functional programs
• feasible for today’s hardware platforms
• can explore variety of architectures…
![Page 25: Optimizing heterogeneous networks David Chu Computer Science Division EECS Department UC Berkeley Qualifying Exam UC Berkeley 16 October 2007.](https://reader036.fdocuments.in/reader036/viewer/2022062516/56649d2e5503460f94a06290/html5/thumbnails/25.jpg)
27
[current work]
optimizing heterogeneous networks
![Page 26: Optimizing heterogeneous networks David Chu Computer Science Division EECS Department UC Berkeley Qualifying Exam UC Berkeley 16 October 2007.](https://reader036.fdocuments.in/reader036/viewer/2022062516/56649d2e5503460f94a06290/html5/thumbnails/26.jpg)
28
outline• declarative sensor networks
• optimizing heterogeneous networks
• motivation
• dimensions for optimization
• execution space by example
• finding optimal
• optimizing for adaptability and competing incentives
• deployment application
![Page 27: Optimizing heterogeneous networks David Chu Computer Science Division EECS Department UC Berkeley Qualifying Exam UC Berkeley 16 October 2007.](https://reader036.fdocuments.in/reader036/viewer/2022062516/56649d2e5503460f94a06290/html5/thumbnails/27.jpg)
29
the protocol problem
• above: workloads vary
• below: greater diversity of networks
• who is going to figure out the best protocol for your situation?
• protocol optimization has a long history [CDO97], [VLC88], [HA89],…
![Page 28: Optimizing heterogeneous networks David Chu Computer Science Division EECS Department UC Berkeley Qualifying Exam UC Berkeley 16 October 2007.](https://reader036.fdocuments.in/reader036/viewer/2022062516/56649d2e5503460f94a06290/html5/thumbnails/28.jpg)
30
implications of declarative
• concise “what” programming
• 1 specification,N possible execution plans
ü
?
![Page 29: Optimizing heterogeneous networks David Chu Computer Science Division EECS Department UC Berkeley Qualifying Exam UC Berkeley 16 October 2007.](https://reader036.fdocuments.in/reader036/viewer/2022062516/56649d2e5503460f94a06290/html5/thumbnails/29.jpg)
31
Planner
Environment- Workload- Network
characteristicsCost
criteria
Optimized execution strategy
Network protocol
specification
![Page 30: Optimizing heterogeneous networks David Chu Computer Science Division EECS Department UC Berkeley Qualifying Exam UC Berkeley 16 October 2007.](https://reader036.fdocuments.in/reader036/viewer/2022062516/56649d2e5503460f94a06290/html5/thumbnails/30.jpg)
32
manually engineering solutions
• packet filtering• offload to resource-rich gateways
• content distribution networks• manual configuration of distribution points
• session state• explicitly choose stateless variants when critical
• quality of service• choose tradeoff between node state or packet state
• routing• domain-specific protocols e.g. DSR, 6lowpan
• distributed event detection• configure detector placement
designparameters:state,rendezvous?
![Page 31: Optimizing heterogeneous networks David Chu Computer Science Division EECS Department UC Berkeley Qualifying Exam UC Berkeley 16 October 2007.](https://reader036.fdocuments.in/reader036/viewer/2022062516/56649d2e5503460f94a06290/html5/thumbnails/31.jpg)
34
toward automated solutions• identify possible executions of program
• search execution space for optimal plan
• but first some rough idea of what we’re after…
Planner
![Page 32: Optimizing heterogeneous networks David Chu Computer Science Division EECS Department UC Berkeley Qualifying Exam UC Berkeley 16 October 2007.](https://reader036.fdocuments.in/reader036/viewer/2022062516/56649d2e5503460f94a06290/html5/thumbnails/32.jpg)
35
packet filter program
% forwarding rulemessage(@Next,Src,Dest,Load) :-
message(@Crt,Src,Dest,Load), nexthop(@Crt,Dest,Next).
% firewall rulereceive(@Crt,Src,Dest,Load) :- message(@Crt,Src,Dest,Load),
Crt == Dest, firewall(@Crt,Load).
% queryreceive(@Crt,Src,Dest,Load)?
recursive relation
base relation
non-recursive relation
![Page 33: Optimizing heterogeneous networks David Chu Computer Science Division EECS Department UC Berkeley Qualifying Exam UC Berkeley 16 October 2007.](https://reader036.fdocuments.in/reader036/viewer/2022062516/56649d2e5503460f94a06290/html5/thumbnails/33.jpg)
36
d
f
ba
c
e
g
nexthopd b
nexthopd b
nexthopf d
nexthopb a
nexthopg c
nexthope d
base relation as table partitioned across nodes
![Page 34: Optimizing heterogeneous networks David Chu Computer Science Division EECS Department UC Berkeley Qualifying Exam UC Berkeley 16 October 2007.](https://reader036.fdocuments.in/reader036/viewer/2022062516/56649d2e5503460f94a06290/html5/thumbnails/34.jpg)
37
d
f
ba
c
e
g
base relation as network graph
![Page 35: Optimizing heterogeneous networks David Chu Computer Science Division EECS Department UC Berkeley Qualifying Exam UC Berkeley 16 October 2007.](https://reader036.fdocuments.in/reader036/viewer/2022062516/56649d2e5503460f94a06290/html5/thumbnails/35.jpg)
39
d
f
ba
c
e
g
firewall
message
receive
![Page 36: Optimizing heterogeneous networks David Chu Computer Science Division EECS Department UC Berkeley Qualifying Exam UC Berkeley 16 October 2007.](https://reader036.fdocuments.in/reader036/viewer/2022062516/56649d2e5503460f94a06290/html5/thumbnails/36.jpg)
40M N
M N
M N
M
A
E
D
B
msg() facts
nexthop() facts
![Page 37: Optimizing heterogeneous networks David Chu Computer Science Division EECS Department UC Berkeley Qualifying Exam UC Berkeley 16 October 2007.](https://reader036.fdocuments.in/reader036/viewer/2022062516/56649d2e5503460f94a06290/html5/thumbnails/37.jpg)
41
r
f
A
E
D
B
![Page 38: Optimizing heterogeneous networks David Chu Computer Science Division EECS Department UC Berkeley Qualifying Exam UC Berkeley 16 October 2007.](https://reader036.fdocuments.in/reader036/viewer/2022062516/56649d2e5503460f94a06290/html5/thumbnails/38.jpg)
42
r
f
A
E
D
B
![Page 39: Optimizing heterogeneous networks David Chu Computer Science Division EECS Department UC Berkeley Qualifying Exam UC Berkeley 16 October 2007.](https://reader036.fdocuments.in/reader036/viewer/2022062516/56649d2e5503460f94a06290/html5/thumbnails/39.jpg)
43
d
f
ba
c
e
g
8
9
10
1
29
firewall
message
receive
varying join selectivity shifts the optimal rendezvous
1: Meet-In-The-Middle (MiM) Rewrite
applications: packet filtering, event detection, CDNs
message
message
![Page 40: Optimizing heterogeneous networks David Chu Computer Science Division EECS Department UC Berkeley Qualifying Exam UC Berkeley 16 October 2007.](https://reader036.fdocuments.in/reader036/viewer/2022062516/56649d2e5503460f94a06290/html5/thumbnails/40.jpg)
44
d
f
ba
c
e
g
firewall
messagemess age+ =
2: Query Scramble Rewrite
receive
HiResVideo EventDetectorFlag
applications: query scrambling, mitigating latency X bandwidth
![Page 41: Optimizing heterogeneous networks David Chu Computer Science Division EECS Department UC Berkeley Qualifying Exam UC Berkeley 16 October 2007.](https://reader036.fdocuments.in/reader036/viewer/2022062516/56649d2e5503460f94a06290/html5/thumbnails/41.jpg)
45
d
f
ba
c
e
g
firewall
message
fire wall+=
receive
3: Semijoin Rewrite
application: semijoin
allowed options
allowedports
![Page 42: Optimizing heterogeneous networks David Chu Computer Science Division EECS Department UC Berkeley Qualifying Exam UC Berkeley 16 October 2007.](https://reader036.fdocuments.in/reader036/viewer/2022062516/56649d2e5503460f94a06290/html5/thumbnails/42.jpg)
46
d
f
ba
c
e
g
firewall
message
receive
connections,
Two separate relations at endpoint.
3b: Traditional Join Placement
application: semijoin
![Page 43: Optimizing heterogeneous networks David Chu Computer Science Division EECS Department UC Berkeley Qualifying Exam UC Berkeley 16 October 2007.](https://reader036.fdocuments.in/reader036/viewer/2022062516/56649d2e5503460f94a06290/html5/thumbnails/43.jpg)
47
d
f
ba
c
e
g
![Page 44: Optimizing heterogeneous networks David Chu Computer Science Division EECS Department UC Berkeley Qualifying Exam UC Berkeley 16 October 2007.](https://reader036.fdocuments.in/reader036/viewer/2022062516/56649d2e5503460f94a06290/html5/thumbnails/44.jpg)
49
d
f
ba
c
e
g
4: Pullback Rewrite
application: alternate between DVR and SR
nexthopd b
nexthopd b
nexthopd bb a
nexthopd bb a
nexthopf d
nexthopb a
nexthopg c
nexthope d
nexthopg cd bb a
![Page 45: Optimizing heterogeneous networks David Chu Computer Science Division EECS Department UC Berkeley Qualifying Exam UC Berkeley 16 October 2007.](https://reader036.fdocuments.in/reader036/viewer/2022062516/56649d2e5503460f94a06290/html5/thumbnails/45.jpg)
50
d
f
ba
c
e
g
4: Pullback Rewrite
message
application: alternate between DVR and SR
nexthopd bb a
nexthopg cd bb a
![Page 46: Optimizing heterogeneous networks David Chu Computer Science Division EECS Department UC Berkeley Qualifying Exam UC Berkeley 16 October 2007.](https://reader036.fdocuments.in/reader036/viewer/2022062516/56649d2e5503460f94a06290/html5/thumbnails/46.jpg)
51
routing layer state proxying
Sensornet
Internetnexthop forwarding table
D
C A
B
source route to Ddistance vector routing
A: D via B
B: D via C
C: D via D C: D via D
4: Pullback Rewrite example
![Page 47: Optimizing heterogeneous networks David Chu Computer Science Division EECS Department UC Berkeley Qualifying Exam UC Berkeley 16 October 2007.](https://reader036.fdocuments.in/reader036/viewer/2022062516/56649d2e5503460f94a06290/html5/thumbnails/47.jpg)
52
ABCL
M’
M
ABC
L’
M,L
M’,L’
ABCM
L
L’M’,L’
M,L
M’
M,L
M
5: SessionState Rewrite
regular
proxy
stateless
![Page 48: Optimizing heterogeneous networks David Chu Computer Science Division EECS Department UC Berkeley Qualifying Exam UC Berkeley 16 October 2007.](https://reader036.fdocuments.in/reader036/viewer/2022062516/56649d2e5503460f94a06290/html5/thumbnails/48.jpg)
53
…
M
M
M
M’
M’
A
B
C
B
L
L’
5: SessionState Rewrite
application: pushing state into packets/proxies
![Page 49: Optimizing heterogeneous networks David Chu Computer Science Division EECS Department UC Berkeley Qualifying Exam UC Berkeley 16 October 2007.](https://reader036.fdocuments.in/reader036/viewer/2022062516/56649d2e5503460f94a06290/html5/thumbnails/49.jpg)
54
…
A
B
C
B
M,L
M,L
M’,L’
M’,L’
M,L L0
5: SessionState Rewrite
application: pushing state into packets/proxies
![Page 50: Optimizing heterogeneous networks David Chu Computer Science Division EECS Department UC Berkeley Qualifying Exam UC Berkeley 16 October 2007.](https://reader036.fdocuments.in/reader036/viewer/2022062516/56649d2e5503460f94a06290/html5/thumbnails/50.jpg)
55
…
M
M
M’
A
B
C
B
L
L’
M,L
M,L
M’,L’
5: SessionState Rewrite
application: pushing state into packets/proxies
prefetch read
defer write
![Page 51: Optimizing heterogeneous networks David Chu Computer Science Division EECS Department UC Berkeley Qualifying Exam UC Berkeley 16 October 2007.](https://reader036.fdocuments.in/reader036/viewer/2022062516/56649d2e5503460f94a06290/html5/thumbnails/51.jpg)
56
summary of rewrite family
![Page 52: Optimizing heterogeneous networks David Chu Computer Science Division EECS Department UC Berkeley Qualifying Exam UC Berkeley 16 October 2007.](https://reader036.fdocuments.in/reader036/viewer/2022062516/56649d2e5503460f94a06290/html5/thumbnails/52.jpg)
57
example from dsn paper:before rewrite% Sample temperature and initiate multihop sendtransmit (@Src, Temperature ) :− thermometer (@Src, Temperature ) , timer (@Src,
collectionTimer , Period ) .
% Prepare message for multihop transmissionmessage (@Src, Src , Dst , Data ) :− transmit (@Src, Data ) , nextHop (@Src, Dst , Next ,
Cost ) ˜ .
% Forward message to next hop parentmessage (@Next, Src , Dst , Data ) :− message (@Crt , Src , Dst , Data ) , nextHop (@Crt
, Dst , Next , Cost ) ˜ , Cr t != Dst .
% Receive when at destinationalert(@Crt , Src , Data ) :− message (@Crt , Src , Dst , Data ) , Crt==Dst, anomaly(@Crt,
Thresh), Data > Thresh.
2
1
3
4
recursive relation
base relation
non-recursive relation
![Page 53: Optimizing heterogeneous networks David Chu Computer Science Division EECS Department UC Berkeley Qualifying Exam UC Berkeley 16 October 2007.](https://reader036.fdocuments.in/reader036/viewer/2022062516/56649d2e5503460f94a06290/html5/thumbnails/53.jpg)
58
message(),nexthop(),nexthop(),…, nexthop(),…, nexthop(),anomaly()
rendezvous()
alert()
![Page 54: Optimizing heterogeneous networks David Chu Computer Science Division EECS Department UC Berkeley Qualifying Exam UC Berkeley 16 October 2007.](https://reader036.fdocuments.in/reader036/viewer/2022062516/56649d2e5503460f94a06290/html5/thumbnails/54.jpg)
59
example from DSN paper:after rewrite% Sample temperature and initiate multihop sendtransmit (@Src, Temperature ) :− thermometer (@Src, Temperature ) , timer (@Src, collectionTimer , Period ) .
% Prepare message for multihop transmissionmessage (@Src, Src , Dst , Data ) :− transmit (@Src, Data ) , nextHop (@Src, Dst , Next , Cost ) ˜ .
% Forward message to next hop parentmessage (@Next, Src , Dst , Data ) :− message (@Crt , Src , Dst , Data ) , nextHop (@Crt, Dst , Next , Cost ) ˜ , Cr t != Dst, -
rendezvous’(@Crt).
% Receive when at destination optimal (middle) pointalert’(@Crt , Crt_, Src , Data ) :− message (@Crt , Src , Dst , Data ) , Crt==Dst, anomaly’(@Crt, Crt_, Thresh_), Data > Thresh,
rendezvous (@Crt).
% Move anomaly’() backwardanomaly’(@Crt, Crt, Thresh) :- anomaly(@Crt,Thresh).anomaly’(@Prev, Crt, Thresh) :- anomaly(@Crt, Crt_, Thresh), nextHop(@Prev, Crt_, Crt, Cost), -rendezvous (@Crt).
% Auxilliary rules to move alert’() to alert()alert’(@Next, Crt_, Src, Data) :- alert’(Crt, Crt_, Src, Data), nextHop(@Crt, Crt_, Next, Cost).alert(@Crt, Src, Data) :- alert’(@Crt, Crt, Src, Data).
2
1
3
4
5
6
rendezvous
rendezvous rendezvou
s
![Page 55: Optimizing heterogeneous networks David Chu Computer Science Division EECS Department UC Berkeley Qualifying Exam UC Berkeley 16 October 2007.](https://reader036.fdocuments.in/reader036/viewer/2022062516/56649d2e5503460f94a06290/html5/thumbnails/55.jpg)
60
finding optimal
• goal: generate optimal rendezvous facts
• get statistics for both workload and resources• synopses: well-understood database summarization
technique• relevant statistics: join selectivity, size of tables, nexthop
distribution, link costs, storage costs
• run optimization• currently custom optimizer algorithms for each rewrite
(rewrites are still general)• should use general purpose dynamic programming optimizer• in both cases, algorithms are exhaustive
![Page 56: Optimizing heterogeneous networks David Chu Computer Science Division EECS Department UC Berkeley Qualifying Exam UC Berkeley 16 October 2007.](https://reader036.fdocuments.in/reader036/viewer/2022062516/56649d2e5503460f94a06290/html5/thumbnails/56.jpg)
61
pullback rewrite: early resultssimulation35 nodes1 sender30 receivers1000 e2e sends
utilize available resource
account for workload
DVR inactive until enough table space for
all receivers.
![Page 57: Optimizing heterogeneous networks David Chu Computer Science Division EECS Department UC Berkeley Qualifying Exam UC Berkeley 16 October 2007.](https://reader036.fdocuments.in/reader036/viewer/2022062516/56649d2e5503460f94a06290/html5/thumbnails/57.jpg)
62
planned experiment 1MiM for event detection
• setup• 30 node sensor network• all nodes send samples via collection tree to
root• root maintains list of anomaly thresholds
• hypothesis• MiM optimization lowers total energy
consumption by identifying optimal tree depth to which to push down anomaly thresholds
![Page 58: Optimizing heterogeneous networks David Chu Computer Science Division EECS Department UC Berkeley Qualifying Exam UC Berkeley 16 October 2007.](https://reader036.fdocuments.in/reader036/viewer/2022062516/56649d2e5503460f94a06290/html5/thumbnails/58.jpg)
63
planned experiment 2SessionState for web server
• setup• 1 server, X clients in P2• each client connects with server with stateful
protocol
• hypothesis• SessionState optimization increases the
number of clients that can connect at overall lower messaging cost by giving “stateful” priority to more frequently connecting clients
![Page 59: Optimizing heterogeneous networks David Chu Computer Science Division EECS Department UC Berkeley Qualifying Exam UC Berkeley 16 October 2007.](https://reader036.fdocuments.in/reader036/viewer/2022062516/56649d2e5503460f94a06290/html5/thumbnails/59.jpg)
status of current work
• captured design/execution space:• rendezvous and state
• identified ways to expose possible executions• MiM Family of Rewrites
• implemented first cut planner
• todo• implement generic rewriter• implement generic optimizer• prove semantic invariance w.r.t. queried relation• evaluate planner on test scenarios
64
![Page 60: Optimizing heterogeneous networks David Chu Computer Science Division EECS Department UC Berkeley Qualifying Exam UC Berkeley 16 October 2007.](https://reader036.fdocuments.in/reader036/viewer/2022062516/56649d2e5503460f94a06290/html5/thumbnails/60.jpg)
65
[future work]
optimizing for adaptability and competing incentives
![Page 61: Optimizing heterogeneous networks David Chu Computer Science Division EECS Department UC Berkeley Qualifying Exam UC Berkeley 16 October 2007.](https://reader036.fdocuments.in/reader036/viewer/2022062516/56649d2e5503460f94a06290/html5/thumbnails/61.jpg)
66
outline• declarative sensor networks
• optimizing heterogeneous networks
• optimizing for adaptability and competing incentives
• why adaptability?
• potential optimization for adaptability
• why competing incentives?
• potential optimization for competing incentives
• deployment application
![Page 62: Optimizing heterogeneous networks David Chu Computer Science Division EECS Department UC Berkeley Qualifying Exam UC Berkeley 16 October 2007.](https://reader036.fdocuments.in/reader036/viewer/2022062516/56649d2e5503460f94a06290/html5/thumbnails/62.jpg)
67
optimize for best case performance?• environment changes quickly
optimizer run infrequently
• partially observable environment e.g. imperfect synopses
• delayed visibility of environment
• adaptability as theme of DB research in last decade [DIR07], [KD98], [IFF+99], [AH00]
![Page 63: Optimizing heterogeneous networks David Chu Computer Science Division EECS Department UC Berkeley Qualifying Exam UC Berkeley 16 October 2007.](https://reader036.fdocuments.in/reader036/viewer/2022062516/56649d2e5503460f94a06290/html5/thumbnails/63.jpg)
68
options for adaptability
• iterating traditional optimizer more often [KD98]• PRO: natural port of algorithms and semantics• CON: state may still be obscured• CON: synopsis collection overhead increases
• run-time adaptability [IFF+99], [AH00]• PRO: continuous distributed planning• CON: unclear what global semantics are achieved
![Page 64: Optimizing heterogeneous networks David Chu Computer Science Division EECS Department UC Berkeley Qualifying Exam UC Berkeley 16 October 2007.](https://reader036.fdocuments.in/reader036/viewer/2022062516/56649d2e5503460f94a06290/html5/thumbnails/64.jpg)
69
adaptive rewrite for latency
% Sample temperature and initiate multihop sendtransmit (@Src, Temperature ) :− thermometer (@Src, Temperature ) , timer (@Src, collectionTimer , Period ) .
% Prepare message for multihop transmissionmessage (@Src, Src , Dst , Data ) :− transmit (@Src, Data ) , nextHop (@Src, Dst , Next , Cost ) ˜ .
% Forward message to next hop parentmessage (@Next, Src , Dst , Data ) :− message (@Crt , Src , Dst , Data ) , nextHop (@Crt, Dst , Next , Cost ) ˜ , Cr t != Dst, -
anomaly’(@Crt, Crt_, Thresh_).
% Receive when at destination optimal (middle) pointalert’(@Crt , Crt_, Src , Data ) :− message (@Crt , Src , Dst , Data ) , Crt==Dst, anomaly’(@Crt, Crt_, Thresh_), Data > Thresh.
% Move anomaly’() backwardanomaly’(@Crt, Crt, Thresh) :- anomaly(@Crt,Thresh).anomaly’(@Prev, Crt, Thresh) :- anomaly(@Crt, Crt_, Thresh), nextHop(@Prev, Crt_, Crt, Cost),
-message(@Crt, Src, Crt_, Data).
% Auxilliary rules to move alert’() to alert()alert’(@Next, Crt_, Src, Data) :- alert’(Crt, Crt_, Src, Data), nextHop(@Crt, Crt_, Next, Cost).alert(@Crt, Src, Data) :- alert’(@Crt, Crt, Src, Data).
2
1
3
4
5
6
dynamic rendezvous
dynamic rendezvous
![Page 65: Optimizing heterogeneous networks David Chu Computer Science Division EECS Department UC Berkeley Qualifying Exam UC Berkeley 16 October 2007.](https://reader036.fdocuments.in/reader036/viewer/2022062516/56649d2e5503460f94a06290/html5/thumbnails/65.jpg)
70
global query plan
m
n
f
r
J1 J2 J3
![Page 66: Optimizing heterogeneous networks David Chu Computer Science Division EECS Department UC Berkeley Qualifying Exam UC Berkeley 16 October 2007.](https://reader036.fdocuments.in/reader036/viewer/2022062516/56649d2e5503460f94a06290/html5/thumbnails/66.jpg)
71
derivation graph
m
n
f
r
Original derivationAugmented derivationDashed represents negation
no unique model(multiple possible outcomes)is well-known
![Page 67: Optimizing heterogeneous networks David Chu Computer Science Division EECS Department UC Berkeley Qualifying Exam UC Berkeley 16 October 2007.](https://reader036.fdocuments.in/reader036/viewer/2022062516/56649d2e5503460f94a06290/html5/thumbnails/67.jpg)
72
potential experiment 1
• setup• 30 node event detection sensor network (similar to
previously planned experiment)• use MiM rewrite in both “best-case” and “adaptible”
mode to find optimal rendezvous
• hypothesis• if anomaly list does not change frequently, “best-
case” performs better• if anomaly list does change frequently, “adaptible”
performs better
![Page 68: Optimizing heterogeneous networks David Chu Computer Science Division EECS Department UC Berkeley Qualifying Exam UC Berkeley 16 October 2007.](https://reader036.fdocuments.in/reader036/viewer/2022062516/56649d2e5503460f94a06290/html5/thumbnails/68.jpg)
73
potential experiment 2
• setup• X clients, 1 server (similar to previously
planned experiment)• relax synopsis accuracy assumptions e.g. most
frequent clients (table distribution) not accurately reported
• hypothesis• as accuracy of synopses decrease, “adaptible”
performs better than “best-case”
![Page 69: Optimizing heterogeneous networks David Chu Computer Science Division EECS Department UC Berkeley Qualifying Exam UC Berkeley 16 October 2007.](https://reader036.fdocuments.in/reader036/viewer/2022062516/56649d2e5503460f94a06290/html5/thumbnails/69.jpg)
74
optimizing for competing incentives• assumed single administrative domain
• sensornets• corporate CDNs• individual ASs
• competing incentives• privately-owned servers & hosts• ISP peering relationships• multiple sensornet queries
• individuals have incentive to deviate from global optimal [MRW+04], [SDK+94]
![Page 70: Optimizing heterogeneous networks David Chu Computer Science Division EECS Department UC Berkeley Qualifying Exam UC Berkeley 16 October 2007.](https://reader036.fdocuments.in/reader036/viewer/2022062516/56649d2e5503460f94a06290/html5/thumbnails/70.jpg)
75
d
f
ba
c
e
g
8
9
10
1
29
firewall
message
receive
3: Semijoin Rewrite
connections,
Two separate relations at endpoint.
PINRELATION(connections)
![Page 71: Optimizing heterogeneous networks David Chu Computer Science Division EECS Department UC Berkeley Qualifying Exam UC Berkeley 16 October 2007.](https://reader036.fdocuments.in/reader036/viewer/2022062516/56649d2e5503460f94a06290/html5/thumbnails/71.jpg)
76
d
f
ba
c
e
g
8
9
10
1
29
messageDomain 1
Domain 2
private plan 1
private plan 2
![Page 72: Optimizing heterogeneous networks David Chu Computer Science Division EECS Department UC Berkeley Qualifying Exam UC Berkeley 16 October 2007.](https://reader036.fdocuments.in/reader036/viewer/2022062516/56649d2e5503460f94a06290/html5/thumbnails/72.jpg)
77
[application]
large scale and fine-graineddebris flow monitoring
![Page 73: Optimizing heterogeneous networks David Chu Computer Science Division EECS Department UC Berkeley Qualifying Exam UC Berkeley 16 October 2007.](https://reader036.fdocuments.in/reader036/viewer/2022062516/56649d2e5503460f94a06290/html5/thumbnails/73.jpg)
78
outline
• declarative sensor networks
• optimizing heterogeneous networks
• optimizing for adaptability and competing incentives
• deployment application
• geological study
• declarative’s role
![Page 74: Optimizing heterogeneous networks David Chu Computer Science Division EECS Department UC Berkeley Qualifying Exam UC Berkeley 16 October 2007.](https://reader036.fdocuments.in/reader036/viewer/2022062516/56649d2e5503460f94a06290/html5/thumbnails/74.jpg)
79
[Left] La Conchita, California – a small seaside community along Highway 101 south of Santa Barbara. This landslide and debris flow occurred in the spring of 1995. A reoccurrence in 2005 claimed 4 lives and resulted in 29 missing persons. [Right] Chehalis, Washington - landslides and debris flows during the winter storms of February 1996. Photographs by R.L. Schuster, U.S. Geological Survey.
![Page 75: Optimizing heterogeneous networks David Chu Computer Science Division EECS Department UC Berkeley Qualifying Exam UC Berkeley 16 October 2007.](https://reader036.fdocuments.in/reader036/viewer/2022062516/56649d2e5503460f94a06290/html5/thumbnails/75.jpg)
80
[Above] The locations of the 2005-2006 and 2006-2007 debris flow deployment sites.[Top Right] Smoke from the Day Fire. [Middle Right] Recently burned hillside in Burbank, CA was the site of two debris flows in 2005-2006 Winter season. [Bottom Right] Base of the channel after debris flow with remaining sediment. [Bottom Left] Burn-resilient vegetation is quickly recovering just a few months after the fires and debris flows.
Harvard Burn Site
Day Fire
![Page 76: Optimizing heterogeneous networks David Chu Computer Science Division EECS Department UC Berkeley Qualifying Exam UC Berkeley 16 October 2007.](https://reader036.fdocuments.in/reader036/viewer/2022062516/56649d2e5503460f94a06290/html5/thumbnails/76.jpg)
82
[Above] Parshall flume used in conjunction with water level logger at the channel’s choke-point. [Top Right] Custom overland flow sensor for fine-grained detection of water runoff. [Bottom Right] Solar-powered base station for actuating and gathering data from the wireless sensor network, shown here connected to laptop during testing.
![Page 77: Optimizing heterogeneous networks David Chu Computer Science Division EECS Department UC Berkeley Qualifying Exam UC Berkeley 16 October 2007.](https://reader036.fdocuments.in/reader036/viewer/2022062516/56649d2e5503460f94a06290/html5/thumbnails/77.jpg)
83
declarative sensornetsin context
• how easy is managing sensornets with declarative programming?
• are the network optimizations useful in helping out a real deployment?• event detection• network management
![Page 78: Optimizing heterogeneous networks David Chu Computer Science Division EECS Department UC Berkeley Qualifying Exam UC Berkeley 16 October 2007.](https://reader036.fdocuments.in/reader036/viewer/2022062516/56649d2e5503460f94a06290/html5/thumbnails/78.jpg)
84
timeline• Fa07:
• complete current work on Optimizing Heterogeneous Networks• implement generic rewriter• implement generic optimizations (concurrent work on optimization framework)• prove rewrite correctness• fully evaluate net planner
• help with USGS 1st deployment• continue engineering and testing hardware and software for 1st round deployment• deploy in selected site, help with issues as they arise
• Sp08+Fa08:• start and complete future work on Optimizing for Adaptability/Competing Incentives
• understand related work, decide on which course to pursue• design and implement generic rewrites• decide which platform(s) to use• augment runtime for runtime decision making
• help with USGS 2nd deployment
• Sp09:• write thesis
![Page 79: Optimizing heterogeneous networks David Chu Computer Science Division EECS Department UC Berkeley Qualifying Exam UC Berkeley 16 October 2007.](https://reader036.fdocuments.in/reader036/viewer/2022062516/56649d2e5503460f94a06290/html5/thumbnails/79.jpg)
85
thanks
collaborators
Joe Hellerstein, Scott Shenker, Ion Stoica,
Lucian Popa, Arsalan Tavakoli
Tsung-Te Lai
Phil Levis, Jung Woo Lee, Aby John,Kevin Klues
Daniel Malmon, Joel Johnson
![Page 80: Optimizing heterogeneous networks David Chu Computer Science Division EECS Department UC Berkeley Qualifying Exam UC Berkeley 16 October 2007.](https://reader036.fdocuments.in/reader036/viewer/2022062516/56649d2e5503460f94a06290/html5/thumbnails/80.jpg)
references• [MFH05] Samuel Madden, Michael J. Franklin, Joseph M. Hellerstein, Wei Hong.
TinyDB: an acquisitional query processing system for sensor networks. ACM Transactions on Database Systems (TODS), Volume 30 , Issue 1, March 2005.
• [YG02] Yong Yao, J. E. Gehrke . The Cougar Approach to In-Network Query Processing in Sensor Networks. Sigmod Record, Volume 31, Number 3. September 2002.
• [LPC04] Philip Levis, Neil Patel, David Culler, and Scott Shenker. Trickle: A Self-Regulating Algorithm for Code Propagation and Maintenance in Wireless Sensor Networks. In Proceedings of the First USENIX/ACM Symposium on Networked Systems Design and Implementation, 2004.
• [DIR07] Amol Deshpande, Zachary Ives, Vijayshankar Raman. Adaptive Query Processing. Foundations and Trends in Databases: Vol. 1: No 1, pp 1-140, 2007.
• [CDO97] C. Castelluccia, W. Dabbous, and S. O’Malley. Generating efficient protocol code from an abstract specification. IEEE/ACM Trans. Netw., 5(4):514–524, 1997.
• [VLC88] S. T. Vuong, A. C. Lau, and R. I. Chan. Semiautomatic implementation of protocols using an estelle-c compiler. IEEE Trans. Softw. Eng., 14(3):384–393, 1988.
• [HA89] D. Hernek and D. P. Anderson. Efficient automated protocol implementation using rtag. Technical Report UCB/CSD-89-526, EECS Department, University of California, Berkeley, Aug 1989.
![Page 81: Optimizing heterogeneous networks David Chu Computer Science Division EECS Department UC Berkeley Qualifying Exam UC Berkeley 16 October 2007.](https://reader036.fdocuments.in/reader036/viewer/2022062516/56649d2e5503460f94a06290/html5/thumbnails/81.jpg)
references (2)• [KD98] N. Kabra and D. J. DeWitt. Efficient mid-query re-optimization of suboptimal
query execution plans. In SIGMOD ’98: Proceedings of the 1998 ACM SIGMOD international conference on Management of data, (New York, NY, USA), pp. 106–117, ACM Press, 1998.
• [IFF+99] Z. G. Ives, D. Florescu, M. Friedman, A. Levy, and D. S. Weld, “An adaptive query execution system for data integration,” in SIGMOD ’99: Proceedings of the 1999 ACM SIGMOD international conference on Management of data, (New York, NY, USA), pp. 299–310, ACM Press, 1999.
• [AH00] R. Avnur and J. M. Hellerstein, “Eddies: continuously adaptive query processing,” in SIGMOD ’00: Proceedings of the 2000 ACM SIGMOD international conference on Management of data, (New York, NY, USA), pp. 261–272, ACM Press, 2000.
• [MRW+04] R. Mahajan, M. Rodrig, D. Wetherall and J. Zahorjan. Experiences Applying Game Theory to System Design. SIGCOMM 2004 Workshop on Practice and Theory of Incentives and Game Theory in Networked Systems (PINS), Portland, OR, August 2004.
• [SDK+94] Michael Stonebraker, Robert Devine, Marcel Kornacker, Witold Litwin, Avi Pfeffer, Adam Sah, and Carl Staelin. An Economic Paradigm for Query Processing and Data Migration in Mariposa, Sequoia 2000 Technical Report 94/49, University of California, Berkeley, CA, Apr. 1994.
![Page 82: Optimizing heterogeneous networks David Chu Computer Science Division EECS Department UC Berkeley Qualifying Exam UC Berkeley 16 October 2007.](https://reader036.fdocuments.in/reader036/viewer/2022062516/56649d2e5503460f94a06290/html5/thumbnails/82.jpg)
88
backup slides
![Page 83: Optimizing heterogeneous networks David Chu Computer Science Division EECS Department UC Berkeley Qualifying Exam UC Berkeley 16 October 2007.](https://reader036.fdocuments.in/reader036/viewer/2022062516/56649d2e5503460f94a06290/html5/thumbnails/83.jpg)
93
evaluation
![Page 84: Optimizing heterogeneous networks David Chu Computer Science Division EECS Department UC Berkeley Qualifying Exam UC Berkeley 16 October 2007.](https://reader036.fdocuments.in/reader036/viewer/2022062516/56649d2e5503460f94a06290/html5/thumbnails/84.jpg)
95
a declarative architecture• why rethink the architecture?
• disparate application requirements
• breaking of traditional abstraction boundaries
• what are the implications?
• architectural flexibility is essential
• put resource management in user’s hands
![Page 85: Optimizing heterogeneous networks David Chu Computer Science Division EECS Department UC Berkeley Qualifying Exam UC Berkeley 16 October 2007.](https://reader036.fdocuments.in/reader036/viewer/2022062516/56649d2e5503460f94a06290/html5/thumbnails/85.jpg)
96
resource management
• memory
• processor
• energy
![Page 86: Optimizing heterogeneous networks David Chu Computer Science Division EECS Department UC Berkeley Qualifying Exam UC Berkeley 16 October 2007.](https://reader036.fdocuments.in/reader036/viewer/2022062516/56649d2e5503460f94a06290/html5/thumbnails/86.jpg)
97
session state program
• % include “forwarding rule”
• % respond to message• msg(@Crt,Dest,Src,NewLoad) :-
msg(@Crt,Src,Dest,Load), Crt == Dest, NewLoad = modify_load(Load).
• % local state update• local(@Crt,NewState) :- msg(@Crt,Dest,Load),
Crt == Dest, local(State), NewState = modify_state(State,Load).
5: SessionState Rewrite