Sterbenz, et al.ITTCWeather Disruption-Tolerant
Self-Optimising Millimeter Mesh Networks:Architecture, Routing Protocols, Performance
22 April 2009
James P.G. Sterbenz,Abdul Jabbar, Justin Rohrer, Egemen Çetinkaya,
Bharatwajan Raman, Victor Frost
Department of Electrical Engineering & Computer ScienceInformation Technology & Telecommunications Research Center
The University of Kansas
http://www.ittc.ku.edu/~jpgshttp://wiki.ittc.ku.edu/resilinets
© 2007–2009 Sterbenz
22 April 2009 Weather Disruption-Tolerant Mesh Networks 2
Sterbenz, et al.ITTC
WDTN Mesh NetworksAbstract
With growing demand for high-speed access to mobile handheld devices, there is a significant cost benefit in deploying fixed wireless-mesh networks for backhaul access. However, enabling reliable broadband access over high-frequency radios (such as millimeter-wave networks) posses a fundamental challenge due to weather disruptions in general and rain attenuation in particular. In this paper, we present an analysis of the impact of precipitation on millimeter-wave mesh networks based on radar measurements of real storms in the Midwest US. Furthermore, we compare two novel algorithms that use physical-layer information to optimize routing at the network layer: P-WARP (Predictive Weather-Assisted Routing Protocol) and XL-OSPF (Cross-Layered Open Shortest Path First). Finally, we present simulation studies to compare the performance of the proposed protocols and evaluate the dependability of the end-user service during weather disruptions.
22 April 2009 Weather Disruption-Tolerant Mesh Networks 3
Sterbenz, et al.ITTC
WDTN Mesh NetworksOutline
• Introduction and motivation• Millimeter-wave mesh networks• Impact of weather disruptions• WDTN algorithms• Simulation model and performance analysis
22 April 2009 Weather Disruption-Tolerant Mesh Networks 4
Sterbenz, et al.ITTC
MotivationWireless vs. Fiber Optic Links
• Wireless data access for untethered network access– supported by wired backbone– fiber optic cables provide high-speed reliable connections⇒ increasingly wireless access to an optical core (MAN+WAN )
22 April 2009 Weather Disruption-Tolerant Mesh Networks 5
Sterbenz, et al.ITTC
MotivationWireless vs. Fiber Optic Links
• Wireless data access for untethered network access– supported by wired backbone– fiber optic cables provide high-speed reliable connections⇒ increasingly wireless access to an optical core (MAN+WAN )
• Deployment barriers to fiber– extremely expensive: ~ $100K/mile– lack of extensive market: rural regions
or– construction and regulatory issues: metropolitan cities
• barrier to new service providers• expensive to lease competitor’s capacity for backhaul
22 April 2009 Weather Disruption-Tolerant Mesh Networks 6
Sterbenz, et al.ITTC
MotivationBroadband Wireless Links
• Wireless alternative to traditional fiber-optic links– point-to-point wired link replacement in MANs and WANs– mesh between base stations and cell towers
• Potential use– backhaul of 3G (and eventually 4G) data– Internet expansion to new areas, such as rural– alternative to add capacity in congested areas, such as cities
22 April 2009 Weather Disruption-Tolerant Mesh Networks 7
Sterbenz, et al.ITTC
MotivationMillimeter-Wave Wireless Links
• Millimeter-wave communication links– exploit recently available commercial radios
• frequency band: 70 – 90 GHz• lightly licensed by FCC in US
– higher data rate potential than microwave links• very-high data rate: 1 – 10 Gb/s• potential replacement for 1/10 GbE and OC-14/192
– significantly cheaper to deploy than fiber
22 April 2009 Weather Disruption-Tolerant Mesh Networks 8
Sterbenz, et al.ITTC
MotivationMillimeter-Wave Wireless Links
• Disadvantages of millimeter-wave links– highly susceptible to weather
• significant rain-based attenuation
⇒unreliable high-speed links• do not meet carrier requirements for backhaul and distribution• reliability, delay, 50 ms restoration
22 April 2009 Weather Disruption-Tolerant Mesh Networks 9
Sterbenz, et al.ITTC
MotivationWeather Disruption-Tolerant Routing
• Problem: slow recovery from rain-based attenuation– many frames lost before detection– no inherent restoration equivalent to SONET/SDH APS
• Proposed solution: compensate at network layer– new routing mechanisms
• predictive• nearly-instantaneous reactive
– exploit weather radar information• permits short-term prediction• weather dynamics longer time scale than network control
22 April 2009 Weather Disruption-Tolerant Mesh Networks 10
Sterbenz, et al.ITTC
WDTN Mesh NetworksMillimeter-Wave Mesh Networks
• Introduction and motivation• Millimeter-wave mesh networks• Impact of weather disruptions• WDTN algorithms• Simulation model and performance analysis
22 April 2009 Weather Disruption-Tolerant Mesh Networks 11
Sterbenz, et al.ITTC
Millimeter-Wave Mesh Networks Architecture
• Mesh architecture– high degree of connectivity
802.163–4G
CO/POP
22 April 2009 Weather Disruption-Tolerant Mesh Networks 12
Sterbenz, et al.ITTC
Millimeter-Wave Mesh Networks Architecture
• Mesh architecture– high degree of connectivity– alternate diverse paths
• mm wave link to CO/POP• alternate mm links
802.163–4G
CO/POP
22 April 2009 Weather Disruption-Tolerant Mesh Networks 13
Sterbenz, et al.ITTC
Millimeter-Wave Mesh Networks Architecture
• Mesh architecture– high degree of connectivity– alternate diverse paths
• severely attenuated mm wave• alternate mm links• alternate lower-freq. RF• fiber bypass (competitor)
• Proposed solution– route around failures
• before they occur
– avoid high error links
802.163–4G
CO/POP
22 April 2009 Weather Disruption-Tolerant Mesh Networks 14
Sterbenz, et al.ITTC
Millimeter-Wave Mesh Networks Open Research Questions
• Impact of weather on mesh network– correlated link failures, link availability
• Actual weather pattern– storm frequency and intensity– geographic coverage with respect to mesh network
• Feasibility– is this approach feasible?– field measurements on deployed infrastructure
22 April 2009 Weather Disruption-Tolerant Mesh Networks 15
Sterbenz, et al.ITTC
WDTN Mesh NetworksImpact of weather disruptions
• Introduction and motivation• Millimeter-Wave Mesh Networks• Impact of weather disruptions• WDTN algorithms• Simulation model and performance analysis
22 April 2009 Weather Disruption-Tolerant Mesh Networks 16
Sterbenz, et al.ITTC
Impact of Weather Disruptions Link
• Impact of weather on individual links• ITU-R P.530 and Crane models
– relates link attenuation to a homogenous rain rate • long term statistics of precipitation probabilities
– cannot predict availability easily of a particular link– does not address correlated link failures of a mesh network– not useful for tolerance of specific events on specific links
22 April 2009 Weather Disruption-Tolerant Mesh Networks 17
Sterbenz, et al.ITTC
Impact of Weather Disruptions Approach
• Evaluate impact of real storms on actual mesh links– collect data on observed storms at a given location– translate storm data to link attenuation
• using geometric model of actual storm
– characterise link behavior and availability during an event
22 April 2009 Weather Disruption-Tolerant Mesh Networks 18
Sterbenz, et al.ITTC
Impact of Weather DisruptionsGeometric Storm Modeling
• Storm model is abstraction of precipitation intensity– cells modelled as ellipses moving along a trajectory– links modelled as line segments– effective attenuation calculated based on link & cell overlap
A B
C
green
red
red
yellow
red
Rs
R1
R2 R3
R4
A B
Cgreen
red
yellow
22 April 2009 Weather Disruption-Tolerant Mesh Networks 19
Sterbenz, et al.ITTC
Impact of Weather Disruptions Actual Observations
• Radar reflectivity data from national weather service• Evaluate effect of real storms on potential networks• Collected and analyzed data
– ~ 30 storms over the Midwest U.S– 3 hypothetical mesh networks in the same region
22 April 2009 Weather Disruption-Tolerant Mesh Networks 20
Sterbenz, et al.ITTC
Impact of Weather Disruptions Observed Storm1 – Rain Distribution
• Millimeter-wave grid location– 38.8621N, 95.3793W
• Storm observed at: – 20:39:26Z 30 Sep 2008
22 April 2009 Weather Disruption-Tolerant Mesh Networks 21
Sterbenz, et al.ITTC
Impact of Weather Disruptions Observed Storm2 – Rain Distribution
• Millimeter-wave grid location– 38.8621N, 95.3793W
• Storm observed at: – 05:04:11Z 22 Apr 2008
22 April 2009 Weather Disruption-Tolerant Mesh Networks 22
Sterbenz, et al.ITTC
Impact of Weather Disruptions Link Characterization
• Channel error rate for links of a 4×4 grid – 10 sec intervals– few links
severelydegraded
– large numbereitherpartiallydegradedor normal
22 April 2009 Weather Disruption-Tolerant Mesh Networks 23
Sterbenz, et al.ITTC
Impact of Weather DisruptionsLink Availability
• Link availability after FEC– Reed Solomon (204,188)
• Link states– three states1: BER < 5×10–8
2: 5×10–8 < BER < 5×10–5
3: BER > 5×10–5
• Modelled asMarkovprocess
22 April 2009 Weather Disruption-Tolerant Mesh Networks 24
Sterbenz, et al.ITTC
Impact of Weather Disruptions Link State Characteristics
• Link state characteristics– state transition probability matrix– state probabilities:
probability of being in each state at given time
State 1 2 3State
Probability0.00046 0.37461
0.40449
0.22091
0.01063
0.98891
0.00459
0.98963
0.00578
1 0.99554
2 0.00417
3 0.00029
22 April 2009 Weather Disruption-Tolerant Mesh Networks 25
Sterbenz, et al.ITTC
WDTN Mesh NetworksWDTN Algorithms
• Introduction and motivation• Millimeter-wave mesh networks• Impact of weather disruptions• WDTN algorithms• Simulation model and performance analysis
22 April 2009 Weather Disruption-Tolerant Mesh Networks 26
Sterbenz, et al.ITTC
Weather Disruption-Tolerant Routing Alternative Algorithm Types
• Reactive– frames in transit are lost– time to converge on new route
• time to detect frame loss > 50 ms restoration• use weather radar to react instantaneously
• Predictive– re-route traffic before link failure– use weather radar to predict the link condition– advance warning in the order of minutes is sufficient
22 April 2009 Weather Disruption-Tolerant Mesh Networks 27
Sterbenz, et al.ITTC
12 13 14 15
8 9 10 11
4 5 6 7
0 1 2 3
Weather Disruption-Tolerant RoutingPredictive Routing Concept
22 April 2009 Weather Disruption-Tolerant Mesh Networks 28
Sterbenz, et al.ITTC
Weather Disruption-Tolerant Routing New Algorithms
• P-WARP: predictive weather-assisted routing prot.– uses weather-radar data to forecast future link conditions– precipitation modelled as {none/light, moderate, heavy}– effective BER calculated and used to adjust link cost
• XL-OSPF: cross-layered OSPF– uses radar to instantaneously estimate attenuation– conventional OSPF but without dead interval detection
22 April 2009 Weather Disruption-Tolerant Mesh Networks 29
Sterbenz, et al.ITTC
Weather Disruption-Tolerant Routing P-WARP
• Predicting link conditions– predictive routing algorithm using weather radar data– effective BER is calculated from radar reflectivity data
• modelled in real-time using the geometric model
– processing done at a core or a subset of nodes• edge nodes with external (Internet) access
• Cost metric– per link cost calculated as
• Cij = cost of link i ↔ j• P = average frame size• BERij = predicted BER• γ = sensitivity factor
Cij = P ×BER ij × γ
22 April 2009 Weather Disruption-Tolerant Mesh Networks 30
Sterbenz, et al.ITTC
Weather Disruption-Tolerant Routing P-WARP
• Link status updates– WLSUs: weather-based link status updates– contains the predicted cost of all links (incremental)
• WLSUs are flooded in to the network by core nodes– single update for all links– generated only when one or more link costs change– significantly reduces protocol overhead
22 April 2009 Weather Disruption-Tolerant Mesh Networks 31
Sterbenz, et al.ITTC
Weather Disruption-Tolerant Routing P-WARP
• Route recomputation– nodes recompute routes using shortest paths first algorithm
• individual nodes do not generate separate LSAs
– network reroutes traffic ahead of the disruption– weather predictions in the order of seconds are sufficient
• route reconvergence is sub second
• Route sensitivity– route flaps avoided using thresholds and hysteresis– changes in BER below BERthresh are ignored– Hthresh is the minimum change in cost that triggers an update
22 April 2009 Weather Disruption-Tolerant Mesh Networks 32
Sterbenz, et al.ITTC
Weather Disruption-Tolerant Routing XL-OSPF
• Standard OSPF– dynamic link state algorithm
• Reacts too slowly causing end-to-end packet loss– link costs do not reflect physical link status– dead interval needed to detect failed links– route convergence is slow
22 April 2009 Weather Disruption-Tolerant Mesh Networks 33
Sterbenz, et al.ITTC
Weather Disruption-Tolerant Routing XL-OSPF
• Cross-layered OSPF– cost metric is proportional to bit error rate
• several mechanisms exist to achieve this reactively– e.g. packet error estimation at the receiver
• could use current weather data to calculate link BER
• Route computation– nodes aware of the quality of all their links– LSAs from other nodes give the complete network status– shortest paths calculated based on link metric– reroute traffic reactively
22 April 2009 Weather Disruption-Tolerant Mesh Networks 34
Sterbenz, et al.ITTC
Weather Disruption-Tolerant Routing XL-OSPF
• Cost metric (same as P-WARP)– per link cost calculated as
• Cij = cost of link ij,• P = average frame size• BERij = predicted BER,• γ = sensitivity factor
• Compared to P-WARP– differs in the mechanism to calculate link costs– reactive instead of predictive– higher overhead, generates per link LSAs
Cij = P ×BER ij × γ
22 April 2009 Weather Disruption-Tolerant Mesh Networks 35
Sterbenz, et al.ITTC
WDTN Mesh NetworksSimulation Model and Performance Analysis
• Introduction and motivation• Millimeter-wave mesh networks• Impact of weather disruptions• WDTN algorithms• Simulation model and performance analysis
22 April 2009 Weather Disruption-Tolerant Mesh Networks 36
Sterbenz, et al.ITTC
SimulationsSimulation Model and Parameters
• ns-2• 16 node mesh: 4×4 grid
– two corner sink nodes connected to Internet (0, 15)– other 14 nodes generate traffic to randomly chosen sink– 2.4 Mb/s CBR over UDP
• Several synthetic storms…– we will look at one example
• Several actual storms…– we will look at one representative example
22 April 2009 Weather Disruption-Tolerant Mesh Networks 37
Sterbenz, et al.ITTC
SimulationsSample Storms
• Synthetic storm– outer ellipse: ~ 30 × 20– four inner ellipses: ~ 5 × 10
• Actual storm– storm 1 from slide 18
22 April 2009 Weather Disruption-Tolerant Mesh Networks 38
Sterbenz, et al.ITTC
SimulationsSynthetic Storm Overlaid on Lawrence, KS
D2
D1
low intensity rain
heavy intensity rain
22 April 2009 Weather Disruption-Tolerant Mesh Networks 39
Sterbenz, et al.ITTC
Synthetic StormPerformance Analysis: Packet Loss
nodeout
linkout
40s OSPF dead interval
10s XL-OSPF HELLO interval
22 April 2009 Weather Disruption-Tolerant Mesh Networks 40
Sterbenz, et al.ITTC
WDTN RoutingPerformance Analysis: Packet Loss
• 100% packet delivery ration with no precipitation• Link attenuation:
– P-WARP reroutes predicatively with no loss– XL-OSPF reroutes with minimal loss after HELLO interval– conventional OSPF must wait for loss detection and recovery
• Node out– attenuation of all links to given node– transit traffic rerouted– but packets sourced and sinked lost until one link returns
• end-to-end recovery necessary
22 April 2009 Weather Disruption-Tolerant Mesh Networks 41
Sterbenz, et al.ITTC
Synthetic StormPerformance Analysis: Cumulative Loss
40s OSPF dead interval
10s OSPF HELLO interval
22 April 2009 Weather Disruption-Tolerant Mesh Networks 42
Sterbenz, et al.ITTC
WDTN RoutingPerformance Analysis: Cumulative Loss
• Cumulative loss statistics– track and compare overall availability during storm event– P-WARP slightly better than XL-OSPF– P-WARP and OSPF significantly better than standard OSPF– static shows worst-case baseline
22 April 2009 Weather Disruption-Tolerant Mesh Networks 43
Sterbenz, et al.ITTC
SimulationsObserved Storm in Northeast Kansas
• Millimeter-wave grid location– 38.8621N, 95.3793W
• Storm observed at: – 20:39:26Z 30 Sep 2008
22 April 2009 Weather Disruption-Tolerant Mesh Networks 44
Sterbenz, et al.ITTC
Observed StormPerformance Analysis: Packet Loss
40s OSPF dead interval
node out
link out
40s OSPF dead interval
10s XL-OSPF HELLO interval
22 April 2009 Weather Disruption-Tolerant Mesh Networks 45
Sterbenz, et al.ITTC
Observed StormPerformance Analysis: Cumulative Loss
40s OSPF dead interval
10s OSPF HELLO interval
22 April 2009 Weather Disruption-Tolerant Mesh Networks 46
Sterbenz, et al.ITTC
Observed StormPerformance Analysis: Delay
40s OSPF dead interval
10s OSPF HELLO interval
22 April 2009 Weather Disruption-Tolerant Mesh Networks 47
Sterbenz, et al.ITTC
WDTN RoutingPerformance Analysis: Delay
• Delay proportional to number of hops– rerouting adds some delay– but not significant within a metro-area mesh
• Congestion avoidance– overprovisioning of mesh essential to avoid congestion– simulation studies based on past precipitation events– drives network engineering for a given network
22 April 2009 Weather Disruption-Tolerant Mesh Networks 48
Sterbenz, et al.ITTC
WDTN Routing Availability
40s OSPF dead interval
10s OSPF HELLO intervalProtocol Availability
synthetic storm Availability
observed storm
P-WARP 0.9638 0.8834
0.8782
0.7313
0.6872
XL-OSPF 0.9554
OSPF 0.9209
Static 0.7304
• Availability– during storm presence in grid neighbourhood– overall availability much higher
• majority of time no storms are present
22 April 2009 Weather Disruption-Tolerant Mesh Networks 49
Sterbenz, et al.ITTC
WDTN Mesh NetworksConclusions
• Overcomes a fundamental limitation – millimeter wave links in the presence of weather events
• Demonstrates a resilient network architecture– P-WARP slightly better than XL-OSPF
• difference important for loss-sensitive traffic
– XL-OSPF significantly better than conventional OSPF– still affected by node outage (storm cell over tower)
• Real case study based on actual radar measurements• Potential solution for data and Internet access in
– rural areas and– metropolitan cities
22 April 2009 Weather Disruption-Tolerant Mesh Networks 50
Sterbenz, et al.ITTC
WDTN Mesh Networks Future Work
• Model additional topologies– hexagonal-packed cellular networks
• Model additional storm types– hurricanes, nor’easters with thundersnow, tropical cyclones– monsoon rains
• Model link alternatives– alternative lower-rate links– fiber bypass
22 April 2009 Weather Disruption-Tolerant Mesh Networks 51
Sterbenz, et al.ITTC
WDTN Mesh Networks Future Work: Alternative Wireless Links
• Mesh architecture– high degree of connectivity– alternate diverse paths
• severely attenuated mm wave• alternate mm links• alternate lower-freq. RF• fiber bypass (competitor)
• Proposed solution– route around failures
• before they occur
– avoid high error links
802.163–4G
CO/POP
22 April 2009 Weather Disruption-Tolerant Mesh Networks 52
Sterbenz, et al.ITTC
WDTN Mesh Networks Future Work: Alternative Fiber Bypass
• Mesh architecture– high degree of connectivity– alternate diverse paths
• severely attenuated mm wave• alternate mm links• alternate lower-freq. RF• fiber bypass (competitor)
• Proposed solution– route around failures
• before they occur
– avoid high error links
802.163–4G
CO/POP
22 April 2009 Weather Disruption-Tolerant Mesh Networks 53
Sterbenz, et al.ITTC
Weather Disruption-Tolerant Nets Publications
• Abdul Jabbar, Justin P. Rohrer, Andrew Oberthaler, Egemen Çetinkaya, Victor S. Frost, and James P.G. Sterbenz,“Performance Comparison of Weather Disruption-Tolerant Cross-Layer Routing Algorithms”,Proceedings of 28th IEEE Conference on Computer Communications (INFOCOM’2009 ),Rio de Janeiro, April 2009[thanks to Merkouris Karaliopoulos for presenting]
• Abdul Jabbar, Bharatwajan Raman, Victor S. Frost, andJames P.G. Sterbenz,“Weather Disruption-Tolerant Self-OptimisingMillimeter Mesh Networks”,Third IFIP/IEEE Workshop on Self-Organizing Systems (IWSOS 2008 ),Vienna/Wein, December 2009, LNCS 5343, pp. 242–255
22 April 2009 Weather Disruption-Tolerant Mesh Networks 54
Sterbenz, et al.ITTC
End
Top Related