CellIQ: Real-Time Cellular Network Analytics at Scale · Slide courtesy: Joey Gonzales Can compute...
Transcript of CellIQ: Real-Time Cellular Network Analytics at Scale · Slide courtesy: Joey Gonzales Can compute...
![Page 1: CellIQ: Real-Time Cellular Network Analytics at Scale · Slide courtesy: Joey Gonzales Can compute destination partitions easily due to the use of geo-partitioner . GeoGraph API class+GeoGraph[V,E](https://reader033.fdocuments.in/reader033/viewer/2022050113/5f4a758e6e76a16cb85d31e0/html5/thumbnails/1.jpg)
CellIQ: Real-Time Cellular Network Analytics at Scale
Anand Iyer#, Li Erran Li+, Ion Stoica# #UC Berkeley +Bell Labs
![Page 2: CellIQ: Real-Time Cellular Network Analytics at Scale · Slide courtesy: Joey Gonzales Can compute destination partitions easily due to the use of geo-partitioner . GeoGraph API class+GeoGraph[V,E](https://reader033.fdocuments.in/reader033/viewer/2022050113/5f4a758e6e76a16cb85d31e0/html5/thumbnails/2.jpg)
Cellular Networks have been seeing exponential growth and become part of our lives
![Page 3: CellIQ: Real-Time Cellular Network Analytics at Scale · Slide courtesy: Joey Gonzales Can compute destination partitions easily due to the use of geo-partitioner . GeoGraph API class+GeoGraph[V,E](https://reader033.fdocuments.in/reader033/viewer/2022050113/5f4a758e6e76a16cb85d31e0/html5/thumbnails/3.jpg)
Image courtesy: Alcatel-Lucent
![Page 4: CellIQ: Real-Time Cellular Network Analytics at Scale · Slide courtesy: Joey Gonzales Can compute destination partitions easily due to the use of geo-partitioner . GeoGraph API class+GeoGraph[V,E](https://reader033.fdocuments.in/reader033/viewer/2022050113/5f4a758e6e76a16cb85d31e0/html5/thumbnails/4.jpg)
What is needed to solve these issues?
Are some regions in the network hotspots? - Better load balancing
How is user traffic moving in the network? - Better resource provisioning
What are the popular handoff sequences? - Troubleshoot handoff related problems
![Page 5: CellIQ: Real-Time Cellular Network Analytics at Scale · Slide courtesy: Joey Gonzales Can compute destination partitions easily due to the use of geo-partitioner . GeoGraph API class+GeoGraph[V,E](https://reader033.fdocuments.in/reader033/viewer/2022050113/5f4a758e6e76a16cb85d31e0/html5/thumbnails/5.jpg)
Cellular Network Analytics Today
![Page 6: CellIQ: Real-Time Cellular Network Analytics at Scale · Slide courtesy: Joey Gonzales Can compute destination partitions easily due to the use of geo-partitioner . GeoGraph API class+GeoGraph[V,E](https://reader033.fdocuments.in/reader033/viewer/2022050113/5f4a758e6e76a16cb85d31e0/html5/thumbnails/6.jpg)
Cellular Network Analytics Today
![Page 7: CellIQ: Real-Time Cellular Network Analytics at Scale · Slide courtesy: Joey Gonzales Can compute destination partitions easily due to the use of geo-partitioner . GeoGraph API class+GeoGraph[V,E](https://reader033.fdocuments.in/reader033/viewer/2022050113/5f4a758e6e76a16cb85d31e0/html5/thumbnails/7.jpg)
Cellular Network Analytics Today
![Page 8: CellIQ: Real-Time Cellular Network Analytics at Scale · Slide courtesy: Joey Gonzales Can compute destination partitions easily due to the use of geo-partitioner . GeoGraph API class+GeoGraph[V,E](https://reader033.fdocuments.in/reader033/viewer/2022050113/5f4a758e6e76a16cb85d31e0/html5/thumbnails/8.jpg)
Problem
Existing cellular network analytic systems do not
support advanced analytic tasks in an efficient manner.
![Page 9: CellIQ: Real-Time Cellular Network Analytics at Scale · Slide courtesy: Joey Gonzales Can compute destination partitions easily due to the use of geo-partitioner . GeoGraph API class+GeoGraph[V,E](https://reader033.fdocuments.in/reader033/viewer/2022050113/5f4a758e6e76a16cb85d31e0/html5/thumbnails/9.jpg)
High Velocity Data Continuous Monitoring
Advanced Tasks
Timely Spatio-Temporal Analysis
Challenges
![Page 10: CellIQ: Real-Time Cellular Network Analytics at Scale · Slide courtesy: Joey Gonzales Can compute destination partitions easily due to the use of geo-partitioner . GeoGraph API class+GeoGraph[V,E](https://reader033.fdocuments.in/reader033/viewer/2022050113/5f4a758e6e76a16cb85d31e0/html5/thumbnails/10.jpg)
CellIQ is a cellular network analytics system that supports rich analysis
tasks efficiently by leveraging domain-specific optimizations
![Page 11: CellIQ: Real-Time Cellular Network Analytics at Scale · Slide courtesy: Joey Gonzales Can compute destination partitions easily due to the use of geo-partitioner . GeoGraph API class+GeoGraph[V,E](https://reader033.fdocuments.in/reader033/viewer/2022050113/5f4a758e6e76a16cb85d31e0/html5/thumbnails/11.jpg)
Cellular Data as Time-Evolving Graphs
Tasks easily expressed in graphs: Hotspot computation è Connected components
Handoff sequences & User traffic è Pregel model
Edge PropertyVertex Property
BS1
UE2
UE1 BS2
UE3
UE4
UE5
![Page 12: CellIQ: Real-Time Cellular Network Analytics at Scale · Slide courtesy: Joey Gonzales Can compute destination partitions easily due to the use of geo-partitioner . GeoGraph API class+GeoGraph[V,E](https://reader033.fdocuments.in/reader033/viewer/2022050113/5f4a758e6e76a16cb85d31e0/html5/thumbnails/12.jpg)
Why Not Use a Graph Parallel Framework?
��
��
���
���
���
���
���
���
���
�������� ���������� ������ �����
�����������������������
������������
Fails to produce results!
Domain specific optimizations key for efficient analysis
![Page 13: CellIQ: Real-Time Cellular Network Analytics at Scale · Slide courtesy: Joey Gonzales Can compute destination partitions easily due to the use of geo-partitioner . GeoGraph API class+GeoGraph[V,E](https://reader033.fdocuments.in/reader033/viewer/2022050113/5f4a758e6e76a16cb85d31e0/html5/thumbnails/13.jpg)
CellIQ Implementation
*Gonzales. et.al. “GraphX: Graph Processing in a Distributed Dataflow Framework”, OSDI 2014
Implemented as a layer on GraphX* Incorporates several domain specific optimizations
GraphX
Spark
Pregel API
PageRank Connected Comp. K-core Triangle Count
LDA SVD++
CellIQ
![Page 14: CellIQ: Real-Time Cellular Network Analytics at Scale · Slide courtesy: Joey Gonzales Can compute destination partitions easily due to the use of geo-partitioner . GeoGraph API class+GeoGraph[V,E](https://reader033.fdocuments.in/reader033/viewer/2022050113/5f4a758e6e76a16cb85d31e0/html5/thumbnails/14.jpg)
Computational Model
BS1UE2
UE1
BS2
UE3
UE4
UE5
![Page 15: CellIQ: Real-Time Cellular Network Analytics at Scale · Slide courtesy: Joey Gonzales Can compute destination partitions easily due to the use of geo-partitioner . GeoGraph API class+GeoGraph[V,E](https://reader033.fdocuments.in/reader033/viewer/2022050113/5f4a758e6e76a16cb85d31e0/html5/thumbnails/15.jpg)
Computational Model
BS1UE2
UE1
BS2
UE3
UE4
UE5
BS1UE2
UE1
BS2
UE3
UE4
UE5
![Page 16: CellIQ: Real-Time Cellular Network Analytics at Scale · Slide courtesy: Joey Gonzales Can compute destination partitions easily due to the use of geo-partitioner . GeoGraph API class+GeoGraph[V,E](https://reader033.fdocuments.in/reader033/viewer/2022050113/5f4a758e6e76a16cb85d31e0/html5/thumbnails/16.jpg)
Computational Model
BS1UE2
UE1
BS2
UE3
UE4
UE5
BS1UE2
UE1
BS2
UE3
UE4
UE5
BS1
UE2
UE1 BS2
UE3
UE4
UE5
![Page 17: CellIQ: Real-Time Cellular Network Analytics at Scale · Slide courtesy: Joey Gonzales Can compute destination partitions easily due to the use of geo-partitioner . GeoGraph API class+GeoGraph[V,E](https://reader033.fdocuments.in/reader033/viewer/2022050113/5f4a758e6e76a16cb85d31e0/html5/thumbnails/17.jpg)
Computational Model: GStreams
BS1UE2
UE1
BS2
UE3
UE4
UE5
BS1UE2
UE1
BS2
UE3
UE4
UE5
BS1
UE2
UE1 BS2
UE3
UE4
UE5
Domain specific graph partitioning Spatial operations
Window operations
![Page 18: CellIQ: Real-Time Cellular Network Analytics at Scale · Slide courtesy: Joey Gonzales Can compute destination partitions easily due to the use of geo-partitioner . GeoGraph API class+GeoGraph[V,E](https://reader033.fdocuments.in/reader033/viewer/2022050113/5f4a758e6e76a16cb85d31e0/html5/thumbnails/18.jpg)
Computational Model: GStreams
BS1UE2
UE1
BS2
UE3
UE4
UE5
BS1UE2
UE1
BS2
UE3
UE4
UE5
BS1
UE2
UE1 BS2
UE3
UE4
UE5
Domain specific graph partitioning Spatial operations
Window operations
![Page 19: CellIQ: Real-Time Cellular Network Analytics at Scale · Slide courtesy: Joey Gonzales Can compute destination partitions easily due to the use of geo-partitioner . GeoGraph API class+GeoGraph[V,E](https://reader033.fdocuments.in/reader033/viewer/2022050113/5f4a758e6e76a16cb85d31e0/html5/thumbnails/19.jpg)
Graph computation frameworks rely on partitioning to minimize communication & balance computation
B C
A D
FE
A DD
B C
D
E
AA
F Machine 1 Machine 2
A
B
C
D
E
F
Graph Partitioning
![Page 20: CellIQ: Real-Time Cellular Network Analytics at Scale · Slide courtesy: Joey Gonzales Can compute destination partitions easily due to the use of geo-partitioner . GeoGraph API class+GeoGraph[V,E](https://reader033.fdocuments.in/reader033/viewer/2022050113/5f4a758e6e76a16cb85d31e0/html5/thumbnails/20.jpg)
Partition geographically close-by entities
Machine 3 Machine 4
3
B CB C
D
E
A
F
Machine 1 Machine 2
CellIQ Graph Partitioning
G H
2D 1D
?
![Page 21: CellIQ: Real-Time Cellular Network Analytics at Scale · Slide courtesy: Joey Gonzales Can compute destination partitions easily due to the use of geo-partitioner . GeoGraph API class+GeoGraph[V,E](https://reader033.fdocuments.in/reader033/viewer/2022050113/5f4a758e6e76a16cb85d31e0/html5/thumbnails/21.jpg)
3 Machine 3 Machine 4
B CB C
D
E
A
F
Machine 1 Machine 2
AB
CD
EF
Graph Partitioning
G HG
H
Random (hashed) partitioning
![Page 22: CellIQ: Real-Time Cellular Network Analytics at Scale · Slide courtesy: Joey Gonzales Can compute destination partitions easily due to the use of geo-partitioner . GeoGraph API class+GeoGraph[V,E](https://reader033.fdocuments.in/reader033/viewer/2022050113/5f4a758e6e76a16cb85d31e0/html5/thumbnails/22.jpg)
3 Machine 3 Machine 4
B CB C
D
E
A
F
Machine 1 Machine 2
AB
CD
EF
Graph Partitioning
G HG
H
Random (hashed) partitioning results in poor spatial locality
![Page 23: CellIQ: Real-Time Cellular Network Analytics at Scale · Slide courtesy: Joey Gonzales Can compute destination partitions easily due to the use of geo-partitioner . GeoGraph API class+GeoGraph[V,E](https://reader033.fdocuments.in/reader033/viewer/2022050113/5f4a758e6e76a16cb85d31e0/html5/thumbnails/23.jpg)
Machine 3 Machine 4
B CB C
D
E
A
F
Machine 1 Machine 2
CellIQ Graph Partitioning
G H
Uses Hilbert space-filling curves
![Page 24: CellIQ: Real-Time Cellular Network Analytics at Scale · Slide courtesy: Joey Gonzales Can compute destination partitions easily due to the use of geo-partitioner . GeoGraph API class+GeoGraph[V,E](https://reader033.fdocuments.in/reader033/viewer/2022050113/5f4a758e6e76a16cb85d31e0/html5/thumbnails/24.jpg)
Machine 3 Machine 4
0 3
2 1 B CB C
D
E
A
F
Machine 1 Machine 2
CellIQ Graph Partitioning
G H
Uses Hilbert space-filling curves Use curve’s distance as the 1-dimensional key
![Page 25: CellIQ: Real-Time Cellular Network Analytics at Scale · Slide courtesy: Joey Gonzales Can compute destination partitions easily due to the use of geo-partitioner . GeoGraph API class+GeoGraph[V,E](https://reader033.fdocuments.in/reader033/viewer/2022050113/5f4a758e6e76a16cb85d31e0/html5/thumbnails/25.jpg)
Machine 3 Machine 4
0 3
2 1 B CB C
D
E
A
F
Machine 1 Machine 2
AB C
D
EF
CellIQ Graph Partitioning
G H G H
Uses Hilbert space-filling curves Use curve’s distance as the 1-dimensional key Range partition the key space
![Page 26: CellIQ: Real-Time Cellular Network Analytics at Scale · Slide courtesy: Joey Gonzales Can compute destination partitions easily due to the use of geo-partitioner . GeoGraph API class+GeoGraph[V,E](https://reader033.fdocuments.in/reader033/viewer/2022050113/5f4a758e6e76a16cb85d31e0/html5/thumbnails/26.jpg)
0 1
2 3
4 7
6 5
8 11
10 9
14 15
12 13
Machine 3 Machine 4
B CB C
D
E
A
F
Machine 1 Machine 2
AB C
D
EF
CellIQ Graph Partitioning
G H G H
Uses Hilbert space-filling curves Use curve’s distance as the 1-dimensional key Range partition the key space
![Page 27: CellIQ: Real-Time Cellular Network Analytics at Scale · Slide courtesy: Joey Gonzales Can compute destination partitions easily due to the use of geo-partitioner . GeoGraph API class+GeoGraph[V,E](https://reader033.fdocuments.in/reader033/viewer/2022050113/5f4a758e6e76a16cb85d31e0/html5/thumbnails/27.jpg)
Computational Model: GStreams
BS1UE2
UE1
BS2
UE3
UE4
UE5
BS1UE2
UE1
BS2
UE3
UE4
UE5
BS1
UE2
UE1 BS2
UE3
UE4
UE5
Domain specific graph partitioning Spatial operations Window operations
![Page 28: CellIQ: Real-Time Cellular Network Analytics at Scale · Slide courtesy: Joey Gonzales Can compute destination partitions easily due to the use of geo-partitioner . GeoGraph API class+GeoGraph[V,E](https://reader033.fdocuments.in/reader033/viewer/2022050113/5f4a758e6e76a16cb85d31e0/html5/thumbnails/28.jpg)
GeoGraph API
class GeoGraph[V, E] { // Broadcast a message to all // vertices within a radius def sendMsg(radius) // Create a spatially aggregated // graph by combining vertices // and edges def spatialAG(reduceV: (V, V) => V, reduceE: (E, E) => E) }
![Page 29: CellIQ: Real-Time Cellular Network Analytics at Scale · Slide courtesy: Joey Gonzales Can compute destination partitions easily due to the use of geo-partitioner . GeoGraph API class+GeoGraph[V,E](https://reader033.fdocuments.in/reader033/viewer/2022050113/5f4a758e6e76a16cb85d31e0/html5/thumbnails/29.jpg)
Tracking user traffic gradients
Goal: Detect and track direction of movement of user groups
![Page 30: CellIQ: Real-Time Cellular Network Analytics at Scale · Slide courtesy: Joey Gonzales Can compute destination partitions easily due to the use of geo-partitioner . GeoGraph API class+GeoGraph[V,E](https://reader033.fdocuments.in/reader033/viewer/2022050113/5f4a758e6e76a16cb85d31e0/html5/thumbnails/30.jpg)
3
B C
A D
F
E
A DD
B C
D
E
AA
F
Tracking user traffic gradients
Base Station
![Page 31: CellIQ: Real-Time Cellular Network Analytics at Scale · Slide courtesy: Joey Gonzales Can compute destination partitions easily due to the use of geo-partitioner . GeoGraph API class+GeoGraph[V,E](https://reader033.fdocuments.in/reader033/viewer/2022050113/5f4a758e6e76a16cb85d31e0/html5/thumbnails/31.jpg)
3
B C
A D
F
E
A DD
B C
D
E
AA
F
Tracking user traffic gradients
![Page 32: CellIQ: Real-Time Cellular Network Analytics at Scale · Slide courtesy: Joey Gonzales Can compute destination partitions easily due to the use of geo-partitioner . GeoGraph API class+GeoGraph[V,E](https://reader033.fdocuments.in/reader033/viewer/2022050113/5f4a758e6e76a16cb85d31e0/html5/thumbnails/32.jpg)
B C
A D
F
E
A DD
B C
D
E
AA
F
Hop-by-hop propagation
Tracking user traffic gradients
![Page 33: CellIQ: Real-Time Cellular Network Analytics at Scale · Slide courtesy: Joey Gonzales Can compute destination partitions easily due to the use of geo-partitioner . GeoGraph API class+GeoGraph[V,E](https://reader033.fdocuments.in/reader033/viewer/2022050113/5f4a758e6e76a16cb85d31e0/html5/thumbnails/33.jpg)
B C
A D
F
E
A DD
B C
D
E
AA
F
Hop-by-hop propagation is inefficient
Tracking user traffic gradients
![Page 34: CellIQ: Real-Time Cellular Network Analytics at Scale · Slide courtesy: Joey Gonzales Can compute destination partitions easily due to the use of geo-partitioner . GeoGraph API class+GeoGraph[V,E](https://reader033.fdocuments.in/reader033/viewer/2022050113/5f4a758e6e76a16cb85d31e0/html5/thumbnails/34.jpg)
Tracking user traffic gradients
B C
A D
F
E
A DD
B C
D
E
AA
F
Instead, CellIQ enables radius based broadcast
![Page 35: CellIQ: Real-Time Cellular Network Analytics at Scale · Slide courtesy: Joey Gonzales Can compute destination partitions easily due to the use of geo-partitioner . GeoGraph API class+GeoGraph[V,E](https://reader033.fdocuments.in/reader033/viewer/2022050113/5f4a758e6e76a16cb85d31e0/html5/thumbnails/35.jpg)
Part. 2
Part. 1
Vertex Table(RDD)
B C
A D
FE
A D
Routing Table in GraphX enables Multicast
D
B C
D
E
AA
F
Machine 1
Machine 2
Edge Table(RDD)
A B
A C
C D
B C
A E
A F
E F
E D
B
C
D
E
A
F
RoutingTable
(RDD)
B
C
D
E
A
F
1
2
1 2
1 2
1
2
Slide courtesy: Joey Gonzales
![Page 36: CellIQ: Real-Time Cellular Network Analytics at Scale · Slide courtesy: Joey Gonzales Can compute destination partitions easily due to the use of geo-partitioner . GeoGraph API class+GeoGraph[V,E](https://reader033.fdocuments.in/reader033/viewer/2022050113/5f4a758e6e76a16cb85d31e0/html5/thumbnails/36.jpg)
RoutingTable
(RDD)
B
C
D
E
A
F
1
2
1 2
1 2
1
2 Part. 2
Part. 1
Vertex Table(RDD)
B C
A D
FE
A DD
B C
D
E
AA
F
Machine 1
Machine 2
Edge Table(RDD)
A B
A C
C D
B C
A E
A F
E F
E D
B
C
D
E
A
FSlide courtesy: Joey Gonzales
Can compute destination partitions easily due to the use of geo-partitioner
![Page 37: CellIQ: Real-Time Cellular Network Analytics at Scale · Slide courtesy: Joey Gonzales Can compute destination partitions easily due to the use of geo-partitioner . GeoGraph API class+GeoGraph[V,E](https://reader033.fdocuments.in/reader033/viewer/2022050113/5f4a758e6e76a16cb85d31e0/html5/thumbnails/37.jpg)
GeoGraph API
class GeoGraph[V, E] { // Broadcast a message to all // vertices within a radius def sendMsg(radius) // Create a spatially aggregated // graph by combining vertices // and edges def spatialAG(reduceV: (V, V) => V, reduceE: (E, E) => E) }
![Page 38: CellIQ: Real-Time Cellular Network Analytics at Scale · Slide courtesy: Joey Gonzales Can compute destination partitions easily due to the use of geo-partitioner . GeoGraph API class+GeoGraph[V,E](https://reader033.fdocuments.in/reader033/viewer/2022050113/5f4a758e6e76a16cb85d31e0/html5/thumbnails/38.jpg)
B C
A D
F
E
A DD
B C
D
E
AA
F
Spatial Clustering
F E DDB’F
Goal: Combine spatially close-by vertices
![Page 39: CellIQ: Real-Time Cellular Network Analytics at Scale · Slide courtesy: Joey Gonzales Can compute destination partitions easily due to the use of geo-partitioner . GeoGraph API class+GeoGraph[V,E](https://reader033.fdocuments.in/reader033/viewer/2022050113/5f4a758e6e76a16cb85d31e0/html5/thumbnails/39.jpg)
Spatial Clustering Two ways to enable spatial aggregation: - Using a (supplied) field in properties - Leverage geo partitioner
00 01
02 03
10 13
12 11
20 23
22 21
32 33
30 31
![Page 40: CellIQ: Real-Time Cellular Network Analytics at Scale · Slide courtesy: Joey Gonzales Can compute destination partitions easily due to the use of geo-partitioner . GeoGraph API class+GeoGraph[V,E](https://reader033.fdocuments.in/reader033/viewer/2022050113/5f4a758e6e76a16cb85d31e0/html5/thumbnails/40.jpg)
Spatial Clustering Two ways to enable spatial aggregation: - Using a (supplied) field in properties - Leverage geo partitioner
00 01
02 03
10 13
12 11
20 23
22 21
32 33
30 31 0 3
2 1
![Page 41: CellIQ: Real-Time Cellular Network Analytics at Scale · Slide courtesy: Joey Gonzales Can compute destination partitions easily due to the use of geo-partitioner . GeoGraph API class+GeoGraph[V,E](https://reader033.fdocuments.in/reader033/viewer/2022050113/5f4a758e6e76a16cb85d31e0/html5/thumbnails/41.jpg)
Computational Model: GStreams
BS1UE2
UE1
BS2
UE3
UE4
UE5
BS1UE2
UE1
BS2
UE3
UE4
UE5
BS1
UE2
UE1 BS2
UE3
UE4
UE5
Domain specific graph partitioning Spatial operations
Window operations
![Page 42: CellIQ: Real-Time Cellular Network Analytics at Scale · Slide courtesy: Joey Gonzales Can compute destination partitions easily due to the use of geo-partitioner . GeoGraph API class+GeoGraph[V,E](https://reader033.fdocuments.in/reader033/viewer/2022050113/5f4a758e6e76a16cb85d31e0/html5/thumbnails/42.jpg)
Tracking Persistent Hotspots
Goal: Detect and track groups of base stations with high traffic volume
Equivalent to finding connected components
![Page 43: CellIQ: Real-Time Cellular Network Analytics at Scale · Slide courtesy: Joey Gonzales Can compute destination partitions easily due to the use of geo-partitioner . GeoGraph API class+GeoGraph[V,E](https://reader033.fdocuments.in/reader033/viewer/2022050113/5f4a758e6e76a16cb85d31e0/html5/thumbnails/43.jpg)
Tracking Persistent Hotspots BS1
BS2 BS3
t1 t2 t3
W
Combining graphs at the end of the window results in many join operations (inefficient)
BS1
BS2
BS1
BS2
![Page 44: CellIQ: Real-Time Cellular Network Analytics at Scale · Slide courtesy: Joey Gonzales Can compute destination partitions easily due to the use of geo-partitioner . GeoGraph API class+GeoGraph[V,E](https://reader033.fdocuments.in/reader033/viewer/2022050113/5f4a758e6e76a16cb85d31e0/html5/thumbnails/44.jpg)
Tracking Persistent Hotspots BS1
BS2 BS3
t1 t2 t3
W
BS1
BS2
BS1
BS2
BS1
BS2 BS3
1 1
1
BS1
BS2 BS3
2 1
1
BS1
BS2 BS3
3 1
1
Apply incremental updates to a cumulative graph
![Page 45: CellIQ: Real-Time Cellular Network Analytics at Scale · Slide courtesy: Joey Gonzales Can compute destination partitions easily due to the use of geo-partitioner . GeoGraph API class+GeoGraph[V,E](https://reader033.fdocuments.in/reader033/viewer/2022050113/5f4a758e6e76a16cb85d31e0/html5/thumbnails/45.jpg)
Tracking Persistent Hotspots BS1
BS2 BS3
t1 t2 t3
BS1
BS2
BS1
BS2
BS1
BS2 BS3
1 1
1
Apply differential updates to a cumulative graph
BS1
BS3
t4
BS1
BS2 BS3
1 2
1
BS1
BS2 BS3
1 3
1
BS1
BS2 BS3
1 2
0
![Page 46: CellIQ: Real-Time Cellular Network Analytics at Scale · Slide courtesy: Joey Gonzales Can compute destination partitions easily due to the use of geo-partitioner . GeoGraph API class+GeoGraph[V,E](https://reader033.fdocuments.in/reader033/viewer/2022050113/5f4a758e6e76a16cb85d31e0/html5/thumbnails/46.jpg)
GStream API
class GStream[V, E] { def graphReduceByWindow( reduceFunc(Graph[V, E], Graph[V, E], fv: (V, V) => V, fe: (E, E) => E): Graph[V, E], invReduceFunc(Graph[V, E], Graph[V, E], fv: (V, V) => V, fe: (E, E) => E): Graph[V, E], windowDuration, slideDuration) }
![Page 47: CellIQ: Real-Time Cellular Network Analytics at Scale · Slide courtesy: Joey Gonzales Can compute destination partitions easily due to the use of geo-partitioner . GeoGraph API class+GeoGraph[V,E](https://reader033.fdocuments.in/reader033/viewer/2022050113/5f4a758e6e76a16cb85d31e0/html5/thumbnails/47.jpg)
graphReduceByWindow
• Implemented using Spark’s cogroupedRDD • Two default reduce functions: graph intersection and union • Further optimizations:
– Co-partition graphs from multiple batches – Reuse indices and routing tables for graphs in the
same window More details in the paper!
![Page 48: CellIQ: Real-Time Cellular Network Analytics at Scale · Slide courtesy: Joey Gonzales Can compute destination partitions easily due to the use of geo-partitioner . GeoGraph API class+GeoGraph[V,E](https://reader033.fdocuments.in/reader033/viewer/2022050113/5f4a758e6e76a16cb85d31e0/html5/thumbnails/48.jpg)
How does CellIQ perform?
![Page 49: CellIQ: Real-Time Cellular Network Analytics at Scale · Slide courtesy: Joey Gonzales Can compute destination partitions easily due to the use of geo-partitioner . GeoGraph API class+GeoGraph[V,E](https://reader033.fdocuments.in/reader033/viewer/2022050113/5f4a758e6e76a16cb85d31e0/html5/thumbnails/49.jpg)
Evaluation Setup
• LTE control plane data from a major cellular network operator • 1 million+ subscribers, live network
• 2 TB data from 1 week
– 1 file per minute, 750k records, 100s of fields/line – 10 collection points, 10 hours per day
• Implemented several analysis tasks
![Page 50: CellIQ: Real-Time Cellular Network Analytics at Scale · Slide courtesy: Joey Gonzales Can compute destination partitions easily due to the use of geo-partitioner . GeoGraph API class+GeoGraph[V,E](https://reader033.fdocuments.in/reader033/viewer/2022050113/5f4a758e6e76a16cb85d31e0/html5/thumbnails/50.jpg)
Benefits of Geo-partitioning
��
��
���
���
���
���
���
���
���
�������� ���������� ������ �����
�����������������������
������������
�������������������� ����������������
![Page 51: CellIQ: Real-Time Cellular Network Analytics at Scale · Slide courtesy: Joey Gonzales Can compute destination partitions easily due to the use of geo-partitioner . GeoGraph API class+GeoGraph[V,E](https://reader033.fdocuments.in/reader033/viewer/2022050113/5f4a758e6e76a16cb85d31e0/html5/thumbnails/51.jpg)
Benefits of Geo-partitioning
��
��
���
���
���
���
���
���
���
�������� ���������� ������ �����
�����������������������
������������
�������������������� ����������������
Small amount of data, movement not noticeable
Default practitioner fails to produce results
![Page 52: CellIQ: Real-Time Cellular Network Analytics at Scale · Slide courtesy: Joey Gonzales Can compute destination partitions easily due to the use of geo-partitioner . GeoGraph API class+GeoGraph[V,E](https://reader033.fdocuments.in/reader033/viewer/2022050113/5f4a758e6e76a16cb85d31e0/html5/thumbnails/52.jpg)
Benefits of Incremental Updates
��
��
���
���
���
���
���
���
���
�������� ���������� ������ �����
�����������������������
������������
������������������������������������
������������������������������������������������������������
![Page 53: CellIQ: Real-Time Cellular Network Analytics at Scale · Slide courtesy: Joey Gonzales Can compute destination partitions easily due to the use of geo-partitioner . GeoGraph API class+GeoGraph[V,E](https://reader033.fdocuments.in/reader033/viewer/2022050113/5f4a758e6e76a16cb85d31e0/html5/thumbnails/53.jpg)
Benefits of Incremental Updates
��
��
���
���
���
���
���
���
���
�������� ���������� ������ �����
�����������������������
������������
������������������������������������
������������������������������������������������������������
2 – 5X improvements
![Page 54: CellIQ: Real-Time Cellular Network Analytics at Scale · Slide courtesy: Joey Gonzales Can compute destination partitions easily due to the use of geo-partitioner . GeoGraph API class+GeoGraph[V,E](https://reader033.fdocuments.in/reader033/viewer/2022050113/5f4a758e6e76a16cb85d31e0/html5/thumbnails/54.jpg)
Benefits of Incremental Updates
��
��
���
���
���
���
���
���
���
�������� ���������� ������ �����
�����������������������
������������
������������������������������������
������������������������������������������������������������
window size affects performance
![Page 55: CellIQ: Real-Time Cellular Network Analytics at Scale · Slide courtesy: Joey Gonzales Can compute destination partitions easily due to the use of geo-partitioner . GeoGraph API class+GeoGraph[V,E](https://reader033.fdocuments.in/reader033/viewer/2022050113/5f4a758e6e76a16cb85d31e0/html5/thumbnails/55.jpg)
Benefits of Differential Updates
��
��
��
��
��
���
�� �� �� �� �� ��� ���
�����������������
����������������
��������������
![Page 56: CellIQ: Real-Time Cellular Network Analytics at Scale · Slide courtesy: Joey Gonzales Can compute destination partitions easily due to the use of geo-partitioner . GeoGraph API class+GeoGraph[V,E](https://reader033.fdocuments.in/reader033/viewer/2022050113/5f4a758e6e76a16cb85d31e0/html5/thumbnails/56.jpg)
Benefits of Differential Updates
��
��
��
��
��
���
�� �� �� �� �� ��� ���
�����������������
����������������
��������������
Larger windows see bigger benefits
Graceful degradation in performance
![Page 57: CellIQ: Real-Time Cellular Network Analytics at Scale · Slide courtesy: Joey Gonzales Can compute destination partitions easily due to the use of geo-partitioner . GeoGraph API class+GeoGraph[V,E](https://reader033.fdocuments.in/reader033/viewer/2022050113/5f4a758e6e76a16cb85d31e0/html5/thumbnails/57.jpg)
Benefits of Radius-based Broadcast
��
����
����
����
����
����
����
�������� ���������� ������ �����
�����������������
������������
����������������������
��� � ��
���
����������
��� ���
���
![Page 58: CellIQ: Real-Time Cellular Network Analytics at Scale · Slide courtesy: Joey Gonzales Can compute destination partitions easily due to the use of geo-partitioner . GeoGraph API class+GeoGraph[V,E](https://reader033.fdocuments.in/reader033/viewer/2022050113/5f4a758e6e76a16cb85d31e0/html5/thumbnails/58.jpg)
Benefits of Radius-based Broadcast
��
����
����
����
����
����
����
�������� ���������� ������ �����
�����������������
������������
����������������������
��� � ��
���
����������
��� ���
���
Larger datasets result in increase in messages exchanges per hop
![Page 59: CellIQ: Real-Time Cellular Network Analytics at Scale · Slide courtesy: Joey Gonzales Can compute destination partitions easily due to the use of geo-partitioner . GeoGraph API class+GeoGraph[V,E](https://reader033.fdocuments.in/reader033/viewer/2022050113/5f4a758e6e76a16cb85d31e0/html5/thumbnails/59.jpg)
CellIQ is a cellular network analytics system that uses domain-specific optimizations to achieve 2x to 5x
improvements
![Page 60: CellIQ: Real-Time Cellular Network Analytics at Scale · Slide courtesy: Joey Gonzales Can compute destination partitions easily due to the use of geo-partitioner . GeoGraph API class+GeoGraph[V,E](https://reader033.fdocuments.in/reader033/viewer/2022050113/5f4a758e6e76a16cb85d31e0/html5/thumbnails/60.jpg)
CellIQ is a cellular network analytics system that uses domain-specific optimizations to achieve 2x to 5x
improvements Ongoing Work: • Using techniques in CellIQ to perform root-cause
analysis on operational LTE Networks • Generalized streaming graph analysis techniques