MobileInsight - CS | Computer Scienceweb.cs.ucla.edu/~zyuan/slides/mobicom16-mobileinsig… · ·...

MobileInsightExtracting and Analyzing

Cellular Network Information on Smartphones Yuanjie Li1, Chunyi Peng2, Zengwen Yuan1, Jiayao Li1, Haotian Deng2, Tao Wang3

1University of California, Los Angeles2The Ohio State University

3Peking University

“Anytime, Anywhere” Cellular Network Service

22

Critical Cellular Operations to Users/Apps

33

Physical Layer (PHY)Link Layer (MAC/RLC/PDCP)

Radio Resource Control (RRC)Mobility Management (MM)Session Management (SM)

HardwareSoftware…

But They are Closed…

44

Physical Layer (PHY)Link Layer (MAC/RLC/PDCP)

Radio Resource Control (RRC)Mobility Management (MM)Session Management (SM)

HardwareSoftware? ?

? ? ?

Can We Have Open Access to Runtime Cellular Network Operations?

55? ? ?

Why my 4G phoneswitches to slow 2G?

Why my phone drainsbattery quickly?

4 signal bars, butwhy no data service?

…

It’s Not That Simple

Finegrained

Fullcoverage

• No approaches cover all necessary features

Analysis In-phoneAt scale

Android APIs ✘ ✔✘ ✘ ✔

Operator-sidecellular analytics ✔ ✘✔ ✔ ✘

External Tools(e.g., QXDM) ✔ ✔ ✘ ✘ ✘

6

Our Solution: MobileInsight

Finegrained

Fullcoverage

• A software tool for commodity phones• A community tool that can be built and shared together

Analysis In-phoneAt scale

Android APIs ✘ ✔✘ ✘ ✔

Operator-sidecellular analytics ✔ ✘✔ ✔ ✘

External Tools(e.g., QXDM) ✔ ✔ ✘ ✘ ✘

7

MobileInsight ✔ ✔ ✔ ✔ ✔

MobileInsight Overview

Hardware

8

01101PHY

MAC/RLC/PDCPRRCMMSM

Software

Monitor

MobileInsight Overview

State 1 State 3

State 2

State 1 State 3

State 2

State 1 State 3

State 2

…

9

HardwareSoftware

Monitor Analyzers API

In-device Runtime MonitorHow to expose runtime cellular messages to user space?

AnalyzersMonitor API 10


HardwareSoftware

Coarse-grainedcellular info

Radio Interface Layer

Challenge: No Ordinary In-device Schemes

Android APIs

…


HardwareSoftware

Coarse-grainedcellular info

Android APIs

Radio Interface Layer

Solution: Side-Channel Across SW-HW Boundary

via USB

Parsers

/dev/diag

Raw cellular messages

Proxy…

Cellular Protocol AnalyticsHow to unveil runtime cellular protocol behaviors?


• Operation logic inference• Network side• Non-standardized, operator-specific

• State dynamics extraction• Device side• Regulated by cellular standards

Two Dimensions for Each Protocol

AnalyzersMonitor API

Handoff decision logic

Protocol Analytics: Tracking State Dynamics• Current protocol state, transition events and causes• RRC: Radio connectivity status and power-saving mode• MM: Device registration status• SM: Data session activity and QoS status


Protocol Analytics: Tracking State Dynamics• Observation: regulated by the cellular standards


RRC conn. setup accept

RRC conn. setup request

Downlink data……

RRC conn. reconfigurationParameters: T1=100ms,

TshortDRX=20msT2=2 TshortDRX

3GPP TS 36.331 V12.5.0 (2015-03) Technical Specification

3rd Generation Partnership Project; Technical Specification Group Radio Access Network; Evolved Universal Terrestrial Radio Access (E-UTRA);

Radio Resource Control (RRC); Protocol specification

(Release 12)

The present document has been developed within the 3rd Generation Partnership Project (3GPP TM) and may be further elaborated for the purposes of 3GPP. The present document has not been subject to any approval process by the 3GPP Organizational Partners and shall not be implemented. This Specification is provided for future development work within 3GPP only. The Organizational Partners accept no liability for any use of this Specification. Specifications and reports for implementation of the 3GPP TM system should be obtained via the 3GPP Organizational Partners' Publications Offices.

Conn. setup

T1

Conn. release

T2Data

Data

Conn. setup

T1Data T1

• Reference state machine + runtime message

16

Protocol Analytics: Inferring Operation Logic• Algorithm to determine protocol configurations and actions• Example: handoff decision logic


BS 1’s handoff decision logic: • Switch to BS 2 (4G) if • Otherwise, switch to BS 3 (3G) if and

RSS1(4G) < �110 dBmRSS3(3G) > �90 dBm

RSS2(4G) > RSS1(4G) + 3 dBm

BS 3 (3G)

BS 2 (4G) BS 1 (4G)

17

Inferring Operation Logic is Not Simple• Challenge #1: Non-standardized, carrier-specific operations• Challenge #2: Internal logic, not visible by end device



RSS1(4G) < �110 dBmRSS3(3G) > �90 dBm

RSS2(4G) > RSS1(4G) + 3 dBm?BS 3 (3G)

BS 2 (4G) BS 1 (4G)

3GPP TS 36.331 V12.5.0 (2015-03) Technical Specification

3rd Generation Partnership Project; Technical Specification Group Radio Access Network; Evolved Universal Terrestrial Radio Access (E-UTRA);

Radio Resource Control (RRC); Protocol specification

(Release 12)

The present document has been developed within the 3rd Generation Partnership Project (3GPP TM) and may be further elaborated for the purposes of 3GPP. The present document has not been subject to any approval process by the 3GPP Organizational Partners and shall not be implemented. This Specification is provided for future development work within 3GPP only. The Organizational Partners accept no liability for any use of this Specification. Specifications and reports for implementation of the 3GPP TM system should be obtained via the 3GPP Organizational Partners' Publications Offices.

18

Observation: Operation Logic is Not Arbitrary• Many network-side operations are stateful


Monitor 3G&4GMonitor 4G

Handoff to 4G Handoff to 3G

RSS1 < -110dBm

RSS1 > -110dBm RSS1 < -110dBmRSS3 > -90dBm

RSS2 > RSS1 + 3dBm


RSS1(4G) < �110 dBmRSS3(3G) > �90 dBm

RSS2(4G) > RSS1(4G) + 3 dBm

BS 3 (3G)

BS 2 (4G) BS 1 (4G)

19

Observation: Operation Logic is Not Arbitrary• Many network-side operations are stateful and interactive




RSS1 < -110dBm


RSS2 > RSS1 + 3dBm

BS 3 (3G)

BS 2 (4G) BS 1 (4G)

Meas Control: Monitor 4GMeas Report: RSS2> RSS1+3

Handoff command: to BS2

BS 1 (4G)

20

Solution: Online state machine inference

State Machine Inference: Partial Recovery• Runtime sample sequence 1


Meas Control: Monitor 4GMeas Report: RSS2> RSS1+3

Handoff command: to BS2

BS 1 (4G)



RSS1 < -110dBm


RSS2 > RSS1 + 3dBm



RSS1 < -110dBm


RSS2 > RSS1 + 3dBm

BS 3 (3G)

BS 2 (4G) BS 1 (4G)

21

State Machine Inference: Partial Recovery• Runtime sample sequence 2


Meas Control: Monitor 4GMeas Report: RSS1<-110

Meas Control: Monitor 3G&4G

BS 1 (4G)



RSS1 < -110dBm


RSS2 > RSS1 + 3dBm



RSS1 < -110dBm


RSS2 > RSS1 + 3dBm

Meas Report: RSS2>-90Handoff command: to BS3



RSS1 < -110dBm


RSS2 > RSS1 + 3dBm



RSS1 < -110dBm


RSS2 > RSS1 + 3dBm

BS 3 (3G)

BS 2 (4G) BS 1 (4G)

22

State Machine Inference: Aggregation a


Meas Control: Monitor 4GMeas Report: RSS1<-110

Meas Control: Monitor 3G&4G

BS 1 (4G)

Meas Report: RSS2>-90Handoff command: to BS3



RSS1 < -110dBm


RSS2 > RSS1 + 3dBm



RSS1 < -110dBm


RSS2 > RSS1 + 3dBm

BS 3 (3G)

BS 2 (4G) BS 1 (4G)

23

MobileInsight APIs

AnalyzersMonitor API More tutorials:http://metro.cs.ucla.edu/mobile_insight/tutorials.html

src =OnlineMonitor()lte_rrc_analyzer =LteRrcAnalyzer()wcdma_rrc_analyzer =WcdmaRrcAnalyzer()lte_rrc_analyzer.set_source(src)wcdma_rrc_analyzer.set_source(src)src.run()

24

Showcase ExamplesHow can MobileInsight stimulate new apps and research?

25

Example 1: Fix Our Phone’s Network Failures• How: Track protocol state dynamics• Root cause: device-side misconfiguration• Fix: disable VoLTE when device is in 3G

Data service setup requestQoS class = 1 (voice)

……

Data service setup rejectCause: QoS unsupportedData service setup request

QoS class = 1 (voice)Data service setup rejectCause: QoS unsupported

Session_Inactive

Active_Pending

Session_Active

Inactive_Pending

Session_Inactive

Active_Pending

Session_Active

Inactive_Pending

Session_Inactive

Active_Pending

Session_Active

Inactive_Pending

26

HardwareSoftware

?4 signal bars, butwhy no data service?

Example 2: Boost Our Phone’s Data Speed• How: Analyze inferred handoff decision logic• Root cause: suboptimal FCFS strategy• Advice: disable 2G when 4G is available

Meas ControlMonitor 2G & 4G

Meas Report: 2G available

Meas Report: 4G available(ignored by base station)

Handoff command: to 2G

Monitor 2G & 4G

Handoff to 4GHandoff to 2G

2G Meas Report 4G Meas Report

Monitor 2G & 4G



Monitor 2G & 4G



27

HardwareSoftware

?2G

Why switch to slow 2Gdespite good 4G coverage?

Research Empowered by MobileInsight• Security loophole detection, failure resolution, handoff advisor, etc.

• iCellular [NSDI’16]: Device-customized multi-carrier roaming

• MMDiag [SIGMETRICS’16]: mobility misconfiguration detection

28

EvaluationCoverage, performance, accuracy and system overhead

29

Wide Coverage of Phone Models

30

Mobile OS

Chipset Feasibility

Android

Qualcomm ✔

MediaTek ✔

Intel XMM ✔

iOS All ✔

Mobile OS

Chipset Feasibility Current Version(2.1.1)

Android

Qualcomm ✔ ✔

MediaTek ✔ ✘

Intel XMM ✔ ✘

iOS All ✔ ✘

• Current version: rooted Android with Qualcomm chipset• MTK/Intel and iOS support: under development

Wide Coverage of Cellular Protocols/Messages• 3G/4G signaling messages and 4G-L1/L2 messages• Characterization of cellular message patterns

Dataset size 245.24GBTotal messages 72,389,300Protocol Layers 4G-PHY (71.8%),

4G-MAC (9.0%), 4G-PDCP (8.3%),3G/4G-RRC (10.0%),3G/4G-MM/SM (0.6%),3GPP2-EvDo/CDMA (0.3%)

31

Real-time Processing of Cellular Messages• 99% messages’ parsing and analyzing within 0.8ms• Worst case observed: 33ms

0

20

40

60

80

100

0 2 4 6 8 10

CD

F (

%)

Proc time (ms)

6PS5

Tribute

32

Accurate Cellular Protocol Analytics• Tracking Protocol State Dynamics: identical as QXDM• Same cellular message sources

• Inference of Handoff Operation Logic• 10-fold cross validation: 87.5%~95.3% prediction accuracy

0

100

200

300

400

500

600

0 100 200 300 400 500 600

Dep

artu

re r

ate

(pkt

/ s)

Arrival rate (pkt / s)

y = x

(a) LG Tribute

0

100

200

300

400

500

600

0 100 200 300 400 500 600

Dep

artu

re r

ate

(pkt

/ s)


y = x

(b) Samsung Galaxy S5

0

100

200

300

400

500

600

0 100 200 300 400 500 600

Dep

artu

re r

ate

(pkt

/ s)


y = x

(c) Huawei Nexus 6PFigure 8: MOBILEINSIGHT message departure rate as a function of arrival rate from hardware cellular interface.

0 5

10 15

0 5000 10000 15000 20000 25000 30000time

(ms)

Tribute S5 6P

(a) Illustration of 30000 samples with light load: <500 msg/s

0

20

40

60

80

100

0 0.5 1 1.5 2

CD

F (

%)

Proc time (ms)

TributeS56P

95

100

0 0.5 1

(b) CDF: light traffic

0

20

40

60

80

100

0 2 4 6 8 10

CD

F (

%)

Proc time (ms)

TributeS56P

95

100

0 5 10 15 20

(c) CDF: heavy trafficFigure 9: MOBILEINSIGHT’s processing time under light (<500

msg/s) and heavy (�500 msg/s) loads.

Table 8: 4G protocol state-relevant parameters from 5 carriers.AT&T T-Mobile Sprint Verizon CMCC

RRC

T1 (ms) 200 (99.9%) 500 (100%) 200 (54.4%) 200 (99.5%) 60 (100%)100 (0.1%) 100 (31.2%) 10 (0.5%)

10 (14.4%)T

shortDRX

(ms) 20 80 0 (85.6%), 80 40 20T2/T

shortDRX

1 1 0 (85.6%), 2 0, 4 2Conn! 10 10 4 10 13idle (s) ±0.9 ±0.6 ±0.5 ±0.8 ±0.5

EMM Encryption EEA2 EEA2 EEA2 EEA2 NULLIntegrity EIA2 EIA2 EIA2 EIA2 EIA2

ESMQoS class 8 6 9 1 (voice), 5 9Max dlink 150Mbps 256Mbps 200Mbps 138Mbps (data), 256Mbps

bitrate 80Kbps (voice)

type of cells. We also observe that intra-frequency handoff (overthe same frequency) is the most common one with the highest pri-ority. The handoff logic varies with carriers. Indeed, operatorshave freedom to customize their policies. We have also applied thesame inference to AT&T and T-Mobile 3G, and verified that sim-ilar per-frequency aggregations are used. One major difference isthat, only intra-frequency handoff is allowed for each 3G frequencyband. This is possibly because that 3G supports soft handoff be-tween cells under same frequency. So intra-frequency handoff ismore preferred to provide more seamless network service.

Given that carriers do not publicize their operation logics, welack the ground truth to directly assess the correctness of our infer-ence algorithm. We thus devise an indirect approach. The idea isto predict whether a handoff indeed occurs based on the inferredhandoff logic. If our inferred state machine is entirely identical tothe one used by the operator, we could predict the handoff with-out errors. The accuracy reflects the completeness and correctnessof prediction. Note that, the inferred one may be incomplete dueto device-based perception only. In a word, the prediction accu-racy serves as a lower bound of the inference correctness. We runthe 10-fold cross validation [49]. Table 9 shows that MOBILEIN-SIGHT yields high prediction accuracy (87.5% to 95.3%) for fourUS carriers. It is slightly lower for Verizon because of a relativelysmaller dataset. This validates that in practice, MOBILEINSIGHTis effective in learning the network-side operation logic. There areonly false positive errors (i.e. the handoff does not occur when the

Table 9: Accuracy for predicting upcoming handoffs.

AT&T T-Mobile Sprint Verizon#Samples 11,050 10,178 10,042 2,741Accuracy 90.7% 91.8% 95.3% 87.5%

Table 11: Log storage overhead in MOBILEINSIGHT.

Control only Control+data Control+data+PHYHex log size (1-hour) 0.36MB 3.89MB 41.40MB

Avg. log growth speed 0.79Kbps 8.64Kbps 92.00KbpsAvg. log size reduction 115.6x 10.62x N/A

radio measurement meets the trigger conditions). No false nega-tive errors are observed since the handoff decision indeed considersradio quality [36]. These results confirm that, our inference is al-most complete, covering most rules used by operators through thedevice-only observations.

7.3 System OverheadCPU and memory. We assess MOBILEINSIGHT’s CPU andRAM usage. We enable all the supported messages (Table 4) andprotocol analyzers (Table 5), and record their usage every secondwhen we export parsed messages and analysis results into a file.We divide the resource consumption into two parts: by MOBILEIN-SIGHT service and by other relevant operations (e.g., file I/O). Fig-ure 11 shows the average CPU and memory usage. Regarding MO-BILEINSIGHT itself, the CPU consumption for monitor and ana-lyzer modules is about 1-3% for S5 and 6P, and 6-7% for LG Trib-ute. For all phone models, the average RAM usage is below 15 MB(maximum < 30MB).Energy consumption. We assess the energy cost by measuringthe phone’s power consumption with/without MOBILEINSIGHT,through a Monsoon power meter [50]. We use only Samsung S5because their back-covers are easy to remove for power measure-ment. Due to space limit, we show the results in three typical sce-narios in Table 10. The results in many other scenarios are similar.On average, MOBILEINSIGHT consumes 11-58 mW extra power.The higher the traffic load, the larger the power value. However, itsrelative ratio remains within 4%, regardless of traffic variations.Storage overhead. We last gauge the storage required for MO-BILEINSIGHT cellular logs. We run MOBILEINSIGHT on SamsungS5 phone in idle mode for 1-hour, and record the raw log size andgrowth speed. We repeat this experiment by enabling different lev-els of cellular message types. Table 11 shows that, the log growthand storage consumption depends on the cellular message typesto be collected. When only the control plane messages (3G/4G-RRC/MM/SM)are enabled, the log growth speed is 0.79Kbps onaverage, which is 115.6x slower than the scenario when all mes-sages are enabled. Note that the data-plane and PHY-layer mes-sages grow in proportion to the mobile user data volume. Withactive user data, the log size would grow even faster. This jus-tifies our optimization of on-demand log collection (§4.3), which

33

Acceptable System Overhead• CPU utilization: 1%-7%

• Memory: 30MB at maximum

• Energy: 11-58mW extra power (on Samsung S5)

34

New Version: v2.1.1• More cellular protocol support

• Cellular data sharing

• New APIs for mobile applications

• In-phone cellular log browser

• …

35

Toward Open and Large-Scale Cellular Datasets• Initial dataset release• 30+ users, 8 US/Chinese network operators• 13-month collection (Jul 2015 – Sep 2016)• ~245GB 3G/4G cellular traces

• Everyone can contribute to the dataset anywhere, anytime!• Online trace submission or background data sharing

36More information:http://metro.cs.ucla.edu/mobile_insight/insightshare.html

New Research Opportunities Made PossibleMobile big data analytics Cellular protocol refinements

Security threats detections Cross-layer app enhancements

37

Conclusion• Open access to cellular operations benefits everyone• Mobile users, researchers, developers and even operators

• MobileInsight: a first effort toward an open cellular world

• More community efforts are needed for extension• A tool for the community and by the community

38

Try MobileInsight and explore more!http://metro.cs.ucla.edu/mobile_insightYuanjie Li1, Chunyi Peng2, Zengwen Yuan1, Jiayao Li1, Haotian Deng2, Tao Wang3

1University of California, Los Angeles2The Ohio State University3Peking University

MobileInsight - CS | Computer Scienceweb.cs.ucla.edu/~zyuan/slides/mobicom16-mobileinsig… · ·...

Documents

Transcript of MobileInsight - CS | Computer Scienceweb.cs.ucla.edu/~zyuan/slides/mobicom16-mobileinsig… · ·...