Modeling Web Quality-of-Experience on Cellular Networks

Athula Balachandran, Vaneet Aggarwal,Emir Halepovic, Jeff Pang, Srinivasan Seshan,

Shobha Venkataraman, He Yan

AT&T Labs- Research CMU

Rise of Mobile

Cisco Visual Networking Index 2013: Global Mobile Traffic Data Update

Familiar?

Why is QoE important?Cellular

Network Factors

Quality of Experience

User (dis)satisfaction

Revenue

Why do we need a QoE model?• Service Quality Monitoring

• Trending• Alarming

• Better system designs and resource allocations schemes

Outline

Cellular Network Factors

1. What are the network factors?2. How to extract network factors?

Quality-of-Experience

3. What QoE metrics to use?4. How to extract QoE metrics?

5. Model the relationship.

Outline

Cellular Network Factors

1. What are the network factors?2. How to extract network factors?

Quality-of-Experience

3. What QoE metrics to use?4. How to extract QoE metrics?

5. Model the relationship.

Cellular Network Factors

Signal Strength

Handovers Failures

Cell load

Throughput

Outline

Cellular Network Factors1. What are the network factors?

2. How to extract network factors?

Quality-of-Experience

3. What QoE metrics to use?4. How to extract QoE metrics?

5. Model the relationship.

Collecting Network Characteristics

• Logs collected at Radio Network Controller– Intermittent: Signal strength, Throughput– Event based: Handovers, Failures

Outline

Cellular Network Factors

1. What are the network factors?2. How to extract network factors?

Quality-of-Experience

3. What QoE metrics to use?4. How to extract QoE metrics?

5. Model the relationship.

QoE Metrics

Session Length Abandonment

Partial Download Ratio (PDR)

Outline

Cellular Network Factors

1. What are the network characteristics?2. How to extract these metrics?

Quality-of-Experience

3. What metrics to use?4. How to extract these metrics?

5. Model the relationship.

Detecting Clicks

• Challenge: Classify embedded objects vs. click from network traces.

• Current Approaches: – Idle-time based– Stream Structure

• Our Approach: Text classification

Performance News, Social, Wiki

Precision is defined as the number of correct clicks identified divided by the total number of clicks identified Recall is de fined as the number of correct clicks identified divided by the total number of clicks.

Outline

Cellular Network Factors

1. What are the network characteristics?2. How to extract these metrics?

Quality-of-Experience

3. What metrics to use?4. How to extract these metrics?

5. Model the relationship.

Correlation Analysis

Signal StrengthHandovers

Failures

Cell load

Throughput

Session Length

Abandonment

Partial Download Ratio (PDR)

Correlation AnalysisCell load

Increasing Cell load leads to worse web QoE

Correlation AnalysisSignal Strength

RSSI: Received Signal Strenth

ECNO ： How well a signal can be distinguished from the noise. Similar to SINR in WiFito signal to noise ratio

Web QoE is interference limited and not power limited.

Correlation AnalysisHandovers

Soft

IRATInter-frequency

IRAT handovers lead to worse QoE

Correlation AnalysisThroughput Failures

Web QoE is more latency-limited than throughput-limited

Downlink

Uplink

Correlation Analysis: Summary

• Cell load, IRAT handovers lead to worse QoE.• Improving ECNO leads to better QoE.• Higher RSSI worse QoE.• All other handovers, throughput, failures do

not have much impact on QoE.

Complex Inter-Dependencies

Signal Strength

Handovers

Failures

Cell load

Throughput

Unified Model

Machine Learning

Network Characteristics Web QoE metrics

QoE Model

Predictive Models

Model ML Algorithm Model RMSE

Estimate PDR Linear Regression 0.1709

Estimate Session length Linear Regression 1.703

Model ML Algorithm Model Accuracy

Predict Abandonment Decision Tree 69.12

Predict Partial Download Decision Tree 73.02

External Factors: Time of day

External Factors: Website

Unified Model

Machine Learning

Network Characteristics Web QoE metrics

QoE Model

External Factors

Predictive Models

Model ML Algorithm Old RMSE UpdatedRMSE

Estimate PDR Linear Regression 0.1709 0.087

Estimate Session length Linear Regression 1.703 1.401

Model ML Algorithm Old Accuracy

Updated Accuracy

Predict Abandonment Decision Tree 69.12 74.30

Predict Partial Download Decision Tree 73.02 83.95

Updated Model

Conclusions

• Web QoE $$• QoE metrics and network parameters– Session Length, Abandonment, PDR text classification– Network parameters RNC logs

• Network parameters impact web QoE– ECNO, Cell load, IRAT handovers

• Build accurate and intuitive models – Complex relationships ML algorithms– Incorporate external factors.

EXTRA SLIDES

Unified Model

Web QoE Model

Web QoE

Signal Strength Handovers

Failures

Cell load

Throughput

1) Estimate partial download ratio – Linear Regression 2) Estimate session length3) Predict partial download – Decision Tree4) Predict user abandonment

• Why is QoE important?• How to measure QoE?• How to improve QoE?• Why measure QoE?