Post on 16-Mar-2018
T-Mobile Technology Meetup - E2E Mobile Device and Network Performance
QoE Lab| Device Development |TPD
August 25, 2014
T-Mobile Confidential 1
Contributors: Jie Hui, Kranthi Sontineni, Zach McPherson, Peter Myron
Agenda
• Introduction to QoE lab
• E2E Performance - Network Side – How YouTube Performance is Improved in T-Mobile Network
• E2E Performance - Device Side – QoE Analyzer demo
– Hands-on trial
T-Mobile Confidential 2
Introduction to QoE Lab
T-Mobile Confidential 3
Why
Make the customer experience more enjoyable!
QoE is measurement of the enjoyment of the customer experience.
Mobile QoE 2.0
Network
Experience Real User Speed
Experience Real User Application
Experience
Crowd-sourced from devices Network-instrumented
1.0 1.5 2.0
Figure originally from : I. Grigorik, “High Performance Browser Networking: What every web developer should know about networking and web performance”. O’Reilley Media, 2013
What
T-Mobile Confidential 6
Measure QoE
Identify Problem
Root Cause Analysis
Optimize
Scaling Through Partnerships and Automation Development
T-Mobile Confidential
Operations
NW Engineering
“How YouTube Performance is Improved in T-Mobile Network”, Velocity 2014
[Info] [Video]
T-Mobile Confidential 8
QoE Analyzer
T-Mobile Confidential 9
QoE Analyzer
Turns data into
something which is
meaningful in Seconds
10
Motivation to build QoE Analyzer
Test QoE Analysis
Automated log parser and charting tool to assist in root cause analysis for QoE issues.
QoE Analyzer TCP log QXDM log App log
TCP analysis Radio analysis
Note: QoE Analyzer implements part of T-Mobile patent “Network Device Trace Correlation”, Jie Hui, Himesh Bagley, Jeff Smith, US20140119196
QoE Analyzer - Design Modem log (.isf)
Packet trace (.pcap)
PDCP/RRC/RLC– Modem QXDM (.isf)
IP/Transport– OS (.pcap)
App/ e.g YouTube (logcat)
UE
Server
IP/Transport (.pcap)
IP/Transport (.pcap)
GGSN/PGW
12
RUM performance
Device QoE
QoE Analyzer
Note: QoE Analyzer implements part of T-Mobile patent “Network Device Trace Correlation”, Jie Hui, Himesh Bagley, Jeff Smith, US20140119196
QoE Analyzer
What Analysis can QoE analyzer do?
• TCP Connection times
• DNS Lookup time
• Throughput graphs
• TCP Failure Analysis
• TCP retransmission analysis per Stream/IP
• TCP sequence Analysis
• IP filtering.
• E2E analysis-Correlate Packets between UE, SGW/PGW and server
• Supports IPV4/IPV6
T-Mobile Confidential 14
PCAP Analysis Radio Analysis Logcat Analysis
• 3G/4G/LTE Radio state
Transition summary
• 3G/4G/Radio bearer
Reconfiguration
response time.
• 3G/4G/RLC UL & DL
PDU Retransmissions.
• LTE PDCP and RLC
throughput graphs
• 3G/4G/ IP packet and
RLC PDU Correlation.
YouTube analysis
• Video Startup time
• Video buffer count
• Buffer Ratio
PCAP Analysis
T-Mobile Confidential 15
Input Output
1 DNS Lookup Time Average: 0.070 Min: 0.019 Max: 0.122
2 TCP -SYN to SYN/ACK RTT Average: 0.069 Min: 0.025 Max: 0.124
3 TCP -SYN/ACK to ACK RTT Average: 0.002 Min: 0.001 Max: 0.006
4 TCP Connection time( TCP SYN -ACK) Average: 0.071 Min: 0.028 Max: 0.125
6 Total Number of TCP retranmissions Total: 2774
7 Total Number of TCP Duplicates Total: 3282
8 Total number of TCP Reset's Total: 6
9 Total number of TCP Packets Total: 19456
10 Total Size of TCP packets MB: 13.686
KPI's Analyzed from wireshark Trace Totals:
TCP Connection Summary
TCP Failure Analysis
T-Mobile Confidential 17
Input Output
TCP Packet Sequence Analysis
T-Mobile Confidential 18
Input-choose specific IP
Output
TCP - Cross IP-Radio and Cross-Node Analysis
T-Mobile Confidential 19
Input-Pcap file from UE, SGW,Server
Output
Radio Analysis
T-Mobile Confidential 20
Radio State Transition Analysis – 3G/4G
T-Mobile Confidential 21
Input Output
RLC UL & DL PDU Retransmission analysis- 3G/4G
T-Mobile Confidential 22
Input Output
Radio State Graph with IP Packet Overlay---3G/4G
T-Mobile Confidential 23
Input Output
Radio State Transition Analysis - LTE
T-Mobile Confidential 24
Input Output
LTE Cross Layer Throughput – PDCP & RLC
T-Mobile Confidential 25
Input Output
Logcat Analysis
T-Mobile Confidential 26
YouTube Video Metrics – Logcat Analysis
T-Mobile Confidential 27
Input Output
Log File StartTime Buffer Count Avg Buffer Buffer Time Rebuff Count MTBR Rebuffer Time Elapsed Time Buffer Ratio
Youtube_4G_2014_03_07_12_57_03.txt 00:02.510 0 00:00.000 00:00.000 1 02:36.220 02:36.220 02:38.730 0.00%
Youtube_4G_2014_03_07_14_50_42.txt 00:01.950 0 00:00.000 00:00.000 1 02:36.120 02:36.120 02:38.070 0.00%
Youtube_4G_2014_03_07_18_38_03.txt 00:28.620 12 00:10.849 02:10.190 13 00:11.872 02:34.330 05:13.140 45.76%
Youtube_4G_2014_03_07_19_49_12.txt 00:29.970 11 00:11.468 02:06.150 12 00:12.879 02:34.550 05:10.670 44.94%
Youtube_4G_2014_03_07_20_19_25.txt 00:30.020 13 00:10.172 02:12.230 14 00:11.016 02:34.230 05:16.480 46.16%
Youtube_4G_2014_03_07_22_01_58.txt 00:28.320 12 00:10.708 02:08.490 13 00:11.858 02:34.150 05:10.960 45.46%
Youtube_4G_2014_03_07_23_56_42.txt 00:28.700 13 00:10.085 02:11.100 14 00:11.031 02:34.440 05:14.240 45.91%
Youtube_4G_2014_03_08_00_28_07.txt 00:28.150 13 00:10.007 02:10.090 14 00:11.041 02:34.570 05:12.810 45.70%
Contact Us
Please send any question or request to get trial access for QoE Anlayzer to DeviceQoELab@T-Mobile.com.
T-Mobile Confidential 28