9th World Telecommunication/ICT Indicators
Meeting (WTIM-11) Mauritius, 7 - 9 December 2011
Contribution to WTIM-11 session Document C/16-E 8 December 2011
English
SOURCE: LIRNEasia
TITLE: Approaches to Measuring Broadband Quality of Service Experience (QoSE)
Approaches to Measuring Broadband Quality of Service
Experience (QoSE) Shazna Zuhyle
World Telecommunication / ICT Indicators Meeting
7-9 December 2011, Mauritius
This work was carried out with the aid of a grant from the International Development Research Centre, Canada and UKaid from the Department for International Development, UK.
Defining broadband access
Modes of Access
Wireless / Mobile
Network
Fixed Network
Mobile Nomadic
LIRNEasia tests QoSE for Fixed and Nomadic use
The user is stationary
Cell handover not required
Cell handover required
2
Defining test parameters Parameters Definitions Method Threshold /
Benchmarks
Download The rate at which data is received from a server Min. file size 1MB; Max. time 3 mins.
-
Upload The rate at which data is sent to a server 2 MB file -
Latency (RTT; Round Trip Time)
The time taken for a packet to reach its destination and back
The average of 10 pings (each ping provides 3 sets of results)
< 300 ms (e.g. India, Singapore)
Jitter The variation of latency (RTT) < 50 ms
Packet loss The fraction of packets that fail to reach its destination
< 3% (e.g. India, Singapore)
Availability The number of times the broadband link can be accessed within a given time frame = (1-F/T) x 100% [where T = total number of attempts; F = number of failed attempts]
> 98% (e.g. India)
3
It’s not just about download speeds
+++ Highly relevant; ++ Very relevant; + Relevant; - Irrelevant
Service Download (kbps)
Upload (kbps)
Latency (Round Trip Time, RTT) (ms)
Jitter (ms)
Packet Loss (%)
Browsing (Text) ++ - ++ - -
Browsing (Media) +++ - ++ + +
Downloading +++ - - - -
Transactions - - ++ + -
Streaming media +++ - ++ ++ ++
VOIP + + +++ +++ +++
Games + + +++ ++ ++
RTT has implications on client-server interactive systems
- Jitter adds to the ‘noise’ of the transmission
- Packet Loss affects streaming media
Source: Gonsalves, T.A & Bharadwaj, A. (2009) 4
Multiple domains; multiple times
4. Averages of multiple readings for each time slot 5. Varying server locations
- ISP domain - International domain - National domain
1. 6 Parameters: Download, Upload, RTT, Jitter, Packet Loss & Availability 2. Multiple days: during the week and on weekends 3. At six time slots: 0800, 1100, 1500, 1800, 2000, 2300 hrs (3 readings per slot)
Diagnostics can be used to identify potential bottle-necks in the
network. Value for money across packages can be compared.
Consumer protection can be enhanced based on data 5
Identifying bottlenecks in the network using the results
0
500
1000
1500
2000
2500
800 1100 1500 1800 2000 2300 Time
Download from ISP Domain
Qubee (1 Mbps) Dhaka, BD Banglalion (512 kbps) Dhaka, BD
Airtel (256 kbps) Bangalore, IN BSNL (256 kbps) Bangalore, IN
Airtel (2 Mbps) Chennai, IN BSNL (256 kbps) Chennai, IN
Airtel ( 2 Mbps) Mumbai, IN MTNL (320 kbps) Mumbai, IN
Airtel (2 Mbps) New Delhi, IN MTNL (512 kbps) New Delhi, IN
SLT (2 Mbps) Colombo, LK Telkom Speed (512 kbps) Jakarta, ID
800 1100 1500 1800 2000 2300 Time
Download from International Domain
The problem is in the International connectivity
Source: September 2011, LIRNEasia 6
The trouble with advertising theoretical speeds
7
0
5
10
15
20
25
30
35
40
45
0800 1100 1500 1800 2000 2300
Del
iver
ed v
s. A
dve
rtis
ed (
%)
Time
Dialog (upto 3.6 Mbps) Colombo, LK Mobitel (upto 1 Mbps) Colombo, LK Three (upto 3 Mbps) Jakarta, ID
Globe Tattoo (upto 3.6 Mbps) Manila, PH Smart (upto 3.6 Mbps) Manila, PH
Source: September 2011, LIRNEasia
Value for money
Source: September 2011, LIRNEasia 8
0
20
40
60
80
100
120
140
0800 1100 1500 1800 2000 2300
Do
wn
load
(kb
ps)
per
do
llar
Time
Qubee (1 Mbps) Dhaka, BD Banglalion (512 kbps) Dhaka, BD Airtel (256 kbps) Bangalore, IN
BSNL (256 kbps) Bangalore, IN Airtel (2 Mbps) Chennai, IN BSNL (256 kbps) Chennai, IN
Airtel ( 2 Mbps) Mumbai, IN MTNL (320 kbps) Mumbai, IN Airtel (2 Mbps) New Delhi, IN
MTNL (512 kbps) New Delhi, IN SLT (2 Mbps) Colombo, LK Telkom Speed (512 kbps) Jakarta, ID
Possible ways to monitor quality
Self regulation by operators
Direct monitoring by
regulators
User satisfaction
surveys
Demand-side (user) testing
Intrusiveness on network
+ +++/+ - +
Regulator participation
++/+/- +++ Depends on survey
-
Operator participation
+++ +++ Depends on survey
-
User participation
- - +++ +++
Subjectivity of results
++/+/- + +++ +
9
+++ High ++ Medium + low - none
Implementing QoSE indicators
• Ideally co-ordinated by regulators
• Follow one methodology so that results are comparative
• Undisclosed locations (not from the office)
• Publicly available results consumers can make informed decisions
• Policies can be based on diagnostics
10
Regulator’s efforts in South Asia
Regulator
Co
nsu
ltan
cy?
Met
ho
d
Pu
blis
he
d?
Re
po
rts
Pu
blis
he
d?
Test
fr
eq
ue
ncy
Parameters
Availability Throughput Latency (RTT)
Jitter Packet Loss
ATRA (AF) The indicators page is inaccessible – www.atra.gov.af
BTRC (BD) - - - - -
BICMA - - - - - - - - -
TRAI (IN) - Quarterly - -
CAM (MV) Website cannot be found
NTA - - - - - -
PTA - -
TRC (LK) - - Monthly - - - 11
Next Steps for LIRNEasia
• The software to be revamped
• Offered to regulators at no cost – just ask
LIRNEasia’s QoSE Diagnostic Tool (Beta):
www.broadbandasia.info
12
Key Takeaways
• There are other metrics apart from download speed that affect a user’s broadband experience
• These metrics (such as RTT, Jitter and Packet Loss) become more important when one goes beyond simply browsing the web
• Diagnostics should ideally be coordinated by regulators; not from a known location
• Results should be made public to benefit consumers
13
Thank You
More Info: www.lirneasia.net
Keywords: Benchmarks, ICT Indicators, Broadband QoSE
Contact: [email protected]
This work was carried out with the aid of a grant from the International Development Research Centre, Canada and UKaid from the Department for International Development, UK.
Top Related