- 1 -© FTW 2014
Assessing and quantifying the impact of network bandwidth fluctuations on Web-
QoEAndreas Sackl1, Pedro Casas1, Raimund
Schatz1, Werner Wiedermann2, Ralf Irmer3
1) FTW Telecommunications Research Center Vienna,
Austria
ETSI TC STQ Workshop on Telecommunication Quality beyond 2015 21-22 October 2015, Vienna, Austria
3) Vodafone Group Research &
Development, UK
2) Telekom Austria Group, Austria
- 2 -© FTW 2014
Introduction
t
Throughput
t
Throughput
constant fluctuations
Resulting Web QoE(well explored)
Resulting Web QoE?
Several (open) research questions: Influence of various fluctuation patterns on Web QoE (Sackl 2014 et al.) Identify KPIs regarding fluctuations Influence of single bandwidth outages on Web QoE How to model throughput fluctuations
Research trend
- 3 -© FTW 2014
Web QoE User Study
Task 3: Browsing a Photo Gallery
Task 1: Browsing a News Site
Task 2: Browsing Google Maps
Web QoE User Study (N=29)
- 4 -© FTW 2014
Single Bandwidth Outages – Results (1)
MOS (Outage Duration) fitting curve
MO
S O
utag
e A
nnoy
ance
notannoying
veryannoying
Photo Gallery News Site
MO
S O
utag
e A
nnoy
ance
e.g. 2 sec. outage
Thro
ughp
ut
30 seconds
Web Usage
- 5 -© FTW 2014
Single Bandwidth Outages – Results (2)
High acceptance rate
50% detection rate
Detection rate is content-dependent
- 6 -© FTW 2014
Conclusions for Single Outages
For Web browsing, outages of 2 or more seconds already impact perceived user quality, whilst outages of up to 4 seconds are still considered “acceptable” by most users
Outages in Web Scenarios are not always detected by the users
The impact of single outages are content-dependent
- 7 -© FTW 2014
Goal & Evaluated Patterns
Pattern P1 Pattern P2 Pattern P3
DBW = Downlink BandwidthADBW = Average Downlink Bandwidth
Progressive outage/recovery e.g. tunnels
Fast changes e.g. heavy loaded cells or interference sources
like trees
High/low profile with fast short-scale
variations
Goal: Identify KPIs of throughput fluctuations to predict resulting MOS values
- 8 -© FTW 2014
Approach (1)
ADT = Average Download Throughput (simple approach), CF = Correcting FactorEADT = Effective Average Download Throughput (novel approach)
MOS = 0.45∗log (DBW)+2.48
Common, non-fluctuations approach i.e. creating simple model from user ratings and constant bandwidth conditions
MOS = 0.45∗log (ADT)+2.48
MOS = 0.45∗log (EADT)+2.48 ✔
✖
(constant throughput)
fluctuation pattern
- 9 -© FTW 2014
Approach (2)
How to calculate the Correcting Factor CF? => 5 models possible
LTD model SLTD model JT model AREA model DOUBLE model
How to get EADT? => EADT=CF x ADT
CF = 1-(t1+t2)/t0
Threshold = ADT/2
- 10 -© FTW 2014
Results of Model Verification (for Google Maps)
Pattern MOS ADT
P1 2.9 5.9 Mbps
P2 3.4 5.1 Mbps
P3 3.1 4.3 Mbps
MOSADT
3.3
3.2
3.1
MOSLTD MOSSLTD MOSJT MOSAREA MOSDOUBLE
3 3.2 3.3 3.2 3
2.9 3.2 2.9 3.1 2.6
2.5 3.1 2.8 3.1 2.3
Directly related to Patterns i.e. most
accurate MOS
Calculated from the pattern
Utilization of fitting curve: MOS = 0.45∗log (ADT)+2.48(Note: fitting curve is extracted from constant BW conditions)
Simplest model (still quite good!)
EADT=CF x ADT
Depends on the model
With the resulting EADT, the MOS value is calculated via the fitting curve MOS = 0.45∗log (EATD)+2.48
- 11 -© FTW 2014
Conclusions for Fluctuation Models
We presented first approach to model throughput fluctuations 5 Models to calculate Correcting Factor CF
- LTD: Considering time below certain throughput threshold- SLTD: Like LTD, but short drops are ignored- JT: Sliding Window- AREA: how long below threshold and how deep- Double: Considering two thresholds
Best fitting Model depends on specific pattern In our study, MOSADT leads to good results Evaluated patterns
might not be optimal for evaluated services More evaluation is needed
- We examined three patterns more types and/or more realistic patterns e.g. from real network traces
- We examined Google Maps more services- 5 ways to calculate CF find more appropriate ways
ReasonableKPIs
- 12 -© FTW 2014
Thank you for your [email protected]
A. Sackl, P. Casas, R. Schatz, J. Lucjan, and R. Irmer. Quantifying the Impact of Network Bandwidth Fluctuations and Outages on Web QoE. In Proc. International
Workshop on Quality of Multimedia Experience, Messinia, Greece, 2015.
- 13 -© FTW 2014
Backup
- 14 -© FTW 2014
Single Bandwidth Outages – Study Design
Evaluated outage durations: 2, 4, 8, 16 seconds Duration of each conditions: 30 seconds (in contrast to
common 2 minute duration) Each outage duration was evaluated three times per User
(reason: short, single outages might be missed)
e.g. 2 sec. outage
Thro
ughp
ut
30 seconds
Web Usage +Questionnaire
about subjective
quality perception
3x per user for each outage duration
- 15 -© FTW 2014
Single Bandwidth Outages – Results (3)News Site
Photo Gallery
Three (short) iterations are feasible and reasonable in the context of Web QoEstudies
- 16 -© FTW 2014
Results of Model Verification
- 17 -© FTW 2014
Approach
ADT = Average Download Throughput (simple approach)EADT = Effective Average Download Throughput (novel approach)CF = Correcting Factor
How to get EADT? => EADT=CF x ADT
- 18 -© FTW 2014
Study Design – Overview & Tasks
Empirical Web QoE user study at FTW´s iLab 29 users (13f/16m) Mean age: 34.7y Median age: 31y 70% students, 43% employees
70% of participants completeduniversity/bachelor studies
- 19 -© FTW 2014
Results of Model Verification
MOS = 0.45∗log (DBW)+2.48Pattern MOS ADTP1 2.9 5.9 MbpsP2 3.4 5.1 MbpsP3 3.1 4.3 Mbps
ADT is calculated via the pattern
- 20 -© FTW 2014
Conclusions for Single Outages
From a methodological point of view, it is reasonable to have- 30 seconds web sessions- Single outage events (instead of outage patterns)- 3 Iterations per condition
- 21 -© FTW 2014
1) Single Bandwidth Outages
- 22 -© FTW 2014
2) Fluctuation Model Verification
Top Related