Perceptual Video Metrics, a new vocabulary for QoE

Jeremy BenningtonCheetah Technologies

Video Quality Management

Quality of Service

Quality of Experience

Perceptual Quality Monitoring

What is a Mean Opinion Score (V-Factor)?

V-FACTOR

4 – Perceptible 3 – Slightly Annoying2 – Annoying

5 – Imperceptible

1 – Very Annoying 1 – Very Annoying

5 – Imperceptible 4 – Perceptible 3 – Slightly Annoying2 – Annoying

V-FACTOR

V-Factor MOS shows a significant change in quality over 12hours Issue with the encoding from the programmer

V-Factor MOS shows a significant change in quality over 3hours Issue with the program (like an old movie)

Gradual degradations of visual quality must be trended over a longer period before alarming

Only monitoring using perceptual metrics can identify these events

Real World Case StudiesGradual Degradations

12 Hours 3 Hours

Sudden drops in quality are common and un-related to MPEG-TS or VCL These events should be alarms for operations These events (when seen at the ingest) should be used to screen trouble

tickets & truck rolls

Real World Case StudiesSudden Degradation

Quality TrendsWeek over Week with threshold violations

Consistent week-over-week average quality Events are easy to find with a threshold violation Useful in determining problematic programs and reducing trouble tickets

Quality Trends Week over Week with periodic trend

Consistent week-over-week periodic quality Events are hard to find with a threshold violation Correlation and moving average are easiest ways to find errors Pattern caused by different programming from midnight to 6am

Quality Trends East versus West Feeds

Consistent east-to-west periodic quality Correlation is useful for determining errors in East feed Could be used to catch errors before they hit the west

Junk in, junk out If what your receive is poor quality only perceptual quality

tools will “see” it...and your customers Poor quality passes thru the system to all the viewers Perceptual quality monitoring identifies content issues to

compliment QoE/QoS

Perceptual Quality Monitoring is essential to both traditional video delivery and newer streaming techniques

“So What”

Working with Programmers Scorecards: Provide each a ranking of their channel(s)

weekly & monthly Identify periodic quality issues: paid programming, shows,

remotes

Empowering Operations Is the issue the programmer or in “our” network? Is the issue the encoding or distribution? Is the issue visible to the subscriber? How often does this happen?

Reducing Cost CSRs: We are aware of the problem CSRs: This is typical for this time of day CSRs & Operations: Reduction in trouble tickets

“So What”

Perceptual Quality Monitoring (PQM) Models the human eye to find visual artifacts Identifies baseline quality & trends; and alerts at ingest &

encode Replaces manual methods of video monitoring at the

Headend

End to End Video Management QoS to determine network fidelity

QoE to estimate QoS and MPEG impacts on subscribers

PQM to objectively determine video quality

QoS, QoE and PQM correlation achieves:

• Prioritization of issues based on impact

• Faster time-to-fix and isolation of issues

• Reduction in costs & increases customer satisfaction

Summary