Does measuring subtitle quality really make any difference?

Presentation byAlex Varley

Welcome to the Future

SubtitHAL 9000

Automatic scanning result:

Reading speed = 182 wpm 91.87% accuracy

This is your second offence.

Your broadcast license will be suspended for 20 hours. You will be automatically fined €40,000.

Metrics are much better.

Easy to measure

Easy to understand

Easy to see errors

Easy to regulate

Clear rules

This type of approach works best for everyone but the viewer

Viewer

Regulator, supplier, broadcaster

Different countries = different rules

Everybody agrees on some rules

Subtitles should be accurate

Non-live subtitles should be synchronised

Subtitles must not cover on-screen information

100%-75%What does accuracy mean?

Accuracy

Delay

Time lag for live subtitles:

3-7 secondsOr delay the broadcast

1 hour4 hours

24 hours

72 hours

96 hours

?

A live program is

Before broadcast

Reading speed

120 wpm – no limits

180 wpm used to be popular

If you make everything measurable then people focus only on the metrics

In Australia quality will be measured by:

• Readability• Synchronisation• Accuracy

The key outcome is can the viewer comprehend the program?

In Australia we recognise that:

Live subtitling is where all of the errors and problems usually occur

So live subtitles should be a last resort,

Even for “Live” programs

Live sport will have live scrolling subtitles

News should have a mix of block and scrolling subtitles.

These all impact on quality of live subtitles

Training

Preparation time

Complexity of output

Regulation

Market pricing

• Do viewers know what to expect?• Are quality errors repeated?• Is every supplier doing the same thing?• Are you acting consistently?• Do you work with suppliers to fix issues?• Is the market working?

As a regulator it is better to focus on:

Punishing one-off errors is a waste of time

But letting cowboys into the market means that everybody has to be a cowboy.

Contact: alex.varley@mediaaccess.org.auwww.mediaaccess.org.au

Thanks to Chris Mikul who sits through the hours of regulator meetings and helped research this presentation.