Cerify – File Based Quality Control Yannick LE DREAU.

Cerify – File Based Quality Control

Yannick LE DREAU

Tuesday 18 April 2023

Outline

The Business Case

Cerify Workflows and Applications


The Business Case


Business types

Segmentation

– Business areas– Content Services– Traditional Broadcasters– Advertiser Services– VoD & IPTV– Post Production Houses

– Applications– Ingest– Repurposing– Delivery


Business area examples

Segments include– Content Services

– Ingest once, re-purpose many– iTunes encoding and delivery– archive/asset management at high bitrate encoded video

– Traditional Broadcasters– Ingest once, re-purpose many– Ingest and playout of high bitrate encoded video

– Advertiser Services– Advertisement/commercial encoding and delivery

– VoD & IPTV– Encoding for web, VoD, mobile

– MPEG 2 Transport streams– 3GPP files– Windows Media

– Post Production Houses Content services– File based workflow and delivery to broadcaster

– Mainly commercials


Typical Applications of different “SD” codecs

Video Codec Typical applications (new)

MPEG-2 Broadcast, cable, VOD, DVD

H.264 / AVC / MPEG-4 Pt 10

Broadcast, cable, VOD, HD, Internet, HD video disks*

WM9/VC-1 Broadcast, HD, Internet TV, HD video discs – Blu-Ray

DV/DVC-Pro 25/50 Broadcast, Acquisition, Editing

MPEG-4 Pt 2 Mobile


Typical Applications of different “HD” codecs

Video Codec Typical applications (new)

DNxHD/VC-3 Editing, Broadcast Playout

AVC-Intra Acquisition, Editing

Pro-Res Editing, iTunes

JPEG 2000 D-Cinema, broadcast internal use/archive

DVC-Pro100 Acquisition, Editing, Broadcast Playout


MPEG-2: Typical Configurations

SDTV, DVD– MP@ML, 720 × 576, 4:2:0 colour format, 25 fps

HDTV– MP@HL, 1920 × 1080i or 1280 × 720p, 4:2:0 colour format, 25 fps

Ingest SD (Sony IMX MPEG 2 I-Frame Only)

422P@ML, 720 × 608, 4:2:2 colour format, 25 fps

Ingest HD (Sony XD-Cam MPEG 2 long GOP)– 422P@HL, 1920x1080 4:2:2 colour format, 25fps


AVC/H.264/MPEG4 part 10

Is an “open standard”, compared to closed formats, like Microsoft's Windows Media

Performs significantly better than any prior standard can, under a wide variety of circumstances in a wide variety of application environments

H.264 can often perform radically better than MPEG-2 video—typically obtaining the same quality at half of the bit rate or less

Can be carried by MPEG-2 Transport Streams making it increasingly used in HD VoD applications.

AVC-Intra is a Panasonic P2 implementation of Intra frame coded AVC. Gaining popularity for acquisition and editing.


DNxHD/VC-3 Avid’s DNxHD is a HD codec technology engineered to create

mastering-quality HD media at reduced bit rates to allow HD editing and production using SD infrastructure.

In January 08 it was approved to be compliant with SMPTE 2019 VC-3

There is support from EVS and Omneon amongst others for playout.

Image © Avid.com


DNxHD Key benefits

– Optimal mastering picture quality

– Reduced storage requirements – Enables real-time HD sharing and collaboration – Improved multi-stream performance

– Specifically engineered for multi-generation compositing and post production

– Full frame raster sampling

– Choice of 8- or 10-bit sampling as well as three user-selectable bit rates

– Open standard: HD Codec source code is licensable free of charge


Apple Pro Res 422

Unlike DNxHD, ProRes 422 provides full functionality at advanced resolutions (2K and 4K cinema) and SD.

It is DCT based, and is therefore simpler to decode than distribution oriented formats like H.264.

Full-width 1920x1080 and 1280x720

4:2:2 chroma sampling

8 and 10-bit sample depth

I frame-only encoding

Variable bit-rate (VBR) encoding

Normal 145 Mbit/s and High-Quality 220 Mbit/s for HD resolution

Normal 42 Mbit/s and High-Quality 63 Mbit/s for SD resolution

Fast encoding and decoding (both at full size and half size)


MXF – Wrapper

Media eXchange Format

Developed as a global wrapper format for all video, audio and metadata

Specification is layered. “OP1A” configuration is most commonly used today (simplest implementation of MXF)

File interchange for networked AV file distribution

Format independent– Picture Size – Interlaced or Progressive– Audio Sampling Rates

Wraps all formats– Current and future formats– Metadata only files allowed – Compressed and uncompressed


Broadcast Operational challenges

Ingest and Playout are in transition– Analogue to Compressed Digital

Methods of storing video are changing– Tape to File based stored on video Servers

Many different new formats– Terrestrial, Satellite, Cable, VoD, IPTV– Sub SD, SD, HD,– MPEG-2, MPEG-4, H.264/AVC, VC-1 etc

Content is being re-purposed at different resolutions and frame rates, using new codecs and at different bit-rates, with different technical requirements

Explosion of content from many sources for different channels and in different formats


The file based QC challenge

15

VisibleErrors

InvisibleErrors


Broadcasting Human Quality Control Challenges

Human QC/play-out testing is time consuming and expensive

Visual/audio errors are easily missed with human play-out– just by blinking or losing concentration for a second– different skill level/experience/training of testers

Simple errors – human may not get constraints correct

Consistency – cannot repeat exactly the same test on content, day after day, week after week

Person cannot easily look inside the file for example:– correctness of encoding used for compression for video, audio– bit-rates of video, audio and overall– syntax errors in the file which may prevent it playing correctly– correct playtime of the video and audio– audio on correct channels– visual quality ...


File Based video Quality Control

File Based Video QC Application Challenges– Traditional video T&M only checks baseband (luma/chroma/signal) levels

in analog and digital video streams– File-based video is different. It must be checked for

– Correct Encoding Syntax – at digital level, audio/video must be encoded without errors, so it plays out correctly at the Customer’s STB / playout device

– Correct Parameters – audio/video bitrates, GOP structure, Color-space, Color depth, Frame size, Frame rate, Aspect ratio, Quantization levels

– Correct Baseband and Quality levels - analog parameters for Signal levels, Luma, Chroma, Gamut and Quality levels of Black frames, Blockiness, Loss of audio, Audio clipping, Video/Audio playtime


Video Quality Control Video QC is a delivery requirement by many

broadcasters. QC Operators would be looking at:

– Black and white level– RGB Gamut and possibly composite gamut– Number of audio channels, level and loudness

peaks, phase– Picture action and graphic safe areas in all

deliverable aspect ratios– Creative quality and attached metadata such as

program duration, commercial breaks and clock details

– Video blanking– Timecode presence and continuity– General picture quality, standards conversion and

compression artefacts, tape drop out

Some of the above can be automated


Why Automate Quality Control?

We can automate our classic QC tools such as WFM and WVR– What do our customers do today?

– They use logging– They use SNMP traps

Broadcasters are moving to a Service Orientated Workflow– Using more file based technology

– File base acquisition– Panasonic P2 Cards– Sony XDCam SxS cards

– File based editing– Apple Final Cut Pro– Avid Media Composer

– File based archive– Isilon– Omneon MediaGrid

– File based Playout– Omneon Spectrum– Grass Valley K2


Why Automate Quality Control

Save time– Non-linear workflow

– QC can happen at the same time as proxy creation/transcoding

Save resources– Skilled Quality Control operators are few and expensive– Automated QC is a tool for the department

Give Consistency– Repeatable results

QC Operators looking at a broadcast monitor cannot easily “look inside” the file

– Are the correct parameters of encoding used for compression for video, audio?

– Are the bit-rates of video, audio and overall correct?– Syntax errors in the encoding may prevent it playing correctly further on

in the workflow– Is the metadata correct?


Content VerificationService Level Agreement (SLA) Audit Points

Cable

Cerify File based Audit point

PostEncode/Reformat

Ingest Edit /Transcode

PlayoutLibrary

Content On Demand On Line Service Provider

Ingest Edit /Transcode Playout

Broadcaster

Archive


What does Cerify test?

Systems level tests– Correct Standard and integrity– File size, Bitrate, Playtime, Number of video/audio streams, CableLabs VOD

compliance, – Video and audio PIDs, Signalling of Closed captions– Playtime of video vs playtime of audio

Video Tests– Correct encoding Standard, Profile, Level and Syntax checks for encoding errors – GOP structure, Frame rate, Bitrate, Frame size, Interlaced/ progressive, Aspect

ratio– Baseband tests including Gamut levels, Luma, Chroma, Signal levels– Letterbox/ Pillarbox, Playtime– Freeze frame– Color depth, Color format (4:2:0, 4:2:2), Copyright– Field order, Telecine pulldown– Quality - Black frames (lead in, lead out and during the video), blockiness,

Quantization


What does Cerify test? - cont

Audio Tests– Correct encoding Standard, Profile, and Syntax checks for encoder errors– Sample rate, Bitrate, Playtime– Number of Channels, Peak and Minimum Signal levels on each channel – Audio Silence (lead in, lead out, during video), clipping, mute, test tones

Reporting– Web based on-screen job reports and detailed drill down to error details– Copy or move file on success or error – Email alerts with test reports – CeriTalk API for fully automated working and integration with Automation /

Asset Management systems


Benefits of Automated QC

Quality Control Strategy– Saves time and money over human playout testing– Works 24/7– Enables all of your file-based content to be checked all of the time

Provides a Content Filter to catch errors that people can easily miss, and perform repeatable objective testing

– Performs automatic, consistent and thorough checks of content files against user-defined templates

– Allows QC staff to concentrate on 5% of rejected files not watching the 95% of good files

Ingest / Transcode / Archiving– Check encoded content at ingest to resolve problems early – prevent problems from

entering your workflow– Monitor your Service Level Agreements and the quality of your supplier’s content– Check before and after transcode, edit and archiving

Playout– Check your files are ready to playout– Improves broadcast QoS by minimizing risk of transmission and reception defects


Cerify Workflows and Applications


Identifying the workflow

There are 3 main types of workflow available with Cerify– Manual job creation and results viewing

– Semi-automatic using Watchfolder/dropbox– Manual viewing of results in web GUI– Automatic email reports generated and sent– Automatic XML reports generated and saved on network– Custom Action Scripts

– CeriTalk API– Uses SOAP– Large list of members of the Cerify Developer Community – new Partners

added monthly– Best way to control Cerify– May require development effort from Automation/Asset Management Partner

Workflow chosen depends on application/size of facility


Cerify Workflow – Network Setup

BROWSE STATIONS FILE SERVER

CERIFY CYC200

Gigabit Network

Gigabit Network


Manual Workflow – monitoring results

Files are added to MediaSets manually

Jobs are created manually

Results are reviewed by an operator


“See and Solve” Test Results Details of Cerify results are

displayed on the user GUI in levels

The amount of detail increases as you “drill down” to lower levels

Details of a specific error alert

Video thumbnails and audio waveform shown in frames surrounding the alert

Details for each file within the “Job”

Alerts for each error within the file

Test summary for each file in “Job”

Alert traffic lights

Thumbnail of erroneous frame


Semi-Automatic Workflow – Cerify Email Reports

Files are added to MediaSets automatically using “watch folders” or “drop boxes”

Jobs are created manually using these mediasets

Results are sent by email and reviewed by an operator

The operator can “click through” from the email to the Cerify web GUI for thumbnail viewing


Semi-Automatic Workflow – Cerify XML Reports

Files are added to MediaSets automatically using “watch folders” or “drop boxes”

Jobs are created manually using these mediasets

Results are stored on a server in XML

They can be reviewed by an operator or picked up by an asset management system

The operator can “click through” from the XML to the Cerify web GUI for thumbnail viewing or an asset management system can use these URLs to gather the thumbnails


Automated Workflow – using the CeriTalk API

Tektronix has a Partner program to aid Cerify Integration– This is the Cerify Developer Community or CDC

3rd Party Applications (typically Asset Management) control Cerify– File is ingested– Job is created on Cerify by automation system– Results are fed back to automation system


Cerify Developer Community - CDC

Note: The community is growing rapidly. Please check with Tektronix for latest list or visit http://www.tek.com/products/video_test/cerify/cdc.html

Working together for integrated solutions with Cerify automated QC


Case Study - VoD

Overview– The customer encodes a master file in HD– SD and HD low bitrate versions are made from the master file– The master file is archived for later re-use

Workflow– High resolution/high bitrate master file ingested manually– Encoded file is moved to a high bitrate watch folder– Transcoder picks up the master file and creates 2 low bitrate versions– Auto Quality Check tests the files in these watch folders for encoding

syntax, parameters and various baseband values and sends emails on error to alert QC Operator

– Dolby DP600 used for Dolby AC-3 encoding, correction of audio metadata and levels

Cost benefits of auto QC– As volume of media goes up costs can remain low– The complete file is tested not just 3 or 5 point spot check


Workflow Example - VoD


Case Study – Broadcast Playout Centre

Overview– Tape and file ingest controlled by Media Asset Management and automation

system– Important decisions on the format taken e.g. AVC-Intra, IMX, MXF, XDCam– Can be complex and highly automated– Uses Cerify’s web services SOAP API– Low number of transcode/un-wrap/re-wrap steps

Workflow– Asset Management requests media to be ingested– After ingest job is created on Cerify for testing– Cerify is polled for the results/progress– Results are provided on completion as XML over SOAP– Results are displayed in GUI

Cost benefits of auto QC– Clear penalties for playout centre failing to schedule a show or commercial –

Service Level Agreements– Quality of the channel brand upheld


Workflow Example - Broadcast Playout Centre


How fast is Cerify?


Cerify Benchmarks

Cerify Lite is difficult to bench mark because the hardware is customer supplied – it will process one file at a time

– Typically on a 3GHz processor with 2Gb RAM– 50Mb/s SD MPEG2 is realtime– 12Mb/s SD MPEG2 is 0.5 x realtime– 8MB/s SD H.264 is 2x realtime

CYC200 & CYM200 will process 4 files at the same time– 50Mb/s SD MPEG2 is realtime– 12Mb/s SD MPEG2 is 0.5 x realtime– 8MB/s SD H.264 is 2x realtime


Cerify Processing

To define the speed– Firstly the codec

– MPEG2 is faster than H.264– The syntax testing takes the most time

– Secondly the bitrate and complexity– I frame only is simpler than long GOP– Low bitrate is quicker than high

– Thirdly the resolution– HD takes longer to process than SD

– Fourthly – the baseband tests– Colour Gamut and luminance tests take more processing power than black

frame detection

The most important test is the syntax– This takes most of the processing

Limit the number of tests switched on– Limit the baseband tests to those most appropriate


Questions?

Cerify – File Based Quality Control Yannick LE DREAU.

Documents

Transcript of Cerify – File Based Quality Control Yannick LE DREAU.