En-hui YangMultimedia Communications

LaboratoryDept. of Electrical & Computer

Engineering

About the Speaker Professor and Canada Research ChairsFellow of the Royal Society of CanadaFellow of the Canadian Academy of

EngineeringCo-founder of SlipStream Data Inc. (now a

subsidiary of Research In Motion)Areas of expertise: Multimedia compression;

information theory; digital communications; image and video coding

Convergence of Mobile Communications and Computing (3COM)

Ten years ago 3C 3CInfo 3CInfoTech SlipStream 3C (Convergence of communication and computing) did not

happen.Today we have witnessed not only 3C, but also 3COM

Global mobile data traffic: 237 petabytes per month in 2010, over three times the entire global Internet traffic in 2000 (Cisco VNI Mobile 2011)

Smart devices Smart phones: 297 million units sold in 2010 (Gartner);

represent about 13 percent of total global handsets in use Tablets: 20 million units sold in 2010 (iHS iSuppli)

Contents and carriers: CTV acquisition by BCE Canwest Global acquisition by Shaw NBC acquisition by Comcast

3COM: Opportunities

The convergence is happening at a speed much faster than most of us

have anticipated. According to the forecast from Cisco VNI Mobile 2011, By 2015, there will be over 7.1 billion mobile-connected

devices (including machine-to-machine modules), roughly one mobile device per capita.

The global mobile data traffic will grow at a compound annual growth rate (CAGR) of 92% from 2010 to 2015, reaching about 6.3 exabytes per month by 2015.

The amount of mobile data traffic generated by tablets will reach 248 petabytes per month in 2015, more than the entire global mobile data traffic in 2010.

By 2015, more than 60% of the global mobile data traffic will be video.

Mobile Video: Traffic Forecast

Mobile Video: Typical ApplicationsLocal video playback (including

downloading, streaming, etc.)Video chatVideo telephonyVideo conferencing Video broadcast/multicastVideo uploading

Behind all these applications is video encoding/decoding

Mobile Video: Bottlenecks

With so much data processed and moved around, two related issues stand out:

BandwidthPower: radio power needed for

communicating, and computing power needed for processing

For real-time communications such as video chat, telephony, and conferencing, the interplay among bandwidth, radio power, and computing power becomes very intriguing.

Mobile Video: Shannon Capacity Formula

/

0

2 1

/

C WbE

N C W

Existing Video Coding Standards: Evolution Path

ITU-T

ISO/IEC

H.120

(1984-1988)

H.261

(1990+)

MPEG-1

(1993)

H.262/MPEG-2

(1994/95

-1998+)

H.263

(1995-2000+)

MPEG-4

Visual

(1998-2001+)

H.264/MPEG-4

AVC

(2003-2006)

1990 1992 1994 1996 1998 2000 2002 2004 2006 2010

H.265/HEVC

2010--

Existing Video Coding Standards: Predictive Video Coding (PVC) Paradigm

Original Frames:

Reconstructed Frames:

I P P P

• All video coding standards proposed so far fall into PVC

Existing Video Coding Standards: Frame Level Structure

Prediction/motion compensation

Existing Video Coding Standards: Good Solutions for Mobile Video?

Each standard improves the compression performance of its predecessors at the cost of significant increase in encoding and decoding complexity.

Compression gains of these standards over their predecessors come, to a large extent, from the improvements in their signal processing components---prediction/motion compensation.

Further gains from this direction, if any, are diminishing for given frame sizes and would come at extreme high computation complexity.

In the pursuit of improving compression performance, computing power for real-time mobile applications is almost forgotten in the development of these standards.

Answer: No

IT Video Coding: Green Solutions for Mobile Video For mobile video, we need green solutions which can

drive up the compression performance while reducing the computing power needed for encoding and decoding.

Such solutions could be offered by what I call information theoretic (IT) video coding.

Instead of driven by signal processing components, IT video coding includes new ways of quantization and coding, tight integration of algorithm innovation and software/hardware implementation, and possibly paradigm shifts such as from PVC to causal video coding.

Within IT video coding, IT methods and signal processing methods (such as prediction/motion compensation) are complementary to each other.

IT Video Coding: Future Standards

IT Video Coding: Early Successes Soft decision quantization (SDQ) and optimal

algorithms for SDQ developed by Yang and his former PhD students/postdocs in recent years and now widely used.

Iterative video coding developed by Yang and his former PhD student.

Causal video coding paradigm: More and Less Coding Theorem: Under some conditions on source frames and distortion, the more frames need to be encoded and transmitted, the less amount of data after encoding has to be actually sent!

Thank you