CS148: Introduction to Computer Graphics and Imaging...

CS148: Introduction to Computer Graphics and Imaging

Compression

CS148 Lecture 16 Pat Hanrahan, Winter 2009

Key Concepts

Lossless vs. lossy compressionKolmolgorev complexityPredictive coding / Huffman codingJPEG / Discrete cosine transform (DCT)JPEG2000 / Wavelets

Image and Video Data Rates

Image 640x480x24b = ~3/4 MB

Full screen Image 1024x768x24b = ~2.5MB

DVD 720x480x24bx30f/s = ~30 MB/s

High Definition DVD 1920x1080x24bx30f/s = ~178MB/s

Film 4000x3000x36bx30f/s = ~1.5GB/s 8 TB for one 90 minute movie!

Lossless vs. Lossy Compression

Lossless All information stored Exact original can be reconstructed

Lossy Some information discarded Goal: discard information humans won’t

notice Much higher compression ratios possible

Kolmogorov Complexity

What is the shortest program that can generate the data?

17 KB JPEG

Re(c), Im(c), xmin, xmax, ymin, ymax (24B)

z0 = x + iy

zn+1 = z2n + c

Run Length Encoding

BWBBBBBBBBBBBBWWWWWWBBBW

BW\{12}B\{6}W\{3}BW

Code Book

Alphabet: A, B, C, DFrequencies: ¼, ¼, ¼, ¼ Code: 00, 01, 10, 11

ABCDBDAC = 8 characters00 01 10 11 01 11 00 11 = 16 bits

¼ * 2 + ¼ * 2 + ¼ * 2 + ¼ * 2 = 2 bits/ch on average

Code Book

Alphabet: A, B, C, DFrequencies: ½, ¼, 1/8, 1/8Code: 0, 10, 110, 111

ABACADAB = 8 characters0 10 0 110 0 111 0 10 = 14 bits

½ * 1 + ¼ * 2 +1/8 * 3 +1/8 * 3 = 1 ¾ bits/ch on average

Entropy

Huffman CodingA

B (.15)

C(.30)

D(.16)

E(.29)

Huffman Coding

A (.10)

B (.15)

C(.30)

D(.16)

E(.29)

AB (.25)

Huffman Coding

A (.10)

B (.15)

C(.30)

D(.16)

E(.29)

AB (.25)

ABD(.41)

Huffman Coding

A (.10)

B (.15)

C(.30)

D(.16)

E(.29)

AB (.25)

ABD(.41)

0 1 0 1

CE(.59)

Huffman Coding

A (.10)

B (.15)

C(.30)

D(.16)

E(.29)

AB (.25)

ABD(.41)

0 1 0 1

CE(.59)

ABCDE(1.0)

Lossy Compression

Chroma subsamplingTransform Coding

Fourier / DCT (JPEG) Wavelets (JPEG2000)

Chroma Subsampling

Transform Coding

X = 2 2 2 2 3 4 5 6 6 6 6 H(X) = 2.0049

D(X) = 2 0 0 0 1 1 1 1 0 0 0 H(D(X)) = 1.3222

Difference Operator

Basis vectors b0, b1, … , bn

Express any vector as

a0 * b0 + a1 * b1 + … + an * bn

where the coefficients ai are scalars.

Pixel Basis

Another Basis

Two Observations on Images

Characterize images by the frequencies that are present

The frequencies in natural images fall off as 1/f That is, high frequencies are less common

The human visual system is less sensitive to high frequencies

That is, it is more important to preserve low frequencies than high frequencies

Discrete Cosine Transform (DCT)

262,144 pixels 43384 largest terms, 16%(dropped 218,760 terms)

262,144 pixels 8353 largest terms, 3.2%(dropped 253,791 terms)

Quantization

DCT Image Quantization Matrix

Quantized DCT Image

Storage Order

error = abs(original – compressed) * 8

Peak Signal-to-Noise Ratio

� Original Image (I) and new image (I’)

MSE =1N

!I(i)" I !(i)!2

RMSE =#

PSNR = 10 log10

"I2max

= 20 log10

!p" q!2 =

(avg(pi)" avg(qi))2

(max(pi)"max(qi))2

(lum(p)" lum(q))2

· · ·

Error Metrics

Wavelets

Haar Wavelets

Scaling function = average

Wavelet = difference

Transform:S = (A + B)/2D = (A – B)/2

Invert:A = S + DB = S – D

Haar Wavelets

6 8 5 9 5 5 6 6

7 7 5 6

Haar Wavelets

6 8 5 9 5 5 6 6

7 7 5 6 -1 -2 0 0

Haar Wavelets

6 8 5 9 5 5 6 6

7 7 5 6 -1 -2 0 0

1 Transform Step

Averages• Smoothed version of signal• Lower resolution image

Differences• Local, high frequencies• Details missing from low resolution part

Haar Wavelets

6 8 5 9 5 5 6 6

7 7 5 6 -1 -2 0 0

-1 -2 0 0

Haar Wavelets

6 8 5 9 5 5 6 6

7 7 5 6 -1 -2 0 0

7 -1 -2 0 05.5

Haar Wavelets

6 8 5 9 5 5 6 6

7 7 5 6 -1 -2 0 0

7 0 -.5 -1 -2 0 05.5

Haar Wavelets

6 8 5 9 5 5 6 6

7 7 5 6 -1 -2 0 0

7 0 -1 -2 0 0

0 -1 -2 0 0

Haar Wavelets

6 8 5 9 5 5 6 6

7 7 5 6 -1 -2 0 0

7 5.5 0 -1 -2 0 0

6.25 0 -1 -2 0 0

Haar Wavelets

6 8 5 9 5 5 6 6

7 7 5 6 -1 -2 0 0

7 0 -1 -2 0 0

6.25.75 0 -1 -2 0 0

Haar Wavelets

6 8 5 9 5 5 6 6

6.25 0 -1 -2 0 0

Full Transform

High Res DetailsMedium Res DetailsLow Res DetailAverage Value

-.5.75

2D Wavelet Transform

Standard– Apply full transform horizontally, then full

transform vertically– Creates long, thin basis functions – bad for

image compressionNon-standard

– Repeatedly apply 1 step horizontally, then 1 step vertically

– Creates square basis functions – good for image compression

What else needs compression?

VideoTextures (different requirements than plain

images)Geometry (including animations)Anything else?

Things to Remember

LosslessNo error, but not lots of compression

LossyError can be driven by human perceptionMore error, but lots of compressionDifferent error metrics (max, avg, perceptual)

Different Basis FunctionsJPEG / Discrete cosine transform (DCT)JPEG2000 / Wavelets

CS148: Introduction to Computer Graphics and Imaging...

Documents

Transcript of CS148: Introduction to Computer Graphics and Imaging...

introduction - Stanford Universitygraphics.stanford.edu/courses/cs148-10-fall/lectures/introduction.pdf · Yi-Ting Yeh Imran Haque others TBD CS148 Han S148 I trod ction to emp Graphics

Conveying Shape: Lighting and Texturing - …graphics.stanford.edu/courses/cs448b-04-winter/lectures/shading/... · Conveying Shape: Lighting and Texturing Pat Hanrahan ... Categories

Curves and Surfaces - graphics.stanford.edu

CS148 - Building Intelligent Robots Lecture 3: Sensors ...cs.brown.edu/courses/cs148/old/2004fall/lecture_slides/cs148_3.pdf · Jenkins – CS148 Building Intelligent Robots Lecture

CS148: Introduction to Computer Graphics and Imaging Input ...

Introduction to Computer Graphics What is Computer ...graphics.stanford.edu/courses/cs148-10-summer/docs/01_intro.pdf · Computer Graphics, 3rd ed. (in a pinch 2nd is ok) Optional:

CMSC 754 - graphics.stanford.edu

Conveying Shape Pat Hanrahan

CS148: Introduction to Computer Graphics and Imaging Pat ...graphics.stanford.edu/courses/cs148-08/lectures/introduction/... · Digital photography Digital video and HDTV New media

CS148: Introduction to Computer Graphics and Imaging Image …graphics.stanford.edu/courses/cs148-09-fall/lectures/... · 2009. 11. 5. · Page 1 CS148: Introduction to Computer Graphics

CS148 Final Project Write Up - Stanford Universityweb.stanford.edu/class/cs148/students/2018/raytracing/10... · 2019. 9. 24. · CS148 Final Project Write Up Fang-Yu Lin SUID: fangyuln

CS148 - Building Intelligent Robots Lecture 4: Control ...cs.brown.edu/courses/cs148/old/2004fall/lecture_slides/cs148_4.pdf · Jenkins – CS148 Building Intelligent Robots Lecture

CS148 Final Image - web.stanford.edu

Radiometry and Photometry - Computer Graphics at …graphics.stanford.edu/courses/cs348b-12/lectures/...Page 2 Radiometry and Photometry CS348B Lecture 4 Pat Hanrahan, Spring 2012

CS148: Introduction to Computer Graphics Final Review Session.

CS 148, Summer 2012 Introduction to Computer …graphics.stanford.edu/courses/cs148-12-summer/assets/...Frustum 5. Viewport Act I: Real-time graphics Act I: Real-time graphics ¢ z

CS148 - Building Intelligent Robots Lecture 2: Robotics ...cs.brown.edu/courses/cs148/old/2004fall/lecture_slides/cs148_2.pdf · Jenkins – CS148 Building Intelligent Robots Lecture

CS148: Introduction to Computer Graphics and Imaging ...graphics.stanford.edu/courses/cs148-09-fall/lectures/transform.pdf · CS148: Introduction to Computer Graphics and Imaging

Lecture11 CS148/248: Interactive Narrative

Lecture 11 CS148/248: Interactive Narrative