Image and Video Compression - Universitas...
Transcript of Image and Video Compression - Universitas...
Image and Video CompressionEE398
Bernd GirodInformation Systems Laboratory
Department of Electrical EngineeringStanford University
Winter 2005/06
Bernd Girod: EE398 Image and Video Compression Introduction no. 2
Image and Video Compression Everywhere …
Fax machinesDigital still camerasDigital camcordersDigital television broadcastingDigital video/versatile disk (DVD)Personal video recorder (PVR, aka TiVo)World Wide WebInternet video streamingVideo conferencing. . .
Bernd Girod: EE398 Image and Video Compression Introduction no. 3
Motivating Image Compression
Binary image (facsimile Group 3)8.5 x 11 in document scanned at 7.7 lines/mm (“fine mode”), 1664pixels/line, with 1 bit/pixel3.255 Mbits for 1 page = 5.65 minutes over 9600 baud connection
Photos on 35 mm filmScanned at 12μ resolution (3656x2664 pixels) with 8 bits per color and 3 colors233 Mbits for 1 photo, 2/3 of 48 Mbyte compact flash card
Bernd Girod: EE398 Image and Video Compression Introduction no. 4
Digital video studio standard ITU-R Rec. 601
Some interesting bit-ratesTerrestial TV broadcasting channel ~20 MbpsDVD (max. 17 GB/length of movie) 10...20 MbpsEthernet/Fast Ethernet <10/100 MbpsDSL downlink 384...2048 kbpsWireless cellular data 9.6...384 kbps
Y Cb CrSampling rate 13.5 MHz 6.75 MHz 6.75 MHzQuantization 8 bit 8 bit 8 bitRaw bit rate 216 MbpsW/o blanking intervals 166 Mbps
Motivating Video Compression
Bernd Girod: EE398 Image and Video Compression Introduction no. 5
Personal Video Recorder
Hard disk drive
MPEG-2 QualitiesBest 7.7 MbpsHigh 5.4 MbpsMedium 3.6 MbpsBasic 2.2 Mbps
Bernd Girod: EE398 Image and Video Compression Introduction no. 6
Introductory Lecture
Goal: provide a first introduction of some key concepts of image compression, without rigorous treatmentLossless vs lossy compressionMeasuring distortion and compressionStatistical and visual redundancyNeed for structured compression schemes
EE398 Organisation
Bernd Girod: EE398 Image and Video Compression Introduction no. 7
Digital Image: Quantized Array of Samples
1 1 2 20 , 0 n N n N≤ < ≤ <
2n2 1N −
1 1N −
00 1
1
2
2
1n[ ]1 2,x n n
[ ] [ ]1 2 1 2, 2 ,Bx n n x n n′=
Real-valued intensities, range
0…1 or –1/2…+1/2
Rounding to nearest integer
Bernd Girod: EE398 Image and Video Compression Introduction no. 8
Multiple Image Components
Color images typically represented by three values per sample location, for red, green and blue primary components
General multi-component image
Examples:Color printing: cyan, magenta, yellow, black dyes, sometimes moreHyperspectral satellite imaging: 100s of channels
[ ] [ ] [ ]1 2 1 2 1 2, , , , , R G Bx n n x n n x n n
[ ]1 2, , 1, 2, , Cx n n c C= …
Bernd Girod: EE398 Image and Video Compression Introduction no. 9
Lossless Compression
Minimize number of bits required to represent original digital image samples w/o any loss of information.All B bits of each sample must be reconstructed perfectly.Achievable compression usually rather limited.Applications
Binary images (facsimile)Medical imagesMaster copy before editingPalettized color images
Bernd Girod: EE398 Image and Video Compression Introduction no. 10
Lossy Compression
Some deviation of decompressed image from original(“distortion”) is often acceptable:
Human visual system might not perceive loss, or tolerate it.Digital input to compression algorithm is imperfect representation of real-world scene
Much higher compression than with lossless.Lossy compression used widely for natural images (e.g. JPEG) and motion video (e.g. MPEG).
Bernd Girod: EE398 Image and Video Compression Introduction no. 11
Lossy Compression: Measuring Distortion
Most commonly employed: Mean Squared Error
. . . or, equivalently, Peak Signal to Noise Ratio
AdvantagesEasy calculationMathematical tractability in optimization problems
DisadvantageNeglects properties of human vision
[ ] [ ]( )1 2
1 2
1 12
1 2 1 20 01 2
1 ˆMSE= , ,N N
n nx n n x n n
N N
− −
= =
−∑ ∑
( )2
10
2 1PSNR=10log dB
MSE
B −
Bernd Girod: EE398 Image and Video Compression Introduction no. 12
Measures of Compression
Image represented by “bit-stream” c of length ||c||.Compare no. of bits w/ and w/o compression
Alternatively
For lossy compression, bit-rate more meaningful than compression ratio, as B is somewhat arbitrary.
1 2compression ratio = N N Bc
1 2
bit-rate = bits/pixelN N
c
Bernd Girod: EE398 Image and Video Compression Introduction no. 13
Typical Bit-rates after Compression
Dependent on image content: consider typical natural imagesLossless compression: (B-3) bpp (bits per pixel)Lossy compression,
Perceived distortion depends on sampling density and contrastAssume viewing on computer monitor, 90 pixels/inch.high quality: 1 bppmoderate quality: 0.5 bppusable quality: 0.25 bpp
Bernd Girod: EE398 Image and Video Compression Introduction no. 14
How Does Compression Work?
Exploit statistical redundancy.Take advantage of patterns in the signal.Describe frequently occuring events efficiently.Lossless coding: only statistical redundancy
Introduce acceptable deviations.Omit “irrelevant” detail that humans cannot perceive.Match the signal resolution (in space, time, amplitude) to the applicationLossy coding: exploit statistical and visual redundancy
Bernd Girod: EE398 Image and Video Compression Introduction no. 15
Statistical Redundancy
Trivial example: Given two B-bit integers (e.g., representing two adjacent pixels)
Assume that only takes on values {0,1}Compression to 1 bpp
Further assume, that Compression to 0.5 bpp
Hope: bit-rate increases only slightly, as long as the above assumptions hold with high probability
{ }1 2, 0,1, , 2 1Bx x ∈ −…
1 2,x x
1 2x x=
Bernd Girod: EE398 Image and Video Compression Introduction no. 16
Joint Histogram of two Horizontally Adjacent Pixels
( ‘Lena’, 512 x 512 pixels, 8 bpp)
100 200 300 400 500
100
200
300
400
500
Amplitude of current pixel
Amplitude of adjacent pixel
PMF
Bernd Girod: EE398 Image and Video Compression Introduction no. 17
Visual Redundancy
For images to be viewed by humans, no need to represent more than the visible resolution in
spacetimebrightnesscolor
Required resolution might depend on image content (“masking”)For some applications, only a specific region of the image might be relevant, e.g., in
medical imagingmilitary imaging
Bernd Girod: EE398 Image and Video Compression Introduction no. 18
Exploiting Limitations of Color Vision
Human visual system has much lower acuity for color hue and saturation than for brightnessUse color transform to facilitate exploiting that property
Note
Cb and Cr often sub-sampled 2:1 relative to Y.
0.299 0.587 0.1140.169 0.331 0.50.5 0.419 0.0813
Y R
Cb G
Cr B
x xx xx x
⎛ ⎞ ⎛ ⎞ ⎛ ⎞⎜ ⎟ ⎜ ⎟ ⎜ ⎟= − − ⋅⎜ ⎟ ⎜ ⎟ ⎜ ⎟⎜ ⎟ ⎜ ⎟ ⎜ ⎟− −⎝ ⎠ ⎝ ⎠ ⎝ ⎠
RGB components (γ-predistorted)
Chrominancecomponents
( ) ( )0.564 0.713Cb B Y Cr R Yx x x x x x= − = −
Luminancecomponent
Bernd Girod: EE398 Image and Video Compression Introduction no. 19
Compression as a Global Mapping
Lossless compression
Lossy compression
( )compressor
M=c x ( )1
decompressor
ˆ M −=x c
image x bit-stream c
reconstructed ˆimage x
1 2
lookup table2 entriesN N B
lookup table
2 entriescfixed length cLookup table
interpretation
1 1M M− −=
( ) ( )( )1arg min ,M D M −
′′= =
cc x x c
Bernd Girod: EE398 Image and Video Compression Introduction no. 20
Typical Structured Compression System
Transform T(x) usually invertibleQuantization not invertible, introduces distortionCombination of encoder and decoder lossless
( )transform
T=y x ( )quantizer
Q=q y ( )encoder
C=c qcoefficients yimage x indices q
( )1
inversetransform
ˆ ˆT −=x y ( )1
dequantizer
ˆ Q−=y q ( )1
decoderC−=q c
indices qˆcoefficients yreconstructed
ˆimage x
bit-stream c
( )Q y( )C q ( )1C− c
Bernd Girod: EE398 Image and Video Compression Introduction no. 21
Outline EE398
Entropy and lossless coding techniquesRun-length coding, fax standardsArithmetic codingRate-distortion limits and quantizationLossless and lossy predictive codingTransform coding, JPEG standard Subband coding, wavelets, JPEG-2000Motion compensated coding, MPEG standards
Bernd Girod: EE398 Image and Video Compression Introduction no. 22
EE398 Organisation
Regularly check class home page: http://www.stanford.edu/class/ee398
Co-instructor: Dr. Markus FlierlAssistants
General TA: David VarodayanISE lab TA: Shantanu RaneCourse assistant: Kelly Yilmaz
Bernd Girod: EE398 Image and Video Compression Introduction no. 23
EE398 Organisation (cont.)
Homeworks7 problem sets, require computer + Matlab (Image Proc. Toolbox)Handed out Tuesdays, due one week later, 2:45 p.m.
Grading Homeworks: 50%Term project: 50%No mid-term, no final
Bernd Girod: EE398 Image and Video Compression Introduction no. 24
EE398 Term Projects - General
Work in groups of 2-3 students, 40-50 hours per personProject proposal required, deadline: February 9Class-room presentations of projects: March 14/16Project report due: March 16Project grade based on
Technical quality, significance, and originality of results 50%Project report 25%Class-room presentation 25%
Bernd Girod: EE398 Image and Video Compression Introduction no. 25
ISE laboratory
Created by equipment grants from Hewlett-Packard, Xerox, and IntelExclusively a teaching laboratoryLocation: Packard room 02120 Linux PCs, 2 Windows PCs, scanners, printers etc.Access:
door combination for lab entry will be provided by TAAccount on ise machine will be provided to all enrolled in class
Bernd Girod: EE398 Image and Video Compression Introduction no. 26
Further reading
Slides available as hand-outs and as pdf files on the webRecommended text books
D. S. Taubman, M. W. Marcellin, „JPEG2000 – Image Compression Fundamentals, Standards, and Practice,“ Kluwer Academic Publishers, 2002.Y. Wang, J. Ostermann, Y.-Q. Zhang, „Video Processing and Communications,“ Prentice-Hall, 2002.
Selected papers will be posted on Web site