Image Processing and Computer Vision: 91. Image and Video Coding Compressing data to a smaller...
-
Upload
derrick-wilkerson -
Category
Documents
-
view
215 -
download
0
Transcript of Image Processing and Computer Vision: 91. Image and Video Coding Compressing data to a smaller...
![Page 1: Image Processing and Computer Vision: 91. Image and Video Coding Compressing data to a smaller volume without losing (too much) information.](https://reader036.fdocuments.in/reader036/viewer/2022062422/56649ee65503460f94bf62dd/html5/thumbnails/1.jpg)
Image Processing and Computer Vision: 9 1
![Page 2: Image Processing and Computer Vision: 91. Image and Video Coding Compressing data to a smaller volume without losing (too much) information.](https://reader036.fdocuments.in/reader036/viewer/2022062422/56649ee65503460f94bf62dd/html5/thumbnails/2.jpg)
Image and Video Coding
Compressing data to a smaller volume without losing (too much) information
![Page 3: Image Processing and Computer Vision: 91. Image and Video Coding Compressing data to a smaller volume without losing (too much) information.](https://reader036.fdocuments.in/reader036/viewer/2022062422/56649ee65503460f94bf62dd/html5/thumbnails/3.jpg)
Image Processing and Computer Vision: 9 3
Why Code Data? To reduce storage volume To reduce transmission time
One colour image 760 by 580 pixels 3 channels, each 8 bits 1.3 Mbyte
Video data Same resolution 25 frames per second 33 Mbyte/second
![Page 4: Image Processing and Computer Vision: 91. Image and Video Coding Compressing data to a smaller volume without losing (too much) information.](https://reader036.fdocuments.in/reader036/viewer/2022062422/56649ee65503460f94bf62dd/html5/thumbnails/4.jpg)
Image Processing and Computer Vision: 9 4
Redundancy Spatial
Correlation between adjacent pixels Chromatic
Correlation between colour channels Temporal
Correlation between adjacent frames Perceptual
Unnoticed losses
![Page 5: Image Processing and Computer Vision: 91. Image and Video Coding Compressing data to a smaller volume without losing (too much) information.](https://reader036.fdocuments.in/reader036/viewer/2022062422/56649ee65503460f94bf62dd/html5/thumbnails/5.jpg)
Image Processing and Computer Vision: 9 5
Example
![Page 6: Image Processing and Computer Vision: 91. Image and Video Coding Compressing data to a smaller volume without losing (too much) information.](https://reader036.fdocuments.in/reader036/viewer/2022062422/56649ee65503460f94bf62dd/html5/thumbnails/6.jpg)
Image Processing and Computer Vision: 9 6
Redundancy - Consequences
Data exceeds information More data than content justifies
Can lose data without losing information
![Page 7: Image Processing and Computer Vision: 91. Image and Video Coding Compressing data to a smaller volume without losing (too much) information.](https://reader036.fdocuments.in/reader036/viewer/2022062422/56649ee65503460f94bf62dd/html5/thumbnails/7.jpg)
Image Processing and Computer Vision: 9 7
Compression Ratio
image compressed of volume
image eduncompress of volumeC
![Page 8: Image Processing and Computer Vision: 91. Image and Video Coding Compressing data to a smaller volume without losing (too much) information.](https://reader036.fdocuments.in/reader036/viewer/2022062422/56649ee65503460f94bf62dd/html5/thumbnails/8.jpg)
Image Processing and Computer Vision: 9 8
Lossy vs Lossless Compression Can lose data without losing
information Lossily compressed images can
look similar to the original Lossy compression has greater C
![Page 9: Image Processing and Computer Vision: 91. Image and Video Coding Compressing data to a smaller volume without losing (too much) information.](https://reader036.fdocuments.in/reader036/viewer/2022062422/56649ee65503460f94bf62dd/html5/thumbnails/9.jpg)
Image Processing and Computer Vision: 9 9
Quality of Decoded Images Measure differences between
Original Coded/decoded images
Options Maximum difference Average difference Subjective scales
![Page 10: Image Processing and Computer Vision: 91. Image and Video Coding Compressing data to a smaller volume without losing (too much) information.](https://reader036.fdocuments.in/reader036/viewer/2022062422/56649ee65503460f94bf62dd/html5/thumbnails/10.jpg)
Image Processing and Computer Vision: 9 10
Example
![Page 11: Image Processing and Computer Vision: 91. Image and Video Coding Compressing data to a smaller volume without losing (too much) information.](https://reader036.fdocuments.in/reader036/viewer/2022062422/56649ee65503460f94bf62dd/html5/thumbnails/11.jpg)
Image Processing and Computer Vision: 9 11
Subjective Quality Measurement Scales - Example
0 Unusable1 Annoying degradation2 Adequate images3 Barely perceptible degradation4 No observable degradation
![Page 12: Image Processing and Computer Vision: 91. Image and Video Coding Compressing data to a smaller volume without losing (too much) information.](https://reader036.fdocuments.in/reader036/viewer/2022062422/56649ee65503460f94bf62dd/html5/thumbnails/12.jpg)
Image Processing and Computer Vision: 9 12
Difference coding Adjacent pixels are similar
Difference is small
Uncompressed:
Compressed:
,,,,,43210 sssss
,,,,,3 42 3210 10 sssssssss
![Page 13: Image Processing and Computer Vision: 91. Image and Video Coding Compressing data to a smaller volume without losing (too much) information.](https://reader036.fdocuments.in/reader036/viewer/2022062422/56649ee65503460f94bf62dd/html5/thumbnails/13.jpg)
Image Processing and Computer Vision: 9 13
Coding Assign 4 bits to a difference Can code –7, … , +8
Overflow? Use –7 and +8 to show larger differences Code –6, … , +7 directly Use overflow codes to indicate shift of
codes
![Page 14: Image Processing and Computer Vision: 91. Image and Video Coding Compressing data to a smaller volume without losing (too much) information.](https://reader036.fdocuments.in/reader036/viewer/2022062422/56649ee65503460f94bf62dd/html5/thumbnails/14.jpg)
Image Processing and Computer Vision: 9 14
![Page 15: Image Processing and Computer Vision: 91. Image and Video Coding Compressing data to a smaller volume without losing (too much) information.](https://reader036.fdocuments.in/reader036/viewer/2022062422/56649ee65503460f94bf62dd/html5/thumbnails/15.jpg)
Image Processing and Computer Vision: 9 15
Predictive Coding If signals are well sampled
Adjacent samples are correlated They have similar values Differences are small Can guess next sample from value of
current
![Page 16: Image Processing and Computer Vision: 91. Image and Video Coding Compressing data to a smaller volume without losing (too much) information.](https://reader036.fdocuments.in/reader036/viewer/2022062422/56649ee65503460f94bf62dd/html5/thumbnails/16.jpg)
Image Processing and Computer Vision: 9 16
Constants are correlation coefficient and mean grey value
Difference between real and predicted values are smaller
Code as for difference coding
II ttˆ 1
![Page 17: Image Processing and Computer Vision: 91. Image and Video Coding Compressing data to a smaller volume without losing (too much) information.](https://reader036.fdocuments.in/reader036/viewer/2022062422/56649ee65503460f94bf62dd/html5/thumbnails/17.jpg)
Image Processing and Computer Vision: 9 17
Run Length Coding Replace runs of equal brightness values
by (length of run, value)
1 2 2 3 3 4 4 4 5 6 5(1 1) (2 2) (2 3) (3 4) (1 5) (1 6) (1 5)
More use when few brightness values e.g. fax
![Page 18: Image Processing and Computer Vision: 91. Image and Video Coding Compressing data to a smaller volume without losing (too much) information.](https://reader036.fdocuments.in/reader036/viewer/2022062422/56649ee65503460f94bf62dd/html5/thumbnails/18.jpg)
Image Processing and Computer Vision: 9 18
Huffman Coding Uses variable length codes Most frequently occurring grey
value has shortest code Least frequently occurring values
have longest codes
![Page 19: Image Processing and Computer Vision: 91. Image and Video Coding Compressing data to a smaller volume without losing (too much) information.](https://reader036.fdocuments.in/reader036/viewer/2022062422/56649ee65503460f94bf62dd/html5/thumbnails/19.jpg)
Image Processing and Computer Vision: 9 19
ExampleSymbol Probability 21 43
ABCDEF
0.40.30.10.10.060.04
0.40.30.10.10.1
0.40.30.20.1
0.40.30.3
0.60.4
![Page 20: Image Processing and Computer Vision: 91. Image and Video Coding Compressing data to a smaller volume without losing (too much) information.](https://reader036.fdocuments.in/reader036/viewer/2022062422/56649ee65503460f94bf62dd/html5/thumbnails/20.jpg)
Image Processing and Computer Vision: 9 20
SymbolProbability 21 43
ABCDEF
0.4 10.3 000.1 0110.1 01000.06 010100.04 01011
0.4 10.3 000.1 0110.1 01000.1 0101
0.4 10.3 000.2 0100.1 011
0.4 10.3 000.3 01
0.6 00.4 1
![Page 21: Image Processing and Computer Vision: 91. Image and Video Coding Compressing data to a smaller volume without losing (too much) information.](https://reader036.fdocuments.in/reader036/viewer/2022062422/56649ee65503460f94bf62dd/html5/thumbnails/21.jpg)
Image Processing and Computer Vision: 9 21
GIF Applicable to images with 256
colours Replace sequences of bytes with
shorter codes Most common sequences use
shortest codes
![Page 22: Image Processing and Computer Vision: 91. Image and Video Coding Compressing data to a smaller volume without losing (too much) information.](https://reader036.fdocuments.in/reader036/viewer/2022062422/56649ee65503460f94bf62dd/html5/thumbnails/22.jpg)
Image Processing and Computer Vision: 9 22
Wavelet coders Wavelet transform organises
image content efficiently Can easily select data to be
discarded
![Page 23: Image Processing and Computer Vision: 91. Image and Video Coding Compressing data to a smaller volume without losing (too much) information.](https://reader036.fdocuments.in/reader036/viewer/2022062422/56649ee65503460f94bf62dd/html5/thumbnails/23.jpg)
Image Processing and Computer Vision: 9 23
Wavelet coders
![Page 24: Image Processing and Computer Vision: 91. Image and Video Coding Compressing data to a smaller volume without losing (too much) information.](https://reader036.fdocuments.in/reader036/viewer/2022062422/56649ee65503460f94bf62dd/html5/thumbnails/24.jpg)
Image Processing and Computer Vision: 9 24
JPEG standard A subcommittee of ISO Optimised for a range of image
subject matter Compression rates can be defined Quality inversely proportional to C
![Page 25: Image Processing and Computer Vision: 91. Image and Video Coding Compressing data to a smaller volume without losing (too much) information.](https://reader036.fdocuments.in/reader036/viewer/2022062422/56649ee65503460f94bf62dd/html5/thumbnails/25.jpg)
Image Processing and Computer Vision: 9 25
Block Transform Codingoriginal image
decompose
transform
quantise
entropy code
Sequence of 8 by 8 blocks - different planes treated separately (RGB, YUV etc.)
Transformed blocks reduce redundancy and concentrate signal energy into a few coefficients discrete cosine transformation (DCT)
Blocks with discarded information - goal is to smooth picture and discard information that will not be missed, e.g. high frequencies
![Page 26: Image Processing and Computer Vision: 91. Image and Video Coding Compressing data to a smaller volume without losing (too much) information.](https://reader036.fdocuments.in/reader036/viewer/2022062422/56649ee65503460f94bf62dd/html5/thumbnails/26.jpg)
Image Processing and Computer Vision: 9 26
Block Transform Encoding
DCT
Zig-zag
run length code entropy
code
quantise
010111000111…..
![Page 27: Image Processing and Computer Vision: 91. Image and Video Coding Compressing data to a smaller volume without losing (too much) information.](https://reader036.fdocuments.in/reader036/viewer/2022062422/56649ee65503460f94bf62dd/html5/thumbnails/27.jpg)
Image Processing and Computer Vision: 9 27
Block Encoding
139 144 149 153144 151 153 156150 155 160 163159 161 162 160
1260 -1 -12 -5 -23 -17 -6 -3 -11 -9 -2 2 -7 -2 0 1
DCT
Zig-zag
run length code
Huffman code
quantise
10011011100011….
79 0 -1 0-2 -1 0 0-1 -1 0 0 0 0 0 0
79 0 -2 -1 -1 -1 0 0 -1 0 0 0 0 0 0 0
0 791 -20 -10 -10 -12 -10 0
Original image
![Page 28: Image Processing and Computer Vision: 91. Image and Video Coding Compressing data to a smaller volume without losing (too much) information.](https://reader036.fdocuments.in/reader036/viewer/2022062422/56649ee65503460f94bf62dd/html5/thumbnails/28.jpg)
Image Processing and Computer Vision: 9 28
Block Transform Decoding
DCT
Zig-zag
run length code entropy
code
quantise
010111000111…..
![Page 29: Image Processing and Computer Vision: 91. Image and Video Coding Compressing data to a smaller volume without losing (too much) information.](https://reader036.fdocuments.in/reader036/viewer/2022062422/56649ee65503460f94bf62dd/html5/thumbnails/29.jpg)
Image Processing and Computer Vision: 9 29
Result of Coding and Decoding
139 144 149 153144 151 153 156150 155 160 163159 161 162 160
Original block
144 146 149 152148 150 152 154155 156 157 158160 161 161 162
Reconstructed block
-5 -2 0 1-4 1 1 2-5 -1 3 5-1 0 1 -2
errors
![Page 30: Image Processing and Computer Vision: 91. Image and Video Coding Compressing data to a smaller volume without losing (too much) information.](https://reader036.fdocuments.in/reader036/viewer/2022062422/56649ee65503460f94bf62dd/html5/thumbnails/30.jpg)
Image Processing and Computer Vision: 9 30
Discrete Cosine Transform
F[u,v] =4C(u)C(v)
n2
n-1
j=0
n-1
k=0
f(j,k) cos(2j+1) u
2ncos
(2k+1) v 2n
with
C(w) =
12
For w = 0
1 otherwise
![Page 31: Image Processing and Computer Vision: 91. Image and Video Coding Compressing data to a smaller volume without losing (too much) information.](https://reader036.fdocuments.in/reader036/viewer/2022062422/56649ee65503460f94bf62dd/html5/thumbnails/31.jpg)
Image Processing and Computer Vision: 9 31
Discussion Where are lossy steps?
Quatisation and subsampling before coding
How is quantisation matrix chosen? Its predefined by the standard after
much experimentation
![Page 32: Image Processing and Computer Vision: 91. Image and Video Coding Compressing data to a smaller volume without losing (too much) information.](https://reader036.fdocuments.in/reader036/viewer/2022062422/56649ee65503460f94bf62dd/html5/thumbnails/32.jpg)
Image Processing and Computer Vision: 9 32
Video coding Many specific standards
AVS, MOV, QT, … One universal standard
MPEG
![Page 33: Image Processing and Computer Vision: 91. Image and Video Coding Compressing data to a smaller volume without losing (too much) information.](https://reader036.fdocuments.in/reader036/viewer/2022062422/56649ee65503460f94bf62dd/html5/thumbnails/33.jpg)
Image Processing and Computer Vision: 9 33
MPEG Standards Standard specifies audio, video
and system layers MPEG-1: low data rates, poor
quality: VHS quality at 1.5Mbits-1
MPEG-2: high quality hence high data rates: studio quality, 15Mbits-1
MPEG-4: low data rates, small images, 64 kbits-1
![Page 34: Image Processing and Computer Vision: 91. Image and Video Coding Compressing data to a smaller volume without losing (too much) information.](https://reader036.fdocuments.in/reader036/viewer/2022062422/56649ee65503460f94bf62dd/html5/thumbnails/34.jpg)
Image Processing and Computer Vision: 9 34
MPEG-1 Audio and video designed to work
at CD ROM speeds: 1.5Mbits-1
Video 1.150Mbits-1
Audio 0.256Mbits-1
System 0.094Mbits-1
![Page 35: Image Processing and Computer Vision: 91. Image and Video Coding Compressing data to a smaller volume without losing (too much) information.](https://reader036.fdocuments.in/reader036/viewer/2022062422/56649ee65503460f94bf62dd/html5/thumbnails/35.jpg)
Image Processing and Computer Vision: 9 35
MPEG-2 Released in 1994 Aimed at digital TV, ATM. Additions for
Interlaced video Scalable video coding Graceful degradation with noise
Implementation of full standard impractical Varying profiles/levels of conformity
![Page 36: Image Processing and Computer Vision: 91. Image and Video Coding Compressing data to a smaller volume without losing (too much) information.](https://reader036.fdocuments.in/reader036/viewer/2022062422/56649ee65503460f94bf62dd/html5/thumbnails/36.jpg)
Image Processing and Computer Vision: 9 36
MPEG-4 Coding specifically for multimedia
objects
![Page 37: Image Processing and Computer Vision: 91. Image and Video Coding Compressing data to a smaller volume without losing (too much) information.](https://reader036.fdocuments.in/reader036/viewer/2022062422/56649ee65503460f94bf62dd/html5/thumbnails/37.jpg)
Image Processing and Computer Vision: 9 37
Coding Algorithms Frame sequence Motion compensation Frame coding
![Page 38: Image Processing and Computer Vision: 91. Image and Video Coding Compressing data to a smaller volume without losing (too much) information.](https://reader036.fdocuments.in/reader036/viewer/2022062422/56649ee65503460f94bf62dd/html5/thumbnails/38.jpg)
Image Processing and Computer Vision: 9 38
Frame Sequence I frames (Intraframes)
Coded independently of any other frame P frames (Predicted frames)
Derived from previous I frame by motion prediction
B frames (Bidirectionally interpolated) Interpolate motion compensated blocks
between I and P frames
![Page 39: Image Processing and Computer Vision: 91. Image and Video Coding Compressing data to a smaller volume without losing (too much) information.](https://reader036.fdocuments.in/reader036/viewer/2022062422/56649ee65503460f94bf62dd/html5/thumbnails/39.jpg)
Image Processing and Computer Vision: 9 39
Motion Compensation Image is divided into macroblocks
(16 x 16 pixels) Matching macroblocks are found
by minimising differences Code differences and macroblock
displacement
![Page 40: Image Processing and Computer Vision: 91. Image and Video Coding Compressing data to a smaller volume without losing (too much) information.](https://reader036.fdocuments.in/reader036/viewer/2022062422/56649ee65503460f94bf62dd/html5/thumbnails/40.jpg)
Image Processing and Computer Vision: 9 40
Frame Coding Use JPEG algorithms
![Page 41: Image Processing and Computer Vision: 91. Image and Video Coding Compressing data to a smaller volume without losing (too much) information.](https://reader036.fdocuments.in/reader036/viewer/2022062422/56649ee65503460f94bf62dd/html5/thumbnails/41.jpg)
Image Processing and Computer Vision: 9 41
Summary Why code data? Redundancy Assessment of compression Lossy vs. lossless compression Algorithms
JPEG, MPEG
![Page 42: Image Processing and Computer Vision: 91. Image and Video Coding Compressing data to a smaller volume without losing (too much) information.](https://reader036.fdocuments.in/reader036/viewer/2022062422/56649ee65503460f94bf62dd/html5/thumbnails/42.jpg)
Image Processing and Computer Vision: 9 42
But what … is it good for?Engineer at the Advanced Computing
Systems Division of IBM, commenting on the microchip in 1968