6.003: Signal Processing

Fourier-based Compression

• Lossy compression • Discrete Cosine Transform (DCT) • JPEG overview

May 5, 2020

Background: Communications Systems

We previously (week 8) talked about the importance of signal processing

in the development of communications techologies.

Examples:

• cellular communications • wifi • broadband • bluetooth • GPS (Global Positioning System) • IOT (Internet of Things) − smart house / smart appliances − smart car − medical devices

• cable • private networks: fire departments, police • radar and navigation systems

Background: Communications Systems

Our focus in week 8 was on matching signals to communications media.

From wires:

mic amp telephone wire amp speaker

to wireless.

mic amp E/M wave amp speaker

Telephone-quality speech contains frequencies from 200 Hz to 3000 Hz.

But a pocket-sized wireless antenna works best at GHz.

We addressed this mismatch using strategies based on modulation.

Compression

Today we will revisit issues in communications systems – this time from the

perspective of reducing the number of bits needed to represent the signal.

We will find that many effective ways to reduce bit-count are based on

frequency representions.

There are two general types of compression:

• lossless – take advantage of statistical dependences in the bitstream

– enormously successful: LZW, Huffman, zip ...

– learn more in 6.006, ...

• lossy – focus on perceptually important information

– enormously successful: JPEG, MP3, MPEG ...

– topic of today’s lecture

We’ll start with sample frequency and quantization, both of which have

enormous effects of bit rates – especially for high-quality signals.

Effects of Sampling Are Easily Heard

Sampling Music

fs = 1 T

• fs = 44.1 kHz • fs = 22 kHz • fs = 11 kHz • fs = 5.5 kHz • fs = 2.8 kHz

J.S. Bach, Sonata No. 1 in G minor Mvmt. IV. Presto

Nathan Milstein, violin

While fs < 5.5 kHz is not acceptable for many purposes, there is little (if any) perceptual difference between 22 kHz and 44 kHz. • the difference is a factor of two in bits! • most lossless coding schemes don’t recapture much of this difference.

– zip reduces both the 22 and 44.1 kHz versions by about 14%.

Effects of Sampling are Easily Seen

Sampling Images

original: 2048× 1536

Effects of Sampling are Easily Seen

Sampling Images

downsampled: 1024× 768

Effects of Sampling are Easily Seen

Sampling Images

downsampled: 512x384

Effects of Sampling are Easily Seen

Sampling Images

downsampled: 256× 192

Effects of Sampling are Easily Seen

Sampling Images

downsampled: 128× 96

Effects of Sampling are Easily Seen

Sampling Images

downsampled: 64× 48

Effects of Sampling are Easily Seen

Sampling Images

downsampled: 32× 24

Effects of Sampling are Easily Seen

Sampling Images

As with audio, low sample rates are not acceptable for many purposes.

However, there is little perceptual change at high sample rates.

Quantization

Another way to reduce bit rate is to reduce the number of bits used to

represent each sample.

­1 ­1

0

0

1

1 Input voltage

O u tp u t v o lt a g e 2 bits 3 bits 4 bits

00

01

10

0 0.5 1 ­1

0

1

Time (second) 0 0.5 1

Time (second) 0 0.5 1

Time (second)

­1 0 1 Input voltage

­1 0 1 Input voltage

Quantization of Audio Signals

We hear sounds that range in amplitude from 1,000,000 to 1.

How many bits would be necessary to code the loudest sounds we can hear

if the quietest sounds us just 1 bit?

Quantization of Audio Signals

We hear sounds that range in amplitude from 1,000,000 to 1.

How many bits would be necessary to code the loudest sounds we can hear

if the quietest sounds us just 1 bit?

bits range

1 2 2 4 3 8 4 16 5 32 6 64 7 128 8 256 9 512

10 1, 024 11 2, 048 12 4, 096 13 8, 192 14 16, 384 15 32, 768 16 65, 536 17 131, 072 18 262, 144 19 524, 288 20 1, 048, 576

Quantization of Audio Signals

Quantizing Music

• 16 bits/sample • 8 bits/sample • 6 bits/sample • 4 bits/sample • 3 bits/sample • 2 bit/sample

J.S. Bach, Sonata No. 1 in G minor Mvmt. IV. Presto

Nathan Milstein, violin

Quantization of Audio Signals

Quantizing Music

• 16 bits/sample • 8 bits/sample • 6 bits/sample • 4 bits/sample • 3 bits/sample • 2 bit/sample

J.S. Bach, Sonata No. 1 in G minor Mvmt. IV. Presto

Nathan Milstein, violin

Quantization of Audio Signals

Quantizing Music

• 16 bits/sample • 8 bits/sample • 6 bits/sample • 4 bits/sample • 3 bits/sample • 2 bit/sample

J.S. Bach, Sonata No. 1 in G minor Mvmt. IV. Presto

Nathan Milstein, violin

Quantization of Audio Signals

Quantizing Music

• 16 bits/sample • 8 bits/sample • 6 bits/sample • 4 bits/sample • 3 bits/sample • 2 bit/sample

J.S. Bach, Sonata No. 1 in G minor Mvmt. IV. Presto

Nathan Milstein, violin

Quantization of Audio Signals

Quantizing Music

• 16 bits/sample • 8 bits/sample • 6 bits/sample • 4 bits/sample • 3 bits/sample • 2 bit/sample

J.S. Bach, Sonata No. 1 in G minor Mvmt. IV. Presto

Nathan Milstein, violin

Quantization of Audio Signals

Quantizing Music

• 16 bits/sample • 8 bits/sample • 6 bits/sample • 4 bits/sample • 3 bits/sample • 2 bit/sample

J.S. Bach, Sonata No. 1 in G minor Mvmt. IV. Presto

Nathan Milstein, violin

Quantization of Audio Signals

Quantizing Music

• 16 bits/sample • 8 bits/sample • 6 bits/sample • 4 bits/sample • 3 bits/sample • 2 bit/sample

J.S. Bach, Sonata No. 1 in G minor Mvmt. IV. Presto

Nathan Milstein, violin

Quantization of Audio Signals

Quantizing Music

• 16 bits/sample • 8 bits/sample • 6 bits/sample • 4 bits/sample • 3 bits/sample • 2 bit/sample

J.S. Bach, Sonata No. 1 in G minor Mvmt. IV. Presto

Nathan Milstein, violin

Quantizing Images

Converting an image from a continuous representation to a discrete repre-

sentation involves the same sort of issues.

This image has 280× 280 pixels, with brightness quantized to 8 bits.

Quantizing Images

8 bit image 7 bit image

Quantizing Images

8 bit image 6 bit image

Quantizing Images

8 bit image 5 bit image

Quantizing Images

8 bit image 4 bit image

Quantizing Images

8 bit image 3 bit image

Quantizing Images

8 bit image 2 bit image

Quantizing Images

8 bit image 1 bit image

Perceptual (Lossy) Compression

Perceptual coding offers additional reductions in the total number of bits

needed to represent audio and video signals.

Examples: JPEG (for pictures), MP3 (for audio), and MPEG for (video).

Each of these compression standards combine lossless coding (such as

LZW, Huffman, zip) with perceptual coding in the frequency domain.

Example: JPEG (Joint Photographic Experts Group) Encoding

• color encoding: RGB → YCrCb • 2D DCT (discrete cosine transform): a kind of Fourier series • quantization to achieve perceptual compression (lossy) • run-length and Huffman encoding (lossless)

We will focus on the DCT and quantization of its components.

• the image is broken into 8 × 8 pixel blocks • each block is represented by its 8 × 8 DCT coefficients • each DCT coefficient is quantized, using higher resolutions for coeffi-

cients with greater perceptual importance

Block Processing

The idea behind block processing is that small patches of an image can be

coded efficiently (with just a few bits).

Start with an image, such as this ball.

Block Processing

The idea behind block processing is that small patches of an image can be

coded efficiently (with few bits).

Break the image into blocks.

Block Processing

The idea behind block processing is that small patches of an image can be

coded efficiently (with few bits).

Represent each block with as few bits as possible.

Energy Compaction

The block has 8×8 = 64 pixels.

Representing each pixel in a block with an 8-bit number

→ a total of 64 bytes for this block.

Energy Compaction

Try coding the 2D DFT instead. Here is the magnitude of the 2D DFT.

log1