Introduction to Source Coding - German University in...

Comm. 502: Communication Theory

Lecture 8

Introduction to Source Coding

-Lampel-Ziv code

Ziv Coding-Lempel

• Huffman coding requires knowledge of a probabilistic model of the source

– This is not necessarily always feasible

• The Lempel-Ziv method specifies fixed-length code words and

does not require prior knowledge of symbol probabilities..

• Lempel-Ziv codes tend to be quite inefficient during the initial

portion of the message ensemble. This compares poorly with

the other existing methods.

• The string must be sufficiently long for the algorithm to achieve

good compression over the full string.

• Lempel-Ziv coding is asymptotically optimal, meaning

that the redundancy approaches zero as the length of the

source ensemble tends to infinity.

• However, for particular finite sequences, the compression

achieved may be far from optimal. (When the method

begins, each source symbol is coded individually, the

method gives about 50% expansion during initial

encoding).

• Lempel-Ziv code assumes that a dictionary exists

containing already coded segments of a sequence of

alphabet symbols. Data is encoded by looking through the

existing dictionary for a match to the next short segment in

the sequence being coded.

Ziv Coding-Lampel

Example: Binary version of Lempel-Ziv Coding

0 0 0 1 0 1 1 1 0 0 1 0 1 0 0 1 0 1…

Code book Index 1 2 3 4 5 6 7 8 9

Subsequence 0 1

Representation

Encoding

Suppose we have the following binary stream:

Ziv Coding-Example: Binary version of Lempel

0 0 0 1 0 1 1 1 0 0 1 0 1 0 0 1 0 1…

Codebook Index 1 2 3 4 5 6 7 8 9

Subsequence 0 1 00

Representation

Encoding


0 0 0 1 0 1 1 1 0 0 1 0 1 0 0 1 0 1…


Subsequence 0 1 00 01

Representation

Encoding


0 0 0 1 0 1 1 1 0 0 1 0 1 0 0 1 0 1…


Subsequence 0 1 00 01 011

Representation

Encoding


0 0 0 1 0 1 1 1 0 0 1 0 1 0 0 1 0 1…


Subsequence 0 1 00 01 011 10

Representation

Encoding


0 0 0 1 0 1 1 1 0 0 1 0 1 0 0 1 0 1…


Subsequence 0 1 00 01 011 10 010

Representation

Encoding


0 0 0 1 0 1 1 1 0 0 1 0 1 0 0 1 0 1…


Subsequence 0 1 00 01 011 10 010 100

Representation

Encoding


0 0 0 1 0 1 1 1 0 0 1 0 1 0 0 1 0 1…


Subsequence 0 1 00 01 011 10 010 100 101

Representation

Encoding


0 0 0 1 0 1 1 1 0 0 1 0 1 0 0 1 0 1…


Subsequence 0 1 00 01 011 10 010 100 101

Representation

(use index) Position of each

bit

11 12 42 21 41 61 62

Encoding


0 0 0 1 0 1 1 1 0 0 1 0 1 0 0 1 0 1…


Subsequence 0 1 00 01 011 10 010 100 101

Representation 11 12 42 21 41 61 62

Source Code

Use Binary representation

of first bit

0010 0011 1001 0100 1000 1100 1101

0010 0011 1001 0100 1000 1100 1101 Source encoded bits

Information bits

Ziv Coding-Example: Binary version of Lempel • The last bit of each subsequence (4th row of last table) is called

innovation bit. This name comes from: its addition to a particular subsequence distinguishes it from all previous subsequences stored in the code book.

• The remaining bits provides the equivalent binary representation of the “pointer” to the root subsequence that matches the one in question except for the innovation bit.

• The decoder uses the pointer to identify the root subsequence and then appends the innovation bit.

• For example, the block 1101 in position 9. The last bit is, 1, is the innovation bit. The remaining bits, 110, point to the root subsequence 10 in position 6. Then put the innovation bit after root sequence. Hence, the block 1101 is decoded into 101 which is correct.

2Ziv Coding Example -Lempel

A A B A B B B A B A A B A B B B A B B A ……

Code book Index 1 2 3 4 5 6 7 8

Subsequence A

Representation

Encoding

We start with the shortest phrase on the left that we haven't seen before.


Codebook Index 1 2 3 4 5 6 7 8

Subsequence A AB

Representation

Encoding

A A B A B B B A B A A B A B B B A B B A ..…



Subsequence A AB ABB

Representation

Encoding




Subsequence A AB ABB B

Representation

Encoding




Subsequence A AB ABB B ABA

Representation

Encoding

A A B A B B B A B A A B A B B B A B B A ……



Subsequence A AB ABB B ABA ABAB

Representation

Encoding




Subsequence A AB ABB B ABA ABAB BB

Representation

Encoding




Subsequence A AB ABB B ABA ABAB BB ABBA

Representation

Encoding

A A B A B B B A B A A B A B B B A B B A




Representation 0A 1B 2B 0B 2A 5B 4B 3A

Encoding

The empty set ; should be considered as the 0'th phrase and encoded by 0.

The second row gives the phrases, and the third row their representation.

That is, when we represent the ABAB from the sixth phrase, we represent it as

5B. This maps to ABAB since the fifth phrase was ABA, and we add B to it.




Representation 0A 1B 2B 0B 2A 5B 4B 3A

Encoding 0000 0011 0101 000 1 0100 1011 1001 0110

A 0

B 1

We have taken the third row of the previous array, expressed all the numbers

in binary (before the comma) and the letters in binary (after the comma).

Note that we mapped A to 0 and B to 1. If you had a larger

alphabet, you would encode the letters by more than one

bit.

Repeat this table by assuming A and B in the first two places in the dictionary

2Ziv Example -Remarks on Lampel

• We have a sequence of 8 symbols in the coded sequence.Each symbol consisting of 4 bits.

• Therefore we have a total number of 32 bits representing 20 characters.

• As in many compression schemes, coding gain is not realized for short sequences.

Digital Communication Systems

Source of Information

User of Information

Source Encoder

Channel Encoder

Modulator

Source Decoder

Channel Decoder

De-Modulator

Channel Communication systems are designed to transmit the information generated by a source to some destination.

The Reasons for Bandpass Modulation

1. To decrease size of Antenna

For efficient radiation of electromagnetic energy, the

size of the radiation antenna should be on the order

of one-tenth or more the wavelength of the radiated

signal.

Then as ,

m/s103, 8 cf

cAntennaofSize

f

Consider the case of several radio stations broadcasting audio baseband

signals directly, without any modulation. They would interfere with each

other because the spectra of all the signals occupy more or less the same

bandwidth.

We can use various audio signals to modulate different carrier

frequencies, thus translating each signal to a different frequency range. If

the various carriers are chosen sufficiently far apart in frequency, the

spectra of the modulating signals will not overlap and thus will not

interfere with each other.

2. Simultaneous Transmission of Several Signal

. Better control of filtering operation3

Easy to keep the interesting signal and to reject all the other signals

using BPF.

• A carrier frequency is used to shift the frequency spectrum of transmitted signals.

• Baseband signals w(t) may be transformed into bandpass signals through multiplication by a carrier (Modulation):

ccc ffWffWtwtw 2

1cos

- fc 0 +fc

W(f)

f - fm 0 +fm

W(f)

f

Band Pass Modulation

Ex: Binary Phase Shift Keying (BPSK)

• In PSK, the information is contained in the instantaneous phase of the modulated carrier.

• For binary PSK, phase states of 0 and 180 are used. • Time domain representation:

1

0)2cos(

)2cos()(0

)2cos()(1

1

1

tfA

tfAts

tfAts

c

c

c

Ex: Phase Shift Keying (PSK) Modulation

1 0 1 1 0 1

Base band Signal X(t)

Band Pass Signal Y(t)

Y(t)=X(t) cos(2πfct)

PSK Demodulation

X(t)cos(2πfct) x

2cos(2πfct)

X(t)[2cos2(2πfct)]

Low Pass Filter X(t)

X(t)[2cos2(2πfct)] =X(t)[1+cos(4πfct)]

=X(t) +X(t)cos(4πfct)]

Base band Signal (i.e., low frequency content)

(desired signal)

High frequency content rejected by the LPF

Without noise

Let A=1

Introduction to Source Coding - German University in...

Documents

Transcript of Introduction to Source Coding - German University in...