Channel Coding - uni-rostock.de · A. Ahrens, Institute of Communications Engineering, University...

Baltic Summer School 2006, Vilnius, Lithuania, 31.07.–14.08.2006

Channel Coding

Andreas Ahrens

Institute of Communications Engineering

University of Rostock, Germany

A. Ahrens, Institute of Communications Engineering, University of Rostock, Germany 1

Contact information Baltic Summer School 2006, Vilnius, Lithuania, 31.07.–14.08.2006

Contact

Privatdozent Dr.-Ing. habil. Andreas Ahrens

University of RostockFaculty of Computer Science and Electrical EngineeringInstitute of Communications EngineeringRichard-Wagner-Str. 3118119 Rostock Germany

Phone: (+49) 0381 / 498-7313Fax: (+49) 0381 / 498-7302

Email: [email protected]


Introduction Baltic Summer School 2006, Vilnius, Lithuania, 31.07.–14.08.2006

Contents (1)

I Introduction

• Error-detection and error-correction

• Channel characteristics, error-structures with and without

memory

I Fundamentals of block codes

• Linear block codes (encoding, decoding)

• Systematic and non-systematic codes

• Cyclic codes (encoding, decoding)

I Fundamentals of convolutional codes

• Description (code tree, state diagram, trellis diagram)

• Encoding, Decoding, Termination



Contents (2)

I Channel Models

• Analysis of error structures

• Error-gap distribution, error-gap density function

• Generation of error structures with memory

I Code concatenating

I Interleaving

I Coding gain

I Exercises and Consultations



Literature (1)

I Anderson, J. B.: Digital Transmission Engineering. Munchen,

New York: IEEE Press, 1999

I Bingham, J. A. C.: The Theory and Practice of Modem Design.

New York: Wiley, 1988

I Bossert, M.: Channel Coding for Telecommunications. New York:

Wiley, 1999

I Bossert, M.: Kanalcodierung. Stuttgart: Teubner, 1998

I Clark, G. C.; Cain, J. B.: Error-Control Coding for digital

Communications. New York: Plenum, 1981

I Friedrichs, B.: Kanalcodierung - Grundlagen und Anwendungen

in Kommunikationssystemen. Berlin: Springer, 1995



Literature (2)

I Goldsmith, A.: Wireless Communications. New York: Cambridge,

2005

I Heise, W.; Quattrochi, P.: Informations- und

Codierungstheorie. Berlin, Heidelberg: Springer, 1995

I Huber, J.: Trelliscodierung. Berlin, Heidelberg: Springer, 1992

I Larrson, R.; Stoica, P.: Space-Time Block Coding for Wireless

Communications. New York: Cambridge, 2003

I Oberg, T.: Modulation, Detection and Coding. Chichester: Wiley,

2001

I Paulraj, A.; Nabar, R.; Gore, D.: Introduction to Space-Time

Wireless Communications. New York: Cambridge, 2003



Literature (3)

I Proakis, J. G.: Digital communications. Boston: McGraw-Hill,

2000

I Proakis, J. G.; Salehi, M.: Grundlagen der

Kommunikationstechnik. Munchen: Pearson-Education, 2004

I Rohling, H.: Einfuhrung in die Informations- und

Codierungstheorie. Stuttgart: Teubner, 1995

I Schneider-Obermann, H.: Kanalcodierung – Theorie und Praxis

fehlerkorrigierender Codes. Braunschweig/Wiesbaden: Vieweg, 1998

I Hamming, R. W.: Information and Coding, Prentice Hall, 1988



HF-channel characteristic

PSfrag replacements reflectiondispersion (scattering)

direct path

diffraction

Time- and frequency dispersion of wireless transmission channelsthrough multipath propagation and mobile objects.



Transmission model

PSfrag replacements

analoguesignal

samplingquanti-zation

com-pression

source encoding

channelencoder

baseband-modulation

LP/BP-transform

analoguechannel

BP/LP-transform

Detectorchanneldecoder

decom-pression

DA-converter

signal re-constr.

digital channelsource decoding

binary sequencebinary sequence

disturbances

Likelihoodinformation



Source coding (1)

Representation of a message or a signal by a value-discrete and atime-discrete sequence (e. g. a binary sequence) with an as smallas possible bit rate.

a) Speech signal:

• Frequency range: 300 Hz to 3.4 kHz

• Sample rate: 8 kHz

• Quantization: 8 bit/sample

Example:

• 64 kbit/s PCM (Pulse Code Modulation)

Research:

• CELP (Codebook Exited Linear Predictive Coding)

6.8 . . . 4.8 kbit/s with approximately the same quality

• MELP (Mixed Excitation Linear Prediction) 2.4 kbit/s



Source coding (2)

b) Audio signal:

• Frequency range: 20 Hz to 20 kHz

• CD (compact disk) sample rate: 44 kHz

• Quantization: 17 bit/sample

Result:

• 700 kbit/s per stereo channel

Research:

• 90 kbit/s per stereo channel for digital broadcast (MPEG4

Audio, DAB (Digital Audio Broadcasting))

Bit stream is highly sensitive against transmission errors →

channel coding.



Error structures without memory

- - - - - - - - - - - - - - - - x - - - - - - x - - - - x- - - x - - - - x - - - - - - - - - - - x x - - - - - - -- - - - - - - - x - - - - x - - - - - - - - x - - - - - -- - - - - - - - - x x - - - - - - - - - - - - - - - x - -- - - - - - - - - - - - - - - - - x - - - - - - - - - - -- - - - x - - - - - - - - - - - x - - - - - - - - - - - -- - - - - - - - - - x - - - - - - x - - - - - - - - - - -- - - - - - x - - - - - x - - x - - - - - - - - - - - - -- - - - - - - - x - - - - - - x - - - - - - - - - - - - x- - - x - - - - - - - - - - - - - - - - - x x x x x - - -x - - - - - - x - - - - - - - - - - x - - - x - - - - - -- - - - x - - - - - - - - - - - - - - x - - - - - - - - -- x - - - x - - - - - - - - - - - x - x - - - - - - - - -- - - - - - - - x - - - - - - - - - - - - - - - - - - - -- - - - - - - - - - - - - - - - x - - - - - - - - - - - -- - - - - - - - x x - - - - - - - - - - - - - - x - - - -- - - - - - - - - - - - - - - - - - x - - - - - - - - - -- x - - - - - - - - - - - - - x - - x - - - - - - - - - -- - - x - - - - - - - - - - - - - - - - - - - x - - - - -- - - - - - - - - - - x - - - - - - - - - - - x - - - - -- - - - - - - - x - - - - - - - - - - - - - x - - - - - -- x - - - - - - - - - - - - - - - x - - - - - - - - - - -- - - - - - - - x x - - - - - - - - - - - - x - - - - - -

Parameter:

x bit error- correct bit



Error structures with memory

x x - x x x - - x x x x x x - x x - - - - - - - - - - - -- - - - - - - x x x x - x - - x x - - x x - - - - - - - -- - - - - - - - - - - - - - - - - - - - - - - - - - - - -- - - - - - - - - - - - - - - - - - - - - - - - - - - - -- - - - - - - - - - - - - - - - - - - - - - - - - - - - -- - - - - - - - - - - - - - - - - - - - - - - - - - - - -- - - - - - - - - - - - - - - - - - - - - - - - - - - - -- - - - - - - - - - - - - - - - - - - - - - - - - x x x x- - - - - - - - - - - - - - - - - - - - - - - - - - - - -- - - - - - - - - - - - - - - - - - - - - - - - - - - - -- - - - - - - - - - - - - - - - - - - - - - - - - - - - -- - - - - - - - - - - - - - - - - x x - - - - - - - - - -- - - - - - - - - - - - - - - - - - - - - - - - - - - - -- - - - - - - - - - - - - - x x x x - - - - - x - - x x -- - - - - - - - - - - - - - - - - - - - - - - - - - - - -- - - - - - - - - - - - - - - - - - - - - - - - - - - - -- - - - x x x - - - - x x x - - - - - - - - - - - - - - -- - - - - - - - - - - - - - - - - - - - - - - - - - - - -- - - - - - - x x x - - - - - - - - - - - - - - - x x - -- - - - - - - - - - - - - - - - - - - - - - - - - - - - -x x x x x - - - - - - - - - - - x x - - x x x x x x - x -- - - - - - - - - - - - - - - - - - - - - - - - - - - - -- - - x x x x x x - - - - - - - - - - - - - - - - - - - -

Parameter:




Channel coding (1)

a) CD (compact disk)

• Interleaved Reed-Solomon codes

• (Protection against error-bursts)

• . . .

b) Coding in data modems:

• Convolutional codes, coded modulation (Ungerboeck 1982),

. . .

• DSL (Digital Subscriber Line): Reed-Solomon forward error

correction and trellis-coded modulation

• . . .



Channel coding (2)

c) Coding in satellite communications:

• Convolutional codes, Reed-Solomon codes, Reed-Muller codes

• Block-coded modulation, turbo-codes

• Code concatenating

• . . .

d) Coding in mobile communications:

• Interleaved convolutional codes, fire codes

• Reed-Solomon codes, Hamming codes

• . . .



Historical development (1)

Year Development

1948 Shannon founded the information theory

1950 one-bit error correctable Hamming codes

1952 Gilbert specified bounds for block codes

1961/62 sequential decoding (Wozencraft, Fano)

1967 Viterbi decoding

1968/69 Berlekamp-Massey algorithm for decoding of BCH (Bose,

Chaudhuri, Hocquenghem)- and RS (Reed Solomon) codes

1968 convolutional codes in Pioneer spacecraft

...



Historical development (2)

Year Development

1974 convolutional codes in Helios spacecraft

1976 Ungerboeck codes (coded modulation)

1982 trellis coded modulation

1985 VLSI (Very Large Scale Integration) realization of RS- and

Viterbi decoder

1988/90 punctured convolutional codes for application in mobile communications

1993 Turbo codes



Other possibilities of coding

a) Encryption:

• Cryptography, DES (Data Encryption Standard), . . .

b) Codes for spectral adaptation (Line codes):

• Pseudo-ternary codes (e. g. HDB-3, AMI), Manchester code,

. . .

c) Codes for frame synchronization or channel identification:

• Barker codes, m-sequences, CAZAC (Constant Amplitude Zero

Autocorrelation) sequences, . . .

d) Codes for spreading:

• m-sequences, Gold codes, orthogonal Gold codes, . . .


Block codes Baltic Summer School 2006, Vilnius, Lithuania, 31.07.–14.08.2006

Basic terms – Block codes

Parameter Description Meaning

codeword c = (c0, c1, c2, · · · , cn−1) sequence of n bits

(valid codeword, sequence of n binary symbols)

information word i = (i0, i1, i2, · · · , im−1) sequence of m bits

(sequence of m binary information symbols)

number of parity bits k = n − m

weight w{c} non-zero vector coordinates

distance h = d{cµ, cν}

minimum distance hmin = minµ6=ν

d{cµ, cν}

code rate R = m

n= n−k

n= 1 − k

nefficiency of the code



Weight, distance

(Hamming) weightThe (Hamming) weight of a vector c is defined as the number ofnon-zero vector coordinates.

(Hamming) distanceThe (Hamming) distance between two vectors cµ and cν is thenumber of coordinates where cµ and cν differ

h = d{cµ, cν} = w{cµ ⊕ cν} .

Minimum distanceThe minimum distance hmin of a code C is the minimum distancebetween two different codewords. For linear codes, the minimumdistance is equal to the minimum weight

hmin = minµ6=ν

d{cµ, cν} = minµ6=ν

w{cµ ⊕ cν} cµ, cν ∈ C .



The finite field GF(2)

Addition table

⊕ 0 1

0 0 1

1 1 0

Multiplication table

� 0 1

0 0 0

1 0 1



Linear block codes – Example (1)

The 2m codewords of a (n, m) code arise by linear combination ofthe basic codewords

Basic codewords of a (7, 2) code

a = (1 1 0 0 1 0 1)

b = (1 0 1 0 0 0 1)

The code has 2m = 22 = 4 valid codewords. These are

all-zero codeword : (0 0 0 0 0 0 0)

a : (1 1 0 0 1 0 1)

b : (1 0 1 0 0 0 1)

a ⊕ b : (0 1 1 0 1 0 0)



Linear block codes – Example (2)

Basic codewords of a (7, 3) code

a = (1 1 0 0 1 0 1) b = (1 0 1 0 0 0 1) c = (1 1 1 0 0 1 0)

The code has 2m = 23 = 8 valid codewords. These are

all-zero codeword : (0 0 0 0 0 0 0)

a : (1 1 0 0 1 0 1)

b : (1 0 1 0 0 0 1)

c : (1 1 1 0 0 1 0)

a ⊕ b : (0 1 1 0 1 0 0)

a ⊕ c : (0 0 1 0 1 1 1)

b ⊕ c : (0 1 0 0 0 1 1)

a ⊕ b ⊕ c : (1 0 0 0 1 1 0)



Generator matrix (1)

The generator matrix is a compact description of how codewordsare generated from information bits in a linear block code.

All basic codewords generate a (m × n) matrix

G =

g0,0 g0,1 · · · g0,n−1

g1,0 g1,1 · · · g1,n−1...

......

...gm−1,0 gm−1,1 · · · gm−1,n−1

=

g0

g1...

gm−1

.

The rows of G are the m linearly independent basic vectors(codewords) g0, g1, · · · , gm−1 of the code. Each codeword c canbe constructed by

c = i · G .



Generator matrix (2)

A systematic linear block code is described by a generatormatrix of the form

G =

1 0 · · · 0 a0,0 a0,1 · · · a0,k−1

0 1 · · · 0 a1,0 a1,1 · · · a1,k−1

......

......

......

0 0 · · · 1 am−1,0 am−1,1 · · · am−1,k−1

= (Im|A) .

Note that any generator matrix of a (n, m) linear block code canbe reduced by row operations and column permutations to agenerator matrix in systematic form.



Parity check matrix (1)

Codewords c = (c0, c1, · · · , cn−1) of a linear code can be definedby the following equation

H · cT = 0(n−m)×1 or c · HT = 01×(n−m)

with H as parity check matrix. The parity check matrix is used todecode linear block codes with generator matrix G.

The ((n − m) × n) parity check matrix H corresponding to agenerator matrix G = (Im|A) is defined as

H =(

AT| I

k

)

.

Between the generator matrix and parity check matrix

G · HT = 0m×(n−m)

is valid.



Parity check matrix (2)

The ((n − m) × n) parity check matrix H corresponding to agenerator matrix G = (Im|A) is defined as

H =(

AT| I

k

)

.

Parity check matrix of a systematic code

H =

a0,0 a1,0 · · · am−1,0 1 0 · · · 0

a0,1 a1,1 · · · am−1,1 0 1 · · · 0...

......

......

...

a0,k−1 a1,k−1 · · · am−1,k−1 0 0 · · · 1

=(

AT| I

k

)

.



Syndrome decoding

Let d be the received codeword resulting from the transmission ofthe codeword c. In the absence of channel errors, it yields: d = c.However, if the transmission is corrupted, one or more of thecodeword symbols in d will differ from those in c. It holds

d = c + f ,

where f = (f0, f1, · · · , fn−1) is the error vector indicating whichsymbols are corrupted by the channel.We define the syndrome S as

S = d · HT = (c + f) · HT = f · HT.

Note that the syndrome S1×(n−m) is a function only of the errorpattern f and not the transmitted codeword c.



Syndrome decoding – Example (1)

(5, 2, 3) code with G and H

G =

(

1 0 1 1 0

0 1 1 0 1

)

H =

1 1 1 0 0

1 0 0 1 0

0 1 0 0 1

.

Syndrome table assuming one bit-error

error position syndromej S

0 (1 1 0)

1 (1 0 1)

2 (1 0 0)

3 (0 1 0)

4 (0 0 1)



Syndrome decoding – Example (2)

Syndrome table assuming two bit-errors

error position syndromej k S

0 1 (0 1 1)

0 2 (0 1 0)

0 3 (1 0 0)

0 4 (1 1 1)

1 2 (0 0 1)

1 3 (1 1 1)

1 4 (1 0 0)

2 3 (1 1 0)

2 4 (1 0 1)

3 4 (0 1 1)



Cyclic codes (1)

The generator polynomial of a (n, m, hmin) code has the degreek = n − m and is of the form

g(x) = g0 + g1 x1 + · · · + gk xk,

where gµ is binary (0 or 1) and g0 = gk = 1. The m-bitinformation sequence is written in polynomial form as

i(x) = i0 + i1 x1 + · · · + im−1 xm−1.

The codeword c(x) is obtained from

c(x) = i(x) · g(x) = c0 + c1 x1 + · · · + cn−1 xn−1.



Cyclic codes (2)

A codeword described by a polynomial c(x) is a valid codeword fora cyclic code with the generator polynomial g(x) if and only ifg(x) divides c(x) with no remainder – that is, if and only if

c(x)

g(x)= q(x)

holds.

Cyclic codeA code is cyclic if each cyclic shift of the coordinates in acodeword results in a codeword. A cyclic shift of a codewordc = (c0, c1, · · · , cn−1) over i coordinates results in the followingcodeword

c = (ci , ci+1, · · · , cn−1, c0, c1, · · · , ci−1) .



Cyclic codes (3)

Description

1. Cyclic codes can be represented by polynomials.

2. Multiplication and division of polynomials can be realized by

shift register.

Encoder for a cyclic (6, 3) codePSfrag replacements

τ0

∆ τ1

∆ τqk−1

g3 g2 g1 g0

0 0 0 i0 i1 i2

c0 c1 c2 c3 c4 c5

memory elements



Cyclic codes (4)

Systematic codes (1)A cyclic code can be put in a systematic form by first multiplyingthe information polynomial i(x) by xn−m = xk , yielding

i(x) · xn−m = im−1 xn−1 + · · · ... + i1 xn−m+1 + i0 · xn−m

. (1)

This shifts the information bits to the m rightmost digits of thecodeword polynomial. If we next divide (1) by g(x), we obtain

i(x) · xn−m

g(x)= q(x) +

r(x)

g(x). (2)

Multiplying (2) by g(x) we obtain

i(x) · xn−m = q(x) · g(x) + r(x) , (3)

where r(x) is the remainder polynomial.



Cyclic codes (5)

Systematic codes (2)This implies that

i(x) · xn−m− r(x) = q(x) · g(x) = c(x)

is a valid codeword since it is divisible by g(x) with no remainder.In the GF(2) we obtain

r(x) = −r(x) .

The systematic codeword has the first m bits consisting of theinformation bits and the last n − m = k bits consisting of theparity bits, as it is required for the systematic form.



Cyclic Hamming Codes – Parameter

Parameter Value

block length n = 2k− 1

number of parity bits k

number of information bits m = n − k

minimum Hamming distance hmin = 3

code rate R = n−k

n= 1 −

k

n



Cyclic Hamming Codes – Examples

k n n − k generator polynomial g(x)

3 7 4 x3 + x + 1

4 15 11 x4 + x + 1

5 31 26 x5 + x2 + 1

6 63 57 x6 + x + 1

7 127 120 x7 + x + 1



(7, 4) Hamming code – valid codewords (1)

i(x) · g(x) c(x)

1 (1) · g(x) x3 + x + 1

2 (x) · g(x) x4 + x2 + x

3 (1 + x) · g(x) x4 + x3 + x2 + 1

4 (x2) · g(x) x5 + x3 + x2

5 (x2 + 1) · g(x) x5 + x2 + x + 1

6 (x2 + x) · g(x) x5 + x4 + x3 + x

7 (x2 + x + 1) · g(x) x5 + x4 + 1

8 (x3) · g(x) x6 + x4 + x3

9 (x3 + 1) · g(x) x6 + x4 + x + 1

10 (x3 + x) · g(x) x6 + x3 + x2 + x

11 (x3 + x2) · g(x) x6 + x5 + x4 + x2

12 (x3 + x + 1) · g(x) x6 + x2 + 1

13 (x3 + x2 + 1) · g(x) x6 + x5 + x4 + x3 + x2 + x + 1

14 (x3 + x2 + x) · g(x) x6 + x5 + x

15 (x3 + x2 + x + 1) · g(x) x6 + x5 + x3 + 1



(7, 4) Hamming code – valid codewords (2)

c(x)

1 x3 + x + 1 (1) · (x3 + x + 1)

2 x4 + x2 + x (x) · (x3 + x + 1)

4 x5 + x3 + x2 (x2) · (x3 + x + 1)

8 x6 + x4 + x3 (x3) · (x3 + x + 1)

7 x5 + x4 + 1 (x4) · (x3 + x + 1)

14 x6 + x5 + x (x5) · (x3 + x + 1)

12 x6 + x2 + 1 (x6) · (x3 + x + 1)

3 x4 + x3 + x2 + 1 (1) · (x4 + x3 + x2 + 1)

6 x5 + x4 + x3 + x (x) · (x4 + x3 + x2 + 1)

11 x6 + x5 + x4 + x2 (x2) · (x4 + x3 + x2 + 1)

15 x6 + x5 + x3 + 1 (x3) · (x4 + x3 + x2 + 1)

9 x6 + x4 + x + 1 (x4) · (x4 + x3 + x2 + 1)

5 x5 + x2 + x + 1 (x5) · (x4 + x3 + x2 + 1)

10 x6 + x3 + x2 + x (x6) · (x4 + x3 + x2 + 1)

13 (x3 + x2 + 1) · g(x) x6 + x5 + x4 + x3 + x2 + x + 1



Cyclic codes (6)

Error detection and error correctionThe codeword polynomial is of the form

c(x) = i(x) · g(x) .

The received codeword can also be written in polynomial form as

d(x) = c(x) + f (x) = i(x) · g(x) + f (x) ,

where f (x) is the error polynomial with coefficients equal to 1where errors occur.

Syndrome polynomialThe syndrome polynomial s(x) for the received codeword is definedas the remainder when d(x) is divided by g(x). The syndromepolynomial s(x) is equivalent to the error polynomial f (x) modulog(x).



Transmission system with syndrome decodingPSfrag replacements

i(x)

i(x)

c(x)

d(x) = c(x) + f(x)

d(x) = c(x) + f(x)

f(x)

c(x) + f(x) + f(x)

f(x)

encoder channel

Determination

syndrome

Determination

error-pattern

decoder


Convolutional Codes Baltic Summer School 2006, Vilnius, Lithuania, 31.07.–14.08.2006

Fundamentals of convolutional codes (1)

Encoder of a convolutional code

PSfrag replacements

information

sequenceencoder

code

sequence

A binary information sequence i is partitioned into blocks iµ with b

bits. Each information block iµ is mapped by the encoder to acode block cµ. It yields

i0, i1, i2, · · · → c0, c1, c2, · · · .

The µth code block cµ depends on the µth information block iµand the m previous information blocks, i. e. the encoder has amemory of m.




Convolutional Encoder

Stage 1 Stage 2 Stage K

1

1 11

2

2 22

n

b bb …

…

… ……PSfrag replacements

to Modulator

b bits



Fundamentals of convolutional codes – Example

Encoder of a (2, 1, 2) convolutional code (1)

Stage 1 Stage 2 Stage 3

PSfrag replacements

to Modulator

b = 1 bit




Encoder of a (2, 1, 2) convolutional code (2)

PSfrag replacementsinformation

sequence

memory elements

codesequence




State diagram of a (2, 1, 2) convolutional code

11

0110

0/00

0/11

1/00

0/10

0/01

1/10

1/01

1/1100




Code tree of a (2, 1, 2) convolutional code

PSfrag

replacem

ents

state

1 0

00

00

00

00

00

00

00

00

00

11

11

11

11

11

11

11

11

10

10

10

10

10

10

10

01

01

01

01

01

01

01




Subtrellis of a (2, 1, 2) convolutional code

state state

00 00

10 10

01 01

11 11

0/00

1/10

1/01

0/11

0/01

1/00

1/11

0/10

PSfrag replacements

input bits/output bits




Trellis diagram of a (2, 1, 2) convolutional code

state

00

10

01

11

0/00

1/11

0/00

1/01

1/11

0/10

0/00

1/10

1/01

0/11

0/01

1/00

1/11

0/10

PSfrag replacements

input bits/output bits




Maximum Likelihood (ML) Detection

ML detection of a convolutional code means finding the most likelysequence of coded symbols given the received sequence of codedsymbols.Solution (hard decision): The coded sequence c(µ) with theminimum Hamming distance to the received sequence dcorresponds to the maximum likelihood sequence

log10 Pr{

d | c(µ)}

= g log10

pe

1 − pe

+ nb log10(1 − pe) . (1)

Since pe � 0.5, (1) is maximized when g is minimized.




Maximum Likelihood (ML) Detection – Example

Log likelihood function

log10 Pr{

d | c(µ)}

= g log10

pe

1 − pe

+ nb log10(1 − pe) ,

with nb = 10 and pe = 10−2.

0 2 4 6 8 10−20

−10

0

PSfrag replacements

error-weight g →

log10Pr{

d|c(

µ) }




The difficulty with ML decoding is that the complexity ofcomputing the log likelihood function (1) grows exponentially withthe memory of the code and this computation must be done forevery possible path through the trellis.

Solution: Viterbi-Algorithm

Viterbi-Algorithm reduces the complexity of ML decoding

by taking advantage of the structure of the path metric

computation.




Viterbi decoder (1)

00 11 01receivedSequence

state

00

10

01

11

0/00

1/11

0/00

1/01

1/11

0/10

0/00

1/10

1/01

0/11

0/01

1/00

1/11

0/10

0 0

02

2

3

3

3

3

2

0

4

4

3

5




Viterbi decoder (2)

10 11 11receivedSequence

state

00

10

01

11

0/00

1/11

0/00

1/01

1/11

0/10

0/00

1/10

1/01

0/11

0/01

1/00

1/11

0/10

0 1

11

3

2

2

5

3

2

2

2

4

3

3




Termination

state

00

10

01

11

0/00 0/000/00

1/10

1/01

0/11 0/

11

0/11

0/01 0/01

1/00

1/11

0/10

0/10

PSfrag replacements

start termination


Digital channels with memory Baltic Summer School 2006, Vilnius, Lithuania, 31.07.–14.08.2006

Digital channels with memory

Received power fluctuations in wireless communication (e. g.caused by mobile transmitter or receiver or constructive anddestructive interference, ...) result in error bursts.

10−2

10−1

100

101

PSfrag replacements rece

ived

pow

er→

time →



Digital channels with memory – Basic terms (1)

Structure of a digital channel

PSfrag replacements

digitalchannel

input sequence output sequence

{ek}

error sequence

{xk} {yk}

Example:

{xk} = · · · 1 0 1 1 1 0 1 0 1 1 · · ·

{yk} = · · · 1 0 1 0 0 1 1 0 1 1 · · ·

How can the characteristic of the digital channel be determined?




Digital channel modelPSfrag replacements

digital channel

input sequence output sequence

{ek}

error sequence

{xk} {yk}

Example:

{xk} = · · · 1 0 1 1 1 0 1 0 1 1 · · ·

{yk} = · · · 1 0 1 0 0 1 1 0 1 1 · · ·

− − − − −−−−−−−

{ek} = · · · 0 0 0 1 1 1 0 0 0 0 · · ·




How can we describe a digital channel with memory?

Parameter:

Error-gap distribution

u(k) = Pr{X ≥ k}

Error-gap density function

v(k) = Pr{X = k}

Definition of an error-gap

PSfrag replacements

k1 k2 k3

bit error




Between u(k) and v(k) the following relationship exists:

u(k) = v(k) + v(k + 1) + v(k + 2) + · · ·

u(k + 1) = v(k + 1) + v(k + 2) + · · · ,

which leads tov(k) = u(k) - u(k+1) . (1)

The digital channel characteristic can be completely defined by the

error gap distribution u(k) if we assume that the distances (gaps)

between neighbouring errors are statistically independent form each

other.




How can we determine v(k) or u(k)?

Solution: Analyzing of error structures of real communication links

Thereby the following relationship was found (Wilhelm 1976)

pb(n) = pe · nα (with pe · n

α� 1).

The block-error rate pb(n) can be described as a function of thebit-error rate pe and the block-length n. Alternatively therelationship

log10 pb(n) = log10 pe + α log10 n . (2)

is valid. The value α describes the linear dependency betweenlog10 pb(n) and log10 n.




The block-error rate pb(n) can be described (in a digital channel)

as a function of the bit-error rate pe and the block-length n.

Example:

0 0.5 1 1.5 2 2.5 310

−2

10−1

100

PSfrag replacements

(1 − α) = 0.0(1 − α) = 0.3(1 − α) = 0.5

blo

ck-e

rror

rate

→

log10 n →



Error structures with memory

x x - x x x - - x x x x x x - x x - - - - - - - - - - - -- - - - - - - x x x x - x - - x x - - x x - - - - - - - -- - - - - - - - - - - - - - - - - - - - - - - - - - - - -- - - - - - - - - - - - - - - - - - - - - - - - - - - - -- - - - - - - - - - - - - - - - - - - - - - - - - - - - -- - - - - - - - - - - - - - - - - - - - - - - - - - - - -- - - - - - - - - - - - - - - - - - - - - - - - - - - - -- - - - - - - - - - - - - - - - - - - - - - - - - x x x x- - - - - - - - - - - - - - - - - - - - - - - - - - - - -- - - - - - - - - - - - - - - - - - - - - - - - - - - - -- - - - - - - - - - - - - - - - - - - - - - - - - - - - -- - - - - - - - - - - - - - - - - x x - - - - - - - - - -- - - - - - - - - - - - - - - - - - - - - - - - - - - - -- - - - - - - - - - - - - - x x x x - - - - - x - - x x -- - - - - - - - - - - - - - - - - - - - - - - - - - - - -- - - - - - - - - - - - - - - - - - - - - - - - - - - - -- - - - x x x - - - - x x x - - - - - - - - - - - - - - -- - - - - - - - - - - - - - - - - - - - - - - - - - - - -- - - - - - - x x x - - - - - - - - - - - - - - - x x - -- - - - - - - - - - - - - - - - - - - - - - - - - - - - -x x x x x - - - - - - - - - - - x x - - x x x x x x - x -- - - - - - - - - - - - - - - - - - - - - - - - - - - - -- - - x x x x x x - - - - - - - - - - - - - - - - - - - -

Parameter:




Error structures without memory

- - - - - - - - - - - - - - - - x - - - - - - x - - - - x- - - x - - - - x - - - - - - - - - - - x x - - - - - - -- - - - - - - - x - - - - x - - - - - - - - x - - - - - -- - - - - - - - - x x - - - - - - - - - - - - - - - x - -- - - - - - - - - - - - - - - - - x - - - - - - - - - - -- - - - x - - - - - - - - - - - x - - - - - - - - - - - -- - - - - - - - - - x - - - - - - x - - - - - - - - - - -- - - - - - x - - - - - x - - x - - - - - - - - - - - - -- - - - - - - - x - - - - - - x - - - - - - - - - - - - x- - - x - - - - - - - - - - - - - - - - - x x x x x - - -x - - - - - - x - - - - - - - - - - x - - - x - - - - - -- - - - x - - - - - - - - - - - - - - x - - - - - - - - -- x - - - x - - - - - - - - - - - x - x - - - - - - - - -- - - - - - - - x - - - - - - - - - - - - - - - - - - - -- - - - - - - - - - - - - - - - x - - - - - - - - - - - -- - - - - - - - x x - - - - - - - - - - - - - - x - - - -- - - - - - - - - - - - - - - - - - x - - - - - - - - - -- x - - - - - - - - - - - - - x - - x - - - - - - - - - -- - - x - - - - - - - - - - - - - - - - - - - x - - - - -- - - - - - - - - - - x - - - - - - - - - - - x - - - - -- - - - - - - - x - - - - - - - - - - - - - x - - - - - -- x - - - - - - - - - - - - - - - x - - - - - - - - - - -- - - - - - - - x x - - - - - - - - - - - - x - - - - - -

Parameter:




Analysis of error structures (1)

Analysis of error-patterns in an erroneous block of the length n

after the error-length l and error-weight g

Example: Error-pattern in a block of the length n = 9

PSfrag replacements n = 9l = 6, g = 3

bit error

Searched: Probability xl ,g of an error-pattern of the error-length l

and error-weight g in an erroneous block of the length n






12

34

56

7

12

34

56

7

0

0.5

1

PSfrag replacements

error-length l

error-weight g

x l,g

Probability xl ,g with n = 7,(1 − α) = 0.0 and pe = 10−2

(memoryless channel,binomial model)






12

34

56

7

12

34

56

7

0

0.2

0.4

0.6

0.8

PSfrag replacements

error-length l

error-weight g

x l,g


(channel with memory)






12

34

56

7

12

34

56

7

0

0.1

0.2

0.3

0.4

PSfrag replacements

error-length l

error-weight g

x l,g


(channel with memory)



Digital channel models (1)

Relationship between pb(n) and u(k)

pb(n) = pe

n−1∑

k=0

u(k) = pe nα

The searched error-gap distribution u(k) can be calculated step bystep and results in:

n = 1 : u(0) = 1n = 2 : u(0) + u(1) = 2α

n = 3 : u(0) + u(1) + u(2) = 3α

n = 4 : u(0) + u(1) + u(2) + u(3) = 4α

· · · : · · · = · · ·

n : u(0) + u(1) + · · · + u(n − 1) = nα




This can be simplified to:

n = 1 : u(0) = 1n = 2 : u(0) + u(1) = 2α

n = 3 : u(0) + u(1)︸︷︷︸

2α

+u(2) = 3α

n = 4 : u(0) + u(1) + u(2)︸︷︷︸

3α

+u(3) = 4α

The following approximation for the searched error-gap functionu(k) can be derived:

u(k) = (k + 1)α− kα (3)




Error-gap function u(k) = Pr{X ≥ k}

u(k) = (k + 1)α− kα (4)

doesn’t fulfill the boundary condition

limn→∞

pb(n) = 1 = limk→∞

pe

k∑

κ=0

u(κ) .

A solution was presented by Wilhelm (1976):

u(k) = ((k + 1)α− kα) e

−β k (5)

withpe ≈ β

α.



Digital channels with memory – Examples (1)

With v(k) = u(k) − u(k + 1) the error-gap density function resultsin:

v(k) = (1 + e−β) (k + 1)α

e−β k

− kαe−β k + (k + 2)α

e−β (k+1)

.

0 2 4 6 8 100

0.1

0.2

0.3

0.4

PSfrag replacements

v(k

)→

k →

(1 − α) = 0.0

(1 − α) = 0.1

(1 − α) = 0.3



Digital channels with memory – Examples (2)

For the generation of error structures with memory a uniformlydistributed random number Y has to be equated with u(k):

Y ≡ u(k) (6)

and the corresponding value of the error-gap k has to bedetermined.


Channel Coding Baltic Summer School 2006, Vilnius, Lithuania, 31.07.–14.08.2006

Coding gain

2 4 6 8 10

10−4

10−2

PSfrag replacements

uncoded transmissioncoded transmission without decodingcoded transmission with decoding

bit-e

rror

rate

→

10 · lg(Es/N0) (in dB) →



Error probability (1)

For a BSC (binary symmetric channel) the probability that exactlyg errors occur during the transmission of a binary codewordconsisting of n bits is

P(g) = pg

e(1 − pe)

n−g.

There exist(

n

g

)

different vectors of weight g . Probability for any g

errors in a block of the length n is given by (binomial distribution)

(

n

g

)

P(g) =

(

n

g

)

pg

e(1 − pe)

n−g.



Error probability (2)

Error correction of fk errors

Codeword error probability Perror is given by

Perror =n

∑

g=fk+1

(

n

g

)

pg

e (1−pe)n−g = 1−

fk∑

g=0

(

n

g

)

pg

e (1−pe)n−g

.

The codeword error probability Perror gives the probability

that a transmitted codeword does not correspond to the

decoded codeword (false decoding or decoding failure).


Channel Coding Baltic Summer School 2006, Vilnius, Lithuania, 31.07.–14.08. 2006

Interleaving

PSfrag replacements

transmitter side receiver side

readin

readout

read out read in

......

· · · · · ·

error-burst



Code concatenation (1)

PSfrag replacements

outerencoder

innerencoder

innerdecoder

outerdecoder

channel

input output

superchannel

Advantages of concatenated codes (1)

very long codes can be constructed by concatenation based on

short codes

concatenated codes can correct burst errors and independent

single errors



Code concatenation (2)

PSfrag replacements

outerencoder

innerencoder

innerdecoder

outerdecoder

channel

input output

superchannel

Advantages of concatenated codes (2)

decoding complexity can be decreased compared with the

decoding of a single code of the same length, because

decoding of the shorter component codes can be done in

separate stages

utilization of different code characteristics, e. g.

error-detection and error-correction



Examples (1)

All Hamming codes have a minimum Hamming distance of 3.

1. What are the error correction and error detection capabilities

of a Hamming code?

Consider a (3, 1) linear block code.

1. Find all codewords of this code!

2. Find the minimum Hamming distance of the code!

A linear (5, 2) code has the codewords c0 = (0 0 0 0 0),

c1 = (0 1 0 1 1), c2 = (1 0 1 0 1) and c3 = (1 1 1 1 0).

1. Find the set of error patterns corresponding to non-detectable

errors!



Examples (2)

Consider a linear code with a given generator matrix G

G =

0 1 0 1 1 0 0

1 0 1 0 1 0 0

0 1 1 0 0 1 0

1 1 0 0 0 0 1

.

1. Find the following code parameters: n, m, k and dmin!

2. Find all codewords of the code!

3. Assuming an information bit sequence of all 0s, find all

minimum weight error pattern f that result in a valid

codeword that is not the all-zero codeword!



Examples (3)

Consider a (5, 2) code with a given generator matrix G

G =

(

1 0 1 1 0

0 1 1 0 1

)

and parity check matrix H

H =

1 1 1 0 0

1 0 0 1 0

0 1 0 0 1

.

1. Is it a systematic code?

2. Is it a linear code?

3. Is the codeword c = (0 1 1 0 1) a valid codeword?

4. Determine the minimum Hamming distance!



Examples (4)

Consider a (7, 4) Hamming code with generator polynomial

g(x) = x3 + x + 1.

1. Determine if the codewords described by the polynomials

d1(x) = x4 + x3 + x2 + 1 and d2(x) = x4 + x3 + x2 + x + 1

are valid codewords for this generator polynomial.

Consider a (15, 11) Hamming code with generator polynomial

g(x) = x4 + x + 1.

1. Determine if the codewords described by the polynomials

d1(x) = x7 + x3 + x + 1 and d2(x) = x6 + x5 + x3 + 1 are

valid codewords for this generator polynomial.



Examples (5)

Consider a (7, 4) Hamming code with the following generatormatrix G

G =

1 1 0 0 1 0 1

1 0 1 0 0 0 1

1 1 1 0 0 1 0

1 1 0 1 0 0 0

1. Transform the generator matrix G in a systematic form!



Examples (6)

1. What is the basic idea of an error-detection or error-correction

strategy? What is the Hamming distance? Which distances

are necessary for an error-detection or error-correction? Please

illustrate the differences between both strategies!

2. Prove that if the sum of two error patterns is a valid

codeword, then each pattern has the same syndrome.

3. Compute the codeword error probability obtained with a (7, 4)

Hamming code on a binary symmetric channel. Assume

bit-error rates of 10−2 and 10−3!



Examples (7)

An (5, 4) linear block code is constructed by shortening a (7, 4)Hamming code. The (7, 4) Hamming code can be described by thefollowing generator matrix

G =

1 1 0 0 1 0 1

1 0 1 0 0 0 1

1 1 1 0 0 1 0

1 1 0 1 0 0 0

.

1. Construct the codewords of the (5, 4) code and list them!

2. What is the minimum distance of the (5, 4) code!



Examples (8)

1. Please explain the principle of a block interleaver! Indicate the

advantages and the disadvantages of a block interleaver!

2. Illustrate the functionality of a convolutional code using an

example of your choice! Please explain the terms: code tree,

state diagram and trellis diagram!

3. For the trellis and tree diagram of the considered (2, 1, 2)

convolutional code, determine the encoder output sequence

assuming the all-zero initial state (0 0) and an information bit

sequence 0 1 0 1 0.


Channel Coding - uni-rostock.de · A. Ahrens, Institute of Communications Engineering, University...

Documents

Transcript of Channel Coding - uni-rostock.de · A. Ahrens, Institute of Communications Engineering, University...