Channel Coding I + IIkom.aau.dk/group/05gr943/literature/turbo/kc1_kap1.pdf · 2005-12-02 · Dirk...

Channel Coding I + IIChannel Coding I + II

Dirk WübbenDepartment of Communications Engineering

Room: N 2380, Phone: 0421/218-2545Email: [email protected]

LectureTuesday, 08:30 – 10:00 in N2420

Wednesday, 14:00 – 16:00 in N2250Dates for exercises will be announced

during lectures.

http://www.ant.uni-bremen.de/teaching/kc

TutorRonald BöhnkeRoom: N2380

Phone [email protected]

Dirk Wübben Channel Coding I 2UniversitätBremen

PreliminariesPreliminaries

� Master students: � Channel Coding I and Channel Coding II are elective courses� Written examination at the end of each semester

� Diplomstudenten� Kanalcodierung als (Wahl)pflichtfach, mündliche Prüfung� Wahlweise dreistündig (Prüfung nach 1 Semester)

oder sechsstündig (Prüfung nach 2 Semestern)

� Documents� Script Kanalcodierung I/II by Volker Kühn (in German), these slides and tasks for

exercises are available in the internet http://www.ant.uni-bremen.de/teaching/kc

� Exercises� Take place on Wednesday, 14:00-16:00 in Room N2250� Dates will be arranged in the lesson and announced by mailling list

[email protected]� Contain theoretical analysis and tasks to be solved in Matlab


LiteratureLiterature

� Books on Channel Coding� B. Friedrichs, Kanalcodierung, Springer Verlag, 1996 � M. Bossert, Kanalcodierung, B.G. Teubner Verlag, 1998, � M. Bossert, Channel Coding for Telecommunications, Wiley 1999� R. E. Blahut, Algebraic Codes for Data Transmission, Cambridge University Press, 2003 � S. Lin, D. J. Costello, Jr., Error Control Coding: Fundamentals and Applications, Prentice- Hall, 2004 � W.C. Huffman, V. Pless, Fundamentals of Error-Correcting Codes, Cambridge University Press, 2003 � S. B. Wicker, Error Control Systems for Digital Communications and Storage, Prentice-Hall, 1995 � J. Huber, Trelliscodierung, Springer Verlag, 1992

� Books on Information Theory� T. M. Cover, J. A. Thomas, Information Theory, Wiley, 1991� R. G. Gallager, Information Theory and Reliable Communication, Wiley, 1968 � R. Johannesson, Informationstheorie - Grundlagen der (Tele-)Kommunikation, Addison-Wesley, 1992

� General Books on Digital Communication� K. D. Kammeyer, Nachrichtenübertragung, B.G. Teubner, 2004 � K.D. Kammeyer, V. Kühn, MATLAB in der Nachrichtentechnik, Schlembach, 2001� J. Proakis, Digital Communications, McGraw-Hill, 2001 � B. Sklar, Digital Communications, Fundamentals and Applications, Prentice-Hall, 2003

� Internet Resources


Outline Channel Coding IOutline Channel Coding I

� Introduction� Declarations and definitions, general principle of channel coding� Structure of digital communication systems

� One Lesson of Information Theory� Probabilities, measure of information� SHANNON‘s channel capacity for different channels

� Linear block codes� Properties of block codes and general decoding principles� Bounds on error rate performance� Representation of block codes with generator and parity check matrices� Cyclic block codes (Reed-Solomon and BCH codes)

� Convolutional Codes� Structure, algebraic and graphical presentation� Distance properties and error rate performance� Optimal decoding with Viterbi algorithm


Outline Channel Coding IIOutline Channel Coding II

� Trelliscoded Modulation (TCM)� Motivation by information theory� TCM of Ungerböck, pragmatic approach by Viterbi, Multilevel codes� Distance properties and error rate performance� Applications (data transmission via modems)

� Concatenated Codes� Serial Concatenation� Parallel Concatenation (Turbo Codes)� Iterative Decoding with Soft-In/Soft-Out decoding algorithms

� Adaptive Error Control� Automatic Repeat Request (ARQ)� Performance for perfect and disturbed feedback channel� Hybrid FEC/ARQ schemes


IntroductionIntroduction

� Basics about Channel Coding� General declarations� Basic principles� Applications of Channel Coding� Declarations and definitions, general principle of channel coding

� Structure of digital communication systems

� Discrete Channel � Statistical description� AWGN and fading channel� Discrete Memoryless Channel (DMC)� Binary Symmetric Channel (BSC and BSEC)


General DeclarationsGeneral Declarations

� Important terms:� Message Amount of transmitted data or symbols by the source� Information Part of message, which is new for the sink� Redundancy Difference of message and information, which is

unknown to the sinkMessage = Information + Redundancy

� Irrelevance Information, which is not of importance to the sink� Equivocation Information, not stemming from sink of interest

� Message is also transmitted in a distinct amount of time� Messageflow Amount of message per time� Informationflow Amount of information per time� Transinformation Amount of error-free information per time transmitted

from the source to the sink


Three Main Areas of CodingThree Main Areas of Coding

� Source coding (entropy coding)� Compression of the information stream so that no significant information is lost,

enabling a perfect reconstruction of the information. � Thus, by eliminating superfluous and uncontrolled redundancy the load on the

transmission system is reduced. � Entropy defines the minimum amount of necessary information

� Channel coding (error-control coding)� Encoder adds redundancy (additional bits) to information bits in order to detect or

even correct transmission errors

� Cryptography� The information is encrypted to make it unreadable to unauthorized persons or to

avoid falsification or deception during transmission.� Decryption is only possible by knowing the encryption key

� Claude E. Shannon (1948): A Mathematical Theory of Communication


Basic PrincipleBasic Principless of Channel Codingof Channel Coding

� Forward Error Correction (FEC)� Added redundancy is used to correct transmission errors at the receiver� Channel condition affects the quality of data transmission � errors after

decoding occur if the error-correction capability of the code is passed� No feedback channel is required� varying reliability, constant bit throughput

� Automatic Repeat Request (ARQ)� Small amount of redundancy is added to detect transmission errors

� retransmission of data in case of a detected error� feedback channel is required

� Channel condition affects the throughput� constant reliability, but varying throughput

� Hybrid FEC/ARQ


Basic Idea of Channel CodingBasic Idea of Channel Coding

� Add redundancy to the information data in order to detect and/orcorrect transmission errors at the receiver

� Channel Coding � Create a vector x of length n>k for an information vector u of length k� In case of a binary alphabet 2n different vectors x exist.

Due to the bijective mapping from u to x only 2k<2n vectors are used.� Find a k-dimensional subset out of an n-dimensional space so that

• Minimum distance between elements within the subset is maximized� effects probability of detecting errors

• Cardinality of subset is as large as possible� Code rate equals the rate of length of the uncoded and the

coded sequence and describes the required expansion of the signalbandwidth

c

kR

n=

• Code is a subset

• Coder is the mapper


Visualizing Distance Properties with Code CubeVisualizing Distance Properties with Code Cube

� Code rate Rc = 1

� No error correction

� No error detection

� Code rate Rc = 2/3

� No error correction

� Detection of single error

� Code rate Rc = 1/3

� Correction of single error

� Detection of 2 errors

010 011

100

110 111

101

001000

dmin=1

010 011

100

110 111

101

001000

dmin=2

010 011

100

110 111

101

001000

dmin=3


Applications of Channel CodingApplications of Channel Coding

� Importance of channel coding increased with digital communications� First use for deep space communications:

� AWGN channel, no bandwidth restrictions, only few receivers (costs negligible)

� Examples:Viking (Mars), Voyager (Jupiter, Saturn), Galileo (Jupiter), ...

� Mass storage � compact disc, digital versatile disc, magnetic tapes

� Digital wireless communications: � GSM, UMTS, WLAN (Hiperlan, IEEE 802.11), ...

� Digital wired communications� Modem transmission (V.90, ...)

� Digital broadcasting � DAB, DVB


digital source

• Source transmits signal d(t) (e.g. analog speech signal)

• Source coding samples, quantizes and compresses analog signal

• Digital Source: comprises analog source and source coding, delivers

digital data vector u = (u0 u1 … uk-1) of length k

Structure of Digital Transmission SystemStructure of Digital Transmission System

uanalogsourceanalogsource

sourceencodersource

encoder


• Channel encoder adds redundancy to u so that errors in

x = (x0 x1 … xn-1) can be detected or even corrected

• Channel encoder may consist of several constituent codes

• Code rate: Rc = k / n



sourceencodersource

encoder

digital source

xchannelencoderchannelencoder


discrete channel

• Modulator maps discrete vector x onto analog waveform and moves it into the transmission band

• Physical channel represents transmission medium– Multipath propagation → intersymbol interference (ISI)– Time varying fading, i.e. deep fades in complex

envelope– Additive noise

• Demodulator: Moves signal back into baseband and performs lowpass filtering, sampling, quantization


• Discrete channel: comprises analog part of modulator, physical channel and analog part of demodulator


sourceencodersource

encoder

digital source


y

modulatormodulator

physicalchannelphysicalchannel

demodulatordemodulator


y

discrete channelChannel decoder: • Estimation of u on the basis of received vector y• y need not to consist of hard quantized values (0,1)• Since encoder may consist of several parts,decoder may also consist of several modules



sourceencodersource

encoder

digital source


uchanneldecoderchanneldecoder

modulatormodulator




y

discrete channel


feedback channel


sourceencodersource

encoder

digital source


sinksink sourcedecodersource

decoder

uchanneldecoderchanneldecoder

modulatormodulator



Citation of Massey: “The purpose of the modulation system is to create a good discrete channel from the modulator input to the demodulator output, and the purpose of the coding system is to transmit the information bits reliably through this discrete channel at the highest practicable rate.”


Discrete ChannelDiscrete Channel

� Discrete channel comprises analog parts of modulator and demodulator as well as physical transmission medium

� Discrete input alphabet Ain={X0, ..., X|Ain|-1}

� Discrete output alphabet Aout={Y0, ..., Y|Aout|-1}

� Probabilities: Pr{Xν}, Pr{Yµ}

� Joint probability of event Xν,Yµ: Pr{Xν,Yµ}

� Transition probabilities: Pr{Yµ | Xν}

discretechannel

xi∈Ain yi∈Aout

{ } { } { }{ } { }

Pr , Pr Pr

Pr Pr

X Y Y X X

X Y Y

ν µ µ ν ν

ν µ µ

=

=



� Description of discrete channel by transition diagram

� Restrictions in this lecture� Binary input alphabet (BPSK): Ain={-1,+1}

� Output alphabet depends on quantization• No quantization Aout=R

• Hard decision (1-bit) Aout=Ain

Y1X1

X|Ain|-1

Pr{Y0 | X0} Y0X0

Y|Aout|-1

Pr{Y1 | X0}



� A-posteriori probabilities:

� For statistical independent elements

� General probability relations

{ } { }Pr Pr 1in outX Y

X Yν µ

ν µ∈ ∈

= =∑ ∑A A

{ }Pr , 1in outX Y

X Yν µ

ν µ∈ ∈

=∑ ∑A A

{ } { } { } { }Pr Pr , Pr Pr ,in outX Y

Y X Y X X Yν µ

µ ν µ ν ν µ∈ ∈

= =∑ ∑A A

{ } { } { }Pr , Pr PrX Y X Y= ⋅ν µ ν µ

{ } { }Pr Pr , jj

a a b=∑{ }Pr 1i

i

a =∑

{ }Pr , 1i ji j

a b =∑∑

{ } { }{ }

Pr ,Pr

Pr

X YX Y

Y

ν µν µ

µ

=

{ } { }{ }

{ } { }{ } { }

Pr , Pr PrPr Pr

Pr Pr

X Y X YX Y X

Y Y

⋅= = =ν µ ν µ

ν µ νµ µ


BayesBayes RuleRule

� Bayes Rule

� Attention

but

{ } { }{ } { } { }

{ }Pr , Pr

Pr PrPr Pr

a b bb a a b

a a= = ⋅ { } { } { }

{ }Pr

Pr PrPr

YY X X Y

X= ⋅ µ

µ ν ν µν

{ } { }{ }

Pr ,Pr 1

Prout outY Y

X YX Y

Yµ µ

ν µν µ

µ∈ ∈

= ≠∑ ∑A A

{ } { }{ }

{ }{ }

Pr , PrPr 1

Pr Prin inX X

Y X YX Y

Y Yν ν

µ ν µν µ

µ µ∈ ∈

= = =∑ ∑A A

{ }Pr 1outY

Y Xµ

µ ν∈

=∑A

and

�Example


BasebandBaseband TransmissionTransmission

� Time-continuous, band-limited signal x(t)∈R

� Average Power per period

� Average Energy of each (real) transmit symbol

� Noise with spectral density (of real noise)� Power

� Signal to Noise Ratio (after matched filtering)

ix

( )n t

iy( )Tg t

( )x t

( ) ( )i si

x t x g t iT= ⋅ −∑

{ }2

s sE E TX µ= ⋅

{ }22 2 /X s s sBE E T E X µσ = = =

( )Rg t

f

0 / 2N

2B

sE 2Xσ,XX NNΦ Φ

channelchannel

2Nσ

( )20 0 2N sN B N Tσ = =

0 / 2NN NΦ =

siT

2

20

/2

sX

N

ES N

N

σσ

= =

1 2A sf T B= =Sampling


BandpassBandpass TransmissionTransmission

� Transmit real part of complex signal shifted to a carrier frequency f0

� With x(t)∈C the complex envelope of the data signal

� Re{} results in two spetra around -f0 and f0� Received signal� Transformation to basis band� Analytical signal achieved by Hilbert transformation (suppress f<0)

� Low pass signal

( )y t+

ix

( )n tɶ

iy( )Tg t

( )x tBP

1√2 · e

−jω0t√2 · ejω0t

{ }Re i

( )x t

( ) ( ){ } ( ) ( ) ( ) ( )0BP 0 02 Re 2 ' cos '' sinj tx t ex t x t t x t tω ω ω= = −

channelchannel

{ }j iH

( )Rg t

( ) ( ) ( ){ }BP BP BPy t y t j y t+ = + ⋅H

( ) ( ) 01TP BP2

j ty t y t e ω−+=

BP BP BP BP( ) ( ) ( ) ( )y t x t h t n t= ∗ +

xBP(t)∈R

doubles spectrum for f >0

( )y tBP


Equivalent Equivalent BasebandBaseband RepresentationRepresentation

� Equivalent received signal

� In this semester, only real signals are investigated (BPSK)� Signal is only given in the real part� Only the real part of noise is of interest

( ) ( ) ( ) ( ) ( ) ( ) ( )0BP BP BP

1

2j ty t e x t h t n tx t h t n t ω+ −= = ∗ + ∗ +

f

0 2N

B

2rs sE E=

212 Xσ

,BP BP BP BPX X N NΦ Φ

212 Nσ

B

-f0 f0

2

20 0

2 2/

2 2s sX

N

B E ES N

B N N

σσ

= = =

f

0 / 2N

B

sE 2Xσ,XX NNΦ Φ

2Nσ

2

20

/2

sX

N

ES N

N

σσ

= =


Probability Density Functions for AWGN ChannelProbability Density Functions for AWGN Channel

-4 -2 0 2 40

0.2

0.4

0.6

0.8

ddd

signal-to-noise-ratio Es/N0 = 2 dB

( )1|| −=ξϑxyp ( )1|| +=ξϑxyp

ϑ

( )ϑyp

-4 -2 0 2 40

0.2

0.4

0.6

0.8

ddd

signal-to-noise-ratio Es/N0 = 6 dB

ϑ

( )1|| −=ξϑxyp ( )1|| +=ξϑxyp

( )ϑyp

2

| 22

( )1( | ) exp

22y x

NN

y Xp y X ν

ν σπσ −= ⋅ − xi

ni

yi


Error Probability of AWGN Channel Error Probability of AWGN Channel

� Signal-to-Noise-Ratio S/N

� Error Probability for antipodal modulation

{ } ( ) ( )

{ } ( ) ( )

( )

|

0 0

0 0

|

2

220

Pr error 1 |

Pr error 1 |

1exp

22

e i y x s s n s s

i y x s s n s s

s s

NN

P x p E T d p E T d

x p E T d p E T d

E Td

∞ ∞

−∞ −∞

∞

= = − = = − = +

= = + = = = −

+ = ⋅ −

∫ ∫∫ ∫

∫

ϑ ξ ϑ η η

ϑ ξ ϑ η η

ηη

σπσ

0 02 2s s s

s

E T ES

N N T N= =

Es : symbol energyN0/2: noise density

( ),s s s sE T E T+ −


Error Probability of AWGN ChannelError Probability of AWGN Channel

� With the error probability becomes

� Using the substitution

or

( )2

000

1exp

s s

ess

E TP d

N TN T

ηη

π

∞ + = ⋅ − ∫

( ) 0s s sE T N Tξ η= +

20 2N sN N Tσ= =

2

00

1 1e erfc

2s

se

E N

EP d

N

∞−

= ⋅ = ∫ ξ ξπ

with error function complement

( ) 0 2s s sE T N Tξ η= +2

0

2

02

21e Q

2s

se

E N

EP d

N

ξ

ξπ

∞−

= ⋅ = ∫ ( )2

21Q

2e d

η

α

α ηπ

∞−

= ∫

( ) ( ) η

α

α α ηπ

∞−= − = ∫ 22

erfc 1 erf e d

with


Probability Density Functions for Frequency Probability Density Functions for Frequency Nonselective Rayleigh Fading ChannelNonselective Rayleigh Fading Channel

xi

ni

ri yi

αi

( ) ( )2 2 22 exp for 0

0 else

s spα

ξ σ ξ σ ξξ ⋅ ≥=

ir-4 -2 0 2 40

0.02

0.04

0.06

0.08

0.1

iα0 2 4

0

0.02

0.04

0.06

0.08

0.1

iy-4 -2 0 2 40

0.02

0.04

0.06

0.08

0.1


Bit Error Rates for AWGN and Flat Rayleigh Bit Error Rates for AWGN and Flat Rayleigh Channel (BPSK modulation)Channel (BPSK modulation)

0 10 20 3010

-6

10-4

10-2

100

17 dB

AWGN Rayleigh

Pb

� AWGN channel:

� Flat Rayleigh fading channel:

� Channel coding for fading channels essential(time diversity)

( )( )

0

0

1erfc /

2

2 /

b s

s

P E N

Q E N

= ⋅

=

0

0

/11

2 1 /s

bs

E NP

E N

= ⋅ − +

Es / N0 in dB


� If line-of-sight connection exist for α, its real part is non-central Gaussian distributed� Rice factor K determines power ratio between line-of-sight path and scattered paths� K = 0 : Rayleigh fading channel� K → ∞ : AWGN channel

(no fading)

� Coefficient α[k] has Rayleigh distributed magnitude with average power 1� Relation between total average power and variance:

� Magnitude is Rician distributed

Rice Fading ChannelRice Fading Channel

2 2/02 2

| |

22 0

( )

0

K Ke I

pα−ξ σ −

α α α

ξ ⋅ ⋅ ξ ξ ≥ ξ = σ σ

for

else

1

1 1'[ ]K

i K Ka k

+ += + ⋅α

1

K

K+

'α

in

1

1 K+

iyix

2(1 ) HP K= + ⋅ σ


Discrete Memoryless Channel (DMC)Discrete Memoryless Channel (DMC)

� Transition probabilities of a discrete memoryless channel (DMC)

� The probability, that exactly m errors occur at distinct positions in a sequence of n is given by

� Probability, that in a sequence of length n exactly m errors occur (at any place)

With giving the number of possibilities to choose melements out of n different elements, withoutregarding the succession (combinations)

{ } { } { }1

0 1 1 0 1 10

Pr Pr , , , , , , Prn

n n i ii

y y y x x x y x−

− −=

= … … = ∏y x

{ } ( )Pr bit of incorrect 1n mm

e em n P P−= ⋅ −

{ } ( )Pr error in a sequence of length 1n mm

e e

nm n P P

m−

= ⋅ ⋅ −

( )!

! !

n n

m m n m

= ⋅ −


Binary Symmetric Channel (BSC and BSEC)Binary Symmetric Channel (BSC and BSEC)

� Binary Symmetric Channel (BSC)

� Binary Symmetric Erasure Channel (BSEC)

{ } { }0 1 1 00

1Pr Pr erfc

2s

e

EY X Y X P

N

= = =

{ } { }

{ } ( )0 0 1 1 1 1

1

0

Pr Pr , , ,

Pr 1

n n

nn

i i ei

x y x y x y

x y P

− −

−

=

= = = = … =

= = −∏

x y

1-Pe

1-Pe

Pe

Pe

Y0

Y1

X0

X1

1-Pe-Pq

1-Pe-Pq

Pe

Pe

Y0

Y1

X0

X1

Y2

Pq

Pq

{ }1

Pr ?e q

q

e

P P y x

x y P y

P y x

− − == = ≠

{ } { } ( )Pr 1 Pr 1 1n

e eP n P≠ = − = = − − ≈x y x y

Channel Coding I + IIkom.aau.dk/group/05gr943/literature/turbo/kc1_kap1.pdf · 2005-12-02 · Dirk...

Documents

Transcript of Channel Coding I + IIkom.aau.dk/group/05gr943/literature/turbo/kc1_kap1.pdf · 2005-12-02 · Dirk...