Channel Coding (Ⅰ)

Copyrights Zhu, Weihong

School of Information Science and Engineering, Shandong University

Principles of the Communications

Channel Coding (Ⅰ)

Linear Block Codes




Basic Theory of Channel Coding

• Channel coding refers to the class of signal transformations designed to improve communications performance by enabling the transmitted signals to better withstand the effects of various channel impairments, such as noise, interference, and fading.

• Channel coding techniques could provide as much as 10dB performance improvement.

• Channel coding can be partitioned into two study areas, waveform (or signal design) coding and structured sequences (or structured redundancy).

• Waveforms coding deals with transforming waveforms into “better waveforms”, to make the detection process less subject to errors.




Basic Theory of Channel Coding

• Structured sequences deals with transforming data sequences into “better sequence”, having structured redundancy (redundant bits). The redundant bits can be used for the detection and correction of errors.

• The encoding procedure provides the coded signal with better distance properties than those of their uncoded counterparts.




Types of Error Control1. Error detection and retransmission.Automatic Repeat Request (ARQ): three types1) stop-and wait ARQ2) Continuous ARQ with pullback3) Continuous ARQ with selective repeatThe major advantage of ARQ over forward error correction is that error

detection requires much simpler decoding equipment and much less redundancy than does error correction.

2. Forward error correction (FEC)Advantages over ARQ: 1) A reverse channel is not available or the

delay with ARQ would be excessive.2) The retransmission strategy is not conveniently implemented3) The expected number of errors, without corrections, would require

excessive retransmissions.




Structured sequences• Channel models1) Discrete Memoryless Channel: discrete input and output2) Binary symmetric Channel

3) Gaussian Channel: discrete input and continuous output• Code Rate and Redundancy: for a block code (n,k), the ratio k/n is

called the code rate. Ratoion (n-k)/n is called redundant rate.

0

1

0

1

1-p

pp

1-p




Repetition Codes• For the block code (n,1) , we could construct the code word as00…00------”0”11…11------”11”Example: n=3, we have 000----”0” ; 111----”1”.Then , at decoding, we choose majority criteria. That is, 000 011001 101010 110 100 111

0 1

In this case, redundant is great.




Linear Block Codes• The encoder transforms a block of k message digits into a longer

block of n codeword digits constructed from a given alphabet of elements. When the alphabet consists of two elements (0 and 1), the code is a binary code.

• The k-bit messages form 2k distinct message sequences, referred to as k-tuples (k元组）。The n-bit blocks can form as many as 2n

distinct sequences, referred to as n-tuples. • Definition: A block code is linear if any linear combination of two

code words is also a code word. In the binary case this requires that if ci and cj are code words, then ci +cj is also a code word, where + denotes component-wise modula-2 addition.

• Definition: The Hamming distance between two code words ci and cj is the number of components at which the two code words differ, and is denoted by d(ci , cj )




Linear Block Codes• Definition: The minimum distance of a code is the minimum

Hamming distance between any two different code words.• Difinition: The Hamminng weight, or the weight of a code word ci is

the number of nonzero components of the code word and is denoted by w(ci ).

• Definition: The minimum weight of a code is the minimum of the weights of the code words except the all-zero code word.

• Theorem: In any linear code, dmin =wmin

• Example: a (6,3) Linear Block Code




Linear Block CodesMessage bits Codeword

000 000000

100 110100

010 011010

110 101110

001 101001

101 011101

011 110011

111 000111

dmin =3. ci +cj is also a codeword




Linear Block Codes• Generator Matrix (生成矩阵）

If k is large, a table lool-up implementation of the encode becomes prohibitive.

In general, we can define a generator metric by the following k×n array:

1 11 12 1

2 21 22 2

1 2

n

n

k k k kn

V v v vV v v v

G

V v v v

⎡ ⎤ ⎡ ⎤⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎢ ⎥= =⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎢ ⎥⎣ ⎦ ⎣ ⎦

If the message bits is m=[m1 ,m2 ,…, mk ]’, then the codeword C is written in matrix notation as

C=mG




Linear Block Codes• The generator matrix of the upper example is

1

2

3

1 1 0 1 0 00 1 1 0 1 01 0 1 0 0 1

VG V

V

⎡ ⎤ ⎡ ⎤⎢ ⎥ ⎢ ⎥= =⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎢ ⎥⎣ ⎦⎣ ⎦

Where V1 , V2 and V3 are three linearly independent vectors. Thus, the code word corresponding to a message bits is a linear combination of the rows of G.• Systematic Linear Block Codes. A systematic (n,k) linear block code is a mapping from a k message digits to an n bits codeword in such a way that part of the sequence generated coincides with the k message digits.




Linear Block Codes• A systematic linear block code will have a generator matrix of the

form11 2 1,( )

21 22 2,( )

1 2 ,( )

1 0 00 1 0

0 0 1

n k

n kk

k k k n k

p p pp p p

G I P

p p p

−

−

−

⎡ ⎤⎢ ⎥⎢ ⎥= ⎡ ⎤ =⎣ ⎦ ⎢ ⎥⎢ ⎥⎣ ⎦

Thus, the cord word C can be written as

1

,

, 1

i

k

ji jj

m i i kc

p m k i n=

≤ ≤⎧⎪= ⎨ + ≤ ≤⎪⎩∑




Linear Block Codes

1

2

3

1 0 0 1 1 00 1 0 0 1 10 0 1 1 0 1

VG V

V

⎡ ⎤ ⎡ ⎤⎢ ⎥ ⎢ ⎥= =⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎢ ⎥⎣ ⎦⎣ ⎦

The systematic generator matrix of the upper example is

Parity-Check Matrix (监督矩阵)Let us define a matrix H, called the parity-check matrix, that will enable us to decode the received vectors.

⎡ ⎤= ⎣ ⎦T

n-kH P I

Where we have TGH = 0

H can be used to test whether a received vector is a valid member of the codeword set. Because C is a codeword generated by G, is and only if CHT=0




Linear Block Codes• Syndrome Testing (伴随式校验)Let r=r1 ,r2 ,…,rn be a received codeword resulting from the transmission

of C=c1 ,c2 ,…cn . We can describe r asr=C+e

Where e=e1 ,e2 , …,en is an error pattern introduced by the channel. There are a total of 2n-1 potential nonzero error pattern in the 2n n- tuples. The syndrome of r is defined as

S=rHT

If S has a value 0, r is a member of the codeword set. If r contains correctable errors, the syndrome has some nonzero value that can earmark the particular error pattern.

S=rHT=(C+e) HT=CHT+eHT=eHT




Linear Block CodesAn important property of linear block codes, fundamental to the

decoding process, is that the mapping between correctable error patterns and syndromes is one to one

For the parity-check matrix1) No column of H can be all zeros, or else an error in the

corresponding codeword position would not affect the syndrome and would be undetectable.

2) All columns of H must be unique. If two columns of H were identical, errors in these two corresponding codeword positions would be indistinguishable.

Example: syndrome test Suppose the codeword C=110101 and the received word is r=110001Find the syndrome vector value S=rHT and verify that it is equal to eHT




Linear Block Codes

1

2

3

100110010011001101

VG V

V

⎡ ⎤ ⎡ ⎤⎢ ⎥ ⎢ ⎥= =⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎢ ⎥⎣ ⎦⎣ ⎦

1 0 1 1 0 01 1 0 0 1 00 1 1 0 0 1

H⎡ ⎤⎢ ⎥= ⎢ ⎥⎢ ⎥⎣ ⎦

110011

101100101

[110001] 110010 [110001] [1,0,0]100

011001010001

T

TS rH

⎡ ⎤⎢ ⎥⎢ ⎥⎡ ⎤⎢ ⎥⎢ ⎥= = = =⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎣ ⎦ ⎢ ⎥⎢ ⎥⎣ ⎦

[100]TS eH= =




Linear Block Codes• Error Correction Syndrome can be used to correct the error since there is a one-to-one

correspondence between correctable error patterns and syndromes.In the hard-decision decoding, we use standard array (标准阵) to

accomplish error correction.Standard array consists of possible received sequences. The first row

contains all the codewords, starting with the all-zeros codeword, and the first column contains all the correctable error patterns. Each row, called a coset (陪集),consists of an error pattern in the first column, called the coset leader (陪集首), followed by the codewords perturbed by that error pattern. The standard array format for an (n.k) code is as follows:




Linear Block Codes1 2 2

2 2 2 2 22

3 2 3 3 32

2 2

22 2 2 2 2

2 2

k

k

k

k

n k n k n k k n k

i

i

in k k

j j i j j

i

C C C Ce C e C e C ee C e C e C e

e C e C e C e

e C e C e C e− − − −

−

+ + ++ + +

×+ + +

+ + +

Note that codeword C1 , the all-zeros codeword, plays two roles. It is one of the code-words, and it can also be thought of as the error pattern e1 --- -the pattern that represents no error, such that r=C.The decoding algorithm calls for replacing a corrupted vector with a valid codeword from the top of the column containing the corrupted vector. Suppose that a codeword Ci is transmitted over a noisy channel, resulting in a received vector Ci +ej . If ej is a coset leader, the received vector will be decoded correctly into the transmitted codeword Ci .




Linear Block Codes• The Syndrome of a CosetEach member of a coset has the same syndrome. The syndrome for

each coset is different from that of any other coset in the code; it is the syndrome that is used to estimate the error pattern.

• Error Correction Decoding1. Calculate the syndrome of r using S=rHT.2. Locate the coset leader (error pattern) ej , whose syndrome equals

rHT

3. This error pattern is assumed to be the corruption caused by the channel.

4. The corrected received vector, or codeword, is identified as U=r+ej .




Linear Block Codes• Example (6,3) standard array.

000000 110100 011010 101110 101001 011101 110011 000111

000001 110101 011011 101111 101000 011100 110010 000110000010 110110 011000 101100 101011 011111 110001 000101000100 110000 011110 101010 101101 011001 110111 000011001000 111100 010010 100110 100001 010101 111011 001111 010000 100100 001010 111110 111001 001101 100011 010111100000 010100 111010 001110 001001 111101 010011 100111010001 100101 001011 111111 111000 001100 100010 010110




Linear Block CodesError pattern Syndrome

000000 000000001 101000010 011000100 110001000 001010000 010100000 100010001 111

Example error correctionAssume that codeword C=101110, r=001110. Show how a decoder, using the

upper table to correct the error.Solution: The syndrome of r is computed: S=[001110]HT=[100]e’=100000. The corrected vector is estimated by

U’=r+e’=001110+1000000=101110=C




Error-detection and Correction Capability

• It should be clear that not all error patterns can be correctly decoded.• For a linear block code set with minimum distance dmin , the error-

correcting capability t is

min 12

dt −⎡ ⎤= ⎢ ⎥⎣ ⎦The error detecting capability e is

min 1e d= −




Cyclic Codes (循环码)• Definition. A cyclic code is a linear block code with the extra condition

that if c is a code word, a cyclic shift of it is also a code word.• Example : the code set {000,110,101, 011} is a cyclic codes.• The structure of cyclic codes.Each cyclic code could be represented by a polynomial as follows:The code word is c=(c1 ,c2 ,…,cn-1 , cn ) , the corresponding polynomial is

1 21 2 1

1

( ) ...n

n i n ni n n

i

c p c p c p c p c p c− − −−

=

= = + + + +∑The codeword polynomial of c(1)=(c2 ,c3 ,…cn-1 ,cn ,c1 ) is

(1) 1 2 22 3 1 1

1

( ) ...

( ) ( 1)

n nn n

n

c p c p c p c p c p c

pc p c p

− −−= + + + + +

= + +




Cyclic Codes (循环码)• In the binary field addition and subtraction are equivalent. Thus

(1)1

(1)

( ) ( ) ( 1)

( ) ( ) (mod( 1))

n

n

pc p c p c p

or c p pc p p

= + +

= +In general, for i shifts we have the code word polynomial

( ) ( ) ( ) (mod( 1))i i nc p p c p p= +Where a(p)=b(p) (mod d(p) ) means that a(p) and b(p) have the same remainder when divided by d(p).Example: c=1101, for n=4. express the codeword in polynomial form, and using upper equation, solve for the third end-around shift of the codeword.Solution : c(p)=p3+p2+1; c(3)p=p3c(p) (mod p4+1)=p6+p5+p3 (mod p4+1)=p3+p2+p. Thus, the codeword is (1110). (right shift 3).




Cyclic Codes (循环码)• Binary Cyclic Code PropertiesWe can generate a cyclic code using a generator polynomial . The

generator polynomial g(p) for an (n,k) cyclic code is unique and is of the form

12( ) ... 1n k n k

n kg p p g p g p− − −−= + + + +

If the information (message) sequence polynomial X(p) is defined by1 2

1 2 1( ) ...k kk kX p x p x p x p x− −−= + + + +

The code word polynomial corresponding to x is given by c(p)=X(p)g(p).A generator polynomial g(p) of an (n,k) cyclic code is a factor of pn+1; that is , pn+1=g(p)h(p). For example,

p7+1=(p3+p+1)(p4+p2+p+1)Using g(p)=(p3+p+1) as the generator polynomial of degree n-k=3, we can generate an (n,k)=(7,4) cyclic code. Or, using g(p)=(p4+p2+p+1), we can generate an (n,k)=(7,3) cyclic code.




Cyclic Codes (循环码)• Encoding in Systematic FormSince the message vector in polynomial form is as follows:

1 21 2 1( ) ...k k

k kX p x p x p x p x− −−= + + + +

In systematic form, the message digits are utilized as part of the codeword. So we multiply X(p) by pn-k, then we divide it by g(p), the remainder is the parity polynomial. That is

( ) ( ) ( ) ( )( ) ( ) mod lo g( )

n k

n k

p X p q p g p p por p p p X p u p

−

−

= +

=

Example : X=1101, g(p)= p3+p+1, generate a systematic (7,4) codeword .Solution: X(p)=p3+p2+1, n=7, k=4, n-k=3;pn-kX(p)=p3(p3+p2+1)=p6+p5+p3, divided it by g(p), we cam writep6+p5+p3=(p3+p2+p+1)(p3+p+1)+1. So the codeword is 1101001.




Cyclic Codes (循环码)and other well-known block codes

• Encoding of Cyclic Codes: The cyclic encoders can be implemented by shift registers.(移位寄存器)。

1. Hamming Codes. Hamming codes are a simple class of block codes characterized by the structure (n,k)=(2m-1,2m-1-m), where m=2,3,…. These codes have a minimum distance of 3. Syndrome decoding is especially suited for Hamming codes.

2. BCH Codes. BCH codes are a generalization of Hamming codes that allow multiple error correction. They are powerful class of cyclic codes that provide a large selection of block lengths, code rates, alphabet sizes, and error correcting capability.

3. Reed-Solomon Codes. RS codes are a subset of BCH codes and , therefore, belong to the family of cyclic codes. They are non- binary codes and useful in situation where errors tend to happen in ‘bursts’ rather than randomly.

Channel Coding (Ⅰ)

Documents

Transcript of Channel Coding (Ⅰ)