exercise in the previous class (1)

Define (one of) (15, 11) Hamming code:

construct a parity check matrix, and

determine the corresponding generator matrix

1

100011010011101

010010101011011

001001100110111

000100011101111

1100

1010

0110

1001

0101

0011

1110

1101

1011

0111

1111

I

generator matrix

parity check matrix

m = 4

24 – 1 nonzero vectors

exercise in the previous class (2)

Let C be a linear code with the following parity check matrix. Show that the minimum distance of C is 4.

2

.

10001110

01001101

00101011

00010111

H

weight 1: HuT = hi = 0...contradiction

weight 2: HuT = hi + hj= 0 ...contradiction

weight 3: HuT = hi + hi + hl = 0, but...

column vectors have odd weight, and hi + hi has even weight

hi + hi = hl never happens...contradiction

weight 4: we have 10001110C

previous class

Hamming code

one-bit error correcting, one of perfect codes

minimum Hamming distance

smallest distance between two codewords

gives the maximum number of correctable errors

easy but “rough” estimation of performance

3

today’s class

two independent topics:

performance evaluation using weight distribution

slightly complicated (ちょっと複雑)

precise and detailed estimation is possible

cyclic code

a subclass of linear codes

advantages in realizations

4

is the minimum distance everything?

If the minimum Hamming distance of C is dmin,

then C can correct (dmin – 1) / 2 or less bit errors.

5

C1 C2 C3

< = ?

Consider three codes with the same code length...

many neighbors small # of neighbors

dmin dmin dmin

C2 seems better than C3...

1 codeword at distance 0 4 codewords at distance 3 3 codewords at distance 4

1 codeword with weight 0 4 codewords with weight 3 3 codewords with weight 4

weight distribution

6

C={000000,100111,010110,110001, 001011,101100,011101,111010}.

weight distribution

in linear codes, all codewords have the same distance distribution

3

4 000000

distance distribution from 000000

from the view of 000000,

performance evaluation with weight distribution

assume...

BSC with bit error probability p

one-bit error correcting

send all-zero codeword and evaluate the probabilities of...

correctly decoded

wrongly decoded

error detected

7

C={000000,100111,010110,110001, 001011,101100,011101,111010}.

r 000000 decoder ?

received vector

1 codeword with weight 0 4 codewords with weight 3 3 codewords with weight 4

the domain of the received vector

the received vector rV6

V6 contains eight decoding territories

8

C={000000,100111,010110,110001, 001011,101100,011101,111010}.

correctly decoded;

if r falls in the territory of 000000

wrongly decoded;

if r falls in the territory of the other codewords

error detected; if r does not fall in the territories

(not all vectors shown)

probability of correct decoding

correctly decoded if r belongs to

T = {000000, 000001, 000010, 000100, 001000, 010000, 100000}

9

r is decoded correctly with probability Pc = (1 – p)6 + 6p(1-p)5

prob. that rT and wH(r) = 0

prob. that rT and wH(r) = 1

(1 – p)6

6p( 1 – p)5

(1 – p)6

p( 1 – p)5

probability of wrong decoding (1)

case1: r belongs to the territory of a weight-3 codeword 001011

T = {001011, 001010, 001001, 001111, 000011, 011011, 101011}

10

prob. that rT and wH(r) = 3

prob. that rT and wH(r) = 2

p3(1 – p)3

3p2( 1 – p)4

prob. that rT and wH(r) = 4 3p4( 1 – p)2

r is decoded to the wrong codeword 001011 with probability 3p2(1 – p)4 + p3(1 – p)3 + 3p4(1 – p)2

... the same discussion holds for other codewords with weight 3

probability of wrong decoding (2)

case2: r belongs to the territory of a weight-4 codeword 100111

T = {100111, 100110, 100101, 100011, 101111, 110111, 000111}

11

prob. that rT and wH(r) = 4

prob. that rT and wH(r) = 3

p4(1 – p)2

4p3( 1 – p)3

prob. that rT and wH(r) = 5 2p5( 1 – p)

r is decoded to the wrong codeword 100111 with probability 4p3(1 – p)3 + p4(1 – p)2 + 2p5(1 – p)

... the same discussion holds for other codewords with weight 4

probability of wrong decoding (3)

wrongly decoded;

Pe = 4×(falls in the territory of one of weight-3 codewords)

+ 3×(falls in the territory of one of weight-4 codewors)

= 4×(3p2(1 – p)4 + p3(1 – p)3 + 3p4(1 – p)2 )

+ 3×(4p3(1 – p)3 + p4(1 – p)2 + 2p5(1 – p))

= 12p2(1 – p)4 + 16p3(1 – p)3 + 15p4(1 – p)2 + 6p5(1 – p)

error detected with probability Pd = 1 – Pc – Pe

12

C={000000,100111,010110,110001, 001011,101100,011101,111010}.

1 codeword with weight 0 4 codewords with weight 3 3 codewords with weight 4

the probabilities in a graph

13

00.10.20.30.40.50.60.70.80.9

1

0 0.1 0.2 0.3 0.4 0.5

pro

bab

iliti

es

bit error probability p

Pc

Pe

Pd

comparison example

(7,4) Hamming code: 0000000, 1000101, 0100111, 0010110,

0010110, 1010011, 0110001, 1110100,

0001011, 1001110, 0101100, 1101001,

0011101, 1011000, 0111010, 1111111

(9, 4) 2D code: 000000000, 000101011, 001001101, 001100110,

010010011, 010111000, 011011110, 011110101,

100010101, 100111110, 101011000, 101110011,

110000110, 110101101, 111001011, 111100000

14

weight 0 3 4 7

#codewords 1 7 7 1

weight 0 4 6

#codewords 1 9 6

comparison of two codes

15

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

0 0.1 0.2 0.3 0.4 0.5

Hamm, Pc

Hamm, Pe

Hamm, Pd

2D, Pc

2D, Pe

2D, Pd

Hamming code seems better, but...

“retransmission”

re-transmission of broken data is allowed in some systems

16

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

0 0.1 0.2 0.3 0.4 0.5

Hamm, Pc

Hamm, Pe

Hamm, Pd

2D, Pc

2D, Pe

2D, Pd

2D, Pc (retrans)

2D code + one-time re-transmission is better for p ≤ 0.28.

half the way...

two independent topics:

performance evaluation using weight distribution

slightly complicated (ちょっと複雑)

precise and detailed estimation is possible

cyclic code

a subclass of linear codes

advantages in realizations

17

cyclic codes

Linear codes are characterized by matrix operations.

Encoders/decoders are realized by combinatorial circuits.

simple but not scalable ... O(n2) for the code length n

cyclic codes (巡回符号)

a subclass of linear codes

Encoders/decoders are realized by shift registers.

complex but scalable ... O(n) complexity

18

codes linear codes

cyclic codes

preliminary (1)

We represent binary vectors by binary polynomials (多項式).

19

11101 x4 + x3 + x2 + x + 1

addition of polynomials: (subtraction)

x4 + x3 + x2 + 1 x3 + x + 1 +)

x4 + x2 + x

11101 01011

10110

+)

multiplication of polynomials:

x4 + x3 + x2 + 1 x3 + x + 1 ×)

x4 + x3 + x2 + 1 x5 + x4 + x3 + x

x7 + x6 + x5 + x3

x7 + x6 + x3 + x2+ x + 1 11001111

11101 01011 ×)

11101 11101

11101 multiply by xm

= left-shift of m bits

preliminary (2)

20

division of polynomials:

x6 + x4 x4 + x3 + x2 + 1 )

x2 + x + 1

x6 + x5 + x4 + x2

x5 + x2 x5 + x4 + x3 + x

x4 + x3 + x2 + x x4 + x3 + x2 + 1

x + 1

11101 ) 1010000 11101

10010 11101

11110 11101

11

111

addition = subtraction = XOR of coefficients

the division circuit is easily implemented by a shift register

division circuit (1)

divide p(x) = x6 + x4 by q(x) = x4 + x3 + x2 + 1...

21

1 1 1 0 1

1 0 1 0 0 0 0

q(x)

p(x)

quotient (商) remainder (剰余)

1. store p(x) to the registers 2. if MSB = 1, then AND gates are activated,

and registers are XOR’ed with q(x) 3. left-shift, and go to step 1

division circuit (2)

22

0 1 0 0 1

1 1 1 0 1

0 0

11101 ) 1010000 11101

010010 11101

11110 11101

11

111

1 0 1 0 0

1 1 1 0 1

0 0

one-cycle of operations

= one-step of the calculus

contents of the register

= remainder of the division

definition of cyclic codes

How to construct an (n, k) cyclic code:

length n, k information symbols, m = n – k parity symbols

Step 1: choose an order-m polynomial G(x) which divides xn + 1.

Step 2: C = {multiples of G(x) with order < n}

Example: n = 7, m = 4:

G(x) = x4 + x3 + x2 + 1 divides x7+1,

because x7+1 = (x3 + x2 + 1) (x4 + x3 + x2 + 1).

G(x)...generator polynomial

23

10000001 11101 )

11101 1101001

11101

11101 11101

0

1101

G(x) x7+1

construction of a cyclic code

Step 1: choose an order-m polynomial G(x) which divides xn + 1.

Step 2: C = {multiples of G(x) with order < n}

n = 7, m = 4: G(x) = x4 + x3 + x2 + 1

24

G(x) (x2+x+0)×

C=

0000000 0011101 0111010 0100111

1110100 1101001 1001110 1010011

G(x) (x2+x+1)× G(x) (x2+x+1)× G(x) (x2+x+1)× G(x) (x2+x+1)× G(x) (x2+x+1)× G(x) (x2+x+1)× G(x) (x2+x+1)×

=

properties of cyclic codes (1)

Lemma: a cyclic code is a linear code

proof: show that, for c1, c2C, we have c1+c2C

c1 C c1 = f1(x)G(x)

c2 C c2 = f2(x)G(x)

c1+c2 = (f1(x)+f2(x)) G(x) C

25

properties of cyclic codes (2)

Lemma: if (an-1, an-2, ..., a0)C, then (an-2, ..., a0, an-1)C

proof:

Let W(x) = an-1xn-1 + ... + a0 and W’(x) = an-2xn-1 + ... + a0x + an-1.

W(x) is a multiple of G(x)

W’(x) = an-2xn-1 + ... + a0x + an-1

= an-1xn + an-2xn-1 + ... + a0x + an-1+ an-1xn

= xW(x) + an-1(xn + 1)

26

multiple of G(x) multiple of G(x)

W’(x) is a multiple of G(x), and (an-2, ..., a0, an-1)C.

A cyclic code C is closed for a cyclic shift.

three approaches for encoding

three approaches for an encoding procedure:

matrix approach

use a generator matrix ... no advantage of cyclic codes

multiplication approach

codeword = (info. symbols) × G(x)

the code not systematic ( p.24)

division approach

slightly complicated (for human)

make the code systematic

easily implemented by shift registers

27

encoding by division

1. represent information symbols by a polynomial A(x)

2. divide A(x)xm by G(x), and let B(x) be the remainder

3. let A(x)xm + B(x) be a codeword

encoding 011, with n = 7, k = 3, m = 4, G(x) = x4 + x3 + x2 + 1

1. A(x) = x + 1

2. A(x)x4 = x5 + x4 = x(x4 + x3 + x2 + 1) + (x3 + x), and B(x) = x3 + x

3. A(x)x4 + B(x)= x5 + x4 + x3 + x, the codeword = 0111010

28

A(x)

D(x) G(x)

B(x)

x4

÷ divisor

dividend

quotient

remainder

W(x) = A(x) x4 + B(x)

did we really make “encoding”?

simple question: Is A(x)xm + B(x) really a codeword?

is A(x)xm + B(x) divided by G(x)?

Yes, note that...

B(x) is a remainder of A(x)xm

in a binary world, A(x)xp + B(x) = A(x)xp – B(x)

“A(x)xp – B(x)” = “remove the remainder”

if A(x)xp – B(x) is divided by G(x), then there is no remainder...

29

example

n = 7, k = 3, m = 4, G(x) = x4 + x3 + x2 + 1

30

data 000 001 010 011 100 101 110 111

A(x) x2+x+0 x2+x+1 x2+x+1 x2+x+1 x2+x+1 x2+x+1 x2+x+1 x2+x+1

A(x)x4

0000000 0010000 0100000 0110000 1000000 1010000 1100000 1110000

B(x) x3+x2+x+0 x3+x2+x+1 x3+x2+x+1 x3+x2+x+1 x3+x2+x+1 x3+x2+x+1 x3+x2+x+1 x3+x2+x+1

A(x)xm + B(x) 0000000 0011101 0100111 0111010 1001110 1010011 1101001 1110100

systematic code

encoder ≈ division circuit O(n) < O(n2) of matrix operation

error “detection” of cyclic codes

error “detection” is easy for cyclic codes

u C u (in a polynomial representation) is divided by G(x)

31

G(x)

= 0 ... no error 0 ... error ÷ divisor

dividend

quotient

remainder received u

one division circuit is used for encoding and error detection

reduces the cost of realization

Cyclic Redundancy Check (CRC)

... used in many communication systems

error “correction” of cyclic codes

general algorithm for all cyclic codes

error-trapping decoder [Kasami 61]

32

Tadao Kasami

1930-2007

E. Berlekamp, 1940- J. L. Massey, 1934-

I. Reed, 1923- (left)

and

G. Solomon,

1930-1996

special algorithms for special cyclic codes

Berlekamp-Massey algorithm for...

BCH (Bose-Chaudhuri-Hocquenghem) codes

Reed-Solomon Codes

summary

performance evaluation using weight distribution

slightly complicated (ちょっと複雑)

precise and detailed estimation is possible

cyclic code

a subclass of linear codes

advantages in realizations

33

exercise

Consider the following code C.

determine the weight distribution of C

compute the “three” probabilities (p. 7), and draw a graph

34

.

10001110

01001101

00101011

00010111

H

We want to construct a cyclic code with n = 7, k = 4, and m =3.

confirm that G(x) = x3 + x2 + 1 can be a generator polynomial

encode 0110

decide if 0001100 is a correct codeword or not