Chapter 3: Channel Coding

(part 3)(part 3)EKT 357 Digital Communications

Chapter 3 (part 3) Overview

• Error Correction techniques

▫ Repetition Techniques▫ Repetition Techniques

▫ Hamming Techniques

Forward Error Correction (FEC)

• The transmitted data are encoded so that thereceiver can detect and correct any errors.

• Can be used in both two-way or one-way• Can be used in both two-way or one-waytransmission.

• FEC is the most common technique used in thedigital communication because of its improvedperformance in correcting the errors.

Forward Error Correction (FEC)

• Improved performance because:

▫ It introduces redundancy in the transmitted data in a controlled way

▫ Noise averaging : the receiver can average out the noise over long time of periods.

Repetition

The simplest form of redundancy is: Repetition!

Sender ReceiverSender Receiver

“0” Did she say “1” ?

I said “0” Sounded like “0”

One more time: “0” Sounded like “0” again

She was sending “0”

Repetition Code

The easiest way to increase accuracy is repetition. Forinstance we can repeat the message 3 times.

Repeat every bit 3 times0110 => 000,111,111,000Error detected if all or any of 3 bits are not the sameError detected if all or any of 3 bits are not the same

000,100,111,000

Repetition Code

• Encoding rule: Repeat each bit 3 times

Example:

1 . 0 . 1 . � 111 . 000 . 111 .1 . 0 . 1 . � 111 . 000 . 111 .

• Decoding rule: Majority vote!

Examples of received codewords:

110 . 000 . 111 . � 1 . 0 . 1 . Error-free!

111 . 000 . 010 .� 1 . 0 . 0 . Error!

What if an Error is Detected?

• Strategy #1: Throw the data out and ask for it tobe sent again.▫ Slow but very low odds of erroneous data

• Strategy #2: Majority rules• Strategy #2: Majority rules▫ 0110 => 000,111,111,000 =>▫ 000,110,111,000 => 000,111,111,000▫ Quicker as compared with strategy#1, but higherodds of error

▫ Was it actually 0010 => 000,000,111,000 or000,100,111,000?

9

Analysis of Repetition Code

• Error can go undetected only if 3 consecutive bits are in error

• 0110 => 000,111,111,000 => 000,000,111,000• 0110 => 000,111,111,000 => 000,000,111,000

• If probability of one-bit error is p, then probability of undetected error is p3

• E.g. one-bit error = 10-5 => undetected error =10-15

• (Assumes independence)

How good is this repetition code?

• The redundancy in repetition of bits is measured by the efficiency or rate of the code, denoted by R:

• R= (#information bits / # bits in codeword) x 100%• R= (#information bits / # bits in codeword) x 100%

• For the 3-repetition code: R=33%

Example 1

• Calculate the rate of the code for the repetition bits of 5 times and 7 times with the original code of 3bits?of 3bits?

How good is this repetition code?

• However this method is not practical:

(i) maximum transmission rate is only 33.33%(i) maximum transmission rate is only 33.33%

(ii) normally the channel will be noisy that weneed to give up so much efficiency.

How good is this repetition code?

• Repeating each bit three times allows us tocorrect one error in each group of three bits, butnot more errors.not more errors.

• Suppose each bit has probability of beingreceived correctly, independently for each bit.

• The probability that a group of three repeatedbits will be decoded incorrectly is

▫ Pi=Pr( 0 errors) +Pr (1 error) = P + 3P (1-P)3 2

How good is this repetition code?

• As n gets bigger, the decoding error probability goes down. Here’s what happens with a 10% probability of transmission error (p=0.1)

Number of repetitions Probability of incorrect decoding

3 0.028

e

• It seems that we can push the error probability as close to zero as we wish by using more repetitions.

• If we let the number of repetitions grow and grow, we can approach perfect reliability !

5 0.086

7 0.002728

9 0.000892

What’s the catch?

The catch is:

As the number of repetitions grows to infinity, the transmission rate shrinks to zero!!!transmission rate shrinks to zero!!!

This means: slow data transmission.

Is there a better (more efficient) error correcting code?

Hamming Code

• Hamming code is a set of error correction codes that can beused to detect and correct bit errors that can occur whencomputer data is moved or stored.

• Invented by Richard W. Hamming in 1948.

• Corrects 1 bit error per codeword• Corrects 1 bit error per codeword

• (like repetition code)

• Code efficiency: R = 4/7 ≈ 57%, 7/11 ≈ 64%

• (compared to rate R ≈ 33% for 3-repetition code)

• Bits in positions that are power of 2 are checkbits/redundancy bit. The rest are data bits.

Hamming Code: Example7-bit

. Hamming codes can only correct single errors.

.

Hamming Code

Raw dataNotation: (bit 1, bit 2, bit 3)

000001001010011100101110111

Hamming Code

• Calculating the number of redundancy bits required. Since number of data bits is 3, the value of r is calculated as value of r is calculated as

▫ 2 >= m+r+1

▫ 2 >= 3+3 +1

• Therefore no. of redundancy bits = 3.

• The various r bits are placed at the position that corresponds to the power of 2, i.e, 1,2,4.

r

3

Add Check Bits

Notation: (bit 1, bit 2, bit 3, bit 4, bit 5, bit 6)

r r 0 r 0 0

r r 0 r 0 1r r 0 r 0 1

r r 0 r 1 0

r r 0 r 1 1

r r 1 r 0 0

r r 1 r 0 1

r r 1 r 1 0

r r 1 r 1 1

Generate the First Check Bit

0 r 0 r 0 0

0 r 0 r 0 1

1 r 0 r 1 01 r 0 r 1 0

1 r 0 r 1 1

1 r 1 r 0 0

1 r 1 r 0 1

0 r 1 r 1 0

0 r 1 r 1 1

Generate the Second Check Bit

0 0 0 r 0 0

0 1 0 r 0 1

1 0 0 r 1 01 0 0 r 1 0

1 1 0 r 1 1

1 1 1 r 0 0

1 0 1 r 0 1

0 1 1 r 1 0

0 0 1 r 1 1

Generate the third Check Bit

0 0 0 0 0 0

0 1 0 1 0 1

1 0 0 1 1 01 0 0 1 1 0

1 1 0 0 1 1

1 1 1 0 0 0

1 0 1 1 0 1

0 1 1 1 1 0

0 0 1 0 1 1

Hamming Code

0 0 0 0 0 0

0 1 0 1 0 1

1 0 0 1 1 0

Blue: Check bitsBlack: Data bits

1 0 0 1 1 0

1 1 0 0 1 1

1 1 1 0 0 0

1 0 1 1 0 1

0 1 1 1 1 0

0 0 1 0 1 1

Error in data bit

• Original data = 111000

• 1st bit = 0 (x)

• 2nd bit = 0 (x)• 2nd bit = 0 (x)

• 4th bit =0 (/)

• 1st +2nd =3rd

• Corrected data = 110000

Example 2

• Given code 1001101.

• Calculate the number of redundancy bits required and the total number of code bitsrequired and the total number of code bits

• Determine the error bit of the code.

• What is the corrected code bits?