STATISTIC & INFORMATION THEORY

(CSNB134)

MODULE 12ERROR DETECTION &

CORRECTION

Noise in Channels

With real channels being imperfect, we have to consider the noise introduced:

Noise is one of the reasons for transmission errors. It may be caused by electrical interference, thermal noise, etc.

A transmission error occurs when a bit sent with aspecific value reaches the drain in a different state (e.g. sent ‘1’, received ‘0’ and vice versa)

Error Detection

All network technologies employ some form of error detection.

The basic idea is to use extra bits added to message to determine if errors have been introduced.

In the extreme case of error detection – we can transmit additional copy of message. If the receiver has received 2 identical messages, then no error has occurred. However this is not practical since it results in 100% redundancy.

Thus, it is best to adopt error detection mechanism which is effective enough to detect most errors yet which yield low level of redundancy (i.e. that require the least extra bits for error detection purposes)

Error Correction

There are also techniques available to correct errors in addition to detect errors (i.e. error correcting codes).

However, error correction techniques often requires many more redundant bits.

Sometimes, for less crucial security application, it is ‘cheaper’ to use error detection and throw away bad messages and / or request for retransmission of erroneous messages (i.e. avoid having to correct any erroneous messages).

Error Detection & Error Correction

Here, we shall study about two methods of error correction and detection, namely:-(i) Parity checking(ii) Hamming Coding

Parity Checking

Parity Bit was inserted in the early days per Byte:7 bits plus 1 parity bit (sometimes still used today – remember ASCII code, 7 data bits + 1 parity bit that make up a byte).

The parity bit identifies an odd / even number of '1' per group of data bits (e.g. 7 data bits)

Example (even parity)Data Bits Data + Parity Bits1011000 10110001 1011010 10110100

Example (odd parity)Data Bits Data + Parity Bits1011000 10110000 1011010 10110101

Note: Number of ‘1’ is even

Note: Number of ‘1’ is odd

Parity Checking (cont.)

Limitations of Party checking error detection:- Detects any odd number of errors.- Cannot detect even number of errors.

(Note: See example below based on even parity)

2-Dimensional Parity Checking

Extension of 1-D parity checking. Example (even parity):

parity bit0 1 0 1 0 0 1 11 1 0 1 0 0 1 01 0 1 1 1 1 0 10 0 0 1 1 1 0 10 1 1 0 1 0 0 11 0 1 1 1 1 1 0

parity byte 1 1 1 1 0 1 1 0

Detects any odd number of errors and all 2-bit errors

Exercise 1

Derive the parity bit P1, P2, P3 and P4 and the parity byte P5P6P7P8P9P10P11P12 assuming even parity checking is being deployed.

parity bit

0 1 1 1 1 0 1 P1

1 0 0 1 0 0 1 P2

1 0 1 1 0 1 0 P3

0 1 0 1 0 1 0 P4

parity byte P5P6P7P8P9P10P11 P12

1

1

0

0 0 0 0 1 0 0 1

1

Exercise 2

Identify erroneous bit/s and erroneous codeword/sassuming even parity is being deployed.

parity bit

1 0 1 1 1 0 1 0

1 1 0 0 0 1 0 1

1 0 1 0 0 1 0 1

1 1 1 1 0 1 0 0

parity byte 0 0 0 1 1 1 1 0Therefore possible error could be either:

the pair of bit A and bit D or the pair of bit B and bit C

Thus, it is capable of detecting any 2 bits error of 2 different code-words, HOWEVER incapable of correcting them.

A B

C D

Exercise 3

Identify erroneous bit/s and erroneous codeword/sassuming even parity is being deployed.

parity bit

0 0 1 0 1 0 0 0

0 0 1 1 0 1 0 0

1 1 1 0 0 1 0 0

1 1 0 1 1 1 1 0

parity byte 0 0 1 1 0 1 1 0Therefore possible error is bit A!

Thus, this 2-D parity checking only allows for 1 bit error correction!!

A

Hamming Coding

Hamming Coding is introduced mainly to increase the capability of error detection and error correction without having too many redundant bits.

Before understanding Hamming Coding, we shall need to learn two other parameters related to this:(1) Hamming Weight- The number of non-zero elements in a codeword c. It is written as w(c).(2) Hamming Distance- The number of places in which two codewords ci and cj differ. It is written as d(ci, cj)

Exercise 4

Let code C consists of 4 codewords:{110101, 010101, 101010, 111100}

Derive the Hamming Weights

Derive the Hamming Distances

ci c1 c2 c3 c4

Codeword 110101

010101

101010

111100

Hamming Weight w(ci)

4 3 3 4

ci cj d(ci,cj) ci cj d(ci,cj)

110101 010101 1 010101 101010 6

110101 101010 5 010101 111100 3

110101 111100 2 101010 111100 3

Hamming Coding (cont.)

(n, k) Hamming Code consists of:- n total number of bits per codeword (i.e. data bits + parity bits)- k total number of data bits per codeword

Suppose m is the total number of parity bits per codeword, n and k can be derived based on the following formulae:(n, k) Hamming Code =(2m – 1, 2m – 1 – m) Hamming Code

Note: n – k = mWhere m = total number of parity bits / control bits

Exercise 5

Suppose that a Hamming Code of n bits consists of k data bits – i.e. (n, k) Hamming Code. (a) If the total control bits m = 3, derive n and k.

(b) If the total control bits m = 4, derive n and k.

Compare the percentage of the redundancy in (a) and (b).

n = 2m – 1 = 23 – 1 = 7 where as k = n – m = 7 – 3 = 4

Percentage of redundancy in (a) = (3/4)x100% = 0.75%Percentage of redundancy in (b) = (4/11)x100% = 0.36%

n = 2m – 1 = 24 – 1 = 15 where as k = n – m = 15 – 4 = 11

Hamming Coding (cont.)

Previously we have learned how to calculate n and k of an (n, k) Hamming Code.

We also know that n - k = m total number of parity check bits in Hamming code.

Unlike in normal parity checking, where parity bit is always placed at the first bit, in Hamming code, the parity check bits are placed at 2x position.

For example for (7,4) Hamming code, where total number of parity bits = 7 – 4 = 3, the parity check bits are placed at:(1) 20 position = 1st bit(2) 21 position = 2nd bit(3) 22 position = 4th bit

Hamming Coding (cont.)

Suppose that a (7, 4) Hamming Code is decoded based as the following formulas:

B7 B6 B5 B4 B3 B2 B1 Remarks

Data Data Data Parity Data Parity Parity Code-word

Data - Data - Data - Parity B1 Even Parity

Data Data - - Data Parity B2 Even parity

Data Data Data Parity - - - B4 Even ParityNote: The parity bits B1, B2 and B4 are

related to the data bits B3, B5, B6 AND B7

B7 affecting B1, B2 & B4

B6 affecting B2 & B4

B5 affecting B1 & B4

B3 affecting B1 & B2

Exercise 6

Based on the previous formulae, derive the Hamming codeword if we want to transmit 1101:B7 B6 B5 B4 B3 B2 B1 Remark

s

Data 1

Data 1

Data 0

Parity 0

Data 1

Parity 1

Parity 0

Code-word

1 - 0 - 1 - 0 B1 Even Parity

1 1 - - 1 1 - B2 Even parity

1 1 0 0 - - - B4 Even Parity

Thus, the Hamming codeword is 1100110

Exercise 7

Based on the previous formulae, derive the Hamming codeword if we want to transmit 0100:B7 B6 B5 B4 B3 B2 B1 Remark

s

Data 0

Data 1

Data 0

Parity 1

Data 0

Parity 1

Parity 0

Code-word

0 - 0 - 0 - 0 B1 Even Parity

0 1 - - 0 1 - B2 Even parity

0 1 0 1 - - - B4 Even Parity

Thus, the Hamming codeword is 0101010

Exercise 8

Detect the erroneous data bit, assuming a single error has occurred in the received Hamming Code 1110110 (which follows previous formulae)B7 B6 B5 B4 B3 B2 B1 Remark

s

Data 1

Data 1

Data 1

Parity 0

Data 1

Parity 1

Parity 0

Code-word

1 - 1 - 1 - 0 ERROR!

1 1 - - 1 1 - OK!

1 1 1 0 - - - ERROR!

B1 & B4 indicate an error!! Common data bits effecting B1 & B4 - {B7, B5} - potential erroneous bits

However, B2 is free of error. Data bits affecting B2 – {B7, B6 and B3} - eliminate B7 from potential erroneous bit list!

Therefore, the erroneous bit is B5

Exercise 9

Detect the erroneous data bit, assuming a single error has occurred in the received Hamming Code 1101010 (which follows previous formulae)B7 B6 B5 B4 B3 B2 B1 Remark

s

Data 1

Data 1

Data 0

Parity 1

Data 0

Parity 1

Parity 0

Code-word

1 - 0 - 0 - 0 ERROR!

1 1 - - 0 1 - ERROR!

1 1 0 1 - - - ERROR!

B1, B2 & B4 indicate an error!! Common data bits effecting B1, B2 & B4 - {B7} - potential erroneous bit

Therefore, the erroneous bit is B7

STATISTIC & INFORMATION THEORY

(CSNB134)

ERROR DETECTION & CORRECTION

--END--