Elementary Coding Elementary Coding Theory Including Theory Including

Hamming and Reed-Hamming and Reed-Solomom Codes with Solomom Codes with Maple and MATLABMaple and MATLAB

Richard KlimaRichard KlimaAppalachian State Appalachian State

UniversityUniversity

Boone, North CarolinaBoone, North Carolina

Neil SigmonNeil SigmonRadford UniversityRadford University

Radford, VirginiaRadford, Virginia

Purpose for PresentationPurpose for Presentation

To demonstrate some applications To demonstrate some applications with error correcting codes with with error correcting codes with MATLAB and MapleMATLAB and Maple Reed-Muller CodesReed-Muller Codes Hamming CodesHamming Codes Reed-Solomon CodesReed-Solomon Codes

Basic Notation Basic Notation TerminologyTerminology

Coding theoryCoding theory is the study of methods is the study of methods for for efficient and accurate transformation efficient and accurate transformation from one place to another.from one place to another.

A code is a set of codewords A code is a set of codewords representing the messages to be representing the messages to be transferred between two parties.transferred between two parties.

An error-correcting code is a code whereAn error-correcting code is a code whereit is possible to detect and correct errors it is possible to detect and correct errors during the transmission of codewords.during the transmission of codewords.

ApplicationsApplications Information transmitted by Internet.Information transmitted by Internet. Wireless communication devices.Wireless communication devices. Compact Discs.Compact Discs. Transmission of information from Transmission of information from

satellites.satellites.

Reed-Muller CodesReed-Muller Codes Invented by I. Reed and D. Muller in Invented by I. Reed and D. Muller in

1954.1954.

Can be constructed using matrices.Can be constructed using matrices.

Can correct a multiple number of Can correct a multiple number of errors.errors.

Was used to perform error correction Was used to perform error correction for the Mariner 9 space probe mission for the Mariner 9 space probe mission to Mars in 1972.to Mars in 1972.

Hadamard MatricesHadamard Matrices

A matrix A matrix H H is a Hadamard is a Hadamard matrix if all entries of matrix if all entries of HH are 1 or -1 are 1 or -1 and ,and ,

where where I I is the identity matrix. is the identity matrix.

A A normalizednormalized Hadamard matrix is Hadamard matrix is

one where the first row and column one where the first row and column of of HH only contain positive ones. only contain positive ones.

nnnIHH T

nn

The only normalized Hadamard The only normalized Hadamard matrices of order one and two arematrices of order one and two are

Subsequent normalized Hadamard Subsequent normalized Hadamard

matrices can be generated using the matrices can be generated using the block matrix formulablock matrix formula

11

11 and 1 21 HH

nn

nnn HH

HHH2

Reed-Muller Code Construction Reed-Muller Code Construction StepsSteps

Construct a normalized Hadamard Construct a normalized Hadamard matrix of order . matrix of order .

Delete the first row and first column of Delete the first row and first column of the normalized Hadamard matrix. the normalized Hadamard matrix. Change all negative ones to zeros. Call Change all negative ones to zeros. Call this matrix this matrix AA..

Change all ones to zeros and zeros to Change all ones to zeros and zeros to ones in the matrix ones in the matrix AA. Call this matrix . Call this matrix BB..

84 k

Attach a column of all ones to the left Attach a column of all ones to the left of the matrix of the matrix AA and a column of all and a column of all zeros to the left of the matrix zeros to the left of the matrix BB. Call . Call these matrices these matrices scriptscriptAA and and scriptscriptBB, , respectively.respectively.

Stack the matrix Stack the matrix scriptscriptAA on top ofon top of scriptscriptBB.. Add an extra row of all zeros Add an extra row of all zeros and an extra row of all ones to the and an extra row of all ones to the bottom of the matrix.bottom of the matrix.

The codewords for the code are The codewords for the code are vectors obtained from the rows for vectors obtained from the rows for the resulting matrix. the resulting matrix.

Each codeword Each codeword cc is a vector is a vector consisting over elements in of consisting over elements in of length 4length 4kk..

2Z

Reed-Muller codes are classified as Reed-Muller codes are classified as

(4(4kk, 2, 2kk) codes, where 4) codes, where 4k k is the length of is the length of

each codeword and each codeword and 22k k is the is the minimum minimum distance distance between codewords.between codewords.

A Reed-Muller code has 8A Reed-Muller code has 8k k codewords.codewords.

Error Correction for Reed-Muller Error Correction for Reed-Muller CodesCodes

Reed-Muller codes are designed to Reed-Muller codes are designed to correct up to correct up to kk – – 1 errors.1 errors.

Suppose a vector Suppose a vector rr is received. This is received. This vector is compared to all codewords vector is compared to all codewords in the code. The vector in the code. The vector rr will be will be corrected to the codeword that corrected to the codeword that differs from it in differs from it in kk – – 1 or less 1 or less positions.positions.

MATLAB ExampleMATLAB Example

We generate the Reed-Muller code We generate the Reed-Muller code that was used for in Mariner 9 space that was used for in Mariner 9 space probe used to take pictures of Mars probe used to take pictures of Mars in 1972.in 1972.

The code was constructed using the The code was constructed using the normalized Hadamard (normalized Hadamard (k = k = 8)8). . The code consisted of 64 codewords The code consisted of 64 codewords of length 32 each. The code could of length 32 each. The code could correct up to 7 errors.correct up to 7 errors.

32H

Before being transmitted, each Before being transmitted, each photograph taken was broken down photograph taken was broken down into a collection of pixels. into a collection of pixels.

Each pixel was assigned one of 64 Each pixel was assigned one of 64 levels of grayness, and then encoded levels of grayness, and then encoded into one of the 64 codewords in the into one of the 64 codewords in the code.code.

Hamming CodesHamming Codes

First published by Richard Hamming First published by Richard Hamming in 1950.in 1950.

Used by the telephone industry and Used by the telephone industry and in computer data transfer.in computer data transfer.

Can correct 1 error.Can correct 1 error.

Code is perfect, which says every Code is perfect, which says every message received with errors is message received with errors is uniquely correctable to a codeword.uniquely correctable to a codeword.

Hamming Codeword Hamming Codeword ConstructionConstruction

Let and be vector Let and be vector spaces with and let spaces with and let GG be a be a

matrix over of full row rank. Thenmatrix over of full row rank. Then

is a subspace of of dimension .is a subspace of of dimension .

The vectors in The vectors in CC form codewords. form codewords.

kZW 2nk nk

2Z

} somefor |{ WwwGvVvC

nZV 2

V k

k2

The matrix The matrix GG is called the generator is called the generator matrix for the code matrix for the code CC..

The matrix The matrix GG is formed by finding the is formed by finding the nullspace of a matrix nullspace of a matrix HH known as the known as the parity check matrixparity check matrix..

Hamming codes form a linear [Hamming codes form a linear [n, kn, k] ] code, where the codewords have a code, where the codewords have a length of length of

nn positions and positions and kk is the dimension of is the dimension of the vector space the code forms.the vector space the code forms.

In Hamming codes, andIn Hamming codes, and

for some integer for some integer mm > > 1.1.

mk m 12

12 mn

The parity check matrix The parity check matrix HH is a matrix is a matrix of size that is formed by of size that is formed by placing as its columns the binary placing as its columns the binary expressions of the integers expressions of the integers . .

The generator matrix The generator matrix G G will be of sizewill be of size

..

If a codeword, then . If a codeword, then .

12,,2,1 m

)12( mm

)12()12( mm m

wGc 0THc

MATLAB ExampleMATLAB Example

In MATLAB, suppose we demonstrate In MATLAB, suppose we demonstrate how to construct a codeword from the how to construct a codeword from the vector vector

ww = = (10111011110).(10111011110).

We encode this message using a [15, We encode this message using a [15, 11] Hamming code with generator 11] Hamming code with generator matrix matrix G G formed from the nullspace of formed from the nullspace of the rows of the parity check matrixthe rows of the parity check matrix

101010101010101

110011001100110

111100001111000

111111110000000

H

Error Correction for Hamming Error Correction for Hamming CodesCodes

Suppose a vector Suppose a vector rr is transmitted and is transmitted and received. If , then received. If , then r r is a is a codeword.codeword.

If not, then the error can be If not, then the error can be corrected by computing corrected by computing where is a vector containing all where is a vector containing all zeros except at the position. zeros except at the position.

0THr

ierc iethi

The error position The error position ii is determined by is determined by using the fact that .using the fact that .

Since and produce the Since and produce the same column, the column number same column, the column number produced by produced by

gives the error position gives the error position ii..

Ti

T HeHr

THrTiHe

THr

MATLAB ExampleMATLAB Example

Using a [15, 11] Hamming code, we Using a [15, 11] Hamming code, we demonstrate how to correct the demonstrate how to correct the following received vector.following received vector.

r = r = ((011100010111110011100010111110))

Reed-Solomon CodesReed-Solomon Codes

Developed by Irving Reed and Developed by Irving Reed and Gustave Solomon in 1960.Gustave Solomon in 1960.

One of the most widely used error One of the most widely used error correcting code techniques.correcting code techniques.

Ideal at correcting Ideal at correcting errorerror burstsbursts, , where several errors occur close where several errors occur close together.together.

Important for the Voyager 2 Important for the Voyager 2 satellite’s transmission of pictures of satellite’s transmission of pictures of the outer planets Uranus and the outer planets Uranus and Neptune.Neptune.

Besides satellite communications, Besides satellite communications, Reed Solomon codes are used with Reed Solomon codes are used with wireless communication and with wireless communication and with compact disks.compact disks.

Example Pictures taken by Example Pictures taken by VoyagerVoyager

Pictures of Uranus and its rings

Example Pictures taken by Example Pictures taken by VoyagerVoyager

Picture of Neptune and one its moons, Triton

The Reed-Solomon code implemented The Reed-Solomon code implemented on Voyager 2 was constructed using on Voyager 2 was constructed using a finite field of a finite field of elements represented by the elements represented by the primitive polynomialprimitive polynomial

Codeword Construction in Codeword Construction in Reed-Solomon CodesReed-Solomon Codes

))(/(][2 xpxZF 25628

1)( 2348 xxxxxp

Codewords for Voyager were obtained by Codewords for Voyager were obtained by taking multiples of the generating taking multiples of the generating polynomialpolynomial

where where a a is a primitive element use to generate . is a primitive element use to generate .

A codeword is formed by computing A codeword is formed by computing the polynomial the polynomial

where where mm((xx)) is a polynomial with is a polynomial with coefficients from the finite field .coefficients from the finite field .

)())(()( 322 axaxaxxg

)(xc

F

F

)()()( xgxmxc

Each codeword is represented by Each codeword is represented by polynomials of degree less than polynomials of degree less than . .

Each codeword is a vector of length Each codeword is a vector of length 255 with the information transformed 255 with the information transformed in the polynomial’s coefficients.in the polynomial’s coefficients.

255128 )(xc

Each polynomial coefficient in the Each polynomial coefficient in the codeword is a polynomial of degree < codeword is a polynomial of degree < . Each polynomial coefficient is a . Each polynomial coefficient is a unique element from the finite field . unique element from the finite field .

On Voyager, full color images were On Voyager, full color images were digitized into binary vectors over .digitized into binary vectors over .

8)( xp

F

2Z

The binary vectors can be represented The binary vectors can be represented in “blocks” of length 8 as the in “blocks” of length 8 as the coefficients of polynomials of degree < coefficients of polynomials of degree < pp((xx)) = 8. For example, = 8. For example,

In binary, the length of each codeword In binary, the length of each codeword converts to a vector of length of converts to a vector of length of

74321)10111001( aaaa

2040)12())((deg 8 xp

A codeword A codeword has the property has the property thatthat

for . This gives a for . This gives a technique for verifying a transmitted technique for verifying a transmitted message is a codeword.message is a codeword.

0)( iac 32,,1 i

)(xc

Error Correction in Error Correction in Reed-Solomon CodesReed-Solomon Codes

If a transmitted message If a transmitted message rr((xx)) is is received that is not a codeword, a received that is not a codeword, a polynomial polynomial ee((xx)) is computed to is computed to correct its coefficients by forming correct its coefficients by forming . .

The Voyager 2 Reed-Solomon code The Voyager 2 Reed-Solomon code could correct up to could correct up to polynomial coefficient errors. This polynomial coefficient errors. This translates to a minimum of 121 translates to a minimum of 121 errors in binary.errors in binary.

)()()( xcxexr

162/32

Finding Finding ee((xx)) involves performing involves performing computations involving the Euclidean computations involving the Euclidean algorithm and polynomial algorithm and polynomial differentiation.differentiation.

Maple ExampleMaple Example

Involves user-written proceduresInvolves user-written procedures bincoeffbincoeff binwordbinword rscoeffrscoeff rseuclidrseuclid

We demonstrate both the codeword We demonstrate both the codeword construction and error correction construction and error correction process.process.

ReferencesReferences

R. E. Klima, N. P. Sigmon, and E. L. R. E. Klima, N. P. Sigmon, and E. L. Stitzinger, Stitzinger, Applications of Abstract Applications of Abstract Algebra with Maple, Algebra with Maple, CRC Press, Llc, Boca CRC Press, Llc, Boca Raton, FL 2000.Raton, FL 2000.

Stephen B. Wicker and Vijay K Bhargava, Stephen B. Wicker and Vijay K Bhargava, editors: editors: Reed-SolomonReed-Solomon Codes and their Codes and their ApplicationsApplications, The Institute of Electrical , The Institute of Electrical and Electronics Engineers, Inc., New York, and Electronics Engineers, Inc., New York, NY 1994.NY 1994.