GOOD2 L05_LDPCCode

7/27/2019 GOOD2 L05_LDPCCode

1/53

Introduction to Low Density Parity Check (LDPC) CodesESE576: Digital Communications

Charles Jeon1

1Student at Department of Electrical & Systems Engineering

University of Pennsylvania

June 15, 2012
http://find/http://goback/


2/53

Outline

1 History

2 Encoding LDPC Codes

3 DecodingHard-Decision DecodingSoft-Decision Decoding

4

Performance

5 Summary


3/53

History of LDPC Codes

LDPC codes were developed by Robert E. Gallager in his doctoraldissertation at MIT in 1960

Some people refer to LDPC codes as Gallager codes in his honor

They were forgotten at that time because they were considered tooimpractical to implement

LDPC codes were rediscovered by MacKay and Neal in 1996

Similar to Turbo codes, LDPC codes are a class of codes that aredecoded iteratively and perform very close to the Shannon limit
http://find/


4/53






http://find/


5/53






http://find/


6/53






http://find/


7/53






http://find/


8/53

Overview of Linear Block Codes

A (n,k) linear block code with data input vector x with length k, and

codeword vector c with length n is a k dimensional vector subspace of{0,1}n

For any linear block code, the codeword vectors c are given by c = xGwhere x is the k-bit input vector and G is the kn generator matrix.

The generator matrix G can be expressed as G = [Ik|P] where Ik is thekk identity matrix and P is a k (n k) matrix

The (n k) n parity check matrix H is defined to be some matrixsuch that cHT = 0 for all codeword vectors c

Each row of H gives a parity check equation for each code bits withsum (modulo-2) of zero
http://find/


9/53






The (n k) n parity check matrix H is defined to be some matrixsuch that cHT = 0 for all codeword vectors c

http://find/


10/53






The (n k) n parity check matrix H is defined to be some matrix

such that cHT = 0 for all codeword vectors c

http://find/


11/53








http://find/


12/53








http://find/


13/53

LDPC Codes are Type of Linear Block Codes

LDPC codes are a class of linear block codes characterized by a sparseparity check matrix H

Sparse implies that most of the elements ofH are 0; therefore, H haslow density of 1s

Expect a large dmin distance

LDPC codes are regular LDPC codes if H has uniform number of 1s

in columns and rows Ifwc and wr are number of 1s in columns and rows, wc n k and

wr n for H to be sparse Usually irregular LDPC codes perform better than regular codes

Sample parity check matrix H

H =

0 1 0 1 1 0 0 11 1 1 0 0 1 0 00 0 1 0 0 1 1 1

1 0 0 1 1 0 1 0
http://find/


14/53









H =

0 1 0 1 1 0 0 11 1 1 0 0 1 0 00 0 1 0 0 1 1 1

1 0 0 1 1 0 1 0
http://find/


15/53









H =

0 1 0 1 1 0 0 11 1 1 0 0 1 0 00 0 1 0 0 1 1 1

1 0 0 1 1 0 1 0
http://find/


16/53

Encoding LDPC Codes

Given some sparse parity check matrix H, we can use row and columnoperations to set H =

PT|Ink

and find the generator matrix G

Since G = [Ik|P], codeword vector c = xG = [x|xP]

Some practical issues with this:

Size ofG becomes very large G is generally not sparse
http://find/


17/53

Encoding LDPC Codes


PT|Ink





d d
http://find/


18/53

Encoding LDPC Codes


PT|Ink





T G h
http://find/


19/53

Tanner Graph

Alternative geometric approach to LDPC codes

Michael Tanner showed that parity check matrix H can be representedeffectively by using a bipartite graph or Tanner graph

Unidirectional graph whose nodes are separated into two classes, where

the edges only connect two nodes that are not in the same class

Tanner Graph has two classes of nodes:

Variable nodes (v): correspond to bit/symbol nodes to columns ofH

For (n

k)

nmatrix

H, we have

n vnodes

Check nodes (c): correspond to parity check equations to the rows ofH

We have n k c nodes

T G h
http://find/


20/53

Tanner Graph







For (n

k)

nmatrix

H, we have

n vnodes

Check nodes (c): correspond to parity check equations to the rows ofH

We have n k c nodes

T G h
http://find/


21/53

Tanner Graph







For (n

k

)n

matrixH

, we haven v

nodes Check nodes (c): correspond to parity check equations to the rows ofH

We have n k c nodes

E l f T G h
http://find/


22/53

Example of Tanner Graph

Example

H =

0 1 0 1 1 0 0 11 1 1 0 0 1 0 00 0 1 0 0 1 1 11 0 0 1 1 0 1 0

v0 v1 v2 v3 v4 v5 v6 v7

c0 c1 c2 c3

LDPC D di i N t A E T k
http://find/


23/53

LDPC Decoding is Not An Easy Task

Similar to Decoding for Turbo Codes, there are two types of iterativedecoding: Hard and Soft Decision Decoding

Has been shown that iterative decoding algorithms of sparse codesperform very close to optimal decoder

Some common algorithms:

Bit-flipping Algorithm Sum-Product Algorithm (Message passing/Belief Propagation

Algorithm) Min-Sum Algorithm

Reduced complexity approximation to the Sum-Product Algorithm

In general, per-iteration complexity of LDPC code is less than that forTurbo codes

More iterations maybe required

http://find/


24/53












25/53










http://find/


26/53










Bit Flipping Algorithm
http://find/


27/53

Bit-Flipping Algorithm

As the name suggests, bits are continually flipped by the algorithm

Algorithm:

Step 1: For each received variable nodes vi, calculate the parity checknode c

j. Terminate the algorithm if all check node c

jare 0.

Step 2: For each check node cj, j= 1,2, . . . consecutively, we flip thebits of the connected variables nodes {vk} in order ifcj = 1 andconverts back to received bits ifcj = 0

Step 3: Go back to Step 1

We will show by a simple example

http://find/


28/53



Algorithm:



jare 0.




http://find/


29/53

Bit Flipping Algorithm


Algorithm:



jare 0.




Bit-Flipping Algorithm (Contd )
http://find/


30/53

Bit Flipping Algorithm (Contd.)

For the Matrix H =

0 1 0 1 1 0 0 11 1 1 0 0 1 0 00 0 1 0 0 1 1 11 0 0 1 1 0 1 0

, assume that anerror free received code word would be c = [1 0 0 1 0 1 0 1]

Since cHT = 0, this is a valid codeword!

Denote blue arrows as 0 and red arrows as 1

v0 v1 v2 v3 v4 v5 v6 v7

c0 c1 c2 c3

Bit-Flipping Algorithm (Contd )
http://find/


31/53


For the Matrix H =

0 1 0 1 1 0 0 11 1 1 0 0 1 0 00 0 1 0 0 1 1 11 0 0 1 1 0 1 0

, assume that anerror free received code word would be c = [1 0 0 1 0 1 0 1]

Since cHT = 0, this is a valid codeword!

Denote blue arrows as 0 and red arrows as 1

v0 v1 v2 v3 v4 v5 v6 v7

c0 c1 c2 c3

Bit-Flipping Algorithm (Contd.)
http://find/


32/53


Now, let us suppose that we have a binary channel and received thecodeword with one error, bit c1 flipped to 1, c

= [1 1 0 1 0 1 0 1]

Following Step 1, we calculate the parity check node cj for all j. Sincec0,c1 = 1, we proceed to Step 2.

v0 v1 v2 v3 v4 v5 v6 v7

c0 = 1 c1 = 1 c2 = 0 c3 = 0

Note: blue arrows are 0 and red arrows are 1

http://find/


33/53

t pp g go t (Co td )

Following the Step 2 in the algorithm, we first flip all the {vk}associated with c0, that is {vk} = {v1,v3,v4,v7}

v0 v1 v2 v3 v4 v5 v6 v7

c0 = 1 c1 = 1 c2 = 0 c3 = 0


http://find/


34/53

pp g g ( )

We then flip the the {vk} associated with c1, since c1 = 1. Similarly,{vk} = {v0,v1,v2,v5}

v0 v1 v2 v3 v4 v5 v6 v7

c0 = 1 c1 = 1 c2 = 0 c3 = 0


http://find/


35/53

pp g g ( )

Proceeding, since c2 = 0, we convert the associated nodes back to thereceived configuration, {vk} = {v2,v5,v6,v7} = {0,1,0,1}

v0 v1 v2 v3 v4 v5 v6 v7

c0 = 1 c1 = 1 c2 = 0 c3 = 0


http://find/


36/53

pp g g ( )

Finally, c3 = 0 and we also convert the associated nodes back to thereceived configuration, {vk} = {v0,v3,v4,v6} = {1,1,0,0}

v0 v1 v2 v3 v4 v5 v6 v7

c0 = 1 c1 = 1 c2 = 0 c3 = 0


http://find/


37/53

g g ( )

Going back to Step 1, we have that c0,c1,c2,c3 = 0 and the Algorithmterminates.

We decide on the code c = [1 0 0 1 0 1 0 1], which is the originalerror-free code

v0 v1 v2 v3 v4 v5 v6 v7

c0 = 0 c1 = 0 c2 = 0 c3 = 0


Bit-Flipping Algorithm Summary


38/53

Therefore, what our algorithm did is the following:

c node received/sent

c0received: v1 1 v3 1 v4 0 v7 1

sent: 0 v1 0 v3 1 v4 0 v7


sent: 0 v0 0 v1 0 v2 0 v5


sent: 0 v2 1 v5 0 v6 1 v7


sent: 1 v0 1 v3 0 v4 0 v6

Belief Propagation Algorithm
http://find/


39/53

BPA builds on the Bit-flipping Algorithm by taking the likelihood of 1and 0 into account

Two stages, one for probabilities of bit nodes vi and one forprobabilities of check nodes cj

Let us denote the following for b {0,1}:

Pi = P{ci = 1|yi} where yi denotes the i-th received bit

qij(b) as the message sent by variable node vi to check node ci rji(b) as the message sent by the check node cj to variable node vi

cj

vi

rji(b)

qij(b)

cj

vi

qji(b)

rji(b)yi

For Step 1 in the Hard Decision, we have that all the variable nodes visend their qij messages. Since no other information is available, we

initialize qij(1) = Pi and qij(0) = 1 Pi

Belief Propagation Algorithm


40/53

BPA builds on the Bit-flipping Algorithm by taking the likelihood of 1and 0 into account

Two stages, one for probabilities of bit nodes vi and one forprobabilities of check nodes cj

Let us denote the following for b {0,1}:

Pi = P{ci = 1|yi} where yi denotes the i-th received bit

qij(b) as the message sent by variable node vi to check node ci rji(b) as the message sent by the check node cj to variable node vi

cj

vi

rji(b)

qij(b)

cj

vi

qji(b)

rji(b)yi

For Step 1 in the Hard Decision, we have that all the variable nodes visend their qij messages. Since no other information is available, we

initialize qij(1) = Pi and qij(0) = 1 Pi

Belief Propagation Algorithm (Contd.)
http://find/


41/53

For Step 2, the check nodes cj calculates their response messages rjiby formula by Gallager,

rji(0) =1

2+

1

2 iVj\i

1 2qij(1)

where rji(1) = 1 rji(0)

This equation is the probability that there are even number of 1samong the variable nodes except vi

cj

vi

rji(b)qij(b)

http://find/


42/53

Using this information, we use the values of rji(0) and rji(1) to findqij(0) and qij(1) given by

qij(0) = Kij(1 Pi) jCi\j

rji(0)

andqij(1) = KijPi

j

Ci\j

rji(1)

where Kij are chosen to ensure qij(0) +qij(1) = 1

Note that Ci\j designates all the check nodes except for cj.

cj

vi

qji(b)

rji(b)yi

http://find/


43/53

Finally, we have all the variable nodes vi update their currentestimation of ci of the codeword ci

Using the ideas of previous slide,

Qi(0) = Ki(1 Pi)jCi

rji(0)

andQi(1) = KiPi

jCi

rji(1)

which calculates the likelihood from 0 and 1 from all check nodes cj

We then have decide on ci by,

ci =

1 ifQi(1) > Qi(0)

0 otherwise

http://find/


44/53



Qi(0) = Ki(1 Pi)jCi

rji(0)

andQi(1) = KiPi

jCi

rji(1)



ci =

1 ifQi(1) > Qi(0)

0 otherwise

http://find/


45/53



Qi(0) = Ki(1 Pi)jCi

rji(0)

andQi(1) = KiPi

jCi

rji(1)



ci =

1 ifQi(1) > Qi(0)

0 otherwise

http://find/


46/53

If the estimated codeword vector c does equal 1, the algorithmterminates.

Otherwise the algorithm goes back to Step 2 and repeats the processuntil termination is ensured either by equaling 1, or reaching maximumnumber of iterations

Performance of LDPC Codes
http://find/


47/53

LDPC Codes are very powerful, perform similar to Turbo codes

1

1P. Jagatheeswari and M. Rajaram

Performance Depends on the Soft-Decision Algorithms
http://find/


48/53

Although we do not have BER Performance of Hard-Decision

decoding, simulations show that performance depends on the selectionof Soft-Decision algorithms

2

2S. Lee

LDPC Codes approach Shannon Capacity
http://find/


49/53

LDPC Codes have the potential to offer both better performance and

lower decoding complexity in many cases3

3Comtech EF Data

DVB-S2 LDPC Codes
http://find/


50/53

The latest version of the standard DVB-S2 uses a combination of anouter BCH code and inner LDPC code a significant upgrade over

DVB-S DVB-S used combination of outer RS code and inner convolutional

code

The codeword length may have either n = 64800 (normal) or

n = 16200 (short)

Summary


51/53

Turbo and LDPC Codes approach very closely to the Shannon limit

S. Chung et al. showed that their LDPC code can reach 0.0045 dB ofthe Shannon Limit (2001)

Some examples of Turbo and LDPC codes used today:

Binary Turbo 8-state: CDMA2000, Universal MobileTelecommunications System (UMTS) Duo-Binary Turbo 8-state: DVB-RCS, DVB-RCT, IEEE 802.16

WiMAX LDPC: DVB-S2

Current research focuses on reducing complexity, Richardson andUrbanke (2001) show the complexity can become almost a linearfunction ofn, rather than n2

Summary


52/53





WiMAX LDPC: DVB-S2


Summary
http://find/


53/53





WiMAX LDPC: DVB-S2

http://find/

GOOD2 L05_LDPCCode

Documents

Transcript of GOOD2 L05_LDPCCode