1Convergence Analysis and Optimal Schedulingfor Multiple Concatenated Codes

Fredrik Brannstrom, Member, IEEE, Lars K. Rasmussen, Senior Member, IEEE, andAlex J. Grant, Senior Member, IEEE

Abstract An interesting practical consideration for decod-ing of serial or parallel concatenated codes with more thantwo components is the determination of the lowest complexitycomponent decoder schedule which results in convergence. Thispaper presents an algorithm that finds such optimal decoderschedule. A technique is also given for combining and projecting aseries of three-dimensional extrinsic information transfer (EXIT)functions onto a single two-dimensional EXIT chart. This isa useful technique for visualizing the convergence thresholdfor multiple concatenated codes and provides a design tool forconcatenated codes with more than two components.

Index Terms EXIT chart, iterative decoding, multiple con-catenated codes, optimal scheduling.

I. INTRODUCTION

S INCE the invention of turbo codes [1] with two parallelconcatenated component codes, the turbo principle hasbeen extended to serially concatenated codes [2], multiple par-allel concatenated codes [3] and multiple serially concatenatedcodes [4].

Iterative decoding of concatenated codes with two com-ponents can be analyzed using two-dimensional extrinsic in-formation transfer (EXIT) charts [5, 6]. These charts may beused to predict convergence thresholds and average decodingtrajectories and have proved to be a useful tool for codeconstruction. EXIT chart analysis has been extended to parallelconcatenated codes with three components [7], resulting ina three-dimensional chart. For three serially concatenatedcodes however, the chart would be four-dimensional and it istherefore difficult to show the decoding trajectory in a singlechart. Without a proper approach, extension of EXIT chart

F. Brannstrom and L. K. Rasmussen are supported by the Swedish ResearchCouncil under Grant 621-2001-2976. F. Brannstrom is also supported byPersonal Computing and Communication (PCC++) under Grant PCC-0201-09.L. K. Rasmussen and A. J. Grant are supported by the Australian Governmentunder ARC Grant DP0344856. The material in this correspondence waspresented in part at the IEEE International Symposium on Information Theory,Yokohama, Japan, June/July 2003 and at the International Symposium onTurbo Codes and Related Topics, Brest, France, September 2003.

F. Brannstrom was with the Department of Computer Engineering,Chalmers University of Technology, Sweden. He is now with the Departmentof Signals and Systems, Chalmers University of Technology, SE-412 96Goteborg, Sweden (e-mail: fredrikb@chalmers.se).

L. K. Rasmussen is with the Institute for Telecommunications Research,University of South Australia, Mawson Lakes, SA 5095, Australia (e-mail:Lars.Rasmussen@unisa.edu.au) and with the Department of Computer Engi-neering, Chalmers University of Technology, SE-412 96 Goteborg, Sweden(e-mail: larsr@ce.chalmers.se).

A. J. Grant is with the Institute for Telecommunications Research, Uni-versity of South Australia, Mawson Lakes, SA 5095, Australia (e-mail:Alex.Grant@unisa.edu.au).

analysis to codes with more than three components becomesunmanageable.

For two component codes, decoding alternates betweenthe two component decoders [5, 6]. With more than twocomponents however, the schedule of decoder activations isno longer obvious. Here, the term activate is used instead ofiterate (in a two-component code, one iteration is the sameas two activations). Previously, fixed schedules have beenused e.g., [4, 7], while a message-passing strategy giving thebest bit-error rate (BER) performance for multiple parallelconcatenated codes has been investigated in [3, 8].

This paper presents a technique for combining and project-ing a series of two- and three-dimensional EXIT functionsonto a single two-dimensional chart. Similar to a system withtwo components, the convergence threshold for a system withmultiple concatenated codes can then be visualized as a tunnelin the projected EXIT chart. An optimization algorithm is alsodescribed, which finds the activation schedule that yields thebest possible performance using the lowest possible decodingcomplexity.

In Section II the system model is specified. Section IIIdefines the different mutual informations and EXIT functions.Section IV introduces the EXIT chart projection and examplesare given demonstrating convergence analysis for multipleconcatenated codes. Section V develops the decoder scheduleoptimization algorithm. Numerical examples are presented inSection VI and concluding remarks are given in Section VII.

II. SYSTEM MODEL

Consider coded transmission of binary data over an additivewhite Gaussian noise (AWGN) channel. With reference tothe examples in Fig. 1 (three serial codes) and Fig. 2 (fourparallel codes), the encoder consists of N serially or parallelconcatenated component codes Cn, n = 1, 2, . . . , N . Therate of code n is Rn and the overall transmission rate isR. Transmission is divided into blocks of L independent andidentically distributed source bits x {1,+1}L. Encoder Cnmaps a sequence of Ln input symbols xn {1,+1}Ln toa sequence of Ln/Rn output symbols yn {1,+1}Ln/Rn ,1 n N . Individual elements of these sequences will bedenoted by xn(i) and yn(j), respectively, i = 1, 2, . . . , Lnand j = 1, 2, . . . , Ln/Rn.

The innermost encoder, whose output is connected tothe channel is a modulator M, mapping blocks of m bitsto one of M = 2m symbols, e.g., BPSK, 4PSK, 8PSK or16QAM symbols [5]. The rate of the mapper is m and the

2-x3 C3 -y3 2 -

x2 C2 -y2 1 -

x1 M s

?

-w

rE(x3)

C13A(y3)

-E(y3)

12

2

E(x2)

-A(x2)C12

A(y2)

-E(y2)

11

1

E(x1)

-A(x1)M1

Fig. 1. Three serially concatenated codes.

x - 1 -x1 C1 -

y1

- 2 -x2 C2 -

y2

- 3 -x3 C3 -

y3

- 4 -x4 C4 -

y4

@@@@

M -s

-r?

w

@@

@@

M1

-A(y1)

C11-E(x1) 11 -

E1(x)

A(x1) 1 A1(x)

-A(y2)

C12-E(x2) 12 -

E2(x)

A(x2) 2 A2(x)

-A(y3)

C13-E(x3) 13 -

E3(x)

A(x3) 3 A3(x)

-A(y4)

C14-E(x4) 14 -

E4(x)

A(x4) 4 A4(x)

Fig. 2. Four parallel concatenated codes.

sequence of symbols s {s1, s2, . . . , sM}L/R is transmittedover the AWGN channel subject to a transmitter averageenergy constraint Es = 1M

Mk=1 |sk|2 = REb, where Eb

is the average bit energy of the source bits. The receiversmatched filter output is r = s+w, where each element in w iszero-mean Gaussian with variance 2w = N0/2 per dimension.The signal-to-noise ratio (SNR) is b , Eb/N0.

Let E(xn) denote a sequence of extrinsic values corre-sponding to xn. Likewise, A(xn) is a sequence of priors forxn. This notation is naturally extended to other sequences, xand yn, 1 n N . Priors and extrinsics will be representedas sequences of log-likelihoods, A(xn) RLn .

The decoder consists of N a posteriori probability (APP)decoders C1n [9], interconnected by interleavers and dein-terleavers 1. Upon activation, each decoder uses its codeconstraint and the most recent priors [8] to update its extrinsicvalues [9], (E(xn), E(yn)) = C1n (A(xn), A(yn)).

Let D(x) RL denote the decision statistics for the sourcebits x after each activation. The hard decision x(i) on elementx(i) is according to

D(x(i))x(i)=+1

x(i)=1

0. (1)

Performance is measured in BER, i.e., the probabilityPr(x(i) 6= x(i)).

A. Serially Concatenated CodesIn a system with N serially concatenated codes [4], the

encoders are interconnected by N 1 interleavers, xn =n(yn+1), 1 n N 1. The outer encoder CN hasLN = L and for the other codes, 1 n N 1, the blocklengths are Ln = Ln+1/Rn+1 = Ln1Rn, and the overallrate is R =

Nn=1Rn. Fig. 1 shows an example with three

components, where the inner encoder is a mapper C1 = Mwith R1 = m. The inner encoder may also model an inter-symbol interference channel [10] or a multiple access channel[11].

All extrinsic values are set to zero before decoding,E(xn) = {0}Ln and E(yn) = {0}Ln/Rn , 1 n N .The inner decoder (or demapper) uses the matched filteroutputs r together with the prior values on x1 to produceextrinsics E(x1) =M1(A(x1), r). After initial activation ofthe demapper, decoding proceeds iteratively according to someactivation schedule of the component decoders. Extrinsic val-ues from decoder n, become priors to the connecting decodersA(yn+1) =

1n (E(xn)) and A(xn1) = n1(E(yn))

respectively. The source priors A(xN ) will always be zero andhence A(x3) is not shown in Fig. 1. The decision statistics

D(x) = E(xN ), (2)

are only updated when C1N is activated.

3B. Parallel Concatenated CodesIn a system with N parallel concatenated codes there are

N interleavers permuting the source sequence into N differentinput sequences, xn = n(x), with Ln = L, 1 n N .

Fig. 2 shows an example with four components. In thisexample, M is a memoryless modulator/multiplexer, mappingbits from yn to antipodal, e.g. BPSK symbols before they aretransmitted in serial over the channel. Since a BPSK mapperhas a rate m = 1, the overall rate is R =

(Nn=1 1/Rn

)1.

The demapper/demultiplexer, M1 produces A(yn), 1 n N and these are constant during the decoding process.Prior to decoding, all extrinsic values are set to zero, En(x) ={0}L, 1 n N . Decoding proceeds iteratively accordingto some schedule of activation of the component decoders andin this case, any of the N decoders can be activated first.

Upon activation, decoder C1n updates extrinsics on thesource bits En(x) = 1n (E(xn)) as a function of the priorvalues A(xn) = n(An(x)), where

An(x) =Ni=1i6=n

Ei(x). (3)

The decision statistics are updated according to

D(x) = En(x) +An(x) =

Ni=1

Ei(x). (4)

In contrast to a serial code, the source bit decisions in a parallelcode are updated every activation according to (1) and (4).

The mapper in Fig. 2 may also map m = N bits (one bitfrom each of the N output sequences yn) to a M = 2N -point constellation. In this case something can be gainedby passing extrinsic values E(yn) from all decoders C1n ,1 n N , to the demapper M1. The demapper is now alsoa decoder that can be activated, leading to a system with N+1components. In this scenario, N interleavers/deinterleaversshould be inserted between the encoders/decoders and themapper/demapper to enhance performance resulting in a hy-brid parallel/serial system. For simplicity, BPSK modulationwill be used for parallel codes in this paper. It is straightfor-ward to apply the principles described in the next sections tohybrid systems.

C. ExamplesTwo example codes will be used in this paper, one serial

and one parallel. The specific combination of component codesand mappers used are not optimal in any sense, and are chosensimply to demonstrate the principles introduced in this paper.

A feed-forward convolutional code (CC) will be denoted asCC(2, 3), where the numbers are octal representation of thegenerator polynomials 1 and 1+D. A recursive convolutionalcode will be denoted as CC(15/13), with generator 1+D+D

3

1+D2+D3[6].

The serial example has three components. With referenceto Fig. 1, C3: CC(2, 3) has two states, R3 = 1/2 and C2:CC(2/3) has two states, R2 = 1. A memoryless 8PSKmapper is used as the inner code with counter-clockwise

labelling 0, 1, 2, 3, 6, 7, 5, 4 (octal representation of the bitsmapped to the signal points). This mapping is chosen to givea low convergence threshold [5, 12]. The concatenated codetransmits R = 1.5 bits per channel use.

The parallel example has four CCs as component codes asin Fig. 2. Two of these are feed-forward and two recursive,and Rn = 1 for n = 1, 2, 3, 4. The component codes areC1: CC(3) (two states), C2: CC(7) (four states), C3: CC(2/3)(two states) and C4: CC(15/13) (eight states), respectively. Nosystematic bits are transmitted. These codes are used togetherwith BPSK modulation producing the overall rate R = 1/4bits per channel use.

III. MUTUAL INFORMATION

Mutual information (MI) [13] is used in EXIT charts topredict the convergence behavior of iterative decoding [57]. Let G R be a continuous random variable and X {+1,1} a uniformly distributed discrete random variable.Define pG|X(|x) as the conditional probability density func-tion (PDF) of G given X = 1. The MI IG = I(X;G) is [6,13]

IG=1

2

x=1

+

pG|X(|x) log2(

2pG|X(|x)pG|X(|x) + pG|X(|x)

)d.

(5)Assume G = X +W is Gaussian with mean X , where

0 is a constant and W is zero-mean Gaussian withvariance 2w. Then

IG = J

(2

w

), (6)

where J is defined as [6]

J() = 1 12

+

e(2/2)2

22 log2(1 + e

)d. (7)

J() is monotonically increasing and therefore has a uniqueinverse function,

= J1(IG) . (8)It is infeasible to express J or its inverse in closed form.However, they can be closely approximated by [12]

J() (1 2H12H2 )H3 , (9)

J1(IG) ( 1H1

log2

(1 I

1H3

G

)) 12H2

. (10)

Numerical optimization, using the Nelder-Mead simplexmethod [14], to minimize the total squared difference be-tween (7) and (9) gives H1 = 0.3073, H2 = 0.8935, andH3 = 1.1064. The solid curve in Fig. 3 shows (7) and theindistinguishable approximation (9).

A. Prior Mutual InformationIn a parallel system using BPSK modulation, e.g. Fig. 2,

the received matched filter output at time i is r(i) =

40 1 2 3 4 5 6 7 80

0.2

0.4

0.6

0.8

1

= J1(I)

I=J()

Fig. 3. The J function and its approximation where H1 = 0.3073, H2 =0.8935, and H3 = 1.1064.

REbyn(i) +w, where the variance of the zero-mean Gaus-

sian noise w is 2w = N0/2. The corresponding prior on yn(i)(calculated by M1 and fed to decoder C1n ) is [6]

A(yn(i)) = ln

(pG|X(r(i)|+1)pG|X(r(i)|1)

)

=4REbN0

r(i) =2u2

yn(i) + u, (11)

In (11), 2u = 8REb/N0 = 8Rb is the variance of the zero-mean Gaussian u = 4

REbN0

w [6]. The average MI [6] for thepriors in (11) is defined as

IA(yn) ,RnLn

Ln/Rni=1

I(yn(i);A(yn(i))), (12)

and can for a parallel code using BPSK be expressed as

IA(yn) = J(

8Rb

), (13)

using (5)(7). IA(yn) remains constant during decoding forall n = 1, 2, . . . , N in a parallel system. Note (13) is also theBPSK constellation-constrained capacity of an AWGN channel[6].

In a serial system, the priors A(yn) are the extrinsic valuesfrom the connecting decoder and hence are not Gaussian as in(11). However, simulation experiments have shown that usinga Gaussian assumption provides an acceptable approximationto the evolution of the true MIs [6]. Therefore, the Gaussianassumption is applied to all prior values, A(xn) and A(yn)for all n = 1, 2, . . . , N , in both the serial and the parallel case,

A(xn) =2x2

xn + sx, (14)

A(yn) =2y2

yn + sy. (15)Here, the elements in sx and sy are zero-mean Gaussian withvariances 2x and 2y , respectively. Using (6) and (12), theaverage MIs for the prior values in (14)(15) are expressedas IA(xn) = J(x) and IA(yn) = J(y), respectively.

B. Extrinsic Mutual InformationThe extrinsic average MIs for decoder n = 1, 2, . . . , N ,

IE(xn) and IE(yn), are expressed similar to (12) and dependon Cn. The extrinsic MIs are calculated using (5) and the con-ditional PDFs of the extrinsic log-likelihoods, pE|X(|xn(i))and pE|Y (|yn(i)). The conditional PDFs are usually esti-mated from histograms of E(xn) and E(yn), obtained byMonte Carlo simulations of decoder C1n for specific values of0 IA(xn) 1 and 0 IA(yn) 1 [6]. In these simulations,the prior values fed to the decoder are modelled according to(14)(15), where x = J1

(IA(xn)

)and y = J1

(IA(yn)

),

respectively.

C. Extrinsic Information Transfer FunctionsThe extrinsic MIs are functions, Txn : [0, 1]2 7 [0, 1] and

Tyn : [0, 1]2 7 [0, 1], of the prior MIs IA(xn) and IA(yn) [7],

IE(xn) = Txn(IA(xn), IA(yn)

), (16)

IE(yn) = Tyn(IA(xn), IA(yn)

). (17)

Note that Txn(0, 0) = Tyn(0, 0) = 0 and Txn(1, 1) =Tyn(1, 1) = 1. As an example, the EXIT functions (16)(17)for CC(2/3) are shown in Figs. 45.IA(yn) is equal to (13) if

REbyn is the transmitted BPSK

symbol as in Fig. 2. If the inner code (mapper M) is notBPSK, as in Fig. 1, the extrinsic MI depends on the mappingof bits to the M > 2 signal points in the constellation andalso on Rb,

IE(x1) = TM(IA(x1), Rb

). (18)

The EXIT functions (16)(18) are usually obtained by MonteCarlo simulations of the individual component codes using theGaussian model in (14) and (15) for the priors as describedabove [6]. More recently it has been shown that for certaincodes and simple channel models, it is possible to computethe EXIT functions [15, 16].

Average MI is not affected by an interleaver or a deinter-leaver. In a system with serially concatenated codes, the priorMIs for decoder n are therefore the extrinsic MIs from thedecoders interconnected with decoder n by an interleaver ora deinterleaver, IA(xn) = IE(yn+1), IA(yn) = IE(xn1), andIA(xN ) = 0 (refer to Section II-A and Fig. 1). From (16)(18)the following 2(N 1) mutually coupled EXIT functions canbe defined for a system with N serially concatenated codes

IE(x1) = TM(IE(y2), Rb

), (19)

IE(xn) = Txn(IE(yn+1), IE(xn1)

), (20)

IE(yn) = Tyn(IE(yn+1), IE(xn1)

), (21)

IE(yN ) = TyN(0, IE(xN1)

), (22)

for all n = 2, 3, . . . , N 1.In a system with parallel concatenated codes, IE(xn) =

IEn(x) and IA(xn) = IAn(x) (refer to Section II-B and Fig. 2).Since the prior values are sums of N 1 extrinsic values (3),they are modelled as sums of N 1 biased Gaussian random

500.2

0.40.6

0.81

00.2

0.40.6

0.810

0.2

0.4

0.6

0.8

1

I E(xn)=Txn

( I A(xn),IA

(yn)

)

IA(yn) IA(xn)

Fig. 4. EXIT function for the input bits of CC(2/3).

variables, (14). Using (7) and (8), the prior MI becomes [7]

IA(xn) = J

Ni=1i6=n

J1(IE(xi)

)2 . (23)

Usually, approximations (9) and (10) are sufficiently accurate.According to (16), (17), (23), and (13) the following N

mutually coupled EXIT functions can be defined for a systemwith N parallel concatenated codes using BPSK modulation

IE(xn) = Txn

J

Ni=1i6=n

J1(IE(xi)

)2, J

(8Rb

) ,

(24)for all n = 1, 2, . . . , N .

Define the average MI on the decision statistics in (2) and(4) as ID(x) , 1L

Li=1 I(x(i);D(x(i))). In the serial case,

ID(x) = IE(xN ) = TxN(0, IA(yN )

)= TxN

(0, IE(xN1)

),

(25)and in the parallel case,

ID(x) = J

N

i=1

J1(IE(xi)

)2 . (26)

ID(x) = 1.0 means full information about the source bits,i.e., BER close to zero. The average MI trajectory accordingto a specific decoder activation schedule may be predictedby repeated application of (19)(22) (serial system) or (24)(parallel system). With more than two components, it is not im-mediately obvious whether different activation schedules resultin different convergence behavior. Subject to a monotonicityassumption on (16)(18), (which seems reasonable for anyuseful code we suspect it always holds) it turns out that thelimit behavior is independent of activation schedule. This issummarized in Assumption 1 and Theorem 1 (proved in theAppendix).

00.2

0.40.6

0.81

00.2

0.40.6

0.810

0.2

0.4

0.6

0.8

1

I E(yn)=Tyn

( I A(xn),IA

(yn)

)

IA(yn) IA(xn)

Fig. 5. EXIT function for the output bits of CC(2/3).

Assumption 1 (Monotonicity): All EXIT functions aremonotonically non-decreasing, T

(IA(x) + x, IA(y) + y

) T(IA(x), IA(y)

), if x 0 and y 0.

Theorem 1: Subject to Assumption 1, the sequence of MIresulting from successive or parallel activations of EXIT func-tions converges monotonically to a limit point, independent ofthe actual activation schedule.Theorem 1 also implies that the MI on the decision statisticswill converge according to (25) and (26). The limiting value(independent of the activation schedule) is defined as theconvergence point ID(x) , ID.

IV. EXIT CHART PROJECTIONSIn an N = 2 component code, decoding alternates between

the two component decoders [6, 17]. EXIT charts can beused to predict the convergence point and the convergencethreshold. The convergence point corresponds to the lowestintersection of the EXIT curves [6]. The convergence thresholdis defined as the b at which a tunnel between the two curvesis opened so that the convergence point is close to one,ID = 1.0 [6].

For two component codes [6, 17], the EXIT chart is two-dimensional for a fixed b. In the serial case [17], the MIsare on the coded bits between the encoders, x1 = 1(y2),while in the parallel case [6] the MIs are on the source bitsx1 = 1(x) and x2 = 2(x). A vertical step in the EXITchart for a two-code system represents activation of decoderone, C11 . A horizontal step in the same EXIT chart representsactivation of decoder two, C12 .

In a system with N = 3 parallel components, there arethree different extrinsic values, E(x1), E(x2), and E(x3),connecting the decoders (cf. Fig. 2 where there are fourextrinsic values). According to (24) the EXIT chart is three-dimensional for fixed b [7]. The convergence threshold is nowthe b value that opens a tube between the three surfaces sothat the trajectory can go from IE(x1) = IE(x2) = IE(x3) = 0to IE(x1) = IE(x2) = IE(x3) = 1 [7].

For N parallel concatenated codes, there will be N extrinsicvalues connecting the decoders and the EXIT chart will be N -

6dimensional. In the special case when all parallel componentcodes are identical (symmetric codes), the EXIT chart can bevisualized in two-dimensions, IE(x1) versus IE(x2), by settingIE(xn) = IE(x2) for all 1 n N . The EXIT functions arethen [7]

IE(x1) = Tx1

(J(

N 1J1(IE(x2))), J(

8Rb

)),

IE(x2) = IE(x1).

In a system with N = 3 serially concatenated codes, thereare four extrinsic values, E(y3), E(x2), E(y2), and E(x1)connecting the three decoders and therefore four extrinsic MIs(19)(22) and Fig. 1. Hence, for a fixed b, the EXIT chart forthree serially concatenated codes is four-dimensional and hardto visualize. For a system with N serial components, the EXITchart will have 2(N 1) dimensions according to (19)(22).Even if several of the serial component codes are the same, thesimplification described above for symmetric parallel codes isnot possible.

Convergence analysis for N > 2 component codes (serialor parallel, symmetric or asymmetric) can however be ac-commodated by projecting the EXIT functions back onto twodimensions [12, 18]. The approach is as follows.

The convergence threshold in b is found when ID(x) =1.0. Since ID(x) = TxN

(0, IE(xN1)

)in a serial system (25),

it suffices to analyze the behavior of IV , IE(xN1) versusIH , IE(yN ). The indexes V and H respectively stand for thevertical and horizontal axes in the EXIT chart.

To obtain ID(x) = 1.0 (26) in a parallel system, at leasttwo constituent codes need to be recursive [19], as in theparallel example code. Without loss of generality, the twocodes with highest indexes, CN1 and CN , in a parallel systemare assumed to be recursive. The behavior of the whole systemcan then be determined by analyzing IV , IE(xN1) versusIH , IE(xN ). Note that the definition of IH depends onwhether the system is serial or parallel.

The convergence analysis can now be made with a two-dimensional EXIT chart, IV = TV(IH, Rb) (vertical axis)versus IH = TH(IV, Rb) (horizontal axis). These two transferfunctions can be found, for a specific R and b, by thefollowing procedure (let be a small constant):

Algorithm 1 (EXIT Chart Projections):1) Let the extrinsic MI for all codes be zero, and IH = 0.2) Activate all decoders except decoder N until IV has

converged to a fixed value and save IV = TV(IH, Rb).3) Let IH = IH + . If IH 1.0, go to Step 2.4) Let the extrinsic MI for all codes be zero and IV = 0.5) Activate all decoders except decoder N 1 until IH has

converged to a fixed value and save IH = TH(IV, Rb).6) Let IV = IV + . If IV 1.0, go to Step 5.7) Output the two EXIT functions: IV = TV(IH, Rb) from

Step 2 and IH = TH(IV, Rb) from Step 5.Step 2 and Step 5 use the EXIT functions in (19)(22)

and (24) to find the convergence values in the serial and theparallel case respectively. By Assumption 1 and Theorem 1 theactual activation schedule is irrelevant, as long as all N 1decoders are activated until no further gain is possible (see alsoAppendix). For example, the conventional activation schedule

0 0.2 0.4 0.6 0.8 10

0.2

0.4

0.6

0.8

1

IH = IE(y3), IE(y2)I V

=I E

(x2),I E

(x1)

IE(x1) = TM(IE(y2), Rb

)IE(x2) = TV

(IE(y3), Rb

)IE(y3) = Ty3

(0, IE(x2)

)

Fig. 6. EXIT chart projection of the serial example code at b = 2.0 dB.

1, 2, . . . , N 1, 1, 2, . . . , N 1, 1, 2, . . . can be used in Step2 in Algorithm 1, since it will give the same results comparedto any other chosen schedule that includes all decoders exceptdecoder N . The small value in Step 3 and 6 is chosenarbitrarily to give sufficient resolution of the final EXIT chart.

The projection is visualized by plotting TV(IH, Rb) versusIH in the same two-dimensional EXIT chart as IV versusTH(IV, Rb). A similar projection was independently devel-oped specifically for three serially concatenated codes in [10].

The solid curve in Fig. 6 is the EXIT function for CC(2, 3)used as an outer code in a serial system. According to(22), IH = IE(yN ) = TyN

(0, IE(xN1)

)= TyN (0, IV). This

shows that independent on how many codes that are seriallyconcatenated, IH is fixed (as in a serial system with two codes)and not depending on R or b. The dashed-dotted curve inFig. 6 is the EXIT function for the 8PSK mapper (19) inthe serial example code in Section II-C at b = 2.0 dB, [5].If these two codes were directly concatenated, making it asystem with two serially concatenated codes, no projectionwould be necessary. A vertical step between the solid curveand the dashed-dotted curve represents an activation of the8PSK demapper, while a horizontal step between the dashed-dotted curve and the solid curve represents an activation ofthe outer decoder. In this scenario, the performance wouldbe very poor due to the presence of a fixed point at aboutIE(x1) = 0.55 bits [6].

In the serial example code in Section II-C the CC(2/3) isinserted between the 8PSK mapper and the outer CC(2, 3).The complete EXIT chart is now four-dimensional (19)(22).By following the procedure of Algorithm 1 and using the EXITfunctions, Txn and Tyn , for the CC(2/3) shown in Figs. 45,the resulting projection is shown in Fig. 6 as the dashed curve.A vertical step between the solid curve and the dashed curve

70 0.2 0.4 0.6 0.8 10

0.2

0.4

0.6

0.8

1

IH = IE(x4)

I V=I E

(x3)

IE(x3) = TV(IE(x4), Rb

)IE(x4) = TH

(IE(x3), Rb

)

Fig. 7. EXIT chart projection of the parallel example code at b = 0.6dB.

represents an unspecified number of activations between theinner demapper and the intermediate decoder, until nothingmore can be gained. A horizontal step between the dashedline and the thick solid line represents a single activation ofthe outer decoder. This projected EXIT chart can thereforebe used to determine which points are attainable, but not thenumber of activations required. The convergence threshold isnow easily determined, in this example it is very close to 2.0dB.

Fig. 7 shows the projection of the parallel example codein Section II-C at b = 0.6 dB, which seems to be close tothe convergence threshold. The projection is made by using theEXIT functions, Txn and Tyn (similar to the ones in Figs. 45),for all four components. Although no systematic bits are used,decoding can successfully start since C3: CC(2/3) has a non-zero extrinsic MI IE(x3) 6= 0 when IA(y3) 6= 0 [12], which canbe concluded from Fig. 4. A vertical step between the solidcurve and the dashed curve in Fig. 7 represents activationsof C11 , C12 and C13 until nothing more can be gained. Ahorizontal step represents activations of C11 , C12 and C14 untilnothing more can be gained.

In a system with N codes, a vertical step represents anunspecified number of activations of all decoders except C1Nand a horizontal step represents activations of all decodersexcept C1N1. The convergence threshold, but not the numberof activations, can easily be determined for the two examplecodes by the projections in Figs. 67.

Note that a horizontal step in a serial system repre-sents a single activation of the outer decoder since none ofthe N 2 innermost decoders have any affect on IH =TyN(0, IE(xN1)

), as long as C1N1 is not activated. In a

parallel system, both IV and IH will depend on R and baccording to (24).

V. OPTIMAL SCHEDULINGIn a system with more than two components it is always

favorable to use the most recently updated information passedfrom all connected decoders [8, 19]. If all codes satisfy As-sumption 1, and all decoders are activated sufficiently manytimes, the same convergence point will be reached independentof activation schedule, according to Theorem 1. The particularschedule however, determines the cost of convergence. Here,the term cost can refer to computational complexity, number ofactivations, decoding delay, power consumptions or any othermeasure of choice.

There are many ways of choosing the activation schedule.One of the most commonly used periodic activation sched-ules for N concatenated codes is 1, 2, . . . , N, 1, 2, . . . , N, . . .(schedule A) [4, 7]. A similar periodic activation scheduleis 1, 2, . . . , N 1, N,N 1, . . . , 2, 1, 2, . . . (schedule B).Another suggestion is to activate the decoder for which thegain in MI is maximal [10].

It is of interest to reach the convergence point with thelowest possible cost. For example, let cn > 0 be a constantproportional to the computational complexity of decoder n,1 n N . We can now define the optimal decoderschedule, which reaches the convergence point with lowesttotal computational complexity c, [12, 18, 20]. This optimalschedule depends on the codes and on b. If cn = 1 for all1 n N , this schedule corresponds to using a minimumnumber of activations. Optimal decoder schedule here meansthat, based on the EXIT functions, no other decoder schedulecan reach the convergence point using less complexity.

All possible activation schedules can be described by anN -state trellis (one state for each decoder). The trellis forthe first six activations in a system with three concatenatedcodes is shown in Fig. 8. Let ak denote a state at activation k.Thus ak = n corresponds to using C1n at the k-th activation.There is a branch between every ak1 = n and ak = n for1 n 6= n N , (nothing is gained by repeated activation ofa decoder and no decoders are allowed to be activated at thesame time [8]).

A path (corresponding to a specific activation schedule)entering state ak = n is denoted by pk = (p1, p2, . . . , pk),where pj {1, 2, . . . , N} for 1 j k 1 and pk = n. Letv = (v1, v2, . . . , vF ) be the associated metric-vector1 whichhas as elements all the extrinsic MIs, the MI on the decisionstatistics, and the total decoding complexity, c, resulting fromthe activations along the path pk.

In a system with N serially concatenated codes there areF 2 = 2(N 1) different extrinsic MIs (19)(22), arrangedtogether with ID(x) and c in the following arbitrary order,

v =(ID(x), c, IE(x1), . . . , IE(xN1), IE(y2), . . . , IE(yN )

).

In a system with N parallel concatenated codes there are F 2 = N different extrinsic MIs (24) and

v =(ID(x), c, IE(x1), . . . , IE(xN )

).

In both cases, convergence to v1 = ID is desired. Element v2represents the total complexity, c, using activation schedule

1This is not a metric in a metric-space sense.

8k = 1

C11

C12

C13

bbbbbb

k = 2

C11

C12

C13

""""""

bbbbbb

k = 3

C11

C12

C13

bbbbbb

eeeeeee%

%%%%%%

""""""

k = 4

C11

C12

C13

bbbbbb

eeeeeee

""""""

bbbbbb%

%%%%%%

""""""

k = 5

C11

C12

C13

bbbbbb

eeeeeee

""""""

bbbbbb%

%%%%%%

""""""

k = 6

C11

C12

C13

. . .

. . .

. . .

Fig. 8. The trellis for the decoding schedule of three codes concatenated in serial (solid paths) or in parallel (solid and dashed paths).

pk,

v2 = c =

kj=1

cpj . (27)

For each state n define the metric update function fn : RF 7RF which produces a new metric-vector using the extrinsic

MIs and the total complexity (all included in v) as inputarguments.

In a system with three serially concatenated codes, thedecoding trellis has three states (see Fig. 8) and the metric-vector is six-dimensional,

v =(ID(x), c, IE(x1), IE(x2), IE(y2), IE(y3)

). (28)

The metric update functions are according to (25), (27) and(19)(22),f1(v) = (v1, v2 + c1, TM(v5, Rb) , v4, v5, v6) , (29)f2(v) = (v1, v2 + c2, v3, Tx2(v6, v3) , Ty2(v6, v3) , v6) , (30)f3(v) = (Tx3(0, v4) , v2 + c3, v3, v4, v5, Ty3(0, v4)) . (31)

The metric-vector in a system with four parallel concatenatedcodes is also six-dimensional

v =(ID(x), c, IE(x1), IE(x2), IE(x3), IE(x4)

). (32)

The metric update functions2 are similar to (29)(31), and use(26), (27), and (24) to update v.

Let Pk and Vk be the sets of all surviving paths/metricsafter k activations and let Pk,n Pk and Vk,n Vk be thesets of paths/metrics entering state n after k activations. Theexhaustive search proceeds as follows.

Algorithm 2 (Brute-force Exhaustive Search):1) Let k = 1. Initialize Pk to only include the N

paths Pk = {(1), (2), . . . , (N)}, and initialize Vk toonly include the corresponding metric-vectors Vk ={f1(0), f2(0), . . . , fN (0)}. Define p and v to be acandidate path/metric with a high initial complexity,v2 =.

2) Increment k with one. For each state n, extend everypath pk1 Pk1,n , with metric v, along each statetransition n n, 1 n 6= n N , producing thenew path pk = (pk1, n) which is added to Pk,n with

2In the Appendix, the first two elements of v are not needed and thusremoved. Similarly, the function fn(v) is only applied on the remainingelements. Despite this difference, the same notation is used.

the corresponding updated metric v = fn(v) added toVk,n.

3) Reduce the set Vk to only include metrics that have alower complexity than the complexity of the candidatepath, i.e., v Vk if v2 < v2 and reduce Pk accordingly.

4) Define a new set of metrics V containing all metricsv Vk that have reached the convergence point v1 =ID. If |V| 6= 0, find the metric with the lowest com-plexity, v = argminvV v2, and replace the candidatepath p by the path corresponding to v.

5) Go to Step 6.6) If |Vk| = 0, output p as the optimal path with a final

set of extrinsic MIs and a total complexity in v. If|Vk| 6= 0, go to Step 2.

In the serial case, it is sufficient to initialize Pk = {(1)}since the optimal path must start by activating the demapper.This means that the dashed paths in Fig. 8 can be removed fora system with three serially concatenated codes. The generalalgorithm above will in a serial case automatically remove allpaths that do not start with p1 = 1, since fn(0) will have zeroextrinsic MIs in v if n 6= 1 according to (20)(22). f1(0) isthe only metric update function in a serial system that willhave a non-zero extrinsic MI (v3 6= 0) since it includes theEXIT function of the innermost encoder (mapper) (19).

Using this brute force exhaustive search, the total number ofpaths |Vk| grows exponentially with k until the first candidatepath is found, and is therefore infeasible for large k.

To reduce the search complexity in Algorithm 2, where themetrics are F -dimensional, an adaptation of the well knownViterbi algorithm [21] can be used. In order to do this we needto define a partial order < on the metrics with the followingproperties: (a) the metrics are monotonically non-decreasing,and (b) deletion of paths entering the same state with smallermetrics according to < does not affect the final outcome.

Definition 1 (Domination): Define a partial ordering on RFas follows. For v,v RF , v < v if and only if vj vj ,for all 3 j F , and v2 v2. We say v dominates v if vhas all F 2 extrinsic MIs higher than v and a lower totalcomplexity.

The operator < does not operate on the first element ofv since that is the element the stopping criterion is basedon. Define the function dom, operating on a set of metricsby discarding all dominated metrics. Thus v Vk will bediscarded if and only if there is another metric Vk v < v.

9TABLE ITHE FIRST 30 ACTIVATIONS FOR DIFFERENT ACTIVATIONS SCHEDULES FOR THE PARALLEL EXAMPLE CODE AT b = 1.0 dB.

Optimal 3, 1, 2, 3, 1, 2, 3, 1, 4, 3, 2, 1, 3, 4, 2, 3, 1, 4, 3, 4, 3, 1, 2, 4, 1, 3, 2, 1, 3, 4 . . . A 1, 2, 3, 4, 1, 2, 3, 4, 1, 2, 3, 4, 1, 2, 3, 4, 1, 2, 3, 4, 1, 2, 3, 4, 1, 2, 3, 4, 1, 2 . . . B 1, 2, 3, 4, 3, 2, 1, 2, 3, 4, 3, 2, 1, 2, 3, 4, 3, 2, 1, 2, 3, 4, 3, 2, 1, 2, 3, 4, 3, 2 . . . C 3, 1, 2, 1, 3, 2, 1, 4, 3, 1, 2, 1, 3, 4, 1, 2, 3, 1, 4, 2, 3, 1, 2, 4, 1, 3, 2, 1, 3, 4, . . . D 1, 2, 3, 1, 2, 3, . . . , 1, 2, 4, 1, 2, 4, . . . , 1, 2, 3, 1, 2, 3, . . . , 1, 2, 4, 1, 2, 4, . . .

0 500 1000 1500 2000 2500 3000 3500107

105

103

101Optimal A: 1,2,3,... B: 1,2,3,2,... C: fix, 2.0 dB

Complexity

BER

2.0 dB

2.2 dB2.6 dB4.0 dB

Fig. 9. Performance in BER of the serial example code versus the totalcomputational complexity for different b.

V = domV only implies 1 |V| |V |.Under Assumption 1, (25)(26) and (27), the metric-vectors

are monotonically non-decreasing, v = fn(v) = v + ,for some 0 (see also Appendix). This implies thatthere exists only one convergence point v1 = ID, since theinitial value of the extrinsic MI is zero. Furthermore, supposethe metric v for a path pk is dominated by the metric v(v < v) for some other path pk entering the same staten at activation k, pk = pk = n. Then the metric for theextended path pk+1 = (pk, i), 1 i N , is fi(v) and themetric for the path pk+1 = (pk, i) is fi(v). Since v < v,Assumption 1 gives that fi(v) < fi(v) and we may thereforeremove pk from consideration (see also Appendix). Thus,Step 5 in Algorithm 2 can be modified to include deletionof dominated paths entering the same state, without affectingthe final outcome.

Algorithm 3 (Viterbi Search):Retain all steps from Algorithm 2, but change Step 5 to:

5) Delete dominated metrics, i.e., let Vk,n = domVk,n forall 1 n N and remove the corresponding pathsfrom Pk,n.

Algorithm 3 can be used as long as all codes have non-decreasing extrinsic MIs according to Assumption 1. Thisrequirement is easily checked by inspection of the EXITfunctions, e.g., the functions in Figs. 45. Algorithm 3 doesnot remove the possibility of exponential growth of the numberof retained paths, but we have observed in practice that thenumber of required paths is usually quite small.

0 100 200 300 400 500 600107

105

103

101Optimal A: 1,2,3,4,... B: 1,2,3,4,3,2,... C: fix, 0.6 dB

Complexity

BER

1.0 dB

0.0 dB-0.4 dB

-0.6 dB

Fig. 10. Performance in BER of the parallel example code versus the totalcomputational complexity for different b.

VI. NUMERICAL EXAMPLES

The simulated BER performance of the two examples inSection II-C versus total complexity is shown in Figs. 910for different b. In the simulations, the source bits are dividedinto blocks of size L = 105. Activation schedule C is fixed tothe optimal activation schedule at the convergence threshold(2.0 dB in the serial example code and 0.6 dB in the parallelexample code). This schedule is found using the Algorithm 3.Schedules A, B, and C are fixed for all b in contrast to theoptimal schedule. Unfortunately, no general structure has beenfound for the optimal activation schedule as can be concludedfrom Table I. This schedule depends on the EXIT functionsof all codes in the system and on b.

The complexity values in the serial example code usedin Fig. 9 (proportional to the decoding time) are c1 = 54,c2 = 30, and c3 = 17. The convergence threshold for thissystem is around 2.0 dB, as predicted by Fig. 6. From Fig. 9it can be concluded that for all b 2.2, 30 60% indecoding complexity can be saved by choosing the optimalschedule instead of schedule B, while 3040% can be savedcompared to schedule A, for all BER < 102. Fig. 9 alsoshows that using the fixed activation schedule C for all valuesof b will give almost the same complexity as if the optimalschedule was chosen. From Fig. 9 it is obvious that the sameperformance will eventually be reached independent of theactivation schedule of the decoders.

In the parallel example, the component codes have the samerate and structure so the decoding complexity is proportionalto the number of states, c1 = 2, c2 = 4, c3 = 2, and c4 =

10

0 0.2 0.4 0.6 0.8 10

0.2

0.4

0.6

0.8

1

IH = IE(x4)

I V=I E

(x3)

IE(x3) = TV(IE(x4), Rb

)IE(x4) = TH

(IE(x3), Rb

)

Fig. 11. EXIT chart projection of the parallel example code at b = 1.0dB, together with the average decoding trajectory for different activationschedules. The markers correspond to the five schedules given in Table I.

8. Fig. 10 shows that for this parallel system, 25 50% indecoding complexity can be saved by choosing the optimalschedule instead of schedule B and 10 20% can be savedcompared to schedule A. The fixed schedule C seems to besomewhere between the optimal schedule and schedule A. Toillustrate the difference between the activation schedules, allschedules for the parallel example code at b = 1.0 dB aregiven in Table I. In Fig. 10, where the BER reach equilibriumafter many activations, it is even more obvious than in Fig. 9that the same performance will always be reached independentof the chosen activation schedule of the decoders.

Fig. 11 shows the EXIT chart projection of the parallelexample code at b = 1.0 dB. The average decoding trajectoryusing the same activation schedules as in Fig. 10 and Table Iare also included. Schedule D (marked with diamonds) isone of the schedules that reach the curves in the projectedEXIT chart, as explained in Section IV. The average decodingtrajectories in Fig. 11 are all based on the EXIT functionson the component codes involved. Fig. 11 shows clearly thatdifferent schedules give different average decoding trajectory,that all lie within the two curves in the projected EXIT chart.It also illustrates that the same convergence point is reachedindependent of decoding schedule, which is also concludedfrom Fig. 10.

The savings in percentage stated here are just examples onhow much can be saved by choosing the optimal scheduleinstead of some other activation schedule. For other combi-nations of component codes than the ones used in these twoexamples the savings can be more or less.

VII. CONCLUSIONWe considered the problem of iterative decoding of multiple

concatenated codes with more than two components. For suchcodes, the activation schedule of the component decoders isnot obvious. Furthermore, EXIT chart analysis is not straight-forward.

We have proposed a technique for projecting several three-dimensional EXIT functions onto two dimensions in order todetermine the convergence threshold for any arbitrary number,N , of multiple concatenated codes. This projection can now beused as a design tool to find combinations of codes that givea desired performance. We have also described a Viterbi-likealgorithm that finds the activation schedule for the componentdecoders, giving the lowest possible decoding complexity.

The only requirement to perform both the projection and tofind the optimal activation schedule is that the EXIT functionsof the component codes, IE(xn) = Txn

(IA(xn), IA(yn)

)and

IE(yn) = Tyn(IA(xn), IA(yn)

), are known and that they are

monotonically non-decreasing.The results from the two example codes presented here show

that 1060% in decoding complexity can be saved by choos-ing the optimal schedule compared to some fixed activationschedule. The decoding complexity can be substituted withdecoding delay or power consumption depending on how theconstants in the search algorithm are chosen.

The results also show that instead of using the periodicactivation schedule 1, 2, . . . , N, 1, 2, . . . , N, . . . the decodercomplexity can be reduced if the optimal activation scheduleat the convergence threshold is used for all b. In fact,in the serial example code, this activation schedule givessimilar performance as the optimal activation schedule (whichdepends on b).

From these two examples it is obvious that the computa-tional complexity of iterative decoding can be substantiallyreduced, without affecting the performance, by wisely choos-ing the decoding schedule.

APPENDIXLet v = (v1, v2, . . . , vN ) [0, 1]N , be an N -vector with

elements 0 vn 1. By assumption, let the functions gn :[0, 1]N 7 [0, 1], n = 1, 2, . . . , N each satisfy

v v = gn(v) gn(v) (33)where henceforth for vectors, v v means vn vn, n =1, 2, . . . , N .

Define f : [0, 1]N 7 [0, 1]N and fn : [0, 1]N 7 [0, 1]N , forn = 1, 2, . . . , N according to

f(v) = (g1(v), g2(v), . . . , gN (v)) (34)fn(v) = (v1, . . . , vn1, gn(v), vn+1, . . . , vN ) . (35)

Note that the assumption (33) implies the relationsv v = f(v) f(v) (36)

and

v v = fn(v) fn(v). (37)

11

Define a region f as

f , {v : v f(v)} . (38)The region in (38) is referred to as the feasible region of f .The definitions in (34), (35), and (38) together with assumption(33) now imply that,

f(v) fn(v), if v f , (39)for any n = 1, 2, . . . , N .

Given a sequence pK = (p1, p2, . . . , pK), where pn {1, 2, . . . , N} and an initial point v0 = s0 f , define thefollowing two sequences for k = 1, 2, . . . ,K,

vk =(vk1 , v

k2 , . . . , v

kN

), f(vk1) (40)

sk =

(uk1 , u

k2 , . . . , u

kN

), fpk(s

k1). (41)Thus, vk is a sequence with parallel, or synchronousupdates and sk is a sequence with serial, or asynchronousupdates [22]. The sequence pK defines the update order forthe serial sequence.

Lemma 1: Let v0 f . Then the sequence vk definedby (40) converges monotonically (in the sense vk vk1) toa unique limit point 0 v 1.

Proof: Since f 1, the lemma results from showingthat vk is monotonically non-decreasing. This will be accom-plished by induction. By assumption, v0 f and

v1 = f(v0) by (40) v0 by (38)

Now supposevk vk1 (42)

Then

vk = f(vk1) by (40) f(vk) by (42) and (36)= vk+1 by (40).

Since (42) holds for k = 1, it holds for all k > 1 by induction.

Henceforth, letv = lim

kvk, (43)

the existence of which is guaranteed by the previous lemma.Lemma 2: Let s0 = v0 f and suppose the sequence pK

is such that for any integer M > 0, there exists a KM Ksuch that each integer 1, 2, . . . , N appears at least M times inthe subsequence pKM = (p1, p2 . . . , pKM ). Then the sequencesk defined by (41) converges monotonically to v.

Proof: Showing that(i) vk sk,

(ii) vk sKk ,for k 1, directly prove the lemma.

By assumption, s0 = v0 f andv1 = f(v0) = f(s0) by (40) fp1(s0) by (39)= s1 by (41)

for any p1 = 1, 2, . . . , N . Suppose

vk sk. (44)Then

vk+1 = f(vk) by (40) f(sk) by (44) and (36). fpk+1(sk) by (39)= sk+1. by (41)

Since (44) holds for k = 1, it holds for all k > 1 by induction,which is part (i).

For part (ii), note that v1 sK1 , since on the left eachelement is updated once, in parallel by the corresponding gn,whereas on the right, each element is updated serially, at leastonce (by the assumption on pK1):

u1p1 = gp1(s0) = gp1(v

0) = v1p1u2p2 = gp2(s

1) gp2(v0) = v1p2.

.

.

It is easy to see that this domination is preserved, vk sKk ,k > 1.

These two lemmas show that both sequences (40) and (41)have the same limit point. Note that this limit point is afunction of the starting point, which was chosen as v0 =s0 f . Different starting points within the feasible region

f could result in different limit points. In fact, selectinga starting point outside the feasible region f can result indifferent limit points for the two sequences.

Using the notation above, the decoding trajectory for aconcatenated code with parallel components can be describedusing (24) as follows:

v , (v1, v2, . . . , vN ) =(IE(x1), IE(x2), . . . , IE(xN )

),

gn(v) , Txn

J

Ni=1i6=n

J1(vi)2

, J

(8Rb

) .

Assumption (33) can be guaranteed if the monotonicity as-sumption of the EXIT functions holds (Assumption 1) togetherwith the knowledge that both the J-function and its inverseis monotonically increasing [6], i.e., J( + ) > J() andJ1(IG + ) > J1(IG), for any > 0.

The decoding trajectory for a concatenated code with serialcomponents can be described in a similar way, where gn(v)is then given by (19)(22). In this case, each activation canupdate one or two elements in v. However, it is straightforwardto redefine fn(v) to be a function that updates several elementsin v and to prove that Lemma 2 is still valid.

Theorem 1 results from direct application of the abovelemmas as follows.

Theorem 1: (restated) Subject to Assumption 1, the se-quence of MI resulting from successive or parallel activationsof EXIT functions converges monotonically to a limit point,independent of the actual activation schedule.

Proof: The elements in v are MIs, and hence 0 v 1.Further, since v0 = 0 and f(0) 0, v0 = 0 is inside the

12

feasible region f defined in (38). This together with Lemma1 and 2 complete the proof of Theorem 1.

REFERENCES[1] C. Berrou, A. Glavieux, and P. Thitimajshima, Near Shannon limit

error-correcting coding and decoding: Turbo-codes, in Proc. IEEEInt. Conf. Commun. (ICC 93), vol. 2, Geneva, Switzerland, May 1993,pp. 10641070.

[2] S. Benedetto and G. Montorsi, Serial concatenation of block andconvolutional codes, IEE Electron. Lett., vol. 32, no. 10, pp. 887888,May 1996.

[3] D. Divsalar and F. Pollara, Multiple turbo codes for deep-space com-munications, TDA Progress Report 42-121, Jet Propulsion Laboratory,Pasadena, CA, May 1995.

[4] S. Benedetto, D. Divsalar, G. Montorsi, and F. Pollara, Analysis,design, and iterative decoding of double serially concatenated codes withinterleavers, IEEE J. Selected Areas Commun., vol. 16, pp. 231244,Feb. 1998.

[5] S. ten Brink, Convergence of iterative decoding, IEE Electron. Lett.,vol. 35, pp. 11171119, June 1999.

[6] , Convergence behavior of iteratively decoded parallel concate-nated codes, IEEE Trans. Commun., vol. 49, pp. 1727 1737, Oct.2001.

[7] , Convergence of multi-dimensional iterative decoding schemes,in Proc. Thirty-Fifth Asilomar Conference on Signals, Systems andComputers, vol. 1, Pacific Grove, CA, Nov. 2001, pp. 270274.

[8] J. Han and O. Y. Takeshita, On the decoding structure for multiple turbocodes, in Proc. IEEE Int. Symp. Inform. Theory (ISIT 01), Washington,DC, June 2001, p. 98.

[9] S. Benedetto, D. Divsalar, G. Montorsi, and F. Pollara, A soft-inputsoft-output APP module for iterative decoding of concatenated codes,IEEE Commun. Lett., vol. 1, pp. 2224, Jan. 1997.

[10] M. Tuchler, Convergence prediction for iterative decoding of threefoldconcatenated systems, in Proc. IEEE Global Commun. Conf. (GLOBE-COM 02), vol. 2, Taipei, Taiwan, Nov. 2002, pp. 13581362.

[11] F. Brannstrom, T. M. Aulin, L. K. Rasmussen, and A. J. Grant,Convergence analysis of iterative detectors for narrow-band multipleaccess, in Proc. IEEE Global Commun. Conf. (GLOBECOM 02),vol. 2, Taipei, Taiwan, Nov. 2002, pp. 13731377.

[12] F. Brannstrom, Convergence analysis and design of multiple concate-nated codes, Ph.D. dissertation, Chalmers Univ. of Techn., Goteborg,Sweden, Mar. 2004.

[13] T. M. Cover and J. A. Thomas, Elements of Information Theory. NewYork, NY: Wiley, 1991.

[14] J. A. Nelder and R. Mead, A simplex method for function minimiza-tion, The Computer Journal, vol. 7, pp. 308313, 1965.

[15] A. Ashikhmin, G. Kramer, and S. ten Brink, Extrinsic informationtransfer functions, information functions, support weights, and duality,in Proc. Int. Symp. on Turbo Codes and Rel. Topics, Brest, France, Sept.2003, pp. 223226.

[16] , Extrinsic information transfer functions: model and erasurechannel properties, IEEE Trans. Inform. Theory, vol. 50, pp. 26572673, Nov. 2004.

[17] S. ten Brink, Design of serially concatenated codes based on itera-tive decoding convergence, in Proc. Int. Symp. on Turbo Codes andRel. Topics, Brest, France, Sept. 2000, pp. 319322.

[18] F. Brannstrom, L. K. Rasmussen, and A. Grant, Optimal schedulingfor multiple serially concatenated codes, in Proc. Int. Symp. on TurboCodes and Rel. Topics, Brest, France, Sept. 2003, pp. 383386.

[19] S. Huettinger and J. Huber, Design of multiple-turbo-codes withtransfer characteristics of component codes, in Proc. Conf. Inform. Sci-ences and Syst. (CISS 02), Princeton University, Mar. 2002.

[20] F. Brannstrom, L. K. Rasmussen, and A. Grant, Optimal scheduling foriterative decoding, in Proc. IEEE Int. Symp. Inform. Theory (ISIT 03),Yokohama, Japan, June/July 2003, p. 350.

[21] G. D. Forney Jr., The Viterbi algorithm, Proc. IEEE, vol. 61, pp.268278, Mar. 1973.

[22] D. P. Bertsekas and J. N. Tsitsiklis, Parallel and Distributed Computa-tion. Englewood Cliffs, NJ: Prentice-Hall, 1989.