PAMFundamentals - nt.tuwien.ac.at€¦ · signaling bandlimited transmit pulse. Note that, r fo Tb...

2 PAM Fundamentals

Contents

2.1 Introduction and Outline . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18

2.2 Baseband PAM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18

2.2.1 Transmitter and Transmit Signal . . . . . . . . . . . . . . . . . . . . . . . . . . 19

2.2.2 Mean, Power Spectrum, and Mean Power of Transmit Signal . . . . . . . . . . 22

2.2.3 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

2.3 Passband PAM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28

2.3.1 Transmitter and Transmit Signal . . . . . . . . . . . . . . . . . . . . . . . . . . 28

2.3.2 Mean, Power Spectrum, and Mean Power of Transmit Signal . . . . . . . . . . 30

2.3.3 Symbol Alphabet Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33

2.3.4 Spectral Efficiency . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38

2.3.5 Spread Spectrum Modulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38

2.3.6 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41

17

18 Chapter 2. PAM Fundamentals

2.1 Introduction and Outline

This chapter discusses pulse amplitude modulation (PAM), which is the most elementary digital mod-

ulation scheme. PAM comes in two flavors, baseband PAM and passband PAM , which define the

topics of the two main sections of this chapter. We will concentrate on passband PAM because it is

predominant in practice. Our discussion in this chapter is limited to the PAM transmitter and the

PAM transmit signal; a discussion of the effects of the channel and the design of PAM receivers is

deferred to Chapters 3–6.

This chapter is organized as follows.

• Section 2.2 considers a baseband PAM transmitter and the transmit signal produced by it.

The mean, power spectral density, and mean power of the baseband PAM transmit signal are

calculated. This discussion provides a basis for our subsequent discussion of passband PAM.

• Section 2.3 considers a passband PAM transmitter and the passband transmit signal. The mean,

power spectral density, and mean power of the passband PAM transmit signal are calculated.

From expressions for the mean power, some simple rules for the design of the symbol alphabet

(signal constellation) are derived. The spectral efficiency of passband PAM is calculated. Finally,

the special case of spread spectrum modulation is briefly studied.

Throughout these lecture notes, the Fourier transform of a continuous-time signal x(t) is defined

as

X(jω) ,

∫ ∞

−∞x(t) e−jωtdt , (2.1)

with inverse transform x(t) = 12π

∫∞−∞X(jω) ejtω dω. The Fourier transform of a discrete-time signal

x[n] is defined as1

X(ejθ) ,

∞∑

n=−∞x[n] e−jθn, (2.2)

with inverse transform x[n] = 12π

∫ π−πX(ejθ) ejnθdθ.

2.2 Baseband PAM

We first consider the case of baseband PAM, where the transmit signal is a real-valued lowpass signal

whose spectrum is centered about DC (i.e., frequency 0).

We begin our discussion with some elementary but fundamental considerations. A binary word

or sequence (b1, b2, · · ·, bL) of length L, i.e., L bits bj ∈ 0, 1 (j = 1, 2, · · ·, L) can take on M = 2L

different “values.” Conversely, if we want to code any one of M different messages by a corresponding

binary word (b1, b2, · · ·, bL) of fixed length L, then the minimum wordlength is L = ⌈ldM⌉, i.e., thesmallest integer ≥ ldM (here, ld , log2). That is, we need L = ⌈ldM⌉ bits to code or represent

1With an abuse of notation, we use the same symbol X for both Fourier transforms.

2.2 Baseband PAM 19

M messages. For example, we need ⌈ld10⌉ = 4 bits to code a number taken from an alphabet of 10

numbers.

In digital communications, we have to transmit a message selected from a finite set of M different

messages, each of which can be represented by a corresponding sequence of bits bj. (Note that M can

be extremely large.) In accordance with Figure 1.1, we assume that the bj’s are the bits at the output

of the channel encoder or, equivalently, at the input of the modulator. Thus, digital modulation means

the mapping of M = 2L different bit sequences b(i)j of length L to corresponding analog signals s(i)(t)

(here, i = 1, 2, · · ·,M is the message index ).

2.2.1 Transmitter and Transmit Signal

We assume that at the input of the modulator, one bit arrives every Tb seconds. Thus, the jth bit bj

corresponds to time jTb. The bit rate is

Rb =1

Tb[bit/s] . (2.3)

Baseband PAM with binary signaling. To represent bj by a signal, we use a pulse g(t − jTb)

obtained by shifting an elementary real-valued transmit pulse g(t) to time jTb. The total transmitted

signal s(t) is formed by multiplying g(t− jTb) by bj (where, for reasons to be explained in Subsection

2.2.2, we will assume bj ∈ −1, 1 instead of bj ∈ 0, 1) and adding all the resulting pulses, i.e.,

s(t) =L∑

j=1

bj g(t−jTb) , bj ∈ −1, 1 . (2.4)

Since each bit bj determines the amplitude of the associated pulse g(t−jTb), the modulation technique

defined by (2.4) is known as pulse amplitude modulation (PAM). An example of the transmit signal

s(t) is shown in part (a) of Figure 2.1. In this example, the transmit pulse g(t) is a rectangular pulse

of duration Tb,

g(t) = rect

(t;Tb2

),

1 , |t| < Tb

2

0 , |t| > Tb2 .

(2.5)

Note that here the shifted transmit pulses g(t − jTb) do not overlap. Thus, as is easily verified, the

bit sequence bj can be recovered from s(t) simply by sampling s(t) at times jTb, i.e., bj = s(jTb).

A problem with the rectangular transmit pulse is that it is not bandlimited, and therefore the

resulting transmit signal s(t) is not bandlimited either. For communications over a bandlimited

channel (such as the telephone/voiceband channel or a wireless link), we need a bandlimited pulse

g(t). Since bandlimitation is not compatible with finite time duration, g(t) must have infinite duration

and thus two pulses g(t − iTb) and g(t − jTb) associated with different bit locations iTb and jTb will

generally overlap. How, then, are we going to separate these pulses at the receiver? This question will

be answered in Section 3.3. Figure 2.1(b) shows the transmit signal s(t) for a bandlimited transmit

pulse g(t).

Baseband PAM with higher-level signaling. As will be made clear later (see Subsection 3.3.1),

the maximum rate with which pulses g(t − jTb) can be transmitted over a bandlimited channel such


1

t

Ts=Tb

1

t

−1

Ts=Tb

3

1

t

−1

−3

Ts=2Tb

−1

3

1

t−1

−3

Ts=2Tb

(a)

(b)

(c)

(d)

Figure 2.1: Baseband PAM transmit signals: (a) Binary signaling using a re tangular transmit pulse,

(b) binary signaling using a bandlimited transmit pulse, ( ) higher-level signaling (symbol alphabet A =−3,−1, 1, 3) using a re tangular transmit pulse, and (d) higher-level signaling using a bandlimited transmitpulse. Note that, for Tb xed, the time s ale in parts ( ) and (d) is dierent from that in parts (a) and (b).

that, in a specific sense, the pulses can be separated at the receiver is limited by the channel bandwidth.

Since for our binary modulation scheme (2.4) the pulse rate equals the bit rate, this means that the

bit rate is limited by the channel bandwidth.

Therefore, we will now generalize our PAM format as follows. We combine blocks of l consecutive

bits bj into higher-level (i.e., nonbinary) “symbols” a[k] that can take on Ma = 2l different real

values/levels, i.e.,

a[k] ∈ A =a(1), a(2), · · ·, a(Ma)

with Ma = 2l . (2.6)

2.2 Baseband PAM 21

Here, A is called the symbol alphabet. Each symbol carries l = ldMa bits. For example, for l = 2 (and,

hence, Ma = 4) we could have A = −3,−1, 1, 3. Then, we do not transmit the bits bj directly but

we transmit the associated sequence of symbols a[k]. That is, our PAM transmit signal is now given

by

s(t) =

K∑

k=1

a[k] g(t−kTs) , a[k] ∈ A =a(1), a(2), · · ·, a(Ma)

, (2.7)

where Ts is the symbol period . Figure 2.1(c),(d) shows possible transmit signals s(t) for the symbol

alphabet A = −3,−1, 1, 3.If we transmit K Ma-ary symbols a[k] according to (2.7), there are M =MK

a different signals s(t)

(and, hence, different messages), corresponding to a total number L = ldM = ldMKa = K ldMa of

transmitted bits.

Since l = ldMa consecutive bits bj were combined into a symbol a[k], the symbol period is

Ts = lTb = (ldMa)Tb , (2.8)

and thus the symbol rate is reduced by a factor of l as compared to the bit rate:

Rs =1

Ts=

1

lTb=

Rb

l=

Rb

ldMa, Rb = lRs = (ldMa)Rs . (2.9)

According to (2.7), the symbol rate Rs is equal to the pulse rate, which is limited by the channel

bandwidth. Hence, for a given channel bandwidth, using Ma-ary symbols instead of the original bits

we can transmit these bits l = ldMa times faster. Whereas the symbol rate (= pulse rate) Rs is still

limited by the channel bandwidth, the bit rate Rb = lRs can in principle be made arbitrarily large by

choosing l (and, thus, Ma = 2l) sufficiently large. (In practice, however, Ma is limited by the power

constraint and the noise of the channel—cf. Subsection 2.3.3.) We note that the “physical unit” for

the symbol rate Rs is symbol/s or baud2.

Bits-to-symbols mapping and Gray code. If we transmit symbols instead of the raw bits, then

the transmitter must perform a mapping by which a block of l consecutive bits bj is converted into

a symbol a[k]. An inverse operation (demapping) must then be performed by the receiver in order

to convert the symbol resulting from the receiver’s decision into bits. What happens if the receiver’s

symbol decision is wrong, i.e., if there occurs a symbol error? How many of the resulting bits will be

wrong? We shall see later that for the Gaussian channel the most likely symbol errors result in symbols

that are nearest neighbors of the symbol transmitted. For example, if we use PAM with symbol (=

amplitude) alphabet A = −3,−1, 1, 3 and if the symbol −1 was transmitted, then the most likely

symbol errors produce the neighboring symbol −3 or 1 whereas the symbol 3 is very unlikely (see

Figure 2.2). This motivates the use of a Gray code for the bits-to-symbols mapping. For a Gray

code, binary words corresponding to neighboring symbols differ in exactly one bit, which means that

a typical symbol error results in only one bit error .

2After Jean-Maurice-Emile Baudot (1845–1903), French engineer, inventor of a telegraphy system widely used in

Baudot’s time.


01 11 1000a

−1−3 1 3

Figure 2.2: PAM symbol alphabet A = −3,−1, 1, 3 and orresponding binary words a ording to a Gray

ode.

Mapping g(t)a[k] ∈ A

(k = 1, 2, · · ·,K)

s(t) =K∑

k=1

a[k] g(t−kTs)bj ∈ 0, 1(j = 1, 2, · · ·, L)

Figure 2.3: Blo k diagram of a baseband PAM transmitter in luding the bits-to-symbols mapping. (Stri tly

speaking, the input to the transmit lter is the Dira impulse train

∑Kk=1 a[k] δ(t − kTs). With pra ti al

PAM modems, the transmitter is usually implemented in dis rete time and thus Dira impulses are not

needed.)

A schematic block diagram of the baseband PAM transmitter including the bits-to-symbols map-

ping is shown in Figure 2.3. Here, the shifted transmit pulses g(t− kTs) are generated by a transmit

filter whose impulse response is g(t). The transmit filter is also known as a pulse shaper .

2.2.2 Mean, Power Spectrum, and Mean Power of Transmit Signal

For any modulation technique, statistical quantities like the mean, mean power, and power spectrum

(power spectral density) of the transmit signal s(t) are important properties. To calculate these

quantities for baseband PAM, we assume that the symbols A[k] are random (note that we will usually

denote random variables by upper-case letters, e.g., we write A[k] instead of a[k] and S(t) instead

of s(t)). More specifically, we assume that the symbol sequence A[k] (−∞ < k < ∞) constitutes

a wide-sense stationary, discrete-time random process with mean3 µA = EA[k], autocorrelation

RA[l] = EA[k]A[k − l], and power spectrum

SA(ejθ) =

∞∑

l=−∞RA[l] e

−jθl . (2.10)

Stationarized process. Even though the symbol process A[k] is stationary, the resulting PAM

process

S(t) =∞∑

k=−∞A[k] g(t−kTs) (2.11)

is not stationary but cyclostationary with period Ts. Thus, the mean and autocorrelation are Ts-

periodic functions of t. For example, the mean of S(t) is obtained as

µS(t) = ES(t) = E

∞∑

k=−∞A[k] g(t−kTs)

=

∞∑

k=−∞EA[k]︸︷︷︸

µA

g(t−kTs) = µA

∞∑

k=−∞g(t−kTs) ,

which evidently is a Ts-periodic function. (Note that usually µA = 0, whence also µS(t) ≡ 0.)

3Here, E· denotes the expectation operator.

2.2 Baseband PAM 23

It is both theoretically convenient and practically desirable to consider a “stationarized” process

S(t) that is wide-sense stationary and thus possesses a power spectral density. We can construct S(t)

by introducing a random time offset T that is uniform in [0, Ts) and statistically independent of the

symbols A[k]. (Note that this stationarization procedure is practically justified because it eliminates

the influence of the location of the time origin, which is arbitrary anyway.) Hence, the new PAM

process is

S(t) , S(t−T ) =∞∑

k=−∞A[k] g(t−T−kTs) . (2.12)

Mean, autocorrelation, and power spectrum. The mean of the stationarized PAM process S(t)

is now obtained as

µS(t) = E

∞∑

k=−∞A[k] g(t−T−kTs)

=∞∑

k=−∞EA[k]︸︷︷︸

µA

Eg(t−T−kTs)︸︷︷︸1Ts

∫ Ts0 g(t−τ−kTs) dτ

= µA1

Ts

∞∑

k=−∞

∫ kTs+Ts

kTs

g(t−τ) dτ

= µA1

Ts

∫ ∞

−∞g(t−τ) dτ

=1

TsµAG(0) , (2.13)

with G(0) =∫∞−∞ g(t) dt. Thus, the mean of S(t) is constant. In a similar manner, one can show that

the autocorrelation of S(t) does not depend on t and is given by

RS(τ) = ES(t) S(t−τ)

=

1

Ts

∞∑

l=−∞RA[l] rg(τ−lTs) , (2.14)

where

rg(τ) ,

∫ ∞

−∞g(t) g(t−τ) dt (2.15)

is the temporal autocorrelation of the pulse g(t). Since µS and RS(τ) do not depend on t, S(t) is

wide-sense stationary. Taking the Fourier transform of RS(τ), we finally obtain the power spectrum

of the stationarized baseband PAM process as

SS(jω) =

∫ ∞

−∞RS(τ) e

−jωτ dτ = ... =1

TsSA(e

jωTs) |G(jω)|2 , (2.16)

with SA(ejθ) as defined in (2.10). Note that the factor SA(e

jωTs) describes the influence of the symbol

statistics while the factor |G(jω)|2 —the energy spectrum (energy spectral density) of g(t)—describes

the influence of the transmit pulse. Furthermore note that SS(jω) does not depend on the phase

spectrum of g(t).


White symbol sequence. A practically important special case arises when the stationary symbol

process A[k] is white, i.e., when two different symbols A[k], A[k′] are uncorrelated:

CA[l] , E(A[k]−µA)(A[k−l]−µA)

= σ2A δ[l] . (2.17)

In particular, this will always be the case if the (random) bits Bj at the modulator input are statistically

independent (recall from Subsection 2.2.1 that disjoint blocks of bits Bj are mapped onto the symbols

A[k]). With the general relation RA[l] = CA[l] + µ2A, we here obtain

RA[l] = σ2A δ[l] + µ2A (2.18)

and, taking the discrete-time Fourier transform,

SA(ejθ) = σ2A + µ2A 2π

∞∑

i=−∞δ(θ−i2π) . (2.19)

Inserting in (2.16), we obtain for the power spectrum of the stationarized transmit signal

SS(jω) =1

Ts

[σ2A + µ2A 2π

∞∑

i=−∞δ(ωTs−i2π)

]|G(jω)|2

=1

Tsσ2A |G(jω)|2 +

1

T 2s

µ2A 2π

∞∑

i=−∞

∣∣∣∣G(ji2π

Ts

)∣∣∣∣2

δ

(ω− i2π

Ts

),

where in the last step we have used the relation δ(aω) = 1|a| δ(ω). Assuming that the bandwidth of

g(t) is < 2π/Ts, which is usually satisfied in practice, we have G(ji2πTs

)= 0 for i 6= 0 and thus

SS(jω) =1

Tsσ2A |G(jω)|2 +

1

T 2s

µ2A 2π |G(0)|2 δ(ω) . (2.20)

Hence, the power spectrum of S(t) consists of a component that is proportional to σ2A and |G(jω)|2,and a Dirac impulse located at ω = 0. The Dirac impulse is usually undesirable; it can easily be

avoided by making the symbols zero-mean, i.e., enforcing µA = 0 (see also below).

Mean signal power and zero-mean symbols. In many applications, transmit signal power is

costly and thus one desires to transmit the data (symbols) with minimum signal power. Still assuming

a white symbol sequence, we obtain the mean power of the PAM process S(t) by integrating SS(jω)

in (2.20) over ω:

PS , ES2(t)

=

1

2π

∫ ∞

−∞SS(jω) dω

=1

Tsσ2A

1

2π

∫ ∞

−∞|G(jω)|2 dω +

1

T 2s

µ2A |G(0)|2∫ ∞

−∞δ(ω) dω

=1

Tsσ2AEg +

1

T 2s

µ2A |G(0)|2, (2.21)

where

Eg ,

∫ ∞

−∞|g(t)|2 dt =

1

2π

∫ ∞

−∞|G(jω)|2 dω (2.22)

2.2 Baseband PAM 25

is the energy of the transmit pulse g(t). This shows that PS consists of a term that is proportional

to the symbol variance σ2A and the transmit pulse energy Eg, and a term that is proportional to the

squared symbol mean µ2A. Since the latter term is nonnegative, we conclude that in order to obtain

small signal power we must choose µA = 0, i.e., zero-mean symbols. The expressions for SA(ejθ),

SS(jω), and PS here simplify to

SA(ejθ) ≡ σ2A (2.23)

SS(jω) =1

Tsσ2A |G(jω)|2 (2.24)

PS =1

Tsσ2A Eg . (2.25)

Note that according to (2.24), for zero-mean and white symbols the power spectrum SS(jω) equals

the energy density spectrum of the pulse g(t) up to a constant factor.

2.2.3 Problems

Problem 2.2.1. Consider a baseband PAM system with Ma = 256 symbol amplitudes and symbolperiod Ts = 2µs. What is the resulting bit rate?

Problem 2.2.2. Consider a baseband PAM system with transmit bandwidth BT = 100 kHz. Findsuitable values of the symbol rate Rs and the number of symbol amplitudes Ma such that a bit rateof Rb = 1Mbit/s is achieved.

Problem 2.2.3. Consider baseband PAM with K = 3 and symbol alphabet A = −3,−1, 1, 3.

a) How many different transmit signals are there?

b) How many bits are transmitted?

c) What is the minimum symbol distance da , minm6=n

∣∣a(m)−a(n)∣∣?

d) Sketch a possible transmit signal s(t), assuming a rectangular transmit pulse of duration Ts (Tsis the symbol period).

e) Sketch a possible transmit signal s(t), assuming a rectangular transmit pulse of duration 32 Ts.

Problem 2.2.4. Consider transmission of the bit sequence (1,0,1,0,0,1,1,1,0,1,0,0). Blocks of 3 bitsare mapped onto symbols a[k]; these symbols are then transmitted using baseband PAM with arectangular pulse of duration = symbol period Ts = 1µs.

a) How many symbol values must be contained in the symbol alphabet?

b) How many symbols are transmitted?

c) Choose some symbol alphabet and sketch the resulting transmit signal.

d) How long does the transmission take?


Problem 2.2.5. Suppose that l successive bits Bj are encoded into a symbol A[k] ∈ A =a(1), a(2), · · ·, a(Ma)

by an invertible mapping. The Bj’s are statistically independent and assume

the values 0 and 1 with equal probability. Show that all Ma = 2l symbols a(m) are equally likely aswell.

Problem 2.2.6. A communication system has a known probability of symbol error, PEs. Calculatethe resulting probability of a bit error, PEb, under the assumption that a Gray code is being usedfor mapping blocks of l bits onto a corresponding symbol.

Note: Assume that a “nearest neighbor” symbol error is the only type of symbol error that mayoccur (since all other symbol errors are extremely unlikely). Thus, a symbol error results in exactlyone bit error, i.e., exactly one out of the l bits is incorrect.

Hint : Use the total probability theorem to decompose PEb in terms of the conditional probabilitiesof the bit error event Eb given the following mutually exclusive events: (1) no symbol error, (2) “nearestneighbor” symbol error.

Problem 2.2.7. Consider the stationarized baseband PAM signal S(t) in (2.12).

a) Derive the result (2.14) for the autocorrelation of S(t).

b) Proceed to derive the result (2.16) for the power spectrum of S(t).

c) Specialize RS(τ) in (2.14) to a white symbol sequence A[k] with mean µA 6= 0 and variance σ2A.

Problem 2.2.8. Consider a stationary symbol sequence A[k] with zero mean and power spectraldensity SA(e

jθ) ≡ 1.

a) Show that EA2[k]

= 1.

b) Calculate the covariance of A[k] and A[k + 3].

c) Can you think of a different symbol power spectrum, i.e., SA(ejθ) 6≡ 1, which also leads to

EA2[k]

= 1?

Problem 2.2.9. Consider baseband PAM with transmit pulse g(t) = sinc(πt/Ts) (where4 sinc(x) ,

sin(x)x ) and white symbols A[k]. Sketch the power spectral density of the transmit signal S(t) for

a) µA 6= 0

b) µA = 0.

Problem 2.2.10. Consider baseband PAM with transmit pulse g(t) = sinc(πt/Ts) and zero-meansymbols A[k] with autocorrelation RA[l] = β δ[l + 1] + δ[l] + β δ[l−1] (β > 0).

a) Find the maximum admissible value of β such that RA[l] is a valid autocorrelation sequence (i.e.,the Fourier transform of RA[l] is nonnegative).

b) Calculate and sketch the power spectral density of the transmit signal S(t) for this value of β.

c) Calculate the mean power of the transmit signal for this value of β.

4Note that in some books and articles the definition sinc(x) = sin(πx)πx

is used.

2.2 Baseband PAM 27

0 32 da

Ma−12 da−Ma−1

2 da

da

− 32 da − 1

2 da12 da

a

Figure 2.4: Typi al symbol alphabet for baseband PAM.

Problem 2.2.11. Consider baseband PAM with transmit pulse g(t) = rect(t;Ts/2). The symbolsA[k] are zero-mean and white. Sketch the power spectral density of the signal received at the outputof an idealized lowpass channel with bandwidth Ωch = 2π/Ts.

Problem 2.2.12. In a partial response system, the symbols are prefiltered in order to achieve specificshapes of the transmit power spectrum. Consider the prefiltering defined by A[k] = Bk−Bk−1, wherethe Bk’s are the input bits represented as Bk ∈ −1, 1. It is assumed that the Bk’s are uncorrelatedwith −1 and 1 equally likely. The transmit pulse g(t) is an idealized lowpass pulse with bandwidthΩg = π/Ts.

a) Calculate the power spectral density of the symbol sequence A[k].

b) Calculate the power spectral density of the stationarized transmit signal S(t).

c) Show that the transmit signal (without stationarization) can be written in the form

S(t) =∞∑

k=−∞Bk h(t−kTb) ,

and provide an expression of h(t) in terms of g(t).

Problem 2.2.13. Consider a symbol alphabet as depicted in Figure 2.4, with Ma even. All symbolsare equally likely.

a) Calculate the mean symbol power PA = EA2[k] as a function of the minimum symbol distance

da. Hint : Use the identity∑n

i=1(2i− 1)2 = n (4n2−1)3 .

b) By what percentage is the squared minimum symbol distance d2a reduced if the mean symbolpower PA is reduced by 10%?

Problem 2.2.14. Repeat part a) of Problem 2.2.13 for the case where the two innermost symbolsare twice as likely as the remaining symbols.

Problem 2.2.15. Consider the symbol alphabet

A =

1

2da,

3

2da, · · ·,

2Ma−1

2da

. (2.26)

All symbols are equally likely.

a) Calculate the mean µA and the mean power PA (cf. Problem 2.2.13).

b) Calculate the mean and the mean power for the centered symbol alphabet in which the symbolsa(m) are replaced by a(m) − µA.


Problem 2.2.16. Consider the shifted symbols a(m)′ , a(m) − α.

a) Show that PA′ , with A′ = A − α, is minimized if and only if α = µA. Hint : In PA′ = EA′2,subtract and add µA from/to A′ = A− α and then expand PA′ such that there appears a term(α− µA)

2.

b) Use the result from a) to argue that zero-mean symbols have minimum mean power (in the sensethat adding a nonzero mean will increase the mean power).

2.3 Passband PAM

Since most channels do not support the transmission of baseband signals, we shall now consider

passband PAM . Here, the transmit signal is a real-valued bandpass signal that is spectrally centered

about a given carrier frequency ωc.

2.3.1 Transmitter and Transmit Signal

This passband PAM transmit signal can be constructed in two steps:

1. First a baseband PAM signal is formed, i.e.,

sLP(t) =

K∑

k=1


. (2.27)

As before, Ts denotes the symbol period and g(t) is some real-valued baseband (lowpass) pulse

with bandwidth Ωg ≡ 2πBg < ωc. The symbols a[k] ∈ A are again derived from blocks of l

consecutive bits bj by using a Gray-code mapping. However, the symbols a(m) or a[k] and, in

turn, the signal sLP(t) now are complex-valued in general, which is an important difference from

conventional baseband PAM as discussed in Section 2.2.

2. The complex-valued baseband PAM signal sLP(t) is transformed into a real-valued passband

PAM signal s(t) that is located in a bandpass range about carrier frequency ωc. This “lowpass-

bandpass transformation” is performed according to

s(t) =√2 Re

sLP(t) e

jωct

=1√2

[sLP(t) e

jωct + s∗LP(t) e−jωct

]. (2.28)

Here, the factor of√2 has been included in order that the energies of sLP(t) and s(t) are equal,

i.e.,EsLP

= Es . (2.29)

An example of the passband PAM transmit signal s(t) along with the associated baseband PAM

signal sLP(t) is shown in Figure 2.5. The Fourier transforms of the signals sLP(t), sLP(t) ejωct, and

s(t) =√2 Re

sLP(t) e

jωctare depicted schematically in Figure 2.6. This figure describes the action

of the lowpass-bandpass transformation in the frequency domain. The lowpass signal sLP(t) is termed

the equivalent complex baseband PAM signal associated with the real-valued passband PAM signal

s(t). A general discussion of bandpass signals and processes can be found in Appendix A.

2.3 Passband PAM 29

1

t−1

Ts=Tb

1

t−1

Ts=Tb

(a)

(b)

Figure 2.5: Baseband and passband PAM transmit signals: (a) Baseband PAM signal sLP(t) (here

assumed real-valued for easy graphi al representation) and (b) asso iated passband PAM signal s(t) =√2 Re

sLP(t) e

jωct.

−ωc −ωc+Ωg ωc ωc+Ωg0 ωc−Ωg−ωc−Ωg

−Ωg Ωg0

|SLP(jω)|

2Ωg

(a)

(b)

(c)

|S(jω)|

|S(jω)|

ω

ω

ω

ωc ωc+Ωgωc−Ωg0

Figure 2.6: S hemati frequen y-domain representation of the lowpass-bandpass transformation: (a) om-

plex lowpass signal sLP(t), (b) frequen y-shifted and s aled signal s(t) =√2 sLP(t) e

jωct, and ( ) bandpass

signal s(t) = Res(t) =√2 Re

sLP(t) e

jωct.


The equivalent complex baseband PAM signal sLP(t) is a lowpass signal whose (one-sided) band-

width is equal to Ωg. As shown in Figure 2.6, the frequency band of the PAM transmit signal s(t)

then is [ωc − Ωg, ωc + Ωg] (considering only positive frequencies; there is also a symmetric band

[−ωc−Ωg, −ωc+Ωg] at negative frequencies since s(t) is real-valued). Because ωc > Ωg, the positive-

frequency and negative-frequency bands do not overlap about frequency 0, and thus s(t) is a true

bandpass signal. The bandwidth of s(t)—which is the transmit bandwidth ΩT ≡ 2πBT —is

ΩT = 2Ωg or equivalently BT = 2Bg . (2.30)

Hence, the bandwidth of passband PAM is twice that of baseband PAM using the same transmit pulse

g(t).

An alternative representation of the passband PAM transmit signal is

s(t) =√2 sLP,R(t) cos(ωct) −

√2 sLP,I(t) sin(ωct) , (2.31)

with the inphase component

sLP,R(t) = ResLP(t) =

K∑

k=1

aR[k] g(t−kTs) , aR[k] = Rea[k] (2.32)

and the quadrature component

sLP,I(t) = ImsLP(t) =K∑

k=1

aI[k] g(t−kTs) , aI[k] = Ima[k] . (2.33)

Note that sLP,R(t) and sLP,I(t) are two real-valued baseband PAM signals using the “symbols” aR[k] =

Rea[k] and aI[k] = Ima[k]. Whereas the representation (2.31) always exists, the expressions

(2.32), (2.33) are only valid for g(t) ∈ R.

A passband PAM transmitter that corresponds to the expression of s(t) in (2.28) and (2.27) is

shown in part (a) of Figure 2.7. A transmitter that corresponds to the representation in (2.31) and

(2.32), (2.33) is shown in Figure 2.7(b).

There are M = MKa different signals s(t) (and, hence, different messages), corresponding to a

total number of L = ldMKa = K ldMa transmitted bits. However, since the symbols a[k] can lie in

a region of the complex plane (rather than in an interval of the real axis as for baseband PAM), the

number of symbols, Ma, can be roughly chosen as the square of the value of Ma chosen for baseband

PAM (given the same symbol power constraint and minimum symbol distance). This means that L

is roughly doubled as compared to baseband PAM, resulting in about twice the bit rate of baseband

PAM. Note, however, that the bandwidth of the transmit signal is doubled as well (cf. Figure 2.6).

2.3.2 Mean, Power Spectrum, and Mean Power of Transmit Signal

For calculation of the mean and the power spectrum of passband PAM, we assume that the symbol

process A[k] is wide-sense stationary5 with mean µA = EA[k], autocorrelation RA[l] = EA[k]A∗[k−l], and power spectrum SA(e

jθ) =∑∞

l=−∞RA[l] e−jθl.

5A complex-valued process A[k] is stationary if the real-part process AR[k] and the imaginary-part process AI[k] are

jointly stationary. This implies that EA[k], EA[k]A∗[k − l], and EA[k]A[k − l] do not depend on k.

2.3 Passband PAM 31

Mapping

Mapping

s(t)

s(t)

√2 ejωct

g(t)

(b)

a[k]

g(t)

sLP(t) =K∑

k=1

a[k] g(t−kTs)Re

g(t)sLP,I(t) =

K∑k=1

aI[k] g(t−kTs)

−√2 sin(ωct)

bj

bj

aI[k]

aR[k]

√2 cos(ωct)

sLP,R(t) =K∑

k=1

aR[k] g(t−kTs)

(a)

a[k]s(t)

Figure 2.7: Blo k diagrams of two dierent implementations of a passband PAM transmitter: (a) Imple-

mentation based on the \ omplex" representation in (2.28) and (2.27), (b) implementation based on the

\real" representation in (2.31) and (2.32), (2.33).

Stationarized process. We model the transmit signal by a stationarized random process that is

constructed as

S(t) ,√2 Re

SLP(t) e

j(ωct+Φ), (2.34)

with the stationarized equivalent complex baseband PAM process

SLP(t) , SLP(t−T ) =

∞∑

k=−∞A[k] g(t−T−kTs) . (2.35)

Here, the random time offset T is uniform in [0, Ts) and statistically independent of the symbols

A[k] (cf. Subsection 2.2.2), and the phase offset Φ in (2.34) is uniform in [0, 2π) and statistically

independent of SLP(t) (cf. Subsection A.3.2 in Appendix A). Note that there are two separate and

independent stationarizations: one by means of the random time offset T in SLP(t−T ) to suppress the

periodicity corresponding to the symbol period Ts, the other by means of the random carrier phase

offset Φ in ej(ωct+Φ) to suppress the periodicity corresponding to the carrier period 2π/ωc. The latter

stationarization step can be omitted if the complex symbols A[k] feature circular symmetry , i.e., if

RA,A∗[l] = EA[k]A[k− l] = 0 or equivalently RAR[l] = RAI

[l] and RAR,AI[l] = −RAI,AR

[l] (cf. Section

A.3).


Mean, autocorrelation, and power spectrum. The stationarized passband PAM process S(t)

is zero-mean (cf. (A.74)),

µS = 0 , (2.36)

and its autocorrelation function is obtained as (cf. (A.75))

RS(τ) = ReRSLP

(τ) ejωcτ

=1

2

[RSLP

(τ) ejωcτ +R∗SLP

(τ) e−jωcτ]. (2.37)

As in (2.14), the autocorrelation function of SLP(t) is given by

RSLP(τ) =

1

Ts

∞∑

l=−∞RA[l] rg(τ−lTs) , (2.38)

with rg(τ) =∫∞−∞ g(t) g(t − τ) dt; the only difference from Subsection 2.2.2 is that now RA[l] is com-

plex-valued.

Taking the Fourier transform of RS(τ) in (2.37) and using

SSLP(jω) =

1

TsSA(e

jωTs) |G(jω)|2 (2.39)

(cf. (2.16)), we finally obtain the power spectral density of S(t) as

SS(jω) =1

2

[SSLP

(j(ω−ωc)

)+ SSLP

(−j(ω+ωc)

)]

=1

2TsSA(ej(ω−ωc)Ts

) ∣∣G(j(ω−ωc)

)∣∣2 +1

2TsSA(e−j(ω+ωc)Ts

) ∣∣G(−j(ω+ωc)

)∣∣2. (2.40)

This means that the baseband PAM spectrum SSLP(jω) = 1

TsSA(e

jωTs) |G(jω)|2 is shifted to ωc

(positive-frequency band), and it is also frequency-reversed and shifted to −ωc (negative-frequency

band). This is shown in Figure 2.8. The bandwidth of S(t) is 2Bg and thus twice that of SLP(t).

Hence, we again see that the bandwidth of passband PAM is twice that of baseband PAM using the

same transmit pulse g(t).

White symbol sequence. In the practically important special case of a white6 and zero-mean

symbol process A[k], we have SA(ejθ) ≡ σ2A (cf. (2.23)) and thus the power spectrum of S(t) simplifies

to

SS(jω) =1

2Tsσ2A∣∣G(j(ω−ωc)

)∣∣2 +1

2Tsσ2A∣∣G(−j(ω+ωc)

)∣∣2. (2.41)

Mean signal power and zero-mean symbols. For white (but not necessarily zero-mean) symbols

A[k], the mean power of S(t) is given by (2.21) with the minor difference that µ2A is replaced by |µA|2,i.e.,

PS = PSLP=

1

Tsσ2A Eg +

1

T 2s

|µA|2 |G(0)|2. (2.42)

6That is, CA[l] = σ2A δ[l], where CA[l] , E

(A[k]−µA)(A[k−l]−µA)

∗= RA[l]− |µA|2.

2.3 Passband PAM 33

0

−ωc

ω

ω

SSLP(jω)

SS(jω)

(a)−Ωg Ωg

ωc ωc+Ωg

2Ωg

Ωg

(b)ωc−Ωg0−ωc−Ωg −ωc+Ωg

Figure 2.8: S hemati representation of (a) the power spe tral density of the stationarized equivalent

omplex baseband PAM pro ess SLP(t) and (b) the power spe tral density of the stationarized passband

PAM pro ess S(t).

Arguing as in Subsection 2.2.2, we conclude that small transmit power PS presupposes µA = 0, i.e.,

zero-mean symbols. Here, the mean transmit power simplifies to

PS =1

Tsσ2AEg . (2.43)

A further reduction of PS can be achieved by reducing σ2A (see next).

2.3.3 Symbol Alphabet Design

Next, we consider the design of the symbol alphabet A =a(1), a(2), · · ·, a(Ma)

. An important differ-

ence from the baseband case (see Subsection 2.2.1) is that the symbols a(m) now are complex-valued.

The geometric constellation of the symbols in the complex plane is called signal constellation. We

first consider the mean and variance/mean power of the (random) symbols A[k].

Mean. We have just seen that minimization of the mean power of the PAM transmit signal requires

that the random symbols A[k] have zero mean, i.e.,

µA = EA[k] =

Ma∑

m=1

a(m) pA(a(m)

)= 0 . (2.44)

Here, pA(a(m)

), P

A[k] = a(m)

denotes the probability7 of the mth symbol a(m), i.e., of the event

A[k] = a(m). In practice, the probability distribution of the symbols is usually uniform, i.e.,

7Note that, theoretically, PA[k] = a(m)

might depend on k since we merely assumed A[k] to be wide-sense stationary.

However, µA and σ2A do not depend on k.


pA(a(m)

)≡ 1

Ma, m = 1, 2, · · ·,Ma . (2.45)

In particular, such a uniform distribution of the A[k] results if the (random) bits Bj from which

the A[k] are obtained are uniformly distributed (i.e., Bj = 0 and Bj = 1 are equally likely) and

statistically independent. For uniform symbol probabilities, the zero-mean condition (2.44) simplifies

to the condition that the arithmetic average of the symbols a(m) be zero:

µA =1

Ma

Ma∑

m=1

a(m) = 0 . (2.46)

This condition can easily be met, and in fact practical symbol alphabets usually have zero average.

Variance or mean power. A second condition for a small mean power of the PAM transmit signal

is that the symbols A[k] have small variance. If µA = 0, which will hereafter be assumed, the variance

is equal to the mean power PA, i.e.,

σ2A = PA = E|A[k]|2 =

Ma∑

m=1

|a(m)|2 pA(a(m)

). (2.47)

For uniform probability distribution of the symbols, this simplifies as

σ2A = PA =1

Ma

Ma∑

m=1

|a(m)|2 . (2.48)

Unfortunately, a reduction of σ2A = PA is possible only within certain limits because it tends to imply

a reduction of the minimum distance between symbols

da , minm6=n

∣∣a(m)− a(n)∣∣ . (2.49)

For a Gaussian channel, this will be shown in Subsection 3.4.3 to entail an increased probability of

a symbol error at the receiver. In particular, a reduction of σ2A = PA could most easily be achieved

by a downscaling of the entire signal constellation. It is evident that this would imply a proportional

reduction of the minimum symbol distance da.

Some signal constellations. Thus, the objective of signal constellation design is to reduce the

mean symbol power without reducing the minimum symbol distance or, said differently, to maximize

the minimum symbol distance while not exceeding a given mean or peak power limit. In the passband

case where symbols are distributed in the 2-D complex plane, there are more degrees of freedom for

such a design than in the 1-D baseband case. However, a truly optimum design is often difficult

to obtain. Furthermore, optimum constellations do not possess a simple geometric structure, and

thus the implementation of the bits ↔ symbol (de)mapping and of the decision performed at the

receiver tends to be computationally intensive. Consequently, most practical modems use one of a few

popular constellations that are suboptimum but have nearly optimum performance. Some popular

signal constellations and related aspects are discussed in the following.

2.3 Passband PAM 35

(a)

a(m)R a

(m)R

(b)

a(m)I

a(m)I

Figure 2.9: Two ASK onstellations: (a) 2-ASK (binary antipodal or BPSK onstellation), (b) 4-ASK.

a(m)I

a(m)I

a(m)Ra

(m)R a

(m)R

(a) (c)(b)

a(m)I

Figure 2.10: Some QAM onstellations: (a) 4-QAM, (b) 16-QAM, ( ) 64-QAM.

• The symbols of an amplitude shift keying (ASK) constellation are all real-valued, as in the

baseband PAM case. The quadrature component is not utilized for transmitting data, so that

the advantage of passband PAM is not exploited. Two examples are shown in Figure 2.9. In

particular, the 2-ASK constellation is also known as the binary antipodal or binary phase shift

keying (BPSK) constellation.

• Examples of quadrature amplitude modulation (QAM) constellations are shown in Figure 2.10.

For QAM constellations, Ma is a power of 4 and thus l = ldMa is even. QAM constellations

are distinguished by their regular rectangular pattern. They feature a practically convenient

“separability” of the real and imaginary symbol components that greatly simplifies the bits

↔ symbol (de)mapping and the decision algorithm used by the receiver. Indeed, assuming an

appropriate bits → symbol mapping and a bandpass channel with additive white Gaussian noise,

the inphase and quadrature signals for a QAM constellation are statistically independent PAM

signals with√Ma = 2l/2 symbol amplitudes each, and the noise components in the inphase and

quadrature branches are statistically independent as well. This can be shown to imply that the

(optimum) decisions regarding the inphase and quadrature components can be made completely

separately and independently of each other. That is, instead of a 2-D decision we have two

simple 1-D decisions.

• Cross constellations are similar to QAM constellations in that they are based on a rectangular

lattice; however, here l is odd (see Figure 2.11). These constellations require slightly more


a(m)Ra

(m)R

a(m)I

a(m)I

a(m)I

a(m)I

a(m)R a

(m)R

Figure 2.11: Some ross onstellations.

a(m)R a

(m)R

a(m)I

a(m)I

a(m)R a

(m)R a

(m)R

a(m)I

(e)

a(m)I

a(m)I

(a) (b) (c) (d)

Figure 2.12: Constellations using PSK: (a) 2-PSK or BPSK (binary antipodal onstellation), (b) 4-PSK

or QPSK (identi al to 4-QAM), ( ) 8-PSK, (d) and (e) ombination of PSK with amplitude modulation.

complicated mappers than QAM constellations. Both the QAM constellations and the cross

constellations are called rectangular constellations.

• Another family of constellations with simple geometry is based on phase shift keying (PSK),

sometimes combined with amplitude modulation (see Figure 2.12). The 2-PSK (usually called

BPSK) constellation is identical to the binary antipodal constellation. The 4-PSK (also called

QPSK) constellation is identical to the 4-QAM constellation. With pure PSK (i.e., no amplitude

modulation), all information about the symbols is borne by the phase of the carrier, which makes

the transmission insensitive to rapid amplitude fluctuations (fading) introduced by the channel.

Another advantage of pure PSK is that the transmit signal tends to have (at least approximately)

constant envelope. Finally, inexpensive receivers can be designed with digital logic. The received

signal can immediately be hard-limited, which preserves the locations of the zero crossings (that

indicate the phase). The hard-limited signal can be processed with digital logic without requiring

A/D conversion. The disadvantage of pure PSK is that because all symbols are constrained to

lie on a common circle, the minimum symbol distance da becomes very small for larger values

of Ma (assuming that PA, and thus the circle’s radius, is fixed).

• A performance improvement over rectangular constellations such as QAM can be achieved with

hexagonal constellations, an example of which is shown in Figure 2.13. The symbols lie on

the vertices of equilateral triangles. The term ‘hexagonal’ refers to the decision regions of

2.3 Passband PAM 37

a(m)R

a(m)I

Figure 2.13: An example of a hexagonal onstellation, with three hexagonal de ision regions shown.

the maximum-likelihood receiver (to be discussed in Chapters 5 and 6) which are also shown

in Figure 2.13. For large Ma, hexagonal constellations are optimum in that they minimize the

extension of a constellation for a given minimum symbol distance da. However, the improvement

over rectangular constellations is only slight, and the bits ↔ symbol (de)mapping and decision

algorithms are significantly more complicated.

• Constellation shaping is a mapping method that achieves a nonuniform symbol probability

pA(a(m)

). The basic idea is to make symbols with large magnitude |a(m)| less likely since these

symbols contribute more to the average power PA =∑Ma

m=1 |a(m)|2 pA(a(m)

). However, practical

difficulties may outweigh the advantages of this approach.

• Most of the signal constellations described above have the practical problem that they are

rotationally invariant for some particular angles of rotation, typically multiples of π/2 or possibly

less in the case of PSK. If the constellation is rotated by such an angle (due to a phase offset

introduced by the channel), the receiver cannot distinguish it from the correct constellation unless

it knows the actual transmitted symbols, which it does not. Thus, the receiver may decide on

the symbols corresponding to a rotated constellation, with the result that the information bits

will be incorrectly demapped.

This ambiguity can be resolved by starting the transmission with a sequence of training or

pilot symbols that is known to the receiver (this is sometimes called a preamble). A safer and

simpler method is differential encoding , in which the information is represented by the change

in constellation position between symbols rather than absolute position. In particular, with

differential PSK (DPSK), the current symbol phase θk (note that a[k] = |a[k]|ejθk) is chosen as

θk = θk−1 +∆k , (2.50)

where θk−1 is the phase of the previous symbol and the phase difference ∆k represents the

information (bits) to be transmitted in the kth symbol interval.

DPSK allows a simplified “differential” detection algorithm that does not require recovery and

tracking of the carrier phase. This, however, results in an SNR penalty of about 1...3 dB.


Differential encoding can be combined with absolute encoding. For the 16-QAM constellation of

Figure 2.10(b), for example, it is common to differentially encode the quadrant (specified by two

bits, since there are four quadrants), while the point within the quadrant (specified by another

two bits, since there are four symbols within each quadrant) is encoded absolutely.

2.3.4 Spectral Efficiency

The spectral efficiency of a digital modulation scheme is defined as the bit rate Rb divided by the

transmit bandwidth BT, i.e., the bandwidth of the transmit signals s(t):

ν ,Rb

BT

[bit/s

Hz

]. (2.51)

For any serial modulation method transmitting an Ma-ary symbol per symbol period of length Ts,

with Ma = 2l, the bit rate is

Rb =l

Ts=

ldMa

Ts. (2.52)

According to (2.30), the transmit bandwidth of passband PAM is twice the bandwidth of the transmit

pulse g(t), i.e.,

BT = 2Bg . (2.53)

Inserting (2.52) and (2.53) into (2.51) yields the spectral efficiency of passband PAM as

ν =ldMa

2TsBg. (2.54)

In Subsection 3.3.1, we shall show that the minimum pulse bandwidth Bg that (theoretically)

allows for transmission free of intersymbol interference is

Bg,min =1

2Ts. (2.55)

Hence, the maximum spectral efficiency of passband PAM is obtained as

νmax =ldMa

2TsBg,min= ldMa = l , (2.56)

which is the number of bits represented by one symbol. This shows that the spectral efficiency of

passband PAM can be made arbitrarily large by choosing Ma sufficiently large. However, Ma is

effectively limited by the constraint on the transmit power and by the channel noise—cf. Sections

2.3.3 and 3.4.

2.3.5 Spread Spectrum Modulation

Transmit bandwidth is costly and thus one is normally interested in using a transmit pulse g(t)

with small bandwidth, i.e., a bandwidth that is not much larger than the (impractical) minimum

bandwidth Bg,min = 1/(2Ts). Spread spectrum modulation, on the other hand, is passband PAM with

a transmit pulse whose bandwidth is deliberately chosen much larger than the minimum bandwidth,

i.e., Bg ≫ Bg,min = 1/(2Ts). There are several reasons for using a large bandwidth:

2.3 Passband PAM 39

• Pulses with a broader spectrum are less sensitive to channel impairments that are highly localized

in frequency, and less vulnerable to narrowband interference (such as jammers, i.e., interfering

signals deliberately emitted by a hostile party in order to disrupt the communication).

• Spread spectrum signals can be concealed, since they may be placed in frequency bands already

occupied by other signals, and in effect may be masked by the other signals. Since spread

spectrum signals have a large bandwidth but small power spectrum levels, it is difficult for third

parties to distinguish them from ambient broadband noise.

• Several different spread spectrum transmissions (users) can share a common (large) bandwidth

without interfering much with one another. The corresponding multiple access technique is

known as code-division multiple access (CDMA).

In view of the first two reasons, it should not be surprising that for a long time spread spectrum mod-

ulation was primarily used for secure military communications. In today’s nonmilitary/commercial

applications, spread spectrum modulation is mostly used in combination with CDMA, e.g., for digital

cellular phone communications (see further below). Here, the bandwidth may be large since it is

shared by several different users. Another important application of spread spectrum modulation is

GPS (global positioning system).

Transmit pulse. Spread spectrum modulation requires ways to generate broadband pulses with

controlled spectral properties. A convenient pulse format for achieving this goal is8

g(t) =

Nc−1∑

n=0

ξn gc(t−nTc) , (2.57)

where gc(t) is an elementary pulse whose translates gc(t−nTc) are called chips, Tc = Ts/Nc is the chip

period , Nc is the number of chips contained in g(t) (termed the spreading factor), and ξn ∈ −1, 1(n = 0, 1, · · ·, Nc−1) is called the spreading sequence. Note that the symbol period Ts is divided into

Nc chip intervals of duration Tc = Ts/Nc each. According to (2.57), the pulse g(t) is formed through

PAM-type modulation9 of the chips gc(t − nTc) by the given spreading sequence ξn. This spreading

sequence is fixed (i.e., it does not depend on the data/symbols a[k] to be transmitted).

In practice, the chip pulse gc(t) is usually a rectangular pulse of length Tc (here, we do not take into

account the bandlimitation and, thus, pulse shaping caused by the subsequent RF stage). The PAM

pulses g(t−kTs) then do not overlap, and the baseband PAM transmit signal sLP(t) =∑K

k=1 a[k] g(t−kTs) can be generated by a multiplier that, in each symbol interval, multiplies the current, temporally

constant value of a[k] by the fast spreading sequence ξn ∈ −1, 1. This multiplication performs a

“chipping” of the temporally constant value of a[k] whereby the bandwidth is increased.

8A different pulse construction is used for frequency-hopping spread spectrum modulation, which will however not be

discussed here.9The PAM-type modulation that is used for constructing the transmit pulse g(t) =

∑Nc−1n=0 ξn gc(t − nTc) should

not be confused with the PAM used for transmitting the data symbols a[k] via the passband PAM transmit signal

s(t) =√2 Re

∑Kk=1 a[k] g(t− kTs) e

jωct.


The (effective) bandwidth of g(t) equals that of the chip pulse gc(t). Broadly speaking, the length

of gc(t) is Tc = Ts/Nc, and thus the bandwidth of gc(t) (and, in turn, the bandwidth of g(t)) is Nc times

the bandwidth of conventional (narrow-band) pulses designed for the symbol period Ts. This yields

the spectral spreading and explains the term spreading factor used for Nc. The Fourier transform of

g(t) is easily shown to be

G(jω) = Gc(jω) Ξ(ejωTc) , (2.58)

where

Ξ(ejθ) =

Nc−1∑

n=0

ξn e−jθn (2.59)

is the Fourier transform of the spreading sequence ξn. Thus, the spreading sequence influences the

pulse spectrumG(jω). Usually one desires both |ξn| ≡ 1 (i.e., constant envelope) and∣∣Ξ(ejθ)

∣∣ ≡ 1; this

can approximately be achieved by choosing ξn as a suitable pseudorandom or pseudonoise sequence

such as a maximum-length shift register sequence. The result is called direct-sequence spread spectrum.

The spreading sequence ξn is a “signature” or “code” whose knowledge allows the receiver to

demodulate (or “de-spread”) the spread spectrum signal. If the chip pulse is rectangular, de-spreading

is simply done by multiplying the received signal by the spreading sequence ξn (same operation as

spreading; note that ξn·ξn ≡ 1 because 1 ·1 = 1 and (−1) · (−1) = 1). In this process of demodulation,

narrowband interference is spectrally spread out and thereby becomes similar to broadband noise,

most of which can be suppressed by a filter that passes the (small) bandwidth of the de-spread signal.

CDMA communications. For multiuser (code-division multiple access, CDMA) communications,

different users i and j are typically assigned orthogonal spreading sequences ξ(i)n and ξ

(j)n . Orthogonality

means that

〈ξ(i)n , ξ(j)n 〉 ,

Nc−1∑

n=0

ξ(i)n ξ(j)n = 0 for all i 6= j . (2.60)

In the process of demodulation for a particular user, the signals of other users—whose spreading

sequences are orthogonal to the spreading sequence of the specific user considered—are theoretically

suppressed. This principle allows many users to occupy the same band and also explains the term

code-division multiple access. In practice, however, distortion introduced by the channel and syn-

chronization errors cause a certain loss of orthogonality and, thus, some interference from other users

remains. This interference is similar to broadband noise.

CDMA is employed by the UMTS10 standard for cellular/mobile communications (in the context

of UMTS, the term “wideband CDMA” is often used). The spreading factor Nc is between 4 and 256

for the FDD11 mode and between 1 and 16 for the TDD12 mode. The chip rate 1/Tc is 3.84Mchip/s.

10UMTS = Universal Mobile Telecommunications System.11FDD = frequency-division duplex: Uplink and downlink are transmitted in different frequency bands.12TDD = time-division duplex: Uplink and downlink are transmitted in the same frequency band but in different time

slots. This mode is currently not deployed in Europe.

2.3 Passband PAM 41

2.3.6 Problems

Problem 2.3.1. Verify (2.29).

Problem 2.3.2. Verify the equivalence of (2.28), (2.27) and (2.31)–(2.33) for g(t) ∈ R.

Problem 2.3.3. Consider a passband PAM system with symbol period Ts = 2µs and symbol alphabetA = 1+ j, −1+ j, −1−j, 1−j. What is the bit rate?

Problem 2.3.4. Consider passband PAM with symbol rate Rs = 2.4 ksymbol/s. How large must wechoose Ma, the number of different symbol values in our symbol alphabet, in order to achieve a bitrate of Rb = 9.6 kbit/s?

Problem 2.3.5. Consider a passband PAM system with transmit bandwidth BT = 3.1 kHz. Findsuitable values for the symbol rate Rs and the number of symbol amplitudes Ma such that a bit rateof Rb = 33.6 kbit/s is achieved. Note: For transmission free of intersymbol interference, the symbolrate is constrained as Rs ≤ BT (cf. Subsection 3.3.1).

Problem 2.3.6. Consider passband PAM with a complex-valued transmit pulse g(t).

a) Find expressions for the inphase and quadrature components of the transmit signal for this caseand compare them to the expressions (2.32) and (2.33) valid for a real-valued g(t).

b) Sketch a corresponding implementation of the PAM transmitter for a complex-valued g(t). Thistransmitter should be similar to that in Figure 2.7(b) in that it includes multiplications by√2 cos(ωct) and −

√2 sin(ωct).

c) Sketch an alternative implementation of the PAM transmitter for complex-valued g(t) that usesbandpass filters with impulse responses g1(t) = 2 Reg(t)ejωct and g2(t) = 2 Img(t)ejωct anda time-varying premultiplication of a[k]. Is this implementation less complex than that in b)?

Problem 2.3.7. Show (2.36) and (2.37).

Problem 2.3.8. Let SLP(t) be a stationary complex-valued lowpass process and let S(t) =√2 Re

SLP(t) e

j(ωct+Φ)as in Subsection 2.3.2. Show that PS = PSLP

(used in (2.42)).

Problem 2.3.9. Consider passband PAM with transmit pulse g(t) = rect(t;Ts/2). The symbols A[k]are zero-mean and white with variance σ2A.

a) Sketch the power spectral density of the transmit signal.

b) Calculate the mean power of the transmit signal.

Problem 2.3.10. Consider passband PAM with symbol alphabet A = 1+ j, −1+ j, −1−j, 1−j.All symbols are equally likely.


a) Calculate the mean symbol power PA and the minimum symbol distance da.

b) All symbols are scaled by a common factor β > 0. Calculate β such that the mean symbol powerbecomes PA = 3.5.

c) What is the resulting minimum symbol distance?

Problem 2.3.11. Consider passband PAM using the following signal constellations:

a) binary antipodal

b) QPSK

c) 8-PSK

d) 16-QAM.

All symbols are equally likely.

• Calculate the symbol mean µA.

• Calculate the mean symbol power PA as a function of the minimum symbol distance.

• Calculate the mean number of nearest-neighbor symbols. (This is defined as N , ENM =∑Mam=1 Nm pA

(a(m)

), where Nm denotes the number of symbols that are “nearest neighbors” to

a(m).)

Problem 2.3.12. Consider passband PAMwith a signal constellation consisting of seven PSK symbolswith radius r > 0 plus the symbol a = 0.

a) Calculate the minimum symbol distance da.

b) Calculate the mean symbol power PA under the assumption that all symbols are equally likely.

c) Show that the mean symbol power PA is not changed if the seven PSK symbols have unequalprobabilities (but the probability of the symbol a = 0 is as before).

d) Suppose that all seven PSK symbols are equally likely, and the probability of the symbol a = 0is larger by a factor β > 1 than the probability of any one of the seven PSK symbols. How largemust β be chosen if PA is to be halved relative to the case where all eight symbols are equallylikely?

Problem 2.3.13. Consider passband PAM with a 16-QAM constellation. All symbols are equallylikely and the symbol sequence is white. The transmit pulse is g(t) = G0 sinc(πt/Ts) with G2

0 =0.1mW. The bit rate is Rb = 100 kbit/s. Calculate the minimum symbol distance da such that themean transmit power is PS = 10mW.

Problem 2.3.14. Consider passband PAM with a symbol alphabet consisting of the two inner symbols±jα/2 and the four outer symbols ±α±jα/2 (with independent signs of the real and imaginary parts).The symbol probability is 1/4 for inner symbols and 1/8 for outer symbols.

a) Design a bits → symbol mapper that achieves these symbol probabilities for a bit sequence thatis white and uniformly distributed (i.e., 0 and 1 equally likely).

2.3 Passband PAM 43

b) Calculate the mean symbol power PA as a function of α.

c) Calculate the mean number of nearest-neighbor symbols (defined as in Problem 2.3.11).

Problem 2.3.15. Calculate the maximum spectral efficiency of baseband PAM for a given Ma andcompare it with that of passband PAM (equation (2.56)). Note: Equation (2.55) also applies tobaseband PAM.

Problem 2.3.16. Consider baseband PAM using symbol alphabet A = −3,−1, 1, 3 and passbandPAM using 16-QAM symbols a[k] = aR[k] + jaI[k] where aR[k], aI[k] ∈ A.

a) Show that the maximum spectral efficiency of the two modulation schemes is identical. Hint :Use the result of Problem 2.3.15.

b) Compare the mean symbol power and the minimum symbol distance for the two modulationschemes.

c) Consider a scaling of the 16-QAM constellation such that the mean symbol power of bothmodulation schemes is equal. What is the resulting reduction of the minimum symbol distanceda of the scaled 16-QAM constellation?

Problem 2.3.17. Show (2.58).

Problem 2.3.18. Consider UMTS transmission with spreading factor Nc = 16. The chip rate is3.84Mchip/s. The bandwidth of the chips in Hz (only counting positive frequencies) is defined to be1/(2Tc).

a) Calculate the chip bandwidth (in Hz).

b) Calculate the transmit bandwidth.

c) Calculate the symbol rate.

d) What is the maximum number of users that can be served when the chip sequences are supposedto be orthogonal?

3Elementary Passband PAM

Systems

Contents


3.2 The Elementary Passband PAM System . . . . . . . . . . . . . . . . . . . 46

3.2.1 Components of the Elementary Passband PAM System . . . . . . . . . . . . . 47

3.2.2 Equivalent Discrete-Time Baseband PAM System . . . . . . . . . . . . . . . . . 51

3.2.3 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55

3.3 Intersymbol Interference and Nyquist Pulses . . . . . . . . . . . . . . . . . 57

3.3.1 ISI-Free Transmission and the Nyquist Criterion . . . . . . . . . . . . . . . . . 58

3.3.2 Examples of Nyquist Pulses . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60

3.3.3 Eye Diagrams . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61

3.3.4 Zero-Forcing Receive Filter Design . . . . . . . . . . . . . . . . . . . . . . . . . 62

3.3.5 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63

3.4 Error Probability of Passband PAM . . . . . . . . . . . . . . . . . . . . . . 65

3.4.1 Probability of a Symbol Error for ISI-Free Transmission . . . . . . . . . . . . . 65

3.4.2 Binary Symbols . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69

3.4.3 Bounds and Approximations for the Symbol Error Probability . . . . . . . . . 72

3.4.4 Probability of a Bit Error . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76

3.4.5 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79

3.5 Matched Filter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83

3.5.1 Definition and Derivation of the Matched Filter . . . . . . . . . . . . . . . . . . 84

3.5.2 Equivalent Discrete-Time Baseband System and Nyquist Criterion . . . . . . . 86

3.5.3 More about the Matched Filter . . . . . . . . . . . . . . . . . . . . . . . . . . . 88

3.5.4 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91

45

46 Chapter 3. Elementary Passband PAM Systems


In this chapter, we consider a complete uncoded passband PAM system that includes a channel

and an elementary receiver. Important aspects to be studied are the linear distortion (intersymbol

interference, ISI) and noise introduced by the channel.

Since the discussion of optimum receivers is deferred until Chapters 5 and 6, the PAM receiver

considered in this chapter is an ad hoc structure comprising a frequency shift, a receive filter, a sampler,

and a quantization device (slicer). Later, in Chapter 6, we shall see that this simple structure is closely

related to the optimum receiver for a Gaussian noise channel— indeed, for such a channel, it is itself

optimum in the ISI-free case. Particular attention is directed towards the design of the transmit

and/or receive filter, which is considered with two different goals in mind:

1. The criterion of ISI-free transmission will lead to the Nyquist pulses discussed in Section 3.3. ISI

and Nyquist pulses are fundamental concepts that will often be encountered in later chapters.

2. The criterion of maximum signal-to-noise ratio at the slicer input defines the matched filter

discussed in Section 3.5. The matched filter is a central component of optimum receivers; it will

be rederived in Chapter 6 using a different viewpoint and optimality criterion.


• Section 3.2 discusses a complete uncoded passband PAM system consisting of a transmitter, a

channel, and an elementary receiver. In particular, we derive a surprisingly simple description

of this system termed the equivalent discrete-time baseband PAM system.

• In Section 3.3, we discuss ISI and how it can be avoided through proper design of the transmit

and/or receive filter. Specifically, we derive the Nyquist criterion and discuss some examples of

Nyquist pulses.

• The probability of a symbol error and the probability of a bit error are calculated in Section

3.4 under the simplifying assumption of ISI-free transmission. Furthermore, useful bounds and

approximations for the symbol error probability are developed.

• In Section 3.5, the receive filter maximizing the signal-to-noise ratio at the slicer input (i.e., the

matched filter) is derived.

3.2 The Elementary Passband PAM System

In this section, we present an elementary passband PAM system without source or channel coding. We

show that it can be equivalently represented as a discrete-time (symbol-rate) baseband PAM system

with complex symbols, signals, pulses, and filters. (We recall that passband PAM transmitters and

transmit signals have previously been discussed in Section 2.3.)

3.2 The Elementary Passband PAM System 47

MappingTransmit

filter

Demapping

Channelfilter

Transmitter

Bandpass-lowpass

transformation

Symbol-ratesampler

Complex slicer

Channel

Lowpass-bandpass

transformation

Receivefilter

Receiver

sLP(t) =K∑

k=1

a[k]g(t−kTs)

s(t)

n(t)

g(t)

y(t) yLP(t)

f(t)

q(t)

t=kTs

a[k]∈A bj ∈0,1

bj ∈0,1 a[k]∈A s(t) =√2Re

sLP(t)e

jωct

y(t)

q[k]

b(t)

Figure 3.1: Elementary passband PAM system.

3.2.1 Components of the Elementary Passband PAM System

The block diagram of the elementary passband PAM system is shown in Figure 3.1. This block

diagram is based on the complex representation of passband PAM as discussed in Subsection 2.3.1.

A schematic frequency-domain representation of the signals involved in the passband PAM system is

depicted in Figure 3.2 (cf. Figure 2.6). The various components of the system are explained in the

following.

1. Mapping : The incoming bit sequence bj (bit period Tb, bit rate Rb = 1/Tb) is converted into

a sequence of symbols a[k]. This is done by dividing the bit sequence into blocks of l bits

and reversibly mapping each bit block to a complex -valued symbol a[k] = aR[k] + jaI[k]. The

symbols a[k] are taken from an Ma-ary alphabet A =a(1), a(2), · · ·, a(Ma)

. There are Ma = 2l

different symbols a(m). The symbol period is Ts = lTb = (ldMa)Tb and the symbol rate is Rs =

Rb/l = Rb/ldMa. A Gray code (see Subsection 2.2.1) is usually employed for the bits→ symbol

mapping.


SN (jω) ≡ N0/2

0

0

0

0

0

ω

ω

ω

ω

ω

SLP(jω)

S(jω)

Y (jω)

YLP(jω)

−ωc ωc

−ωc ωc

−ωc ωc

−ωc ωc

Y (jω)

−2ωc

Figure 3.2: S hemati frequen y-domain representation of various signals of the passband PAM system

shown in Figure 3.1. (The nal ltering is done impli itly by the re eive lter f(t).)

2. Transmit filter : The complex-valued symbols1 a[k] are passed through a transmit filter (pulse

shaping filter) with impulse response g(t). The output signal is (cf. (2.27))

sLP(t) =K∑

k=1


. (3.1)

Usually, the transmit filter’s impulse response g(t) is real-valued since a complex-valued transmit

filter would result in undesired “cross talk” between the inphase symbol components aR[k] and

the quadrature symbol components aI[k]. We assume that the transmit filter is a lowpass filter

with bandwidth Ωg ≡ 2πBg. Then sLP(t) is a lowpass signal with the same bandwidth Ωg.

1Theoretically, the input to the transmit filter is the impulse train∑K

k=1 a[k] δ(t−kTs). With practical PAM modems,

the transmitter is implemented in discrete time and hence Dirac impulses are not needed.


Lowpassfilter

√2 e−jωct

y(t) yLP(t)y(t)

Figure 3.3: Bandpass-lowpass transformation system.

3. Lowpass-bandpass transformation: The complex-valued lowpass signal sLP(t) is converted into

the corresponding real-valued bandpass signal (cf. (2.28) and Figure 2.7)

s(t) =√2 Re

sLP(t) e

jωct

=√2 Re

[K∑

k=1

a[k] g(t−kTs)]ejωct

. (3.2)

The effect of this lowpass-bandpass transformation is depicted in the frequency domain by the

upper part of Figure 3.2. We assume that ωc > Ωg so that s(t) is a true bandpass signal, as

discussed in Subsection 2.3.1.

4. Channel : The channel introduces linear, time-invariant distortion, which is modeled by an LTI

(bandpass) filter with real-valued impulse response b(t), and it adds stationary noise N(t) with

power spectral density SN (jω). Typically, N(t) will be assumed zero-mean and white, i.e.,

SN (jω) ≡ N0/2. Also, we will often assume that N(t) is a Gaussian process.

5. Bandpass-lowpass transformation: The real-valued signal y(t) received at the output of the chan-

nel is converted into a complex-valued baseband signal. This can be achieved by the “bandpass-

lowpass transformation system” depicted in Figure 3.3 (cf. Figure A.4(a) in Appendix A). The

real-valued signal y(t) is first frequency-shifted by frequency −ωc through multiplication by2√2 e−jωct, i.e.,

y(t) = y(t) ·√2 e−jωct . (3.3)

This causes the signal band about ωc to be shifted to baseband and the signal band about −ωc

to be shifted to a band about −2ωc. Subsequently, the undesired band about −2ωc and the

noise components outside the desired signal band would have to be suppressed by an (idealized)

lowpass filter. However, this lowpass filtering is done by the subsequent receive filter (see Item

6), thus obviating the need for the idealized lowpass filter. A frequency-domain illustration of

the effect of this bandpass-lowpass transformation is provided by the lower part of Figure 3.2.

In a practical digital implementation of the receiver, this bandpass-lowpass transformation is

preceded by an analog anti-aliasing filter (not shown in Figure 3.1).

2The carrier factor e−jωct must be available at the receiver with correct frequency ωc and correct phase. In practice,

this is achieved by a carrier recovery (carrier estimation) unit in the receiver. The nontrivial problem of carrier recovery

will not be considered here.


6. Receive filter : The complex-valued signal y(t) formed above (consisting of a desired baseband

component, an undesired component about frequency −2ωc, and white noise) is passed through

a receive filter with impulse response f(t). This receive filter has a lowpass characteristic, and

thus it suppresses the signal component about frequency −2ωc. It also suppresses most of the

out-of-band noise, i.e., the noise components that lie outside the signal band. For reasons that

will become clear later, f(t) is generally complex-valued.

7. Symbol-rate sampler : The output of the receive filter—denoted by q(t)— is sampled with sam-

pling rate equal to the symbol rate. This produces the complex-valued sequence3

q[k] , q(kTs) . (3.4)

8. Decision device (slicer, quantizer): Each sampler output q[k] is converted into a “detected”4

symbol a[k]. Whereas q[k] can be any complex-valued number, a[k] is a valid symbol taken from

our symbol alphabet A =a(1), a(2), · · ·, a(Ma)

. Hopefully, the detected symbol a[k] is correct,

i.e., equal to the symbol a[k] that was transmitted. In the opposite case a[k] 6= a[k], a symbol

error has occurred.

At this point, we assume that the decision device performs a simple rounding operation (this

is known as a “slicer” or “quantizer”). That is, the detected symbol a[k] is chosen as the valid

symbol a ∈ A which, in the complex plane, is closest to the sampler output5 q[k]:

a[k] = argmina∈A

∣∣q[k]− a∣∣ = argmin

a∈A

√(qR[k]− aR)2 + (qI[k]− aI)2 . (3.5)

Note that q[k] − a is a complex number and |q[k] − a| is its modulus, i.e., |q[k] − a| =√(qR[k]− aR)2 + (qI[k]− aI)2. We shall see in Chapter 6 that for ISI-free transmission and

Gaussian noise such a symbol-by-symbol quantization is optimum.

9. Demapping : Finally, each detected symbol a[k] is demapped into the corresponding binary word,

thereby inverting the bits→ symbol mapping performed by the transmitter (Item 1). This results

in a “detected” bit sequence bj with bit rate Rb = lRs. If no symbol errors have occurred, the

detected bit sequence equals the original bit sequence, i.e., bj ≡ bj. In the presence of symbol

errors, some of the detected bits will be incorrect. (These bit errors, or at least some of them, can

still be corrected if the bits are protected by a channel code.) If a Gray code was employed for

the bits→ symbol mapping in Item 1, then an incorrectly detected symbol a[k] that is adjacent

to the correct symbol a[k] results in only a single bit error.

3Actually, the output of the sampling stage is q[k] = q(kTs + τ0), where τ0 is a time offset that ideally should be

equal to the total delay introduced by the transmit, channel, and receive filters. In practice, τ0 is estimated by a timing

recovery (synchronization) unit in the receiver. The nontrivial problem of timing recovery will not be considered here.

In what follows, we set τ0 = 0 for simplicity.4The relation to detection theory will be worked out in Chapters 5 and 6.5The notation argminx∈X f(x) should be read as “the x ∈ X minimizing f(x).”


It should be noted that the receiver defined in Items 5–9 performs a symbol-by-symbol deci-

sion/detection since each sampler output q[k] is separately converted to a symbol a[k], without refer-

ence to the other sampler outputs q[l] (l 6= k). Alternatively, one could imagine performing sequence

detection where the entire sequence q[k] as a whole is mapped to a detected symbol sequence a[k].Sequence detection for passband PAM will be discussed in Chapter 6.

At this point, we do not claim that the operations performed by our receiver are optimum in any

sense. However, we shall see in Chapter 6 that, for a Gaussian noise channel, there is a close relation

to the optimum receiver. Indeed, for such a channel, our receiver is optimum if there is no ISI and if

the receive filter f(t) is chosen appropriately (matched filter, see Section 3.5).

Even though we defer the issue of overall optimality to Chapters 5 and 6, there are some design

issues that can be addressed within our current framework. While we cannot influence the properties

of the channel (channel impulse response b(t) and noise power spectrum SN (jω)), we can, and must,

somehow design the transmit filter g(t) and the receive filter f(t). Two fundamental aspects of this

design will occupy a considerable part of the remainder of this chapter.

3.2.2 Equivalent Discrete-Time Baseband PAM System

We now analyze the passband PAM system described in the last subsection. We will see that it can be

represented very compactly by an equivalent discrete-time (symbol-rate) baseband PAM system. This

representation radically simplifies the analysis of passband PAM systems, and therefore it will be the

basis for our further treatment of passband PAM in subsequent sections.

Effect of the channel. The transmit signal in (3.2) is

s(t) =√2 Re

sLP(t) e

jωct

=1√2

[sLP(t) e


], (3.6)

with the complex-valued baseband PAM signal

sLP(t) =

K∑

k=1

a[k] g(t−kTs) . (3.7)

In a similar manner, the real-valued bandpass channel filter b(t) can be represented by a complex-

valued lowpass filter bLP(t) as6

b(t) = 2 RebLP(t) e

jωct

= bLP(t) ejωct + b∗LP(t) e

−jωct . (3.8)

According to Figure 3.1, the output of the channel or, equivalently, the input to the receiver is given

by

y(t) = s(t) ∗ b(t) + n(t) , (3.9)

6We here use the factor 2 instead of√2 since this preserves the filter gain in the pass band; for example, an idealized

bandpass filter corresponds to an idealized lowpass filter (both with gain 1 in the pass band).


where ∗ denotes the convolution operation. Using (3.6) and (3.8), the signal component of y(t)

(without the noise n(t)) is obtained as

ys(t) , s(t) ∗ b(t)

=1√2

[sLP(t) e


]∗[bLP(t) e

jωct + b∗LP(t) e−jωct

]

=1√2

[ [sLP(t) e

jωct]∗[bLP(t) e

jωct]

︸︷︷︸[sLP(t) ∗ bLP(t)] ejωct

+[sLP(t) e

jωct]∗[b∗LP(t) e

−jωct]

︸︷︷︸0

+[s∗LP(t) e

−jωct]∗[bLP(t) e

jωct]

︸︷︷︸0

+[s∗LP(t) e

−jωct]∗[b∗LP(t) e

−jωct]

︸︷︷︸[s∗LP(t) ∗ b∗LP(t)] e

−jωct

](3.10)

=1√2

[[sLP(t) ∗ bLP(t)] ejωct + [s∗LP(t) ∗ b∗LP(t)] e−jωct

]. (3.11)

In (3.10), the expressions below the braces follow from the fact that sLP(t) is bandlimited with band-

width Ωg < ωc; these expressions can easily be verified in the frequency domain where all convolu-

tions become simple multiplications. In particular, we have[sLP(t) e

jωct]∗[b∗LP(t) e

−jωct]= 0 and[

s∗LP(t) e−jωct

]∗[bLP(t) e

jωct]= 0 since the positive-frequency and negative-frequency bands do not

overlap (cf. Figure 3.2).

Bandpass-lowpass transformation. The first stage of the receiver is a bandpass-lowpass trans-

formation combined with the receive filter f(t). The signal y(t) = ys(t)+n(t) is first frequency-shifted

by −ωc, which according to (3.3), (3.9), and (3.11) gives

y(t) = y(t) ·√2 e−jωct = sLP(t) ∗ bLP(t)︸︷︷︸

baseband component

+ [s∗LP(t) ∗ b∗LP(t)] e−j2ωct

︸︷︷︸undesired component about −2ωc

+√2 n(t) e−jωct

︸︷︷︸noise component

. (3.12)

Next, this signal is passed through the complex-valued receive filter f(t). Let

q(t) , y(t) ∗ f(t) =

∫ ∞

−∞y(t′) f(t−t′) dt′ (3.13)

denote the filter output. Assuming that f(t) is a lowpass filter with cutoff frequency ≤ ωc, the

undesired signal component about frequency −2ωc is suppressed. Hence, we obtain

q(t) = sLP(t) ∗ bLP(t) ∗ f(t) + n(t) ∗ f(t) , where n(t) ,√2 n(t) e−jωct . (3.14)

According to this result, the central part of our passband PAM system (i.e., without the bits →symbols mapping, slicer, and symbols → bits demapping) can be equivalently represented as the

baseband PAM system depicted in part (a) of Figure 3.4. Note that the filters bLP(t) and f(t) and

the signals sLP(t), n(t), and y(t) are complex-valued in general.


a[k]∈A sLP(t)

a[k]∈A q[k]

(a)

(b)

n(t)

y(t) q(t)

t= kTs

q[k]g(t) bLP(t) f(t)

p[k]

z[k]

Figure 3.4: Equivalent simplied representations of the entral part of the passband PAM system: (a)

Equivalent baseband PAM system, (b) equivalent dis rete-time baseband PAM system. Note: The signal

y(t) in part (a) is slightly dierent from the y(t) in (3.12) in that it does not ontain the undesired omponentabout −2ωc. However, sin e this omponent is suppressed by the re eive lter f(t), this dieren e is of no onsequen e with respe t to the subsequent signals q(t) and q[k].

Equivalent baseband PAM signal. Inserting (3.7) into (3.14), we obtain

q(t) =

[K∑

k=1

a[k] g(t−kTs)]∗ bLP(t) ∗f(t) + n(t) ∗ f(t) (3.15)

=

K∑

k=1

a[k][g(t−kTs) ∗ bLP(t) ∗f(t)

]+ n(t) ∗ f(t) (3.16)

=

K∑

k=1

a[k] p(t−kTs) + z(t) , (3.17)

with the equivalent overall complex baseband pulse

p(t) , g(t) ∗ bLP(t) ∗f(t) (3.18)

and the equivalent complex filtered noise

z(t) , n(t) ∗ f(t) =√2[n(t) e−jωct

]∗f(t) . (3.19)

According to (3.18), the equivalent overall complex baseband pulse p(t) subsumes the joint action of

the transmit pulse g(t), the equivalent lowpass channel filter bLP(t), and the receive filter f(t). The

Fourier transform of p(t) is

P (jω) = G(jω)BLP(jω)F (jω) . (3.20)

Leaving aside the noise component z(t), the receive filter output signal q(t) in (3.17) has the form

of the baseband PAM signal sLP(t) =∑K

k=1 a[k] g(t − kTs) in (3.7), albeit with a pulse p(t) that is

different from the transmit pulse g(t) since it has been changed by the channel and receive filters.


Thus, the processing consisting of lowpass-bandpass transformation, channel filter, bandpass-lowpass

transformation, and receive filter preserves the structure of the original baseband PAM signal sLP(t)

while modifying its pulse.

Noise spectrum. There remains to calculate the power spectral density of the noise component7

Z(t) = N(t) ∗f(t) in (3.17), (3.19). We first have

SZ(jω) = SN (jω) |F (jω)|2 . (3.21)

The autocorrelation of N(t) =√2N(t) e−jωct is given by

RN(τ) = E

N(t) N∗(t−τ)

= E

[√2N(t) e−jωct

][√2N(t−τ) e−jωc(t−τ)

]∗

= 2EN(t)N∗(t−τ)

e−jωcτ = 2RN (τ) e−jωcτ (3.22)

and thus

SN(jω) = 2SN

(j(ω + ωc)

), (3.23)

so that (3.21) becomes

SZ(jω) = 2SN(j(ω+ωc)

)|F (jω)|2. (3.24)

In particular, if the channel noise N(t) is white with power spectral density SN (jω) = N0/2, then

SZ(jω) = N0 |F (jω)|2 . (3.25)

The equivalent discrete-time baseband PAM system. Finally, the noisy baseband PAM signal

q(t) =∑K

l=1 a[l] p(t− lTs) + z(t) in (3.17) is sampled at the symbol-rate time instants t = kTs. Let

q[k] , q(kTs) =

∫ ∞

−∞y(t) f(kTs−t) dt (3.26)

denote the sampled version of q(t). We obtain

q[k] =

K∑

l=1

a[l] p(kTs−lTs) + z(kTs) . (3.27)

We can write this as an equivalent discrete-time baseband PAM system:

q[k] =K∑

l=1

a[l] p[k−l] + z[k] , (3.28)

with the discrete-time impulse response

p[k] , p(kTs) (3.29)

and the discrete-time noise

z[k] , z(kTs) . (3.30)

7We temporarily write N(t), N(t), and Z(t) instead of n(t), n(t), and z(t) since these quantities are now considered

random.


The equivalent discrete-time baseband PAM system is depicted in Figure 3.4(b). The symbols a[k] are

filtered by the discrete-time LTI filter p[k], and discrete-time noise z[k] is added. The entire system

operates at the symbol rate Rs = 1/Ts. Note that all quantities involved (q[k], a[k], p[k], z[k]) are

complex-valued in general. The significant simplification resulting from the equivalent discrete-time

baseband PAM system should be evident upon comparing Figure 3.1 and Figure 3.4(b).

Spectral description. Sampling in the time domain corresponds to periodic continuation in the

frequency domain. More specifically, the (discrete-time) Fourier transform of the equivalent discrete-

time channel filter p[k] = p(kTs) is given by8

P (ejθ) ,

∞∑

k=−∞p[k] e−jθk =

1

Ts

∞∑

i=−∞P

(jθ− i2πTs

). (3.31)

That is, P (ejθ) is a periodized version of 1TsP(j θTs

), where the period is θ = 2π (corresponding to ω =

2π/Ts, i.e., 2π times the symbol rate 1/Ts). We recall from (3.20) that P (jω) = G(jω)BLP(jω)F (jω).

In order to derive a similar relation for the power spectral density of the equivalent discrete-time

noise Z[k], we consider the autocorrelation sequence of Z[k]:

RZ [l] , EZ[k]Z∗[k− l]

= E

Z(kTs)Z

∗((k−l)Ts)

= EZ(t)Z∗(t− lTs)

= RZ(lTs) , (3.32)

where the stationarity of Z(t) has been used. That is, RZ [l] is a sampled version of RZ(τ) =

EZ(t)Z∗(t − τ)

. Hence, the power spectral density SZ(e

jθ) of Z[k] is a periodized version of

the (scaled) power spectral density SZ(jω) of Z(t):

SZ(ejθ) ,

∞∑

l=−∞RZ [l] e

−jθl =1

Ts

∞∑

i=−∞SZ

(jθ− i2πTs

), (3.33)

where SZ(jω) = 2SN(j(ω+ωc)

)|F (jω)|2 according to (3.24). Note the similarity of (3.31) and (3.33).

For the important special case of white channel noise N(t), insertion of (3.25) yields

SZ(ejθ) = N0

1

Ts

∞∑

i=−∞

∣∣∣∣F(jθ− i2πTs

)∣∣∣∣2

. (3.34)

The representation by the equivalent discrete-time baseband PAM system radically simplifies the

analysis and design of passband PAM systems. It will be the basis for our further treatment of

passband PAM in subsequent sections.

3.2.3 Problems

Problem 3.2.1. Develop a “real-valued” representation of the equivalent discrete-time basebandPAM system in (3.28), i.e., specify the relation between qR[k], qI[k] on the one hand and aR[l], aI[l]on the other. Show that in general there occurs “cross talk” between the inphase component and the

8With an abuse of notation, we use the same symbol P for the Fourier transform P (jω) of p(t) and the (discrete-time)

Fourier transform P (ejθ) of p[k].


quadrature component. Which condition on p[k] or, equivalently, P (ejθ) guarantees cross-talk-freetransmission?

Problem 3.2.2. Consider a passband PAM system with g(t) = f(t) = 1√Ts

sinc(π tTs

)where sinc(x) =

sin(x)x . The channel is distortionless (BLP(jω) ≡ 1) and the channel noise is white (SN (jω) ≡ N0/2).

a) Calculate P (jω).

b) Calculate P (ejθ) and p[k].

c) How does the slicer input q[k] depend on the symbols a[l]?

d) Calculate the power spectrum and mean power of the equivalent discrete-time baseband noiseZ[k].

Problem 3.2.3. Consider a passband PAM system in which the transmit and receive filters areidealized lowpass filters with bandwidth Bg = Bf = 1/(2Ts) and the transfer function of the channelfilter has triangular shape,

BLP(jω) =

1− |ω|

2π/Ts, |ω| < 2π

Ts

0 , otherwise .(3.35)

The channel noise is white with power spectrum N0/2.

a) Sketch the Fourier transform P (ejθ) of the equivalent discrete-time baseband pulse p[k].

b) Sketch the power spectrum and autocorrelation function of the equivalent discrete-time basebandnoise Z[k].

Problem 3.2.4. Consider a passband PAM system in which the receive filter is an idealized lowpassfilter with bandwidth Bf = 1/(2Ts). The power spectrum of the channel noise is (at least within thetransmission band about carrier frequency ωc) SN (jω) = Kω2.

a) Sketch the power spectrum SZ(ejθ) of the discrete-time baseband noise Z[k].

b) Calculate the mean power of the discrete-time baseband noise Z[k].

Problem 3.2.5. Consider a passband PAM system with equivalent baseband channel noise powerspectrum SN (jω). The receive filter is given by

F (jω) =

1√SN(jω)

, |ω| < πTs

0 , otherwise.(3.36)

Calculate the power spectrum and mean power of the equivalent discrete-time baseband noise Z[k].

Problem 3.2.6. Consider a passband PAM system in which the receive filter is omitted for the sakeof design simplicity. Calculate the noise variance at the slicer input for white channel noise N(t).

3.3 Intersymbol Interference and Nyquist Pulses 57

Problem 3.2.7. Sketch a pulse p(t) such that the output of the sampler is q[k] = a[k − 1] + z[k](delay by one symbol period). How must the passband PAM receiver be modified so that this delaydoes not result in symbol errors?

Problem 3.2.8. Show that the noise variance at the sampler output can be equivalently expressed asσ2Z = 1

2π

∫∞−∞ SZ(jω) dω and σ2Z = 1

2π

∫ π−π SZ(e

jθ) dθ. Hint : You should also consider the time domain.

Problem 3.2.9. Consider a passband PAM system as described and analyzed in Subsections 3.2.1and 3.2.2, however with the difference that there is a frequency offset ω∆ and a phase offset φ at thereceiver, i.e., (3.3) is replaced with

y(t) = y(t) ·√2 ej[(−ωc+ω∆)t+φ] . (3.37)

Derive the equivalent discrete-time baseband system for this case.

Problem 3.2.10. Consider a passband PAM system as described and analyzed in Subsections 3.2.1and 3.2.2, however with the difference that there is a time offset τ at the receiver, i.e., the output ofthe receive filter is sampled at the time instants t = kTs + τ .

a) Derive the equivalent discrete-time baseband system for this case.

b) The transmit, (baseband) channel, and receive filters are given by

G(jω) = BLP(jω) = F (jω) = rect

(ω ;

3

2

π

Ts

). (3.38)

Calculate the Fourier transform of the equivalent discrete-time channel filter, P (ejθ), and discussits dependence on the sampling phase offset τ .

3.3 Intersymbol Interference and Nyquist Pulses

The complex numbers q[k] in (3.28) form the input to the slicer (quantizer) and are used for deter-

mining the detected symbols a[k] in a symbol-by-symbol manner. Let us consider the slicer input at

a specific time index k, assuming an infinite sequence of transmitted symbols a[l] (−∞ < l < ∞).

Modifying the summation range in (3.28) accordingly, we obtain

q[k] =∞∑

l=−∞a[l] p[k−l] + z[k] (3.39)

=

∞∑

l=−∞a[k−l] p[l] + z[k] (3.40)

= a[k]︸︷︷︸desired symbol

p[0] + a[k−1] p[1] + a[k−2] p[2] + · · ·︸︷︷︸ISI term containing past symbols

+ a[k+1] p[−1] + a[k+2] p[−2] + · · ·︸︷︷︸ISI term containing future symbols

+ z[k] . (3.41)


This expression shows that q[k] depends not only on the desired/current symbol a[k]; it is also influ-

enced by the past symbols a[k − 1], a[k − 2], · · · and the future symbols a[k + 1], a[k + 2], · · · . This

undesirable effect is known as intersymbol interference (ISI). The influence that the past and future

symbols have upon q[k] depends on the values of p[l] for l > 0 and l < 0, respectively.

3.3.1 ISI-Free Transmission and the Nyquist Criterion

We desire that ISI be avoided, i.e., that

q[k]!= a[k] + z[k] . (3.42)

Comparing with (3.40), we see that a necessary and sufficient condition for ISI-free transmission is

that the overall discrete-time pulse p[k] is given by

p[k] = δ[k] =

1 , k = 0

0 , k 6= 0(3.43)

or, equivalently formulated in terms of p(t),

p(kTs) = δ[k] . (3.44)

Let us calculate the (discrete-time) Fourier transform of both sides of (3.43) or (3.44). Using (3.31)

with θ/Ts = ω and recalling that the Fourier transform of δ[k] is 1, we obtain

1

Ts

∞∑

i=−∞P

(j(ω − i

2π

Ts

))≡ 1 . (3.45)

This is the famous Nyquist criterion. It is the frequency-domain version of the time-domain condition

p(kTs) = δ[k], formulated in terms of P (jω). The essential point is that the shifted replicas of P (jω)

add to a constant. Pulses p(t) that satisfy the Nyquist criterion, i.e., that yield ISI-free transmission,

are called Nyquist pulses. Nyquist pulses and the Nyquist criterion are illustrated in Figure 3.5.

From Figure 3.5(b), it is clear that bandlimited pulses p(t) with bandwidth Ωp < π/Ts or, equiv-

alently, Bp < 1/(2Ts) cannot satisfy the Nyquist criterion since there would be “gaps” (intervals of

zero value) in 1Ts

∑∞i=−∞ P

(j(ω − i 2πTs

)). Hence, the minimum bandwidth of a Nyquist pulse is

Ωp,min =π

Tsor Bp,min =

1

2Ts=Rs

2. (3.46)

Conversely, the maximum symbol rate that can be transmitted without ISI by means of passband PAM

using a given pulse bandwidth Bp is

Rs,max = 2Bp . (3.47)

The difference Bp − Bp,min = Bp − 1/(2Ts) ≥ 0 is called excess bandwidth. Note that according to

(3.20), Bp ≥ 1/(2Ts) requires that the bandwidths of all filters involved in p(t)— i.e., g(t), bLP(t),

and f(t)—are ≥ 1/(2Ts). Since the transmit bandwidth (i.e., the bandwidth of the passband PAM


−3Ts −2Ts 2Ts 3Ts

(a)

πTs

2πTs

ω

p(t)

1TsP(j(ω+ 2π

Ts))

1TsP (jω)

(b)

1

1TsP(j(ω− 2π

Ts))

1

−Ts 0 Tst

0− 2πTs

− πTs

Figure 3.5: Illustration of Nyquist pulses and the Nyquist riterion: (a) Time-domain hara terization|

the samples of p(t) at t = kTs must be equal to δ[k], (b) frequen y-domain hara terization| the shifted

Fourier transform versions

1TsP(j(ω − i 2πTs

))must add to 1 (Nyquist riterion).

transmit signal s(t) =√2 Re

∑∞k=−∞ a[k] g(t − kTs) · ejωct

) is given by BT = 2Bg, it follows that

BT,min = 2Bg,min = 2Bp,min = 21

2Ts=

1

Ts= Rs . (3.48)

Thus, a given (desired) symbol rate Rs constitutes a lower bound on the required transmit bandwidth

BT: for ISI-free transmission, the transmit bandwidth must be at least the symbol rate. Equivalently,

a given transmit bandwidth constitutes an upper bound on the symbol rate that can be achieved:

Rs,max = BT . (3.49)

That is, for ISI-free transmission, the symbol rate cannot be larger than the transmit bandwidth. For

example, the maximum symbol rateRs that can be transmitted without ISI over a telephone/voiceband

channel with bandwidth BT = 3.1 kHz is 3.1 ksymbol/s. Carefully note that this relation to the

transmit bandwidth applies to the symbol rate Rs, not the bit rate Rb.

In order to understand more clearly the implications of the Nyquist criterion regarding the shape

of P (jω), let us suppose that p(t) is bandlimited with bandwidth Ωp bounded as π/Ts ≤ Ωp ≤ 2π/Ts.

That is, the excess bandwidth is ≤ π/Ts, which implies that only adjacent terms in the sum in (3.45)

overlap. This bandwidth restriction is usually satisfied in practice provided that p(t) is bandlimited at

all (in particular, it is satisfied in Figure 3.5(b)). Let us consider (3.45) on the fundamental frequency

interval 0 ≤ ω ≤ 2π/Ts. Due to our excess bandwidth restriction, only the i = 0 and i = 1 terms in


(3.45) are nonzero on this interval. Hence, (3.45) implies

1

Ts

[P (jω) + P

(j(ω− 2π

Ts

))]≡ 1 for 0 ≤ ω ≤ 2π

Ts. (3.50)

This is a symmetry condition that constrains the shape of the roll-offs of P (jω) about ω = π/Ts and

ω = −π/Ts (see Figure 3.5(b)).

3.3.2 Examples of Nyquist Pulses

Next, we discuss three important examples of Nyquist pulses.

• Rectangular pulse. The rectangular pulse of length T < 2Ts,

p(t) = rect

(t ;T

2

)with T < 2Ts , (3.51)

is a Nyquist pulse since it satisfies p(kTs) = δ[k]. More generally, any pulse p(t) that is zero for

|t| ≥ Ts and one at the origin (p(0) = 1) satisfies the Nyquist criterion. Note, however, that

these pulses are not bandlimited.

• Sinc pulse. The sinc pulse

p(t) = sinc

(πt

Ts

), (3.52)

with sinc(x) = sin(x)x , has a rectangular Fourier transform

P (jω) = Ts rect

(ω ;

π

Ts

). (3.53)

The Nyquist criterion is satisfied. Note that p(t) is bandlimited; in fact, this is the extreme

case of minimum bandwidth Ωp,min = π/Ts or Bp,min = 1/(2Ts), i.e., zero excess bandwidth.

The temporal decay of p(t) is merely as 1/|t|, and thus the temporal concentration of p(t)

is poor. Whereas the sinc pulse is a theoretically important limiting case, it is not practical

since it corresponds to a non-realizable idealized lowpass filter; furthermore, it would lead to a

catastrophic sensitivity to sampling phase errors (see Subsection 3.3.3).

• Raised-cosine pulse. The bandlimited Nyquist pulses that are commonly used in practice are

the raised-cosine pulses

p(t) = wα(t) sinc

(πt

Ts

)with wα(t) =

cos(απ t

Ts

)

1−(2α t

Ts

)2 , (3.54)

with the roll-off factor α ∈ [0, 1]. The Fourier transform of p(t) can be shown to be given by

P (jω) =

Ts , |ω| ≤ (1−α) πTs

Ts2

[1− sin

(Ts2α

(|ω|− π

Ts

))], (1−α) π

Ts≤ |ω| ≤ (1+α) π

Ts

0 , |ω| ≥ (1+α) πTs.

(3.55)


−Ts Ts−2Tst

− πTs

00 πTs

2πTs

ω

Ts

(b)(a)

2Ts − 2πTs

1p(t)

P (jω)

Figure 3.6: Raised- osine pulses: (a) Time-domain fun tion, (b) Fourier transform.

This shows that p(t) is bandlimited with bandwidth

Ωp = (1+α)π

Ts= (1+α)Ωp,min . (3.56)

The excess bandwidth is απ/Ts, which is α times the minimum bandwidth Ωp,min = π/Ts. The

“raised-cosine” shaped roll-off of P (jω) occurs between (1−α)π/Ts and (1+α)π/Ts (see Figure

3.6). A larger roll-off factor α implies a slower spectral roll-off, a larger bandwidth, and a better

temporal concentration of the pulse. For α = 0, the raised-cosine pulse degenerates to the sinc

pulse (with rectangular Fourier transform, i.e., discontinuous roll-off), which is an impractical

limiting case. Since for α 6= 0 the raised-cosine pulses decay with respect to time as 1/|t|3, theycan be approximated (in discrete time) using FIR filters.

3.3.3 Eye Diagrams

An eye diagram consists of many overlaid traces of small sections of the real part or the imaginary

part of the baseband PAM signal at the output of the receive filter (see (3.17)),

q(t) =∞∑

k=−∞a[k] p(t−kTs) + z(t) ,

for many different symbol sequences a[k]. Eye diagrams can easily be generated using an oscilloscope

triggered according to the symbol timing. They are useful design and diagnosis tools during the

analytical and simulation design phase of modems, and they also allow a quick check of the performance

of modems in the field. If the random data symbols a[k] are statistically independent (and thus

ergodic), and if a sufficient number of signal traces are overlaid, then an eye diagram summarizes all

possible ISI waveforms.

An eye diagram provides a visual representation of several relevant features of the signal, as shown

in Figure 3.7 for the real part of q(t) in the noise-free case, assuming a real-valued, binary-antipodal

symbol alphabet:

• The transmission is free of ISI if (in the absence of noise) all traces pass through the two symbol

points marked by circles. (In the general case, this must be checked both for the real part and

the imaginary part of q(t).)


c a

b

Figure 3.7: Eye diagram (noise-free ase) for a binary symbol alphabet and its salient features.

(a) (b)

Figure 3.8: Eye diagrams for raised- osine pulses with (a) 25% and (b) 100% ex ess bandwidth ( orre-

sponding to α = 1/4 and α = 1, respe tively).

• With ISI, i.e., when the pulse p(t) does not satisfy the Nyquist criterion, the eye will tend to

close vertically and thus noise may cause the two symbols to be confused. Hence, the vertical eye

opening (labeled “a” in Figure 3.7) indicates the presence and amount of ISI and the immunity

to noise. A wider vertical eye opening indicates less ISI and, thus, better noise immunity. If the

eye is completely closed vertically, then symbol errors may occur even in the absence of noise.

• The ideal sampling instants are at the points of maximum vertical eye opening. It follows

that the horizontal eye opening (“b”) indicates the immunity to errors in the sampling time

phase. A wider horizontal eye opening indicates better immunity. In particular, the sinc pulse

in (3.52) has maximum vertical eye opening (since it is a Nyquist pulse) but zero horizontal eye

opening . Thus, the sinc pulse presupposes perfect timing recovery, which is not practical. In

the practically important case of a raised-cosine pulse, the horizontal eye opening is wider for a

larger excess bandwidth, i.e., a larger roll-off factor α, as shown in Figure 3.8.

• The slope of the inside eye lid (“c”) indicates the sensitivity to jitters in the timing phase.

3.3.4 Zero-Forcing Receive Filter Design

According to (3.20), the Fourier transform of the overall pulse p(t) can be written as

P (jω) = H(jω)F (jω) , (3.57)

where

H(jω) , G(jω)BLP(jω) (3.58)


is the Fourier transform of the received pulse h(t) , g(t) ∗ bLP(t). The received pulse is the (base-

band) pulse that is “seen” at the input of the receiver. It summarizes the transmit pulse g(t) and the

(baseband) channel filter bLP(t).

We now assume that the received pulse h(t) is given. Can we design the receive filter f(t) such

that ISI is avoided (or “forced to zero”)? Let us suppose that H(jω) 6= 0 for |ω| ≤ Ω0, where Ω0 is

larger than the minimum bandwidth Ωp,min = π/Ts (i.e., the excess bandwidth of the filter H(jω) is

> 0). Furthermore, let PN(jω) be the Fourier transform of an arbitrary Nyquist pulse with bandwidth

Ω0. ISI will be avoided if we design the receive filter f(t) such that P (jω) = PN(jω). Due to (3.57),

this can be achieved by setting

F (jω) =

PN(jω)

H(jω), |ω| ≤ Ω0

0 , |ω| > Ω0 .

(3.59)

Note that we can set F (jω) = 0 for |ω| > Ω0 since here PN(jω) = 0 as well.

Of course, this “zero-forcing design” presupposes knowledge of the transfer function of the channel

filter, BLP(jω), which in many applications is not available. Apart from this limitation, an important

drawback of the zero-forcing design is that F (jω) will be very large at frequencies where H(jω) =

G(jω)BLP(jω) is very small, and hence the noise (which is filtered only by F (jω)) will be enhanced

(amplified) at these frequencies. Indeed, with SZ(jω) = 2SN(j(ω+ωc)

)|F (jω)|2 according to (3.24),

the noise power at the slicer input (i.e., the mean power of the filtered noise Z[k] = Z(kTs)) is obtained

as

PZ = EZ2[k] = EZ2(t) =1

2π

∫ ∞

−∞SZ(jω) dω =

1

π

∫ Ω0

−Ω0

SN(j(ω+ωc)

) ∣∣∣∣PN(jω)

H(jω)

∣∣∣∣2

dω . (3.60)

Note that H(jω) occurs in the denominator of the integrand, so small values of H(jω) will lead to

noise enhancement. An alternative to the zero-forcing design is the mean-square error design that will

be discussed in a different context in Subsection 4.2.2.

3.3.5 Problems

Problem 3.3.1. Consider a pulse p(t) whose Fourier transform is given by

P (jω) =

Ts

(1− |ω|

2π/Ts

), |ω| < 2π

Ts


a) Sketch P (jω) and P (ejθ).

b) Is p(t) a Nyquist pulse?

Problem 3.3.2. Extend the Nyquist criterion such that it allows a time delay by an integer multipleof the symbol period, i.e., the output of the sampler is allowed to be q[k] = a[k− k0]+ z[k], with somegiven k0 ∈ Z.


Problem 3.3.3. Consider a passband PAM system with channel bandwidth Bc = 1.7 kHz. What isthe maximum symbol rate for ISI-free transmission?

Problem 3.3.4. Passband PAM is used to transmit a bit rate of Rb = 33.6 kbit/s over a bandpasschannel with bandwidth Bc = 3.1 kHz. A raised-cosine pulse with 50% excess bandwidth is used.Calculate the number of symbols, Ma, that must be contained in the symbol alphabet.

Problem 3.3.5. Consider passband PAM with symbol rate Rs = 2ksymbol/s. What is the transmitbandwidth if a raised-cosine pulse with roll-off factor α = 0.25 is used?

Problem 3.3.6. Passband PAM is used to transmit a bit rate of Rb = 12kbit/s over a passbandchannel with bandwidth Bc = 8kHz. The symbol alphabet consists of four symbols. Calculate themaximum possible excess bandwidth.

Problem 3.3.7. Consider a pulse of the form p(t) = w(t) sinc(π tTs

).

a) Show that p(t) is a Nyquist pulse if and only if w(0) = 1.

b) Derive an expression for the Fourier transform of p(t).

c) Assume that w(t) is bandlimited to the band [−Bw, Bw]. What is the bandwidth of p(t)?

Problem 3.3.8. Consider the pulse p(t) = sinc(2π t

Ts

)(note that the argument of the sinc function

is 2π tTs, not π t

Ts). Check if p(t) is a Nyquist pulse by operating

a) in the time domain;

b) in the frequency domain.

Problem 3.3.9. Repeat Problem 3.3.8 for p(t) = sinc(π2

tTs

).

Problem 3.3.10. Calculate the energy of a raised-cosine pulse.

Problem 3.3.11. According to the manual of a digital passband PAM modem, the receive filter isgiven by f(t) = sinc

(π tTs

)(up to implementation inaccuracies which are to be disregarded) and the

overall pulse p(t) (assuming a distortionless channel) is a raised-cosine pulse with roll-off factor 0.5.Is this possible?

Problem 3.3.12. Consider passband PAM with G(jω) = F (jω) =√R(jω), where R(jω) is the

Fourier transform of a raised-cosine pulse. The channel is distortionless (B(jω) ≡ 1) with white noise(SN (jω) ≡ N0/2). Calculate p[k] and SZ(e

jθ).

Problem 3.3.13. Consider passband PAM with zero-mean and white symbols a[k]. The transmitpulse is given by G(jω) =

√R(jω), where R(jω) is the Fourier transform of a raised-cosine pulse with

roll-off factor α. Calculate the mean power of the transmit signal and discuss its dependence on α.

3.4 Error Probability of Passband PAM 65

Problem 3.3.14. For digital telephony, an analog speech signal is sampled with sampling frequency8 kHz and quantized with 8 bits per sample. Determine the minimum channel bandwidth for ISI-freetransmission by means of passband PAM, assuming that the following signal constellation is used:

a) BPSK;

b) QPSK;

c) 16-QAM.

Problem 3.3.15. Consider passband PAM transmission over a bandpass channel with bandwidthBc = 10kHz. Calculate the bit rate for ISI-free transmission, assuming that the following signalconstellation and overall pulse are used:

a) QPSK, with p(t) having minimum bandwidth;

b) BPSK, with p(t) having 50% excess bandwidth;

c) 16-QAM, with p(t) having 100% excess bandwidth.

Problem 3.3.16. Sketch an eye diagram for a symbol alphabet consisting of four equally spacedreal-valued symbols, assuming

a) ideal transmission (i.e., free of ISI, timing jitter, and noise);

b) transmission corrupted by the following imperfections (consider them separately):

• ISI;

• timing jitter;

• noise.

Problem 3.3.17. The transmit pulse g(t) of a passband PAM system is a raised-cosine pulse with50% roll-off. The (baseband) channel filter is given by bLP(t) = e−t/τ u(t), where u(t) denotes the unitstep function. Calculate the receive filter transfer function F (jω) according to the zero-forcing receivefilter design.

3.4 Error Probability of Passband PAM

In this section, we consider the probability of a symbol error and the probability of a bit error for a

passband PAM system under the simplifying assumptions of Gaussian noise and ISI-free transmission.

We will see that an “almost closed-form” expression of the symbol error probability exists only in

certain special cases. However, in Subsection 3.4.3 we will derive simple and useful bounds and

approximations for the symbol error probability.

3.4.1 Probability of a Symbol Error for ISI-Free Transmission

We first consider the probability of a symbol error, assuming a channel with linear distortion and

Gaussian noise. With ISI present, calculating the symbol error probability is a difficult task in general.

Hence, we shall assume that there is no ISI. More specifically, we assume that the overall pulse p(t) is


R(3)

R(2)p[0]a(1)

p[0]a(3)

p[0]a(2)

R(1)

Figure 3.9: De ision regions R(m)for Ma = 3. (The region R(1)

is shaded.)

a Nyquist pulse up to an arbitrary gain factor p[0] that may be 6= 1, i.e. (cf. (3.43)–(3.45)),

p[k] = p(kTs) = p[0]δ[k] ,1

Ts

∞∑

i=−∞P

(j(ω − i

2π

Ts

))≡ p[0] . (3.62)

It should be observed that this notion of “ISI-free transmission” does not mean that the channel filter

b(t) may not introduce linear distortion; rather, it means that the entire system as characterized by

the overall pulse p(t) (consisting of transmit filter, channel filter, and receive filter) is ISI-free.

Slicer operation and decision regions. According to (3.40), for ISI-free transmission as described

above the noisy signal samples at the slicer input are given by

Q[k] = p[0]A[k] + Z[k] , −∞ < k <∞ . (3.63)

For a given transmitted symbol a ∈ A =a(1), a(2), · · ·, a(Ma)

, the corresponding signal component

(without noise) at the slicer input would be p[0]a. Accordingly, the slicer is assumed to implement

the rounding operation

A[k] = argmina∈A

∣∣Q[k]− p[0]a∣∣ . (3.64)

(This is slightly different from (3.5) since now we include the scaling factor p[0].) An equivalent

formulation of the slicer operation is

A[k] =

a(1) , Q[k] ∈ R(1)

a(2) , Q[k] ∈ R(2)

...

a(Ma) , Q[k] ∈ R(Ma) ,

(3.65)

where the mth decision region R(m) consists of all those points of the complex plane that are closest

to the associated scaled symbol p[0]a(m), as opposed to all other scaled symbols p[0]a(n) (n 6= m). The

construction of the decision regions R(m) is depicted in Figure 3.9. Note that all decision regions are

disjoint and their union is the entire complex plane.


Conditional symbol error probability, Part I. We are now ready to consider the probability of

a symbol error

PEs , PA[k] 6= A[k]

. (3.66)

Assuming that the processes A[k] and N(t) are strict-sense stationary, PEs does not depend on k.

According to the total probability theorem, we have

PEs =

Ma∑

m=1

PEs∣∣A[k]=a(m)

pA(a(m)

), (3.67)

with pA(a(m)

)= P

A[k] = a(m)

. Thus, we have expressed PEs in terms of the conditional symbol

error probability PEs∣∣A[k]=a(m)

, i.e., the symbol error probability given that a specific symbol a(m)

was transmitted. This conditional symbol error probability will now be studied. For A[k] = a(m), a

symbol error occurs if and only if (iff) A[k] 6= a(m), which according to (3.65) is the case iff Q[k] 6∈ R(m).

Hence,

PEs∣∣A[k]=a(m)

= P

A[k] 6= a(m)

∣∣A[k]=a(m)

= PQ[k] 6∈ R(m)

∣∣A[k]=a(m). (3.68)

We now recall that Q[k] = p[0]A[k] + Z[k]. Under the condition that A[k] = a(m), we have

Q[k] = p[0]a(m) + Z[k] . (3.69)

Thus, Q[k] 6∈ R(m) is equivalent to

Z[k] 6∈ R(m) , (3.70)

where R(m) denotes the decision region R(m) shifted by p[0]a(m) (note that R(m) is located about the

origin of the complex plane since p[0]a(m) is mapped to 0). Therefore, (3.68) becomes

PEs∣∣A[k]=a(m)

= P

Z[k] 6∈ R(m)

∣∣A[k]=a(m)

= PZ[k] 6∈ R(m)

, (3.71)

where in the last step we have used the statistical independence of the noise random variable Z[k] and

the transmitted symbol A[k]. This last expression for PEs∣∣A[k]=a(m)

can be written as the integral

of the probability density function fZ(z) of the complex random variable Z[k] over the complement

of the region R(m):

PEs∣∣A[k]=a(m)

=

∫

z /∈R(m)

fZ(z) dz =

∫∫

(zR,zI) /∈R(m)

fZR,ZI(zR, zI) dzR dzI . (3.72)

These two integrals, just as the two probability density functions fZ(z) and fZR,ZI(zR, zI), are equiva-

lent because the complex random variable Z[k] = ZR[k] + jZI[k] is equivalent to the two-dimensional

(2-D) real random variable (ZR[k], ZI[k]). Note that in the second integral, we are interpreting R(m)

not as a region of the complex plane but as a region of the 2-D real plane, which of course is fully

equivalent.

Noise at slicer input. To further develop the integral in (3.72), we need to specify fZ(z). We

first calculate the variance σ2Z of Z[k]. For channel noise N(t) being zero-mean, white, and stationary


with power spectral density SN (jω) = N0/2, the noise Z(t) obtained after the receive filter F (jω) is

zero-mean with power spectral density SZ(jω) = N0 |F (jω)|2 (see (3.25)). Thus, the noise variance at

the slicer input is (recall that Z[k] is zero-mean and thus its variance equals its mean power, which

equals the mean power of Z(t))

σ2Z = E|Z[k]|2

= E

|Z(t)|2

=

1

2π

∫ ∞

−∞N0 |F (jω)|2 dω = N0Ef , (3.73)

where Ef is the energy of the receive filter impulse response f(t).

We now assume that the channel noise N(t) is a Gaussian random process. This is a reasonably

accurate model for many practical channels9. Since Z(t) is derived from N(t) through a linear filtering

operation and Z[k] = Z(kTs), the random variables Z[k] are jointly complex Gaussian. Moreover, if

the bandwidth of the receive filter f(t) is smaller than the carrier frequency ωc—i.e., if the pass band

of the bandpass version of f(t) does not overlap DC, which is always satisfied in practice— then it

can be shown (see Subsection A.3.1 in Appendix A) that Z[k] is a stationary, zero-mean, circularly

symmetric random process. Due to circular symmetry and Gaussianity, the real part ZR[k] and the

imaginary part ZI[k] are statistically independent and Gaussian with identical variances equal to σ2Z/2

(see (A.43), noting that mean powers and variances are equal since the processes are zero-mean).

Hence, the probability density function of Z[k] is

fZ(z) = fZR,ZI(zR, zI)

= fZR(zR) fZI

(zI)

=1√

2πσ2Z/2exp

(−1

2

( zR

σZ/√2

)2) 1√2πσ2Z/2

exp

(−1

2

( zI

σZ/√2

)2)

=1

πσ2Zexp

(−z

2R + z2Iσ2Z

)(3.74)

=1

πσ2Zexp

(−|z|2σ2Z

), (3.75)

with σ2Z = N0Ef according to (3.73). Note that fZ(z) is a function of |z| and thus circularly symmetric

(rotationally invariant) about the origin of the complex plane.

Conditional symbol error probability, Part II. With (3.74), the conditional symbol error prob-

ability (3.72) becomes

PEs∣∣A[k]=a(m)

=

1

πσ2Z

∫∫

(zR,zI) /∈R(m)

exp

(−z

2R + z2Iσ2Z

)dzR dzI . (3.76)

This is depicted schematically in Figure 3.10. Alternatively, the conditional error probability

PEs∣∣A[k] = a(m)

can be expressed in terms of the conditional probability of a correct decision,

PCs∣∣A[k]=a(m)

(which is often easier to calculate):

9The practical relevance of the Gaussian model can be explained by the central limit theorem, which states that under

certain conditions the superposition of n independent random variables yields a Gaussian random variable as n → ∞.


0

fZR,ZI(zR, zI)

R(1)

Figure 3.10: Cal ulation of the onditional symbol error probability PEs |A[k]=a(1)

for Gaussian noise:

The noise probability density fun tion fZR,ZI(zR, zI) =

1πσ2

Zexp(− z2R+z2I

σ2Z

)is integrated over the omplement

(shaded) of the shifted de ision region R(1).

PEs |A[k]=a(m) = 1 − PCs |A[k]=a(m)

= 1 −∫∫

R(m)fZR,ZI

(zR, zI) dzR dzI

= 1 − 1

πσ2Z

∫∫

R(m)

exp

(−z

2R + z2Iσ2Z

)dzR dzI . (3.77)

For regions R(m) of general shape, the above integrals cannot be calculated in closed form. For some

special shapes, however, they can be expressed in terms of the Q-function (see Subsection 3.4.2).

Unconditional symbol error probability. Inserting (3.76) into (3.67) yields the unconditional

symbol error probability as

PEs =1

πσ2Z

Ma∑

m=1

[∫∫

(zR,zI) /∈R(m)

exp

(−z

2R + z2Iσ2Z

)dzR dzI

]pA(a(m)

). (3.78)

Alternatively, inserting (3.77) into (3.67) yields

PEs = 1 − 1

πσ2Z

Ma∑

m=1

[∫∫

R(m)

exp

(−z

2R + z2Iσ2Z

)dzR dzI

]pA(a(m)

). (3.79)

3.4.2 Binary Symbols

In the simplest case, that of binary symbols A[k] ∈ a(1), a(2), the decision regions R(1), R(2) are

two complementary half-planes as depicted in Figure 3.11. An example is given by BPSK symbols

a(1), a(2) = −1, 1. As we shall see in Subsection 3.4.3, the binary case is more generally important

than it may appear at first sight.

To simplify matters, we can transform all complex variables (equivalently, rotate the complex

plane) such that, say, the real axis passes through the scaled symbols p[0]a(1) and p[0]a(2). After this

rotation, the real axis is orthogonal to the straight line separating the two half-planes (the dashed

line in Figure 3.11(a)). Since the original 2-D probability density function fZR,ZI(zR, zI) is circularly

symmetric, i.e., rotationally invariant, the probability density function of the real part Z ′R[k] of the


R(1)

fZ(z−p[0]a(1)

)

d

d

z′R

(a)

(b)

R(2)

p[0]a(2)

0 d/2

p[0]a(1)

fZ′R(z′R)

d

Figure 3.11: Cal ulation of the symbol error probability for the ase of binary symbols: (a) S aled sym-

bols and 2-D probability density fun tion of the noise random variable Z[k] (drawn around p[0]a(1)), (b)equivalent 1-D problem (after shift by p[0]a(1) and rotation).

new random variable Z ′[k] still is Gaussian with zero mean and variance σ2Z/2. Assuming that a(1) was

transmitted, the error event Z[k] /∈ R(1) now is equivalent to the event Z ′R[k] > d/2 with d = |p[0]|da,

where da = |a(2)−a(1)| is the distance between the symbols (see Figure 3.11(b)). Hence, the conditional

symbol error probability is obtained as (recall that σ2Z = N0Ef )

PEs |A[k]=a(1)

= Q

(d/2

σZ/√2

)= Q

(d√2σZ

)= Q

(|p[0]|da√2N0Ef

), (3.80)

where Q(x) is the Q-function.10 Because of symmetry, the same result is obtained for the other

10The Q-function is defined as

Q(x) ,1√2π

∫ ∞

x

e−ξ2/2dξ , x ∈ R . (3.81)

Q(x) is the probability that a Gaussian random variable X with mean 0 and variance 1 is larger than x, i.e., Q(x) =PX > x. More generally, if X is a Gaussian random variable with mean µX and variance σ2

X , then

PX > x = Q(x− µX

σX

). (3.82)

The Q-function is a strictly decreasing function. A small decrease/increase of the argument x tends to produce a verylarge increase/decrease of Q(x). We note the relation Q(−x) = 1 − Q(x). The graph of the Q-function is provided inFigure 3.12. There exist the following lower and upper bounds,

1√2π

1

x

(1− 1

x2

)e−x2/2 < Q(x) <

1√2π

1

xe−x2/2 , (3.83)

which are quite tight for arguments x > 3. Another useful upper bound is

Q(x) ≤ 1

2e−x2/2 . (3.84)


4 4.5 5 5.5 6

0 1 2 3 41•10-5

1•10-4

1•10-3

1•10-2

1•10-1

10

1

1•10-5

1•10-4

1•10-6

1•10-7

1•10-8

1•10-9

1•10-10

0 1 2 3 4

4 4.5 5 5.5 6

Q(x)

Q(x)

x

x

Figure 3.12: Graph of the Q-fun tion.


conditional symbol error probability PEs |A[k]=a(2)

. Inserting into (3.67), we then obtain for the

unconditional symbol error probability

PEs = Q(

|p[0]|da√2N0Ef

)pA(a(1))+ Q

(|p[0]|da√2N0Ef

)pA(a(2))

= Q(

|p[0]|da√2N0Ef

)[pA(a(1))+ pA

(a(2))].

But pA(a(1))+ pA

(a(2))= 1, and thus

PEs = Q(

|p[0]|da√2N0Ef

)with da = |a(2)−a(1)| . (3.85)

Note that PEs does not depend on the symbol probability distribution pA(a(m)

). Finally, note that

in the binary case one symbol corresponds to one bit, so the symbol error probability equals the bit

error probability.

3.4.3 Bounds and Approximations for the Symbol Error Probability

We have seen above that for passband PAM, closed-form expressions of the symbol error probability

can only be found for simple special shapes of the decision regions R(m), e.g., in the binary case where

the decision regions are half-planes. In the general case, then, it is useful to have at least simple

closed-form bounds on the symbol error probability.

Upper bound (union bound). According to (3.64), the receiver detects the symbol a ∈ A for

which p[0]a is closest to Q[k]. Let us assume that a specific symbol a(m) was transmitted. Under

this condition, a symbol error occurs if and only if (iff) Q[k] is outside the decision region R(m),

which is the case iff Q[k] is closer to some other scaled symbol p[0]a(n) (n 6= m) than to p[0]a(m) or,

equivalently, iff Q[k] is not closest to p[0]a(m). For example, given that a(1) was transmitted, an error

occurs if Q[k] is closer to p[0]a(2) than to p[0]a(1), or if Q[k] is closer to p[0]a(3) than to p[0]a(1), or

if Q[k] is closer to p[0]a(4) than to p[0]a(1), etc. Phrased mathematically, the error event “Q[k] is not

closest to p[0]a(m)” is equivalent to the union of events11∨

n 6=m Emn, where Emn (n 6= m) denotes the

event that Q[k] is closer to p[0]a(n) than to p[0]a(m). Hence, the conditional symbol error probability,

given that the symbol a(m) was transmitted, becomes

PEs∣∣A[k]=a(m)

= P

Q[k] not closest to p[0]a(m)

∣∣A[k]=a(m)

= P

∨

n 6=m

Emn

∣∣∣∣∣A[k]=a(m)

≤∑

n 6=m

PEmn

∣∣A[k]=a(m). (3.86)

The above inequality is the union bound of probability theory. Note that this inequality becomes an

equality iff all events Emn are mutually exclusive, which is only true in the binary case.

11Here,∨

n6=m Emn denotes the event that one or more events Emn are true.


p[0]a(3)

p[0]a(3)

H13

p[0]a(3)

p[0]a(2)

p[0]a(1)

p[0]a(1)

p[0]a(1)

p[0]a(2)

p[0]a(2)

R(1)

R(1)

H12

Figure 3.13: Geometry of the union bound for Ma = 3 and m = 1: there is R(1) = H12 ∪H13.

The union bound (3.86) can be interpreted geometrically as shown in Figure 3.13 for the case

Ma = 3 (cf. Figure 3.10). The complement R(m) of the decision region R(m) is the union of Ma − 1

half-planes Hmn, each of which is associated to one of the other scaled symbols p[0]a(n) (n 6= m):

R(m) =⋃

n 6=m

Hmn . (3.87)

Note that the half-plane Hmn corresponds to the event Emn since

Emn = “Q[k] is closer to p[0]a(n) than to p[0]a(m)” = “Q[k] ∈ Hmn” (given A[k]=a(m)) . (3.88)

The conditional symbol error probability,

PEs∣∣A[k]=a(m)

= P

Q[k] 6∈ R(m)

∣∣A[k]=a(m)

= PQ[k] ∈ R(m)

∣∣A[k]=a(m), (3.89)

is the integral of fQ|A(q |a(m)

), the conditional probability density function of Q[k] given A[k] = a(m),

over R(m) =⋃

n 6=mHmn. Since the half-planes Hmn overlap in general, the integral of fQ|A(q |a(m)

)

over R(m) is ≤ the sum of the integrals over all half-planes Hmn, which gives the union bound (3.86).


Note that the integral of fQ|A(q |a(m)

)over Hmn is the probability of the event Q[k] ∈ Hmn (given

A[k]=a(m)), which is equal to the event Emn (given A[k]=a(m)).

The event Emn =“Q[k] is closer to p[0]a(n) than to p[0]a(m)” is in turn equivalent to the symbol

error event Es in the binary case, i.e., in the case where the symbol alphabet consists only of the two

symbols a(m) and a(n) (compare Figure 3.13 with Figure 3.11). Thus, we can apply (3.85) to obtain

PEmn

∣∣A[k]=a(m)

= Q(dmn√2σZ

)(3.90)

with dmn =∣∣p[0]

(a(m) − a(n)

)∣∣ and σZ =√N0Ef . This probability is sometimes termed a pairwise

error probability (PEP). Using (3.90), the union bound in (3.86) finally becomes

PEs∣∣A[k]=a(m)

≤∑

n 6=m

Q(dmn√2σZ

)=∑

n 6=m

Q(∣∣p[0]

(a(m)−a(n)

)∣∣√

2N0Ef

). (3.91)

Lower bound. A lower bound on PEs∣∣A[k]=a(m)

is easily obtained by noting that any one of the

half-planes Hmn is completely contained in R(m) (see Figure 3.13); hence, the integral of fQ|A(q |a(m)

)

over R(m) must be ≥ the integral over any single half-plane Hmn. Equivalently,

PEs∣∣A[k]=a(m)

= P

Q[k] ∈ R(m)

∣∣A[k]=a(m)

≥ PQ[k] ∈ Hmn

∣∣A[k]=a(m)

= Q(dmn√2σZ

), n 6= m. (3.92)

Since this bound is valid for all n 6= m, we take the tightest bound. This is obtained by using the

value of n maximizing Q(

dmn√2σZ

)or, equivalently, minimizing dmn =

∣∣p[0](a(m) − a(n)

)∣∣. Let

d(m) , minn 6=m

dmn = |p[0]| d(m)a with d(m)

a , minn 6=m

∣∣a(m)−a(n)∣∣ (3.93)

denote the minimum distance from p[0]a(m) to any other scaled symbol p[0]a(n). The tighest lower

bound is then given by

PEs∣∣A[k]=a(m)

≥ Q

(d(m)

√2σZ

)= Q

(|p[0]| d(m)

a√2N0Ef

). (3.94)

Combining this lower bound with the union bound in (3.91), we finally obtain the double bound

Q(d(m)

√2σZ

)≤ P

Es∣∣A[k]=a(m)

≤∑

n 6=m

Q(dmn√2σZ

), (3.95)

where d(m) = |p[0]| minn 6=m

∣∣a(m)−a(n)∣∣, dmn =

∣∣p[0](a(m) − a(n)

)∣∣, and σZ =√N0Ef .

Useful approximations. As before, we consider the conditional symbol error probability

PEs∣∣A[k] = a(m)

for a given transmitted symbol a(m). Let us assume that there are Nm other


symbols a(n) (n 6= m) that have the minimum distance d(m)a = minn 6=m

∣∣a(m) − a(n)∣∣ from a(m), i.e.,

the symbol a(m) has Nm nearest neighbors. The union bound (right-hand side in (3.91)) now becomes

∑

n 6=m

Q(dmn√2σZ

)= NmQ

(d(m)

√2σZ

)+ remaining terms (i.e., terms with dmn > d(m))

≈ NmQ(d(m)

√2σZ

), (3.96)

where, as before, d(m) = |p[0]|d(m)a . For this approximation of the union bound, the Ma − 1 − Nm

remaining terms with larger distances have been neglected; this is usually justified since the Q-function

decreases rapidly already for a small increase in its argument (provided that the “signal-to-noise ratio”d(m)√2σZ

is not too small). In general, this approximation of the union bound is not itself an upper bound

on the conditional symbol error probability PEs∣∣A[k]=a(m)

. However, it is nevertheless extremely

useful as a simple approximation of PEs∣∣A[k]=a(m)

:

PEs∣∣A[k]=a(m)

≈ NmQ

(d(m)

√2σZ

). (3.97)

This approximation is quite good if d(m)√2σZ

is not too small and if the distances dmn with dmn > d(m)

are not too close to d(m). The approximation shows that PEs∣∣A[k]=a(m)

depends primarily on the

minimum scaled symbol distance d(m) = |p[0]|d(m)a and on the number Nm of nearest neighbors. The

dependence on d(m) is very strong (since it is via the Q-function) whereas the dependence on Nm is

only linear. Hence, if Nm is not too large, a cruder but even simpler approximation is obtained by

replacing Nm with 1:

PEs∣∣A[k]=a(m)

≈ Q

(d(m)

√2σZ

). (3.98)

Unconditional symbol error probability. With PEs =∑Ma

m=1 PEs∣∣A[k] = a(m)

pA(a(m)

),

the above bounds and approximations for the conditional symbol error probability PEs∣∣A[k]=a(m)

immediately yield bounds and approximations for the unconditional symbol error probability PEs.In particular, (3.95) yields the double bound

Ma∑

m=1

Q(d(m)

√2σZ

)pA(a(m)

)≤ PEs ≤

Ma∑

m=1

∑

n 6=m

Q(dmn√2σZ

) pA

(a(m)

). (3.99)

Similarly, (3.97) yields the approximation

PEs ≈Ma∑

m=1

NmQ(d(m)

√2σZ

)pA(a(m)

). (3.100)

This approximation can be further simplified. Let dmin denote the globally minimum distance, i.e.,

dmin = minm

d(m) = |p[0]| minm

d(m)a = |p[0]| da , (3.101)


with the globally minimum symbol distance

da , minm

d(m)a = min

m

minn 6=m

∣∣a(m)−a(n)∣∣

= minm,n:m6=n

∣∣a(m)−a(n)∣∣ . (3.102)

Then, in the sum (3.100) the terms with the globally minimum distance will be dominant, so that the

other terms can be neglected and we obtain

PEs ≈∑

m: d(m)=dmin

NmQ(dmin√2σZ

)pA(a(m)

)=

[∑

m: d(m)=dmin

Nm pA(a(m)

)]Q(dmin√2σZ

). (3.103)

In many cases, the signal constellation (symbol alphabet) has a regular structure such that all

symbols have the same minimum distance from their nearest neighbor(s), i.e., d(m)a ≡ da and hence

d(m) ≡ dmin. Here, (3.100) and (3.103) both simplify to

PEs ≈[

Ma∑

m=1

Nm pA(a(m)

)]Q(dmin√2σZ

)= N Q

(dmin√2σZ

)= N Q

(|p[0]| da√2N0Ef

), (3.104)

where

N , ENM =

Ma∑

m=1

Nm pA(a(m)

)(3.105)

is the mean number of nearest neighbors. These results show that PEs is critically dependent on the

minimum symbol distance da, as previously claimed in Subsection 2.3.3.

Symbol error probability in terms of mean symbol power. The minimum symbol distance

da is related to the mean symbol power PA as

da = γ√PA . (3.106)

Here, the factor γ depends on the shape of the signal constellation and on the probability distribution

of the symbols, and it is invariant to a common scaling of the symbols12. Inserting (3.106) in (3.104),

we see that our approximation for PEs can also be expressed in terms of PA,

PEs ≈ N Q(|p[0]| γ

√PA

2N0Ef

). (3.107)

3.4.4 Probability of a Bit Error

In the previous subsection, we have calculated the probability of a symbol error, PEs. In practice,

however, one is primarily interested in the probability of a bit error, denoted PEb in what follows.

The relation between these two error probabilities depends on the symbol alphabet, the bits→ symbol

mapping, and the transmit probabilities. Often, this relation can be calculated exactly.

12More specifically, if all symbols a(m) ∈ A are multiplied by a complex factor K, i.e., a(m)′ = Ka(m), then PA′ =

|K|2PA and da′ = |K|da, and there is again da′ = γ√PA′ .


Bit error probability for a Gray code. In the following, we derive a simple approximate relation

between the symbol error probability PEs and the bit error probability PEb for the practically

important case where a Gray code is used for the bits → symbol mapping. As discussed in Subsection

2.2.1, a Gray code has the desirable property that a symbol error for which the (incorrectly) detected

symbol is adjacent to the transmitted symbol (i.e., a nearest-neighbor symbol) results in only a single

bit error. For the Gaussian channel with sufficiently high signal-to-noise ratio, it follows from the

shape of the Q-function that such a “nearest-neighbor symbol error” is the only type of symbol error

we need to be concerned about since the probability of any other symbol error will be much smaller

than that of a nearest-neighbor symbol error.

Each bit is part of a symbol that consists of l = ldMa bits. With regard to the detection of this

symbol, the following events can occur:

• Event Cs: No symbol error, i.e., the symbol is detected correctly (hence, there does not occur a

bit error either, so PEb |Cs = 0);

• Event E1: Nearest-neighbor symbol error;

• Event E2: Other symbol error (much less likely than E1, so PE2 ≪ PE1 and thus PEs ≈PE1).

Since these three events exhaust all possibilities and are mutually exclusive, the bit error probability

can be decomposed (according to the total probability theorem) as follows:

PEb = PEb |Cs︸︷︷︸0

PCs + PEb |E1 PE1︸︷︷︸≈PEs

+ PEb |E2 PE2︸︷︷︸≪PE1

≈ PEb |E1PEs . (3.108)

Since a nearest-neighbor symbol error implies exactly one bit error, i.e., exactly one of the l bits

is incorrect, we have PEb |E1 = 1/l. So we finally obtain the following relation between bit error

probability PEb and symbol error probability PEs for the case where a Gray code is used:

PEb ≈ 1

lPEs =

1

ldMaPEs . (3.109)

Bit error probability versus Eb/N0. We will finally express the bit error probability PEb as a

function of Eb/N0, where

Eb , PS Tb (3.110)

is the mean transmit energy per bit , i.e., the mean energy that is expended on transmitting one bit.

Here, PS is the mean transmit power previously considered in Subsection 2.3.2 and Tb is the bit

period. The representation of PEb versus Eb/N0 is particularly important because it allows a fair

and meaningful comparison with other modulation schemes.


10

16–QAM

5 15

Eb

N0/dB

PE

b

0 2010−10

10−9

10−8

10−7

10−6

10−5

10−3

10−4

10−2

10−1

100

BPSKQPSK

8–PSK

Figure 3.14: PEb vs. Eb/N0: approximation (3.112) with

|p[0]|√EgEf

= 1 (distortionless hannel and

mat hed lter ase). Note that this approximation is ina urate for low SNR.

For a zero-mean and white symbol sequence, PS = PAEg/Ts according to (2.25). Furthermore,

Tb = Ts/(ldMa). Thus, Eb in (3.110) is obtained as

Eb =PAEg

Ts

TsldMa

=PAEg

ldMa. (3.111)

It follows that PA can be expressed in terms of Eb as PA = ldMaEg

Eb. Inserting this into (3.107) and

using (3.109), we obtain

PEb ≈ αQ(β

√Eb

N0

), with α =

NldMa

, β = γ

√ldMa

2

|p[0]|√EgEf

. (3.112)

Here, N and γ depend on the shape of the signal constellation and on the probability distribution of

the symbols. This result is valid under the following assumptions: the transmission is ISI-free, the

channel noise is white and Gaussian, all symbols have equal nearest-neighbor distance, a Gray code

is used for the bits → symbol mapping, and the SNR (i.e., Eb/N0) is sufficiently high so that the

probability of a “non-nearest-neighbor symbol error” is negligible.

In Figure 3.14, the approximation (3.112) is plotted for various signal constellations. For the

calculations underlying this figure, we have assumed equally likely symbols and we have set |p[0]|√EgEf

= 1,

which is satisfied if B(jω) ≡ 1 and if a matched filter is used for f(t) (see Section 3.5). We see that

the PEb-vs.-Eb/N0 performance of 8-PSK and 16-QAM is similar, and that of BPSK (= binary

antipodal) and QPSK (= 4-QAM) is altogether equal. The PEb-vs.-Eb/N0 performance of 8-PSK

and 16-QAM is significantly poorer than that of BPSK and QPSK. This is mainly because for a given

value of Eb/N0, the symbols of the 8-PSK and 16-QAM constellations have to be more densely spaced


than the symbols of the BPSK and QPSK constellations. This is the price paid for the higher spectral

efficiency of 8-PSK and 16-QAM.

3.4.5 Problems

Problem 3.4.1. Consider passband PAM usingMa ≥ 4 symbols a(m) equally spaced on the real axis.

a) Calculate the mean number of nearest neighbors for the following situations:

• All symbols are equally likely.

• Each one of the two outermost symbols is twice as likely as each one of the inner symbols.

• Each one of the two outermost symbols is half as likely as each one of the inner symbols.

b) Discuss the dependence of your results on Ma. What happens as Ma → ∞?

Problem 3.4.2. Consider passband PAM using a 16-QAM signal constellation. Calculate the meannumber of nearest neighbors for the following situations:

a) All symbols are equally likely.

b) Each one of the four corner symbols is twice as likely as each one of the other symbols.

Problem 3.4.3. Consider passband PAM transmission corrupted by white channel noise with powerspectral density N0/2 = 2 ·10−5 W/Hz. The symbol rate is 12 ksymbol/s. Calculate the noise varianceat the slicer input for the following choices of the receive filter f(t) = F0 f(t) with F

20 = 6 ·105 s−2:

a) f(t) is a raised-cosine pulse with α = 0.5.

b) f(t) is an idealized lowpass filter with cutoff frequency fc =1

2Tsand gain Ts in the pass band.

c) f(t) = e−t/τ u(t) with τ = 0.1ms.

Problem 3.4.4. Prove the bounds in (3.83). Hint : Use the definition of the Q-function as an integralof the Gaussian density and integrate by parts.

Problem 3.4.5. Compare Q(x) with the bounds in (3.83) for

a) x = 2;

b) x = 3;

c) x = 4;

d) x = 5.

Problem 3.4.6. Consider passband PAM transmission using binary symbols A[k] ∈− da

2 ,da2

with

da = 1. The transmission is ISI-free with |p[0]| = 1.5. The noise component at the slicer input, Z[k],is zero-mean circularly symmetric complex Gaussian with variance σ2Z = 2 · 0.152.

a) Calculate the symbol/bit error probability.


b) By what factor is the symbol/bit error probability increased if da is reduced to 0.9?

Problem 3.4.7. Consider passband PAM with real symbol alphabet A =±(i− 1

2

)i=1,2,···,Ma/2

.

All symbols are equally likely. The transmission is ISI-free. The noise component at the slicer input,Z[k], is zero-mean circularly symmetric complex Gaussian.

a) Calculate an exact expression of the symbol error probability. Evaluate this expression forMa = 8, |p[0]| = 1.5, and σ2Z = 3.47 · 10−3.

b) Consider a scaling of the signal constellation— i.e., all symbols are scaled by a common factorK—and reformulate the general expression found in part a) as a function of the mean symbolpower PA (plus other parameters).

Problem 3.4.8. Consider ISI-free passband PAM transmission with |p[0]| = 1.5. All symbols areequally likely. The noise component at the slicer input, Z[k], is zero-mean circularly symmetriccomplex Gaussian with variance σ2Z = 2.5 · 10−3.

a) Calculate the probability of a symbol/bit error for the binary symbol alphabet A =− 1

2 ,12

.

b) On the right side of the symbol 12 , a third real symbol is added whose distance from 1

2 is largerthan 1, i.e., the symbol alphabet now is A =

−1

2 ,12 ,

32+ǫ

with ǫ > 0. Calculate the probability

of a symbol error and discuss its dependence on ǫ. For which values of ǫ is the influence of thethird symbol on the symbol error probability negligible? (You are free to define the exactmeaning of “negligible.”)

Problem 3.4.9. Consider passband PAM transmission using the symbol alphabet A =− 3

2 , −12 ,

12 ,

32

. The channel noise is Gaussian and white with power spectral density N0/2 = 0.8 · 10−5 W/Hz.

The receive filter is given by f(t) = F0 e−t/τ u(t) with F 2

0 = 6 · 105 s−2 and τ = 0.1ms. The overallpulse p(t) is a scaled raised-cosine pulse with roll-off factor 0.25 and |p[0]| = 0.1

√W.

a) Calculate the probability of a symbol error for the case of equally likely symbols.

b) Assume now that the two outer symbols are equally likely and the two inner symbols are equallylikely but each one of the two inner symbols is more likely, by a factor K > 1, than each one ofthe outer symbols. How large must K be if the probability of a symbol error is to be larger by10% than the value obtained for equally likely symbols in part a)?

Problem 3.4.10. Consider passband PAM transmission using the symbol alphabet A =− 1

2 ,12

.

The two symbols are equally likely. The channel noise is Gaussian and white with power spectraldensity N0/2. The receive filter is given by f(t) = F0 e

−t/τ u(t) with F 20 = 6 · 105 s−2 and τ = 0.1ms.

The overall pulse p(t) is a scaled raised-cosine pulse with roll-off factor 0.25 and |p[0]| = 0.1√W.

Calculate the maximum value of N0 such that the symbol error probability is at most 10−5.

Problem 3.4.11. Consider passband PAM transmission of a symbol sequence A[k] ∈− 1

2 ,12

over

a distortionless channel (B(jω) ≡ 1) with white Gaussian noise (power spectral density N0/2 =10−5 W/Hz). The transmit pulse is given by

g(t) = G0 cos

(πt

Ts

)rect

(t;Ts2

)with G2

0 = 10mW , Ts = 20ms . (3.113)

The receiver comprises an idealized lowpass filter of bandwidth B = 1/Ts, a symbol-rate sampler, anda slicer. Calculate an approximation for the symbol error probability that is based on the assumptionthat the distortion of the transmit pulse caused by the receive filter is negligible.


Problem 3.4.12. Consider passband PAM transmission using the QPSK signal constellation A =1, j ,−1, −j. The transmission is ISI-free with |p[0]| = 1.5. The real part and imaginary part ofthe noise at the slicer input are statistically independent Gaussian random variables with identicalstandard deviation σZ/

√2 = 0.25.

a) Sketch the signal constellation.

b) Sketch the signal constellation as seen at the slicer input and sketch the decision regions.

c) Let Q denote the input to the slicer. Assume that the symbol −1 was transmitted. Calculatethe (conditional) probability of the event “Q is closer to j than to −1” and the (conditional)probability of the event “Q is closer to 1 than to −1” and compare the two results.

d) Calculate the conditional symbol error probability given that the symbol −1 was transmitted.Furthermore calculate an upper bound (the union bound) and a lower bound on this conditionalsymbol error probability, and compare your results.

e) Calculate the unconditional symbol error probability and the corresponding upper and lowerbounds. Do these results depend on the transmit probabilities of the symbols?

Problem 3.4.13. Consider ISI-free passband PAM transmission using a 16-QAM signal constellationwith minimum symbol distance da. All symbols are equally likely. The real part and imaginary partof the noise at the slicer input are statistically independent Gaussian random variables with varianceσ2Z/2.

a) Calculate the probability of a symbol error.

b) Calculate the approximation (3.104).

c) Compare the results of parts a) and b) for |p[0]|da/(√2σZ) = 4.

Problem 3.4.14. Consider ISI-free passband PAM transmission using a 4N -QAM signal constellationwith minimum symbol distance da. All symbols are equally likely. The real part and imaginary partof the noise at the slicer input are statistically independent Gaussian random variables with varianceσ2Z/2.

a) Calculate the approximation (3.104) for the probability of a symbol error.

b) What do you obtain for N → ∞?

Problem 3.4.15. The ITU-T13 standard V.29 for voiceband data transmission at 9.6 kbit/s uses thesignal constellation shown in Figure 3.15.

a) What is the symbol rate?

b) Sketch the decision regions and explain why exact calculation of the symbol error probability isdifficult.

c) Assume, as usual, that the transmission is ISI-free and that the real part and imaginary partof the noise at the slicer input are statistically independent Gaussian random variables withvariance σ2Z/2. Calculate the union bound on the conditional symbol error probability giventhat the symbol 1 + j was transmitted.

13ITU-T = Telecommunication Standardization Sector of the International Telecommunication Union.


1

3

5

1 3 5

Ima[k]

Rea[k]

Figure 3.15: Signal onstellation of the V.29 standard.

d) Assume in addition that all symbols are equally likely. Calculate the approximation (3.103) forthe (unconditional) probability of a symbol error. (Why does the approximation (3.104) notapply?) Compare to the result of c) for |p[0]| = 1 and σ2Z/2 = 0.04.

e) Still assuming equally likely symbols, calculate the mean symbol power PA = E|A[k]|2 of theV.29 signal constellation.

f) Consider a 16-QAM signal constellation whose minimum symbol distance da is chosen such thatthe mean symbol power of the 16-QAM constellation is equal to that of the V.29 constellationobtained in part e). Calculate the approximation (3.104) for the symbol error probability of the16-QAM constellation. Compare your result with the approximate symbol error probability ofthe V.29 constellation calculated in part d).

Problem 3.4.16. Consider ISI-free passband PAM transmission (with |p[0]| = 1) using binary sym-bols A[k] ∈

− da

2 ,da2

that are equally likely. The real part ZR[k] and imaginary part ZI[k] of the

noise component at the slicer input are statistically independent. Furthermore, ZR[k] is exponentiallydistributed, i.e., fZR

(ζ) = λ e−λζ u(ζ). Calculate the probability of a symbol error.

Problem 3.4.17. Consider ISI-free passband PAM transmission (with |p[0]| = 1) using the symbolalphabet A =

− 1

2 ,12

. The symbol −1

2 occurs with probability pA(− 1

2

)= p. The real part ZR[k]

and imaginary part ZI[k] of the noise component at the slicer input are statistically independent.Furthermore, ZR[k] is uniformly distributed in the interval [−1, 1). Anticipating that p > 1

2 , themodem designer chose the decision threshold of the slicer as 1

4 .

a) Calculate the probability of a symbol error, PEs, as a function of p.

b) For which value of p is PEs minimized, and what is the resulting minimum value of PEs?c) How much is p allowed to deviate from its optimum value (calculated in part b)) if PEs is

supposed to increase by no more than 10%?

Problem 3.4.18. Consider the QPSK signal constellation A = 1, j ,−1, −j and symbol probabil-ities pA(1) = pA(j) = 0.3, pA(−1) = 0.2. Shift the constellation such that PA is minimized withoutchanging the symbol error probability.

3.5 Matched Filter 83

Problem 3.4.19. Consider ISI-free passband PAM transmission over a channel with white Gaussiannoise (power spectrum N0/2). For simplicity, the transmit and receive filters are assumed to beidealized lowpass filters with zero excess bandwidth. Compare the bandwidths and symbol errorprobabilities (use approximation (3.104) if it simplifies matters) obtained for the following signalconstellations, assuming equal bit rate Rb and equal mean transmit power PS :

• BPSK;

• QPSK;

• 16-QAM.

Hint : Note that the assumption is equal mean transmit power, not equal mean symbol power. Alsonote that the symbol rates of the different modulation schemes may be different.

Problem 3.4.20. Consider passband PAM transmission using 16 symbols and a Gray code. Thesignal-to-noise ratio is high. What is the maximum symbol error probability if the bit error probabilityis specified to be no larger than 10−6?

3.5 Matched Filter

Next, we consider the design of the receive filter f(t). Suppose that the received pulse

h(t) = g(t) ∗ bLP(t) (3.114)

is given. The received pulse encompasses the transmit filter g(t) and the channel filter bLP(t). What

is the best receive filter f(t) for the given h(t)? We can understand “best” to mean minimum symbol

error probability PEs. Unfortunately, a closed-form expression of PEs is not available for general

signal constellations. However, for ISI-free transmission, equal nearest-neighbor distances da, white

Gaussian channel noise with power spectrum N0/2, and sufficiently high signal-to-noise ratio, the

following approximation for PEs is provided by (3.104):

PEs ≈ PEs , N Q(

|p[0]| da√2N0Ef

). (3.115)

(This approximation becomes exact in the binary case, cf. Subsection 3.4.2.) Minimization of the

approximate symbol error probability PEs with respect to f(t) is equivalent to maximization of the

argument of the Q-function,

κ ,|p[0]| da√2N0Ef

, (3.116)

with respect to f(t). This quotient depends on the receive filter f(t) via |p[0]| = |(h ∗ f)(0)| in the

numerator and via Ef in the denominator.

We will next consider a different approach to optimizing f(t), which will be seen to be equivalent

to maximization of κ.


3.5.1 Definition and Derivation of the Matched Filter

Definition of the matched filter. For ISI-free transmission, the input to the slicer is given by (cf.

(3.63))

Q[k] = Qs[k] + Z[k] , with Qs[k] , p[0]A[k] . (3.117)

Here, Qs[k] and Z[k] are respectively the signal component and noise component at the slicer input.

Let us consider the signal-to-noise ratio (SNR) at the slicer input that is defined as

SNR ,PQs

σ2Z, (3.118)

where PQs = E|Qs[k]|2 and σ2Z are respectively the signal power and noise variance (= mean noise

power because Z[k] is zero-mean) at the slicer input. We have PQs = |p[0]|2 E|A[k]|2 = |p[0]|2 PA

and σ2Z = N0Ef , and thus

SNR =|p[0]|2PA

N0Ef. (3.119)

We see that√SNR = |p[0]|

√PA√

N0Efis proportional to14 κ = |p[0]|da√

2N0Efin (3.116). This shows that maximiza-

tion of√SNR—or, equivalently, of SNR— is equivalent to maximization of κ and, thus, equivalent

to minimization of the approximate symbol error probability PEs in (3.115). Note, however, that

the expression (3.115) is only valid for Gaussian white noise whereas the SNR expression (3.119) does

not assume Gaussianity of the white noise.

The receive filter f(t) maximizing the SNR at the slicer input for a given received pulse h(t) will

be termed matched filter and denoted fMF(t):

fMF(t) , argmaxf(·)

SNR = argmaxf(·)

|p[0]|2PA

N0Ef. (3.120)

If the white channel noise is Gaussian, fMF(t) also minimizes the approximate symbol error probability

PEs in (3.115).

Some mathematical fundamentals. For easy calculation of fMF(t), we need a few mathematical

fundamentals (cf. Subsection B.2.1 in Appendix B). The inner product of two signals x(t) and y(t) is

defined as

〈x, y〉 ,

∫ ∞

−∞x(t) y∗(t) dt =

1

2π

∫ ∞

−∞X(jω)Y ∗(jω) dω . (3.121)

Note that 〈y, x〉 = 〈x, y〉∗. The inner product is linear in the first argument:

〈cx, y〉 = c〈x, y〉 (c is a complex constant factor); 〈x1+ x2, y〉 = 〈x1, y〉+ 〈x2, y〉 . (3.122)

14Recall from (3.106) that da = γ√PA, where the factor γ depends on the shape of the signal constellation and on

the probability distribution of the symbols but does not depend on the receive filter f(t) that we wish to optimize.


The norm of a signal x(t) is defined as

‖x‖ ,√

〈x, x〉 =

√∫ ∞

−∞|x(t)|2 dt =

√1

2π

∫ ∞

−∞|X(jω)|2 dω . (3.123)

It is seen that the energy of x(t), Ex =∫∞−∞ |x(t)|2 dt = 1

2π

∫∞−∞ |X(jω)|2 dω, is the squared norm:

Ex = ‖x‖2 and ‖x‖ =√Ex. The Cauchy-Schwarz inequality bounds the magnitude of an inner

product as

|〈x, y〉| ≤ ‖x‖ ‖y‖ , (3.124)

with equality if and only if the signals x(t) and y(t) are “colinear,” i.e., x(t) = c y(t).

Derivation of the matched filter. We are now ready to derive the matched filter by straightfor-

ward application of the Cauchy-Schwarz inequality. With p(t) = h(t) ∗ f(t), we obtain

p[0] = p(0) =

∫ ∞

−∞h(0−τ) f(τ) dτ = 〈f, h〉 with h(t) , h∗(−t) . (3.125)

Hence, the SNR in (3.119) can be written as

SNR =PA

N0

|〈f, h〉|2‖f‖2 , (3.126)

and the matched filter can be written as (note that PA and N0 do not depend on f(t))

fMF(t) = argmaxf(·)

|〈f, h〉|2‖f‖2 . (3.127)

Now, setting in the Cauchy-Schwarz inequality (3.124) x(t) = f(t) and y(t) = h(t), dividing by ‖f‖,and squaring, (3.124) becomes

|〈f, h〉|2‖f‖2 ≤ ‖h‖2, (3.128)

with equality if and only if f(t) = c h(t). Hence, |〈f,h〉|2‖f‖2 is maximized for f(t) = c h(t), and thus the

matched filter is obtained as

fMF(t) = c h(t) = c h∗(−t) , (3.129)

with arbitrary c∈C, c 6=0.

Discussion. It follows from (3.128) that the maximum value of |〈f,h〉|2‖f‖2 is given by ‖h‖2 = ‖h‖2 = Eh.

Inserting into (3.126) yields the maximum SNR as

SNRmax = SNR∣∣∣f(t)=fMF(t)

=PA

N0Eh , (3.130)

which does not depend on c. Moreover, it is important to note that this maximum SNR depends on

the energy of the received pulse h(t) but not otherwise on the precise shape of h(t).


Furthermore, the maximum value of |p[0]|√Ef

= |〈f,h〉|‖f‖ is given by

√Eh. Inserting this into (3.115), we

obtain the minimum approximate symbol error probability for Gaussian white noise as

PEsmin = PEs∣∣∣f(t)=fMF(t)

= N Q(da√2

√Eh

N0

). (3.131)

This is the minimum possible approximate symbol error probability that can be achieved for the given

received pulse h(t). It is achieved in the absence of ISI and when the matched filter is used. Since

in the presence of ISI the symbol error probability can only be higher, N Q(

da√2

√EhN0

)is known as

the matched filter bound . The matched filter bound depends on the energy Eh but not otherwise on

the precise shape of h(t). The corresponding approximate bit error probability PEb was shown as a

function of Eb/N0 in Figure 3.14.

The transfer function (frequency response) of the matched filter fMF(t) = c h∗(−t) is

FMF(jω) = cH∗(jω) . (3.132)

We see that the magnitude response is |FMF(jω)| ∝ |H(jω)|. This means that the matched filter

passes more signal energy at those frequencies ω where |H(jω)| is larger, i.e., where the received

signal is stronger (note that this corresponds to a higher SNR since the noise level, N0/2, is the same

at all frequencies). This makes sense since our goal was to maximize the SNR at the slicer input.

Furthermore, the phase response is argFMF(jω)

= φ0 − argH(jω) (where φ0 = argc). This

means that the matched filter compensates the phase response of the received pulse h(t) up to a

constant: the dispersion inherent to h(t) (variation with respect to ω of the group delay of h(t)) is

compensated so that the components of h(t) ∗ fMF(t) at various frequencies are optimally “aligned”

at time t = 0. This corresponds to a maximization of |(h ∗ f)(0)| and, thus, a maximization of the

signal power at the slicer input, |Qs[k]|2 = |p[0]A[k]|2 = |(h ∗ f)(0)|2 |A[k]|2.

3.5.2 Equivalent Discrete-Time Baseband System and Nyquist Criterion

We now consider passband PAM transmission using the matched filter. Without loss of generality,

we set c = 1 for simplicity, resulting in the receive filter f(t) = fMF(t) = h∗(−t). The equivalent

baseband PAM system using f(t) = h∗(−t) is depicted in part (a) of Figure 3.16 (cf. Figure 3.4).

Overall pulse. For f(t) = h∗(−t), the overall pulse p(t) is given by

p(t) = h(t) ∗ fMF(t) = h(t) ∗ h∗(−t) =

∫ ∞

−∞h(t′)h∗(t′−t) dt′ = rh(t) . (3.133)

This is the autocorrelation function of the received pulse h(t). The Fourier transform of p(t) is

P (jω) = H(jω)FMF(jω) = H(jω)H∗(jω) = |H(jω)|2 . (3.134)

With p(t) = rh(t), the equivalent discrete-time pulse p[k] = p(kTs) is obtained as

p[k] = ρh[k] , (3.135)


a[k]g(t)

h(t)

bLP(t)

n(t)

y(t)h∗(−t)

q(t)

t = kTs

q[k]

a[k]ρh[k]

q[k]

(b)

(SN (jω) ≡ N0

)

z[k]

(a)

(SZ(e

jθ) = N0Sh(ejθ))

Figure 3.16: (a) Equivalent baseband PAM system with sampled mat hed lter ( f. Figure 3.4(a)), (b) orresponding equivalent dis rete-time baseband PAM system ( f. Figure 3.4(b)).

with the autocorrelation sequence

ρh[k] , rh(kTs) =

∫ ∞

−∞h(t)h∗(t−kTs) dt . (3.136)

The Fourier transform of the equivalent discrete-time pulse p[k] = ρh[k] is given by (recall that

ρh[k] = rh(kTs) and that the Fourier transform of rh(t) is |H(jω)|2)

P (ejθ) = Sh(ejθ) ,

∞∑

k=−∞ρh[k] e

−jθk =

∞∑

k=−∞rh(kTs) e

−jθk =1

Ts

∞∑

i=−∞

∣∣∣∣H(jθ− i2πTs

)∣∣∣∣2

. (3.137)

The function Sh(ejθ) is the Fourier transform of the autocorrelation sequence ρh[k]; because of the

periodic continuation (folding) in (3.137), it is called the folded spectrum associated to the received

pulse h(t).

Noise power spectrum. Let us next consider the power spectrum of the equivalent discrete-time

noise Z[k] (the noise component at the slicer input) for white channel noise N(t) with power spectrum

SN (jω) ≡ N0/2 and for the matched receive filter f(t) = h∗(−t). Using F (jω) = H∗(jω) in (3.34),

we obtain

SZ(ejθ) = N0

1

Ts

∞∑

i=−∞

∣∣∣∣H(jθ− i2πTs

)∣∣∣∣2

= N0 Sh(ejθ) . (3.138)

Equivalent discrete-time baseband system. We are now ready to specialize the equivalent

discrete-time baseband PAM system (3.28) for the matched filter case f(t) = h∗(−t). With p[k] = ρh[k]

according to (3.135), the slicer input as given by (3.28) is obtained as

q[k] =

K∑

l=1

a[l] ρh[k− l] + z[k] , (3.139)


where the power spectrum of Z[k] is given by (3.138). This is depicted in Figure 3.16(b) (cf. Figure

3.4(b)). We see that the equivalent discrete-time filter has impulse response ρh[k] and transfer function

Sh(ejθ); furthermore, we have shown above that the power spectrum of the equivalent discrete-time

noise Z[k] equals N0Sh(ejθ). Thus, the folded spectrum Sh(e

jθ) determines both the transfer function

of the equivalent discrete-time filter and the power spectrum of the equivalent discrete-time noise.

Matched filter and Nyquist criterion. Suppose we use the matched filter, so p(t) = rh(t) and

P (jω) = |H(jω)|2. In addition, we also desire ISI-free transmission15, i.e., p(t) must be a Nyquist

pulse. It follows that P (jω) = |H(jω)|2 must satisfy the Nyquist criterion (3.45), i.e.,

1

Ts

∞∑

i=−∞

∣∣∣∣H(j(ω − i

2π

Ts

))∣∣∣∣2

≡ 1 or equivalently Sh(ejθ) ≡ 1 . (3.140)

Since H(jω) = G(jω)BLP(jω), this constrains the design of the transmit filter G(jω). Note that not

the Fourier transform H(jω) of the received pulse, but its squared magnitude |H(jω)|2 must satisfy

the Nyquist criterion! Consequently, the magnitude |H(jω)| must be the square root of the Fourier

transform of a Nyquist pulse. This means that

H(jω) =√PN(jω) e

jΦ(ω) , (3.141)

where PN(jω) is the Fourier transform of some Nyquist pulse and Φ(ω) is an arbitrary phase function.

If, in particular, PN(jω) is the Fourier transform of a raised-cosine pulse, then the pulse h(t) defined

by H(jω) =√PN(jω) is known as a root-raised-cosine pulse.

Reformulated in the time domain, (3.140) becomes

ρh[k] ≡∫ ∞

−∞h(t)h∗(t−kTs) dt = δ[k] . (3.142)

It is easily verified that this implies that the shifted pulses hk(t) , h(t−kTs) are orthonormal , i.e.,

〈hk, hl〉 ≡∫ ∞

−∞h(t−kTs)h∗(t−lTs) dt = δk,l ≡

1, k = l

0, k 6= l .(3.143)

Furthermore, inserting (3.140) into (3.138) yields

SZ(ejθ) ≡ N0 , (3.144)

which means that the discrete-time noise Z[k] is white. We can summarize these results as follows:

For white channel noise and ISI-free transmission using the matched receive filter, the shifted pulses

h(t− kTs) are orthonormal and the discrete-time noise at the slicer input is white.

3.5.3 More about the Matched Filter

We finally discuss some important further aspects as well as an extension of the matched filter.15Note that we assumed ISI-free transmission when deriving the matched filter. However, the matched filter can of

course also be used in the presence of ISI. In that case, the expressions (3.115) and (3.119) are no longer valid, and thus

the matched filter is no longer optimum in the sense of maximizing the SNR. Nevertheless, we shall see in Section 6.2

that even in this case the matched filter is an important part of the optimum receiver.


Unknown/distortionless channel. In many cases, the channel filter B(jω) is completely unknown

and hence the received pulse spectrum H(jω) = G(jω)BLP(jω) is unknown too. This means that

the receive filter f(t) cannot be matched to the received pulse h(t) (unless we have some means

of estimating h(t)). In this situation, a natural default strategy is to match the receive filter to

the transmit pulse g(t), which corresponds to the assumption that the channel is distortionless, i.e.,

B(jω) ≡ 1. (In order to compensate for the actual channel distortion, this matched filter is usually

followed by an adaptive equalizer, see Chapter 4). Thus, the matched filter here becomes

fMF(t) = g∗(−t) , FMF(jω) = G∗(jω) . (3.145)

In the absence of channel distortion, condition (3.140) for ISI-free transmission then implies that the

squared magnitude of the Fourier transform of the transmit pulse, |G(jω)|2, must satisfy the Nyquist

criterion. Such a transmit pulse g(t) is the root-raised-cosine pulse previously considered in Subsection

3.5.2. Indeed, the root-raised-cosine pulse is the transmit/receive pulse used in most PAM modems.

Causal matched filter. If the received pulse h(t) is causal, then the matched filter fMF(t) = h∗(−t)is necessarily anticausal . If the causal pulse h(t) has finite length Th, i.e., h(t) = 0 for t < 0 and t > Th,

then the delayed matched filter fMF(t− Th) = h∗(−(t− Th)

)= h∗(Th − t) is causal. (You may wish

to sketch h(t), fMF(t) = h∗(−t), and fMF(t − Th) in order to check this.) Now, the slicer input

q[k] = q(kTs)— i.e., the output q(t) = (y ∗fMF)(t) of the matched filter sampled at t = kTs, cf. Figure

3.16—can be rewritten as follows:

q[k] =[y(t) ∗ fMF(t)

](kTs) =

∫ ∞

−∞y(t) fMF(kTs−t) dt =

∫ ∞

−∞y(t) fMF(kTs+Th−t−Th) dt

=

∫ ∞

−∞y(t) fMF

((τ− t)− Th

)dt∣∣∣τ = kTs+Th

=[y(t) ∗ fMF(t−Th)

](kTs+Th) . (3.146)

This is the output of the delayed matched filter fMF(t−Th), which is causal, sampled at t = kTs+Th.

Thus, the causal version of the matched filter can be used if its output is sampled with a delay of Th.

In practice, the received pulse h(t) will always be effectively zero for t > T if T is chosen sufficiently

large, and thus we can always use a causal matched filter.

Matched filter for colored noise. What happens if the channel noise N(t) is not white but colored

with power spectral density SN (jω)? We will use a noise whitening filter to reduce the colored noise

problem to the white noise problem whose solution we already know. Let us assume that SN (jω)

is finite and nonzero for all ω (actually, this must be true only within our transmission band). We

can then represent the matched filter for colored noise, fMF(t), as the series connection of a noise

whitening filter w(t) and some other filter f ′MF(t) that has yet to be determined:

FMF(jω) = W (jω)F ′MF(jω) . (3.147)

This is depicted in Figure 3.17.

According to (3.23), the noise spectrum at the input of the receive filter f(t) is

SN (jω) = 2SN(j(ω+ωc)

). (3.148)


H(jω) H ′(jω) =W (jω)H(jω)

︸︷︷︸SN (jω) SN ′(jω) ≡ 1

F ′MF(jω)W (jω)

FMF(jω)

Figure 3.17: Mat hed lter for olored noise: representation using a noise whitening lter.

As will be verified presently, the noise whitening filter is given by

W (jω) =1√

SN (jω)ejΦ(ω) =

1√2SN

(j(ω+ωc)

) ejΦ(ω) , (3.149)

where Φ(ω) is an arbitrary phase function. The two-stage representation in (3.147) is possible since

(with our assumptions on SN (jω)) the noise whitening filter W (jω) in (3.149) is invertible, i.e., we

could always choose F ′MF(jω) such that the action of W (jω) is compensated.

We now verify that the noise at the output of the noise whitening filter is indeed white (with power

spectral density N0/2 = 1):

SN ′(jω) = |W (jω)|2 SN(jω) =

1

SN (jω)SN(jω) ≡ 1 . (3.150)

Therefore, the SNR at the input of the slicer is maximized by taking F ′MF(jω) to be the matched filter

for white noise. This matched filter, however, must be matched to the received pulse after the noise

whitening filter, whose Fourier transform is

H ′(jω) = W (jω)H(jω) =1√

SN (jω)ejΦ(ω)H(jω) . (3.151)

From (3.132),

F ′MF(jω) = c

[H ′(jω)

]∗= c

1√SN (jω)

e−jΦ(ω)H∗(jω) . (3.152)

Inserting (3.149) and (3.152) into (3.147), we finally obtain the matched filter for colored noise as

FMF(jω) = W (jω)F ′MF(jω)

=1√

SN(jω)

ejΦ(ω) c1√

SN(jω)

e−jΦ(ω)H∗(jω)

= cH∗(jω)SN(jω)

= cH∗(jω)

2SN(j(ω+ωc)

) . (3.153)

Note that this does not depend on the phase response Φ(ω) of the noise whitening filter. The matched

filter for white noise, equation (3.132), is readily reobtained as a special case for SN (jω) ≡ N0/2:


FMF(jω)∣∣∣SN (jω)≡N0/2

= cH∗(jω)N0

= c′H∗(jω) . (3.154)

3.5.4 Problems

Problem 3.5.1. Consider passband PAM transmission using symbol alphabet A =− 3

2 , −12 ,

12 ,

32

.

All symbols are equally likely. The channel noise is white with power spectral density N0/2 =2 ·10−5 W/Hz. The receive filter is given by f(t) = F0 e

−t/τ u(t) with F 20 = 1.3mW and τ = 3ms. The

overall pulse p(t) is a scaled raised-cosine pulse with roll-off factor 0.25 and |p[0]| = 68.2µJ. Calculatethe SNR at the slicer input. Hint : Cf. Problem 3.4.9.

Problem 3.5.2. Consider passband PAM transmission of a symbol sequence A[k] ∈− 1

2 ,12

over

a distortionless channel (B(jω) ≡ 1) with white noise (power spectral density N0/2 = 10−5 W/Hz).The transmit pulse is given by

g(t) = G0 cos

(πt

Ts

)rect

(t;Ts2

)with G2

0 = 10mW , Ts = 20ms . (3.155)

The receiver comprises an idealized lowpass filter of bandwidth B = 1/Ts, a symbol-rate sampler, anda slicer. Calculate an approximation for the SNR at the slicer input, using the assumption that thedistortion of the transmit pulse caused by the receive filter is negligible. Hint : Cf. Problem 3.4.11.

Problem 3.5.3. Consider the signals

x(t) = A cos(ω0t) rect

(t;T

2

), y(t) = A cos(ω0t+ θ) rect

(t;T

2

), with T =

2π

ω0. (3.156)

a) Calculate ‖x‖, ‖y‖, 〈x, y〉, and ‖x− y‖.b) Sketch these quantities as a function of θ.

c) Verify that the Cauchy-Schwarz inequality is satisfied.

d) Verify that the triangle inequality ‖x+ y‖ ≤ ‖x‖+ ‖y‖ is satisfied.

Problem 3.5.4. Consider the following signals:

0 Tt

1

s(1)(t)

0 3T4

Tt

1

s(2)(t)

0 T4

Tt

1

s(3)(t)

a) Calculate the norms and the inner product of s(1)(t) and s(2)(t).

b) Calculate the norm of the signal s(1)(t) + s(2)(t) using the results of part a). Hint : Express‖s(1) + s(2)‖2 in terms of ‖s(1)‖2, ‖s(2)‖2, and 〈s(1), s(2)〉.

c) Calculate a signal a(t) that lies in the subspace spanned by s(1)(t) and s(2)(t) (i.e., a(t) is alinear combination of s(1)(t) and s(2)(t)) and is orthogonal to s(1)(t) (i.e., 〈a, s(1)〉 = 0).


d) Calculate the signal b(t) in the subspace spanned by s(1)(t) and s(2)(t) that is closest to s(3)(t).

Problem 3.5.5. Show that the quantities ‖x‖, ‖y‖, 〈x, y〉, and ‖x − y‖ are not changed by thefollowing transformations of both x(t) and y(t):

a) Time shift by t0.

b) Frequency shift by ω0 (i.e., multiplication by ejω0t).

c) Time scaling according to x(t) →√|a| x(at) and y(t) →

√|a| y(at), with a 6= 0.

Problem 3.5.6. Assume ISI-free passband PAM transmission of a binary symbol sequence over a

channel with white Gaussian noise. The received pulse is h(t) =√

2Ts

rect(t− Ts

4 ;Ts4

).

a) Calculate the receive filter f(t) minimizing the symbol error probability. (This receive filter isto be used in what follows.)

b) Calculate the resulting minimum symbol error probability.

c) The symbol alphabet is A =− 1

2 ,32

; the two symbols are equally likely. Determine the

appropriate decision threshold of the slicer.

d) Is f(t) causal?

e) Verify that the transmission is indeed ISI-free.

Problem 3.5.7. Assume passband PAM transmission corrupted by white noise. The received pulse

is h(t) =√

2Ts

(1 + jt) rect(t− Ts

4 ;Ts4

). Calculate the impulse response of the matched receive filter,

and sketch its real and imaginary parts.

Problem 3.5.8. Consider passband PAM transmission of a single symbol A[1] with mean power PA

over a channel with white noise. The receive filter f(t) is the matched filter corresponding to thereceived pulse h(t). Due to a synchronization error (timing offset τ), the sampling instant is Ts + τinstead of Ts.

a) Calculate the resulting slicer input Q[1].

b) Calculate the SNR at the slicer input.

c) Show that both |Q[1]| and the SNR are even functions of τ that assume their maximum at τ = 0.

d) Suppose that the actual time offset τ 6= 0 is known at the time the modem is designed. Modifythe matched filter such that this time offset is taken into account and no performance loss isincurred.

Problem 3.5.9. Consider passband PAM with a root-raised-cosine transmit filter, i.e., G(jω) =√R(jω) where R(jω) is the Fourier transform of a raised-cosine pulse. The channel is distortionless

(B(jω) ≡ 1) with white noise.

a) Calculate the matched receive filter (provide a frequency-domain expression).


b) Calculate the resulting equivalent discrete-time pulse p[k] and the power spectrum of the equiv-alent discrete-time noise Z[k].

Problem 3.5.10. Consider passband PAM with received pulse h(t) = e−atu(t) where a > 0.

a) Calculate the matched receive filter for white channel noise.

b) Calculate the resulting equivalent discrete-time pulse p[k] and the power spectrum of the equiv-alent discrete-time noise Z[k].

Problem 3.5.11. Consider passband PAM with received pulse

h(t) =

∞∑

l=0

αl v(t−lTs) , α ∈ R , |α| < 1 , (3.157)

where all shifted pulses v(t− lTs) are orthogonal (i.e., their inner product is zero). The channel noiseis white (power spectral density N0/2). Assume that the matched receive filter is used. Calculate theresulting equivalent discrete-time pulse p[k] and the power spectrum of the equivalent discrete-timenoise Z[k].

Problem 3.5.12. Consider passband PAM transmission of a white symbol sequence A[k] with meanpower PA over a distortionless channel with white noise (power spectral density N0/2). The transferfunction of the transmit pulse is

G(jω) = Ts [∆(jω)]β with 0 ≤ β ≤ 1 , (3.158)

where ∆(jω) has triangular shape,

∆(jω) =

1− |ω|

2π/Ts, |ω| < 2π

Ts


a) Choose a receive filter F (jω) such that ISI-free transmission is obtained and the resulting overallpulse spectrum P (jω) is continuous. (This F (jω) is to be used in the following.)

b) For which β is the receive filter F (jω) a matched filter?

c) How do the mean signal and noise power at the slicer input depend on the parameter β?

d) For which value of β is the SNR at the slicer input maximized? Why is the resulting optimumreceive filter not a matched filter?

Problem 3.5.13. Consider a received pulse h(t) that is causal and has finite duration Th. Sketch theimpulse response of the delayed matched filter fMF(t− Th) and verify that it is causal.

Problem 3.5.14. Consider a passband PAM system with transmit pulse

g(t) = sin

(8π

t

Ts

)rect

(t− Ts

2;Ts2

)(3.160)

and white channel noise.


a) Calculate and sketch the impulse response of the causal matched filter. What are the appropriatesampling instants?

b) Assume that the sampling instants are advanced by τ (note: advanced, not delayed!). For thesenew sampling instants, calculate the causal filter that maximizes the SNR at the slicer input.Sketch the impulse response of this filter and calculate the resulting SNR as a function of τ .

Hint : In part b), use the Cauchy-Schwarz inequality and the fact that f(t) = f(t)u(t) due to causality.

Problem 3.5.15. Consider a passband PAM system with carrier frequency ωc, received pulseh(t) = exp

(− t2

2τ2

), and colored channel noise with power spectrum (within the effective transmit

band about ωc, only considering positive frequencies) SN (jω) = exp(τ2(ω−ωc)2). Calculate the im-

pulse response and transfer function of the matched filter for the colored noise as specified. Hint :

F

1√2π τ

exp(− t2

2τ2

)= exp

(− τ2ω2

2

).

Problem 3.5.16. Consider a passband PAM system with carrier frequency ωc. The transmit filter isan idealized lowpass filter with bandwidth Bg = 1/(2Ts). The channel is distortionless (B(jω) ≡ 1).The power spectrum of the channel noise is (at least within the transmission band about ωc, onlyconsidering positive frequencies) SN (jω) = Kω2. Calculate and sketch the transfer function of thematched receive filter for the colored noise as specified.

Problem 3.5.17. Consider passband PAM with received pulse h(t) and colored channel noise (powerspectrum SN (jω)). Assume that the appropriate matched filter (i.e., for the colored noise) is used.

a) Calculate the resulting equivalent discrete-time pulse p[k]. Under which condition is the trans-mission ISI-free?

b) Calculate the resulting power spectrum of the equivalent discrete-time noise Z[k]. Under whichcondition is Z[k] white?

c) Calculate the SNR at the slicer input under the assumption of ISI-free transmission.

4 Equalization

Contents


4.2 Linear Equalizer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97

4.2.1 Zero-Forcing Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98

4.2.2 Mean-Square Error Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99

4.2.3 Adaptive Linear Equalizer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105

4.2.4 Fractionally Spaced Linear Equalizer . . . . . . . . . . . . . . . . . . . . . . . . 111

4.2.5 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115

4.3 Decision-Feedback Equalizer . . . . . . . . . . . . . . . . . . . . . . . . . . . 119

4.3.1 Mean-Square Error Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 120

4.3.2 Variations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125

4.3.3 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125

95

96 Chapter 4. Equalization


In this chapter, we augment the receiver of the elementary passband PAM system described in the

previous chapter by an equalizer . The basic function of the equalizer is to combat ISI, which can be

motivated as follows. In practice, the transmit filter g(t) and the receive filter f(t) typically are fixed

filters that are designed without knowledge of the channel filter b(t). Often, these filters are chosen

as root-raised-cosine filters so that the receive filter f(t) is matched to the transmit filter g(t) and the

two filters taken together form a Nyquist filter. This means that unless the channel is distortionless

(i.e., unless B(jω) ≡ const), the total pulse p(t) = g(t) ∗ bLP(t) ∗ f(t) is not a Nyquist pulse and hence

the entire PAM system is not ISI-free. An equalizer can then be used to reduce the ISI.

Before going into details, we first summarize some basic facts about equalizers and briefly outline

the material treated in this chapter.

• Two fundamental equalizer structures are the linear equalizer and the decision-feedback equalizer .

The linear equalizer is simply a discrete-time filter placed between the sampler and the slicer. The

decision-feedback equalizer has an additional feedback loop in which the past detected symbols

are filtered by a “feedback” filter and then subtracted from the output of the “feedforward”

filter. The decision-feedback equalizer is a nonlinear system because it contains the (nonlinear)

slicer.

• The two most important strategies for calculating an equalizer’s filter coefficients are the zero-

forcing (ZF) strategy that attempts to remove all of the ISI (usually at the cost of substantial

noise amplification) and the mean-square error (MSE) strategy that aims at minimizing the total

error caused by ISI and noise.

• A feedforward equalizer filter operating at the symbol rate is theoretically sufficient in the case

of an ideal sampling phase; however, it is very sensitive to sampling-phase errors. An alternative

is provided by the fractionally spaced equalizer that operates at a higher rate (usually double

symbol rate). Because aliasing is avoided, fractionally spaced equalizers can compensate for

arbitrary (constant) sampling phase errors.

• In most practical implementations, the channel characteristics are unknown and may be time-

varying. Therefore, an adaptive equalizer is typically used whose filter coefficients are automat-

ically adjusted and continually updated by an adaptation algorithm.

PAM receiver structures using equalizers are generally suboptimum. However, they have certain

advantages over the optimum PAM receiver to be studied in Chapter 6: they are considerably less

complex to implement, do not require knowledge of the channel impulse response, are capable of

“tracking” time-varying channels, and (when combined with channel coding) may perform almost as

well as the optimum receiver.

4.2 Linear Equalizer 97

q[k] x[k]

t=kTs

a[k]

√2 e−jωct

d[k]f(t)y(t) y(t) q(t)

Figure 4.1: Passband PAM re eiver with linear equalizer.

d[k]a[k]a[k]

p[k]x[k]

z[k]

q[k]

Figure 4.2: PAM system with linear equalizer.


• The linear equalizer is considered in Section 4.2. We discuss the ZF design assuming infinite

equalizer length and the MSE design for finite and infinite equalizer length. Based on the MSE

design, we then develop an adaptive version of the linear equalizer. Finally, the fractionally

spaced linear equalizer is discussed.

• Section 4.3 considers the decision-feedback equalizer . The MSE design is discussed in detail,

whereas the ZF design as well as the fractionally spaced and adaptive versions are only briefly

mentioned.

4.2 Linear Equalizer

As in Chapter 3, we consider a passband PAM system1 represented in the equivalent baseband domain.

The standard PAM receiver of Chapter 3 contains a receive filter f(t) ↔ F (jω), a symbol-rate sampler,

and a slicer (see Figure 3.1). A linear equalizer operating at the symbol rate is a discrete-time filter

that is placed between the symbol-rate sampler and the slicer. In what follows, d[k] ↔ D(ejθ) will

denote the impulse response and transfer function of the equalizer filter. The passband PAM receiver

including the linear equalizer is shown in Figure 4.1.

According to Subsection 3.2.2, the PAM system up to the equalizer (including the transmit fil-

ter, lowpass-bandpass transformation, channel, bandpass-lowpass transformation, receive filter, and

symbol-rate sampler but not the equalizer) can be compactly represented by an equivalent discrete-

time baseband PAM system with filter p[k] ↔ P (ejθ) and noise z[k] with power spectral density

SZ(ejθ) (see Figure 3.4(b)). Using this compact representation, the entire passband PAM system

including the equalizer filter is depicted in Figure 4.2.

1Our discussion in this chapter is also valid for baseband PAM systems. The only essential difference for a baseband

PAM system is that all symbols, pulses, and filters are real-valued.


The equalizer can be viewed as a symbol estimator , i.e., its output x[k] is supposed to be an

estimate of the current symbol a[k]. In other words, x[k] should be a good approximation to a[k].

The function of the subsequent slicer then is to remove the residual difference (error) between x[k]

and a[k], and produce a “detected” symbol a[k]∈A which, hopefully, is equal to a[k].

4.2.1 Zero-Forcing Design

We will now address the design of the linear equalizer. A simple design strategy is the zero-forcing

(ZF) strategy that attempts to remove all of the ISI. The ZF linear equalizer can be written in the

frequency domain as

DZF(ejθ) =

1

P (ejθ), (4.1)

provided that this system exists (note that its existence requires that P (ejθ) does not vanish on an

interval). Indeed, the transfer function of the total system including the equalizer is (see Figure 4.2)

P (ejθ)DZF(ejθ) = P (ejθ)

1

P (ejθ)≡ 1 , (4.2)

which corresponds to the total absence of ISI. Note that even if p[k] has finite length (i.e., it corresponds

to an FIR filter), dZF[k] must be expected to have infinite length; also, in general dZF[k] need not be

causal.

If the receive filter is the matched filter, i.e., f(t) = h∗(−t), then P (ejθ) = Sh(ejθ) (see (3.137)).

The ZF linear equalizer is here given by

DZF(ejθ) =

1

Sh(ejθ). (4.3)

Noise enhancement. The great disadvantage of the ZF linear equalizer is its effect on the noise.

Let U [k] denote the noise at the output of the equalizer. The power spectral density of U [k] is

SU (ejθ) = SZ(e

jθ)∣∣DZF(e

jθ)∣∣2 =

SZ(ejθ)∣∣P (ejθ)∣∣2 , (4.4)

and thus the variance (mean power2) of U [k] is obtained as

σ2U = PU =1

2π

∫ π

−π

SZ(ejθ)∣∣P (ejθ)∣∣2 dθ . (4.5)

Hence, values of P (ejθ) near zero will result in significant noise amplification (noise enhancement).

This is analogous to the zero-forcing receive filter design discussed in Subsection 3.3.4.

Optimum receive filter. The choice of the receive filter f(t) ↔ F (jω) influences the ZF linear

equalizer and consequently the noise enhancement. Let us determine the F (jω) that yields minimum

noise enhancement in the case of white channel noise, i.e., SN (jω) ≡ N0/2. According to (3.34), the

2Note that U [k] is zero-mean since the channel noise N(t) is assumed zero-mean.


power spectral density of the discrete-time noise Z[k] at the input of the equalizer is

SZ(ejθ) = N0

1

Ts

∞∑

i=−∞

∣∣∣∣F(jθ− i2πTs

)∣∣∣∣2

. (4.6)

Furthermore, with P (jω) = H(jω)F (jω) we have from (3.31)

P (ejθ) =1

Ts

∞∑

i=−∞H

(jθ− i2πTs

)F

(jθ− i2πTs

). (4.7)

Inserting these expressions in (4.4) yields

SU (ejθ) = N0

1Ts

∑∞i=−∞

∣∣F(j θ−i2π

Ts

)∣∣2∣∣∣ 1Ts

∑∞i=−∞ F

(j θ−i2π

Ts

)H(j θ−i2π

Ts

)∣∣∣2 . (4.8)

For θ arbitrary but fixed, let us define the inner product 〈F,H〉θ , 1Ts

∑∞i=−∞ F

(j θ−i2π

Ts

)H∗(j θ−i2π

Ts

)

with induced norm ‖F‖θ =√

〈F,F 〉θ =√

1Ts

∑∞i=−∞

∣∣F(j θ−i2π

Ts

)∣∣2. We can then write (4.8) very

compactly as

SU (ejθ) = N0

‖F‖2θ|〈F,H∗〉θ|2

. (4.9)

From the Cauchy-Schwarz inequality3 (cf. (3.128)), it now follows that

|〈F,H∗〉θ|2

‖F‖2θ≤ ‖H∗‖2θ , (4.10)

with equality if and only if (iff) F(j θ−i2π

Ts

)is proportional to H∗(j θ−i2π

Ts

)for all i ∈ Z. This means

that the right-hand side of (4.9) is minimized iff F(j θ−i2π

Ts

)is proportional to H∗(j θ−i2π

Ts

). Since θ

was assumed fixed but arbitrary, this must be true for all values of θ. Hence, SU (ejθ) is minimized iff

F (jω) = cH∗(jω) or equivalently f(t) = c h∗(−t). We have thus shown that the noise enhancement

is minimized for all frequencies (and hence also the integral noise enhancement expressed by σ2Uis minimized) iff the receive filter is the matched filter . For f(t) = c h∗(−t), we have SZ(e

jθ) =

N0 |c|2Sh(ejθ) and P (ejθ) = c Sh(ejθ) (cf. (3.137), (3.138)). Inserting in (4.4), we obtain the minimum

noise power spectrum at the equalizer output as

[SU(e

jθ)]min

= SU (ejθ)∣∣∣f(t) = c h∗(−t)

=N0 |c|2Sh(ejθ)∣∣c Sh(ejθ)

∣∣2 =N0

Sh(ejθ), (4.11)

which does not depend on c. Note that[SU (e

jθ)]min

will be large at those frequencies where Sh(ejθ)

is small.

4.2.2 Mean-Square Error Design

Rather than eliminating all the ISI at the risk of a large noise enhancement, it is wiser to allow some

residual ISI and minimize the total error that is jointly caused by ISI and noise. Specifically, in this

3It is easily shown that 〈F,H〉θ satisfies all axioms of an inner product; hence, the Cauchy-Schwarz inequality holds.


q[k+L−1]q[k+L] q[k−L+1]

x[k] a[k]

d[−L] d[−L+1]

zLa[k] q[k−L]

z−1 z−1

d[L−1] d[L]

p[k]

z[k]

q[k]z−1

Figure 4.3: PAM system with FIR linear equalizer.

subsection, we will derive the linear equalizer that minimizes the mean-square error (MSE) at the

slicer input. This equalizer is termed the MSE linear equalizer . The infinite-length MSE equalizer

can be derived quite easily in the frequence domain. However, we shall first derive the finite-length

(FIR) MSE equalizer and subsequently consider the infinite-length MSE equalizer as a limiting case.

This is done for two reasons: first, FIR equalizers are used in practice4, and second, the FIR version

of the MSE equalizer will provide a basis for developing an adaptive equalizer in Subsection 4.2.3.

The FIR linear equalizer. An FIR linear equalizer with 2L + 1 filter coefficients5 d[k] (k =

−L,−L+1, · · · , L) is depicted in Figure 4.3. Note that the equalizer filter is defined noncausal6

because it contains the nonrealizable, noncausal “time advance system” zL, but it can easily be made

causal by omitting this time advance system. Assuming an infinite sequence of transmitted symbols

A[k] (−∞ < k <∞), the input to the equalizer is

Q[k] =

∞∑

l=−∞p[l]A[k−l] + Z[k] , (4.12)

where A[k] and Z[k] are considered random. The equalizer’s output signal is given by

X[k] =L∑

l=−L

d[l]Q[k−l] = dHq[k] (4.13)

(the superscript H stands for conjugate transposition), where we introduced the vectors of size (2L+

1)× 1

d ,

d∗[−L]d∗[−L+1]

...d∗[L]

, q[k] ,

Q[k+L]Q[k+L−1]

...Q[k−L]

. (4.14)

4We did not consider an FIR version of the ZF linear equalizer because it typically does not exist: a finite-length

equalizer is generally unable to completely remove the ISI.5In digital communications, FIR filters are also termed transversal filters and the filter coefficients are termed tap

coefficients or tap weights.6The filter coefficients d[k] are defined for k = −L,−L+1, · · ·, L since this will allow us to obtain the infinite-length

MSE equalizer simply by letting L → ∞.


Note that the nonrandom vector d contains the (conjugate) filter coefficients7 d∗[k] and the random

vector8 q[k] contains all the input samples Q[l] which at time k are contained in the equalizer’s shift

register and are multiplied by filter coefficients (cf. Figure 4.3).

Definition and calculation of the MSE. The task of the equalizer is to estimate the symbols

A[k] actually transmitted. That is, the equalizer output (simultaneously, slicer input) X[k] should be

close to the current symbol A[k]. The corresponding estimation error is

E[k] , X[k] −A[k] = dHq[k]−A[k] , (4.15)

where (4.13) has been used. Note that E[k] is a random process. Hereafter we assume that the random

processes A[k] and Z[k] are jointly stationary, whence also the random processes Q[k], X[k], and E[k]

are jointly stationary. The mean-square error (MSE) ǫ is defined as the mean power of E[k],

ǫ , E|E[k]|2

. (4.16)

By definition, the MSE linear equalizer is given by the filter coefficient vector d for which the MSE ǫ

is minimized.

Using (4.15), we obtain for the MSE

ǫ = E(

dHq[k]−A[k])(

dHq[k]−A[k])∗

= EdHq[k] (dHq[k])∗︸︷︷︸

dHq[k]qH [k]d

− E

dHq[k]A∗[k]

− E

A[k] (dHq[k])∗︸︷︷︸

qH [k]d

+ E

A[k]A∗[k]

= dHEq[k]qH [k]

d − dHE

q[k]A∗[k]

− E

A[k]qH [k]

d + E

|A[k]|2

. (4.17)

Let us introduce the (2L+ 1)× (2L+ 1) correlation matrix of the equalizer input vector q[k] as

RQ , Eq[k]qH [k]

, (4.18)

with elements

[RQ]i,j = EQ[k + L− (i−1)]Q∗[k + L− (j−1)]

= EQ[k]Q∗[k − (j−i)]

= RQ[j−i] , i, j = 1, 2, · · · , 2L+1 . (4.19)

Furthermore, we introduce the (2L+ 1)× 1 cross-correlation vector of q[k] and A[k] as

rQ,A , Eq[k]A∗[k]

, (4.20)

with elements[rQ,A]i = E

Q[k + L− (i−1)]A∗[k]

, i = 1, 2, · · · , 2L+1 . (4.21)

7The entries of d are defined to be d∗[k] rather than d[k] because this will result in simpler expressions at a later

point.8We here use lowercase boldface symbols for vectors even when these vectors are random. Uppercase boldface symbols

are reserved for matrices.


Since all processes involved are jointly stationary, RQ and rQ,A do not depend on k. The MSE in

(4.17) can now be written as

ǫ = dHRQd − dHrQ,A − rHQ,Ad + PA , (4.22)

with PA = E|A[k]|2

.

The optimum equalizer filter. Our goal is to choose the equalizer coefficient vector d such that

the MSE ǫ is minimized. In what follows, we assume that the input correlation matrix RQ is invertible

(nonsingular). Using a “completing the square” argument, the MSE in (4.22) can then be rewritten

as

ǫ = (d−dMSE)HRQ (d−dMSE) + ǫmin , (4.23)

with

dMSE = R−1Q rQ,A and ǫmin = PA − rHQ,AR

−1Q rQ,A . (4.24)

Indeed, (4.23) can be developed as

ǫ = dHRQd − dHRQdMSE − dHMSERQd + dH

MSERQdMSE + ǫmin . (4.25)

Comparing corresponding terms of (4.25) and (4.22), it is readily verified that these two expressions

of ǫ are equivalent with dMSE and ǫmin as given by (4.24).

Equation (4.23) shows that the MSE, viewed as a function of the filter coefficient vector d, consists

of the constant term ǫmin ≥ 0 and the quadratic form (d − dMSE)HRQ (d − dMSE). The matrix RQ

inducing this quadratic form is a correlation matrix and thus positive definite or semidefinite; because it

was assumed invertible, it is positive definite. This means that the quadratic form (d−dMSE)HRQ (d−

dMSE) is nonnegative and that it is zero iff d − dMSE = 0. It follows that the MSE is minimized iff

the quadratic form is zero, i.e., iff d− dMSE = 0 or equivalently d = dMSE. Thus, dMSE = R−1Q rQ,A

is indeed the coefficient vector of the MSE linear equalizer, i.e., the coefficient vector that minimizes

the MSE. We can visualize the MSE function ǫ(d) as a hyperparaboloid over the (2L+1)-dimensional

coefficient space (coefficient hyperplane). This is illustrated in Figure 4.4 for a two-dimensional,

real-valued coefficient vector9. Note in particular that ǫ(d) has a unique minimum, ǫmin, which is

assumed at d = dMSE.

The Wiener-Hopf equation. Evidently, dMSE = R−1Q rQ,A is the solution to the linear equation

RQd = rQ,A . (4.26)

In estimation theory, an equation of this type (i.e., the system matrix is a correlation matrix of some

stationary process Q[k] and the right hand side is a cross-correlation vector involving Q[k]) is known

as a Wiener-Hopf equation. We also note at this point that the MSE linear equalizer is a special

case of the noncausal version of the Wiener filter , which is the MSE-optimum linear estimator of a

stationary signal process corrupted by stationary noise.

9We use dimension 2 for ease of illustration; of course, 2L+ 1 may not be equal to 2.


d[2]

ǫ(d)

ǫmin

d[1]

dMSE

Figure 4.4: MSE surfa e ǫ(d) = ǫmin + (d − dMSE)HRQ (d − dMSE) for a two-dimensional oeÆ ient

ve tor d.

Let us consider the Wiener-Hopf equation (4.26) in more detail. For the sake of simplicity,

the symbol sequence A[k] is assumed white and zero-mean with mean power PA, so that RA[l] =

EA[k]A∗[k − l]

= PAδ[l]. Furthermore, the processes A[k] and Z[k] are assumed uncorrelated and

Z[k] is assumed zero-mean. With Q[k] = p[k] ∗A[k] + Z[k] (see (4.12)), it then follows from (4.19)

and (4.21) that the elements of the correlation matrix RQ and of the cross-correlation vector rQ,A are

related to p[k], PA, and RZ [l] as

[RQ]i,j = RQ[j−i] = PA ρp[j−i] + RZ [j−i] , i, j = 1, 2, · · · , 2L+1 (4.27)

[rQ,A]i = PA p[L−i+1] , i = 1, 2, · · ·2L+1 , (4.28)

where

ρp[k] , p[k]∗ p∗[−k] =

∞∑

i=−∞p[i] p∗[i−k] . (4.29)

Inserting (4.27) and (4.28) into (4.26), suitably transforming the summation index, and taking the

complex conjugate yields the system of linear equations

L∑

l=−L

[PA ρp[k−l] + RZ [k−l]

]d[l] = PA p

∗[−k] , k = −L,−L+1, · · · , L , (4.30)

which is equivalent to (4.26) under the assumptions stated above.

The infinite-length case. The (noncausal) MSE linear equalizer with infinite length can be ob-

tained as a limiting case of the FIR MSE linear equalizer simply by letting L → ∞. Equation (4.30)


then becomes the convolution relation

[PA ρp[k] + RZ [k]

]∗ d[k] = PA p

∗[−k] , −∞ < k <∞ . (4.31)

Taking the Fourier transform of both sides, we obtain the equivalent frequency-domain relation

[PA∣∣P (ejθ)

∣∣2 + SZ(ejθ)]D(ejθ) = PAP

∗(ejθ) . (4.32)

The transfer function of the infinite-length MSE linear equalizer then follows as

DMSE(ejθ) =

PAPA |P (ejθ)|2 + SZ(ejθ)

P ∗(ejθ) . (4.33)

This result shows that the infinite-length MSE linear equalizer consists of two stages:

1. A discrete-time matched filter10 p∗[−k] ↔ P ∗(ejθ). For a causal equivalent channel filter p[k],

the matched filter p∗[−k] is anticausal.

2. A zero-phase filter with nonnegative transfer function PAPA |P (ejθ)|2 +SZ(ejθ)

. The corresponding

impulse response has even symmetry and thus is noncausal.

For an intuitive understanding of the operation of the infinite-length MSE linear equalizer, the follow-

ing special situations are of interest:

• At frequencies θ where there is no channel noise, i.e., SZ(ejθ) = 0, the infinite-length MSE linear

equalizer reduces to the ZF linear equalizer DZF(ejθ) in (4.1):

SZ(ejθ) = 0 ⇒ DMSE(e

jθ) =PA

PA |P (ejθ)|2 P∗(ejθ) =

1

P (ejθ)= DZF(e

jθ) . (4.34)

Thus, the ISI contribution associated with frequency θ is completely removed, without any noise

enhancement penalty.

• At frequencies θ where P (ejθ) = 0 but SZ(ejθ) 6= 0 (i.e., no signal, only noise), the transfer

function of the infinite-length MSE linear equalizer is zero, DMSE(ejθ) = 0, and hence the noise

is totally suppressed. In contrast, the transfer function of the ZF linear equalizer would be

infinite, DZF(ejθ) → ∞, and thus there would be infinite noise enhancement. However, while

it is true that for P (ejθ) = 0 the infinite-length MSE linear equalizer does not cause any noise

enhancement, it does not equalize the channel to a flat frequency response, and thus there will

be substantial residual ISI.

• At frequencies θ where the noise is dominant, i.e., SZ(ejθ) ≫ PA |P (ejθ)|2, we obtain from (4.33)

the following approximation:

SZ(ejθ) ≫ PA |P (ejθ)|2 ⇒ DMSE(e

jθ) ≈ PAP ∗(ejθ)SZ(ejθ)

. (4.35)

10More precisely, p∗[−k] would be the filter matched to p[k] if the noise z[k] were white. However, z[k] is not necessarily

white.


This is the transfer function of the (discrete-time) matched filter associated to the filter transfer

function P (ejθ) and noise spectrum SZ(ejθ) (cf. (3.153)). Note also that DMSE(e

jθ) is smaller

when SZ(ejθ) is larger (noise suppression).

The power spectral density of the noise U [k] at the output of the infinite-length MSE linear

equalizer is

SU (ejθ) = SZ(e

jθ)∣∣DMSE(e

jθ)∣∣2 = SZ(e

jθ)P 2A |P (ejθ)|2

[PA |P (ejθ)|2 + SZ(ejθ)

]2 , (4.36)

and the variance of U [k] is σ2U = 12π

∫ π−π SU(e

jθ) dθ. As observed above, SU (ejθ) = 0 at those frequencies

where P (ejθ) = 0. It can be shown that for P (ejθ) and SZ(ejθ) given, SU (e

jθ) is ≤ the SU (ejθ) obtained

with the ZF linear equalizer (see (4.4)).

4.2.3 Adaptive Linear Equalizer

Thus far, we have assumed that the relevant properties of the channel (the impulse response or transfer

function of the equivalent discrete-time channel filter and, for the MSE design, also the autocorrelation

function or power spectral density of the equivalent discrete-time noise) are known. This assumption

is rarely satisfied in practice. Therefore, in this subsection, we will discuss a method— in fact, the

most frequently used method— for adaptively adjusting the coefficients of the equalizer filter during

data transmission, without explicit knowledge of the channel characteristics. The resulting “adaptive

linear equalizer” is a signal-adaptive version of the linear equalizer. It is not, however, a linear system

itself because the filter coefficients depend on the input signal via the adaptation algorithm.

The gradient algorithm. We will first present a simple method for calculating the MSE equalizer

dMSE in an iterative manner. The starting-point for developing this method is the geometric intuition

provided by Figure 4.4 and the fact that dMSE is the unique minimum of the MSE surface ǫ(d).

Consider a given coefficient vector (or point in the d-hyperplane) di as shown in Figure 4.5. From

di, we wish to proceed to a new point di+1 at which the MSE is smaller, i.e., ǫ(di+1) < ǫ(di). The

d-direction in which we locally obtain the fastest decrease of ǫ(d) is given by the negative gradient of

ǫ(d) at the point di, which will be denoted −∇ǫ(di). Therefore, let us consider the recursion

di+1 = di −µ

2∇ǫ(di) , µ > 0 , (4.37)

i.e., the new point di+1 is derived from the old point di by moving a little in the direction of the

negative gradient11. The iterative algorithm defined by the recursion (4.37) is known as the gradient

algorithm. If the “step size” µ > 0 is sufficiently small so that di+1 does not overshoot too much

beyond the position dMSE of the unique minimum, the new point di+1 will have a smaller MSE than

the old point di, i.e., ǫ(di+1) < ǫ(di) as desired, and it will thus be closer to dMSE. If we continue the

recursion (4.37), we will get closer and closer to dMSE. This shows that the gradient algorithm (4.37)

(with sufficiently small µ) is guaranteed to converge to dMSE. This is true for an arbitrary choice of

the initial point d0 with which the recursion is started.

11The factor of 1/2 in µ/2 is arbitrary; it is included to avoid a factor in the final formulations (4.39) and (4.42).


d[2]

di

dMSE

ǫ(di)

−∇ǫ(di)

ǫmin

d[1]

ǫ(d)

Figure 4.5: MSE surfa e ǫ(d) = ǫmin + (d−dMSE)HRQ (d−dMSE) and negative gradient −∇ǫ(d) at a

point di.

We still have to calculate ∇ǫ(d). In a passband PAM system, d is complex-valued and the gradient

is defined by12 ∇ǫ(d) , ∇dRǫ(d) + j∇dI

ǫ(d), where dR = Red and dI = Imd. With (4.22), the

gradient can be calculated as13

∇ǫ(d) = ∇dHRQd− dHrQ,A − rHQ,Ad+ PA

= . . . = 2

(RQd− rQ,A

). (4.38)

Thus, (4.37) becomes

di+1 = di − µ(RQdi − rQ,A

). (4.39)

On the other hand, there is

Eq[k]E∗[k]

= E

q[k]

(qH [k]d −A∗[k]

)= E

q[k]qH [k]

d − E

q[k]A∗[k]

= RQd − rQ,A , (4.40)

where (4.15) has been used. Hence, an equivalent expression for the gradient in (4.38) is

∇ǫ(d) = 2Eq[k]E∗[k]

. (4.41)

12The gradient ∇ǫ(d) of a function ǫ(d) of a complex vector d is not a complex derivative in the function-theoretic

sense (which would require the function to be analytic). Rather, the complex elements dn = dn,R + jdn,I of d are

considered as real vectors (dn,R, dn,I); ǫ(d) is accordingly considered as a function ǫ(dR,dI) of the real vectors dR, dI;

and ∇dR ǫ(d) and ∇dI ǫ(d) are the partial derivatives of ǫ(dR,dI) with respect to dR and dI, respectively.13This expression is also valid in the case of baseband PAM, where d is real-valued.


With this, (4.37) can alternatively be written as

di+1 = di − µEq[k]E∗[k]

∣∣∣d=di

, (4.42)

where it should be noted that Eq[k]E∗[k]

= RQd− rQ,A depends on d and does not depend on k.

There is one practical problem with the gradient algorithm in its present form: without knowledge

of the channel characteristics, the MSE gradient ∇ǫ(d) = 2Eq[k]E∗[k]

= 2(RQd − rQ,A) cannot

be calculated. We shall address this problem after having performed a convergence analysis of the

gradient algorithm.

Convergence analysis of the gradient algorithm. The recursion (4.39) can be written as

di =(I− µRQ

)di−1 + µrQ,A , (4.43)

where I is the identity matrix of size (2L + 1) × (2L + 1). Subtracting dMSE from both sides of this

equation yields

di − dMSE =(I− µRQ

)di−1 + µrQ,A − dMSE

=(I− µRQ

)di−1 + µRQdMSE − dMSE

=(I− µRQ

)di−1 −

(I− µRQ

)dMSE

=(I− µRQ

)(di−1 − dMSE

), (4.44)

where dMSE = R−1Q rQ,A has been used. Let us write

∆di , di − dMSE (4.45)

for the deviation of the filter coefficient vector di from the optimum vector dMSE. The recursion (4.44)

can then be written as

∆di =(I− µRQ

)∆di−1 =

(I− µRQ

)2∆di−2 = ... =

(I− µRQ

)i∆d0 , (4.46)

where d0 is the initial vector with which the iteration is started. Fast convergence of the gradient

algorithm means that ∆di becomes small already after a few iterations. This will be the case (for all

initial vectors d0) if and only if the matrix(I − µRQ

)ibecomes small already for moderate values

of i. Evidently, this latter property depends on RQ (i.e., on the statistical properties of the equalizer

input Q[k]) and on the step size µ.

To further analyze this dependence, we use the spectral decomposition of the correlation matrix

RQ,

RQ =2L+1∑

l=1

λl eleHl , (4.47)

where λl and el are the eigenvalues and eigenvectors, respectively, of RQ. Since RQ is positive

definite Hermitian, the eigenvalues are positive and the eigenvectors are orthonormal. The spectral


decomposition of(I− µRQ

)ican be shown to be

(I− µRQ

)i=

2L+1∑

l=1

(1− µλl)i ele

Hl . (4.48)

Inserting this into (4.46) yields

∆di =

[2L+1∑

l=1

(1− µλl)i ele

Hl

]∆d0 =

2L+1∑

l=1

(1− µλl)i ξl el , with ξl , eHl ∆d0 . (4.49)

We see that ∆di is the sum of 2L+ 1 orthogonal vectors (“modes”) (1 − µλl)i ξl el. The behavior of

the lth mode for growing i is governed by |1 − µλl|, i.e., by the product µλl of step size µ and lth

eigenvalue λl. In particular, ∆di will converge to zero if and only if |1 − µλl| < 1 for all l, which is

easily seen to be equivalent to 0 < µ < 2/λl for all l or

0 < µ <2

λmaxwith λmax , max

l=1,···,2L+1λl . (4.50)

In general, values of µ near 0 or 2λmax

will lead to slow convergence. For fast convergence, we would

like to have |1 − µλl| ≈ 0 for all l. Evidently, this is possible only if all eigenvalues λl are approxi-

mately equal. As a global measure characterizing the “difference of the eigenvalues,” let us define the

eigenvalue spread λmax/λmin where λmin , minl=1,···,2L+1 λl. In a mathematical context, λmax/λmin

is also known as the condition number of the matrix RQ. Note that λmax/λmin ≥ 1.

For Q[k] zero-mean and white, RQ = σ2QI. It follows that λl ≡ σ2Q, i.e., all eigenvalues are equal

and thus λmax/λmin = 1. Here, (4.49) becomes

∆di =(1− µσ2Q

)i 2L+1∑

l=1

ξl el . (4.51)

It is seen that ∆di can be made equal to zero after only a single iteration by choosing the step size

as µ = 1/σ2Q so that 1− µσ2Q = 0.

On the other hand, if the eigenvalue spread is large, i.e., λmax/λmin ≫ 1 (or, equivalently, if RQ is

ill-conditioned), then the gradient algorithm will converge slowly. The reason is that no matter how

we choose µ within the convergence interval defined by (4.50), |1 − µλl| will not be close to zero for

some value(s) of l.

A geometric interpretation of these results is shown in Figure 4.6 for a two-dimensional, real-

valued coefficient vector. The figure shows the (elliptic) contours of constant MSE of the quadratic

MSE surface (cf. Figure 4.5). Note that the principal axes of the ellipses are in the direction of the

eigenvectors el and the lengths of the principal axes are proportional to14 1/√λl, i.e., the reciprocals

14Indeed, inserting the spectral decomposition of RQ, equation (4.47), into the quadratic-form expression of the MSE,

equation (4.23), we obtain after some simple manipulations the “diagonal” quadratic form

ǫ =

2L+1∑

l=1

λl

∣∣dl− dMSE,l

∣∣2 + ǫmin =

2L+1∑

l=1

∣∣∣∣dl− dMSE,l

1/√λl

∣∣∣∣2

+ ǫmin , where dl , eHl d and dMSE,l , e

Hl dMSE . (4.52)

In the two-dimensional case (depicted in Figure 4.6), the contours of constant MSE described by (4.52) are ellipses

centered about dMSE, with the principal axes pointing in the direction of the eigenvectors el and the lengths of the

principal axes proportional to 1/√λl.


d[2]

d[1]

dMSE

d0

d[2]

d[1]

dMSE

d0

(a) (b)

Figure 4.6: Geometri interpretation of the onvergen e of the gradient algorithm for (a) large eigenvalue

spread, λmax/λmin ≫ 1, and (b) small eigenvalue spread, λmax/λmin ≈ 1.

of the square roots of the eigenvalues λl. Thus, for a larger eigenvalue spread the ellipses are more

eccentric. It is clear from the figure that a larger eccentricity implies a slower convergence since the

direction of the negative gradient tends to be more different from the direction pointing to the location

of the minimum, and hence more iterations are needed to approach the minimum.

Convergence and power spectrum. The convergence rate of the gradient algorithm is related to

the shape of the power spectral density SQ(ejθ) of the (stationary) process Q[k] that forms the input

to the equalizer filter. Recall from (4.19) that

[RQ]i,j = RQ[j−i] , i, j = 1, 2, · · · , 2L+1 , (4.53)

where (see (4.27))

RQ[l] = PA ρp[l] +RZ [l] (4.54)

is the autocorrelation sequence of Q[k]. Since RQ[l] is the inverse Fourier transform of the power

spectral density SQ(ejθ), it is clear that the eigenvalues of RQ depend on SQ(e

jθ). Specifically, they

can be shown to be bounded by the minimum and maximum of SQ(ejθ), i.e.,

[SQ(e

jθ)]min

≤ λl ≤[SQ(e

jθ)]max

. Hence, in particular,

[SQ(e

jθ)]min

≤ λmin ≤ λmax ≤[SQ(e

jθ)]max

. (4.55)

In the limiting case of infinite equalizer length, L→ ∞, the correlation matrix RQ has infinite size

and its eigenvalues λl can be shown to become equal to the power spectral density SQ(ejθ), with the

frequency variable θ formally taking the place of the eigenvalue index l. In particular, we then have

λmin =[SQ(e

jθ)]min

and λmax =[SQ(e

jθ)]max

and thus the eigenvalue spread becomes

λmax

λmin=

[SQ(e

jθ)]max[

SQ(ejθ)]min

. (4.56)

We see that for infinite equalizer length, the eigenvalue spread (condition number) of RQ is equal to

the “spectral spread” of the input process Q[k]. Hence, fastest convergence of the gradient algorithm

is obtained for a flat spectrum, i.e., for a white input process.


These results can be expected to remain approximately valid also in the finite-length (FIR) case,

provided that the filter length is not too small. In particular, if the input spectrum is nearly zero at

certain frequencies or in certain frequency bands, then [SQ(ejθ)]max/[SQ(e

jθ)]min will be large and the

gradient algorithm must be expected to converge slowly. Due to (4.54), the input spectrum is given

by

SQ(ejθ) = PA |P (ejθ)|2 + SZ(e

jθ) , (4.57)

where P (ejθ) is the transfer function of the equivalent discrete-time channel filter without equalizer.

Thus, zeros in P (ejθ) will result in slow convergence of the gradient algorithm unless SZ(ejθ) is large.

The stochastic gradient algorithm (LMS algorithm). If we do not know the transfer function

and noise power spectral density of the channel, we are not able to calculate the correlations RQ and

rQ,A. Hence, the MSE gradient ∇ǫ(d) = 2(RQd− rQ,A) cannot be calculated either.

We recall from (4.41) that the gradient can also be expressed as

∇ǫ(d) = 2Eq[k]E∗[k]

. (4.58)

Unfortunately, we cannot perform the expectation operation since in practice we observe only a sin-

gle realization of the processes q[k] and E[k]. Therefore, let us simply omit the expectation and

substitute the realization 2q[k]E∗[k]— i.e., the “noisy” or “stochastic” gradient— for the gradient

2Eq[k]E∗[k]

. Then, instead of the original gradient algorithm in (4.42),

di+1 = di − µEq[k]E∗[k]

∣∣∣d=di

, (4.59)

we obtain the stochastic gradient algorithm

dk+1 = dk − µq[k]E∗[k] . (4.60)

Here, dk is the current filter coefficient vector, q[k] is the current filter input vector, E[k] = dHk q[k]−

A[k] is the current error at the slicer input, and dk+1 is the filter coefficient vector to be used in

the next symbol interval. It is important to note that the iteration index i has been replaced by the

symbol time index k. That is, in each symbol interval we perform exactly one iteration using the

current quantities dk, q[k], and E[k].

Whereas dk and q[k] are available at the receiver, the error E[k] = dHk q[k] − A[k] cannot be

calculated by the receiver since the true symbols A[k] are unknown. This problem can be solved

as follows. During an initial start-up phase, a predefined sequence of symbols A[k] is transmitted.

This training sequence or preamble is known to the receiver and thus allows the equalizer to form the

error signal E[k] and to converge sufficiently. Subsequently, i.e., during normal operation, a “decision-

directed” mode is adopted in which the detected symbols A[k] are used instead of the unknown true

symbols A[k]. Since the equalizer has already converged during the start-up phase, there will be

A[k] = A[k] most of the time. In this decision-directed mode, the equalizer is even able to track slow

time-variations of the channel characteristics.

The stochastic gradient algorithm is also known as the LMS algorithm, where LMS stands for

least mean square. Note, however, that the LMS algorithm does not perform an exact minimization


of the MSE. Indeed, whereas the gradient algorithm is guaranteed to converge to the optimum filter

coefficient dMSE (assuming that the step size µ is chosen according to (4.50)), the stochastic gradient

(LMS) algorithm will come close to dMSE for an appropriate choice of the step size but it will forever

fluctuate about dMSE due to the stochastic “driving term” µq[k]E∗[k] in (4.60). For a smaller step

size µ, this coefficient fluctuation (or “coefficient misadjustment”) will be smaller but this comes at

the cost of slower convergence and poorer tracking capability. For a larger step size µ (assuming it

is still within the stability range), we obtain faster convergence and faster tracking, but also a larger

coefficient misadjustment.

The stability condition (4.50) is no longer valid for the stochastic gradient algorithm. Indeed, it can

be shown that convergence of the stochastic gradient algorithm (up to stochastic fluctuations about

dMSE as explained above) requires the step size µ to be chosen much smaller than for the gradient

algorithm. However, it is still true that fastest convergence of the stochastic gradient algorithm is

obtained for a flat input spectrum whereas spectral zeros lead to slow convergence.

The stochastic gradient (LMS) algorithm can be modified in various ways. Furthermore, an im-

portant alternative to stochastic gradient algorithms (LMS-type algorithms) is given by the family of

recursive least-squares (RLS) algorithms. These algorithms converge much faster than the LMS-type

algorithms but their computational complexity is much higher.

4.2.4 Fractionally Spaced Linear Equalizer

The equalizers discussed above are in trouble in one way or another when the transfer function of

the equivalent discrete-time channel filter has zero or near-zero values, i.e., when P (ejθ) ≈ 0 for some

value(s) of θ. We recall that P (ejθ) subsumes the transmit filter G(jω), the channel filter B(jω), the

receive filter F (jω), and the symbol-rate sampler. A zero value of P (ejθ) may be due to a zero value of

the actual channel transfer function B(jω)—which cannot be equalized by any means—but it may

also be due to the symbol-rate sampling. Indeed, the symbol-rate sampling causes aliasing , i.e., the

superposition of different frequency components. Depending on the sampling phase, this superposition

may be constructive or destructive. In the latter case, there may occur a zero or near-zero value of

P (ejθ) even though the actual channel transfer function B(jω) is nonzero at the corresponding ω.

Influence of the sampling phase. Let us take a closer look at this aliasing mechanism. Thus far,

we have assumed that the received signal (filtered by the receive filter f(t)) is sampled at the correct

sampling instants t = kTs. We then have (see (3.31))

P (ejθ) =1

Ts

∞∑

i=−∞P

(jθ− i2πTs

), (4.61)

with P (jω) = G(jω)BLP(jω)F (jω). In practice, however, there will be some sampling phase offset τ

so that the sampling instants are t = kTs + τ . Here, it is easily shown that the transfer function of

the equivalent discrete-time channel filter is given by

P (ejθ) =1

Ts

∞∑

i=−∞P

(jθ− i2πTs

)ejτ

θ−i2πTs . (4.62)


|P (ejθ)|

∣∣ 1TsP(j θ+2π

Ts

)∣∣ ∣∣ 1TsP(j θTs

)∣∣ ∣∣ 1TsP(j θ−2π

Ts

)∣∣

−2π −π πθ

(a)

−2π −π π 2π

θ

(b)

2π0

0

1

1

Figure 4.7: Aliasing due to symbol-rate sampling for p(t) being a Nyquist pulse: (a) Spe tral omponents1TsP(j θ−i2π

Ts

)ejτ

θ−i2πTs

(with phase fa tors ejτθ−i2π

Tsnot shown) for i = −1, 0, 1, (b) |P (ejθ)| a ording to

(4.62) for destru tive aliasing.

To illustrate the effect of the phase factors ejτθ−i2π

Ts , let us consider a total pulse p(t) that is a Nyquist

pulse. The aliasing due to symbol-rate sampling is depicted in Figure 4.7. Specifically, at frequency

θ = π (corresponding to ω = π/Ts), the spectral components15 P(j πTs

)and P

(j π−2π

Ts

)are exactly

equal— they both take on the value Ts/2 since p(t) is a Nyquist pulse—but the phase factors ejτθ−i2π

Ts

may cause destructive superposition. Indeed, for θ = π we obtain from (4.62)

P (ejπ) =1

Ts

[P

(jπ

Ts

)

︸︷︷︸Ts/2

ejτπTs + P

(jπ−2π

Ts

)

︸︷︷︸Ts/2

ejτπ−2πTs

]

=1

2

[ejπ

τTs + e−jπ τ

Ts

]

= cos(πτ

Ts

). (4.63)

In particular, for τ = Ts/2 (which means sampling exactly midway between the ideal sampling instants,

i.e., maximum sampling phase offset), we have

P (ejπ)∣∣∣τ=Ts/2

= cos(π1

2

)= 0 . (4.64)

15We assume excess bandwidth < π/Ts or equivalently < 1/(2Ts). Then, as shown in Figure 4.7(a), at frequency θ = π

only the spectral components corresponding to i = 0 and i = 1 overlap.


a[k]

a[k]

t=kTs

q(t)

√2 e−jωct

x[k]y(t)y(t)

√2 e−jωct

q(t)

(a)

(b)

y(t) y(t)f(t) dc(t)

x[k]

f(t)

t=nTs

2

q[n]=q(nTs

2

)dFSE[n]

Figure 4.8: Derivation of the fra tionally spa ed equalizer: (a) Passband PAM re eiver using a ontinuous-

time equalizer, (b) equivalent dis rete-time realization using a fra tionally spa ed equalizer.

Note that this zero value of P (ejπ) is entirely due to the symbol-rate sampling with maximum (worst-

case) phase offset. Without a phase offset (i.e., τ = 0), we would obtain P (ejπ)∣∣τ=0

= cos(0) = 1.

The fractionally spaced equalizer. Troublesome aliasing phenomena could be avoided altogether

if we implemented the equalizer before the symbol-rate sampling, i.e., as a continuous-time sys-

tem dc(t) ↔ Dc(jω) (see part (a) of Figure 4.8). However, since for technological reasons such

a continuous-time implementation is difficult if not impossible, we wish to develop an equivalent

discrete-time implementation. For this, our sampling has to be sufficiently fast so that aliasing is

avoided; in other words, the sampling condition must be satisfied. For excess bandwidth < π/Ts or

equivalently < 1/(2Ts), the bandwidth of the transmit signal (considered in the equivalent baseband

domain) is < 1/Ts. This shows that a sampling frequency of 2/Ts, i.e., double symbol rate, is sufficient

to avoid aliasing16.

The resulting PAM receiver is depicted in Figure 4.8(b). The continuous-time receive filter f(t) is

followed by a sampler operating at the double symbol rate, a fractionally spaced equalizer with impulse

response dFSE[n], a downsampling by a factor of 2, and a slicer which, of course, operates at the symbol

rate. The fractionally spaced equalizer is a discrete-time filter operating at the double symbol rate. In

what follows, we will allow this filter to have infinite length. The downsampling performs a conversion

from double symbol rate to symbol rate by discarding every second sample.

We shall now show that the receiver in Figure 4.8(b) using an infinite-length fractionally spaced

equalizer dFSE[n] ↔ DFSE(ejθ) is fully equivalent to the receiver in Figure 4.8(a) using a continuous-

time equalizer dc(t) ↔ Dc(jω), provided that the two equalizer filters are appropriately related. Our

derivation will make use of the following two results.

• Since q(t), the output of the receive filter f(t) or equivalently input to the continuous-time

equalizer dc(t), is bandlimited with (one-sided) bandwidth B < 1/Ts, it can be reconstructed

16We assume that the receive filter f(t) causes the channel noise to be bandlimited to the signal band.


from its samples taken with sampling frequency 2/Ts or equivalently sampling period Ts/2:

q(t) =

∞∑

n=−∞q[n] sinc

(πt− nTs/2

Ts/2

)with q[n] = q

(nTs2

), (4.65)

where sinc(α) = sinαα .

• Furthermore, again due to the bandlimitation of q(t), we can assume without loss of generality

that the continuous-time equalizer dc(t) is itself bandlimited with (one-sided) bandwidth B <

1/Ts. It can then be shown that the inner product of dc(t) with the shifted sinc functions

sinc(π t−nTs/2

Ts/2

)gives the samples of dc(t) at the instants t = nTs

2 up to a constant factor (see

equation (B.32) in Appendix B for a proof of this relation):

∫ ∞

−∞dc(t) sinc

(πt− nTs/2

Ts/2

)dt =

Ts2dc

(nTs2

). (4.66)

We are now ready to show that the receiver using dFSE[n] is equivalent to the one using dc(t).

Consider the input to the slicer in Figure 4.8(a). This is simultaneously the output of the equalizer

filter dc(t) sampled at t = kTs, and thus given by

x[k] =

∫ ∞

−∞q(t) dc(kTs−t) dt . (4.67)

Inserting (4.65), interchanging the order of summation and integration, using sinc(−α) = sinc(α), and

using (4.66), we obtain

x[k] =

∫ ∞

−∞

[ ∞∑

n=−∞q[n] sinc

(πt− nTs/2

Ts/2

)]dc(kTs−t) dt

=

∞∑

n=−∞q[n]

[∫ ∞

−∞dc(kTs−t) sinc

(πt− nTs/2

Ts/2

)dt

]

=

∞∑

n=−∞q[n]

[∫ ∞

−∞dc(t) sinc

(πkTs − t− nTs/2

Ts/2

)dt

]

=∞∑

n=−∞q[n]

[ ∫ ∞

−∞dc(t) sinc

(πt− (2k−n)Ts/2

Ts/2

)dt

︸︷︷︸Ts2

dc((2k−n)Ts2 )

]

=

∞∑

n=−∞q[n] dFSE[2k−n] , (4.68)

where we have set

dFSE[n] ,Ts2dc

(nTs2

). (4.69)

On the other hand, let us consider the receiver in Figure 4.8(b). The output of the fractionally spaced

equalizer dFSE[n] is given by∑∞

n=−∞ q[n] dFSE[m−n]. Downsampling this by a factor of 2 (i.e., letting

m→ 2k), we obtain the input to the slicer as∑∞

n=−∞ q[n] dFSE[2k−n]. This is equal to (4.68). Thus,


Dc

(j θ+2π

Ts/2

)Dc

(j θTs/2

)DFSE(e

jθ)

2πθ

Dc

(j θ−2π

Ts/2

)

−2π −π 0 π

Figure 4.9: Illustration of the relation (4.70) onne ting the transfer fun tion of the ontinuous-time

equalizer, Dc(jω), and the transfer fun tion of the fra tionally spa ed equalizer, DFSE(ejθ).

we have shown that the receiver in Figure 4.8(b) is indeed equivalent to the receiver in Figure 4.8(a)

provided that the impulse responses of the two equalizer filters are related according to (4.69).

From (4.69), it follows that the transfer functions of the two equalizers are related as

DFSE(ejθ) =

Ts2

1

Ts/2

∞∑

i=−∞Dc

(jθ− i2πTs/2

)=

∞∑

i=−∞Dc

(jθ− i2πTs/2

). (4.70)

This is depicted in Figure 4.9. Since dc(t) is bandlimited with bandwidth < 1/Ts and sampled at

rate 2/Ts, the individual spectra do not overlap, i.e., there does not occur aliasing. In particular, on

the fundamental frequency interval the transfer functions Dc(jω) and DFSE(ejθ) are equivalent,

DFSE(ejθ) = Dc

(j

θ

Ts/2

)for − π < θ < π . (4.71)

The continuous-time equalizer dc(t) ↔ Dc(jω) can be designed according to the ZF criterion

(cf. the ZF receive filter design discussed in Subsection 3.3.4) or according to the MSE criterion.

The associated fractionally spaced equalizer dFSE[n] ↔ DFSE(ejθ) can then be obtained via (4.69) or

equivalently (4.71).

With the fractionally spaced equalizer, a sampling phase offset is no problem any more. Indeed,

a sampling phase offset τ can be described in the frequency domain simply by a phase factor ejτω

that can be compensated by a continuous-time allpass filter. Since a fractionally spaced equalizer

with infinite length can emulate any appropriately bandlimited continuous-time system, this allpass

filtering (bandlimited to the signal band) can equivalently be performed by the fractionally spaced

equalizer. This shows that a noncausal, infinite-length, fractionally spaced equalizer can compensate

arbitrary sampling phase offsets. (With a practical fractionally spaced equalizer that is causal and

finite-length, the allpass filter can only be approximated and there is a delay.)

We finally note that there also exist fractionally spaced versions of the finite-length (FIR) MSE

linear equalizer and of the adaptive linear equalizer.

4.2.5 Problems

Problem 4.2.1. In Subsection 4.2.2, only an outline of the derivation of (4.30) was given. Provide adetailed derivation of (4.27), (4.28), and (4.30).


Problem 4.2.2. Show that for P (ejθ) and SZ(ejθ) given and for θ arbitrary but fixed, the noise power

spectrum SU(ejθ) at the output of the infinite-length MSE linear equalizer (see (4.36)) is ≤ the noise

power spectrum SU(ejθ) at the output of the ZF linear equalizer (see (4.4)).

Problem 4.2.3. At the input of a ZF linear equalizer, the equivalent pulse is p[k] = e−aku[k] witha > 0 (here, u[k] is the discrete-time unit step, i.e., u[k] = 1 for k ≥ 0 and u[k] = 0 for k < 0), andthe equivalent noise is zero-mean and white with power spectral density SZ(e

jθ) ≡ σ2Z .

a) Calculate the transfer function of the ZF linear equalizer.

b) Calculate the equivalent pulse at the output of the ZF linear equalizer.

c) Calculate and sketch the power spectral density of the noise U [k] at the output of the ZF linearequalizer.

d) Calculate the variance/mean power of U [k].

e) Calculate the noise enhancement, i.e., the ratio σ2U/σ2Z .

Problem 4.2.4. At the input of a ZF linear equalizer, the equivalent pulse is p[k] = aδ[k+1]+ δ[k]+aδ[k − 1] with a ∈ R, and the equivalent noise is zero-mean and white with power spectral densitySZ(e

jθ) ≡ σ2Z .

a) Calculate the transfer function of the ZF linear equalizer.

b) Calculate and sketch the power spectral density of the noise U [k] at the output of the ZF linearequalizer.

c) Discuss your results for various values of a.

Problem 4.2.5. The received pulse in a PAM system is given by h(t) = e−atu(t) where a > 0. Thechannel noise N(t) is white with power spectral density N0/2. The baseband section of the receiverconsists of a receive filter f(t), a symbol-rate sampler, a ZF linear equalizer, and a slicer.

a) Specify the receive filter f(t) that minimizes the noise enhancement caused by the equalizer.This receive filter is to be used in what follows.

b) Calculate the equivalent discrete-time pulse p[k] at the input of the equalizer.

c) Calculate the transfer function of the ZF linear equalizer.

d) Calculate and sketch the power spectral density of the noise U [k] at the output of the ZF linearequalizer.

Problem 4.2.6. Consider a noncausal, linear equalizer filter of length 2L+ 1 with impulse responsed[k], k = −L,−L+1, · · ·, L. Let p[k] (−∞ < k < ∞) denote the equivalent pulse at the input of theequalizer.

a) Provide an expression for the equivalent pulse p(d)[k] at the output of the equalizer filter.

b) If d[k] was the ZF equalizer, there would be p(d)[k] = δ[k] for all k; however, in general thisis impossible due to the finite length of the equalizer. Therefore, we relax our ZF conditionby requiring that p(d)[k] = δ[k] on the finite interval k = −L,−L+1, · · ·, L whereas p(d)[k] isarbitrary outside that interval. Formulate a system of linear equations whose solution yields the


equalizer d[k], k = −L,−L+1, · · ·, L satisfying this relaxed ZF condition. (You need not solvethis system of equations.)

c) Instead of p(d)[k] = δ[k] for k = −L,−L + 1, · · ·, L, we now desire p(d)[k] ≈ δ[k] for k =−K,−K+1, · · ·,K with K ≥ L. The approximation error is defined by

ǫd ,

K∑

k=−K

(p(d)[k]− δ[k]

)2. (4.72)

Derive a system of linear equations that the optimum equalizer (i.e., the equalizer d[k], k =−L,−L+1, · · ·, L minimizing ǫd) has to satisfy. Compare your result with the system of linearequations obtained in part b). What happens when K = L?

Problem 4.2.7. Let E[k] denote the error at the slicer input obtained with the MSE equalizer dMSE.Show that E[k] is statistically orthogonal to the equalizer input vector q[k], i.e., E

q[k]E∗[k]

= 0

(orthogonality principle of linear mean-square estimation).

Problem 4.2.8. Consider passband PAM transmission with QPSK symbols A[k] ∈ 1, j,−1,−j. Allsymbols are equally likely and the symbol sequence A[k] is white. At the input of a linear equalizer,the equivalent pulse is p[k] = 2−ku[k] (here, u[k] = 1 for k ≥ 0 and u[k] = 0 for k < 0), and theequivalent noise is zero-mean and white with power spectral density SZ(e

jθ) ≡ 1.

a) Calculate the MSE linear equalizer with L = 1 (i.e., 3 taps).

b) Calculate the variance/mean power of the noise U [k] at the output of the MSE linear equalizer.

c) Calculate the noise enhancement, i.e., the ratio σ2U/σ2Z .

Problem 4.2.9. Frequency-domain derivation of the infinite-length MSE linear equalizer. The inputto an infinite-length linear equalizer is

Q[k] =

∞∑

l=−∞p[l]A[k−l] + Z[k] , (4.73)

where A[k] is stationary and white with mean power PA, Z[k] is stationary and zero-mean with powerspectral density SZ(e

jθ), and A[k] and Z[k] are uncorrelated. The MSE can be expressed as

ǫ , E|E[k]|2

=

1

2π

∫ π

−πSE(e

jθ) dθ , (4.74)

where SE(ejθ) is the power spectral density of the error E[k] at the slicer input.

a) Derive an expression of SE(ejθ) in terms of D(ejθ) (the equalizer’s transfer function), PA, P (e

jθ),and SZ(e

jθ).

b) Calculate the transfer function D(ejθ) minimizing ǫ, and compare your result to (4.33).

Hint : According to (4.74) and the expression of SE(ejθ) found in part a), ǫ can be minimized by

minimizing SE(ejθ) for each θ separately. Minimization of SE(e

jθ) for arbitrary but fixed θ is mosteasily done by a “completing the square” procedure that results in an expression of SE(e

jθ) structurallyanalogous to (4.23). From this expression, DMSE(e

jθ) can be found by inspection.

Problem 4.2.10. Redo Problem 4.2.9 for a colored symbol sequence A[k] with power spectrumSA(e

jθ).


Problem 4.2.11. Show (4.38) under the simplifying assumption that all quantities involved arereal-valued. Hint : First calculate the gradient of dTAd (with A symmetric, i.e., AT = A) and ofaTd = dTa.

Problem 4.2.12. Let λl denote the eigenvalues of the correlation matrix RQ. Assuming nonsingu-larity (and, thus, positive definiteness) of RQ, there is 0 < λmin ≤ λl ≤ λmax. According to (4.49),the convergence properties of the gradient algorithm are determined by the behavior of (1− µλl)

i fori = 1, 2, · · · . According to (4.50), the gradient algorithm converges if and only if 0 < µ < 2/λmax.

a) Discuss the behavior of (1− µλl)i for λl = λmin and λl = λmax, using the following values of the

step size:

• µ = ε > 0 (i.e., a very small, positive quantity);

• µ = 2/λmax − ε > 0 (i.e., slightly less than 2/λmax);

• µ = 1/λmax.

b) Consider the step size µ0 = 2/(λmin + λmax).

• Verify that 1− µ0λmax = −(1− µ0λmin) and, thus, |1− µ0λmax| = |1− µ0λmin|.• Calculate |1− µ0λ| for λ = (λmin + λmax)/2.

• Sketch f(λ) = |1− µ0λ| for λmin ≤ λ ≤ λmax.

Problem 4.2.13. Consider a passband PAM system in which the Fourier transform P (jω) =G(jω)BLP(jω)F (jω) of the total pulse p(t) is given by

P (jω) = rect

(ω ;

3

2

π

Ts

). (4.75)

Due to a sampling phase offset at the receiver, the sampling instants are t = kTs + τ .

a) Calculate the Fourier transform of the resulting equivalent discrete-time pulse, P (ejθ). SketchP (ejθ) for τ = Ts/4, τ = Ts/2, and τ = 3Ts/4. (Cf. Exercise 3.2.10.)

b) Calculate the power spectral density of the noise U [k] at the output of the ZF linear equalizer,assuming white equivalent channel noise (SZ(e

jθ) ≡ σ2Z). Sketch SU (ejθ) for the three values of

τ given in part a).

Problem 4.2.14. Show (4.66), assuming that dc(t) is bandlimited with (one-sided) bandwidth B <1/Ts. Hint : Apply Parseval’s relation

∫∞−∞ x(t) y∗(t) dt = 1

2π

∫∞−∞X(jω)Y ∗(jω) dω to the left-hand

side of (4.66).

Problem 4.2.15. In Subsection 4.2.4 (see (4.67)–(4.69)), it was shown by means of a time-domainderivation that

x[k] ,

∫ ∞

−∞q(t) dc(kTs−t) dt =

∞∑

n=−∞q[n] dFSE[2k−n] for dFSE[n] =

Ts2dc

(nTs2

), (4.76)

where q[n] = q(nTs/2) and where q(t) and dc(t) are bandlimited as explained in Subsection4.2.4. Illustrate this equivalence in the frequency domain by sketching the Fourier transforms ofx[k] =

∫∞−∞ q(t) dc(kTs− t) dt and of x[k] =

∑∞n=−∞ q[n] dFSE[2k−n] for generic shapes of the Fourier

transforms of q(t) and dc(t).

4.3 Decision-Feedback Equalizer 119

q(t)

t=kTs

x[k]

b[k]

√2 e−jωct

f(t) d[k]a[k]y(t) q[k]y(t)

v[k]

Figure 4.10: Passband PAM re eiver with de ision-feedba k equalizer.

Problem 4.2.16. In Subsection 4.2.4, the fractionally spaced linear equalizer was derived for excessbandwidth Ωexcess ≤ π/Ts. Develop a fractionally spaced equalizer for π/Ts < Ωexcess ≤ 2π/Ts (i.e.,Ωexcess may be as large as 2π/Ts). Which oversampling factor is required in this case?

Problem 4.2.17. Consider a PAM system in which the received pulse h(t) is a root-raised-cosinepulse with roll-off factor α = 0.5. The receiver uses a matched receive filter f(t) = h∗(−t) and asampler with sampling phase offset τ = Ts/4.

a) Assume sampling at the double symbol rate. Calculate and sketch the transfer function of thefractionally spaced ZF linear equalizer for the given sampling phase offset.

b) Assume sampling at the symbol rate. Calculate and sketch the transfer function of the ZF linearequalizer for the given sampling phase offset. Compare with the result of part a) and discussthe noise enhancement occurring in either case.

4.3 Decision-Feedback Equalizer

ISI consists of precursor ISI (interference from future symbols) and postcursor ISI (interference from

past symbols). Assuming that the slicer decisions are correct most of the time, the past symbols are

effectively known. The postcursor ISI can then be canceled without noise enhancement by subtracting

a replica of it from the slicer input. This replica can be formed by passing the estimated past symbols

(i.e., the symbols previously detected by the slicer) through a suitable filter.

This idea leads to the decision-feedback equalizer whose block diagram is shown in Figure 4.10 (cf.

Figure 4.1). The decision-feedback equalizer consists of a feedforward filter d[k] ↔ D(ejθ) that filters

the output q[k] of the symbol-rate sampler and a feedback filter v[k] ↔ V (ejθ) that feeds back the

detected symbols, i.e., the slicer output a[k]. The feedforward filter attempts to make the ISI causal

or, equivalently, to suppress the precursor ISI; it thus acts as a precursor equalizer . The feedback filter

combined with the subtraction stage attempts to cancel the postcursor ISI; it thus acts as a postcursor

equalizer . Both the feedforward filter and the feedback filter are linear filters; however, the overall

decision-feedback equalizer is a nonlinear system because it contains the (nonlinear) slicer.

An obvious problem with the decision-feedback equalizer is error propagation: any symbols that

are detected incorrectly (due to decision errors of the slicer) are fed back and thus will influence the

decision of subsequent symbols, which may cause further decision errors. However, if the channel SNR


a[k]p[k]

q[k]

z[k]

q[k+L]

q[k+L]

d[−L] d[−L+1] d[L]

x[k] x[k]− b[k]

b[k]

a[k]

a[k−1]z−1

v[K]

z−1 z−1

z−1 z−1

zL

q[k+L−1] q[k−L]

a[k−K]

v[2] v[1]

a[k−2]

Figure 4.11: PAM system with FIR de ision-feedba k equalizer.

is above a certain threshold, the benefit of reduced noise enhancement outweighs the detrimental effect

of error propagation.

As for the linear equalizer, two design critera for the decision-feedback equalizer are the ZF criterion

that attempts to remove all of the ISI and the MSE criterion that allows some residual ISI in order to

reduce the noise enhancement introduced by the feedforward filter. We will discuss the MSE decision-

feedback equalizer in what follows. The ZF decision-feedback equalizer generally requires filters of

infinite length and will not be considered.

4.3.1 Mean-Square Error Design

The mean-square error (MSE) design minimizes the overall MSE at the slicer input. As we will see,

the derivation of the MSE decision-feedback equalizer is partly analogous to the derivation of the MSE

linear equalizer in Subsection 4.2.2.

The FIR decision-feedback equalizer. A decision-feedback equalizer using FIR filters is depicted

in Figure 4.11. The feedforward filter has 2L+ 1 coefficients d[k], k = −L,−L+1, · · · , L (this filter


can be made causal by introducing a delay by L samples, which amounts to removing the “time

advance system” zL in Figure 4.11). The feedback filter has K coefficients v[k], k = 1, 2, · · · ,K. This

is an admissible loop filter since it is causal with zero-delay coefficient v[0] equal to zero, i.e., v[k] = 0

for k ≤ 0 (note that a nonzero zero-delay coefficient v[0] would result in a delayless loop that is not

realizable).

We now discuss the operation of the decision-feedback equalizer.

• Equalizer input. We assume an infinite sequence of transmitted symbols A[k] (−∞ < k < ∞)

and a general equivalent discrete-time baseband PAM system with filter p[k] ↔ P (ejθ) and

additive noise Z[k] with power spectral density SZ(ejθ). The input to the equalizer is then given

by (cf. (4.12))

Q[k] =

∞∑

l=−∞p[l]A[k−l] + Z[k] . (4.77)

• Output of feedforward filter. The feedforward filter is fully analogous to the FIR linear equalizer

of Subsection 4.2.2. Its output is (cf. (4.13))

X[k] =

L∑

l=−L

d[l]Q[k−l] = dHq[k] , (4.78)

with the (2L+ 1)× 1 vectors (cf. (4.14))

d ,

d∗[−L]d∗[−L+1]

...d∗[L]

, q[k] ,

Q[k+L]Q[k+L−1]

...Q[k−L]

. (4.79)

• Output of feedback filter. Assuming the slicer decisions to be correct, i.e., A[k] ≡ A[k], the

output of the feedback filter is given by

B[k] =

K∑

l=1

v[l]A[k−l] = vHa[k] , (4.80)

with the K × 1 vectors

v ,

v∗[1]v∗[2]...

v∗[K]

, a[k] ,

A[k−1]A[k−2]

...A[k−K]

. (4.81)

• Error signal. The slicer input is X[k] −B[k]; hence, the error at the slicer input is obtained as

E[k] = (X[k]−B[k]) − A[k] = dHq[k]− vHa[k]−A[k] .

We can write the error more compactly as

E[k] = dH q[k]−A[k] , (4.82)


with the (2L + 1 + K) × 1 “stacked” coefficient vector d and the (2L + 1 + K) × 1 “stacked”

input vector q[k] given by

d ,

(d−v

), q[k] ,

(q[k]

a[k]

). (4.83)

Optimum equalizer filters. By definition, the MSE decision-feedback equalizer is given by those

filters d and v for which the MSE ǫ = E|E[k]|2

is minimized. Deriving these optimum equalizer

filters is not difficult. Indeed, a comparison of (4.82) with (4.15) shows that we have formulated the

error E[k] in the same way as for the MSE linear equalizer; the only difference is that the vectors d

and q[k] are replaced by d and q[k], respectively. Thus, we can directly reuse the result (4.26) with

obvious modifications. This gives the following “compound Wiener-Hopf equation” for the optimum

compound coefficient vector d,

RQ d = rQ,A . (4.84)

Here, we have defined the (2L+ 1 +K)× (2L+ 1 +K) correlation matrix

RQ , Eq[k]qH [k]

= E

(q[k]

a[k]

)(qH [k] aH [k]

)=

(Eq[k]qH [k] Eq[k]aH [k]Ea[k]qH [k] Ea[k]aH [k]

)

=

(RQ RQ,A

RHQ,A RA

), (4.85)

where RQ , Eq[k]qH [k], RQ,A , Eq[k]aH [k], and RA , Ea[k]aH [k]. Furthermore, we have

defined the (2L+ 1 +K)× 1 correlation vector

rQ,A , Eq[k]A∗[k]

= E

(q[k]

a[k]

)A∗[k]

=

(Eq[k]A∗[k]Ea[k]A∗[k]

)=

(rQ,A

rA

), (4.86)

where rQ,A , Eq[k]A∗[k] and rA , Ea[k]A∗[k].We now assume that all processes involved are jointly stationary, from which it follows that RQ

and rQ,A do not depend on k. Furthermore, for the sake of simplicity the symbol sequence A[k] is

assumed white and zero-mean with mean power PA, so that RA[l] = EA[k]A∗[k − l]

= PA δ[l]. It

then follows that

[RA]i,j = EA[k−i]A∗[k−j]

= RA[j−i] = PAδ[j−i] , i, j = 1, 2, · · · ,K (4.87)

[rA]i = EA[k−i]A∗[k]

= 0 , i = 1, 2, · · · ,K , (4.88)

or equivalently

RA = PA I (4.89)

rA = 0 . (4.90)

With these relations, the compound Wiener-Hopf equation (4.84) becomes


(RQ RQ,A

RHQ,A PAI

)(d−v

)=

(rQ,A

0

), (4.91)

or equivalently

RQd − RQ,Av = rQ,A (4.92)

RHQ,Ad − PAv = 0 . (4.93)

We first determine the feedforward filter d. Solving (4.93) for v and inserting the result into (4.92),

we obtain the following equation in d,

(RQ − 1

PARQ,AR

HQ,A

)d = rQ,A , (4.94)

whose formal solution is

dMSE =

(RQ − 1

PARQ,AR

HQ,A

)−1

rQ,A . (4.95)

From dMSE, the optimum feedback filter vMSE can be derived according to (4.93),

vMSE =1

PARH

Q,A dMSE (4.96)

=1

PARH

Q,A

(RQ − 1

PARQ,AR

HQ,A

)−1

rQ,A . (4.97)

The Wiener-Hopf equation. The Wiener-Hopf equation for the feedforward filter d is given by

(4.94). Let us consider this equation in more detail. With Q[k] = p[k] ∗A[k] + Z[k] (see (4.77)), and

assuming that the processes A[k] and Z[k] are uncorrelated with Z[k] zero-mean, it can be shown that

the elements of the correlation matrices/vectors RQ, RQ,A, and rQ,A are related to p[k], PA, and RZ [l]

as (cf. (4.27), (4.28))

[RQ]i,j = RQ[j−i] = PA ρp[j−i] + RZ [j−i] , i, j = 1, 2, · · · , 2L+1 (4.98)

[RQ,A]i,j = PA p[L+j−i+1] , i = 1, 2, · · ·2L+1 , j = 1, 2, · · · ,K (4.99)

[rQ,A]i = PA p[L−i+1] , i = 1, 2, · · · , 2L+1 , (4.100)

where (cf. (4.29))

ρp[k] , p[k]∗ p∗[−k] =

∞∑

i=−∞p[i] p∗[i−k] . (4.101)

Inserting into (4.94) yields after some manipulations the following system of linear equations,

L∑

l=−L

[PA ρp[k, l] + RZ [k−l]

]d[l] = PA p

∗[−k] , k = −L,−L+1, · · · , L , (4.102)

where we have set


ρp[k, l] , ρp[k−l] −K∑

i=1

p[i−l] p∗[i−k]

=

∞∑

i=−∞p[i−l] p∗[i−k] −

K∑

i=1

p[i−l] p∗[i−k]

=∑

i 6∈ [1,K]

p[i−l] p∗[i−k] . (4.103)

Equation (4.102) is only superficially similar to the analogous equation (4.30) that we obtained for the

MSE linear equalizer. The essential difference is that ρp[k, l] is a function of both k and l individually

whereas ρp[k − l] in (4.30) depended only on the difference k − l. Hence, (4.102) does not become a

convolution for K → ∞ and L → ∞, which means that the infinite-length case cannot be elegantly

handled in the frequency domain as in the case of the linear equalizer.

Equivalent discrete-time baseband PAM system seen at the output of the optimum feed-

forward filter. Consider the output X[k] of the optimum feedforward equalizer dMSE[k]. Inserting

(4.77) into (4.78) with d[k] = dMSE[k], we obtain

X[k] =

L∑

l=−L

dMSE[l]

[ ∞∑

m=−∞p[m]A[k−l−m] + Z[k−l]

]

=L∑

l=−L

dMSE[l]

[ ∞∑

i=−∞p[i−l]A[k−i]

]+

L∑

l=−L

dMSE[l] Z[k−l]

=∞∑

i=−∞

[L∑

l=−L

dMSE[l] p[i−l]]A[k−i] +

L∑

l=−L

dMSE[l] Z[k−l]

=

∞∑

i=−∞p(d)[i]A[k−i] + Z(d)[k] , (4.104)

with

p(d)[k] ,

L∑

l=−L

dMSE[l] p[k−l] , Z(d)[k] ,

L∑

l=−L

dMSE[l] Z[k−l] . (4.105)

Here, p(d)[k] is the total pulse shape obtained at the output of the optimum feedforward equalizer

dMSE[k], and Z(d)[k] is the noise obtained at the same point. The pulse p(d)[k] subsumes the trans-

mitter, channel, receive filter, symbol-rate sampler, and optimum feedforward equalizer.

There is an interesting relation involving p(d)[k]. Insertion of (4.99) into (4.96) yields the following

relation between the impulse responses of the optimum feedforward and feedback filters:

vMSE[k] =L∑

l=−L

dMSE[l] p[k−l] , k = 1, 2, · · · ,K . (4.106)

Comparing with (4.105), we see that

vMSE[k] = p(d)[k] , k = 1, 2, · · · ,K . (4.107)


This means that the impulse response of the optimum feedback equalizer filter vMSE[k] equals the pulse

shape obtained at the output of the optimum feedforward equalizer for k = 1, 2, · · · ,K.

Equivalent discrete-time baseband PAM system seen at the slicer input. We finally cal-

culate the pulse shape and noise obtained at the slicer input. With (4.104), (4.80), and (4.107), the

slicer input is obtained as

X[k] −B[k] =∞∑

l=−∞p(d)[l]A[k−l] + Z(d)[k] −

K∑

l=1

vMSE[l]︸︷︷︸p(d)[l]

A[k−l]

=∑

l 6∈ [1,K]

p(d)[l]A[k−l] + Z(d)[k]

=

∞∑

l=−∞p(d,v)[l]A[k−l] + Z(d)[k] , (4.108)

with

p(d,v)[k] ,

0 , k = 1, 2, · · · ,Kp(d)[k] , otherwise.

(4.109)

This shows that the optimum feedback filter completely cancels the postcursor ISI over the interval

k = 1, 2, · · · ,K covered by the length of the feedback filter . Furthermore note that the noise still

is Z(d)[k], which means that subtracting the output B[k] of the feedback filter does not enhance (or

otherwise modify) the noise at the slicer input .

4.3.2 Variations

In view of the formal analogy of the MSE decision-feedback equalizer to the MSE linear equalizer, it is

not surprising that the MSE decision-feedback equalizer can be approximated by an adaptive version

using the LMS algorithm (cf. Subsection 4.2.3).

Furthermore, the feedforward filter of the decision-feedback equalizer can be implemented in a

fractionally spaced mode— i.e., operating at the double symbol rate with its output downsampled by

a factor of 2— in order to avoid detrimental effects of sampling phase offsets (cf. Subsection 4.2.4).

Of course, the feedback filter must always operate at the symbol rate.

4.3.3 Problems

Problem 4.3.1. Consider an MSE decision-feedback equalizer with L = 0 and K = 2 (i.e., thefeedforward filter has one tap and the feedback filter has two taps). All processes involved are jointlystationary, and the symbol sequence A[k] is white and zero-mean with mean power PA. The followingquantities are known: E|Q[k]|2 = PQ and EQ[k]A∗[k−i] = γi for i = 0, 1, 2.

a) Specify the matrices RQ, RQ,A, and RA as well as the vectors rQ,A and rA.

b) Calculate the filter coefficients of the MSE decision-feedback equalizer.


Problem 4.3.2. Consider a decision-feedback equalizer with a given feedforward filter d[k]. (Thisneed not be the MSE-optimum feedforward filter.) All processes involved are jointly stationary, andthe symbol sequence A[k] is white and zero-mean with mean power PA.

a) Calculate the MSE-optimum feedback filter v[k] of length K (i.e., the feedback filter that mini-mizes the MSE at the slicer input for the given feedforward filter d[k]).

b) Show that the decision-feedback equalizer using the MSE-optimum feedback filter from part a)compensates the causal part of the ISI in the interval k = 1, 2, · · ·,K. (As always, the slicerdecisions are assumed to be correct, i.e., A[k] = A[k]).

PAMFundamentals - nt.tuwien.ac.at€¦ · signaling bandlimited transmit pulse. Note that, r fo Tb...

Documents

Transcript of PAMFundamentals - nt.tuwien.ac.at€¦ · signaling bandlimited transmit pulse. Note that, r fo Tb...