TSKS01 Digital Communication Lecture 12 › TSKS01 › lectures › 12 › F12-HT10.pdf · Started...
Transcript of TSKS01 Digital Communication Lecture 12 › TSKS01 › lectures › 12 › F12-HT10.pdf · Started...
2010-11-19
Linköpings universitet 1
TSKS01 Digital Communication Lecture 12
Source Coding
Summing up the course
Mikael Olofsson
Department of EE (ISY)
Div. of Communication Systems
Last Time
Finished Error Control Coding
� Bounds: Hamming bound and Singleton bound
� CRC codes
Started Source Coding
� Tree Codes
� The Huffman Algorithm
� Kraft’s Inequality: ∃ tree code w. lengths l1,…,lN iff holds.
� Entropy
2010-11-19 TSKS01 Digital Communication - Lecture 12 2
121
≤∑=
−N
i
li
Recall Huffman Coding 1(2)
2010-11-19 TSKS01 Digital Communication - Lecture 12 3
0.8
0.1
0.05
0.05
0.8
0.1
0.1
0.8
0.2
1.01
01
01
0
a1
a2
a3
a4
Recall Huffman Coding 2(2)
2010-11-19 TSKS01 Digital Communication - Lecture 12 4
0.8
0.1
0.05
0.05
0.8
0.1
0.1
0.8
0.2
1.01
01
01
0
1
00
011
010
a1
a2
a3
a4
c(1) =
c(2) =
c(3) =
c(4) =
1233
0.80.20.150.151.3mL =
0.80.10.050.05
pi
100011010
c(i) li pi li ( ) 022.1log1
2 ≈−= ∑=
N
iii ppAHEntropy:
( ) 278.0≈− AHmLRedundancy:
54.1log2 ≈
Lm
NCompr. ratio:
2010-11-19
Linköpings universitet 2
2010-11-19 TSKS01 Digital Communication - Lecture 12 5
Entropy
2010-11-19 TSKS01 Digital Communication - Lecture 12 6
Simplified Huffman Code for an Extended Source
2010-11-19 TSKS01 Digital Communication - Lecture 12 7
A Markov Source witn N = 3 Result of Source Coding
Almost equally probable almost uncorrelated bits.
2010-11-19 TSKS01 Digital Communication - Lecture 12 8
The closer we get to entropy, the closer we get to almost equally
probable and almost uncorrelated bits.
2010-11-19
Linköpings universitet 3
TSKS01 Digital Communication – Exam
2010-11-19 TSKS01 Digital Communication - Lecture 12 9
Three parts: Introductory task – At least one of two.
Question part 2 × 5p At least 3 points.
Problem part 4 × 5p At least 6 points.
Grades: Grade 3 (ECTS C): 14p
Grade 4 (ECTS B): 19p Max: 30p
Grade 5 (ECTS A): 24p
Allowed aids: Tables & Formulas in Signal Theory.
Pocket calculator with empty memory.
2010-11-19 TSKS01 Digital Communication - Lecture 12 10
A One-way Telecommunication System
Channel
Source encoder
Source decoder
Source
Destination
Channel encoder Modulator
Channel decoder
De-modulator
Source coding
Channel coding
Packing
Unpacking
Error control
Error correction
Digital to analog
Analog to digital
Medium
Digital modulation
2010-11-19 TSKS01 Digital Communication - Lecture 12 11
Cables – LTI Filtering
Mathematically: Linear differential equation
LTI system, impulse response, frequency response
Model:
Two wires:
2010-11-19 TSKS01 Digital Communication - Lecture 12 12
Optical fibers – Multimode Propagation
Different paths - different distances
⇒ different delays ⇒ pulse spreading
Light pulse in Light pulse out
2010-11-19
Linköpings universitet 4
2010-11-19 TSKS01 Digital Communication - Lecture 12 13
Radio – Multipath Transmission – Fading
( ) ( )1
N
k kk
v t s tρ τ=
= −∑Output:
DelayReflection coeff.
2010-11-19 TSKS01 Digital Communication - Lecture 12 14
Thermal Noise
A resistor:
Thermal movements of electrons
⇒ Random local currents
⇒ Random local voltages
⇒ Random total voltage
Model:
)(tV
∑=
=N
kk tItI
1
)()(
+ −
R
)()( tIRtV ⋅=
Enormous
⇒ I(t) Gaussian
⇒ V(t) Gaussian
Short pulses, almost unit impulses⇒ I(t1) & I(t2) almost independent
for t1 ≠ t2
(Almost) White Gaussian Noise
2010-11-19 TSKS01 Digital Communication - Lecture 12 15
Function Spaces – Vector Spaces
M signals, span N dimensions, .
Ortho-normal basis: (span the same dimensions) .
The signals expressed in this basis:
Corresponding vectors:
2010-11-19 TSKS01 Digital Communication - Lecture 12 16
Geometric Interpretation
Signals:
Vectors:
ON-basis:
Inner products:
. Zero outside .and
Result:
2010-11-19
Linköpings universitet 5
2010-11-19 TSKS01 Digital Communication - Lecture 12 17
AWGN – Additive White Gaussian Noise
Orthogonal noise components are statistically independent.
X1 & X2 independent
Y1 & Y2 independent
σXi= σYi
= RW ( f ) = N0/2 PSD = Variance
Y2
Y1
X1
X2
2 2
2010-11-19 TSKS01 Digital Communication - Lecture 12 18
1 3
3
1
–1
φ0
φ1
2s
0s
1s
ML Decision Regions – for AWGN Channel
?
B2
B1
B0
x
Interprete as the nearest signal.x
!
Detection:Decision regions consisting of all points closest to a signal point.
Notation:Bi is the decision region of the signal vector si. Thus also of the signal si (t)and of the message ai.
The borders are orthogonal to straight lines between signals:
In 2 dimensions: Lines.In 3 dimensions: Planes.In more dim: Hyperplanes.
Borders cut the lines mid-way.
2010-11-19 TSKS01 Digital Communication - Lecture 12 19
Exact Expression of Error Probability
1 3
3
1
–1
φ0
φ1
2s
0s
1s
B2
B1
B0
Interprete as the nearest signal.x
We had:
{ }∑−
=
=∉=1
0e |Pr
1 M
iii aABX
MP
{ }∑∑−
= ≠
=∈=1
0
|Pr1 M
i ijij aABX
M
Hard to calculate!!
Approximate and/or bound.
2010-11-19 TSKS01 Digital Communication - Lecture 12 20
The Union Bound
1 3
3
1
–1
φ0
φ1
2s
0s
1s
B2
B1
B0
Interprete as the nearest signal.x
Define overestimated regions:
( ) ( ){ }ijji sxdsxdxB ,,:, <=
B0,1B0,2
An upper bound based on overestimating the decision regions. We had:
{ }∑∑−
= ≠
=∈=1
0e |Pr
1 M
i ijij aABX
MP
{ }∑∑−
= ≠
=∈≤1
0,e |Pr
1 M
i ijiji aABX
MP
Overestimated error probability:
∑∑−
= ≠
≤
1
0 0
,e
2
1 M
i ij
ji
N
dQ
MP
Back to the one-dim case:
( )jiji ssdd ,, =Distances:
2010-11-19
Linköpings universitet 6
2010-11-19 TSKS01 Digital Communication - Lecture 12 21
The Nearest Neighbour Approximation
1 3
3
1
–1
φ0
φ1
2s
0s
1s
Interprete as the nearest signal.x
∑∑−
= ≠
≤
1
0 0
,e
2
1 M
i ij
ji
N
dQ
MP
We had the union bound:
0,11,0 dd =
1,22,1 dd =
0,22,0 dd =
∑ ∑−
= =
≈
1
0 : 0
mine
min,2
1 M
i ddj jiN
dQ
MP
Nearest neighbour approximation:
Dominated by the smallest distance.
jiji
dd ,min min≠
=
min,:# ddjn jii ==
∑−
=
=
1
0 0
min
2
1 M
ii
N
dQn
M
mind=
2010-11-19 TSKS01 Digital Communication - Lecture 12 22
On-Off Keying (OOK)
2010-11-19 TSKS01 Digital Communication - Lecture 12 23
Binary Phase-Shift Keying (BPSK)
2010-11-19 TSKS01 Digital Communication - Lecture 12 24
Binary Frequency-Shift Keying (BFSK)
2010-11-19
Linköpings universitet 7
2010-11-19 TSKS01 Digital Communication - Lecture 12 25
Amplitude-Shift Keying (ASK)
2010-11-19 TSKS01 Digital Communication - Lecture 12 26
8-PSK
2010-11-19 TSKS01 Digital Communication - Lecture 12 27
16-QAM (Quadrature Amplitude Modulation)
2010-11-19 TSKS01 Digital Communication - Lecture 12 28
Frequency-Shift Keying (FSK)
2010-11-19
Linköpings universitet 8
www.liu.se 2010-11-19 TSKS01 Digital Communication - Lecture 12 30