Post on 06-Jun-2020
Low-Power MPEG-4
HE-AAC Version-2 Encoder
Chi-Min LiuHan-Wen Hsu
Chung-Han YangWen-Chieh Lee
http://psplab.csie.nctu.edu.tw
Outline
Introduction
Overview of CEMFB and CMFB
Parameter Extraction in CMFB
Down-Mixing by Phase Adjustment
Quality Measurement
Introduction
HE-AAC v.2 SBR+PS+AAC.
“Low-Power” Encoder A Fast Computation algorithm by simplifying the
computation from complex domain to real domainwith a limited quality degradation.
SBR
AACPS
Introduction
SBR — Bandwidth extension
Original Spectrum Decoded AAC LF Spectrum
HF Generation in SBR HF Adjustment in SBR
Introduction
PS coding — Stereo extension
Introduction
HE-AAC Encoder
SynthesisCEMFB
AACEncoder
SBREncoder
StereoParameterExtractor
HybridAnalysis
Filterbank
Down-Mixing
AACSBRPSBit-Stream Formatter
AnalysisCEMFB
L
RM
Introduction
Modules involved in “Low Power” Encoders Complex Exponential Filterbanks.
Data Calculation in Complex Domain
SynthesisCEMFB
AACEncoder
SBREncoder
StereoParameterExtractor
HybridAnalysis
Filterbank
Down-Mixing
AACSBRPSBit-Stream Formatter
AnalysisCEMFB
L
RM
Introduction
Addressed Problem Our previous papers on Low Power SBR in AES 122
This paper on PS.
SynthesisCEMFB
AACEncoder
SBREncoder
StereoParameterExtractor
HybridAnalysis
Filterbank
Down-Mixing
AACSBRPSBit-Stream Formatter
AnalysisCEMFB
L
RM
Outline
Introduction
Overview of CEMFB and CMFB
Parameter Extraction in CMFB
Down-Mixing by Phase Adjustment
Quality Measurement
CMFB (Cosine Modulated Filter Bank)
CEMFB (Complex-Exponential Modulated Filter Bank)
0Q1Q 2Q 3Q*
0Q*
1Q*
2Q*
3Q
04-channel CMFB
0Q1Q 2Q 3Q
04-channel CEMFB
Overview of CEMFB and CMFB
Overview of CEMFB and CMFB
Why is CEMFB adopted?CEMFB CMFB
X 2 X 2X 1 X 1
Aliasing Free Aliasing Suffering
Overview of CEMFB and CMFB
Aliasing in CMFB
4-channel CMFB0
0Y0
0Y
0
1Y0
1Y
0
2Y0
2Y
0
3Y0
3Y
Freq.-π/2M π/2M -π π0
)(zHk
][nx
1
0Y3
0Y1
0Y 2
0Y2
0Y 3
0Y
2
1Y1
1Y3
1Y 1
1Y3
1Y 2
1Y
2
2Y3
2Y1
2Y 2
2Y 3
2Y1
2Y
3
3Y1
3Y2
3Y2
3Y 3
3Y1
3Y
Freq.-π/2M π/2M -π π0
M M )(zFk
][ˆ nxk
0
1Y0
1Y
0
0Y0
0Y
0
2Y0
2Y
0
3Y0
3Y
Aliasing Eliminationby a Mutually
“Perfect” Cancellation
Overview of CEMFB and CMFB
Aliasing in CEMFB
0
3Y
Freq.-π/2M π/2M -π π0
0
0Y
0
1Y
0
2Y
)(zHk
][nx
4-channel CEMFB
M M
2
3Y 3
3Y 0
3Y1
3Y
Freq.-π/2M π/2M -π π0
1
0Y3
0Y0
0Y2
0Y
3
1Y 0
1Y 1
1Y2
1Y
0
2Y2
2Y3
2Y1
2Y
)(zFk
][ˆ nxk
0
3Y
0
0Y
0
1Y
0
2Y
Aliasing Elimination by near -zero overlap
Outline
Introduction
Overview of CEMFB and CMFB
Parameter Extraction in CMFB Signal Modeling
IID (Inter-channel Intensity Difference)
ICC (Inter-Channel Coherence)
Down-Mixing by Phase Adjustment
Quality Measurement
-20 -15 -10 -5 0 5 10 15 20-0.8
-0.6
-0.4
-0.2
0
0.2
0.4
0.6
0.8
Parameter Extraction in CEMFB —
Analytic Modeling for CEMFB Subband Signals
Hilbert transform
0 0
0 )2/(sin2
][
2
n
nn
nnh
, ][][][ˆ
k
knxkhnx
j
j)( jeH
90-degree phase shifter
-π π
ω
2
Parameter Extraction in CEMFB —
Analytic Modeling for CEMFB Subband Signals
CMFB subband
CEMFB subband
][ˆ][][ nxjnxnx
-π π
ω
1
1
1
)( jejH
Image part is cancelled.
Analytic signal
otherwise
eXeX
j
j
,0
0,2
real
image][nx
Parameter Extraction in CEMFB —
Cross-Correlation Function
Cross-correlation
Signal energy
ICC (Inter-Channel Coherence)
)()()(][][)( *
,
*
,
jjj
yx
k
yx eYeXeRnkykxnr
deYeXr jj
yx )()()0( *
21
,
deXeXr jj
xx )()(2
1)0( *
,
Energy of real signal (CMFB)
Energy of analytic signal (CEMFB)
Energy relation
Parameter Extraction in CEMFB —
Energy Relation of Real and Analytic Signals
0
*
24
, )()()0( deXeXrE jj
xxx
xx EE 2
0
**
, )()(2
2)()(
2
1)0( deXeXdeXeXrE jjjj
xxx
Parameter Extraction in CEMFB —
Cross-Correlation Relation of Real and Analytic Signals
ICC for real signals (CMFB)
0
22
21
*
21
,,
)()()(
)()()0(
deRdeIjeR
deYeXr
jjj
jj
yxyx
-π π
ω
1
)()(* jj eYeX
The image term of X*Y are cancelled mutually.
Parameter Extraction in CEMFB —
Cross-Correlation Relation of Real and Analytic Signals
ICC for analytic signals (CEMFB)
00
24
0
*
24
,,
)()(
)()()0(
deIjdeR
deYeXr
jj
jj
yxyx
-π π
ω
4
)()(4 * jj eYeX
The image term of X*Y are kept.
Parameter Extraction in CEMFB —
Cross-Correlation Relation of Real and Analytic Signals
ICC for real and analytic signals
ICC relations
)0()()()0( ,21
0
*
22
,, yx
jj
yxyx rdeYeXr
0
24
,, )(2 deIj j
yxyx
yxyx ,, 2
Parameter Extraction in CEMFB —
Extraction ofIID and ICC Parameters
)()(2
)(2log10
)(
)(log10)( ,1010,
biidbe
be
be
bebiid rl
r
l
r
l
rl
h
l
k
kk
L
n
Ll nnklbe
1
0
2
21,)(
h
l
k
kk
L
n
Lr nnkrbe
1
0
2
21,)(
xx EE 2
IID (Inter-channel Intensity Difference) approximation
)()( ,,biidbiid rlrl
Parameter Extraction in CEMFB —
Extraction ofIID and ICC Parameters
ICC approximation
,1,1,)(1
0
2
*
2,
h
l
k
kk
L
n
LLrl nnkrnnklbe
)()(
)(
)(2)(2
)(2
)()(
)()(
,,,
, bebe
be
bebe
be
bebe
bebicc
rl
rl
rl
rl
rl
rl
rl
xx EE 2 yxyx ,, 2
)()(
)()(
,
, bebe
bebicc
rl
rl
rl
Down-Mixing by Phase AdjustmentCEMFB —
Summary
Relations between the real and analytic signals
Approximation of IID and ICC
xx EE 2
yxyx ,, 2
)()(
)()(
,
, bebe
bebicc
rl
rl
rl
)()( ,,biidbiid rlrl
Only one channel needs CEMFB.
Outline
Introduction
Overview of CEMFB and CMFB
Parameter Extraction in CMFB
Down-Mixing by Phase Adjustment
Quality Measurement
Down-Mixing by Phase Adjustment
Adjustment Phase Extraction
CEMFB Subband
CMFB Subband
Down-mixing Subband
ie21
][][][21 nyenxnd i
1
0
2
21
)2,0[][][maxarg
N
n
i nyenx
1
0
* ][][N
n
nynx
1
0
*
1
0
*
][][
][][
N
n
N
nj
nynx
nynx
e
KLT-like Criterion
Down-Mixing by Phase Adjustment—
Comparison for sc03.wav
with Phase Adjustment
without Phase Adjustment
ODG: -2.86
ODG: -3.06
Down-Mixing by Phase Adjustment—
Tone Modulation Artifact
Down-Mixing by Phase Adjustment—
Smooth Method
Cosine smooth connection of coefficient vector
FrameiFramei+1
k0
i
1i
time index
n
FrameiFramei+1
k0
i
1i
time index
n 2
cos2
11
iiii
k
nn
Down-Mixing by Phase Adjustment—
Artifact Reduction by Smooth Method
Outline
Introduction
Overview of CEMFB and CMFB
Parameter Extraction in CEMFB
Down-Mixing by Phase Adjustment
Quality Measurement
Complex HE-AAC V.2
Complex HE-AAC V.2 + KLT Down-Mix
LP HE-AAC V.2 Real HE-AAC V.2 LP HE-AAC V.2 + KLT-like Down-Mix
es01 -2.21 -2.18 -2.25 -2.24 -2.29
es02 -2.71 -2.73 -2.7 -2.7 -2.8
es03 -2.82 -2.84 -2.79 -2.79 -2.85
sc01 -3.3 -3.18 -3.32 -3.33 -3.24
sc02 -3 -3.05 -3 -3.02 -3.09
sc03 -2.62 -2.65 -2.56 -2.74 -2.57
si01 -2.75 -3.02 -2.72 -2.77 -2.95
si02 -2.63 -2.42 -2.68 -2.74 -2.46
si03 -1.99 -2.19 -2.03 -2.02 -2.31
sm01 -2.89 -3.05 -2.86 -2.96 -3.06
sm02 -3.15 -3.05 -3.13 -3.23 -3.13
sm03 -2.52 -2.55 -2.52 -2.56 -2.51
AVG -2.7158 -2.7425 -2.7133 -2.7583 -2.7717
ODG Comparison @ 36kbps
Real HE-AAC v.2 ODG: -3.23
LP HE-AAC v.2ODG: -3.13
AVG ODG Comparison @ 24k, 36k and 48k bps
AVG ODG
-3.5
-3.3
-3.1
-2.9
-2.7
-2.5
-2.3
-2.1
48k 36k 24k
OD
G
Complex Low Power Real
Remarks
Low Power Merits Computation, Data Movement, Memory.
SynthesisCEMFB
AACEncoder
SBREncoder
StereoParameterExtractor
HybridAnalysis
Filterbank
Down-Mixing
AACSBRPSBit-Stream Formatter
AnalysisCEMFB
L
RM
Computational Complexity
Filter Bank 32 (=2048/64) 64-pt DCTs-IV are saved for per
frame.
PS and SBR reduce about 50% cost for IID, ICC, energy, and tonality computation.
Complex LP LP + KLT-Like Adustment
128-64 Analysis FB
2*264-pt DCT-IV
2+1 64-pt DCT-IV
2+1 64-pt DCT-IV
64-32 Downsampled Synthesis FB
1 64-pt DCT-IV
1 32-pt DCT-IV
1 64-pt DCT-IV
Q & Ahttp://psplab.csie.nctu.edu.tw
Computational Complexity
Filter Bank
Complex: 5 64-pt DCTs.
LP: 3 64-pt DCTs, 1 32-pt DCTs.
LP + KLT-like: 4 64-pt DCTs.
PS and SBR reduces about 50% cost for IID, ICC, energy, and tonality computation.
Adjustment Phase Extraction
CEMFB Subband
CMFB Subband
Down-mixing Subband
ie21
For AAC
Adjustment Phase Extraction
CEMFB Subband
CMFB Subband
Down-mixing Subband
ie21
21
Real Part
For SBR
備註
48k 36k 24k
Com LP All Real com LP All Real com LP All Real
es01 -1.36 -1.47 -1.45 -2.21 -2.25 -2.24 -3.17 -3.26 -3.3
es02 -1.43 -1.45 -1.45 -2.71 -2.7 -2.7 -3.47 -3.46 -3.49
es03 -1.61 -1.61 -1.61 -2.82 -2.79 -2.79 -3.69 -3.68 -3.69
sc01 -3.24 -3.26 -3.28 -3.3 -3.32 -3.33 -3.39 -3.41 -3.44
sc02 -2.88 -2.87 -2.91 -3 -3 -3.02 -3.24 -3.21 -3.23
sc03 -2.29 -2.25 -2.52 -2.62 -2.56 -2.74 -2.96 -2.96 -2.99
si01 -2.59 -2.55 -2.59 -2.75 -2.72 -2.77 -3 -2.99 -3.08
si02 -2.37 -2.41 -2.52 -2.63 -2.68 -2.74 -3.31 -3.29 -3.27
si03 -1.63 -1.74 -1.72 -1.99 -2.03 -2.02 -2.85 -3 -3.03
sm01 -2.66 -2.64 -2.76 -2.89 -2.86 -2.96 -3.42 -3.46 -3.52
sm02 -2.91 -2.81 -2.9 -3.15 -3.13 -3.23 -3.52 -3.49 -3.52
sm03 -2.4 -2.4 -2.46 -2.52 -2.52 -2.56 -2.84 -2.88 -2.91
average -2.2808 -2.2883 -2.3475 -2.7158 -2.7133 -2.7583 -3.2383 -3.2575 -3.2892