MPEG-2 Decoding in a Stream Programming Language · – Teleport Messaging (PPOPP 05) – Compiling...
Transcript of MPEG-2 Decoding in a Stream Programming Language · – Teleport Messaging (PPOPP 05) – Compiling...
![Page 1: MPEG-2 Decoding in a Stream Programming Language · – Teleport Messaging (PPOPP 05) – Compiling for Communication-Exposed Architectures (ASPLOS 02) – Load-Balanced Rendering](https://reader036.fdocuments.in/reader036/viewer/2022071004/5fc13301c7dd1d39e924a8d8/html5/thumbnails/1.jpg)
1
MPEG-2 Decoding in a Stream Programming Language
Matthew Drake, Hank Hoffmann, Rodric Rabbah andSaman Amarasinghe
Massachusetts Institute of Technology
IPDPSRhodes, April 2006
http://cag.csail.mit.edu/streamit
![Page 2: MPEG-2 Decoding in a Stream Programming Language · – Teleport Messaging (PPOPP 05) – Compiling for Communication-Exposed Architectures (ASPLOS 02) – Load-Balanced Rendering](https://reader036.fdocuments.in/reader036/viewer/2022071004/5fc13301c7dd1d39e924a8d8/html5/thumbnails/2.jpg)
2
• Graphics• Cryptography• Databases• Object recognition• Network processing
and security • Scientific codes• …
Stream Application Domain
![Page 3: MPEG-2 Decoding in a Stream Programming Language · – Teleport Messaging (PPOPP 05) – Compiling for Communication-Exposed Architectures (ASPLOS 02) – Load-Balanced Rendering](https://reader036.fdocuments.in/reader036/viewer/2022071004/5fc13301c7dd1d39e924a8d8/html5/thumbnails/3.jpg)
3
Parallel Programmer’s Dilemma
Rapid prototyping- MATLAB- Ptolemy
highParallel Performance
MalleabilityPortability
Productivity
high
low
Natural parallelization- StreamIt
Automatic parallelization- FORTRAN compilers- C/C++ compilers
Manual parallelization- C/C++ with MPI
Optimal parallelization- assembly code
![Page 4: MPEG-2 Decoding in a Stream Programming Language · – Teleport Messaging (PPOPP 05) – Compiling for Communication-Exposed Architectures (ASPLOS 02) – Load-Balanced Rendering](https://reader036.fdocuments.in/reader036/viewer/2022071004/5fc13301c7dd1d39e924a8d8/html5/thumbnails/4.jpg)
4
programmability
domain specificoptimizations
enable parallelexecution
target tiled architectures, clusters, DSPs, multicores,
graphics processors, …
simple and effective optimizations for domain
specific abstractions
boost productivity, enable faster development and
rapid prototyping
Compiler-Aware Language Design
![Page 5: MPEG-2 Decoding in a Stream Programming Language · – Teleport Messaging (PPOPP 05) – Compiling for Communication-Exposed Architectures (ASPLOS 02) – Load-Balanced Rendering](https://reader036.fdocuments.in/reader036/viewer/2022071004/5fc13301c7dd1d39e924a8d8/html5/thumbnails/5.jpg)
5
StreamIt Project• Language Semantics / Programmability
– StreamIt Language (CC 02) – Programming Environment in Eclipse (P-PHEC 05)
• Optimizations / Code Generation– Phased Scheduling (LCTES 03)– Cache Aware Optimization (LCTES 05)
• Domain Specific Optimizations– Linear Analysis and Optimization (PLDI 03)– Optimizations for bit streaming (PLDI 05)– Linear State Space Analysis (CASES 05)
• Parallelism– Teleport Messaging (PPOPP 05)– Compiling for Communication-Exposed
Architectures (ASPLOS 02)– Load-Balanced Rendering
(Graphics Hardware 05)• Applications
– SAR, DSP benchmarks, JPEG, – MPEG [IPDPS 06], DES and
Serpent [PLDI 05], …
StreamIt Program
Front-end
Stream-AwareOptimizations
Uniprocessorbackend
Clusterbackend
Rawbackend
IBM X10backend
C C per tile +msg code
StreamingX10 runtime
Annotated Java
MPI-likeC
![Page 6: MPEG-2 Decoding in a Stream Programming Language · – Teleport Messaging (PPOPP 05) – Compiling for Communication-Exposed Architectures (ASPLOS 02) – Load-Balanced Rendering](https://reader036.fdocuments.in/reader036/viewer/2022071004/5fc13301c7dd1d39e924a8d8/html5/thumbnails/6.jpg)
6
In This Talk• MPEG-2 Overview
• StreamIt Application Development : MPEG-2 Decoding
• Natural expression of – Program structure– Parallelism– Data distribution
• Emphasis on programmability– Comparison/Contrast with C
![Page 7: MPEG-2 Decoding in a Stream Programming Language · – Teleport Messaging (PPOPP 05) – Compiling for Communication-Exposed Architectures (ASPLOS 02) – Load-Balanced Rendering](https://reader036.fdocuments.in/reader036/viewer/2022071004/5fc13301c7dd1d39e924a8d8/html5/thumbnails/7.jpg)
7
Video Compression Algorithms
• Commonly used• Order of magnitude reduction in data
needed for representation• Decreases storage requirements• Internet and wireless transmission feasible
![Page 8: MPEG-2 Decoding in a Stream Programming Language · – Teleport Messaging (PPOPP 05) – Compiling for Communication-Exposed Architectures (ASPLOS 02) – Load-Balanced Rendering](https://reader036.fdocuments.in/reader036/viewer/2022071004/5fc13301c7dd1d39e924a8d8/html5/thumbnails/8.jpg)
8
MPEG-2 Overview
• Temporal compression eliminates redundancies between pictures
• Spatial compression eliminates data within a picture based on a human perception model
![Page 9: MPEG-2 Decoding in a Stream Programming Language · – Teleport Messaging (PPOPP 05) – Compiling for Communication-Exposed Architectures (ASPLOS 02) – Load-Balanced Rendering](https://reader036.fdocuments.in/reader036/viewer/2022071004/5fc13301c7dd1d39e924a8d8/html5/thumbnails/9.jpg)
9
MPEG-2 Temporal Compression
![Page 10: MPEG-2 Decoding in a Stream Programming Language · – Teleport Messaging (PPOPP 05) – Compiling for Communication-Exposed Architectures (ASPLOS 02) – Load-Balanced Rendering](https://reader036.fdocuments.in/reader036/viewer/2022071004/5fc13301c7dd1d39e924a8d8/html5/thumbnails/10.jpg)
10
Blocks
MPEG-2 Spatial Compression
![Page 11: MPEG-2 Decoding in a Stream Programming Language · – Teleport Messaging (PPOPP 05) – Compiling for Communication-Exposed Architectures (ASPLOS 02) – Load-Balanced Rendering](https://reader036.fdocuments.in/reader036/viewer/2022071004/5fc13301c7dd1d39e924a8d8/html5/thumbnails/11.jpg)
11
Color Channels
MPEG-2 Spatial Compression
![Page 12: MPEG-2 Decoding in a Stream Programming Language · – Teleport Messaging (PPOPP 05) – Compiling for Communication-Exposed Architectures (ASPLOS 02) – Load-Balanced Rendering](https://reader036.fdocuments.in/reader036/viewer/2022071004/5fc13301c7dd1d39e924a8d8/html5/thumbnails/12.jpg)
12
DCT
MPEG-2 Spatial Compression
2
507
-34
-3 3 141 2 -3 -227 9 -2 -1 -1-18 9 1 -1-1 5 2 1-1-2 2 1 -1-1 1 1-1 -2 1 -1
0
00 000
0
0 0 00 0 0000 0 00
000 0 000 00
horizontal frequency
vertical frequency
![Page 13: MPEG-2 Decoding in a Stream Programming Language · – Teleport Messaging (PPOPP 05) – Compiling for Communication-Exposed Architectures (ASPLOS 02) – Load-Balanced Rendering](https://reader036.fdocuments.in/reader036/viewer/2022071004/5fc13301c7dd1d39e924a8d8/html5/thumbnails/13.jpg)
13
Quantization
MPEG-2 Spatial Compression
2
507
-34
-3 3 141 2 -3 -227 9 -2 -1 -1-18 9 1 -1-1 5 2 1-1-2 2 1 -1-1 1 1-1 -2 1 -1
0
00 000
0
0 0 00 0 0000 0 00
000 0 000 00
horizontal frequency
vertical frequency
63 -17
-1 120 1 -111 32 3
1000
0 0 000 0 0 00
0 0 0 0 000 0 0 0 00
0 0 0 0 0 000 0 0 0 0 000 0 0 0 0 000 0 0 0 0 00
horizontal frequency
vertical frequency
![Page 14: MPEG-2 Decoding in a Stream Programming Language · – Teleport Messaging (PPOPP 05) – Compiling for Communication-Exposed Architectures (ASPLOS 02) – Load-Balanced Rendering](https://reader036.fdocuments.in/reader036/viewer/2022071004/5fc13301c7dd1d39e924a8d8/html5/thumbnails/14.jpg)
14
ZigZag
MPEG-2 Spatial Compression
63 -17
-1 120 1 -111 32 3
1000
0 0 000 0 0 00
0 0 0 0 000 0 0 0 00
0 0 0 0 0 000 0 0 0 0 000 0 0 0 0 000 0 0 0 0 00
horizontal frequency
vertical frequency
63 -17
-1 120 1 -111 32 3
1
horizontal frequency
vertical frequency
![Page 15: MPEG-2 Decoding in a Stream Programming Language · – Teleport Messaging (PPOPP 05) – Compiling for Communication-Exposed Architectures (ASPLOS 02) – Load-Balanced Rendering](https://reader036.fdocuments.in/reader036/viewer/2022071004/5fc13301c7dd1d39e924a8d8/html5/thumbnails/15.jpg)
15
Huffman Coded
MPEG-2 Spatial Compression
63 -17
-1 120 1 -111 32 3
1
horizontal frequency
vertical frequency
Output
![Page 16: MPEG-2 Decoding in a Stream Programming Language · – Teleport Messaging (PPOPP 05) – Compiling for Communication-Exposed Architectures (ASPLOS 02) – Load-Balanced Rendering](https://reader036.fdocuments.in/reader036/viewer/2022071004/5fc13301c7dd1d39e924a8d8/html5/thumbnails/16.jpg)
16
• Variable length decoding• Spatial decoding
– block decoding in parallel with motion vector decoding
• Temporal decoding– all color channels motion
compensated in parallel
• Color space conversion and data ordering
Stream Composition ofMPEG-2 Decoder
![Page 17: MPEG-2 Decoding in a Stream Programming Language · – Teleport Messaging (PPOPP 05) – Compiling for Communication-Exposed Architectures (ASPLOS 02) – Load-Balanced Rendering](https://reader036.fdocuments.in/reader036/viewer/2022071004/5fc13301c7dd1d39e924a8d8/html5/thumbnails/17.jpg)
17
Picture Reorder
joiner
joiner
IDCT
IQuantization
splitter
splitter
VLDmacroblocks, motion vectors
frequency encodedmacroblocks differentially coded
motion vectors
motion vectorsspatially encoded macroblocks
recovered picture
ZigZag
Saturation
Channel Upsample Channel Upsample
Motion Vector Decode
Y Cb Cr
quantization coefficients
picture type
<QC>
<QC>
reference picture
Motion Compensation
<PT1> referencepicture
Motion Compensation
<PT1>reference picture
Motion Compensation
<PT1>
<PT2>
Repeat
Color Space Conversion
<PT1, PT2>
add VLD(QC, PT1, PT2);
add splitjoin {
split roundrobin(N∗B, V);
add pipeline {add ZigZag(B);add IQuantization(B) to QC;add IDCT(B);add Saturation(B);
}add pipeline {
add MotionVectorDecode();add Repeat(V, N);
}
join roundrobin(B, V);}
add splitjoin {split roundrobin(4∗(B+V), B+V, B+V);
add MotionCompensation(4∗(B+V)) to PT1;for (int i = 0; i < 2; i++) {
add pipeline {add MotionCompensation(B+V) to PT1;add ChannelUpsample(B);
}}
join roundrobin(1, 1, 1);}
add PictureReorder(3∗W∗H) to PT2;
add ColorSpaceConversion(3∗W∗H);
MPEG bit stream
Application Design
• Structured block level diagram describes computation and flow of data
• Conceptually easy to understand– Clean abstraction of
functionality
![Page 18: MPEG-2 Decoding in a Stream Programming Language · – Teleport Messaging (PPOPP 05) – Compiling for Communication-Exposed Architectures (ASPLOS 02) – Load-Balanced Rendering](https://reader036.fdocuments.in/reader036/viewer/2022071004/5fc13301c7dd1d39e924a8d8/html5/thumbnails/18.jpg)
18
Picture Reorder
joiner
joiner
IDCT
IQuantization
splitter
splitter
VLDmacroblocks, motion vectors
frequency encodedmacroblocks differentially coded
motion vectors
motion vectorsspatially encoded macroblocks
recovered picture
ZigZag
Saturation
Channel Upsample Channel Upsample
Motion Vector Decode
Y Cb Cr
quantization coefficients
picture type
<QC>
<QC>
reference picture
Motion Compensation
<PT1> referencepicture
Motion Compensation
<PT1>reference picture
Motion Compensation
<PT1>
<PT2>
Repeat
Color Space Conversion
<PT1, PT2>
add VLD(QC, PT1, PT2);
add splitjoin {
split roundrobin(N∗B, V);
add pipeline {add ZigZag(B);add IQuantization(B) to QC;add IDCT(B);add Saturation(B);
}add pipeline {
add MotionVectorDecode();add Repeat(V, N);
}
join roundrobin(B, V);}
add splitjoin {split roundrobin(4∗(B+V), B+V, B+V);
add MotionCompensation(4∗(B+V)) to PT1;for (int i = 0; i < 2; i++) {
add pipeline {add MotionCompensation(B+V) to PT1;add ChannelUpsample(B);
}}
join roundrobin(1, 1, 1);}
add PictureReorder(3∗W∗H) to PT2;
add ColorSpaceConversion(3∗W∗H);
MPEG bit stream
StreamIt Philosophy
• Preserve program structure– Natural for application
developers to express
• Leverage program structure to discover parallelism and deliver high performance
• Programs remain clean– Portable and malleable
![Page 19: MPEG-2 Decoding in a Stream Programming Language · – Teleport Messaging (PPOPP 05) – Compiling for Communication-Exposed Architectures (ASPLOS 02) – Load-Balanced Rendering](https://reader036.fdocuments.in/reader036/viewer/2022071004/5fc13301c7dd1d39e924a8d8/html5/thumbnails/19.jpg)
19
StreamIt Philosophy
output to player
Picture Reorder
joiner
joiner
IDCT
IQuantization
splitter
splitter
VLDmacroblocks, motion vectors
frequency encodedmacroblocks differentially coded
motion vectors
motion vectorsspatially encoded macroblocks
recovered picture
ZigZag
Saturation
Channel Upsample Channel Upsample
Motion Vector Decode
Y Cb Cr
quantization coefficients
picture type
<QC>
<QC>
reference picture
Motion Compensation
<PT1> referencepicture
Motion Compensation
<PT1>reference picture
Motion Compensation
<PT1>
<PT2>
Repeat
Color Space Conversion
<PT1, PT2>
add VLD(QC, PT1, PT2);
add splitjoin {
split roundrobin(N∗B, V);
add pipeline {add ZigZag(B);add IQuantization(B) to QC;add IDCT(B);add Saturation(B);
}add pipeline {
add MotionVectorDecode();add Repeat(V, N);
}
join roundrobin(B, V);}
add splitjoin {split roundrobin(4∗(B+V), B+V, B+V);
add MotionCompensation(4∗(B+V)) to PT1;for (int i = 0; i < 2; i++) {
add pipeline {add MotionCompensation(B+V) to PT1;add ChannelUpsample(B);
}}
join roundrobin(1, 1, 1);}
add PictureReorder(3∗W∗H) to PT2;
add ColorSpaceConversion(3∗W∗H);
MPEG bit stream
![Page 20: MPEG-2 Decoding in a Stream Programming Language · – Teleport Messaging (PPOPP 05) – Compiling for Communication-Exposed Architectures (ASPLOS 02) – Load-Balanced Rendering](https://reader036.fdocuments.in/reader036/viewer/2022071004/5fc13301c7dd1d39e924a8d8/html5/thumbnails/20.jpg)
20
Picture Reorder
joiner
joiner
IDCT
IQuantization
splitter
splitter
VLD
ZigZag
Saturation
Channel Upsample Channel Upsample
Motion Vector Decode
<QC>
<QC>
reference picture
Motion Compensation
<PT1> referencepicture
Motion Compensation
<PT1>reference picture
Motion Compensation
<PT1>
<PT2>
Repeat
Color Space Conversion
<PT1, PT2>
add VLD(QC, PT1, PT2);
add splitjoin {
split roundrobin(N∗B, V);
add pipeline {add ZigZag(B);add IQuantization(B) to QC;add IDCT(B);add Saturation(B);
}add pipeline {
add MotionVectorDecode();add Repeat(V, N);
}
join roundrobin(B, V);}
add splitjoin {split roundrobin(4∗(B+V), B+V, B+V);
add MotionCompensation(4∗(B+V)) to PT1;for (int i = 0; i < 2; i++) {
add pipeline {add MotionCompensation(B+V) to PT1;add ChannelUpsample(B);
}}
join roundrobin(1, 1, 1);}
add PictureReorder(3∗W∗H) to PT2;
add ColorSpaceConversion(3∗W∗H);
MPEG bit stream
splitjoins
filters
pipelines
Stream Abstractions in StreamIt
![Page 21: MPEG-2 Decoding in a Stream Programming Language · – Teleport Messaging (PPOPP 05) – Compiling for Communication-Exposed Architectures (ASPLOS 02) – Load-Balanced Rendering](https://reader036.fdocuments.in/reader036/viewer/2022071004/5fc13301c7dd1d39e924a8d8/html5/thumbnails/21.jpg)
21
StreamIt Language Highlights
• Filters
• Pipelines
• Splitjoins
• Teleport messaging
![Page 22: MPEG-2 Decoding in a Stream Programming Language · – Teleport Messaging (PPOPP 05) – Compiling for Communication-Exposed Architectures (ASPLOS 02) – Load-Balanced Rendering](https://reader036.fdocuments.in/reader036/viewer/2022071004/5fc13301c7dd1d39e924a8d8/html5/thumbnails/22.jpg)
22
Example StreamIt Filter
float→float filter FIR (int N) {
work push 1 pop 1 peek N {float result = 0;for (int i = 0; i < N; i++) {
result += weights[i] ∗ peek(i);}push(result);pop();
}}
0 1 2 3 4 5 6 7 8 9 10 11 input
output
FIR0 1
![Page 23: MPEG-2 Decoding in a Stream Programming Language · – Teleport Messaging (PPOPP 05) – Compiling for Communication-Exposed Architectures (ASPLOS 02) – Load-Balanced Rendering](https://reader036.fdocuments.in/reader036/viewer/2022071004/5fc13301c7dd1d39e924a8d8/html5/thumbnails/23.jpg)
23
Compiler Managed Buffers
• Sliding window computation is very common in multimedia and scientific codes
• There are various implementation strategies for managing peek buffers
• Compiler recognizes peek buffers and chooses best implementation strategy for an architecture
Circular Buffer: Copy-Shift:0 1 2 3 4 5 6 7
N
8 9 0 1
N
2 3 4 5 6 78 9
![Page 24: MPEG-2 Decoding in a Stream Programming Language · – Teleport Messaging (PPOPP 05) – Compiling for Communication-Exposed Architectures (ASPLOS 02) – Load-Balanced Rendering](https://reader036.fdocuments.in/reader036/viewer/2022071004/5fc13301c7dd1d39e924a8d8/html5/thumbnails/24.jpg)
24
FIR Filter in C
void FIR(int* src, int* dest, int* srcIndex, int* destIndex, int srcBufferSize, int destBufferSize,int N) {
float result = 0.0;for (int i = 0; i < N; i++)result += weights[i] * src[(*srcIndex + i) % srcBufferSize];dest[*destIndex] = result;*srcIndex = (*srcIndex + 1) % srcBufferSize;*destIndex = (*destIndex + 1) % destBufferSize;
}
• FIR functionality obscured by buffer management details
• Programmer must commit to a particular buffer implementation strategy
![Page 25: MPEG-2 Decoding in a Stream Programming Language · – Teleport Messaging (PPOPP 05) – Compiling for Communication-Exposed Architectures (ASPLOS 02) – Load-Balanced Rendering](https://reader036.fdocuments.in/reader036/viewer/2022071004/5fc13301c7dd1d39e924a8d8/html5/thumbnails/25.jpg)
25
StreamIt Language Highlights
• Filters
• Pipelines
• Splitjoins
• Teleport messaging
![Page 26: MPEG-2 Decoding in a Stream Programming Language · – Teleport Messaging (PPOPP 05) – Compiling for Communication-Exposed Architectures (ASPLOS 02) – Load-Balanced Rendering](https://reader036.fdocuments.in/reader036/viewer/2022071004/5fc13301c7dd1d39e924a8d8/html5/thumbnails/26.jpg)
26
Example StreamIt Pipeline• Pipeline
– Connect components in sequence
– Expose pipeline parallelism Column_iDCTs
Row_iDCTs
float→float pipeline 2D_iDCT (int N) {
add Column_iDCTs(N);add Row_iDCTs(N);
}
![Page 27: MPEG-2 Decoding in a Stream Programming Language · – Teleport Messaging (PPOPP 05) – Compiling for Communication-Exposed Architectures (ASPLOS 02) – Load-Balanced Rendering](https://reader036.fdocuments.in/reader036/viewer/2022071004/5fc13301c7dd1d39e924a8d8/html5/thumbnails/27.jpg)
27
From Figures 7-1 and 7-4 of the MPEG-2 Specification (ISO 13818-2, P. 61, 66)
Preserving Program Structureint->int pipeline BlockDecode(
portal<InverseQuantisation> quantiserData, portal<MacroblockType> macroblockType) {
add ZigZagUnordering();add InverseQuantization() to quantiserData, macroblockType;add Saturation(-2048, 2047);add MismatchControl();add 2D_iDCT(8); add Saturation(-256, 255);
}
Can be reused for JPEG decoding
exchange of control-relevant information
quantiserData,
quantiserData,
![Page 28: MPEG-2 Decoding in a Stream Programming Language · – Teleport Messaging (PPOPP 05) – Compiling for Communication-Exposed Architectures (ASPLOS 02) – Load-Balanced Rendering](https://reader036.fdocuments.in/reader036/viewer/2022071004/5fc13301c7dd1d39e924a8d8/html5/thumbnails/28.jpg)
28
Decode_Picture {for (;;) {
parser()for (;;) {
decode_macroblock();motion_compensation();if (condition)
then break;}
}frame_reorder();
}
decode_macroblock() {parser();motion_vectors();for (comp=0;comp<block_count;comp++) {
parser();Decode_MPEG2_Block();
}}
motion_vectors() {parser();decode_motion_vectorparser();
}
Decode_MPEG2_Block() {for (int i = 0;; i++) {
parsing();ZigZagUnordering();inverseQuantization();if (condition) then
break;}
}
motion_compensation() {for (channel=0;channel<3;channel++)
form_component_prediction();for (comp=0;comp<block_count;comp++) {
Saturate();IDCT();Add_Block();
}}
In Contrast: C Code Excerpt
• Explicit for-loops iterate through picture frames• Frames passed through global arrays, handled with pointers• Mixing of parser, motion compensation, and spatial decoding
EXTERN unsigned char *backward_reference_frame[3];EXTERN unsigned char *forward_reference_frame[3];EXTERN unsigned char *current_frame[3];...etc...
![Page 29: MPEG-2 Decoding in a Stream Programming Language · – Teleport Messaging (PPOPP 05) – Compiling for Communication-Exposed Architectures (ASPLOS 02) – Load-Balanced Rendering](https://reader036.fdocuments.in/reader036/viewer/2022071004/5fc13301c7dd1d39e924a8d8/html5/thumbnails/29.jpg)
29
StreamIt Language Highlights
• Filters
• Pipelines
• Splitjoins
• Teleport messaging
![Page 30: MPEG-2 Decoding in a Stream Programming Language · – Teleport Messaging (PPOPP 05) – Compiling for Communication-Exposed Architectures (ASPLOS 02) – Load-Balanced Rendering](https://reader036.fdocuments.in/reader036/viewer/2022071004/5fc13301c7dd1d39e924a8d8/html5/thumbnails/30.jpg)
30
Example StreamIt Splitjoin• Splitjoin
– Connect components in parallel
– Expose data parallelism and data distribution
joiner
splitterfloat→float splitjoin Row_iDCT (int N){
split roundrobin(N);for (int i = 0; i < N; i++) {
add 1D_iDCT(N);}join roundrobin(N);
}
![Page 31: MPEG-2 Decoding in a Stream Programming Language · – Teleport Messaging (PPOPP 05) – Compiling for Communication-Exposed Architectures (ASPLOS 02) – Load-Balanced Rendering](https://reader036.fdocuments.in/reader036/viewer/2022071004/5fc13301c7dd1d39e924a8d8/html5/thumbnails/31.jpg)
31
joiner
iDCT
splitter
iDCTiDCT iDCT
joiner
iDCT
splitter
iDCTiDCT iDCT
• Splitjoin– Connect components
in parallel– Expose data
parallelism and data distribution
Example StreamIt Splitjoin
float→float splitjoin Row_iDCT (int N){
split roundrobin(N);for (int i = 0; i < N; i++) {
add 1D_iDCT(N);}join roundrobin(N);
}joiner
splitter
joiner
splitter
![Page 32: MPEG-2 Decoding in a Stream Programming Language · – Teleport Messaging (PPOPP 05) – Compiling for Communication-Exposed Architectures (ASPLOS 02) – Load-Balanced Rendering](https://reader036.fdocuments.in/reader036/viewer/2022071004/5fc13301c7dd1d39e924a8d8/html5/thumbnails/32.jpg)
32
StreamIt Parallel Performance
0.25
2.82
1.00
0
0.5
1
1.5
2
2.5
3
C Reference,1 Tile
StreamIt,1 Tile
StreamIt,16 Tiles
2D Discrete Cosine Transform on MIT Raw Architecture
Spee
dup
![Page 33: MPEG-2 Decoding in a Stream Programming Language · – Teleport Messaging (PPOPP 05) – Compiling for Communication-Exposed Architectures (ASPLOS 02) – Load-Balanced Rendering](https://reader036.fdocuments.in/reader036/viewer/2022071004/5fc13301c7dd1d39e924a8d8/html5/thumbnails/33.jpg)
33
add splitjoin {split roundrobin(4*(B+V), B+V, B+V);
add MotionCompensation();for (int i = 0; i < 2; i++) {
add pipeline {add MotionCompensation();add ChannelUpsample(B);
}}
join roundrobin(1, 1, 1);}
joiner 1:1:1
splitter 4:1:1
recovered picture
Channel Upsample
Channel Upsample
Cr
MotionCompensation
MotionCompensation
Motion Compensation
Y Cb
Naturally Expose Data Distribution
Y Cb Cr
1 2
3 4 5 6
scatter macroblocks according to chroma format
gather one pixel at a time
![Page 34: MPEG-2 Decoding in a Stream Programming Language · – Teleport Messaging (PPOPP 05) – Compiling for Communication-Exposed Architectures (ASPLOS 02) – Load-Balanced Rendering](https://reader036.fdocuments.in/reader036/viewer/2022071004/5fc13301c7dd1d39e924a8d8/html5/thumbnails/34.jpg)
34
Stream Graph Malleability
joiner
splitter 4,1,1
Upsample Upsample
MCMCMC
Y Cb Cr
1 2
3 4 5 6
Y Cb Cr
1 2
3 4
5
87
6
4:2:2 chroma format
4:2:0 chroma format
joiner
splitter 1,1
Upsample Upsample
MCMCMC
splitter 4,(2+2)
joiner
![Page 35: MPEG-2 Decoding in a Stream Programming Language · – Teleport Messaging (PPOPP 05) – Compiling for Communication-Exposed Architectures (ASPLOS 02) – Load-Balanced Rendering](https://reader036.fdocuments.in/reader036/viewer/2022071004/5fc13301c7dd1d39e924a8d8/html5/thumbnails/35.jpg)
35
StreamIt Code Sample
// C = blocks per chroma channel per macroblock// C = 1 for 4:2:0, C = 2 for 4:2:2add splitjoin {
split roundrobin(4*(B+V), 2*C*(B+V));
add MotionCompensation();add splitjoin {
split roundrobin(B+V, B+V);
for (int i = 0; i < 2; i++) {add pipeline {
add MotionCompensation()add ChannelUpsample(C,B);
}}
join roundrobin(1, 1);}
join roundrobin(1, 1, 1);}
blue = code added or modified to support 4:2:2 format
joiner
splitter 4,1,1
Upsample Upsample
MCMCMC
joiner
splitter 1,1
Upsample Upsample
MCMCMC
splitter 4,(2+2)
joiner
![Page 36: MPEG-2 Decoding in a Stream Programming Language · – Teleport Messaging (PPOPP 05) – Compiling for Communication-Exposed Architectures (ASPLOS 02) – Load-Balanced Rendering](https://reader036.fdocuments.in/reader036/viewer/2022071004/5fc13301c7dd1d39e924a8d8/html5/thumbnails/36.jpg)
36
In Contrast: C Code Excerpt
/* Y */form_component_prediction(src[0]+(sfield?lx2>>1:0),dst[0]+(dfield?lx2>>1:0),
lx,lx2,w,h,x,y,dx,dy,average_flag);
if (chroma_format!=CHROMA444) {lx>>=1; lx2>>=1; w>>=1; x>>=1; dx/=2;
}if (chroma_format==CHROMA420) {
h>>=1; y>>=1; dy/=2;}
/* Cb */form_component_prediction(src[1]+(sfield?lx2>>1:0),dst[1]+(dfield?lx2>>1:0),
lx,lx2,w,h,x,y,dx,dy,average_flag);
/* Cr */form_component_prediction(src[2]+(sfield?lx2>>1:0),dst[2]+(dfield?lx2>>1:0),
lx,lx2,w,h,x,y,dx,dy,average_flag);
Adjust values used for address calculations depending on the chroma format used.
blue = pointers used for address calculations
![Page 37: MPEG-2 Decoding in a Stream Programming Language · – Teleport Messaging (PPOPP 05) – Compiling for Communication-Exposed Architectures (ASPLOS 02) – Load-Balanced Rendering](https://reader036.fdocuments.in/reader036/viewer/2022071004/5fc13301c7dd1d39e924a8d8/html5/thumbnails/37.jpg)
37
StreamIt Language Highlights
• Filters
• Pipelines
• Splitjoins
• Teleport messaging
![Page 38: MPEG-2 Decoding in a Stream Programming Language · – Teleport Messaging (PPOPP 05) – Compiling for Communication-Exposed Architectures (ASPLOS 02) – Load-Balanced Rendering](https://reader036.fdocuments.in/reader036/viewer/2022071004/5fc13301c7dd1d39e924a8d8/html5/thumbnails/38.jpg)
38
Teleport Messaging
• Avoids muddling data streams with control relevant information
• Localized interactions in large applications– A scalable alternative to
global variables or excessive parameter passing
Order
IQ
VLD
MCMCMC
![Page 39: MPEG-2 Decoding in a Stream Programming Language · – Teleport Messaging (PPOPP 05) – Compiling for Communication-Exposed Architectures (ASPLOS 02) – Load-Balanced Rendering](https://reader036.fdocuments.in/reader036/viewer/2022071004/5fc13301c7dd1d39e924a8d8/html5/thumbnails/39.jpg)
39
Motion Prediction and Messaging
portal<MotionCompensation> PT;
add splitjoin {split roundrobin(4*(B+V), B+V, B+V);
add MotionCompensation() to PT;for (int i = 0; i < 2; i++) {
add pipeline {add MotionCompensation() to PT;add ChannelUpsample(B);
}}
join roundrobin(1, 1, 1);}
joiner
splitter
recovered picture
Channel Upsample
Channel Upsample
Cr
MotionCompensation
MotionCompensation
Motion Compensation
Y Cb
picture type
![Page 40: MPEG-2 Decoding in a Stream Programming Language · – Teleport Messaging (PPOPP 05) – Compiling for Communication-Exposed Architectures (ASPLOS 02) – Load-Balanced Rendering](https://reader036.fdocuments.in/reader036/viewer/2022071004/5fc13301c7dd1d39e924a8d8/html5/thumbnails/40.jpg)
40
• Looks like method call, but timed relative to data in the stream
• Simple and precise for user– Exposes dependences to compiler– Adjustable latency– Can send upstream or downstream
void setPicturetype(int p) {reconfigure(p);
}
TargetFilter x;if newPictureType(p) {
x.setPictureType(p) @ 0;}
Teleport Messaging Overview
![Page 41: MPEG-2 Decoding in a Stream Programming Language · – Teleport Messaging (PPOPP 05) – Compiling for Communication-Exposed Architectures (ASPLOS 02) – Load-Balanced Rendering](https://reader036.fdocuments.in/reader036/viewer/2022071004/5fc13301c7dd1d39e924a8d8/html5/thumbnails/41.jpg)
41
Frame Reordering
DecodePicture
Decode Macroblock
Decode Motion Vectors
Motion Compensation
Decode Block
Motion Compensation For Single Channel
Saturate
IDCT
ZigZagUnorderingInverse
Quantization
File Parsing
Global Variable Space
The MPEG Bitstream
Output Video
Messaging Equivalent in C
![Page 42: MPEG-2 Decoding in a Stream Programming Language · – Teleport Messaging (PPOPP 05) – Compiling for Communication-Exposed Architectures (ASPLOS 02) – Load-Balanced Rendering](https://reader036.fdocuments.in/reader036/viewer/2022071004/5fc13301c7dd1d39e924a8d8/html5/thumbnails/42.jpg)
42
Language Comparison: Programmer’s Perspective
C StreamItCorrectness and Performance
Mixed together Separation of concerns
Buffer management
Programmer managed
Compiler managed
Scheduling Programmer managed
Compiler managed
![Page 43: MPEG-2 Decoding in a Stream Programming Language · – Teleport Messaging (PPOPP 05) – Compiling for Communication-Exposed Architectures (ASPLOS 02) – Load-Balanced Rendering](https://reader036.fdocuments.in/reader036/viewer/2022071004/5fc13301c7dd1d39e924a8d8/html5/thumbnails/43.jpg)
43
C StreamItMemory Model Global address
spaceDistributed (private) address spaces
Parallelism Implicit Explicit
Communication Obscured Exposed
Transformations Limited Global
Language Comparison: Compiler’s Perspective
![Page 44: MPEG-2 Decoding in a Stream Programming Language · – Teleport Messaging (PPOPP 05) – Compiling for Communication-Exposed Architectures (ASPLOS 02) – Load-Balanced Rendering](https://reader036.fdocuments.in/reader036/viewer/2022071004/5fc13301c7dd1d39e924a8d8/html5/thumbnails/44.jpg)
44
Implementation
• Functional MPEG-2 decoder – Encoder recently completed
• Developed by 1 programmer in 8 weeks
• 2257 lines of code – Vs. 3477 lines of C code in MPEG-2 reference
• 48 static streams, 643 instantiated filters
![Page 45: MPEG-2 Decoding in a Stream Programming Language · – Teleport Messaging (PPOPP 05) – Compiling for Communication-Exposed Architectures (ASPLOS 02) – Load-Balanced Rendering](https://reader036.fdocuments.in/reader036/viewer/2022071004/5fc13301c7dd1d39e924a8d8/html5/thumbnails/45.jpg)
45
Related Work• Synchronous Dataflow and Extensions
– Synchronous Piggybacked Dataflow– C. Park, J. Chung, S. Ha 1999– C. Park, J. Jung, S. Ha 2002
– Blocked Dataflow– D.-I. Ko, S. S. Bhattacharyya 2005
– Hierarchical Dataflow– S. Neuendorffer, E. Lee 2004
• Implementations– MPEG2 Decoding and Encoding
– E. Iwata, K. Olukotun 1998– Parallel MPEG4 Encoding
– I. Assayad, P. Gerner, S. Yovine, V. Bertin 2005• Stream Oriented Languages
– Esterel, Lustre, Signal, Lucid, Cg, Brook, Spindle, StreamC, Occam, Parallel Haskell, Sisal
![Page 46: MPEG-2 Decoding in a Stream Programming Language · – Teleport Messaging (PPOPP 05) – Compiling for Communication-Exposed Architectures (ASPLOS 02) – Load-Balanced Rendering](https://reader036.fdocuments.in/reader036/viewer/2022071004/5fc13301c7dd1d39e924a8d8/html5/thumbnails/46.jpg)
46
Ongoing and Future Work
• MPEG-2 performance evaluation
• Inter-language interfaces– StreamIt to native C, and vice versa
• More applicationswe want to hear from you!
![Page 47: MPEG-2 Decoding in a Stream Programming Language · – Teleport Messaging (PPOPP 05) – Compiling for Communication-Exposed Architectures (ASPLOS 02) – Load-Balanced Rendering](https://reader036.fdocuments.in/reader036/viewer/2022071004/5fc13301c7dd1d39e924a8d8/html5/thumbnails/47.jpg)
47
Conclusions• StreamIt language preserves program
structure– Natural for programmers
• Parallelism and communication naturally exposed– Compiler managed buffers, and portable
parallelization technology
• StreamIt increases programmer productivity, doesn’t sacrifice performance
![Page 48: MPEG-2 Decoding in a Stream Programming Language · – Teleport Messaging (PPOPP 05) – Compiling for Communication-Exposed Architectures (ASPLOS 02) – Load-Balanced Rendering](https://reader036.fdocuments.in/reader036/viewer/2022071004/5fc13301c7dd1d39e924a8d8/html5/thumbnails/48.jpg)
48
The End
http://cag.csail.mit.edu/streamit