8/6/2019 Parallel Processing (1)
1/54
7/17/20
X L SONG SONGPARALLEL PROCESSING
THNG TIN V HC PHN
1. Thng tin chung:
1.1. Tn hc phn: X l song song (Parallel Processing)
1.2. M hc phn: KH.KM.515
1.3. S tn ch: 2 TC
1.4. Loi hc phn: Bt buc
1.5. Cc hc phn tin quyt: C s d liu phn tn
THNG TIN V HC PHN2. Mc tiu ca hc phn:2.1. Kin thc: cung cp cho ngi hc cc kin thcv my tnh song song, cch xy dng cc thut tonsong song.
2.2. K nng: s dng cng c lp trnh song song nhMPI, JAVA, VPM ... ngi hc phi ci t c mt sthut ton song song c bn.
2.3. Thi hc tp: ngi hc phi tham d y
cc gi l thuyt v tho lun.
THNG TIN V HC PHN3. Chnh sch i vi hc phnTham gia hc tp trn lp: i hc y , tch cc tholun ni dung bi ging, tham gia cha bi tp v chunb bi v tt - 20%.
Kh nng t hc, t nghin cu: hon thnh bi tiulun (assignment) theo tng c nhn. Bi kim tra nhgi gia k - 20%.
Kt qu thi cui k - 60%.
THNG TIN V HC PHN3. Phn b s tit:
L thuyt: 20
Tiu lun, c thm: 8
Tho lun: 2
NI DUNG CHNG TRNHPHN 1: TNH TON SONG SONG
Chng 1 KIN TRC V CC LOI MY TINH SONG SONG
Chng 2 CC THNH PHN CA MY TINH SONG SONG
Chng 3 GII THIU V LP TRNH SONG SONG
Chng 4 CC M HNH LP TRNH SONG SONG
Chng 5 THUT TON SONG SONG
PHN 2: X L SONG SONG CC C S D LIU
(c thm)
Chng 6 TNG QUAN V C S D LIU SONG SONG
Chng 7 TI U HA TRUY VN SONG SONG
Chng 8 LP LCH TI U CHO CU TRUY VN SONG SONG
http://../CSDL%20phan%20tan/Slide/Chuong%203.ppthttp://../CSDL%20phan%20tan/Slide/Chuong%202.ppthttp://../CSDL%20phan%20tan/Slide/Chuong%201.ppt8/6/2019 Parallel Processing (1)
2/54
7/17/20
TI LIU THAM KHO
[0] on vn Ban, Nguyn Mu Hn,X l song song v phn tn,NXB KH&KT, 2006.
[1] Ananth Grama, Anshui Guptal George Karipis, Vipin Kumar,
Introduction to Parallel Computing, Pearson, 2003
[2] Barry Wilkingson, Michael Allen, Parallel Programming,Technigues and Applications Using Networked Workstations and
Parallel Computers, Prentice Hall New Jersey, 1999
[3] M. Sasikumar, Dinesh Shikhare, P. Ravi Prakash, Introduction to
Parallel Processing, Prentice - Hall, 2000
[4] Seyed H. Roosta, Parallel Processing and Parallel Algorithms,
Theory and Computation, Springer 1999.
[5] Michael J. Quinn, Parallel Computing Theory and Practice,
MaGraw-Hill,1994
[6] Shaharuddin Salleh, Albert Y. Zomaya, Scheduling in Parallel
Computing Systems, Kluwer Academic Publisher, 1999.
TI LIU THAM KHO
[7] Clement T.Yu, Weiyi Meng, Principles of Database Query
Processing for Advanced Applications, Morgan Kaufman Inc.,
1998. 185-225.
[8] Hasan Waqar, Optimization of SQL Query for ParallelMachines, Springer, 1995.
[9] Hong W., Parallel Query Processing Using Shared Memory
Multiprocessors and Disk Arrays, Univesity of California, 1992.
[10] Hua, K.A., Parallel Database Technology, University of
Central Florida Orlande FL 32846-2362, 1997.
[11] Zomaya A. Y. and Shaharuddin Salleh, Scheduling in
parallel computing systems, Kluwer Academic Publishers, 1999.
A CH LIN H
TS. NGUYN MU HNKHOA CNG NGH THNG TINTRNG I HC KHOA HC - I HC HU77, NGUYN HU HU
IN THOI:CQ: 054 382 6767D: 01255213579
EMAIL: [email protected]
back10
PHN 1:
TNH TON SONG SONG
Nguyn Mu HnKhoa CNTT-HKH HU
[email protected] | [email protected]
11
CHNG 1. KIN TRC CC LOI MY TNH SONG SONG
NI DUNG
1.1 Gii thiu chung
1.2 Kin trc my tnh kiu Von Neumann
1.3 Phn loi my tnh song song
1.4 Kin trc my tnh song song
12
1.1 Gii thiu chung
X l song song (XLSS) l g?
Trong x l tun t:
Bi ton c tch thnh mt chui cc cu lnh ri rcCc cu lnh c thc hin mt cch tun t
Ti mi thi im ch thc hin c mt cu lnh
mailto:[email protected]:[email protected]:[email protected]:[email protected]:[email protected]:[email protected]8/6/2019 Parallel Processing (1)
3/54
7/17/20
13
1.1 Gii thiu chung (tt)
1 CPU
n gin
Chm qu !!!
14
1.1 Gii thiu chung (tt)
Trong x l song song
Bi ton c tch thnh nhiu phn v c th thc hin
ng thi.
Mi phn c tch thnh cc lnh ri rc
Mi lnh c thc hin t nhng CPU khc nhau
15
1.1 Gii thiu chung (tt)
Nhiu CPU
Phc tp hn
Nhanh hn !!!
16
1.1 Gii thiu chung (tt)
XLSS l mt qu trnh x l gm nhiu tin trnhc kch hot ng thi v cng tham gia gii
quyt mt vn trn h thng c nhiu b x l.
Vy x l song song l g?
17
1.1 Gii thiu chung (tt)
Ti sao phi x l song song?
Yu cu ca ngi s dng: Cn thc hin mt khi lng ln cng vic Thi gian x l phi nhanh
Yu cu thc t: Trong thc t khng tn ti my tnh c b nh v hnv kh nng tnh ton v hn.Trong thc t c nhiu bi ton m my tnh x l tun t(XLTT) kiu von Neumann khng p ng c.S dng h thng nhiu BXL thc hin nhng tnhton nhanh hn nhng h n BXL. Gii quyt c nhng bi ton ln hn, phc tp hn
18
1.1 Gii thiu chung (tt)
S khc nhau c bn gia XLSS v XLTT:
X l tun t X l song song
Mi thi im ch thc hincmtphp ton
Mi thi im c th thchin cnhiuphp ton
Thi gian thc hin phpton chm
Thi gian thc hin phpton nhanh
8/6/2019 Parallel Processing (1)
4/54
7/17/20
19
1.1 Gii thiu chung (tt)
i tng no s dng my tnh song song?
20
1.1 Gii thiu chung (tt)
Tnh kh thi ca vic XLSS?
Tc x l ca cc BXL theo kiu Von Neumann b gii
hn, khng th ci tin thm c.
Gi thnh ca phn cng (CPU) gim, to iu kin xy
dng nhng h thng c nhiu BXL vi gi c hp l.
S pht trin cng ngh mch tch hp cao VLSI (very large
scale integration) cho php to ra nhng h phc hp c
hng triu transistor trn mt chip.
21
1.1 Gii thiu chung (tt)
Nhng thnh phn lin quan n vn XLSS:
Kin trc my tnh song song
Phn mm h thng (h iu hnh),
Thut ton song song
Ngn ng lp trnh song song, v.v.
nh ngha my tnh song song (MTSS):
MTSS l mt tp cc BXL (thng l cng mt loi) kt
ni vi nhau theo mt kiu no c th hp tc vi nhau
cng hot ng v trao i d liu vi nhau.
22
1.1 Gii thiu chung (tt)
Tiu ch nh gi mt thut ton song song
i vi thut ton tun tthi gian thc hin thut ton.
khng gian b nh.
kh nng lp trnh.
i vi thut ton song songcc tiu chun nh thut ton tun t.
nhng tham s v s BXL: s BXL, tc x l.
kh nng ca cc b nh cc b.
s truyn thng.
thao tc I/O.
23
1.2 Kin trc my tnh kiu Von Neumann
John von Neumann (1903 1957): nh ton hc ngi Hungary
S dng khi nim lu tr chng trnh (stored-program concept)
Von Neumann computer c cc c im sau:
B nh c dng lu tr chng trnh v d liu
Chng trnh c m ho (code) my tnh c th hiu c
D liu l nhng thng tin n gin c s dng bi chng trnh
CPU np (fetch) nhng lnh v d liu t b nh, gii m (decode) v thc
hin tun t chng. 24
1.2 Kin trc my tnh kiu Von Neumann
My tnh kiu V.Neumann c xy dng t cc khi c s:
B nh: lu tr d liu
Cc n v logic v s hcALU: thc hin cc php tonCc phn t x l: iu khin CU v truyn d liu I/O
ng truyn d liu: BUS
B nh
B x l
Ghi d liu c d liuCu lnh
http://en.wikipedia.org/wiki/1903http://en.wikipedia.org/wiki/1957http://en.wikipedia.org/wiki/1957http://en.wikipedia.org/wiki/1957http://en.wikipedia.org/wiki/19038/6/2019 Parallel Processing (1)
5/54
7/17/20
25
1.3 Phn loi my tnh song song
Tiu ch phn loi my tnh song song?
a) Da trn lnh, dng d liu v cu trc b nh
(Flynn)
b) Da trn kin trc: (xem 1.4)
Pipelined Computers
Dataflow Architectures
Data Parallel Systems
Multiprocessors
Multicomputers26
1.3 Phn loi my tnh song song
Michael Flynn (1966)
SISD: Single Instruction Stream, Single Data Stream
n lung lnh, n lung d liu
SIMD: Single Instruction Stream, Multiple Data Stream
n lung lnh, a lung d liu
MISD: Multiple Instruction Stream, Single Data Stream
a lung lnh, n lung d liu
MIMD: Multiple Instruction Stream, Multiple Data Stream
a lung lnh, a lung d liu
27
1.3 Phn loi my tnh song song
M hnh SISD -n lung lnh, n lung d liu
c im
Ch c mt CPU
mi thi im ch thc hin mt lnh v ch c/ghi
mt mc d liu
C mt thanh ghi, gi l b m chng trnh (program
counter), c s dng np a ch ca lnh tip theo
khi x l tun t
Cc cu lnh c thc hin theo mt th t xc nh
y chnh l m hnh my tnh truyn thng kiu Von Neumann
28
1.3 Phn loi my tnh song song
n v iukhin
B nh
BXL s hc
Lung lnhLungd liu
Lungkt qu
Tn hiu iukhin
M hnh SISD -n lung lnh, n lung d liu (tt)
V d minh ha
29
1.3 Phn loi my tnh song song
M hnh SIMD -n lung lnh, a lung d liu C mt n v iu khin (CU) iu khin nhiu n v
x l (PE) CU pht sinh tn hiu iu khin n cc n v x l
n lung lnh: cc n v x l thc hin cng mt lnh
trn cc mc d liu khc nhau
a lung d liu: mi n v x l c lung d liu ring
y l kiu tnh ton lp li cc n v s hc trong CPU,
cho php nhng n v khc nhau thc hin trn nhng
ton hng khc nhau, nhng thc hin cng mt lnh.
My tnh SIMD c th h tr x l kiu vector, trong c
th gn cc phn t ca vector cho cc phn t x l
tnh ton ng thi. 30
1.3 Phn loi my tnh song song
M hnh ca kin trc SIMD vi b nh phn tn
M hnh SIMD -n lung lnh, a lung d liu (tt)
CU
PU1 LM1
PUn LMn
DS
DS
DS
DS
ISIS
Program loaded
from host
Data sets
loaded from host
IS: Instruction Stream PU : Processing Unit
LM : Local Memory DS : Data Stream
8/6/2019 Parallel Processing (1)
6/54
7/17/20
31
1.3 Phn loi my tnh song song
Cc my tnh trn th trng c sn xut theo m hnh
SIMD: ILLIAC IV, DAPv Connection Machine CM-2
M hnh SIMD -n lung lnh, a lung d liu (tt)
32
1.3 Phn loi my tnh song song
M hnh MISD -a lung lnh, n lung d liu
c im: a lung lnh: c th thc hin nhiu lnh trn cng mt mc
d liu
n lung d liu: cc PU x l trn cng mt lung d liu
Kin trc kiu ny c th chia thnh hai nhm:Cc my tnh yu cu mi n v x l (PU) nhn nhng
lnh khc nhau thc hin trn cng mt mc d liu.Cc my tnh c cc lung d liu c chuyn tun t
theo dy cc CPU lin tip-gi l kin trc hnh ng-x ltheo vector thng qua mt dy cc bc, trong mibc thc hin mt chc nng v sau chuyn kt qucho PU thc hin bc tip theo.
33
1.3 Phn loi my tnh song song
M hnh MISD a lung lnh, n lung d liu (tt)
Memory:
(Program,
Data) PU1 PU2 PUn
CU1 CU2 CUn
DS DS DSIS IS
ISIS
IS
DS
I/O
MISD architecture (the systolic array)
IS: Instruction Stream PU : Processing Unit CU : Control Unit
LM : Local Memory DS : Data Stream
34
1.3 Phn loi my tnh song song
M hnh MISD a lung lnh, n lung d liu (tt)
V d minh ha
35
1.3 Phn loi my tnh song song
M hnh MIMD -a lung lnh, a lung d liu
Mi BXL c th thc hin nhng lung lnh (chng trnh)
khc nhau trn cc lung d liu ring.
Hu ht cc h thng MIMD u c b nh ring v cng c
th truy cp vo c b nh chung (global) khi cn, do vy
gim thiu c s trao i gia cc BXL trong h thng.
Nhn xt:y l kin trc phc tp nht, nhng n l m hnh h trxl song song cao nhtCc my tnh c sn xut theo kin trc ny:BBN Butterfly, Alliant FX, iSPCca Intel
36
1.3 Phn loi my tnh song song
M hnh MIMD a lung lnh, a lung d liu (tt)
CU1 PU1
Shared
Memory
IS
IS DSI/O
CUn PUnIS DS
I/O
IS
MIMD architecture with shared memory
8/6/2019 Parallel Processing (1)
7/54
7/17/20
37
1.3 Phn loi my tnh song song
M hnh MIMD a lung lnh, a lung d liu (tt)
Nhn xt:MIMD l kin t rc phc tp nht, nhng n l m hnh h tr x l songsong cao nhtCc my tnh c snxut theo kin trc ny:BBN Butterfly, Alliant FX, iSPCca Intel 38
1.4 Kin trc my tnh song song
Mt vi nhn xt: Theo Flynn: c hai h kin trc quan trng cho cc my tnh
song song: SIMD v MIMD. Nhng kin trc khc c thxp theo hai mu .
Mc tiu ca x l song song l khaithc n mc ti acc kh nng s dng ca cc thit b phn cngnhm giiquyt nhanh nhng bi ton t ra trong thc t.
Kin trc phn cng l trong sut i vi ngi lp trnh Trong kin trc tun t c th tn dng tc cc nhanh
ca BXL thc hin x l song song theo nguyn l chias thi gian v chia s ti nguyn.
Nhng chng trnh song song trn my n BXL c ththc hin c nu c HH cho php nhiu tin trnh cngthc hin, ngha l c th xem h thng nh l a b x l.
39
1.4 Kin trc my tnh song song
Song song ha trong my tnh tun t:a. a n v chc nng: Cc my tnh truyn thng ch c mt n v s hc v
logic (ALU) trong BXL. mi thi im n ch c th thchin mt chc nng.
My tnh song songc nhiu n v x l(PE). Nhng nvnyc th cngnhauthc hin song song.V d:myCDC 6600(1964) c 10 PE c t chc trongmt BXL. Nhng n v chc nng ny c lp vi nhau vc th thc hin ng thi.
Xy dngb lp lch ti u phn chia cc cu lnh thc
hin sao cho tn dng c ti a cc n v x l cngnh cc ti nguyn ca my tnh.
40
1.4 Kin trc my tnh song song
Song song ha trong my tnh tun t (tt):
b. X l theo nguyn l hnh ng trong CPU
Cu lnh c chia thnh cc giai on (stage-phase)
Ti mt thi im c th c nhiu lnh c ti vo v cthc hin trong nhng bc khc nhau
Cc giai on thc hin khc nhau ca mi cu lnh c ththc hin gi u nhau.
u ra ca giai on ny c th l u vo ca giai ontip theo
Thc hin theo nguyn l hnh ng s hiu qu hn vkhng cn vng m d liu.
41
1.4 Kin trc my tnh song song
Song song ha trong my tnh tun t(tt):b. X l theo nguyn l hnh ng trong CPU V d: Pha 1: np cu lnh v t b nh (Instruction Fetch) Pha 2: gii m (Instruction decode) Pha 3: xc nh cc ton hng (Operand Fetch) Pha 4: thc hin cu lnh (Instruction Execute) Pha 5: lu tr kt qu (Write-Back)... qu trnh ny c th phn cho mi PE thc hin mt
cng vic. Theo cch , ti mt thi im BXL c ththc hin c nhiu cu lnh gi u nhau. Trc khimt cu lnh kt thc thc hin th cu lnh tip theo c th thc hin pha gii m, cu lnh khc li c thc np v, v.v. 42
1.4 Kin trc my tnh song song
IF: Instruction Fetch
ID: Instruction decode
OF: Operand Fetch
IE: Instruction ExecuteWB: Write-Back
Instruction i IF ID OF IE
IF ID OF IE
IF ID OF IE
IF ID OF IE
IF ID OF IE
Instruction i+1
Instruction i+2
Instruction i+3
Instruction i+4
Instruction # 1 2 3 4 5 6 7 8
WB
WB
WB
WB
WB
Cycles
8/6/2019 Parallel Processing (1)
8/54
7/17/20
43
1.4 Kin trc my tnh song song
V d: Thc hin tun t v hnh ng ca 2 tin trnh gm 4 giai on
Pha 1 Pha 2 Pha 3 Pha 4
Pha 1 Pha 2 Pha 3 Pha 4 Pha 1 Pha 2 Pha 3 Pha 4
Gi s mt tin trnh c chia thnh 4 giai on:
Thc hin tun t 2 tin trnh phi qua 8 giai on:
Thc hin theo hnh ng ch cn tri qua 5 giai on:
Pha 1 Pha 2 Pha 3 Pha 4
Pha 1 Pha 2 Pha 3 Pha 4
Tng thi gian tnh ton tun t l: 2 * (S1 + S2 + S3 + S4)Tng thi gian tnh ton hnh ng l: S1 + S2 + S3 + S4 + S4 44
1.4 Kin trc my tnh song song
Song song ha trong my tnh tun t(tt):
c. S gi u CPU v cc thao tc vo/ra (I/O)
Cc php vo/ra c th thc hin ng thi i vi
nhiu nhim v tnh ton khc nhau bng cch sdng nhng b iu khin vo/ra, cc knh hay
nhng BXL vo/ra khc nhau.
Nhiu my tnh hin nay c nhiu b iu khin
thit b vo/ra, cho php a x l vo/ra lm tng
c tc trao i d liu gia cc thit b ngoi
vi CPU.
45
1.4 Kin trc my tnh song song
d. Cc h thng b nhphn cp
Do tc thc hin ccphp ton trong BXL nhanhhn rt nhiu vic c dliu vo b nh trong
Cc thanh ghi c s dngtrc tip cho ALU nn bnh cache c xem nhvng m gia BXL v bnh chnh
Khi d liu c chuyn tb nh cache vo b nh
chnh th ng thi c thchuyn d liu t cache vocho CPU x l
CPU(Registers)
Cache
Main Memory
Fixed Disks
Magnetic
Tapes
Tng kh
nng lutr
Tng v tc truy cp
46
1.4 Kin trc my tnh song song
Song song ha trong my tnh tun t(tt):e. a chng trnh v chia s thi gian Thc hin song song da vo h iu hnh a nhim,
phn mm a lung, a tin trnh. H iu hnh a nhim thng gii quyt cc trng hp:
Trong cng mt khong thi gian, c nhiu tin trnhcng truy cp vo d liu t thit b vo/ra chung
Mt tin trnh tnh ton vi cng cao c th tmthi chim dng CPU lm vic, trong khi mt tin
trnh khc trc khng i hi phi kt thc cng vicsm phi ngng li.
B lp lch chia s thi gian lm nhim v phn chia
CPU cho mi tin trnh mt khong thi gian c nh To cc BXL o: mi tin trnh c cung cp mt mi
trng c xem nh mt BXL thc hin ring chotin trnh .
47
1.4 Kin trc my tnh song song
M hnh tru tng ca my tnh song song
Mc ch: mun th hin c nhng kh nng tnh ton caMTSS m khng quan tm n nhng rng buc c th ca
nhng my tnh c trong thc t.
Ch : khi xy dng cc thut ton song song, chng ta qui
c l pht trin thut ton cho m hnh tru tng ny,
sau nh x sang nhng my tnh c th vi mt s cc
rng buc no .
48
1.4 Kin trc my tnh song song
a. My tnh truy cp ngu nhin song song PRAM
Cha mt n v iu khin CU
Mt b nh chung
Mt tp khng gii hn cc BXL
Mi BXL li c b nh ring v c mt ch s duy nht
c s dng xc nh a ch trong qu trnh trao
i cc tn hiu v qun l cc ngt.
Tt c cc BXL u chia s b nh chung vi yu cu
khng b gii hn.Cc cu lnh c th bt u thc hin bt k thi im
no, bt k v tr no ca b nh (ring hoc chung)
8/6/2019 Parallel Processing (1)
9/54
7/17/20
49
1.4 Kin trc my tnh song song
a. My tnh truy cp ngu nhin song song PRAM
y cng l m hnh tng qut cho my tnh song song kiu MIMD
Private memory
P1
Private memory
P2
Private memory
Pn
Interconnection network
Global memory
CU
50
1.4 Kin trc my tnh song song
Mt s iu cn lu khi pht trin nhng thut toncho cc MTSS tng qut
Khng b gii hn v s lng BXL
Mi v tr ca b nh u truy cp c bi bt k BXL
no
Khng gii hn v dung lng b nh chung chia s
trong h thng
Cc BXL c th c bt k mt v tr no ca b nh,
ngha l khng cn phi ch nhng BXL khc kt
thc cng vic truy cp vo b nh.
51
1.4 Kin trc my tnh song song
Mt s iu cn lu khi chuyn nhng thut ton xydng cho MTSS tng qut sang my tnh c th Phi p dng mt s cc rng buc m bo chng
trnh thc hin c trn nhng my tnh . V hnh thc, phi thc hin mt trong nhng iu kin
sau: EREW: loi tr vn xung t c/ghi
(Exclusive Read + Exclusive Write)
CREW: cho php c ng thi, nhng khng chophp xung t khi ghi(Concurrent Read + Exclusive Write)
CRCW: Cho php c, ghi ng thi(Concurrent Read + Concurrent Write)
52
1.4 Kin trc my tnh song song
b. Kin trc SIMD
Cu trc:
Cc phn t x l (PE) u c iu hnh bi mt
n v iu khin (CU).
Cc phn t x l nhn c cng mt lnh t CU
nhng hot ng trn nhng tp d liu khc nhau.
c tnh:
Phn tn vic x l trn nhiu phn cng
Thao tc ng thi trn nhiu phn t d liu
Thc hin cng mt tnh ton trn tt c cc phn t d
liu.
53
1.4 Kin trc my tnh song song
b. Kin trc SIMD (tt)
CU
PE2
IS
PEn
PE1
.
.
.
Global memory
Result
IS
DSn
DS2
DS1
M hnh kin trckiu SIMD
IS: Instruction Stream PE : Processing Element
LM : Local Memory DS : Data Stream
54
1.4 Kin trc my tnh song song
V d
X1
X2
X4
X3X= No
Yes
X1
X3
X4
X2
Mt s PE kim tra X= ,mt s khc ri
Mt s PE kim tra X ,mt s khc ri
Tt c PE
(a) thc hin theo SISD,
(b) thc hin theo SIMDTrong , X1, X2, X3, v X4 l cc
khi cc cu lnh
8/6/2019 Parallel Processing (1)
10/54
7/17/20
55
1.4 Kin trc my tnh song song
b. Kin trc MISD
BXL hnh ng chnh l BXL kiu MISD
Nguyn l hnh ng(pipeline):da trn nguyn tc:
Phn on hoc chia nh mt tin trnh thnh mt s tin
con thc hin trong cc pha lin tip. Mi mt giai on ca mt tin trnh c thc hin tun t.
Sau khi thc hin xong mt pha th bt u thc hin giai
on ca tin trnh tip theo.
Mi pha thc hin xong s truyn kt qu cho pha tip theo.
Tm li, theo nguyn l hnh ng:
Khi mt giai on cng vic ang thc hin th mt giai
on khc c th np d liu vo, v d liu vo ca giai
onny cth l kt qu ca giai on trc n.56
1.4 Kin trc my tnh song song
V d: Thc hin tun t v hnh ng ca 2 tin trnh gm 4 giai on
Pha 1 Pha 2 Pha 3 Pha 4
Pha 1 Pha 2 Pha 3 Pha 4 Pha 1 Pha 2 Pha 3 Pha 4
Gi s mt tin trnh c chia thnh 4 giai on:
Thc hin tun t 2 tin trnh phi qua 8 giai on:
Thc hin theo hnh ng ch cn tri qua 5 giai on:
Pha 1 Pha 2 Pha 3 Pha 4
Pha 1 Pha 2 Pha 3 Pha 4
Tng thi gian tnh ton tun t l: 2 * (S1 + S2 + S3+ S4)Tng thi gian tnh ton hnh ng l: S1 + S2 + S3 + S4 + S4
57
1.4 Kin trc my tnh song song
Nguyn l hnh ng c th p dng theo hai mc:- Hnh ng theo n v s hc:
Cc n v s hc v logic ALU c t chc thnh mng,cc php ton bn trong c thc hin theo nguyn lhnh ng.
CUALU
ALU. . .
ALU
B nh
X l hnh ng theo ALU58
1.4 Kin trc my tnh song song
- Hnh ng theo n v cu lnh:Cc n v iu khin CU c phn on v t chc theohnh ng.
CU
. . .CU
CU
ALU
B nh
X l hnh ng theo CU
59
1.4 Kin trc my tnh song song
Xy dng hnh ng vng trn gia cc BXL, b nh v mnglin kt
Php ton thc hin bi CU theo kin trc ny c th chia thnh 5 giai on:G1: c d liu: c d liu t b nh chia s (Shared Memory).
G2:Chuyn ti d liu: chuyn d liu t b nh ti cc phn t x l PEthng qua mng c (Read Network).G3. Thc hin cu lnh: s dng PE thc hin cc cu lnh.G4. Chuyn ti d liu: chuyn cc kt qu t cc PE ti b nh
thng qua mng ghi (Write Network).G5. Lu tr d liu : ghi li cc kt qu vo b nh chia s.
SharedMemory
SharedMemory
WriteNetwork
ReadNetwork
V d v mt hnh ng vng trn
60
1.4 Kin trc my tnh song song
Mt v d n gin
Tnh tng n phn t ca mt mng a:
For (i=0; i
8/6/2019 Parallel Processing (1)
11/54
7/17/20
61
1.4 Kin trc my tnh song song
Tnh tng n phn t ca mt mng a:
sin = sout + a[i]
sin sout
a[0] a[1] a[2] a[3] a[4]
Thc hin theo ng ng
sin sout sin sout sin soutin sout
62
1.4 Kin trc my tnh song song
V d: Sp xp n phn t bng k thut ng ng
Recv(&number, Pi-1);
If (number > x) {send(&x,Pi-1);
x= number;
} else send(number, Pi+1);
tng thut ton (insertion sort): P0 nhn mt dy cc s
Lu tr s ln nht Chuyn cc s nh hn i Nu s nhn c ln hn s lu tr hin thi th thay thn v chuyn s nh hn i
63
1.4 Kin trc my tnh song song
4,3,1,2,5
5
5
5 2
5 3 1
5 4 2
5 4 3 1
5 4 3 2
5 4 3 2 1
P0 P1 P2 P3 P4
4,3,1,2
4,3,12
4,31
14 5 2
3
4 2
3 1
2
1
64
1.4 Kin trc my tnh song song
c. B x l mng tm thu SAP (Systolic Array Processor)
Do Kung vLeiserson xut 1978
L mt mng cc phn t x l d liu (DPU) c kt ni
cc b vi nhau.
Mi DPU c xem nh mt t bo, mt trong mng, bao
gm:
Mt s thanh ghi (registers)
Mt b cng (adder)
Cc mch iu khin
n v logic-s hc ALU.
65
1.4 Kin trc my tnh song song
c. B x l mng tm thu SAP (cont.)
SystolicArray Controller
HostProcessor
Tn hiu
D liu vo
Kt qu
Kin trc b x l mng tm thu
Trong kin trc SAP,B iu khin (Controller) lm nhim v giao din cho BXLchnh (Host Processor) v gi cc tn hiu iu khin qutrnh vo/ra d liu cho SA.Hot ng ca h thng theo tng nhp v lp li mt cchu n tn dng c kh nng song song ca tt ccc phn t x l 66
1.4 Kin trc my tnh song song
c. B x l mng tm thu SAP (tt)
DPU:DataprocessingUnits
http://hyperlink/Leiserson.pptxhttp://hyperlink/Leiserson.pptxhttp://hyperlink/Leiserson.pptxhttp://hyperlink/Leiserson.pptxhttp://hyperlink/Leiserson.pptx8/6/2019 Parallel Processing (1)
12/54
7/17/20
67
1.4 Kin trc my tnh song song
c. B x l mng tm thu SAP (cont.)
SA c th t chc theo nhiu cu hnh tp khc nhau
(a) mng tuyn tnh
(b) mng hnh tam gic
(c) mng hai chiu hnh vung68
1.4 Kin trc my tnh song song
c. B x l mng tm thu SAP (tt)
Xt bi ton nhn 2 ma trn c 2x2: AxB=C
S dng b nh SAP hai chiu hnh vung tnh
22221221222122112121
22121211122112111111
2221
1211
2221
1211
2221
1211
****
****
*
*
babacbabac
babacbabac
bac
cc
cc
bb
bb
aa
aa
k
kjikij
69
1.4 Kin trc my tnh song song
4 5 6
7 8 9
1 2 3
a11 a12
a21
a22
b22
b21b12
b11
c11
c12
c21
c22
Nhp theo ct
Nhp theo hng
Nhp 1:Nhp a11, b11 vo s 1 v tnh a11* b11Nhp b21 vo s 4 v a12vo s 2
Nhp 2:Truyn b11 t s 1 sang s 2Truyn a11 t 1 sang 4 v tnh a11* b21Truyn b11 t 1 sang 2 v tnh a12* b11Truyn b12 t 4 sang 5 v a12 t 2 sang 5 v tnh a12* b21 . Tnh c11=a12*b11+a12*b21Nhn tip b12vo 4 v a21 vo s 2
Nhp 3:Truyn c11 t s 5sang s 9Truyn a21 *b11 t 2 sang 6
Truyn b12 t 5 sang 6Nhp a22 vo 3 v nhp b22 vo 7
Nhp 4:Truyn a22 t s 3 sang s 6
Tnh a22* b21Truyn a21 *b11 t 2 sang 6Cng dn kt qu c cuyn t 2:S cho c21=a21 *b11 +a22* b21Chuyn c11 t 9 ra v gn cho c11
70
1.4 Kin trc my tnh song song
d. Kin trc MIMD Loi a BXL hoc cn gi l h thng a my tnh Trong mi BXL c n v iu khin (CU) ring v thc
hin chng trnh ring ca mnh.
Nhng c trng ca MIMD: X l phn tn trn mt s BXL c lp Chia s vi nhau mt s ti nguyn, trong c h thng
b nh. Mi BXL thao tc c lp v c th thc hin ng thi vi
nhau. Mi BXL c th thc hin mt chng trnh ring.
71
1.4 Kin trc my tnh song song
CU 1
PE2
IS1
PEn
PE1
.
.
.
CU 2
CU n
.
.
.
DS
IS2
ISn
Kin trc MIMD72
Cu hi chng 1
1. Nu c trng c bn ca kin trc my tnh tun t ca VN2. Cc kin trc my tnh c th c phn loi nh th no?da vo nhng yu t no phn loi?
3. Mt h thng nh th no c gi l my tnh song song?4. My tnh kiu MIMD khc vi mng cc my tnh nh thno?5. Nu nguyn l x l theo hnh ng. Nhng bi ton c nhngtnh cht g th thch hp vi kin trc x l hnh ng?6. Cn bao nhiu nhp thc hin nhn hai ma trn 100 100trn SAP c 100100 phn t x l? Nu s dng h10001000 PE th h no tt hn (nhanh hn)?7. Mt cng vic c chia thnh m cng vic con, mi cngvic con i hi mt n v thi gian thc hin. Hi cn baonhiu n v thi gian h hnh ng gm m-b x l tuyntnh (gm m-pha thc hin) thc hin c nhim v chotrc?
8/6/2019 Parallel Processing (1)
13/54
7/17/20
73
CHNG 2. CC THNH PHN CA MT SONG SONG
NI DUNG
2.1 B nh ca MTSS
2.2 Mng kt ni cc thnh phn ca MTSS
2.3 Chng trnh dch v h iu hnh
2.4 Kt lun
74
2.1 B nh ca MTSS
Mt s vn cn quan tm trong kin trc MTSS
S dng nhiuthanh ghi:
s lm gim hiu ng ph ca cc ngt
S dng khng gian nh ln: lm gim hiu ng ph ca s i ch khi thc hin mt
lnh. tng hiu qu trao i d liu ca h thng.
Xy dng b lp lch (Scheduling): nhm cp pht hu hiu
tng nhim v n l cho cc BXL cho mt cch ng .
ng b cc BXL (Synchronization):iu khin nhiu tin trnh
hot ng ng thi, cng truy cp n mt s hu hn cc ti
nguyn dng chung, trnh c s tc nghn (deadlock)
75
2.1 B nh ca MTSS
Mt s vn cn quan tm trong kin trc MTSS (cont.)Thit k cu hnh mng:tp trung vo vic kt ni gia BXLvi BXL, gia BXL vi b nh trong h thng. Cu hnh tpca mng kt ni l vn rt quan trng trong thit k hthng song song.
Phn on( Partitioning): xc nh mc song song trongcc thut ton xc nh c cc lung x l ng thi.S phn on c th th hin nhiu mc khc nhau: mclnh, mc th tc, hoc mc chng trnh, v.v.
m bo tin cy (Reliability):nu c mt BXL no khngthc hin c cng vic m nhim th mt BXL khc sc thay th m bo trong mi tnh hung cng vicchung vn c hon thnh.
76
2.1 B nh ca MTSS
B nh ca MTSS c chia thnh nhiu mcB nh mc 1: mc cao nht, c dung lng nh nht, truycp nhanh v t nht, thng gn cht vi mi BXL thnhb nh cc b (local memory-khc vi b nh chia s).B nh mc 2: Truy cp chm hn v r hn mc 1, v.v.V nguyn tc, d liu c chuyn i gia cc mc lncn ca cc b nh v hon ton c iu khin bi b nhmc 1.V l thuyt, trong cu trc phn cp b nh, tc truy cpb nh trung bnh gn bng tc truy cp mc cao nht(mc 1), nhng chi ph ca cc n v nh trung bnh li gnvi gi ca b nh mc thp nht (mc n).
77
BXL
B nh mc 1
B nh mc n
B nh mc 2
Mc thp nht
Mc cao nht Nh, nhanh v t
Ln, chm v r
2.1 B nh ca MTSS
Phn cp h thng b nh
78
2.1B nh ca MTSS
2.1.1 B nh kt hp (AM Associative Memory)AM bao gm cc nh (cell)
sq
k
a
mR/W
Select
Key
Argument
Read/Write
Output
Match
Mi nh ca AM c 4 u vo v hai u raCc u vo (input) ca mi nh bao gm: . Bit i s a,
. Bit c/ghi R/Wxc nh thao tc tng ng cn thc hin
. Bit kho k
. Bit la chn s xc nh nh thch hp cho vic thc hin c/ghi.Hai kt qu u ra:. Bit i snh mch ra d liu c lu trong b nh c i snh cvi bit i s a.. Bit kt qu ra q.
http://hyperlink/thanh%20ghi.pptxhttp://hyperlink/Ngat.pptxhttp://hyperlink/lenh.pptxhttp://hyperlink/B%E1%BB%99%20nh%E1%BB%9B%20c%E1%BB%A7a%20MT%20tu%E1%BA%A7n%20t%E1%BB%B1.pptxhttp://hyperlink/B%E1%BB%99%20nh%E1%BB%9B%20c%E1%BB%A7a%20MT%20tu%E1%BA%A7n%20t%E1%BB%B1.pptxhttp://hyperlink/B%E1%BB%99%20nh%E1%BB%9B%20c%E1%BB%A7a%20MT%20tu%E1%BA%A7n%20t%E1%BB%B1.pptxhttp://hyperlink/lenh.pptxhttp://hyperlink/Ngat.pptxhttp://hyperlink/thanh%20ghi.pptx8/6/2019 Parallel Processing (1)
14/54
7/17/20
79
2.1 B nh ca MTSS - Cu trc ca b nh kt hp
0
1...
m-1
Argument Register Mask Register
Buffer Register
TagsMatch
Register
Input Input
Output
Array of memory locations
0 1 . . . n-1
80
2.1 B nh ca MTSS
V d, tm trong AM tt c cc t c 8 bit cao nht cha gi tr
(1101 1100)2 v tr li t u tin c tm thy.
Gi tr (1101 1100)2 l i s tm
Thanh ghi nh du (mask register) l 8 bit cao nht. Qu
trnh tm kim thc hin nh sau:1. T cn tm (1101 1100)2 c np vo thanh ghi i s
Argument Register
2.on m chng ta quan tm l 8 bit cao nht, nhng bit
ny c a vo thanh ghi nh du Mask Register
nh du.
3. Tt c cc t trong b nh c so snh song song vi
thanh ghi i s.
81
2.1 B nh ca MTSS
2.1.2 M hnh b nh truy cp ngu nhin song song
B nh PRAM gm:
m vng b nh ln chia s cho p b x l.
c s dng lu tr d liu v l vng trao i d liu
gia cc BXL.
ccBXL c th truy cp vo b nh PRAM ng thi v c th
hot ng mt cch d b.
V d, b x l Pi ghi d liu vo mt vng nh v d liu ny c
th c Pj truy cp, ngha l Piv Pjc th dng b nh chung
trao i vi nhau.M hnh loi ny c cc phng thc truy cp sau:
82
2.1 B nh ca MTSS
Cc phng thc truy cp b nhConcurrent Read (CR): nhiu BXL c th c ng thi cngmt nh.
Exlusive Read (ER):p BXL c c pvng nh khc nhauv mi BXL c c duy nht mt vng nh v mi vng nhc c bi mt BXL.
Concurrent Write (CW):cng mt thi im cho php nhiuBXL ghi vo cng mt vng nh.
Exlusive Write (EW):p BXL ghi c vo pvng nh khcnhau. Mi BXL ghi c vo mt vng nh v mi vng nh
c ghi bi mt BXL.Ngitaphn CWthnh cc loinh sau:
83
2.1 B nh ca MTSS
Ghi ng thi c u tin (Priority CW):cc BXL c gn mc utin v khi c nhiu BXL mun ghi d l iu vo mt vng nh th utin cho BXL c mc u tin cao nht. Cc mc u tin c th gn
tnh hoc ng theo nhng qui tc c xc nh khi thc hin.Ghi ng thi chung(Common CW):Khi cc BXL ghi cng mt gitr th chng c php ghi vo cng mt vng nh. Trong trnghp ny, mt BXL s c chn ghi d liu .Ghi ng thi t do (Arbitrary CW):mt s BXL mun ghi d liung thi vo mt vng nh, nhng c mt BXL c php thay igi tr ca vng nh . Trong trng hp ny, chng ta phi ch racch la chn BXL thc hin vic ghi.Ghi ng thi ngu nhin (Random CW):BXL c la chn ghi d liu l ngu nhin.Ghi ng thi t hp (Combining CW):tt c cc d liu m ccBXL nh ghi ng thi vo b nh c t hp li thnh mt gi tr.Gi tr ny s c ghi vo b nh . 84
2.1 B nh ca MTSS
2.1.2 B nh chia s (Share Memory)c trng:
Dunglng ln
CcBXL u c th truycp vo SM
CcBXL c th hot ng c lp nhng lun chias a ch
cc nh(khng c c quyn)
Nhngthayi nidung nh cthc hin bi mtBXL
no scnhn thy bi ccBXL khc.
Cc my tnh sdng b nhchias cphn lm 2loi:
UMA (Uniform MemoryAccess) NUMA (NonUniform MemoryAccess)
Share Memory
http://hyperlink/b%E1%BB%99%20nh%E1%BB%9B%20AM.pptxhttp://hyperlink/Truy%20c%E1%BA%ADp%20d%E1%BB%8B%20b%E1%BB%99.pptxhttp://hyperlink/Truy%20c%E1%BA%ADp%20d%E1%BB%8B%20b%E1%BB%99.pptxhttp://hyperlink/b%E1%BB%99%20nh%E1%BB%9B%20AM.pptx8/6/2019 Parallel Processing (1)
15/54
7/17/20
85
2.1 B nh ca MTSS
a. M hnh UMA ca b nh chia s
c im:Cc BXL lm vic nh c ch chuyn mch tp trung (central
switching mechanism) iu khin vic truy cp ti b nh
chia s. Cc BXL u c th truy cp n b nh chia s ton cc
(global shared memory)
Thi gian truy cp vo b nh l nh nhau i vi mi BXL.
Mt BXL ny c th trao i thng tin vi mt BXL khc bng
cch ghi thng tin vo b nh ton cc v BXL kia s c d
liu ti cng v tr trong b nh.
86
2.1 B nh ca MTSS
a. M hnh UMA ca b nh chia s (tt)
Nu tt c cc BXL ca my tnh u c thi gian truy cp n
cc thit b l nh nhau th gi l my tnh a b x l i xng
- Symmetric MultiProcessor (SMP) machines
My tnh vi kin trc UMA cn c gi: CC-UMA - Cache
Coherent UMA
P1
Switching mechanism
I/O
P2
Pn
Memory banks
C1 C2 Cn
Pi
Ci
Processor i
Cache i
87
2.1 B nh ca MTSS
a. M hnh UMA ca b nh chia s (tt)
88
2.1 B nh ca MTSS
b. M hnh NUMA ca b nh chia sc im B nhchias c phn tn v chiathnh cc moun nhc lp. B nh chia s c phn tn cho tt c cc BXL thnh bnh cc bv tt ccc moun nh s l b nh chung chocc BXL. M hnh NUMA thng c to thnh t hai hoc nhiuSMPs ni vi li vi nhau bi mt ng truyn vt l. Mt SMP c th truy cp trc tip n mt SMP khc Cc BXL c php truy cp ng thi ti mt hay nhiu m
un nh v cth hotng c lp vi nhau. Khng phi tt c cc BXL u c thi gian truy cp n ccbnh l nh nhau. Truy cp b nh qua link s chm hn
89
2.1 B nh ca MTSS
b. M hnh NUMA ca b nh chia s (tt)
Network
Cache
P
Mem Cache
P
Mem
Cache
P
Mem Cache
P
Mem
NUMA vi Distributed Memory
90
2.1 B nh ca MTSS
2.1.3. B nh phn tn
c im:
C mt ng kt ni giacc BXL
Cc BXL u c b nh ring.
a ch b nh ca mt BXL no khng c bit bi BXLkhc. Do khng c khi nim khng gian a ch tng th(global address space) qua cc BXL khc.
V BXL ny khng th t do c/ghi vo bnhcamt BXLkhc nn khi nim Cache coherent khng p dng y.
Khi cn thit ngi lp trnhc nhim v chuynd liu t nh ca BXL ny sang nh ca mt BXL khc.
http://hyperlink/Cache%20coheren.pptxhttp://hyperlink/Cache%20coheren.pptxhttp://hyperlink/Cache%20coheren.pptxhttp://hyperlink/Cache%20coheren.pptx8/6/2019 Parallel Processing (1)
16/54
7/17/20
91
2.1 B nh ca MTSS
2.1.3. B nh phn tn
92
2.1 B nh ca MTSS
2.1.3. B nh a my tnh
Mi nttrongh thng a my tnh cng chnh l mt my tnhc b nh ring, khngchias vi nhngBXL khc. Cc BXL traoi vinhau thng qua vic gi v nhn cc thng ip (messages)Khng tn ti b nh chia s chung m mi BXL c b nh ccb ring ca chng.Vic trao i d liu trong mng theo m hnh point to pointthng qua s lin kt tnh gia cc BXL.
Memory BXL
BXLMemory
Memory BXL
Memory BXL
c im:
93
2.1 B nh ca MTSS
2.1.4 My tnh vi b nh lai (Hybrid Memory) Kt hp gia kin trc chia s b nh chung v b nh phn
tn.
H thng l nhiu cm my tnh khc nhau. Mi cm l cc
my tnh vi b nh c kin trc chia s v c b nh ton cc
ring cho cm .
Cc BXL trong mt cm chia s b nh chung c th truy cp
n b nh ton cc ring ca cm .
Thnh phn b nh phn tn c hiu nh l mt cm cc
b nh ton cc ca mi cm.
Cc BXL ch c th truy cp n b nh ton cc trong thnhphn chia s b nh phn tn ca chng, ch khng truy cp
c b nh ca cc thnh phn chia s b nh chung khc. 94
2.1 B nh ca MTSS
6. My tnh vi b nh lai
Hybrid Distributed-Shared Memory
95
2.1 B nh ca MTSS
Tm li,Hai iu cn phi quan tm trong thit k b nh ca h thngsong song. l,
Bng thng(bandwidth): ng kt ni cc BXL, b nh tr b nh(memory latency), tr b nh c hiu nhl khong thi gian t khi BXL yu cu d liu t b nhn khiBXL nhn c d liu tng ng.
Trong h thng my tnh song song bng thng phi rng thc hin vic truyn thng ca cc BXL.
Khi cc mun nh c chia s th v ic cnh tranh s dngkhng gian nh phi c cc tiu ha. Do , tr b nhcng phi c cc tiu ha.
96
2.2 Mng kt ni cc thnh phn ca MTSS
Mt vi nhn xt:Trong hu ht cc kin trc song song nh: nhng h
a BXL chia s b nh, a b x l a b nh , v.v. thvn quan trng nht trong thit k l xc nh s kt
ni gia cc BXL v b nh vi nhau.
Mt kin trc l tng l kin trc trong , mi BXL
u kt ni c vi cc BXL cn li. Khi n to ra
mt th y .
V d, nu h c p BXL th s c p*(p 1)/2 ng
lin kt. D nhn thy kin trc loi ny s rt phc
tp, nht l khi p ln.
8/6/2019 Parallel Processing (1)
17/54
7/17/20
97
2.2 Mng kt ni cc thnh phn ca MTSS
C hai loi cu hnh tp cho mng lin kt: lin kt
tnh v lin kt ng.
Mng lin kt tnh l mng cc thnh phn ca hthng my tnh, trong cc BXL, b nh c lin
kt vi nhau mt cch c nh, khng thay i c.
Mng lin kt ngl mng cc thnh phn ca h
thng my tnh, trong s lin kt gia cc BXL, b
nh c th thay i c.98
2.2 Mng kt ni cc thnh phn ca MTSS
2.2.1 Lin kt tuyn tnh v vng (linear and ring)
a. Lin kt tuyn tnhCc BXL c lin kt vi nhau theo dy v c nh s theo
th t tng dnP0 P1 Pn-1. . .
Nhn xt:
Trong mng ln kt tuyn tnh, tr hai phn t u v cui, tt
c cc BXL bn trong u c hai lng ging l BXL trc v sau
n.
y l dng lin kt n gin, nhng d liu cng cn phi
chuyn qua nhiu BXL, do s truyn thng d liu gia cc
BXL, c bit l gia BXL u v cui s b chm li khi s BXL
kh ln.
99
2.2 Mng kt ni cc thnh phn ca MTSS
2.2.1 Lin kt tuyn tnh v vngb. Lin kt vng
c t chc tng t nh lin kt tuyn tnh nhng BXL uv cui c ni vng vi nhau
Trong mng lin kt vng, s trao i gia cc BXL c th thchin theo mt chiu, gi l mng n, hoc theo c hai chiu gil mng kp.
Tt nhin s truyn thng trong mng lin kt vng, nht l gianhng BXL xa nhau th cng vn b tr.
P2
P6 P5
P3
P4P0
Nhn xt:
100
2.2 Mng kt ni cc thnh phn ca MTSS
2.2.2 Mng lin kt xo trn hon ho (Perfect Shuffle Exchange)Gis c N BXL: P0, P1, , PN-1, vi N l ly tha ca 2.Trong mng lin kt xo trn hon ho, ng lin kt mtchiu tPi tiPjc xc nh nh sau:
2i vi 0 i N/2 - 12i+1-N vi N/2 i N - 1
Mng lin kt xo trn hon ho mt chiu vi N = 8=2 3,trong s lin kt xo trn c k hiu bng mi tn.
j = {
P0 P1 P2 P3P4 P5
P6 P7
101
2.2 Mng kt ni cc thnh phn ca MTSS
Mt biu din khc ca mng lin kt xo trn hon ho
P001
P010
P011
P100
P000
P101
P110
P111
P001
P010
P011
P100
P000
P101
P110
P111
102
2.2 Mng kt ni cc thnh phn ca MTSS
2.2.2 Mng lin kt xo trn hon ho hai chiu
Mng lin kt xo trn hon ho hai chiu vi N = 8,trong s lin kt xo trn c k hiu bng mi tn hai chiu.
P0 P1 P2 P3P4 P5
P6 P7
8/6/2019 Parallel Processing (1)
18/54
7/17/20
103
2.2 Mng kt ni cc thnh phn ca MTSS
2.2.3 Mng lin kt li hai chiu (two-dimentional mesh)c im:Mi BXL c lin kt vi bn lng ging: trn, di, bn tri vbn phi.Mng lin kt lihaichiuc s dng thit k cc mytnh ILLIAC IV, MPP (Massively Parallel Processor), DAP (ICLDistributed Array Processor), v.v.C hai dng lin kt li:
(a) Li khng quay vng (b) Li quay vng104
2.2 Mng kt ni cc thnh phn ca MTSS
2.2.4 Mng lin kt siu khi hoc hnh khi n-chiuGi s c n b x l: P0, P1, , Pn-1, vi n = 2q, q 0. Nu miBXL cn lin kt vi ng qb x l ln cn th c th dngmng lin kt theo hnh siu khi n chiu.Trong mng lin kt hnh khi cc ch s ca cc BXL cnh nh phn, hai BXL c gi l lng ging vi nhau nunhn ch s ca chng sai khc nhau ng mt bit.
Mng lin kt hnh khi vi n=8 (q=3)
2
P0000
5
1
4
76
3
P001
P100
P101
P111P110
P010 P011
105
2.2 Mng kt ni cc thnh phn ca MTSS
Mng lin kt hnh khi vi n=16 (q=4)
2
0
5
1
4
76
3
2
0
5
1
4
76
3
5 = 0101
1 = 0001
4 = 010013 = 1101
106
2.2 Mng kt ni cc thnh phn ca MTSS
2.2.5 Mng lin kt hnh sao (star) K hiu l Sn C n! b x l Nhn ca mi BXL s tng ng vi mt hon v ca n k hiu. BXL Pi c kt ni vi BXL Pj j nhn c t i bng cchthay k hiu th k ca hon v n phn t 1,..., n, vi 2kn.
P1234
P3214 P2134
P3124
P1324
P2314
V d, trong S4 (n = 4) c 4! = 24 BXL.Hai BXL P2134 v P3124 l c lin kt vinhau v 3124 nhn c t 2134 bngcch i ch v tr u (s 2) vi v trth ba (s 3). Tng t suy ra P2134 vP4132 l clin kt vi nhau.
107
Mng lin kt hnh sao vi 24 b x l
P3412
P4312 P1432
P1342 P4132
P3142
P4231
P3241 P2431
P2341 P3421
P4321
P1234
P3214 P2134
P2314 P3124
P1324
P2413
P4213 P1423
P1243 P4123
P2143
108
2.2 Mng kt ni cc thnh phn ca MTSS
2.2.5 Mng lin kt Butterfly
(k+1)2k nt c chia thnh (k+1) ranks, mi rank cha n=2k nt.
Cc ranks c nh s t 0 n k
K hiu Node(i,j): nt th j trn rank th i
Node(i,j) c ni vi 2 nt trn rank i-1: node(i-1,j) v node (i-1,m),
y m l s nguyn c c bng cch o bit th i trong xu nh
phn biu din j
Nu node(i,j) c ni n node(i-1,m), th node (i,m) c ni vi
node(i-1,j)
8/6/2019 Parallel Processing (1)
19/54
7/17/20
Rank 0
Rank 1
Rank 2
Rank 3
Node(1,5): i=1, j=5
j = 5 = 101 (binary)
i=1
001 = 1
Node(1,5) c ni vinode(0,5) v node(0,1)
0 1 2 3 4 5 6 7
Mng lin kt Butterfly vi 4 ranks v 32 BXL
2.2 Mng kt ni cc thnh phn ca MTSS
2.2.5 Mng lin kt Butterfly (tt)
109
Mt thp chiu cao d gm (4d+1-1)/3BXL c phn tn trong d+1 mc ccc tnh cht sau:
C 4d-2
BXL mc d. C 4d-1 BXL mc d-1, v
C 4d BXL mc d-2
Ch c 01 BXL mc d
i vi mc x:
c ni vi 04 nt lng ging cng mt mc nu x
8/6/2019 Parallel Processing (1)
20/54
7/17/20
C k tng.
Mi tng gm cc thanh cho c m input v n output. tng
cng m*n thanh cho. Mng bao gm mk input v nk output; do n mng cmk*n
k chuyn mch.
Cc kt ni chuyn mch cho php to mt ng dnchnh xc t cc input n cc output.
Nu A l a ch ch ca mt kt ni mong mun trn c sca n, th cc s ca A biu din cc thanh cho t ra thit lp kt ni theo d kin .
2.2 Mng kt ni cc thnh phn ca MTSS
2.2.10 Mng lin kt Delta
115
2.2 Mng kt ni cc thnh phn ca MTSS
2.2.10 Mng lin kt Delta
116
mng delta 3 tng, c 23*23 ng kt ni, dng 2*2 thanh cho.Mi tng c 23 input v 23 output.
tng 1 tng 2 tng 3
M0
M1
M2
M3
M4
M5
M6
M7
P0
P1
P2
P3
P4
P5
P6
P7
117
2.2 Mng kt ni cc thnh phn ca MTSS
2.2.8 Mt s mng kt ni khc
Tham kho:[6] M. Sasikumar, Dinesh Shikhare, P. Ravi Prakash, Introduction to ParallelProcessing, Prentice Hall. 2002[7] Seyed H. Roosta, Parallel Processing and Parallel Algorithms, Theory andComputation, Springer, 2006.[8] Michael J. Quinn, Parallel Computing Theory and Practice, MaGraw-Hill,2004
118
2.4 Kt lun
Vn trng tm ca chng 21. Tm hiu cc ti nguyn v mi quan h ca chng trong h
thng tn dng c ht kh nng x l song song.2. Cc b nh c t chc thnh b nh kt hp, b nh truy
cp ngu nhin, b nh chia s, v.v. l cc m hnh chnh chovic thit k b nh MTSS.
3. Vn quan trng trong thit k kin trc ca MTSS l xcnh cch kt ni cc b x l vi nhau sao cho hiu qunht.
4. Cc b x l c th kt ni theo mng lin kt tnh hoc linkt ng.
5. Kho st mt s cu hnh tp ca mng lin kt cc b xl nh: lin kt tuyn tnh, lin kt xo trn, lin kt li, linkt hnh sao v lin kt hnh khi...
119
2.3 Chng trnh dch v cc h iu hnh
Cu hi cui chng1. Nu nhng vn cn quan tm khi thit k kin trc my
tnh song song
2. B nh kt hp l g? nu nguyn l hat ng ca b nhkt hp.
3. Ti sao mng lin kt li ng vai tr quan trng trong kintrc MTSS?
4. Da vo nh ngha chung ca mng lin kt hnh khi xydng cu trc tp cho mng lin kt hnh khi cho 16 b xl.
5. Nu nhng c trng c bn ca chng trnh dch songsong?
6. Nu mc ch chnh ca h iu hnh cho my tnh songsong?
7. Xy dng mng lin kt theo m hnh xo trn hon ho cho16 phn t. 120
Tiu lun
1. Kin trc kt ni mngDe Bruijn, Mng cy nh phn, Mng Delta, Mng Butterfly,Mng Omega, Mng Pyramid ([7] Seyed H. Roosta, Parallel Processing
and Parallel Algorithms, Theory and Computation, Springer, page 72-80)
2. Chng trnh dch song song ([7] Seyed H. Roosta, ParallelProcessing and Parallel Algorithms, Theory and Computation,Springer, page 83-90)
3. H iu hnh a x l ([7] Seyed H. Roosta, Parallel Processing andParallel Algorithms, Theory and Computation, Springer, page 91-100)
4. Tm hiu h iu hnh Linus-Windows (lin quan n a lung atin trnh)
8/6/2019 Parallel Processing (1)
21/54
7/17/20
121
CHNG 3. GII THIU V LP TRNH SONG SONG
NI DUNG
1.Nhng khi nim c s ca lp trnh song song
2.Cc ngn ng lp trnh song song
3.Cc loi ph thuc d liu trong chng trnh
4.Phng php bin i chng trnh tun tsang chng trnh song song
122
3.1 Gii thiu chung
S khc nhau c bn gia LT tun t v LT song song
Trong mi trng lp trnh tun t:
Cc cu lnh c thc hin tun t.
Mi chng trnh thc hin s to ra nhng tin trnh bn
trong h thng m ngi lp trnh khng quan st c.
Mi cu lnh thc hin khng gy tr ngi cho cc cu lnh
khc trong chng trnh.
123
3.1 Gii thiu chung
Trong mi trng songsong:Cc cu lnh ca chng trnh c th thc hin an xen lnnhau. cng mt thi im c th c nhiu hn mt lnh cthc hin, ngha l mi chng trnh s t ch thc hin cctin trnh ca mnh.Cc chng trnh phi tng tc vi nhau v vic thc hinca chng nh hng ti nhp thc hin ca nhau.Trong LTSS, ngi lp trnh khng ch vit chng trnh, dliu nh trong mi trng tun t m cn phi s dng cccng c ng b ho (synchronize) v iu khin s tngtcgia cc tin trnh.Ngi lp trnh cn to ra v lp lch cho cc tin trnh, nghal s thc hin chng trnh c th nhn thy c bi ngilp trnh.
124
3.1 Gii thiu chung
Vn quan trng trong LTSS l phi tn dng c khnngca cc bx l.
C hai cch tip cn tn dng khai thc cc b x l:
1. Pht trin nhng ngn ng lp trnh cho php th hin
c vic thc hin song song mc thut ton, v d nh
Fortran, C, v.v.
2. Xy dng nhng chng trnh dch mnh nhn
dng c cc phn on chng trnh c th thc hin
song song hay tun t.
Hai cch tip cn trn l b sung cho nhau, nu ch p dngmt cch th khng hiu qu.
125
3.1 Gii thiu chung
Mt s phng php tip cn trong lp trnh song song:
Lp trnh song song kiu SIMD vi b nh chia s, trong
truy cp b nh l ng b (Synchronous). Lp trnh song song kiu MIMD vi b nh chia s, trong
truy cp b nh l d b (Asynchronous). Lp trnh song song kiu MIMD vi b nh phn tn, trong
truy cp b nh l d b. Lp trnh song song kiu SPMD vi b nh chia s, trong
truy cp b nh l d b. Lp trnh song song kiu SPMD vi b nh phn tn, trong
truy cp b nh l d b. Lp trnh song song kiu MPMD vi b nh chia s, trong
truy cp b nh l d b.
126
3.2 Cc ngn ng lp trnh song song
Cc yu cu i vi mt NNLT song songNgoi cc yu cu i vi mt NNLT tun t, mt NNLT song song
cn phic cc chc nng sau:
1. Cung cp cho ngi lp trnh nhng c ch khi to,
ngb v trao i gia cc tin trnh.
2. To ra c nhng chng trnh c lp vi my tnh
3. Phi h tr to ra c nhng chng trnh d c, d
vit, d chuyn i, v.v.
4. M hnh ho c victhc hin song song.
5. C kh nng iu chnh cc tnh hung m cc tin
trnh i hi phi trao i, tng tc vi nhau.
http://hyperlink/Ti%E1%BA%BFn%20tr%C3%ACnh.pptxhttp://hyperlink/Ti%E1%BA%BFn%20tr%C3%ACnh.pptxhttp://hyperlink/Ti%E1%BA%BFn%20tr%C3%ACnh.pptx8/6/2019 Parallel Processing (1)
22/54
7/17/20
127
3.2 Cc ngn ng lp trnh song song
Cc tnh hung thng gp trong LT song song
1. Ti mt thi im c mt s tin trnh mun truy cp vo mt
ti nguyn chung hoc cp nht vo mt bin chia s. M
nhng ti nguyn ch cho php mt tin trnh truy cp ti
mi thi im.2. Khi mt tin trnh c quyn truy cp vo ti nguyn chung
th n s dng ti nguyn nhng khng c ngn cn
hot ng ca nhng tin trnh khc.
3. Khi mt s tin trnh cng kt hp thc hin mt s php
ton trn c s quan st hnh ng ca nhauth ngi lp
trnh phi lp lch cho nhng tin trnh .
C hai cch gii quyt cc tnh hung trn: 128
3.2 Cc ngn ng lp trnh song song
1.Tt c cc tin trnh phi s dng cu trc d liuchung tin trong vic cp nht v trao i vi
nhau.
2.Tt c cc tin trnh thc hin s ng b bng cch s dnghaihm wait()v pass(). Khi trao ivi nhau, mt tin trnh ch nhn c mt tnhiu ca tin trnh i tc v khi nhn c th haitin trnh trao i trc tip vi nhau.
129
3.2 Cc ngn ng lp trnh song song
Tng qut, c hai cch pht trin NNLT song song:
1. M rng nhng ngn ng lp trnh tun t hin c,
b sung thm nhng cu trc mi thc hin
c song song v gii quyt c s xung t
trong truy cp d liu.
2. Xy dng mt ngn ng lp trnh song song mi.
130
3.2 Cc ngn ng lp trnh song song
3.2.1 V d minh ha
a. Ci t song song trn kin trc UMA vi 4 BXL:
n i
n
rm
1n in
mr
1
2)(
trong
Bi ton: Xc nh phng sai (variance) ca mt mu c
cho bi mt danh sch cc s thc r1, r2, ...., rn . Ngha l phitnh:
BXL1 BXL2 BXL4
Memory I/O Devices
Switching mechanism
. . .. . .
BXL3
131
3.2 Cc ngn ng lp trnh song song
Ci t song song:Cc s thc ri c lu tr trong b nh chia s.
Bn bin sau y cng c to ra trong b nh chia s:
Sum (lu tng gi tr):
Mean (lu gi tr trung bnh):
Sum of square (lu tng bnh phng)
Variance (lu gi tr phng sai)
Qu trnh thc hin song song cm t nh sau:
n
irS 1
n i
n
rm
1
2
1)( mrSS
n
i
n
mr
n
SSn
i
2
1)(
132
n i
n
rm
1
ni
n
mr
1
2)(
Sum ofSum Mean Squares Variance
Values
36
Shared memory
0
541542233124
Process 4Temporary
Process 3Temporary
Process 2
Temporary
Process 1Temporary
8
7
11
10
Sum ofSum Mean Squares Variance
Values
36
Shared memory
3 0
541542233124
Process 4Temporary
Process 3
Temporary
Process 2Temporary
Process 1Temporary
8
7
11
10
(a): Mi tin trnh cng 3gi tr c phn chiavlu vo bin tm thica n. Sau khi cc tin
trnh thc hin xongvic t nh cc tng con,s c mt tin trnhcng dn cc gi tr nyli v lu vo bin Sum bnh chia s.
(b): Mt tintrnh n s
tnh gi tr trung bnh v
lu vo bin b nh
chia s Mean.
n=12; (ri)=(4,2,1,3,3,2,2,4,5,1,4,5)
8/6/2019 Parallel Processing (1)
23/54
7/17/20
133
n
i
n
rm
1
n
i
n
mr
1
2)(
Sum ofSum Mean Squares Variance
Values
36
Shared memory
3 22 1.83
541542233124
Process 4Temporary
Process 3Temporary
Process 2Temporary
Process 1Temporary
1
6
6
9
(c): Mi tin trnh phi xc
nh:(xi-Mean)2, (xi-Mean)2
lu vo bin tm thi v
cui cng cng dn lu
vo bin ca b nh chia
s.
(d): Mt tin trnh n xcnh variance bng cchchia gi tr ca Sum of
Squares cho 12.
Sum of
Sum Mean Squares Variance
Values
36
Shared memory
3 22
541542233124
Process 4Temporary
Process 3
Temporary
Process 2
Temporary
Process 1Temporary
1
6
6
9
n=12; (ri)=(4,2,1,3,3,2,2,4,5,1,4,5)
134
3.2 Cc ngn ng lp trnh song song
3.2.1 V d minh ha (tip)
b. Ci t song song trn NUMA vi kin trc siu khi 4 BXL:
Bi ton: Xc nh phng sai (variance) ca mt mu c
cho bi mt danh sch cc s thc r1, r2, ...., rn . Ngha l phi
tnh:
n i
n
rm
1n in
mr
1
2)(trong
Ci t song songKin trc ny khng c b nh chia s chung; n gi tr r1, r2,...., rn cphn phi cc b nh cc bcami BXL.Mi nt c 4 bin: 2 bin lu tr cc gi tr cng dn lSum v Sum of Squares; mt bin lu gi tr trung bnhMean v bin cn li lu tr gi tr phng sai variance.Qu trnh tip tc thc hin c m t nh hnh di y.
135
3.2 Cc ngn ng lp trnh song song
Nhc li,
c im ca NUMA:
C mt ng kt ni cc BXL li vi nhau.
B nh chia s c phn tn cho tt c cc BXL thnh b
nh cc b v tt c cc moun nh s l b nh chung cho
cc BXL.Cc BXL c php truy cp ng thi ti mt hay nhiu m
un nh v c th hot ng c lp vi nhau.
Network
CacheP
Mem CacheP
Mem
CacheP
Mem CacheP
Mem
NUMA vi Distributed Memory
136
Sum1
Mean
Sum2
Variance
Values
4 2 1
7 Sum1
Mean
Sum2
Variance
Values
3 3 2
8
Sum1
Mean
Sum2
Variance
Values
2 4 5
11 Sum1
Mean
Sum2
Variance
Values
1 4 5
10
Sum1
Mean
Sum2
Variance
Values
4 2 1
36
3
Sum1
Mean
Sum2
Variance
Values
3 3 2
36
3
Sum1
Mean
Sum2
Variance
Values
2 4 5
36
3
Sum1
Mean
Sum2
Variance
Values
1 4 5
36
3
(b):Cc BXL thc hin hon i vcng dn, sau cng chia cho 12 xc nh Mean.
(a): (b):
(a):Cc tin trnh tnh tng cc gi trti mi nt.
n
i
n
rm
1
ni
n
mr
1
2)(
n=12; (ri)=(4,2,1,3,3,2,2,4,5,1,4,5)
137
Sum1
Mean
Sum2
Variance
Values
4 2 1
36
3
6
Sum1
Mean
Sum2
Variance
Values
3 3 2
36
3
1
Sum1
Mean
Sum2
Variance
Values
2 4 5
36
3
6
Sum1
Mean
Sum2
Variance
Values
1 4 5
36
3
9
Sum1
Mean
Sum2
Variance
Values
4 2 1
36
3
22
1.83
Sum1
Mean
Sum2
Variance
Values
3 3 2
36
3
22
1.83
Sum1
Mean
Sum2
Variance
Values
2 4 5
36
3
22
1.83
Sum1
Mean
Sum2
Variance
Values
1 4 5
36
3
22
1.83
(c) Mi tin trnh, vi cc gi trr ing hin c ti nt ca mnh sthc hin t nh (xi-Mean)2 v luvao bin Sum2.
(c): (d):
(d) Cc BXL thc hin hon iv cng dn, sau cng chiacho 12 xc nh Variance.
n
i
n
rm
1
ni
n
mr
1
2)(
n=12; (ri)=(4,2,1,3,3,2,2,4,5,1,4,5)
138
3.2 Cc ngn ng lp trnh song song
Gii thiu mt s s ngn ng lp trnh song song
3.2.2 Fortran 90
Fortran 90 l ngn ng lp trnh chun ANSI. Fortran 90 l ngn ng lp trnh song song theo d liu
c nng cp t Fortran 77 bng cch b sung thm mt
s c tnh nh:
Kiu d liu do User nh ngha
Php ton v Array, con tr
Cc BXL h tr vic s dng cc kiu d liu Short
Integer, Packed logical,...
Cp pht b nh ng.
http://hyperlink/Fortran%20language.pptxhttp://hyperlink/ANSI.pptxhttp://hyperlink/Packed%20logical.pptxhttp://hyperlink/Packed%20logical.pptxhttp://hyperlink/ANSI.pptxhttp://hyperlink/Fortran%20language.pptx8/6/2019 Parallel Processing (1)
24/54
7/17/20
139
3.2 Cc ngn ng lp trnh song song
Ngi lp trnh Fortran 90 da vo m hnh song song tng
t nh PRAM gm c nhng thnh phn:
Mt CU (n v iu khin)
Mt n v x l logic, s hc ALU B nh chia s.
Mt tp cc BXL
CPU thc hin cc cu lnh tun t bng cch truy cp vo
cc bin c lu tr trong b nh chia s.
n v vector lu tr, c, ghi d liu vo b nh chia s v
c CPU iu khin thc hin song song.
140
3.2 Cc ngn ng lp trnh song song - Fortran 90
Fortran 90c cc kiu d liu chun:REAL, INTEGER, CHARACTER, LOGICAL, v.v.Dng khai bo tng qut:
Type[(kind)] [, attribute] :: Variable-ListTrong , Typel kiu c s hoc kiu c nh ngha bi NSD kind l phn tu chn cng mt s kiu c s dng
nh ngha v kiu c thV d: CHARACTER (LEN = 10) :: s1nh ngha bin s1 kiu CHARACTER c th cha 10 k t. [, attribute] l danh sch cc thuc tnh ca Fortran c s
dng nh ngha cc c tnh ring ca danh sch ccbin.
+:: phn t phn cch gia phn m t kiu v danh sch ccbin.
Variable-List:danh sch cc bin
141
3.2 Cc ngn ng lp trnh song song - Fortran 90
V d v khai bo v mng:INTEGER, DIMENSION (1 : 10) :: SA{Bin mng SA c 10 phn t kiu s nguyn.}+ Khai bo mt hng mng:(/ 2,4,6,8,10 /) hoc (/ (I, I = 2, 10, 2) /)+ Fortran 90 cho php p dng cc php ton s hc cho cc
bin mng, ngha l ton hng ca cc php ton s hc (+,-, *, /) c th l bin n hoc bin mng.
V d v php ton trn bin mng:INTEGER A(10), B(10), CDO I = 1, 10, 1
A(I) = B(I) + C
END DOTa c th vit ngn gn hn.
A = B + C142
3.2 Cc ngn ng lp trnh song song - Fortran 90
Cu trc chng trnh Fortran
PROGRAM
IMPLICIT NONE
[phn c t]
[phn thc hin]
[phn chng trnh con]
END PROGRAM
143
3.2 Cc ngn ng lp trnh song song - Fortran 90
Xy dng mt chng trnh FORTRAN
144
V d: tnh s t cng thc bng NNLT Fortran 90
1. INTEGER, PARAMETER :: N = 131000
2. INTEGER, PARAMETER :: LONG =SELECTED_REAL_KIND(13,99)
3. REAL(KIND=LONG) PI, WIDTH4. INTEGER, DIMENSION(N)::id
5. REAL(KIND=LONG), DIMENSION(N)::X, Y
6. WIDTH = 1.0_LONG/N
7. ID = (/(I,I = 1, N)/)
8. X = (ID 0.5)* WIDTH
9. Y = 4.0 / (1.0 + X * X)
10.PI = SUM(Y) * WIDTH
11.10 FORMAT(PI = , F14.12)
12.PRINT 10, PI
13.END
.
khai bo tham s N ls cc on con ti a tnh tch phn
khai bo tham s LONG sdng cho cc bin s thc viphn t nh c t nht 13 ch sv phn ng (m) thuckhong [10-99,1099]. Cc binthc ny c khai bo dng3 v 5
xc nh rng ca mikhong con. Bin mng ID gmN phn t khai bo d ng 4,ckhito dng 7
tnh ton song song im giacami khong contnh ton song song gi t r hmtng ngti ccim gia X
gihm tnh tng SUM
Khai bo bin mng id cN phn t kiu s nguyn
8/6/2019 Parallel Processing (1)
25/54
7/17/20
145
Tiu lun: Fortran 90 Tutorial (02 hc vin)
Professor Dr.Shene
Department of Computer Science
Michigan Technological University
www.cs.mtu.edu/~shene/COURSES/cs201/NOTES/fortran.html
http://www.fortran-2000.com/
146
3.2 Cc ngn ng lp trnh song song - OCCAM
3.2.3 Lp trnh song song vi OCCAMGii thiu v OCCAM: NNLT song song c Inmos Company (Anh) pht trin
nm 1988, Mc ch chnh l thit k v ci t cc chip c gi l
transputer. Chng trnh Occam thng nhiu tin trnh v chng c
th c nh x sang mt s cc transputerbt k thchin song song v trao i d liu vi nhau thng qua ccknh vo/ra.
Occam l NNLT bc cao, c s dng lp trnh chonhng h thng gm nhiu my tnh kt ni vi nhau, hoccc h phn tn.
Trong Occam, cc hnh ng c th thc hin song songc gi l tin trnh v mi cu lnh cn phi khai bo nhmt tin trnh.
147
M phng m hnh lp trnh trong Occam
A: Cc tin trnh thchin tun t
B: Cc tin trnh thchin song song
C: Cc tin trnh thchin tun t
D: Cc tin trnh thchin song song
E: Cc tin trnh thchin tun t
A
B
C
D
E
148
3.2 Cc ngn ng lp trnh song song - Occam
Cc tin trnh trong Occam
C ba tin trnh nguyn thu trong Occam:
Tin trnh gn: thay i gi tr ca cc bin
Tin trnh Input: nhn d liu vo t cc knh vo (cng vo)
Tin trnh Output: gi d liu ra cc knh ra.
Cu trc ngn ng
Tin trnh gn:
sum := partion1 + partion2
/*gn gtr cho bin sum l tng hai gtr partion1v partion2 */
Tin trnh Input: user1 ? x
/*gn gi tr t knh user1 cho x */ Tin trnh Output: C ! x * x
/*gi gi tr x2 ra knh C */
149
3.2 Cc ngn ng lp trnh song song - Occam
Cc cu trc iu khin
SEQ: cu trc tun t, cc thnh phn trong tin trnh ny
c thc hin theo th t v tin trnh ny kt thc khithnh phn cui cng thc hin xong. Mi thnh phn ca
tin trnh ny c th l mt tin trnh song song.
V d:
SEQ
channel1 ? partial1
channel2 ? partial2
sum:= partial1 + partial2
partial3 ! Sum
Cc s nguyn c nhp
vo tun t t 2 knh khc
nhau sau cng li v
chuyn sang knh 3
150
3.2 Cc ngn ng lp trnh song song - Occam
Cc cu trc iu khin (cont.)
PAR: cu trc song song, cc thnh phn ca tin trnh ny
c thc hin ng thi v tin trnh ny s kt thc khi ttc cc thnh phn ca n u kt thc.
V d:
SEQ
PAR
channel1 ? partial1channel2 ? partial2sum:= partial1 + partial2
Cc s nguyn cnhp vo song song t 2knh khc nhau sau cng li
http://www.cs.mtu.edu/~shene/COURSES/cs201/NOTES/fortran.htmlhttp://www.fortran-2000.com/http://hyperlink/occam%20(programming%20language).pptxhttp://hyperlink/Transputer.pptxhttp://hyperlink/Ti%E1%BA%BFn%20tr%C3%ACnh.pptxhttp://hyperlink/Ti%E1%BA%BFn%20tr%C3%ACnh.pptxhttp://hyperlink/Ti%E1%BA%BFn%20tr%C3%ACnh.pptxhttp://hyperlink/Transputer.pptxhttp://hyperlink/occam%20(programming%20language).pptxhttp://www.fortran-2000.com/http://www.fortran-2000.com/http://www.fortran-2000.com/http://www.cs.mtu.edu/~shene/COURSES/cs201/NOTES/fortran.html8/6/2019 Parallel Processing (1)
26/54
7/17/20
151
3.2 Cc ngn ng lp trnh song song - Occam
Cc cu trc iu khin (cont.)
ALT: cu trc tuyn chn, chn mt trong cc thnh phn ca
tin trnh thc hin nu n tho mn iu kin la chn
v tin trnh ny kt thc khi thnh phn c la chn ktthc. (xem v d tnh tch phn)
152
3.2 Cc ngn ng lp trnh song song - Occam
Cc cu trc khc
1. Cu trc iu khin IF, gi l tin trnh iu kin chn mt
tin trnh thnh phn khi biu thc Boolean c gi tr true;
2. Cu trc lp WHILE, gi l tin trnh lp thc hin lp li
tin trnh thnh phn cho n khi biu thc iu kin
Boolean nhn gi tr true.
3. Cu trc lp FOR, gi l tin trnh lp c s vng lp xc
nh trc, i = [0 FOR n]
V d: PAR i = [0 FOR 9]: to ra 10 tin trnh song song, i
nhn gi tr t 0 n 9
153
V d: Gi s c hai tin trnh ging nhau cng nhn d liu vo vcng dn vo tng.
CHAN In1, In2:CHAN Out1, Out2:VAR Sum1, Sum2:
SEQ &cc lnh di y thc hin tun t
Sum1 := 0 &gn gi tr u cho Sum1 v Sum2Sum2 := 0PAR &bt u thc hin song song
-- Tin trnh th nhtWhile TRUE
VAR Item1:SEQ
In1 ? Item1 &gn gi tr knh In1 cho Item1Sum1 := Sum1 + Item1Out1 ! Item1
-- Tin trnh th haiWhile TRUE
VAR Item1:
SEQIn2 ? Item2Sum2 := Sum2 + Item2Out2 ! Item2
154
3.2 Cc ngn ng lp trnh song song - Occam
S trao i gia cc tin trnh
Trong v d trn, cc tin trnh khng cn trao i vi nhau v
mi tin trnh u s dng cc bin cc b ca ring n.
Tuy nhin, khi c nhiu tin trnh mun trao i d liu vi
nhau th phi trao i trn cng mt knh truyn d liu.
Ngha l, mt tin trnh gi d liu ra mt knh truyn v
tin trnh kia nhn d liu t knh .
Trong Occam, mi tin trnh thc hin trn mt b x l v
truyn thng ip (Passed-Message) ti nhng b x l lncn theo kin trc hnh khi.
155
1. DEF N = 400000:2. DEF PROCESS = 8:
3. DEF CHUNK = N / PROCESS:
4. CHAN sum[PROCESS]5. PAR6. PAR i = [0 FOR PROCESS]7. REAL64x, localsum, width:8. SEQ9. localsum := 0.010. width := 1.0 / N11. x := ((i * CHUNK) + 0.5) * width12. SEQ i = [0 FOR CHUNK]13. SEQ14. localsum:=localsum+(4.0/(1.0+(x*x))))15. x := x + width
V d: tnh s t cng thc bng NNLT Occam
cont.
Tdng 1-3:nh ngha cchng: N l s khong chia; PROCESSES l s t in
trnh s c to; CHUNK l s khong
cho mi t in trnh (chng trnh thc hinchnh xc nn chn Nl bi s caPROCESSES
Dng 4: nh ngha mtknh cho mi t in trnh.
Tin trnh s sdngknhny gi dl iu ca n(subtotal) ra tin trnh bnngoi (grand total)
T dng 5-31: l nhng t intrnh song song v tun t,gm c (PROCESS+1) tintrnh. Mi tin trnh trong cctin trnh thc hin song songt dng 6-17, s xc nh tngdin tch cc hnh ch nhtc chia s
156
16. localsum := localsum * width17. sum[i] ! localsum18. REAL64 pi :
19. INT got [PROCESS] :20. SEQ21. pi := 0.022. SEQ i = [0 FOR PROCESS]23. got[i] := FALSE24. SEQ i = [0 FOR PROCESS]25. REAL64 y:26. SEQ27. ALT i = [0 FOR PROCESS]28. (got[i] = FALSE) & sum[i] ? Y29. got[i] := TRUE30. pi := pi + y31. Output ! Approximation to pi is: ; pi
V d: tnh s t cng thc bng NNLT Occam
T dng 18-31: l tintrnh cui cng, xcnh tng sau cng vchuyn cho t in trnh
output in ktqu pi
8/6/2019 Parallel Processing (1)
27/54
7/17/20
157
Tiu lun (02 hc vin):
OCCAM Language
http://en.wikipedia.org/wiki/Occam_programming_language
http://www.wotug.org/
Tiu lun (02 hc vin): ThreadMentor
http://cs.mtu.edu/~shene/NSF-3/e-Book/index.html
158
3.2.4 Lp trnh song song vi PVM (Parallel Virtual Machine)
PVM c pht trin bi University of Tennessee, Oak Ridge
National Laboratory and Emory University (1989)
PVM l mt phn mm s dng cho h thng bao gm cc
my tnh khng thun nht (heterogeneous) c kt ni vi
nhau x l song song.
PVM thng c s dng cho nhng my tnh ni mng
trong mi trng UNIX hoc Windows
3.2 Cc ngn ng lp trnh song song - PVM
159
Cc c im chnh ca PVM:Thc hin theo m hnh truyn thng ip (Message Passing)H tr s kt ni khng thun nht(Heterogeneity):PVM h trs kt ni ca nhiu my tnh, nhiu mng my tnh v nhiuloi chng trnh ng dng khc nhau.H tr a b x l:PVM s dng nhng kh nng truyn thngip trong h a b x l khai thc ht kh nng ca phncng.Tnh ton da trn tin trnh:n v iu khin thc hin songsong trong PVM l mt tc v, l mt lung (thread) lmnhim v iu khin s truyn thng v tnh ton.Thay i cu hnh theo yu cu:Cc chng trnh c th thchin trn tp cc my c la chn theo yu cu ca NSD.
3.2 Cc ngn ng lp trnh song song - PVM
160
PVM x l tt c cc vn :
nh tuyn truyn thng ip
chuyn i d liu
lp lch trong mng my tnh
H thng PVM gm hai thnh phn chnh:1. Khi pvmd (hoc pvm3) t thng tr trn tt c cc my
tnh to ra my o (cc BXL gi lp).
2. Th vin cc chng trnh con giao din ca pvm:cha cc
chng trnh con truyn thng ip, qun l cc tin trnh,
phi hp cc tc v v thay i cc my o.
3.2 Cc ngn ng lp trnh song song - PVM
161
M hnh tnh ton ca PVM
3.2 Cc ngn ng lp trnh song song - PVM
Nhp d liu v phn on
Xut d liu v hin thkt qu
My tnh 1 My tnh 2
162
Mt kin trc ca PVM
3.2 Cc ngn ng lp trnh song song - PVM
MPP
Bridge
SIMD
Cluster 1
Cluster 2
Cluster 3
http://en.wikipedia.org/wiki/Occam_programming_languagehttp://www.wotug.org/http://cs.mtu.edu/~shene/NSF-3/e-Book/index.htmlhttp://hyperlink/Message%20passing.pptxhttp://hyperlink/Thread.pptxhttp://hyperlink/Thread.pptxhttp://hyperlink/Message%20passing.pptxhttp://cs.mtu.edu/~shene/NSF-3/e-Book/index.htmlhttp://cs.mtu.edu/~shene/NSF-3/e-Book/index.htmlhttp://cs.mtu.edu/~shene/NSF-3/e-Book/index.htmlhttp://cs.mtu.edu/~shene/NSF-3/e-Book/index.htmlhttp://cs.mtu.edu/~shene/NSF-3/e-Book/index.htmlhttp://www.wotug.org/http://en.wikipedia.org/wiki/Occam_programming_language8/6/2019 Parallel Processing (1)
28/54
7/17/20
163
Phng thc thc hin chng trnh trong PVM: Nhng chng trnh vit bng C/C++, Fortran 77 c th cha
nhng li gi cc hm th vin ca PVM (y l nhng ngnng lp trnh c PVM h tr)
Cc chng trnh c dch theo kin trc ca h thng(host pool), cc tp m ch (object file) c t vo nhngni m mi my tnh ca h thng u truy cp c.
Cc USER to ra bn sao ca tc v ch (master) hoc khing mt tc v thc hin cc ng dng ca mnh.
Mt tin trnh c khi ng bi mt tin trnh khc cgi l tin trnh t (slave).
Nhng tin trnh ny c th thc hin mt s tnh ton cc bv trao i vi nhau gii quyt bi ton t ra.
3.2 Cc ngn ng lp trnh song song - PVM
164
Mt vi thng tin v PVM: User mun chy mt ng dng trn PVM th phi khi ng
PVM v to ra mt my o. Mt ng dng PVM c th c gi t du nhc ca h iu
hnh UNIX mt host no Cc Users c th cu hnh my o v thc hin cc ngdng ca mnh vi PMV
Trong PVM mi tin trnh u c th giao tip v/hoc ngb vi nhng tin trnh khc
Cc files PVM c th truy cp a ch:http://www.netlib.org/pvm3/index.html
Nhiu my tnh h tr h thng PVM thc hin nh: BBNButterfly TC2000, 80386/486 PC chy vi UNIX, ThinkingMachines CM-2, CM-5, Cray-2, Cray-5MP, HP-9000 PA-RISC, Intel Paragon, Silicon Graphics IRIS, Sun 4, DEC MicroVAX, v.v.
3.2 Cc ngn ng lp trnh song song - PVM
165
Gii thiu mt chng trnh trn PVM
Chng trnh in ra mn hnh (master) nh danh ca tc v (ID task)
nhn c t hm pvm_mytid(),
Sau s dng cc hm:
pvm_spawn(): to ra mt bn sao v gi chng trnh hello_other
pvm_recv(): nhn v thc hin khi thng ip gi n.
pvm_exit(): kt thc chng trnh trong h thng PVM.
3.2 Cc ngn ng lp trnh song song - PVM
166
Gii thiu mt chng trnh trn PVM(cont.)
Chng trnh slave:
pvm_parent() xc nh ID taskca master.
Sau dng cc hm di y xc nh hostname v chuyn
hotname ny cho master:
pvm_initsend(): khi to buffer gi
pvm_pkstr(): t mt xu vo buffer gi i
pvm_upkstr(): c mt xu vo buffer
pvm_send(): chuyn d liu buffer ti tin trnh nhn c xc
nh bi pvm_mptid.
3.2 Cc ngn ng lp trnh song song - PVM
167
/* Chng trnh master c tn Hello.c */
#include pvm3.h
main(){
int cc, tid, msg;
char buf[100];
printf(Master ID number %x\n, pvm_mytid());
cc= pvm_spawn(Hello_other,(char**)0,0,, 1, &tid);
if(cc== 1){
msg = 1;
pvm_recv(tid, msg);
pvm_upkstr(buf);
printf(From master %x: %s\n, tid, buf);
} else printf(Cannot start hello_other\n);
pvm_exit();
}
3.2 Cc ngn ng lp trnh song song - PVM
to ra mt bn sao v gichng trnh hello_other
nhn v thc hin kh ithng ip gin
c mt xu vo buffer
kt thc chng t rnhtrong h thng PVM 168
/* Chng trnh slave c tn: hello_other.c */#include pvm3.h
main(){int ptid, msg;
char buf[100];ptid = pvm_parent();strcpy(buf, Hello world from );gethostname(buff + strlen(buf), 64);
msg = 1;pvm_initsend(PvmDataDefault);pvm_pkstr(buf);pvm_send(ptid, msg);pvm_exit();}
3.2 Cc ngn ng lp trnh song song - PVM
pvm_parent() nhn mt tc vca tintrnh t ttintrnh ch
pvm_initsend() khito buffer gi
pvm_pkstr() t mt xuvo buffer gi i
pvm_send() chuyn d liu buffer titintrnh nhn c xc nhbi ptid.
http://www.netlib.org/pvm3/index.htmlhttp://www.netlib.org/pvm3/index.html8/6/2019 Parallel Processing (1)
29/54
8/6/2019 Parallel Processing (1)
30/54
7/17/20
3
175
2. Phn ph thuc d liu (Data Anti-Dependency): l s ph thuc d liu gia S1 v S2 khi DEF(S2)USE(S1) S ph thuc ny xut hin khi chng ta s dng li tn gi
ca cc bin, mt bin c s dng trong S1 v sau c nh ngha li S2.
Khi xut hin phn ph thuc d liu gia cc cu lnh thchng cng khng thc hin song song c.
Quan h phn ph thuc d liu c k hiu: V d: cc ph thuc d4, d6 . d7 l cc phn ph thuc d liu.
3.3 S ph thuc d liu v th u tin ca cc tin trnh
S1 S2 S3
d4 d6
Phn ph thuc d liu
d7
S1: A := B + C; S2: B := A + E ; S3: A := A + B
176
3. Ph thuc d liu ra (Data Output Dependency): l s ph thuc d liu gia S1 v S2 khi DEF(S2)DEF(S1) S ph thuc ny xut hin do hai nguyn nhn: S dng li tn ca cc bin (dng chung) Tnh tng gi tr ca cng mt bin.
Nu nhng lnh ny thc hin ng thi th chng s ghi cc gi tr vo cng mt nh. Do vy, cn phi xc nhchnh xc th t thc hin ngn nga vic s dngnhng gi tr khng ng.
Quan h ph thuc d liu ra c k hiu l: V d: ph thuc d5 l loi ph thuc d liu ra.
3.3 S ph thuc d liu v th u tin ca cc tin trnh
S1S3
d5
Ph thuc d liu ra
S1: A := B + C;S3: A := A + B
DEF(S1)DEF(S3) ={A}
177
4. Ph thuc d liu vo (Data Input Dependency): l s ph thuc d liu gia S1 v S2 khi USE(S2)USE(S1) Bi v cc lnh ny ch truy cp (c) v khng lm thay i gi
tr ca cc bin , do vy cc lnh ny c th thc hin theobt k th t no cng c, ngha l c th thc hin songsong.
Quan h ph thuc d liu vo c k hiu: V d
3.3 S ph thuc d liu v th u tin ca cc tin trnh
S2: B := A + E ; S3: A := A + B
USE(S3) USE(S2) ={A}
S2 S3
Ph thuc d liu vo
d3
178
3.3 S ph thuc d liu v th u tin ca cc tin trnh
5. Ph thuc iu khin d liu (Data Control Dependency): l s ph thuc d liu gia S1 v S2 khi s thc hin ca
lnh ny ph thuc vo gi tr ca cc bin c tnh lnhkia.
Quan h ph thuc iu khin d liuc k hiu l:
S4: P = (B => 0)S5: if (P is true) then
D = 1else D = 2endif
S4 S5
Ph thuc iu khin d liu
179
3.3 S ph thuc d liu v th u tin ca cc tin trnh
Nhn xt: tng c mc song song ca chng trnh chng ta cth kh hoc gim ccphn ph thuc d liuvph thuc d
liu ra trong th ph thuc d liu. Thng thng kh cc ph thuc d liu ny l t li tncho cc bin trnh vicchia s cc bin . V d: Xt on chng trnh sau:
for i = 1, n, 1X = A[i] + B[i]Y[i] = 2 * XX = C[i] * D[i]P = X + 15
endfor
for i = 1, n, 1X = A[i] + B[i]Y[i] = 2 * XXX = C[i] * D[i]P = XX + 15
endfor
Nhn xt: C phn ph thuc d liu. Nu hai lnh sau ta t X thnh XX th s tora hai on chng trnh c th thc hin song song: 180
3.3 S ph thuc d liu v th u tin ca cc tin trnh
+ S dng cc bin khc nhau loi bph thuc d liu ra.V d:
A = B + CA = C * X
A = B + CA1 = C * X
Nhn xt: Hai lnh trn c s ph thuc d liu ra. C thloi b ph thuc ny bngcchthay bin A ca cu lnh th hai bng A1
DEF(S2) DEF(S1) = {A}
8/6/2019 Parallel Processing (1)
31/54
7/17/20
3
181
3.3 S ph thuc d liu v th u tin ca cc tin trnh
+ S dng cc bin khc nhau loi bphn ph thuc dliu
V d:
Nhn xt: Hai lnh trn c quan h phn ph thuc d liu. C th loi b s ph thuc ny bng cch sdng mt bin khc cho cu lnh th hai
A = B + CB = C * X A = B + CB1 = C * X
DEF(S2)USE(S1) = {B}
182
3.3 S ph thuc d liu v th u tin ca cc tin trnh
+ Thc hin mt s php bin i (php th) cng c th loib cphn ph thuc d liu.
V d: Xt dy cc cu lnh sau:
S1: A = B + C
S2: B = A * 3S3: A = 2 * CS4: P = (B => 0)
if (P is true) thenS5: D = 1S6: else D = 2
endif
S1
S3 S2
S4
S5 S6
th ph thuc d liu
Nhn xt: x l song song, th cn thit ph i lo i b i mt s loi phthuc dliu cth.
183
V d: loi b nhng quan h phn ph thuc d liu v phthuc d liu kt qu
S2: B = (B + C) * 3S3: A = 2 * CS4: P = (B => 0)
if (P is true) thenS5: D = 1S6: else D = 2
endif
S1: A = B + CS2: B = A * 3S3: A = 2 * CS4: P = (B => 0)
if (P is true) thenS5: D = 1S6: else D = 2
endif
S3 S2
S4
S5 S6
th ph thuc d liu rt gn
S1
S3 S2
S4
S5 S6
DEF(S2)USE(S1) ={B} (phn ph thuc d liu)
184
3.3 S ph thuc d liu v th u tin ca cc tin trnh
for (i = 0; i < N; i++)A = A + i;
Nhn xt: Trong mi bc lp u phi s dng nhng tn bin khc nhau nu munloi b phn ph thuc d liu. iu ny kh thc hin c trong trng
hp s vng lp ln hoc bt k. Quan h ph thuc d liu l khng c tnh bc cu. Ngha l nu S2 phthuc vo S1, S3 ph thuc vo S2 th khng kt lun c S3 ph thuc dliu vo S1.
Tuy nhin, nhng php bin i n gin nh th khng philc no cng p dng thnh cng. V d, xt chu trnh sau:
i = 0;aa: if(i > N) stop;
a = a + i;i++;goto aa;
khng s dng FOR
185
3.3 S ph thuc d liu v th u tin ca cc tin trnh
DO iA[2*i+1] = B[i] (1)D[i] = A[2*i] (2)
ENDDO
Nu xem mng A nh mt thc th n th hai lnh trn l phthuc theo dng d liu (v A xut hin trong DEF ca (1) vcng xut hin trong USE ca (2)).Nhng nu ta xt thm ch s mng th trong cu lnh (1) sdng cc phn t mng A c ch s l; cn (2) li s dngphn t mng A c ch s chn.Do , y khng c s ph thuc d liu.
Ph thuc theo chu trnh v theo mngMc ch: kh ph thuc d liu trong cc vng lp.Xt on chng trnh sau:
hai cu lnh (1) v (2)c thc hin song songc khng??
186
3.3 S ph thuc d liu v th u tin ca cc tin trnh
Xt v d sau:FOR I=1 DO
A[I] = A[I-1] + 1 (3)
Chu trnh trn c th khai trin thnh:A[1] = A[0] + 1; A[2] = A[1] + 1;A[3] = A[2] + 1; A[4] = A[3] + 1; . . .
Vn : khi xy dng cc chu trnh lp phi xt xem nhng dylp khc nhau ca chng c th thc hin song song ckhng?
Ni cch khc, khi mt b x l P thc hin mt dy cc culnh vi i = 1 (bin iu khin vng lp) th b x l Q c th cngthc hin dy cc cu lnh vi i=2 hay khng?
8/6/2019 Parallel Processing (1)
32/54
7/17/20
3
187
3.3 S ph thuc d liu v th u tin ca cc tin trnh
Xt hai vng lp th iv thjtng ng vi I = i v I = j.Khng mt tnh tng qut ta c th gi thit i < j. Chu trnh s cdng:
. . .
A[i] = A[i-1] + 1;. . .A[j] = A[j-1] + 1;. . .
Nhn xt:Cc tp trn khng cha bin n m ch c cc bin c ch sv khi ivjkhc nhau th cc bin khc nhau.Hai tp DEF(i)= {A[i]} v USE(j) = {A[j-1]} c cha phn tchung khi i = j 1, V phng trnh i = j 1 c v s nghim
Vy kt lun chu trnh trn khng thc hin song song c.
USE(i) = {A[i-1]}
USE(j) = {A[j-1]}DEF(i) = {A[i]}DEF(j) = {A[j]}
188
3.3 S ph thuc d liu v th u tin ca cc tin trnh
Nhn xt: Ni chung, vic xc nh s ph thuc theo cch nh trn lbi ton kh. S phc tp s xut hin bi v c nhiu dngphng trnh khng hin nhin. Nhng phng trnh ny khngc li gii trc tip trong cc cng c ton hc kinh in. Trng hp tng qut, khi xt nvng lp m dn n dngphng trnh a1x1 + a2x2 + + anxn = c vi ai v c l cc snguyn (gi l phng trnh Diophantine) th kt lun chu trnhkhng th thc hin song song.
Lu : Phng trnh Diophantine c dng tng qut:a1x1 + a2x2 + + anxn = c vi ai v c l cc s nguyn.
Nm 1991 Zima v Chapman chng minh c rng nuc s chung ln nht (a1,a2, an) chia ht cho cth phngtrnh Diophantine c mt nghim nguyn.
Trng hp tng qut, gii phng trnh trn l khng th.
189
3.4 Bin i chng trnh
Mc ch:1. Loi b c s ph thuc ca cc cu lnh trong ch.trnh.2. Khc phc c nhng tr ngi ca cch xc nh cc vnglp c th thc hin song song dn ti phng trnh Diophantine.
3.4.1 Cc bin qui npBin quinp l loi bin trong chu trnh m cc gi tr l in tipca n to ra dy cp s cng.
V d: xt on chng trnh sau:
m = 0;
DO i = 1 to Nm = m + k;x[m] = a[i]
END
{m l bin qui np}
190
3.4 Bin i chng trnh
Nhn xt: Hai cu lnh trn c s ph thuc d liu v mc s dng
cho gia cc vng lp. Khi vng lp trc tnh gi tr ca mth mli c s dng
lnh th hai nn chng phi thc hin tun t.
Nu mi vng lp chng ta d bo c gi tr ca mth cths loi bc sph thucd liu lin quan n m. Gi tr kl bt bin trong cc vng lp v c cng lin tip
vo sau mi vng lp, do vy, vng lp th i, m = i * k.
m = 0;Do i = 1 to N
m = m + k;x[m] = a[i]
End
Do i = 1 to Nx[i*k] = a[i]
End
191
3.4 Bin i chng trnh
3.4.2 S ph thuc tinPh thuc tin l loi phn ph thuc d liu gia cc vnglp ca chu trnh.
Do i = 1 to Nx[i] = x[i+1]
End
Nhn xt:Chu trnh khng thc hin song song cChu trnh lm nhim v sao tt c cc phn t ca mt mngsang chnh mng vi ch s gim i mt, do vy n phi thchin tun t.Cch gii quytS dng 2 mng: mt bn gc v mt bn sao ca mng gc vit li chng trnh
Do i = 1 to Nx1[i] = x[i]
EndDo i = 1 to N
x[i] = x1[i+1]End
C th thc hin song song ?
C th
192
3.4 Bin i chng trnh
3.4.3 S ph thuc liTrng hp cc phn t ca mng trong chu trnh li cxc nh thng qua nhng phn t ng trc chng (li v
pha trc)Do i = 1 to N
x[i] = x[i-1] + y[i]End
Nhn xt:Vic loi b nhng ph thuc d liu kiu ny khng n ginKhng c gii php chung loi b ph thuc d liu trongtrng tng qut. Nu c gii php loi b th gi phi tr cngkh t.
Hin nhin khng cn s ph thuc li trong chu trnh, tuy nhinthi gian tnh ton trong mi cu lnh li tng ln kh nhiu.
Do i = 1 to Nx[i] = x[0]+y[1]+y[2]+ +y[i]
End
8/6/2019 Parallel Processing (1)
33/54
7/17/20
3
193
3.4 Bin i chng trnh
3.4.4 S phn tch chu trnh (ph thuc li)Tch mt chu trnh thnh nhiu chu trnh con thc hin song
song
Do i = 1 to N
a[i] = b[i] + c[i];c[i] = a[i-1]
End
Nhn xt: S tham chiu gia cc mng av c dn n s ph thucgia cc vng lp ca chu trnh. Gi tr c[i] lnh th nht lun l nhng gi tr lu tr c camng c (c xc nh lnh th 2) a[i-1] c s dng lnh th hai l gi tr c tnh vng lp trc (vng th i -1)
Do i = 1 to Na[i] = b[i] + c[i];
EndDo i = 1 to N
c[i] = a[i-1]End
194
3.4 Bin i chng trnh
3.4.4 S phn tch chu trnh (ph thuc tin)
Do i = 1 to N
a[i] = b[i] + c[i];c[i] = a[i+1]
End
Nhn xt: Phn t c[i] lnh hai c tnh theo gi tr ca a[i+1], l gitr trc khi n c cp nht lnh th nht Ngc li c[i] c s dng lnh th nht li l gi tr ctrc khi c cp nht lnh th hai.
Do i = 1 to Nx[i] = c[i];c[i] = a[i+1]
EndDo i = 1 to N
a[i] = b[i] + x[i];End
Cch gii quyt Sao mng c sang mng ph x. Kt qu bin i chu trnh nythnh hai chu trnh trong khng cn s ph thuc d liu
195
3.4 Bin i chng trnh
3.3.5 Cc chu trnh lng nhauXt trng hp nhiu chu trnh lng nhau (chu trnh nhiu ch s)
Do i = 1 to NDo j = 1 to N
x[i,j] = x[i, j-1] + z[i];EndNhn xt:
Mng zc s dng trong chu trnh hon ton khng gy ras ph thuc no c. Ch s th nht ca mng xl i, cng chnh l ch s ca chutrnh, nhng khng c s tham chiu cho gia cc vng lp.Vy, chu trnh lp bn trong (chu trnh lp theo i) c th thchin song song. Ch s th hai ca mng xlj, l ch s ca chu trnh th nhtdo c s ph thuc dng d liu ca cc ln lp vngngoi. Cng nh 3.4.3, trng hp ny khng c cch bini n gin v dng kh song song. 196
Bi tp
3.1 Xc nh s ph thuc d liu ca cc lnh trong chu trnhsau:Do i = 1 to N
e[i] = x[i] - z[i];
a[i+1] = e[i] + 2*d[i]a[i] = e[i]
end3.2 Hai chu trnh sau c tng ng v ni dung tnh ton hay
khng? hy bnh lun v kh nng thc hin song song cacc chu trnh .
1. Do i = 1 to N
a[i] = a[i+1] + iend2. Do i = N downto 1
a[i] = a[i+1] + iend
197
Bi tp
3.3 Xc nh tt c cc s ph thuc d liu trong onchng trnh sau:
int a[MAX];read(a);for(i = 0; i < N; i++)
for(j = 0; j < i; j++){a[i] = max(a[i], a[j]);
a[j] = min(a[i], a[j]);}3.4 Loi b cc ph thuc d liu ra v phn ph thuc d liu
ca chu trnh saufor(i = 0; i < N; i++){
x = a[i]+ b[i]);y[i] = 2 * x;
} 198
Bi tp
3.5 Phn tch on chng trnh sau, xc nh cc ph thuc d
liu v v th ph thuc d liu ca on chng trnh .A = B + C;for(i = 0; i < N; i++){
D[i] = A + b[i]);S = b[i] * 5;T = T + S;
}3.6 Vit chng trnh gii phng trnh bc hai v v th ph
thuc d liu ca n.
8/6/2019 Parallel Processing (1)
34/54
7/17/20
3
199
CHNG 4. CC M HNH V LP TRNH SONG SONG
NI DUNG
1. Lp trnh b nh chia s
2. Lp trnh song song da vo cc tin trnh
3. Lp trnh song song da vo cc lung
4. Lp trnh theo m hnh truyn thng ip.
5. Lp trnh trn cm my tnh vi PVM
200
1. LP TRNH B NH CHIA S
Mt vi ch (1/4)
1. H thng a b x l i xng SMP(symmetric multiprocessor Sysstem)
Cc b x l l nh nhau
Khng c nhng BXL c bit x l vo/ra
Khng c BXL c gn nhim v c bit no khc.2. nghin cu v XLSS, chng ta khng nht thit phi c h
a b x l vt l.
3. Trong mi trng UNIX, chng ta c th to ra nhiu tin trnh
khc nhau trong h thng v chng c s dng m
phng lp trnh a b x l.
4. Hu ht cc h UNIX u cho php to ra mt s cc tin trnh
bt k v chng c lp lch cho nhng b x l thch hp
201
1. LP TRNH B NH CHIA S
Mt vi ch (2/4)5. V nguyn tc, chng trnh l tp cc tin trnh v vic vit
chng trnh l c lp vi cc b x l, vic phn chia cc
tin trnh cho cc b x l l cng vic ca h iu hnh..
6. Trong nhng h thng a nhim nh UNIX, s b x l v s
cc tin trnh khng nht thit phi bng nhau. Nhng nn t
chc mt tin trnh c mt BXL m nhim tin vic lp lch.
7. Mt vn quan trng cn xem xt khi XLSS l cn bao nhiu
nt(BXL) chng trnh song song thc hin hiu qu nht?
8. Nhiu chng trnh c xy dng ph thuc vo mt cu
hnh xc nh. Ni chung, loi chng trnh ph thuc nh vyl khng tt, v khi b x l bn th cc tin trnh tip theo phi
ch. 202
1. LP TRNH B NH CHIA S
Mt vi ch (3/4)
Trong mi trng lp trnh chia s b nh c hai rng bucquan trng:
1. Mt tin trnh c th ch mt khong thi gian bt k gia hailnh cn thc hin. Gi s b x l Pthc hin mt chngtrnh c mt 100 lnh, b x l Qthc hin chng trnh c 10lnh v cng bt u thc hin ng thi. Thm ch, tt c cclnh c tc thc hin nh nhau cng khng th ni rng Qs kt thc trc P.
2. Khng th xem cc lnh thc hin l nguyn t mc ccngn ng lp trnh. V d, mt lnh n gin nh: a = a + 1 s
l mt dy bn lnh trong ngn ng my. M ta cng bit rng,cc tin trnh v h iu hnh ch nhn bit c cc cu lnhca ngn ng my.
203
1. LP TRNH B NH CHIA S
Mt vi ch (3/4)
Trong lp trnh b nh chia s:
Cc tc v (tasks) s c/ghi d liu t b nh chia s qua
khng gian a ch b nh chung (common address space)
C nhng c ch khc nhau nh (locks/semaphores) iu
khin vic truy nhp n b nh chia s
Ngi lp trnh khng cn m t vic truyn thng d liu, do
vic vit chng trnh s n gin.
Kh qun l d liu v khng can thip trc tip qu trnh truyn
d liu.204
1.1 LP TRNH B NH CHIA S DA VO TIN TRNH
To lp tin trnh
C php: id = create_process(N);
Mc ch: to ra N tin trnh v mt tin trnh cha
thc hin cu lnh .
Nh th, c N+1 tin trnh nh nhau c to ra
v mi gi tr ca idc gn tng ng cho
mt tin trnh.
8/6/2019 Parallel Processing (1)
35/54
7/17/20
3
205
V d: to N tin trnh v giao n+1 Nhim v thc hin ssong
id = create_process(N);
switch(id){case 0: do NhiemVu0 ; break;case 1: do NhiemVu1 ; break;case 2: do NhiemVu2 ; break;
. . .case N: do NhiemVuN ; break;
}
1.1 LP TRNH B NH CHIA S DA VO TIN TRNH
206
1.1 LP TRNH B NH CHIA S DA VO TIN TRNH
b) Hy b tin trnh:
C php:join_process(N, id);
Mc ch: cho php tin trnh c gi tr id trong N tin trnh
khi to c tip tc thc hin nhng phn vic tun t cnli, cn nhng tin trnh khc kt thc.
Nu ta t sau join_process(N, id) mt cu lnh th cu lnh
ny s khng c thc hin cho n khi tt c cc tin trnh
u thc hin join_process(N, id). Do vn x l song
song khng xut hinm l x l tun t.
207
Vn l cc tin trnh c to lp v truy cp d liu canhau nh th no?
Mt mt mt tin trnh c th mun gi mt phn d liu cc bcho ring mnh, khng cho nhng tin trnh khc truy cp tinhng d liu .
Mt khc, n cng mun trao i thng tin vi cc tin trnhkhc.
X l vn che du hay chia s thng tin nh th no cntu thuc vo m hnh m chng ta s dng:tin trnh (process) haylung(thread).
Cc tin trnh trong UNIX c s dng nh cc n v tnhton c lp. Theo mc nh, vic tnh ton v cp nht b nhca mt tin trnh l khng nhn thy c t cc tin trnh
khc. i vi lung, tt c cc thng tin, theo mc nh, l nhn thyc.
1.1 LP TRNH B NH CHIA S DA VO TIN TRNH
208
Ch (1/3):
Khi mun s dng b nh chung, Ngi lp trnh cn phi xin
cp pht b nh v sau khi s dng xong phi gii phng
chng.
C hai hm c s thc hin iu ny:
shared(m, &id):xincp pht mbyte b nh chia s cho
tin trnh id.
free_shm():gii phng b nh c cp.
Cc hm shared(m, &id), free_shm() c gi l hm iu
phi v chia s b nh.
1.1 LP TRNH B NH CHIA S DA VO TIN TRNH
209
V d: Bi ton loi tr nh
Top Related