Parallel Processing (1)

8/6/2019 Parallel Processing (1)

1/54

7/17/20

X L SONG SONGPARALLEL PROCESSING

THNG TIN V HC PHN

1. Thng tin chung:

1.1. Tn hc phn: X l song song (Parallel Processing)

1.2. M hc phn: KH.KM.515

1.3. S tn ch: 2 TC

1.4. Loi hc phn: Bt buc

1.5. Cc hc phn tin quyt: C s d liu phn tn

THNG TIN V HC PHN2. Mc tiu ca hc phn:2.1. Kin thc: cung cp cho ngi hc cc kin thcv my tnh song song, cch xy dng cc thut tonsong song.

2.2. K nng: s dng cng c lp trnh song song nhMPI, JAVA, VPM ... ngi hc phi ci t c mt sthut ton song song c bn.

2.3. Thi hc tp: ngi hc phi tham d y

cc gi l thuyt v tho lun.

THNG TIN V HC PHN3. Chnh sch i vi hc phnTham gia hc tp trn lp: i hc y , tch cc tholun ni dung bi ging, tham gia cha bi tp v chunb bi v tt - 20%.

Kh nng t hc, t nghin cu: hon thnh bi tiulun (assignment) theo tng c nhn. Bi kim tra nhgi gia k - 20%.

Kt qu thi cui k - 60%.

THNG TIN V HC PHN3. Phn b s tit:

L thuyt: 20

Tiu lun, c thm: 8

Tho lun: 2

NI DUNG CHNG TRNHPHN 1: TNH TON SONG SONG

Chng 1 KIN TRC V CC LOI MY TINH SONG SONG

Chng 2 CC THNH PHN CA MY TINH SONG SONG

Chng 3 GII THIU V LP TRNH SONG SONG

Chng 4 CC M HNH LP TRNH SONG SONG

Chng 5 THUT TON SONG SONG

PHN 2: X L SONG SONG CC C S D LIU

(c thm)

Chng 6 TNG QUAN V C S D LIU SONG SONG

Chng 7 TI U HA TRUY VN SONG SONG

Chng 8 LP LCH TI U CHO CU TRUY VN SONG SONG
http://../CSDL%20phan%20tan/Slide/Chuong%203.ppthttp://../CSDL%20phan%20tan/Slide/Chuong%202.ppthttp://../CSDL%20phan%20tan/Slide/Chuong%201.ppt


2/54

7/17/20

TI LIU THAM KHO

[0] on vn Ban, Nguyn Mu Hn,X l song song v phn tn,NXB KH&KT, 2006.

[1] Ananth Grama, Anshui Guptal George Karipis, Vipin Kumar,

Introduction to Parallel Computing, Pearson, 2003

[2] Barry Wilkingson, Michael Allen, Parallel Programming,Technigues and Applications Using Networked Workstations and

Parallel Computers, Prentice Hall New Jersey, 1999

[3] M. Sasikumar, Dinesh Shikhare, P. Ravi Prakash, Introduction to

Parallel Processing, Prentice - Hall, 2000

[4] Seyed H. Roosta, Parallel Processing and Parallel Algorithms,

Theory and Computation, Springer 1999.

[5] Michael J. Quinn, Parallel Computing Theory and Practice,

MaGraw-Hill,1994

[6] Shaharuddin Salleh, Albert Y. Zomaya, Scheduling in Parallel

Computing Systems, Kluwer Academic Publisher, 1999.

TI LIU THAM KHO

[7] Clement T.Yu, Weiyi Meng, Principles of Database Query

Processing for Advanced Applications, Morgan Kaufman Inc.,

1998. 185-225.

[8] Hasan Waqar, Optimization of SQL Query for ParallelMachines, Springer, 1995.

[9] Hong W., Parallel Query Processing Using Shared Memory

Multiprocessors and Disk Arrays, Univesity of California, 1992.

[10] Hua, K.A., Parallel Database Technology, University of

Central Florida Orlande FL 32846-2362, 1997.

[11] Zomaya A. Y. and Shaharuddin Salleh, Scheduling in

parallel computing systems, Kluwer Academic Publishers, 1999.

A CH LIN H

TS. NGUYN MU HNKHOA CNG NGH THNG TINTRNG I HC KHOA HC - I HC HU77, NGUYN HU HU

IN THOI:CQ: 054 382 6767D: 01255213579

EMAIL: [email protected]

back10

PHN 1:

TNH TON SONG SONG

Nguyn Mu HnKhoa CNTT-HKH HU

[email protected] | [email protected]

11

CHNG 1. KIN TRC CC LOI MY TNH SONG SONG

NI DUNG

1.1 Gii thiu chung

1.2 Kin trc my tnh kiu Von Neumann

1.3 Phn loi my tnh song song

1.4 Kin trc my tnh song song

12

1.1 Gii thiu chung

X l song song (XLSS) l g?

Trong x l tun t:

Bi ton c tch thnh mt chui cc cu lnh ri rcCc cu lnh c thc hin mt cch tun t

Ti mi thi im ch thc hin c mt cu lnh
mailto:[email protected]:[email protected]:[email protected]:[email protected]:[email protected]:[email protected]


3/54

7/17/20

13

1.1 Gii thiu chung (tt)

1 CPU

n gin

Chm qu !!!

14


Trong x l song song

Bi ton c tch thnh nhiu phn v c th thc hin

ng thi.

Mi phn c tch thnh cc lnh ri rc

Mi lnh c thc hin t nhng CPU khc nhau

15


Nhiu CPU

Phc tp hn

Nhanh hn !!!

16


XLSS l mt qu trnh x l gm nhiu tin trnhc kch hot ng thi v cng tham gia gii

quyt mt vn trn h thng c nhiu b x l.

Vy x l song song l g?

17


Ti sao phi x l song song?

Yu cu ca ngi s dng: Cn thc hin mt khi lng ln cng vic Thi gian x l phi nhanh

Yu cu thc t: Trong thc t khng tn ti my tnh c b nh v hnv kh nng tnh ton v hn.Trong thc t c nhiu bi ton m my tnh x l tun t(XLTT) kiu von Neumann khng p ng c.S dng h thng nhiu BXL thc hin nhng tnhton nhanh hn nhng h n BXL. Gii quyt c nhng bi ton ln hn, phc tp hn

18


S khc nhau c bn gia XLSS v XLTT:

X l tun t X l song song

Mi thi im ch thc hincmtphp ton

Mi thi im c th thchin cnhiuphp ton

Thi gian thc hin phpton chm

Thi gian thc hin phpton nhanh


4/54

7/17/20

19


i tng no s dng my tnh song song?

20


Tnh kh thi ca vic XLSS?

Tc x l ca cc BXL theo kiu Von Neumann b gii

hn, khng th ci tin thm c.

Gi thnh ca phn cng (CPU) gim, to iu kin xy

dng nhng h thng c nhiu BXL vi gi c hp l.

S pht trin cng ngh mch tch hp cao VLSI (very large

scale integration) cho php to ra nhng h phc hp c

hng triu transistor trn mt chip.

21


Nhng thnh phn lin quan n vn XLSS:

Kin trc my tnh song song

Phn mm h thng (h iu hnh),

Thut ton song song

Ngn ng lp trnh song song, v.v.

nh ngha my tnh song song (MTSS):

MTSS l mt tp cc BXL (thng l cng mt loi) kt

ni vi nhau theo mt kiu no c th hp tc vi nhau

cng hot ng v trao i d liu vi nhau.

22


Tiu ch nh gi mt thut ton song song

i vi thut ton tun tthi gian thc hin thut ton.

khng gian b nh.

kh nng lp trnh.

i vi thut ton song songcc tiu chun nh thut ton tun t.

nhng tham s v s BXL: s BXL, tc x l.

kh nng ca cc b nh cc b.

s truyn thng.

thao tc I/O.

23


John von Neumann (1903 1957): nh ton hc ngi Hungary

S dng khi nim lu tr chng trnh (stored-program concept)

Von Neumann computer c cc c im sau:

B nh c dng lu tr chng trnh v d liu

Chng trnh c m ho (code) my tnh c th hiu c

D liu l nhng thng tin n gin c s dng bi chng trnh

CPU np (fetch) nhng lnh v d liu t b nh, gii m (decode) v thc

hin tun t chng. 24


My tnh kiu V.Neumann c xy dng t cc khi c s:

B nh: lu tr d liu

Cc n v logic v s hcALU: thc hin cc php tonCc phn t x l: iu khin CU v truyn d liu I/O

ng truyn d liu: BUS

B nh

B x l

Ghi d liu c d liuCu lnh
http://en.wikipedia.org/wiki/1903http://en.wikipedia.org/wiki/1957http://en.wikipedia.org/wiki/1957http://en.wikipedia.org/wiki/1957http://en.wikipedia.org/wiki/1903


5/54

7/17/20

25


Tiu ch phn loi my tnh song song?

a) Da trn lnh, dng d liu v cu trc b nh

(Flynn)

b) Da trn kin trc: (xem 1.4)

Pipelined Computers

Dataflow Architectures

Data Parallel Systems

Multiprocessors

Multicomputers26


Michael Flynn (1966)

SISD: Single Instruction Stream, Single Data Stream

n lung lnh, n lung d liu

SIMD: Single Instruction Stream, Multiple Data Stream

n lung lnh, a lung d liu

MISD: Multiple Instruction Stream, Single Data Stream

a lung lnh, n lung d liu

MIMD: Multiple Instruction Stream, Multiple Data Stream

a lung lnh, a lung d liu

27


M hnh SISD -n lung lnh, n lung d liu

c im

Ch c mt CPU

mi thi im ch thc hin mt lnh v ch c/ghi

mt mc d liu

C mt thanh ghi, gi l b m chng trnh (program

counter), c s dng np a ch ca lnh tip theo

khi x l tun t

Cc cu lnh c thc hin theo mt th t xc nh

y chnh l m hnh my tnh truyn thng kiu Von Neumann

28


n v iukhin

B nh

BXL s hc

Lung lnhLungd liu

Lungkt qu

Tn hiu iukhin

M hnh SISD -n lung lnh, n lung d liu (tt)

V d minh ha

29


M hnh SIMD -n lung lnh, a lung d liu C mt n v iu khin (CU) iu khin nhiu n v

x l (PE) CU pht sinh tn hiu iu khin n cc n v x l

n lung lnh: cc n v x l thc hin cng mt lnh

trn cc mc d liu khc nhau

a lung d liu: mi n v x l c lung d liu ring

y l kiu tnh ton lp li cc n v s hc trong CPU,

cho php nhng n v khc nhau thc hin trn nhng

ton hng khc nhau, nhng thc hin cng mt lnh.

My tnh SIMD c th h tr x l kiu vector, trong c

th gn cc phn t ca vector cho cc phn t x l

tnh ton ng thi. 30


M hnh ca kin trc SIMD vi b nh phn tn

M hnh SIMD -n lung lnh, a lung d liu (tt)

CU

PU1 LM1

PUn LMn

DS

DS

DS

DS

ISIS

Program loaded

from host

Data sets

loaded from host

IS: Instruction Stream PU : Processing Unit

LM : Local Memory DS : Data Stream


6/54

7/17/20

31


Cc my tnh trn th trng c sn xut theo m hnh

SIMD: ILLIAC IV, DAPv Connection Machine CM-2

M hnh SIMD -n lung lnh, a lung d liu (tt)

32


M hnh MISD -a lung lnh, n lung d liu

c im: a lung lnh: c th thc hin nhiu lnh trn cng mt mc

d liu

n lung d liu: cc PU x l trn cng mt lung d liu

Kin trc kiu ny c th chia thnh hai nhm:Cc my tnh yu cu mi n v x l (PU) nhn nhng

lnh khc nhau thc hin trn cng mt mc d liu.Cc my tnh c cc lung d liu c chuyn tun t

theo dy cc CPU lin tip-gi l kin trc hnh ng-x ltheo vector thng qua mt dy cc bc, trong mibc thc hin mt chc nng v sau chuyn kt qucho PU thc hin bc tip theo.

33


M hnh MISD a lung lnh, n lung d liu (tt)

Memory:

(Program,

Data) PU1 PU2 PUn

CU1 CU2 CUn

DS DS DSIS IS

ISIS

IS

DS

I/O

MISD architecture (the systolic array)

IS: Instruction Stream PU : Processing Unit CU : Control Unit


34


M hnh MISD a lung lnh, n lung d liu (tt)

V d minh ha

35


M hnh MIMD -a lung lnh, a lung d liu

Mi BXL c th thc hin nhng lung lnh (chng trnh)

khc nhau trn cc lung d liu ring.

Hu ht cc h thng MIMD u c b nh ring v cng c

th truy cp vo c b nh chung (global) khi cn, do vy

gim thiu c s trao i gia cc BXL trong h thng.

Nhn xt:y l kin trc phc tp nht, nhng n l m hnh h trxl song song cao nhtCc my tnh c sn xut theo kin trc ny:BBN Butterfly, Alliant FX, iSPCca Intel

36


M hnh MIMD a lung lnh, a lung d liu (tt)

CU1 PU1

Shared

Memory

IS

IS DSI/O

CUn PUnIS DS

I/O

IS

MIMD architecture with shared memory


7/54

7/17/20

37


M hnh MIMD a lung lnh, a lung d liu (tt)

Nhn xt:MIMD l kin t rc phc tp nht, nhng n l m hnh h tr x l songsong cao nhtCc my tnh c snxut theo kin trc ny:BBN Butterfly, Alliant FX, iSPCca Intel 38


Mt vi nhn xt: Theo Flynn: c hai h kin trc quan trng cho cc my tnh

song song: SIMD v MIMD. Nhng kin trc khc c thxp theo hai mu .

Mc tiu ca x l song song l khaithc n mc ti acc kh nng s dng ca cc thit b phn cngnhm giiquyt nhanh nhng bi ton t ra trong thc t.

Kin trc phn cng l trong sut i vi ngi lp trnh Trong kin trc tun t c th tn dng tc cc nhanh

ca BXL thc hin x l song song theo nguyn l chias thi gian v chia s ti nguyn.

Nhng chng trnh song song trn my n BXL c ththc hin c nu c HH cho php nhiu tin trnh cngthc hin, ngha l c th xem h thng nh l a b x l.

39


Song song ha trong my tnh tun t:a. a n v chc nng: Cc my tnh truyn thng ch c mt n v s hc v

logic (ALU) trong BXL. mi thi im n ch c th thchin mt chc nng.

My tnh song songc nhiu n v x l(PE). Nhng nvnyc th cngnhauthc hin song song.V d:myCDC 6600(1964) c 10 PE c t chc trongmt BXL. Nhng n v chc nng ny c lp vi nhau vc th thc hin ng thi.

Xy dngb lp lch ti u phn chia cc cu lnh thc

hin sao cho tn dng c ti a cc n v x l cngnh cc ti nguyn ca my tnh.

40


Song song ha trong my tnh tun t (tt):

b. X l theo nguyn l hnh ng trong CPU

Cu lnh c chia thnh cc giai on (stage-phase)

Ti mt thi im c th c nhiu lnh c ti vo v cthc hin trong nhng bc khc nhau

Cc giai on thc hin khc nhau ca mi cu lnh c ththc hin gi u nhau.

u ra ca giai on ny c th l u vo ca giai ontip theo

Thc hin theo nguyn l hnh ng s hiu qu hn vkhng cn vng m d liu.

41


Song song ha trong my tnh tun t(tt):b. X l theo nguyn l hnh ng trong CPU V d: Pha 1: np cu lnh v t b nh (Instruction Fetch) Pha 2: gii m (Instruction decode) Pha 3: xc nh cc ton hng (Operand Fetch) Pha 4: thc hin cu lnh (Instruction Execute) Pha 5: lu tr kt qu (Write-Back)... qu trnh ny c th phn cho mi PE thc hin mt

cng vic. Theo cch , ti mt thi im BXL c ththc hin c nhiu cu lnh gi u nhau. Trc khimt cu lnh kt thc thc hin th cu lnh tip theo c th thc hin pha gii m, cu lnh khc li c thc np v, v.v. 42


IF: Instruction Fetch

ID: Instruction decode

OF: Operand Fetch

IE: Instruction ExecuteWB: Write-Back

Instruction i IF ID OF IE

IF ID OF IE

IF ID OF IE

IF ID OF IE

IF ID OF IE

Instruction i+1

Instruction i+2

Instruction i+3

Instruction i+4

Instruction # 1 2 3 4 5 6 7 8

WB

WB

WB

WB

WB

Cycles


8/54

7/17/20

43


V d: Thc hin tun t v hnh ng ca 2 tin trnh gm 4 giai on

Pha 1 Pha 2 Pha 3 Pha 4

Pha 1 Pha 2 Pha 3 Pha 4 Pha 1 Pha 2 Pha 3 Pha 4

Gi s mt tin trnh c chia thnh 4 giai on:

Thc hin tun t 2 tin trnh phi qua 8 giai on:

Thc hin theo hnh ng ch cn tri qua 5 giai on:



Tng thi gian tnh ton tun t l: 2 * (S1 + S2 + S3 + S4)Tng thi gian tnh ton hnh ng l: S1 + S2 + S3 + S4 + S4 44


Song song ha trong my tnh tun t(tt):

c. S gi u CPU v cc thao tc vo/ra (I/O)

Cc php vo/ra c th thc hin ng thi i vi

nhiu nhim v tnh ton khc nhau bng cch sdng nhng b iu khin vo/ra, cc knh hay

nhng BXL vo/ra khc nhau.

Nhiu my tnh hin nay c nhiu b iu khin

thit b vo/ra, cho php a x l vo/ra lm tng

c tc trao i d liu gia cc thit b ngoi

vi CPU.

45


d. Cc h thng b nhphn cp

Do tc thc hin ccphp ton trong BXL nhanhhn rt nhiu vic c dliu vo b nh trong

Cc thanh ghi c s dngtrc tip cho ALU nn bnh cache c xem nhvng m gia BXL v bnh chnh

Khi d liu c chuyn tb nh cache vo b nh

chnh th ng thi c thchuyn d liu t cache vocho CPU x l

CPU(Registers)

Cache

Main Memory

Fixed Disks

Magnetic

Tapes

Tng kh

nng lutr

Tng v tc truy cp

46


Song song ha trong my tnh tun t(tt):e. a chng trnh v chia s thi gian Thc hin song song da vo h iu hnh a nhim,

phn mm a lung, a tin trnh. H iu hnh a nhim thng gii quyt cc trng hp:

Trong cng mt khong thi gian, c nhiu tin trnhcng truy cp vo d liu t thit b vo/ra chung

Mt tin trnh tnh ton vi cng cao c th tmthi chim dng CPU lm vic, trong khi mt tin

trnh khc trc khng i hi phi kt thc cng vicsm phi ngng li.

B lp lch chia s thi gian lm nhim v phn chia

CPU cho mi tin trnh mt khong thi gian c nh To cc BXL o: mi tin trnh c cung cp mt mi

trng c xem nh mt BXL thc hin ring chotin trnh .

47


M hnh tru tng ca my tnh song song

Mc ch: mun th hin c nhng kh nng tnh ton caMTSS m khng quan tm n nhng rng buc c th ca

nhng my tnh c trong thc t.

Ch : khi xy dng cc thut ton song song, chng ta qui

c l pht trin thut ton cho m hnh tru tng ny,

sau nh x sang nhng my tnh c th vi mt s cc

rng buc no .

48


a. My tnh truy cp ngu nhin song song PRAM

Cha mt n v iu khin CU

Mt b nh chung

Mt tp khng gii hn cc BXL

Mi BXL li c b nh ring v c mt ch s duy nht

c s dng xc nh a ch trong qu trnh trao

i cc tn hiu v qun l cc ngt.

Tt c cc BXL u chia s b nh chung vi yu cu

khng b gii hn.Cc cu lnh c th bt u thc hin bt k thi im

no, bt k v tr no ca b nh (ring hoc chung)


9/54

7/17/20

49


a. My tnh truy cp ngu nhin song song PRAM

y cng l m hnh tng qut cho my tnh song song kiu MIMD

Private memory

P1

Private memory

P2

Private memory

Pn

Interconnection network

Global memory

CU

50


Mt s iu cn lu khi pht trin nhng thut toncho cc MTSS tng qut

Khng b gii hn v s lng BXL

Mi v tr ca b nh u truy cp c bi bt k BXL

no

Khng gii hn v dung lng b nh chung chia s

trong h thng

Cc BXL c th c bt k mt v tr no ca b nh,

ngha l khng cn phi ch nhng BXL khc kt

thc cng vic truy cp vo b nh.

51


Mt s iu cn lu khi chuyn nhng thut ton xydng cho MTSS tng qut sang my tnh c th Phi p dng mt s cc rng buc m bo chng

trnh thc hin c trn nhng my tnh . V hnh thc, phi thc hin mt trong nhng iu kin

sau: EREW: loi tr vn xung t c/ghi

(Exclusive Read + Exclusive Write)

CREW: cho php c ng thi, nhng khng chophp xung t khi ghi(Concurrent Read + Exclusive Write)

CRCW: Cho php c, ghi ng thi(Concurrent Read + Concurrent Write)

52


b. Kin trc SIMD

Cu trc:

Cc phn t x l (PE) u c iu hnh bi mt

n v iu khin (CU).

Cc phn t x l nhn c cng mt lnh t CU

nhng hot ng trn nhng tp d liu khc nhau.

c tnh:

Phn tn vic x l trn nhiu phn cng

Thao tc ng thi trn nhiu phn t d liu

Thc hin cng mt tnh ton trn tt c cc phn t d

liu.

53


b. Kin trc SIMD (tt)

CU

PE2

IS

PEn

PE1

.

.

.

Global memory

Result

IS

DSn

DS2

DS1

M hnh kin trckiu SIMD

IS: Instruction Stream PE : Processing Element


54


V d

X1

X2

X4

X3X= No

Yes

X1

X3

X4

X2

Mt s PE kim tra X= ,mt s khc ri

Mt s PE kim tra X ,mt s khc ri

Tt c PE

(a) thc hin theo SISD,

(b) thc hin theo SIMDTrong , X1, X2, X3, v X4 l cc

khi cc cu lnh


10/54

7/17/20

55


b. Kin trc MISD

BXL hnh ng chnh l BXL kiu MISD

Nguyn l hnh ng(pipeline):da trn nguyn tc:

Phn on hoc chia nh mt tin trnh thnh mt s tin

con thc hin trong cc pha lin tip. Mi mt giai on ca mt tin trnh c thc hin tun t.

Sau khi thc hin xong mt pha th bt u thc hin giai

on ca tin trnh tip theo.

Mi pha thc hin xong s truyn kt qu cho pha tip theo.

Tm li, theo nguyn l hnh ng:

Khi mt giai on cng vic ang thc hin th mt giai

on khc c th np d liu vo, v d liu vo ca giai

onny cth l kt qu ca giai on trc n.56


V d: Thc hin tun t v hnh ng ca 2 tin trnh gm 4 giai on


Pha 1 Pha 2 Pha 3 Pha 4 Pha 1 Pha 2 Pha 3 Pha 4

Gi s mt tin trnh c chia thnh 4 giai on:

Thc hin tun t 2 tin trnh phi qua 8 giai on:

Thc hin theo hnh ng ch cn tri qua 5 giai on:



Tng thi gian tnh ton tun t l: 2 * (S1 + S2 + S3+ S4)Tng thi gian tnh ton hnh ng l: S1 + S2 + S3 + S4 + S4

57


Nguyn l hnh ng c th p dng theo hai mc:- Hnh ng theo n v s hc:

Cc n v s hc v logic ALU c t chc thnh mng,cc php ton bn trong c thc hin theo nguyn lhnh ng.

CUALU

ALU. . .

ALU

B nh

X l hnh ng theo ALU58


- Hnh ng theo n v cu lnh:Cc n v iu khin CU c phn on v t chc theohnh ng.

CU

. . .CU

CU

ALU

B nh

X l hnh ng theo CU

59


Xy dng hnh ng vng trn gia cc BXL, b nh v mnglin kt

Php ton thc hin bi CU theo kin trc ny c th chia thnh 5 giai on:G1: c d liu: c d liu t b nh chia s (Shared Memory).

G2:Chuyn ti d liu: chuyn d liu t b nh ti cc phn t x l PEthng qua mng c (Read Network).G3. Thc hin cu lnh: s dng PE thc hin cc cu lnh.G4. Chuyn ti d liu: chuyn cc kt qu t cc PE ti b nh

thng qua mng ghi (Write Network).G5. Lu tr d liu : ghi li cc kt qu vo b nh chia s.

SharedMemory

SharedMemory

WriteNetwork

ReadNetwork

V d v mt hnh ng vng trn

60


Mt v d n gin

Tnh tng n phn t ca mt mng a:

For (i=0; i


11/54

7/17/20

61


Tnh tng n phn t ca mt mng a:

sin = sout + a[i]

sin sout

a[0] a[1] a[2] a[3] a[4]

Thc hin theo ng ng

sin sout sin sout sin soutin sout

62


V d: Sp xp n phn t bng k thut ng ng

Recv(&number, Pi-1);

If (number > x) {send(&x,Pi-1);

x= number;

} else send(number, Pi+1);

tng thut ton (insertion sort): P0 nhn mt dy cc s

Lu tr s ln nht Chuyn cc s nh hn i Nu s nhn c ln hn s lu tr hin thi th thay thn v chuyn s nh hn i

63


4,3,1,2,5

5

5

5 2

5 3 1

5 4 2

5 4 3 1

5 4 3 2

5 4 3 2 1

P0 P1 P2 P3 P4

4,3,1,2

4,3,12

4,31

14 5 2

3

4 2

3 1

2

1

64


c. B x l mng tm thu SAP (Systolic Array Processor)

Do Kung vLeiserson xut 1978

L mt mng cc phn t x l d liu (DPU) c kt ni

cc b vi nhau.

Mi DPU c xem nh mt t bo, mt trong mng, bao

gm:

Mt s thanh ghi (registers)

Mt b cng (adder)

Cc mch iu khin

n v logic-s hc ALU.

65


c. B x l mng tm thu SAP (cont.)

SystolicArray Controller

HostProcessor

Tn hiu

D liu vo

Kt qu

Kin trc b x l mng tm thu

Trong kin trc SAP,B iu khin (Controller) lm nhim v giao din cho BXLchnh (Host Processor) v gi cc tn hiu iu khin qutrnh vo/ra d liu cho SA.Hot ng ca h thng theo tng nhp v lp li mt cchu n tn dng c kh nng song song ca tt ccc phn t x l 66


c. B x l mng tm thu SAP (tt)

DPU:DataprocessingUnits
http://hyperlink/Leiserson.pptxhttp://hyperlink/Leiserson.pptxhttp://hyperlink/Leiserson.pptxhttp://hyperlink/Leiserson.pptxhttp://hyperlink/Leiserson.pptx


12/54

7/17/20

67


c. B x l mng tm thu SAP (cont.)

SA c th t chc theo nhiu cu hnh tp khc nhau

(a) mng tuyn tnh

(b) mng hnh tam gic

(c) mng hai chiu hnh vung68


c. B x l mng tm thu SAP (tt)

Xt bi ton nhn 2 ma trn c 2x2: AxB=C

S dng b nh SAP hai chiu hnh vung tnh

22221221222122112121

22121211122112111111

2221

1211

2221

1211

2221

1211

****

****

*

*

babacbabac

babacbabac

bac

cc

cc

bb

bb

aa

aa

k

kjikij

69


4 5 6

7 8 9

1 2 3

a11 a12

a21

a22

b22

b21b12

b11

c11

c12

c21

c22

Nhp theo ct

Nhp theo hng

Nhp 1:Nhp a11, b11 vo s 1 v tnh a11* b11Nhp b21 vo s 4 v a12vo s 2

Nhp 2:Truyn b11 t s 1 sang s 2Truyn a11 t 1 sang 4 v tnh a11* b21Truyn b11 t 1 sang 2 v tnh a12* b11Truyn b12 t 4 sang 5 v a12 t 2 sang 5 v tnh a12* b21 . Tnh c11=a12*b11+a12*b21Nhn tip b12vo 4 v a21 vo s 2

Nhp 3:Truyn c11 t s 5sang s 9Truyn a21 *b11 t 2 sang 6

Truyn b12 t 5 sang 6Nhp a22 vo 3 v nhp b22 vo 7

Nhp 4:Truyn a22 t s 3 sang s 6

Tnh a22* b21Truyn a21 *b11 t 2 sang 6Cng dn kt qu c cuyn t 2:S cho c21=a21 *b11 +a22* b21Chuyn c11 t 9 ra v gn cho c11

70


d. Kin trc MIMD Loi a BXL hoc cn gi l h thng a my tnh Trong mi BXL c n v iu khin (CU) ring v thc

hin chng trnh ring ca mnh.

Nhng c trng ca MIMD: X l phn tn trn mt s BXL c lp Chia s vi nhau mt s ti nguyn, trong c h thng

b nh. Mi BXL thao tc c lp v c th thc hin ng thi vi

nhau. Mi BXL c th thc hin mt chng trnh ring.

71


CU 1

PE2

IS1

PEn

PE1

.

.

.

CU 2

CU n

.

.

.

DS

IS2

ISn

Kin trc MIMD72

Cu hi chng 1

1. Nu c trng c bn ca kin trc my tnh tun t ca VN2. Cc kin trc my tnh c th c phn loi nh th no?da vo nhng yu t no phn loi?

3. Mt h thng nh th no c gi l my tnh song song?4. My tnh kiu MIMD khc vi mng cc my tnh nh thno?5. Nu nguyn l x l theo hnh ng. Nhng bi ton c nhngtnh cht g th thch hp vi kin trc x l hnh ng?6. Cn bao nhiu nhp thc hin nhn hai ma trn 100 100trn SAP c 100100 phn t x l? Nu s dng h10001000 PE th h no tt hn (nhanh hn)?7. Mt cng vic c chia thnh m cng vic con, mi cngvic con i hi mt n v thi gian thc hin. Hi cn baonhiu n v thi gian h hnh ng gm m-b x l tuyntnh (gm m-pha thc hin) thc hin c nhim v chotrc?


13/54

7/17/20

73

CHNG 2. CC THNH PHN CA MT SONG SONG

NI DUNG

2.1 B nh ca MTSS

2.2 Mng kt ni cc thnh phn ca MTSS

2.3 Chng trnh dch v h iu hnh

2.4 Kt lun

74

2.1 B nh ca MTSS

Mt s vn cn quan tm trong kin trc MTSS

S dng nhiuthanh ghi:

s lm gim hiu ng ph ca cc ngt

S dng khng gian nh ln: lm gim hiu ng ph ca s i ch khi thc hin mt

lnh. tng hiu qu trao i d liu ca h thng.

Xy dng b lp lch (Scheduling): nhm cp pht hu hiu

tng nhim v n l cho cc BXL cho mt cch ng .

ng b cc BXL (Synchronization):iu khin nhiu tin trnh

hot ng ng thi, cng truy cp n mt s hu hn cc ti

nguyn dng chung, trnh c s tc nghn (deadlock)

75

2.1 B nh ca MTSS

Mt s vn cn quan tm trong kin trc MTSS (cont.)Thit k cu hnh mng:tp trung vo vic kt ni gia BXLvi BXL, gia BXL vi b nh trong h thng. Cu hnh tpca mng kt ni l vn rt quan trng trong thit k hthng song song.

Phn on( Partitioning): xc nh mc song song trongcc thut ton xc nh c cc lung x l ng thi.S phn on c th th hin nhiu mc khc nhau: mclnh, mc th tc, hoc mc chng trnh, v.v.

m bo tin cy (Reliability):nu c mt BXL no khngthc hin c cng vic m nhim th mt BXL khc sc thay th m bo trong mi tnh hung cng vicchung vn c hon thnh.

76

2.1 B nh ca MTSS

B nh ca MTSS c chia thnh nhiu mcB nh mc 1: mc cao nht, c dung lng nh nht, truycp nhanh v t nht, thng gn cht vi mi BXL thnhb nh cc b (local memory-khc vi b nh chia s).B nh mc 2: Truy cp chm hn v r hn mc 1, v.v.V nguyn tc, d liu c chuyn i gia cc mc lncn ca cc b nh v hon ton c iu khin bi b nhmc 1.V l thuyt, trong cu trc phn cp b nh, tc truy cpb nh trung bnh gn bng tc truy cp mc cao nht(mc 1), nhng chi ph ca cc n v nh trung bnh li gnvi gi ca b nh mc thp nht (mc n).

77

BXL

B nh mc 1

B nh mc n

B nh mc 2

Mc thp nht

Mc cao nht Nh, nhanh v t

Ln, chm v r

2.1 B nh ca MTSS

Phn cp h thng b nh

78

2.1B nh ca MTSS

2.1.1 B nh kt hp (AM Associative Memory)AM bao gm cc nh (cell)

sq

k

a

mR/W

Select

Key

Argument

Read/Write

Output

Match

Mi nh ca AM c 4 u vo v hai u raCc u vo (input) ca mi nh bao gm: . Bit i s a,

. Bit c/ghi R/Wxc nh thao tc tng ng cn thc hin

. Bit kho k

. Bit la chn s xc nh nh thch hp cho vic thc hin c/ghi.Hai kt qu u ra:. Bit i snh mch ra d liu c lu trong b nh c i snh cvi bit i s a.. Bit kt qu ra q.
http://hyperlink/thanh%20ghi.pptxhttp://hyperlink/Ngat.pptxhttp://hyperlink/lenh.pptxhttp://hyperlink/B%E1%BB%99%20nh%E1%BB%9B%20c%E1%BB%A7a%20MT%20tu%E1%BA%A7n%20t%E1%BB%B1.pptxhttp://hyperlink/B%E1%BB%99%20nh%E1%BB%9B%20c%E1%BB%A7a%20MT%20tu%E1%BA%A7n%20t%E1%BB%B1.pptxhttp://hyperlink/B%E1%BB%99%20nh%E1%BB%9B%20c%E1%BB%A7a%20MT%20tu%E1%BA%A7n%20t%E1%BB%B1.pptxhttp://hyperlink/lenh.pptxhttp://hyperlink/Ngat.pptxhttp://hyperlink/thanh%20ghi.pptx


14/54

7/17/20

79

2.1 B nh ca MTSS - Cu trc ca b nh kt hp

0

1...

m-1

Argument Register Mask Register

Buffer Register

TagsMatch

Register

Input Input

Output

Array of memory locations

0 1 . . . n-1

80

2.1 B nh ca MTSS

V d, tm trong AM tt c cc t c 8 bit cao nht cha gi tr

(1101 1100)2 v tr li t u tin c tm thy.

Gi tr (1101 1100)2 l i s tm

Thanh ghi nh du (mask register) l 8 bit cao nht. Qu

trnh tm kim thc hin nh sau:1. T cn tm (1101 1100)2 c np vo thanh ghi i s

Argument Register

2.on m chng ta quan tm l 8 bit cao nht, nhng bit

ny c a vo thanh ghi nh du Mask Register

nh du.

3. Tt c cc t trong b nh c so snh song song vi

thanh ghi i s.

81

2.1 B nh ca MTSS

2.1.2 M hnh b nh truy cp ngu nhin song song

B nh PRAM gm:

m vng b nh ln chia s cho p b x l.

c s dng lu tr d liu v l vng trao i d liu

gia cc BXL.

ccBXL c th truy cp vo b nh PRAM ng thi v c th

hot ng mt cch d b.

V d, b x l Pi ghi d liu vo mt vng nh v d liu ny c

th c Pj truy cp, ngha l Piv Pjc th dng b nh chung

trao i vi nhau.M hnh loi ny c cc phng thc truy cp sau:

82

2.1 B nh ca MTSS

Cc phng thc truy cp b nhConcurrent Read (CR): nhiu BXL c th c ng thi cngmt nh.

Exlusive Read (ER):p BXL c c pvng nh khc nhauv mi BXL c c duy nht mt vng nh v mi vng nhc c bi mt BXL.

Concurrent Write (CW):cng mt thi im cho php nhiuBXL ghi vo cng mt vng nh.

Exlusive Write (EW):p BXL ghi c vo pvng nh khcnhau. Mi BXL ghi c vo mt vng nh v mi vng nh

c ghi bi mt BXL.Ngitaphn CWthnh cc loinh sau:

83

2.1 B nh ca MTSS

Ghi ng thi c u tin (Priority CW):cc BXL c gn mc utin v khi c nhiu BXL mun ghi d l iu vo mt vng nh th utin cho BXL c mc u tin cao nht. Cc mc u tin c th gn

tnh hoc ng theo nhng qui tc c xc nh khi thc hin.Ghi ng thi chung(Common CW):Khi cc BXL ghi cng mt gitr th chng c php ghi vo cng mt vng nh. Trong trnghp ny, mt BXL s c chn ghi d liu .Ghi ng thi t do (Arbitrary CW):mt s BXL mun ghi d liung thi vo mt vng nh, nhng c mt BXL c php thay igi tr ca vng nh . Trong trng hp ny, chng ta phi ch racch la chn BXL thc hin vic ghi.Ghi ng thi ngu nhin (Random CW):BXL c la chn ghi d liu l ngu nhin.Ghi ng thi t hp (Combining CW):tt c cc d liu m ccBXL nh ghi ng thi vo b nh c t hp li thnh mt gi tr.Gi tr ny s c ghi vo b nh . 84

2.1 B nh ca MTSS

2.1.2 B nh chia s (Share Memory)c trng:

Dunglng ln

CcBXL u c th truycp vo SM

CcBXL c th hot ng c lp nhng lun chias a ch

cc nh(khng c c quyn)

Nhngthayi nidung nh cthc hin bi mtBXL

no scnhn thy bi ccBXL khc.

Cc my tnh sdng b nhchias cphn lm 2loi:

UMA (Uniform MemoryAccess) NUMA (NonUniform MemoryAccess)

Share Memory
http://hyperlink/b%E1%BB%99%20nh%E1%BB%9B%20AM.pptxhttp://hyperlink/Truy%20c%E1%BA%ADp%20d%E1%BB%8B%20b%E1%BB%99.pptxhttp://hyperlink/Truy%20c%E1%BA%ADp%20d%E1%BB%8B%20b%E1%BB%99.pptxhttp://hyperlink/b%E1%BB%99%20nh%E1%BB%9B%20AM.pptx


15/54

7/17/20

85

2.1 B nh ca MTSS

a. M hnh UMA ca b nh chia s

c im:Cc BXL lm vic nh c ch chuyn mch tp trung (central

switching mechanism) iu khin vic truy cp ti b nh

chia s. Cc BXL u c th truy cp n b nh chia s ton cc

(global shared memory)

Thi gian truy cp vo b nh l nh nhau i vi mi BXL.

Mt BXL ny c th trao i thng tin vi mt BXL khc bng

cch ghi thng tin vo b nh ton cc v BXL kia s c d

liu ti cng v tr trong b nh.

86

2.1 B nh ca MTSS

a. M hnh UMA ca b nh chia s (tt)

Nu tt c cc BXL ca my tnh u c thi gian truy cp n

cc thit b l nh nhau th gi l my tnh a b x l i xng

- Symmetric MultiProcessor (SMP) machines

My tnh vi kin trc UMA cn c gi: CC-UMA - Cache

Coherent UMA

P1

Switching mechanism

I/O

P2

Pn

Memory banks

C1 C2 Cn

Pi

Ci

Processor i

Cache i

87

2.1 B nh ca MTSS

a. M hnh UMA ca b nh chia s (tt)

88

2.1 B nh ca MTSS

b. M hnh NUMA ca b nh chia sc im B nhchias c phn tn v chiathnh cc moun nhc lp. B nh chia s c phn tn cho tt c cc BXL thnh bnh cc bv tt ccc moun nh s l b nh chung chocc BXL. M hnh NUMA thng c to thnh t hai hoc nhiuSMPs ni vi li vi nhau bi mt ng truyn vt l. Mt SMP c th truy cp trc tip n mt SMP khc Cc BXL c php truy cp ng thi ti mt hay nhiu m

un nh v cth hotng c lp vi nhau. Khng phi tt c cc BXL u c thi gian truy cp n ccbnh l nh nhau. Truy cp b nh qua link s chm hn

89

2.1 B nh ca MTSS

b. M hnh NUMA ca b nh chia s (tt)

Network

Cache

P

Mem Cache

P

Mem

Cache

P

Mem Cache

P

Mem

NUMA vi Distributed Memory

90

2.1 B nh ca MTSS

2.1.3. B nh phn tn

c im:

C mt ng kt ni giacc BXL

Cc BXL u c b nh ring.

a ch b nh ca mt BXL no khng c bit bi BXLkhc. Do khng c khi nim khng gian a ch tng th(global address space) qua cc BXL khc.

V BXL ny khng th t do c/ghi vo bnhcamt BXLkhc nn khi nim Cache coherent khng p dng y.

Khi cn thit ngi lp trnhc nhim v chuynd liu t nh ca BXL ny sang nh ca mt BXL khc.
http://hyperlink/Cache%20coheren.pptxhttp://hyperlink/Cache%20coheren.pptxhttp://hyperlink/Cache%20coheren.pptxhttp://hyperlink/Cache%20coheren.pptx


16/54

7/17/20

91

2.1 B nh ca MTSS

2.1.3. B nh phn tn

92

2.1 B nh ca MTSS

2.1.3. B nh a my tnh

Mi nttrongh thng a my tnh cng chnh l mt my tnhc b nh ring, khngchias vi nhngBXL khc. Cc BXL traoi vinhau thng qua vic gi v nhn cc thng ip (messages)Khng tn ti b nh chia s chung m mi BXL c b nh ccb ring ca chng.Vic trao i d liu trong mng theo m hnh point to pointthng qua s lin kt tnh gia cc BXL.

Memory BXL

BXLMemory

Memory BXL

Memory BXL

c im:

93

2.1 B nh ca MTSS

2.1.4 My tnh vi b nh lai (Hybrid Memory) Kt hp gia kin trc chia s b nh chung v b nh phn

tn.

H thng l nhiu cm my tnh khc nhau. Mi cm l cc

my tnh vi b nh c kin trc chia s v c b nh ton cc

ring cho cm .

Cc BXL trong mt cm chia s b nh chung c th truy cp

n b nh ton cc ring ca cm .

Thnh phn b nh phn tn c hiu nh l mt cm cc

b nh ton cc ca mi cm.

Cc BXL ch c th truy cp n b nh ton cc trong thnhphn chia s b nh phn tn ca chng, ch khng truy cp

c b nh ca cc thnh phn chia s b nh chung khc. 94

2.1 B nh ca MTSS

6. My tnh vi b nh lai

Hybrid Distributed-Shared Memory

95

2.1 B nh ca MTSS

Tm li,Hai iu cn phi quan tm trong thit k b nh ca h thngsong song. l,

Bng thng(bandwidth): ng kt ni cc BXL, b nh tr b nh(memory latency), tr b nh c hiu nhl khong thi gian t khi BXL yu cu d liu t b nhn khiBXL nhn c d liu tng ng.

Trong h thng my tnh song song bng thng phi rng thc hin vic truyn thng ca cc BXL.

Khi cc mun nh c chia s th v ic cnh tranh s dngkhng gian nh phi c cc tiu ha. Do , tr b nhcng phi c cc tiu ha.

96


Mt vi nhn xt:Trong hu ht cc kin trc song song nh: nhng h

a BXL chia s b nh, a b x l a b nh , v.v. thvn quan trng nht trong thit k l xc nh s kt

ni gia cc BXL v b nh vi nhau.

Mt kin trc l tng l kin trc trong , mi BXL

u kt ni c vi cc BXL cn li. Khi n to ra

mt th y .

V d, nu h c p BXL th s c p*(p 1)/2 ng

lin kt. D nhn thy kin trc loi ny s rt phc

tp, nht l khi p ln.


17/54

7/17/20

97


C hai loi cu hnh tp cho mng lin kt: lin kt

tnh v lin kt ng.

Mng lin kt tnh l mng cc thnh phn ca hthng my tnh, trong cc BXL, b nh c lin

kt vi nhau mt cch c nh, khng thay i c.

Mng lin kt ngl mng cc thnh phn ca h

thng my tnh, trong s lin kt gia cc BXL, b

nh c th thay i c.98


2.2.1 Lin kt tuyn tnh v vng (linear and ring)

a. Lin kt tuyn tnhCc BXL c lin kt vi nhau theo dy v c nh s theo

th t tng dnP0 P1 Pn-1. . .

Nhn xt:

Trong mng ln kt tuyn tnh, tr hai phn t u v cui, tt

c cc BXL bn trong u c hai lng ging l BXL trc v sau

n.

y l dng lin kt n gin, nhng d liu cng cn phi

chuyn qua nhiu BXL, do s truyn thng d liu gia cc

BXL, c bit l gia BXL u v cui s b chm li khi s BXL

kh ln.

99


2.2.1 Lin kt tuyn tnh v vngb. Lin kt vng

c t chc tng t nh lin kt tuyn tnh nhng BXL uv cui c ni vng vi nhau

Trong mng lin kt vng, s trao i gia cc BXL c th thchin theo mt chiu, gi l mng n, hoc theo c hai chiu gil mng kp.

Tt nhin s truyn thng trong mng lin kt vng, nht l gianhng BXL xa nhau th cng vn b tr.

P2

P6 P5

P3

P4P0

Nhn xt:

100


2.2.2 Mng lin kt xo trn hon ho (Perfect Shuffle Exchange)Gis c N BXL: P0, P1, , PN-1, vi N l ly tha ca 2.Trong mng lin kt xo trn hon ho, ng lin kt mtchiu tPi tiPjc xc nh nh sau:

2i vi 0 i N/2 - 12i+1-N vi N/2 i N - 1

Mng lin kt xo trn hon ho mt chiu vi N = 8=2 3,trong s lin kt xo trn c k hiu bng mi tn.

j = {

P0 P1 P2 P3P4 P5

P6 P7

101


Mt biu din khc ca mng lin kt xo trn hon ho

P001

P010

P011

P100

P000

P101

P110

P111

P001

P010

P011

P100

P000

P101

P110

P111

102


2.2.2 Mng lin kt xo trn hon ho hai chiu

Mng lin kt xo trn hon ho hai chiu vi N = 8,trong s lin kt xo trn c k hiu bng mi tn hai chiu.

P0 P1 P2 P3P4 P5

P6 P7


18/54

7/17/20

103


2.2.3 Mng lin kt li hai chiu (two-dimentional mesh)c im:Mi BXL c lin kt vi bn lng ging: trn, di, bn tri vbn phi.Mng lin kt lihaichiuc s dng thit k cc mytnh ILLIAC IV, MPP (Massively Parallel Processor), DAP (ICLDistributed Array Processor), v.v.C hai dng lin kt li:

(a) Li khng quay vng (b) Li quay vng104


2.2.4 Mng lin kt siu khi hoc hnh khi n-chiuGi s c n b x l: P0, P1, , Pn-1, vi n = 2q, q 0. Nu miBXL cn lin kt vi ng qb x l ln cn th c th dngmng lin kt theo hnh siu khi n chiu.Trong mng lin kt hnh khi cc ch s ca cc BXL cnh nh phn, hai BXL c gi l lng ging vi nhau nunhn ch s ca chng sai khc nhau ng mt bit.

Mng lin kt hnh khi vi n=8 (q=3)

2

P0000

5

1

4

76

3

P001

P100

P101

P111P110

P010 P011

105


Mng lin kt hnh khi vi n=16 (q=4)

2

0

5

1

4

76

3

2

0

5

1

4

76

3

5 = 0101

1 = 0001

4 = 010013 = 1101

106


2.2.5 Mng lin kt hnh sao (star) K hiu l Sn C n! b x l Nhn ca mi BXL s tng ng vi mt hon v ca n k hiu. BXL Pi c kt ni vi BXL Pj j nhn c t i bng cchthay k hiu th k ca hon v n phn t 1,..., n, vi 2kn.

P1234

P3214 P2134

P3124

P1324

P2314

V d, trong S4 (n = 4) c 4! = 24 BXL.Hai BXL P2134 v P3124 l c lin kt vinhau v 3124 nhn c t 2134 bngcch i ch v tr u (s 2) vi v trth ba (s 3). Tng t suy ra P2134 vP4132 l clin kt vi nhau.

107

Mng lin kt hnh sao vi 24 b x l

P3412

P4312 P1432

P1342 P4132

P3142

P4231

P3241 P2431

P2341 P3421

P4321

P1234

P3214 P2134

P2314 P3124

P1324

P2413

P4213 P1423

P1243 P4123

P2143

108


2.2.5 Mng lin kt Butterfly

(k+1)2k nt c chia thnh (k+1) ranks, mi rank cha n=2k nt.

Cc ranks c nh s t 0 n k

K hiu Node(i,j): nt th j trn rank th i

Node(i,j) c ni vi 2 nt trn rank i-1: node(i-1,j) v node (i-1,m),

y m l s nguyn c c bng cch o bit th i trong xu nh

phn biu din j

Nu node(i,j) c ni n node(i-1,m), th node (i,m) c ni vi

node(i-1,j)


19/54

7/17/20

Rank 0

Rank 1

Rank 2

Rank 3

Node(1,5): i=1, j=5

j = 5 = 101 (binary)

i=1

001 = 1

Node(1,5) c ni vinode(0,5) v node(0,1)

0 1 2 3 4 5 6 7

Mng lin kt Butterfly vi 4 ranks v 32 BXL


2.2.5 Mng lin kt Butterfly (tt)

109

Mt thp chiu cao d gm (4d+1-1)/3BXL c phn tn trong d+1 mc ccc tnh cht sau:

C 4d-2

BXL mc d. C 4d-1 BXL mc d-1, v

C 4d BXL mc d-2

Ch c 01 BXL mc d

i vi mc x:

c ni vi 04 nt lng ging cng mt mc nu x


20/54

7/17/20

C k tng.

Mi tng gm cc thanh cho c m input v n output. tng

cng m*n thanh cho. Mng bao gm mk input v nk output; do n mng cmk*n

k chuyn mch.

Cc kt ni chuyn mch cho php to mt ng dnchnh xc t cc input n cc output.

Nu A l a ch ch ca mt kt ni mong mun trn c sca n, th cc s ca A biu din cc thanh cho t ra thit lp kt ni theo d kin .


2.2.10 Mng lin kt Delta

115


2.2.10 Mng lin kt Delta

116

mng delta 3 tng, c 23*23 ng kt ni, dng 2*2 thanh cho.Mi tng c 23 input v 23 output.

tng 1 tng 2 tng 3

M0

M1

M2

M3

M4

M5

M6

M7

P0

P1

P2

P3

P4

P5

P6

P7

117


2.2.8 Mt s mng kt ni khc

Tham kho:[6] M. Sasikumar, Dinesh Shikhare, P. Ravi Prakash, Introduction to ParallelProcessing, Prentice Hall. 2002[7] Seyed H. Roosta, Parallel Processing and Parallel Algorithms, Theory andComputation, Springer, 2006.[8] Michael J. Quinn, Parallel Computing Theory and Practice, MaGraw-Hill,2004

118

2.4 Kt lun

Vn trng tm ca chng 21. Tm hiu cc ti nguyn v mi quan h ca chng trong h

thng tn dng c ht kh nng x l song song.2. Cc b nh c t chc thnh b nh kt hp, b nh truy

cp ngu nhin, b nh chia s, v.v. l cc m hnh chnh chovic thit k b nh MTSS.

3. Vn quan trng trong thit k kin trc ca MTSS l xcnh cch kt ni cc b x l vi nhau sao cho hiu qunht.

4. Cc b x l c th kt ni theo mng lin kt tnh hoc linkt ng.

5. Kho st mt s cu hnh tp ca mng lin kt cc b xl nh: lin kt tuyn tnh, lin kt xo trn, lin kt li, linkt hnh sao v lin kt hnh khi...

119

2.3 Chng trnh dch v cc h iu hnh

Cu hi cui chng1. Nu nhng vn cn quan tm khi thit k kin trc my

tnh song song

2. B nh kt hp l g? nu nguyn l hat ng ca b nhkt hp.

3. Ti sao mng lin kt li ng vai tr quan trng trong kintrc MTSS?

4. Da vo nh ngha chung ca mng lin kt hnh khi xydng cu trc tp cho mng lin kt hnh khi cho 16 b xl.

5. Nu nhng c trng c bn ca chng trnh dch songsong?

6. Nu mc ch chnh ca h iu hnh cho my tnh songsong?

7. Xy dng mng lin kt theo m hnh xo trn hon ho cho16 phn t. 120

Tiu lun

1. Kin trc kt ni mngDe Bruijn, Mng cy nh phn, Mng Delta, Mng Butterfly,Mng Omega, Mng Pyramid ([7] Seyed H. Roosta, Parallel Processing

and Parallel Algorithms, Theory and Computation, Springer, page 72-80)

2. Chng trnh dch song song ([7] Seyed H. Roosta, ParallelProcessing and Parallel Algorithms, Theory and Computation,Springer, page 83-90)

3. H iu hnh a x l ([7] Seyed H. Roosta, Parallel Processing andParallel Algorithms, Theory and Computation, Springer, page 91-100)

4. Tm hiu h iu hnh Linus-Windows (lin quan n a lung atin trnh)


21/54

7/17/20

121

CHNG 3. GII THIU V LP TRNH SONG SONG

NI DUNG

1.Nhng khi nim c s ca lp trnh song song

2.Cc ngn ng lp trnh song song

3.Cc loi ph thuc d liu trong chng trnh

4.Phng php bin i chng trnh tun tsang chng trnh song song

122

3.1 Gii thiu chung

S khc nhau c bn gia LT tun t v LT song song

Trong mi trng lp trnh tun t:

Cc cu lnh c thc hin tun t.

Mi chng trnh thc hin s to ra nhng tin trnh bn

trong h thng m ngi lp trnh khng quan st c.

Mi cu lnh thc hin khng gy tr ngi cho cc cu lnh

khc trong chng trnh.

123

3.1 Gii thiu chung

Trong mi trng songsong:Cc cu lnh ca chng trnh c th thc hin an xen lnnhau. cng mt thi im c th c nhiu hn mt lnh cthc hin, ngha l mi chng trnh s t ch thc hin cctin trnh ca mnh.Cc chng trnh phi tng tc vi nhau v vic thc hinca chng nh hng ti nhp thc hin ca nhau.Trong LTSS, ngi lp trnh khng ch vit chng trnh, dliu nh trong mi trng tun t m cn phi s dng cccng c ng b ho (synchronize) v iu khin s tngtcgia cc tin trnh.Ngi lp trnh cn to ra v lp lch cho cc tin trnh, nghal s thc hin chng trnh c th nhn thy c bi ngilp trnh.

124

3.1 Gii thiu chung

Vn quan trng trong LTSS l phi tn dng c khnngca cc bx l.

C hai cch tip cn tn dng khai thc cc b x l:

1. Pht trin nhng ngn ng lp trnh cho php th hin

c vic thc hin song song mc thut ton, v d nh

Fortran, C, v.v.

2. Xy dng nhng chng trnh dch mnh nhn

dng c cc phn on chng trnh c th thc hin

song song hay tun t.

Hai cch tip cn trn l b sung cho nhau, nu ch p dngmt cch th khng hiu qu.

125

3.1 Gii thiu chung

Mt s phng php tip cn trong lp trnh song song:

Lp trnh song song kiu SIMD vi b nh chia s, trong

truy cp b nh l ng b (Synchronous). Lp trnh song song kiu MIMD vi b nh chia s, trong

truy cp b nh l d b (Asynchronous). Lp trnh song song kiu MIMD vi b nh phn tn, trong

truy cp b nh l d b. Lp trnh song song kiu SPMD vi b nh chia s, trong

truy cp b nh l d b. Lp trnh song song kiu SPMD vi b nh phn tn, trong

truy cp b nh l d b. Lp trnh song song kiu MPMD vi b nh chia s, trong

truy cp b nh l d b.

126

3.2 Cc ngn ng lp trnh song song

Cc yu cu i vi mt NNLT song songNgoi cc yu cu i vi mt NNLT tun t, mt NNLT song song

cn phic cc chc nng sau:

1. Cung cp cho ngi lp trnh nhng c ch khi to,

ngb v trao i gia cc tin trnh.

2. To ra c nhng chng trnh c lp vi my tnh

3. Phi h tr to ra c nhng chng trnh d c, d

vit, d chuyn i, v.v.

4. M hnh ho c victhc hin song song.

5. C kh nng iu chnh cc tnh hung m cc tin

trnh i hi phi trao i, tng tc vi nhau.
http://hyperlink/Ti%E1%BA%BFn%20tr%C3%ACnh.pptxhttp://hyperlink/Ti%E1%BA%BFn%20tr%C3%ACnh.pptxhttp://hyperlink/Ti%E1%BA%BFn%20tr%C3%ACnh.pptx


22/54

7/17/20

127


Cc tnh hung thng gp trong LT song song

1. Ti mt thi im c mt s tin trnh mun truy cp vo mt

ti nguyn chung hoc cp nht vo mt bin chia s. M

nhng ti nguyn ch cho php mt tin trnh truy cp ti

mi thi im.2. Khi mt tin trnh c quyn truy cp vo ti nguyn chung

th n s dng ti nguyn nhng khng c ngn cn

hot ng ca nhng tin trnh khc.

3. Khi mt s tin trnh cng kt hp thc hin mt s php

ton trn c s quan st hnh ng ca nhauth ngi lp

trnh phi lp lch cho nhng tin trnh .

C hai cch gii quyt cc tnh hung trn: 128


1.Tt c cc tin trnh phi s dng cu trc d liuchung tin trong vic cp nht v trao i vi

nhau.

2.Tt c cc tin trnh thc hin s ng b bng cch s dnghaihm wait()v pass(). Khi trao ivi nhau, mt tin trnh ch nhn c mt tnhiu ca tin trnh i tc v khi nhn c th haitin trnh trao i trc tip vi nhau.

129


Tng qut, c hai cch pht trin NNLT song song:

1. M rng nhng ngn ng lp trnh tun t hin c,

b sung thm nhng cu trc mi thc hin

c song song v gii quyt c s xung t

trong truy cp d liu.

2. Xy dng mt ngn ng lp trnh song song mi.

130


3.2.1 V d minh ha

a. Ci t song song trn kin trc UMA vi 4 BXL:

n i

n

rm

1n in

mr

1

2)(

trong

Bi ton: Xc nh phng sai (variance) ca mt mu c

cho bi mt danh sch cc s thc r1, r2, ...., rn . Ngha l phitnh:

BXL1 BXL2 BXL4

Memory I/O Devices

Switching mechanism

. . .. . .

BXL3

131


Ci t song song:Cc s thc ri c lu tr trong b nh chia s.

Bn bin sau y cng c to ra trong b nh chia s:

Sum (lu tng gi tr):

Mean (lu gi tr trung bnh):

Sum of square (lu tng bnh phng)

Variance (lu gi tr phng sai)

Qu trnh thc hin song song cm t nh sau:

n

irS 1

n i

n

rm

1

2

1)( mrSS

n

i

n

mr

n

SSn

i

2

1)(

132

n i

n

rm

1

ni

n

mr

1

2)(

Sum ofSum Mean Squares Variance

Values

36

Shared memory

0

541542233124

Process 4Temporary

Process 3Temporary

Process 2

Temporary

Process 1Temporary

8

7

11

10


Values

36

Shared memory

3 0

541542233124

Process 4Temporary

Process 3

Temporary

Process 2Temporary

Process 1Temporary

8

7

11

10

(a): Mi tin trnh cng 3gi tr c phn chiavlu vo bin tm thica n. Sau khi cc tin

trnh thc hin xongvic t nh cc tng con,s c mt tin trnhcng dn cc gi tr nyli v lu vo bin Sum bnh chia s.

(b): Mt tintrnh n s

tnh gi tr trung bnh v

lu vo bin b nh

chia s Mean.

n=12; (ri)=(4,2,1,3,3,2,2,4,5,1,4,5)


23/54

7/17/20

133

n

i

n

rm

1

n

i

n

mr

1

2)(


Values

36

Shared memory

3 22 1.83

541542233124

Process 4Temporary

Process 3Temporary

Process 2Temporary

Process 1Temporary

1

6

6

9

(c): Mi tin trnh phi xc

nh:(xi-Mean)2, (xi-Mean)2

lu vo bin tm thi v

cui cng cng dn lu

vo bin ca b nh chia

s.

(d): Mt tin trnh n xcnh variance bng cchchia gi tr ca Sum of

Squares cho 12.

Sum of

Sum Mean Squares Variance

Values

36

Shared memory

3 22

541542233124

Process 4Temporary

Process 3

Temporary

Process 2

Temporary

Process 1Temporary

1

6

6

9

n=12; (ri)=(4,2,1,3,3,2,2,4,5,1,4,5)

134


3.2.1 V d minh ha (tip)

b. Ci t song song trn NUMA vi kin trc siu khi 4 BXL:

Bi ton: Xc nh phng sai (variance) ca mt mu c

cho bi mt danh sch cc s thc r1, r2, ...., rn . Ngha l phi

tnh:

n i

n

rm

1n in

mr

1

2)(trong

Ci t song songKin trc ny khng c b nh chia s chung; n gi tr r1, r2,...., rn cphn phi cc b nh cc bcami BXL.Mi nt c 4 bin: 2 bin lu tr cc gi tr cng dn lSum v Sum of Squares; mt bin lu gi tr trung bnhMean v bin cn li lu tr gi tr phng sai variance.Qu trnh tip tc thc hin c m t nh hnh di y.

135


Nhc li,

c im ca NUMA:

C mt ng kt ni cc BXL li vi nhau.

B nh chia s c phn tn cho tt c cc BXL thnh b

nh cc b v tt c cc moun nh s l b nh chung cho

cc BXL.Cc BXL c php truy cp ng thi ti mt hay nhiu m

un nh v c th hot ng c lp vi nhau.

Network

CacheP

Mem CacheP

Mem

CacheP

Mem CacheP

Mem

NUMA vi Distributed Memory

136

Sum1

Mean

Sum2

Variance

Values

4 2 1

7 Sum1

Mean

Sum2

Variance

Values

3 3 2

8

Sum1

Mean

Sum2

Variance

Values

2 4 5

11 Sum1

Mean

Sum2

Variance

Values

1 4 5

10

Sum1

Mean

Sum2

Variance

Values

4 2 1

36

3

Sum1

Mean

Sum2

Variance

Values

3 3 2

36

3

Sum1

Mean

Sum2

Variance

Values

2 4 5

36

3

Sum1

Mean

Sum2

Variance

Values

1 4 5

36

3

(b):Cc BXL thc hin hon i vcng dn, sau cng chia cho 12 xc nh Mean.

(a): (b):

(a):Cc tin trnh tnh tng cc gi trti mi nt.

n

i

n

rm

1

ni

n

mr

1

2)(

n=12; (ri)=(4,2,1,3,3,2,2,4,5,1,4,5)

137

Sum1

Mean

Sum2

Variance

Values

4 2 1

36

3

6

Sum1

Mean

Sum2

Variance

Values

3 3 2

36

3

1

Sum1

Mean

Sum2

Variance

Values

2 4 5

36

3

6

Sum1

Mean

Sum2

Variance

Values

1 4 5

36

3

9

Sum1

Mean

Sum2

Variance

Values

4 2 1

36

3

22

1.83

Sum1

Mean

Sum2

Variance

Values

3 3 2

36

3

22

1.83

Sum1

Mean

Sum2

Variance

Values

2 4 5

36

3

22

1.83

Sum1

Mean

Sum2

Variance

Values

1 4 5

36

3

22

1.83

(c) Mi tin trnh, vi cc gi trr ing hin c ti nt ca mnh sthc hin t nh (xi-Mean)2 v luvao bin Sum2.

(c): (d):

(d) Cc BXL thc hin hon iv cng dn, sau cng chiacho 12 xc nh Variance.

n

i

n

rm

1

ni

n

mr

1

2)(

n=12; (ri)=(4,2,1,3,3,2,2,4,5,1,4,5)

138


Gii thiu mt s s ngn ng lp trnh song song

3.2.2 Fortran 90

Fortran 90 l ngn ng lp trnh chun ANSI. Fortran 90 l ngn ng lp trnh song song theo d liu

c nng cp t Fortran 77 bng cch b sung thm mt

s c tnh nh:

Kiu d liu do User nh ngha

Php ton v Array, con tr

Cc BXL h tr vic s dng cc kiu d liu Short

Integer, Packed logical,...

Cp pht b nh ng.
http://hyperlink/Fortran%20language.pptxhttp://hyperlink/ANSI.pptxhttp://hyperlink/Packed%20logical.pptxhttp://hyperlink/Packed%20logical.pptxhttp://hyperlink/ANSI.pptxhttp://hyperlink/Fortran%20language.pptx


24/54

7/17/20

139


Ngi lp trnh Fortran 90 da vo m hnh song song tng

t nh PRAM gm c nhng thnh phn:

Mt CU (n v iu khin)

Mt n v x l logic, s hc ALU B nh chia s.

Mt tp cc BXL

CPU thc hin cc cu lnh tun t bng cch truy cp vo

cc bin c lu tr trong b nh chia s.

n v vector lu tr, c, ghi d liu vo b nh chia s v

c CPU iu khin thc hin song song.

140

3.2 Cc ngn ng lp trnh song song - Fortran 90

Fortran 90c cc kiu d liu chun:REAL, INTEGER, CHARACTER, LOGICAL, v.v.Dng khai bo tng qut:

Type[(kind)] [, attribute] :: Variable-ListTrong , Typel kiu c s hoc kiu c nh ngha bi NSD kind l phn tu chn cng mt s kiu c s dng

nh ngha v kiu c thV d: CHARACTER (LEN = 10) :: s1nh ngha bin s1 kiu CHARACTER c th cha 10 k t. [, attribute] l danh sch cc thuc tnh ca Fortran c s

dng nh ngha cc c tnh ring ca danh sch ccbin.

+:: phn t phn cch gia phn m t kiu v danh sch ccbin.

Variable-List:danh sch cc bin

141


V d v khai bo v mng:INTEGER, DIMENSION (1 : 10) :: SA{Bin mng SA c 10 phn t kiu s nguyn.}+ Khai bo mt hng mng:(/ 2,4,6,8,10 /) hoc (/ (I, I = 2, 10, 2) /)+ Fortran 90 cho php p dng cc php ton s hc cho cc

bin mng, ngha l ton hng ca cc php ton s hc (+,-, *, /) c th l bin n hoc bin mng.

V d v php ton trn bin mng:INTEGER A(10), B(10), CDO I = 1, 10, 1

A(I) = B(I) + C

END DOTa c th vit ngn gn hn.

A = B + C142


Cu trc chng trnh Fortran

PROGRAM

IMPLICIT NONE

[phn c t]

[phn thc hin]

[phn chng trnh con]

END PROGRAM

143


Xy dng mt chng trnh FORTRAN

144

V d: tnh s t cng thc bng NNLT Fortran 90

1. INTEGER, PARAMETER :: N = 131000

2. INTEGER, PARAMETER :: LONG =SELECTED_REAL_KIND(13,99)

3. REAL(KIND=LONG) PI, WIDTH4. INTEGER, DIMENSION(N)::id

5. REAL(KIND=LONG), DIMENSION(N)::X, Y

6. WIDTH = 1.0_LONG/N

7. ID = (/(I,I = 1, N)/)

8. X = (ID 0.5)* WIDTH

9. Y = 4.0 / (1.0 + X * X)

10.PI = SUM(Y) * WIDTH

11.10 FORMAT(PI = , F14.12)

12.PRINT 10, PI

13.END

.

khai bo tham s N ls cc on con ti a tnh tch phn

khai bo tham s LONG sdng cho cc bin s thc viphn t nh c t nht 13 ch sv phn ng (m) thuckhong [10-99,1099]. Cc binthc ny c khai bo dng3 v 5

xc nh rng ca mikhong con. Bin mng ID gmN phn t khai bo d ng 4,ckhito dng 7

tnh ton song song im giacami khong contnh ton song song gi t r hmtng ngti ccim gia X

gihm tnh tng SUM

Khai bo bin mng id cN phn t kiu s nguyn


25/54

7/17/20

145

Tiu lun: Fortran 90 Tutorial (02 hc vin)

Professor Dr.Shene

Department of Computer Science

Michigan Technological University

www.cs.mtu.edu/~shene/COURSES/cs201/NOTES/fortran.html

http://www.fortran-2000.com/

146

3.2 Cc ngn ng lp trnh song song - OCCAM

3.2.3 Lp trnh song song vi OCCAMGii thiu v OCCAM: NNLT song song c Inmos Company (Anh) pht trin

nm 1988, Mc ch chnh l thit k v ci t cc chip c gi l

transputer. Chng trnh Occam thng nhiu tin trnh v chng c

th c nh x sang mt s cc transputerbt k thchin song song v trao i d liu vi nhau thng qua ccknh vo/ra.

Occam l NNLT bc cao, c s dng lp trnh chonhng h thng gm nhiu my tnh kt ni vi nhau, hoccc h phn tn.

Trong Occam, cc hnh ng c th thc hin song songc gi l tin trnh v mi cu lnh cn phi khai bo nhmt tin trnh.

147

M phng m hnh lp trnh trong Occam

A: Cc tin trnh thchin tun t

B: Cc tin trnh thchin song song

C: Cc tin trnh thchin tun t

D: Cc tin trnh thchin song song

E: Cc tin trnh thchin tun t

A

B

C

D

E

148

3.2 Cc ngn ng lp trnh song song - Occam

Cc tin trnh trong Occam

C ba tin trnh nguyn thu trong Occam:

Tin trnh gn: thay i gi tr ca cc bin

Tin trnh Input: nhn d liu vo t cc knh vo (cng vo)

Tin trnh Output: gi d liu ra cc knh ra.

Cu trc ngn ng

Tin trnh gn:

sum := partion1 + partion2

/*gn gtr cho bin sum l tng hai gtr partion1v partion2 */

Tin trnh Input: user1 ? x

/*gn gi tr t knh user1 cho x */ Tin trnh Output: C ! x * x

/*gi gi tr x2 ra knh C */

149


Cc cu trc iu khin

SEQ: cu trc tun t, cc thnh phn trong tin trnh ny

c thc hin theo th t v tin trnh ny kt thc khithnh phn cui cng thc hin xong. Mi thnh phn ca

tin trnh ny c th l mt tin trnh song song.

V d:

SEQ

channel1 ? partial1

channel2 ? partial2

sum:= partial1 + partial2

partial3 ! Sum

Cc s nguyn c nhp

vo tun t t 2 knh khc

nhau sau cng li v

chuyn sang knh 3

150


Cc cu trc iu khin (cont.)

PAR: cu trc song song, cc thnh phn ca tin trnh ny

c thc hin ng thi v tin trnh ny s kt thc khi ttc cc thnh phn ca n u kt thc.

V d:

SEQ

PAR

channel1 ? partial1channel2 ? partial2sum:= partial1 + partial2

Cc s nguyn cnhp vo song song t 2knh khc nhau sau cng li
http://www.cs.mtu.edu/~shene/COURSES/cs201/NOTES/fortran.htmlhttp://www.fortran-2000.com/http://hyperlink/occam%20(programming%20language).pptxhttp://hyperlink/Transputer.pptxhttp://hyperlink/Ti%E1%BA%BFn%20tr%C3%ACnh.pptxhttp://hyperlink/Ti%E1%BA%BFn%20tr%C3%ACnh.pptxhttp://hyperlink/Ti%E1%BA%BFn%20tr%C3%ACnh.pptxhttp://hyperlink/Transputer.pptxhttp://hyperlink/occam%20(programming%20language).pptxhttp://www.fortran-2000.com/http://www.fortran-2000.com/http://www.fortran-2000.com/http://www.cs.mtu.edu/~shene/COURSES/cs201/NOTES/fortran.html


26/54

7/17/20

151


Cc cu trc iu khin (cont.)

ALT: cu trc tuyn chn, chn mt trong cc thnh phn ca

tin trnh thc hin nu n tho mn iu kin la chn

v tin trnh ny kt thc khi thnh phn c la chn ktthc. (xem v d tnh tch phn)

152


Cc cu trc khc

1. Cu trc iu khin IF, gi l tin trnh iu kin chn mt

tin trnh thnh phn khi biu thc Boolean c gi tr true;

2. Cu trc lp WHILE, gi l tin trnh lp thc hin lp li

tin trnh thnh phn cho n khi biu thc iu kin

Boolean nhn gi tr true.

3. Cu trc lp FOR, gi l tin trnh lp c s vng lp xc

nh trc, i = [0 FOR n]

V d: PAR i = [0 FOR 9]: to ra 10 tin trnh song song, i

nhn gi tr t 0 n 9

153

V d: Gi s c hai tin trnh ging nhau cng nhn d liu vo vcng dn vo tng.

CHAN In1, In2:CHAN Out1, Out2:VAR Sum1, Sum2:

SEQ &cc lnh di y thc hin tun t

Sum1 := 0 &gn gi tr u cho Sum1 v Sum2Sum2 := 0PAR &bt u thc hin song song

-- Tin trnh th nhtWhile TRUE

VAR Item1:SEQ

In1 ? Item1 &gn gi tr knh In1 cho Item1Sum1 := Sum1 + Item1Out1 ! Item1

-- Tin trnh th haiWhile TRUE

VAR Item1:

SEQIn2 ? Item2Sum2 := Sum2 + Item2Out2 ! Item2

154


S trao i gia cc tin trnh

Trong v d trn, cc tin trnh khng cn trao i vi nhau v

mi tin trnh u s dng cc bin cc b ca ring n.

Tuy nhin, khi c nhiu tin trnh mun trao i d liu vi

nhau th phi trao i trn cng mt knh truyn d liu.

Ngha l, mt tin trnh gi d liu ra mt knh truyn v

tin trnh kia nhn d liu t knh .

Trong Occam, mi tin trnh thc hin trn mt b x l v

truyn thng ip (Passed-Message) ti nhng b x l lncn theo kin trc hnh khi.

155

1. DEF N = 400000:2. DEF PROCESS = 8:

3. DEF CHUNK = N / PROCESS:

4. CHAN sum[PROCESS]5. PAR6. PAR i = [0 FOR PROCESS]7. REAL64x, localsum, width:8. SEQ9. localsum := 0.010. width := 1.0 / N11. x := ((i * CHUNK) + 0.5) * width12. SEQ i = [0 FOR CHUNK]13. SEQ14. localsum:=localsum+(4.0/(1.0+(x*x))))15. x := x + width

V d: tnh s t cng thc bng NNLT Occam

cont.

Tdng 1-3:nh ngha cchng: N l s khong chia; PROCESSES l s t in

trnh s c to; CHUNK l s khong

cho mi t in trnh (chng trnh thc hinchnh xc nn chn Nl bi s caPROCESSES

Dng 4: nh ngha mtknh cho mi t in trnh.

Tin trnh s sdngknhny gi dl iu ca n(subtotal) ra tin trnh bnngoi (grand total)

T dng 5-31: l nhng t intrnh song song v tun t,gm c (PROCESS+1) tintrnh. Mi tin trnh trong cctin trnh thc hin song songt dng 6-17, s xc nh tngdin tch cc hnh ch nhtc chia s

156

16. localsum := localsum * width17. sum[i] ! localsum18. REAL64 pi :

19. INT got [PROCESS] :20. SEQ21. pi := 0.022. SEQ i = [0 FOR PROCESS]23. got[i] := FALSE24. SEQ i = [0 FOR PROCESS]25. REAL64 y:26. SEQ27. ALT i = [0 FOR PROCESS]28. (got[i] = FALSE) & sum[i] ? Y29. got[i] := TRUE30. pi := pi + y31. Output ! Approximation to pi is: ; pi

V d: tnh s t cng thc bng NNLT Occam

T dng 18-31: l tintrnh cui cng, xcnh tng sau cng vchuyn cho t in trnh

output in ktqu pi


27/54

7/17/20

157

Tiu lun (02 hc vin):

OCCAM Language

http://en.wikipedia.org/wiki/Occam_programming_language

http://www.wotug.org/

Tiu lun (02 hc vin): ThreadMentor

http://cs.mtu.edu/~shene/NSF-3/e-Book/index.html

158

3.2.4 Lp trnh song song vi PVM (Parallel Virtual Machine)

PVM c pht trin bi University of Tennessee, Oak Ridge

National Laboratory and Emory University (1989)

PVM l mt phn mm s dng cho h thng bao gm cc

my tnh khng thun nht (heterogeneous) c kt ni vi

nhau x l song song.

PVM thng c s dng cho nhng my tnh ni mng

trong mi trng UNIX hoc Windows

3.2 Cc ngn ng lp trnh song song - PVM

159

Cc c im chnh ca PVM:Thc hin theo m hnh truyn thng ip (Message Passing)H tr s kt ni khng thun nht(Heterogeneity):PVM h trs kt ni ca nhiu my tnh, nhiu mng my tnh v nhiuloi chng trnh ng dng khc nhau.H tr a b x l:PVM s dng nhng kh nng truyn thngip trong h a b x l khai thc ht kh nng ca phncng.Tnh ton da trn tin trnh:n v iu khin thc hin songsong trong PVM l mt tc v, l mt lung (thread) lmnhim v iu khin s truyn thng v tnh ton.Thay i cu hnh theo yu cu:Cc chng trnh c th thchin trn tp cc my c la chn theo yu cu ca NSD.


160

PVM x l tt c cc vn :

nh tuyn truyn thng ip

chuyn i d liu

lp lch trong mng my tnh

H thng PVM gm hai thnh phn chnh:1. Khi pvmd (hoc pvm3) t thng tr trn tt c cc my

tnh to ra my o (cc BXL gi lp).

2. Th vin cc chng trnh con giao din ca pvm:cha cc

chng trnh con truyn thng ip, qun l cc tin trnh,

phi hp cc tc v v thay i cc my o.


161

M hnh tnh ton ca PVM


Nhp d liu v phn on

Xut d liu v hin thkt qu

My tnh 1 My tnh 2

162

Mt kin trc ca PVM


MPP

Bridge

SIMD

Cluster 1

Cluster 2

Cluster 3
http://en.wikipedia.org/wiki/Occam_programming_languagehttp://www.wotug.org/http://cs.mtu.edu/~shene/NSF-3/e-Book/index.htmlhttp://hyperlink/Message%20passing.pptxhttp://hyperlink/Thread.pptxhttp://hyperlink/Thread.pptxhttp://hyperlink/Message%20passing.pptxhttp://cs.mtu.edu/~shene/NSF-3/e-Book/index.htmlhttp://cs.mtu.edu/~shene/NSF-3/e-Book/index.htmlhttp://cs.mtu.edu/~shene/NSF-3/e-Book/index.htmlhttp://cs.mtu.edu/~shene/NSF-3/e-Book/index.htmlhttp://cs.mtu.edu/~shene/NSF-3/e-Book/index.htmlhttp://www.wotug.org/http://en.wikipedia.org/wiki/Occam_programming_language


28/54

7/17/20

163

Phng thc thc hin chng trnh trong PVM: Nhng chng trnh vit bng C/C++, Fortran 77 c th cha

nhng li gi cc hm th vin ca PVM (y l nhng ngnng lp trnh c PVM h tr)

Cc chng trnh c dch theo kin trc ca h thng(host pool), cc tp m ch (object file) c t vo nhngni m mi my tnh ca h thng u truy cp c.

Cc USER to ra bn sao ca tc v ch (master) hoc khing mt tc v thc hin cc ng dng ca mnh.

Mt tin trnh c khi ng bi mt tin trnh khc cgi l tin trnh t (slave).

Nhng tin trnh ny c th thc hin mt s tnh ton cc bv trao i vi nhau gii quyt bi ton t ra.


164

Mt vi thng tin v PVM: User mun chy mt ng dng trn PVM th phi khi ng

PVM v to ra mt my o. Mt ng dng PVM c th c gi t du nhc ca h iu

hnh UNIX mt host no Cc Users c th cu hnh my o v thc hin cc ngdng ca mnh vi PMV

Trong PVM mi tin trnh u c th giao tip v/hoc ngb vi nhng tin trnh khc

Cc files PVM c th truy cp a ch:http://www.netlib.org/pvm3/index.html

Nhiu my tnh h tr h thng PVM thc hin nh: BBNButterfly TC2000, 80386/486 PC chy vi UNIX, ThinkingMachines CM-2, CM-5, Cray-2, Cray-5MP, HP-9000 PA-RISC, Intel Paragon, Silicon Graphics IRIS, Sun 4, DEC MicroVAX, v.v.


165

Gii thiu mt chng trnh trn PVM

Chng trnh in ra mn hnh (master) nh danh ca tc v (ID task)

nhn c t hm pvm_mytid(),

Sau s dng cc hm:

pvm_spawn(): to ra mt bn sao v gi chng trnh hello_other

pvm_recv(): nhn v thc hin khi thng ip gi n.

pvm_exit(): kt thc chng trnh trong h thng PVM.


166

Gii thiu mt chng trnh trn PVM(cont.)

Chng trnh slave:

pvm_parent() xc nh ID taskca master.

Sau dng cc hm di y xc nh hostname v chuyn

hotname ny cho master:

pvm_initsend(): khi to buffer gi

pvm_pkstr(): t mt xu vo buffer gi i

pvm_upkstr(): c mt xu vo buffer

pvm_send(): chuyn d liu buffer ti tin trnh nhn c xc

nh bi pvm_mptid.


167

/* Chng trnh master c tn Hello.c */

#include pvm3.h

main(){

int cc, tid, msg;

char buf[100];

printf(Master ID number %x\n, pvm_mytid());

cc= pvm_spawn(Hello_other,(char**)0,0,, 1, &tid);

if(cc== 1){

msg = 1;

pvm_recv(tid, msg);

pvm_upkstr(buf);

printf(From master %x: %s\n, tid, buf);

} else printf(Cannot start hello_other\n);

pvm_exit();

}


to ra mt bn sao v gichng trnh hello_other

nhn v thc hin kh ithng ip gin

c mt xu vo buffer

kt thc chng t rnhtrong h thng PVM 168

/* Chng trnh slave c tn: hello_other.c */#include pvm3.h

main(){int ptid, msg;

char buf[100];ptid = pvm_parent();strcpy(buf, Hello world from );gethostname(buff + strlen(buf), 64);

msg = 1;pvm_initsend(PvmDataDefault);pvm_pkstr(buf);pvm_send(ptid, msg);pvm_exit();}


pvm_parent() nhn mt tc vca tintrnh t ttintrnh ch

pvm_initsend() khito buffer gi

pvm_pkstr() t mt xuvo buffer gi i

pvm_send() chuyn d liu buffer titintrnh nhn c xc nhbi ptid.
http://www.netlib.org/pvm3/index.htmlhttp://www.netlib.org/pvm3/index.html


29/54


30/54

7/17/20

3

175

2. Phn ph thuc d liu (Data Anti-Dependency): l s ph thuc d liu gia S1 v S2 khi DEF(S2)USE(S1) S ph thuc ny xut hin khi chng ta s dng li tn gi

ca cc bin, mt bin c s dng trong S1 v sau c nh ngha li S2.

Khi xut hin phn ph thuc d liu gia cc cu lnh thchng cng khng thc hin song song c.

Quan h phn ph thuc d liu c k hiu: V d: cc ph thuc d4, d6 . d7 l cc phn ph thuc d liu.

3.3 S ph thuc d liu v th u tin ca cc tin trnh

S1 S2 S3

d4 d6

Phn ph thuc d liu

d7

S1: A := B + C; S2: B := A + E ; S3: A := A + B

176

3. Ph thuc d liu ra (Data Output Dependency): l s ph thuc d liu gia S1 v S2 khi DEF(S2)DEF(S1) S ph thuc ny xut hin do hai nguyn nhn: S dng li tn ca cc bin (dng chung) Tnh tng gi tr ca cng mt bin.

Nu nhng lnh ny thc hin ng thi th chng s ghi cc gi tr vo cng mt nh. Do vy, cn phi xc nhchnh xc th t thc hin ngn nga vic s dngnhng gi tr khng ng.

Quan h ph thuc d liu ra c k hiu l: V d: ph thuc d5 l loi ph thuc d liu ra.


S1S3

d5

Ph thuc d liu ra

S1: A := B + C;S3: A := A + B

DEF(S1)DEF(S3) ={A}

177

4. Ph thuc d liu vo (Data Input Dependency): l s ph thuc d liu gia S1 v S2 khi USE(S2)USE(S1) Bi v cc lnh ny ch truy cp (c) v khng lm thay i gi

tr ca cc bin , do vy cc lnh ny c th thc hin theobt k th t no cng c, ngha l c th thc hin songsong.

Quan h ph thuc d liu vo c k hiu: V d


S2: B := A + E ; S3: A := A + B

USE(S3) USE(S2) ={A}

S2 S3

Ph thuc d liu vo

d3

178


5. Ph thuc iu khin d liu (Data Control Dependency): l s ph thuc d liu gia S1 v S2 khi s thc hin ca

lnh ny ph thuc vo gi tr ca cc bin c tnh lnhkia.

Quan h ph thuc iu khin d liuc k hiu l:

S4: P = (B => 0)S5: if (P is true) then

D = 1else D = 2endif

S4 S5

Ph thuc iu khin d liu

179


Nhn xt: tng c mc song song ca chng trnh chng ta cth kh hoc gim ccphn ph thuc d liuvph thuc d

liu ra trong th ph thuc d liu. Thng thng kh cc ph thuc d liu ny l t li tncho cc bin trnh vicchia s cc bin . V d: Xt on chng trnh sau:

for i = 1, n, 1X = A[i] + B[i]Y[i] = 2 * XX = C[i] * D[i]P = X + 15

endfor

for i = 1, n, 1X = A[i] + B[i]Y[i] = 2 * XXX = C[i] * D[i]P = XX + 15

endfor

Nhn xt: C phn ph thuc d liu. Nu hai lnh sau ta t X thnh XX th s tora hai on chng trnh c th thc hin song song: 180


+ S dng cc bin khc nhau loi bph thuc d liu ra.V d:

A = B + CA = C * X

A = B + CA1 = C * X

Nhn xt: Hai lnh trn c s ph thuc d liu ra. C thloi b ph thuc ny bngcchthay bin A ca cu lnh th hai bng A1

DEF(S2) DEF(S1) = {A}


31/54

7/17/20

3

181


+ S dng cc bin khc nhau loi bphn ph thuc dliu

V d:

Nhn xt: Hai lnh trn c quan h phn ph thuc d liu. C th loi b s ph thuc ny bng cch sdng mt bin khc cho cu lnh th hai

A = B + CB = C * X A = B + CB1 = C * X

DEF(S2)USE(S1) = {B}

182


+ Thc hin mt s php bin i (php th) cng c th loib cphn ph thuc d liu.

V d: Xt dy cc cu lnh sau:

S1: A = B + C

S2: B = A * 3S3: A = 2 * CS4: P = (B => 0)

if (P is true) thenS5: D = 1S6: else D = 2

endif

S1

S3 S2

S4

S5 S6

th ph thuc d liu

Nhn xt: x l song song, th cn thit ph i lo i b i mt s loi phthuc dliu cth.

183

V d: loi b nhng quan h phn ph thuc d liu v phthuc d liu kt qu

S2: B = (B + C) * 3S3: A = 2 * CS4: P = (B => 0)


endif

S1: A = B + CS2: B = A * 3S3: A = 2 * CS4: P = (B => 0)


endif

S3 S2

S4

S5 S6

th ph thuc d liu rt gn

S1

S3 S2

S4

S5 S6

DEF(S2)USE(S1) ={B} (phn ph thuc d liu)

184


for (i = 0; i < N; i++)A = A + i;

Nhn xt: Trong mi bc lp u phi s dng nhng tn bin khc nhau nu munloi b phn ph thuc d liu. iu ny kh thc hin c trong trng

hp s vng lp ln hoc bt k. Quan h ph thuc d liu l khng c tnh bc cu. Ngha l nu S2 phthuc vo S1, S3 ph thuc vo S2 th khng kt lun c S3 ph thuc dliu vo S1.

Tuy nhin, nhng php bin i n gin nh th khng philc no cng p dng thnh cng. V d, xt chu trnh sau:

i = 0;aa: if(i > N) stop;

a = a + i;i++;goto aa;

khng s dng FOR

185


DO iA[2*i+1] = B[i] (1)D[i] = A[2*i] (2)

ENDDO

Nu xem mng A nh mt thc th n th hai lnh trn l phthuc theo dng d liu (v A xut hin trong DEF ca (1) vcng xut hin trong USE ca (2)).Nhng nu ta xt thm ch s mng th trong cu lnh (1) sdng cc phn t mng A c ch s l; cn (2) li s dngphn t mng A c ch s chn.Do , y khng c s ph thuc d liu.

Ph thuc theo chu trnh v theo mngMc ch: kh ph thuc d liu trong cc vng lp.Xt on chng trnh sau:

hai cu lnh (1) v (2)c thc hin song songc khng??

186


Xt v d sau:FOR I=1 DO

A[I] = A[I-1] + 1 (3)

Chu trnh trn c th khai trin thnh:A[1] = A[0] + 1; A[2] = A[1] + 1;A[3] = A[2] + 1; A[4] = A[3] + 1; . . .

Vn : khi xy dng cc chu trnh lp phi xt xem nhng dylp khc nhau ca chng c th thc hin song song ckhng?

Ni cch khc, khi mt b x l P thc hin mt dy cc culnh vi i = 1 (bin iu khin vng lp) th b x l Q c th cngthc hin dy cc cu lnh vi i=2 hay khng?


32/54

7/17/20

3

187


Xt hai vng lp th iv thjtng ng vi I = i v I = j.Khng mt tnh tng qut ta c th gi thit i < j. Chu trnh s cdng:

. . .

A[i] = A[i-1] + 1;. . .A[j] = A[j-1] + 1;. . .

Nhn xt:Cc tp trn khng cha bin n m ch c cc bin c ch sv khi ivjkhc nhau th cc bin khc nhau.Hai tp DEF(i)= {A[i]} v USE(j) = {A[j-1]} c cha phn tchung khi i = j 1, V phng trnh i = j 1 c v s nghim

Vy kt lun chu trnh trn khng thc hin song song c.

USE(i) = {A[i-1]}

USE(j) = {A[j-1]}DEF(i) = {A[i]}DEF(j) = {A[j]}

188


Nhn xt: Ni chung, vic xc nh s ph thuc theo cch nh trn lbi ton kh. S phc tp s xut hin bi v c nhiu dngphng trnh khng hin nhin. Nhng phng trnh ny khngc li gii trc tip trong cc cng c ton hc kinh in. Trng hp tng qut, khi xt nvng lp m dn n dngphng trnh a1x1 + a2x2 + + anxn = c vi ai v c l cc snguyn (gi l phng trnh Diophantine) th kt lun chu trnhkhng th thc hin song song.

Lu : Phng trnh Diophantine c dng tng qut:a1x1 + a2x2 + + anxn = c vi ai v c l cc s nguyn.

Nm 1991 Zima v Chapman chng minh c rng nuc s chung ln nht (a1,a2, an) chia ht cho cth phngtrnh Diophantine c mt nghim nguyn.

Trng hp tng qut, gii phng trnh trn l khng th.

189

3.4 Bin i chng trnh

Mc ch:1. Loi b c s ph thuc ca cc cu lnh trong ch.trnh.2. Khc phc c nhng tr ngi ca cch xc nh cc vnglp c th thc hin song song dn ti phng trnh Diophantine.

3.4.1 Cc bin qui npBin quinp l loi bin trong chu trnh m cc gi tr l in tipca n to ra dy cp s cng.

V d: xt on chng trnh sau:

m = 0;

DO i = 1 to Nm = m + k;x[m] = a[i]

END

{m l bin qui np}

190

3.4 Bin i chng trnh

Nhn xt: Hai cu lnh trn c s ph thuc d liu v mc s dng

cho gia cc vng lp. Khi vng lp trc tnh gi tr ca mth mli c s dng

lnh th hai nn chng phi thc hin tun t.

Nu mi vng lp chng ta d bo c gi tr ca mth cths loi bc sph thucd liu lin quan n m. Gi tr kl bt bin trong cc vng lp v c cng lin tip

vo sau mi vng lp, do vy, vng lp th i, m = i * k.

m = 0;Do i = 1 to N

m = m + k;x[m] = a[i]

End

Do i = 1 to Nx[i*k] = a[i]

End

191

3.4 Bin i chng trnh

3.4.2 S ph thuc tinPh thuc tin l loi phn ph thuc d liu gia cc vnglp ca chu trnh.

Do i = 1 to Nx[i] = x[i+1]

End

Nhn xt:Chu trnh khng thc hin song song cChu trnh lm nhim v sao tt c cc phn t ca mt mngsang chnh mng vi ch s gim i mt, do vy n phi thchin tun t.Cch gii quytS dng 2 mng: mt bn gc v mt bn sao ca mng gc vit li chng trnh

Do i = 1 to Nx1[i] = x[i]

EndDo i = 1 to N

x[i] = x1[i+1]End

C th thc hin song song ?

C th

192

3.4 Bin i chng trnh

3.4.3 S ph thuc liTrng hp cc phn t ca mng trong chu trnh li cxc nh thng qua nhng phn t ng trc chng (li v

pha trc)Do i = 1 to N

x[i] = x[i-1] + y[i]End

Nhn xt:Vic loi b nhng ph thuc d liu kiu ny khng n ginKhng c gii php chung loi b ph thuc d liu trongtrng tng qut. Nu c gii php loi b th gi phi tr cngkh t.

Hin nhin khng cn s ph thuc li trong chu trnh, tuy nhinthi gian tnh ton trong mi cu lnh li tng ln kh nhiu.

Do i = 1 to Nx[i] = x[0]+y[1]+y[2]+ +y[i]

End


33/54

7/17/20

3

193

3.4 Bin i chng trnh

3.4.4 S phn tch chu trnh (ph thuc li)Tch mt chu trnh thnh nhiu chu trnh con thc hin song

song

Do i = 1 to N

a[i] = b[i] + c[i];c[i] = a[i-1]

End

Nhn xt: S tham chiu gia cc mng av c dn n s ph thucgia cc vng lp ca chu trnh. Gi tr c[i] lnh th nht lun l nhng gi tr lu tr c camng c (c xc nh lnh th 2) a[i-1] c s dng lnh th hai l gi tr c tnh vng lp trc (vng th i -1)

Do i = 1 to Na[i] = b[i] + c[i];

EndDo i = 1 to N

c[i] = a[i-1]End

194

3.4 Bin i chng trnh

3.4.4 S phn tch chu trnh (ph thuc tin)

Do i = 1 to N

a[i] = b[i] + c[i];c[i] = a[i+1]

End

Nhn xt: Phn t c[i] lnh hai c tnh theo gi tr ca a[i+1], l gitr trc khi n c cp nht lnh th nht Ngc li c[i] c s dng lnh th nht li l gi tr ctrc khi c cp nht lnh th hai.

Do i = 1 to Nx[i] = c[i];c[i] = a[i+1]

EndDo i = 1 to N

a[i] = b[i] + x[i];End

Cch gii quyt Sao mng c sang mng ph x. Kt qu bin i chu trnh nythnh hai chu trnh trong khng cn s ph thuc d liu

195

3.4 Bin i chng trnh

3.3.5 Cc chu trnh lng nhauXt trng hp nhiu chu trnh lng nhau (chu trnh nhiu ch s)

Do i = 1 to NDo j = 1 to N

x[i,j] = x[i, j-1] + z[i];EndNhn xt:

Mng zc s dng trong chu trnh hon ton khng gy ras ph thuc no c. Ch s th nht ca mng xl i, cng chnh l ch s ca chutrnh, nhng khng c s tham chiu cho gia cc vng lp.Vy, chu trnh lp bn trong (chu trnh lp theo i) c th thchin song song. Ch s th hai ca mng xlj, l ch s ca chu trnh th nhtdo c s ph thuc dng d liu ca cc ln lp vngngoi. Cng nh 3.4.3, trng hp ny khng c cch bini n gin v dng kh song song. 196

Bi tp

3.1 Xc nh s ph thuc d liu ca cc lnh trong chu trnhsau:Do i = 1 to N

e[i] = x[i] - z[i];

a[i+1] = e[i] + 2*d[i]a[i] = e[i]

end3.2 Hai chu trnh sau c tng ng v ni dung tnh ton hay

khng? hy bnh lun v kh nng thc hin song song cacc chu trnh .

1. Do i = 1 to N

a[i] = a[i+1] + iend2. Do i = N downto 1

a[i] = a[i+1] + iend

197

Bi tp

3.3 Xc nh tt c cc s ph thuc d liu trong onchng trnh sau:

int a[MAX];read(a);for(i = 0; i < N; i++)

for(j = 0; j < i; j++){a[i] = max(a[i], a[j]);

a[j] = min(a[i], a[j]);}3.4 Loi b cc ph thuc d liu ra v phn ph thuc d liu

ca chu trnh saufor(i = 0; i < N; i++){

x = a[i]+ b[i]);y[i] = 2 * x;

} 198

Bi tp

3.5 Phn tch on chng trnh sau, xc nh cc ph thuc d

liu v v th ph thuc d liu ca on chng trnh .A = B + C;for(i = 0; i < N; i++){

D[i] = A + b[i]);S = b[i] * 5;T = T + S;

}3.6 Vit chng trnh gii phng trnh bc hai v v th ph

thuc d liu ca n.


34/54

7/17/20

3

199

CHNG 4. CC M HNH V LP TRNH SONG SONG

NI DUNG

1. Lp trnh b nh chia s

2. Lp trnh song song da vo cc tin trnh

3. Lp trnh song song da vo cc lung

4. Lp trnh theo m hnh truyn thng ip.

5. Lp trnh trn cm my tnh vi PVM

200

1. LP TRNH B NH CHIA S

Mt vi ch (1/4)

1. H thng a b x l i xng SMP(symmetric multiprocessor Sysstem)

Cc b x l l nh nhau

Khng c nhng BXL c bit x l vo/ra

Khng c BXL c gn nhim v c bit no khc.2. nghin cu v XLSS, chng ta khng nht thit phi c h

a b x l vt l.

3. Trong mi trng UNIX, chng ta c th to ra nhiu tin trnh

khc nhau trong h thng v chng c s dng m

phng lp trnh a b x l.

4. Hu ht cc h UNIX u cho php to ra mt s cc tin trnh

bt k v chng c lp lch cho nhng b x l thch hp

201


Mt vi ch (2/4)5. V nguyn tc, chng trnh l tp cc tin trnh v vic vit

chng trnh l c lp vi cc b x l, vic phn chia cc

tin trnh cho cc b x l l cng vic ca h iu hnh..

6. Trong nhng h thng a nhim nh UNIX, s b x l v s

cc tin trnh khng nht thit phi bng nhau. Nhng nn t

chc mt tin trnh c mt BXL m nhim tin vic lp lch.

7. Mt vn quan trng cn xem xt khi XLSS l cn bao nhiu

nt(BXL) chng trnh song song thc hin hiu qu nht?

8. Nhiu chng trnh c xy dng ph thuc vo mt cu

hnh xc nh. Ni chung, loi chng trnh ph thuc nh vyl khng tt, v khi b x l bn th cc tin trnh tip theo phi

ch. 202


Mt vi ch (3/4)

Trong mi trng lp trnh chia s b nh c hai rng bucquan trng:

1. Mt tin trnh c th ch mt khong thi gian bt k gia hailnh cn thc hin. Gi s b x l Pthc hin mt chngtrnh c mt 100 lnh, b x l Qthc hin chng trnh c 10lnh v cng bt u thc hin ng thi. Thm ch, tt c cclnh c tc thc hin nh nhau cng khng th ni rng Qs kt thc trc P.

2. Khng th xem cc lnh thc hin l nguyn t mc ccngn ng lp trnh. V d, mt lnh n gin nh: a = a + 1 s

l mt dy bn lnh trong ngn ng my. M ta cng bit rng,cc tin trnh v h iu hnh ch nhn bit c cc cu lnhca ngn ng my.

203


Mt vi ch (3/4)

Trong lp trnh b nh chia s:

Cc tc v (tasks) s c/ghi d liu t b nh chia s qua

khng gian a ch b nh chung (common address space)

C nhng c ch khc nhau nh (locks/semaphores) iu

khin vic truy nhp n b nh chia s

Ngi lp trnh khng cn m t vic truyn thng d liu, do

vic vit chng trnh s n gin.

Kh qun l d liu v khng can thip trc tip qu trnh truyn

d liu.204

1.1 LP TRNH B NH CHIA S DA VO TIN TRNH

To lp tin trnh

C php: id = create_process(N);

Mc ch: to ra N tin trnh v mt tin trnh cha

thc hin cu lnh .

Nh th, c N+1 tin trnh nh nhau c to ra

v mi gi tr ca idc gn tng ng cho

mt tin trnh.


35/54

7/17/20

3

205

V d: to N tin trnh v giao n+1 Nhim v thc hin ssong

id = create_process(N);

switch(id){case 0: do NhiemVu0 ; break;case 1: do NhiemVu1 ; break;case 2: do NhiemVu2 ; break;

. . .case N: do NhiemVuN ; break;

}


206


b) Hy b tin trnh:

C php:join_process(N, id);

Mc ch: cho php tin trnh c gi tr id trong N tin trnh

khi to c tip tc thc hin nhng phn vic tun t cnli, cn nhng tin trnh khc kt thc.

Nu ta t sau join_process(N, id) mt cu lnh th cu lnh

ny s khng c thc hin cho n khi tt c cc tin trnh

u thc hin join_process(N, id). Do vn x l song

song khng xut hinm l x l tun t.

207

Vn l cc tin trnh c to lp v truy cp d liu canhau nh th no?

Mt mt mt tin trnh c th mun gi mt phn d liu cc bcho ring mnh, khng cho nhng tin trnh khc truy cp tinhng d liu .

Mt khc, n cng mun trao i thng tin vi cc tin trnhkhc.

X l vn che du hay chia s thng tin nh th no cntu thuc vo m hnh m chng ta s dng:tin trnh (process) haylung(thread).

Cc tin trnh trong UNIX c s dng nh cc n v tnhton c lp. Theo mc nh, vic tnh ton v cp nht b nhca mt tin trnh l khng nhn thy c t cc tin trnh

khc. i vi lung, tt c cc thng tin, theo mc nh, l nhn thyc.


208

Ch (1/3):

Khi mun s dng b nh chung, Ngi lp trnh cn phi xin

cp pht b nh v sau khi s dng xong phi gii phng

chng.

C hai hm c s thc hin iu ny:

shared(m, &id):xincp pht mbyte b nh chia s cho

tin trnh id.

free_shm():gii phng b nh c cp.

Cc hm shared(m, &id), free_shm() c gi l hm iu

phi v chia s b nh.


209

V d: Bi ton loi tr nh

Parallel Processing (1)

Documents

Transcript of Parallel Processing (1)