[Harvard CS264] 04 - Intermediate-level CUDA Programming

Lecture #4: Intermediate-level CUDA | February 15th, 2011

Nicolas Pinto (MIT, Harvard) [email protected]

Massively Parallel ComputingCS 264 / CSCI E-292

mailto:[email protected]

mailto:[email protected]

Administrivia

• HW1: due Fri 2/18/11 (this week)

• Projects: think about it, consult the staff

• New guest lecturers!

• Max Lin (Google), Kurt Messersmith et al. (Amazon), David Rich et al. (Microsoft)

During this course,

we’ll try to

and use existing material ;-)

“ ”

adapted for CS264

Todayyey!!

Outline

• CUDA Language & APIs (overview)

• Threading/Execution (cont’d)

• Memory/Communication (cont’d)

• Tools

• Libraries

!"#$$%&$'()*+,-.(/$$01(234-5(63*7,-5(8-,9:+5,;<(

=3*<+,.4;(>(?@;;4:A(B3C,;43(/$$0

/DEFD/$$0

G

! !"#$%&"'()*%)(%(+,-#)./-(.'(#/01.$"2()/(%-2(

"3"#,)"'(/-()*"(*/')

! 4%$$%5$"(/-$6(+&/0(%-/)*"&()*"(*/')

! 7,-#)./-'(8.)*/,)(%-6(49!:(2"#$'1"# %&"(

*/')(56(2"+%,$)

! 4%-(,'"(!!"#$%!! %-2(!!&'()*'!!+

)/;")*"&

! !"#$%&"'()*%)(%(+,-#)./-(.'(#/01.$"2()/(%-2(

"3"#,)"'(/-()*"(2"<.#"

! 4%$$%5$"(+&/0()*"(*/')

! 9'"2(%'()*"("-)&6(1/.-)(+&/0(*/')()/(2"<.#"

! 49!:(1&/<.2"'(%('")(/+(5,.$)=.-(<"#)/&()61"'>

! *",-./+0*",-./+*",-1/+0*",-1/+*",-2/+0*",-2/+*",-3/+0*",-3/+

! $"#-%./+0$"#-%./+$"#-%1/+0$"#-%1/+$"#-%2/+0$"#-%2/+$"#-%3/+0$"#-%3/

! )4%./+0)4%./+)4%1/+0)4%1/+)4%2/+0)4%2/+)4%3/+0)4%3/+

! 5#46./+05#46./+5#461/+05#461/+5#462/+05#462/+5#463/+05#463/+

! 75#,%./+75#,%1/+75#,%2/+75#,%3+

! 4%-(#/-')&,#)(%(<"#)/&()61"(8.)*('1"#.%$(

+,-#)./->

8,9'!!"#$%&'(%):(;/+(.!"#$

! 4%-(%##"''("$"0"-)'(/+(%(<"#)/&()61"(8.)*(

!"#$%&!"'$%&!"($%&!")$*

('*(,-<=

! &)82 .'(%('1"#.%$(<"#)/&()61"

! ?%0"(%'(0)4%2@("3#"1)(#%-(5"(#/-')&,#)"2(

+&/0(%('#%$%&()/(+/&0(%(<"#)/&>

:$*,5,-/+./+.>

! 49!:(1&/<.2"'(+/,&(;$/5%$@(5,.$)=.-(<%&.%5$"'

! %"-',&?&=@(@5#*9?&=@(@5#*9A)8@(

6-)&A)8

! +',-.&/0&/&1&)822&34&10)4%22&

! :##"''.5$"(/-$6(+&/0(2"<.#"(#/2"

! 4%--/)()%A"(%22&"''

! 4%--/)(%''.;-(<%$,"

!"#$$%&$'()*+,-.(/$$01(234-5(63*7,-5(8-,9:+5,;<(

=3*<+,.4;(>(?@;;4:A(B3C,;43(/$$0

/DE/D/$$0

#

! !"#$!%&'()*'+),-./'+0'1(2*'('3014*+-./'

4)0563+7''890:*'"0'+;*'%*+(9

! <-):+')*9*(:*'=>?$>@A'B01B*5

! C*30.5')*9*(:*'-.'D?$>@')*EF)-++*.

! G4*.'C06)3*

! H0+',*+'('I-(B9*':096+-0.

&*I-3*

%69+-EJ)03*::0) %69+-EJ)03*::0)

!

!"#$%&'()"*+ !"#$%&'()"*+

,(-.#(&'()"*+

/0 /0

/0 /0

/0 /0

! !

/0 /0

/0 /0

/0 /0

! !

%69+-EJ)03*::0)

!"#$%&'()"*+

/0 /0

/0 /0

/0 /0

! !

! ";*'K6.5(1*.+(9'6.-+'-:'+;*'!"#$%&'(#)*$!!)#

! !':3(9()L'1.23%(45*(#.1."2 K90(+-./'40-.+'!MN

! G.*'%69+-49,$!55'4*)'39032'3,39*

! 678 #OOOE@PQ'30149-(.+

! C+)*(1'4)03*::0):'()*'/)064*5'-.+0'&+,"-.

(#)*$!!)#!

! %69+-E4)03*::0):')6.'-.'/9',&%"#:1;(5

! !'.61B*)'0K'169+-E4)03*::0):'K0)1'('/$0-*$

%69+-EJ)03*::0)

<(3.1;(*1!"#$%&

'()"*+

!

/;*($)&0*"#(11"*

/=$*(>&'()"*+

?%"@$%&'()"*+

A"21;$2;&'()"*+

8(B;C*(&'()"*+

/;*($)&0*"#(11"*1&=$-(&$##(11&;"D

'()"*+&8+5( E##(11 /=$*.23

R*/-:+*): R*(5$S)-+* J)-I(+*

M03(9'%*10), R*(5$S)-+* J)-I(+*

C;()*5'%*10), R*(5$S)-+* %69+-EJ)03*::0)

T90B(9'%*10), R*(5$S)-+* &*I-3*

80.:+(.+'%*10), R*(5 &*I-3*

"*U+6)*'%*10), R*(5 &*I-3*

Language

!"#$$%&$'()*+,-.(/$$01(234-5(63*7,-5(8-,9:+5,;<(

=3*<+,.4;(>(?@;;4:A(B3C,;43(/$$0

/DEFD/$$0

G

! !"#$%&"'()*%)(%(+,-#)./-(.'(#/01.$"2()/(%-2(

"3"#,)"'(/-()*"(*/')

! 4%$$%5$"(/-$6(+&/0(%-/)*"&()*"(*/')

! 7,-#)./-'(8.)*/,)(%-6(49!:(2"#$'1"# %&"(

*/')(56(2"+%,$)

! 4%-(,'"(!!"#$%!! %-2(!!&'()*'!!+

)/;")*"&

! !"#$%&"'()*%)(%(+,-#)./-(.'(#/01.$"2()/(%-2(

"3"#,)"'(/-()*"(2"<.#"

! 4%$$%5$"(+&/0()*"(*/')

! 9'"2(%'()*"("-)&6(1/.-)(+&/0(*/')()/(2"<.#"

! 49!:(1&/<.2"'(%('")(/+(5,.$)=.-(<"#)/&()61"'>

! *",-./+0*",-./+*",-1/+0*",-1/+*",-2/+0*",-2/+*",-3/+0*",-3/+

! $"#-%./+0$"#-%./+$"#-%1/+0$"#-%1/+$"#-%2/+0$"#-%2/+$"#-%3/+0$"#-%3/

! )4%./+0)4%./+)4%1/+0)4%1/+)4%2/+0)4%2/+)4%3/+0)4%3/+

! 5#46./+05#46./+5#461/+05#461/+5#462/+05#462/+5#463/+05#463/+

! 75#,%./+75#,%1/+75#,%2/+75#,%3+

! 4%-(#/-')&,#)(%(<"#)/&()61"(8.)*('1"#.%$(

+,-#)./->

8,9'!!"#$%&'(%):(;/+(.!"#$

! 4%-(%##"''("$"0"-)'(/+(%(<"#)/&()61"(8.)*(

!"#$%&!"'$%&!"($%&!")$*

('*(,-<=

! &)82 .'(%('1"#.%$(<"#)/&()61"

! ?%0"(%'(0)4%2@("3#"1)(#%-(5"(#/-')&,#)"2(

+&/0(%('#%$%&()/(+/&0(%(<"#)/&>

:$*,5,-/+./+.>

! 49!:(1&/<.2"'(+/,&(;$/5%$@(5,.$)=.-(<%&.%5$"'

! %"-',&?&=@(@5#*9?&=@(@5#*9A)8@(

6-)&A)8

! +',-.&/0&/&1&)822&34&10)4%22&

! :##"''.5$"(/-$6(+&/0(2"<.#"(#/2"

! 4%--/)()%A"(%22&"''

! 4%--/)(%''.;-(<%$,"!"#$$%&$'()*+,-.(/$$01(234-5(63*7,-5(8-,9:+5,;<(

=3*<+,.4;(>(?@;;4:A(B3C,;43(/$$0

/DE/D/$$0

#

! !"#$!%&'()*'+),-./'+0'1(2*'('3014*+-./'

4)0563+7''890:*'"0'+;*'%*+(9

! <-):+')*9*(:*'=>?$>@A'B01B*5

! C*30.5')*9*(:*'-.'D?$>@')*EF)-++*.

! G4*.'C06)3*

! H0+',*+'('I-(B9*':096+-0.

&*I-3*

%69+-EJ)03*::0) %69+-EJ)03*::0)

!

!"#$%&'()"*+ !"#$%&'()"*+

,(-.#(&'()"*+

/0 /0

/0 /0

/0 /0

! !

/0 /0

/0 /0

/0 /0

! !

%69+-EJ)03*::0)

!"#$%&'()"*+

/0 /0

/0 /0

/0 /0

! !

! ";*'K6.5(1*.+(9'6.-+'-:'+;*'!"#$%&'(#)*$!!)#

! !':3(9()L'1.23%(45*(#.1."2 K90(+-./'40-.+'!MN

! G.*'%69+-49,$!55'4*)'39032'3,39*

! 678 #OOOE@PQ'30149-(.+

! C+)*(1'4)03*::0):'()*'/)064*5'-.+0'&+,"-.

(#)*$!!)#!

! %69+-E4)03*::0):')6.'-.'/9',&%"#:1;(5

! !'.61B*)'0K'169+-E4)03*::0):'K0)1'('/$0-*$

%69+-EJ)03*::0)

<(3.1;(*1!"#$%&

'()"*+

!

/;*($)&0*"#(11"*

/=$*(>&'()"*+

?%"@$%&'()"*+

A"21;$2;&'()"*+

8(B;C*(&'()"*+

/;*($)&0*"#(11"*1&=$-(&$##(11&;"D

'()"*+&8+5( E##(11 /=$*.23

R*/-:+*): R*(5$S)-+* J)-I(+*

M03(9'%*10), R*(5$S)-+* J)-I(+*

C;()*5'%*10), R*(5$S)-+* %69+-EJ)03*::0)

T90B(9'%*10), R*(5$S)-+* &*I-3*

80.:+(.+'%*10), R*(5 &*I-3*

"*U+6)*'%*10), R*(5 &*I-3*

Language

© NVIDIA Corporation 2010 54

!"#$%&'()*$+(,%-.*./

0+'.'(*.'($12($/3'4(25(."#$%&'(&*$+(23'.*$%2#4(%#(4%#67'(3.'8%4%2#

!!"#$%"&'9(:%.'8$(&*33%#6($2(+*.:1*.'(;<=>*4$(-"$(721'.(*88".*8/(?4''(3.26@(6"%:'(52.(:'$*%74ABC*&37'49(!!()$"&*'+,!!-*."&*'+,!!./0"&*+1'

"#$%"&',9(82&3%7'($2(&"7$%37'(%#4$."8$%2#4<721'.(-"$(+%6+'.(*88".*8/(?D("73(2.(7'44ABC*&37'49(()$"&*'+,-*."&*'+,./0"&*+1'

0+'(2#(-!"3(4!5346,82&3%7'.(23$%2#(52.8'4('E'./("#$%"&',$2(82&3%7'($2(!!"#$%"&'

Unit of Least Precision (ULP) is the gap between the floating-point numbers nearest a given real number

Language

© NVIDIA Corporation 2010

CUDA APIs

API allows the host to manage the devicesAllocate memory & transfer dataLaunch kernels

High level of abstraction - start here!

More control, more verbose

(OpenCL: Similar to CUDA C Driver API)

(aka “Device” API)

API

!"#$$%&$'()*+,-.(/$$01(234-5(63*7,-5(8-,9:+5,;<(

=3*<+,.4;(>(?@;;4:A(B3C,;43(/$$0

/D/#D/$$0

&

! !"#$%&'($)*+,$(-.$/0*123#+$4567,2*6+$4*08

! '#127#$9:6:;#9#6,

! <#9*0=$9:6:;#9#6,

! >,0#:9$9:6:;#9#6,

! ?1#6,$9:6:;#9#6,

! !#@,50#$9:6:;9#6,

! A/#6BCD'20#7,E$26,#0*/#0:F2G2,=

! !"#$)*+,$(-.$2+$#@/*+#3$:+$,H*$3244#0#6,$

!"#$%&

! !"#$G*H$G#1#G$'#127#$(-.$I/0#42@8$75J

! !"#$"2;"$G#1#G$K56,29#$(-.$I/0#42@8$753:J

! >*9#$,"26;+$7:6$F#$3*6#$,"0*5;"$F*,"$(-.+L$*,"#0+$:0#$+/#72:G2M#3

! %:6$F#$92@#3$,*;#,"#0$IH2,"$7:0#J

! (GG$B-&$7*9/5,26;$2+$/#04*09#3$*6$:$3#127#! !*$:GG*7:,#$9#9*0=L$056$:$/0*;0:9L$#,7$*6$,"#$":03H:0#L$H#$6##3$:$!"#$%"&%'()"*)

! '#127#$7*6,#@,+$:0#$F*563$N8N$H2,"$"*+,$,"0#:3+$IO5+,$G2P#$A/#6BCQJ! >*L$#:7"$"*+,$,"0#:3$9:=$":1#$:,$9*+,$*6#$3#127#$7*6,#@,

! (63L$#:7"$3#127#$7*6,#@,$2+$:77#++2FG#$40*9$*6G=$*6#$"*+,$,"0#:3

! (GG$3#127#$(-.$7:GG+$0#,506$:6$#00*0D+577#++$

7*3#$*4$,=/#8$+,-"./0)

! (GG$056,29#$(-.$7:GG+$0#,506$:6$#00*0D+577#++$

7*3#$*4$,=/#$%/!12--'-3)

! (6$26,#;#0$1:G5#$H2,"$M#0*$R$6*$#00*0

! %/!14")51.)2--'-L$%/!14")2--'-6)-$(7

! K56,29#$(-.$7:GG+$:5,*9:,27:GG=$262,2:G2M#

! '#127#$(-.$7:GG+$95+,$7:GG$%/8($)

! !"#$420+,$I*/,2*6:GSJ$+,#/$2+$,*$#659#0:,#$,"#$

:1:2G:FG#$3#127#+

! %/9"#$%"4")+'/()

! %/9"#$%"4")

! %/9"#$%"4"):1;"

! %/9"#$%"4")<')10=";'->

! %/9"#$%"4")?))-$@/)"

! !

!"#$$%&$'()*+,-.(/$$01(234-5(63*7,-5(8-,9:+5,;<(

=3*<+,.4;(>(?@;;4:A(B3C,;43(/$$0

/DE/D/$$0

#

! !"#$!%&'()*'+),-./'+0'1(2*'('3014*+-./'

4)0563+7''890:*'"0'+;*'%*+(9

! <-):+')*9*(:*'=>?$>@A'B01B*5

! C*30.5')*9*(:*'-.'D?$>@')*EF)-++*.

! G4*.'C06)3*

! H0+',*+'('I-(B9*':096+-0.

&*I-3*

%69+-EJ)03*::0) %69+-EJ)03*::0)

!

!"#$%&'()"*+ !"#$%&'()"*+

,(-.#(&'()"*+

/0 /0

/0 /0

/0 /0

! !

/0 /0

/0 /0

/0 /0

! !

%69+-EJ)03*::0)

!"#$%&'()"*+

/0 /0

/0 /0

/0 /0

! !

! ";*'K6.5(1*.+(9'6.-+'-:'+;*'!"#$%&'(#)*$!!)#

! !':3(9()L'1.23%(45*(#.1."2 K90(+-./'40-.+'!MN

! G.*'%69+-49,$!55'4*)'39032'3,39*

! 678 #OOOE@PQ'30149-(.+

! C+)*(1'4)03*::0):'()*'/)064*5'-.+0'&+,"-.

(#)*$!!)#!

! %69+-E4)03*::0):')6.'-.'/9',&%"#:1;(5

! !'.61B*)'0K'169+-E4)03*::0):'K0)1'('/$0-*$

%69+-EJ)03*::0)

<(3.1;(*1!"#$%&

'()"*+

!

/;*($)&0*"#(11"*

/=$*(>&'()"*+

?%"@$%&'()"*+

A"21;$2;&'()"*+

8(B;C*(&'()"*+

/;*($)&0*"#(11"*1&=$-(&$##(11&;"D

'()"*+&8+5( E##(11 /=$*.23

R*/-:+*): R*(5$S)-+* J)-I(+*

M03(9'%*10), R*(5$S)-+* J)-I(+*

C;()*5'%*10), R*(5$S)-+* %69+-EJ)03*::0)

T90B(9'%*10), R*(5$S)-+* &*I-3*

80.:+(.+'%*10), R*(5 &*I-3*

"*U+6)*'%*10), R*(5 &*I-3*

API

Runtime API(high-level)

!"#$$%&$'()*+,-.(/$$01(234-5(63*7,-5(8-,9:+5,;<(

=3*<+,.4;(>(?@;;4:A(B3C,;43(/$$0

/D/#D/$$0

E

! !"#$%&$%#'(()$%*%+$,-#$%&-.'%!"#$%&!$'$(

&$%/$.%*%+$,-#$%'*"+0$%(1%.23$%)*+$%&!$

! 4*"%"(&%#5$*.$%*%#(".$6.%&-.'%!")(,)-$.($

! 78".-9$%:;<%35(,-+$)%*%)-930-1-$+%-".$51*#$%

1(5%#5$*.-"/%*%#(".$6.=

! !"+.'$(#$%&!$)/"0(

! !"+.1$(#$%&!$

! :"+%.'$%8)$180=

! !"+.)2//3$#$%&!$

! !"#$%&$%'*,$%*%#(".$6.%>)*!/0($,(?%#*"%

*00(#*.$%9$9(52@%#*00%*%A;B%18"#.-("%$.#C%%

! 4(".$6.%-)%-930-#-.02%*))(#-*.$+%&-.'%#5$*.-"/%

.'5$*+

! D(%)2"#'5("-E$%*00%.'5$*+)%>4;B%'().%&-.'%

A;B%.'5$*+)?%#*00%!")(,140!2-/0&5$

! F*-.)%1(5%*00%A;B%.*)G)%.(%1-"-)'%

! :00(#*.$HI5$$%9$9(52=

! !"6$7899/!:;!"6$7<-$$

! <"-.-*0-E$%9$9(52=

! !"6$73$(

! 4(32%9$9(52=

! !"6$7!=4>(/#:;!"6$7!=4#(/>:;

!"6$7!=4#(/#

! F'$"%*00(#*.-"/%9$9(52%1(5%.'$%2/3(@%#*"%

8)$%!"##$% H%&'( H%!!")

! !5%8)$%!"6$7899/!>/3(@%!"6$7<-$$>/3(

! D'$)$%18"#.-(")%*00(#*.$%'().%9$9(52%.'*.%-)%

)"*'+#$%,'-

! ;$51(59*"#$%-935(,$+%1(5%#(32%.(H15(9%

3*/$J0(#G$+%'().%9$9(52

! :00(#*.$HI5$$%9$9(52=

! !"+.6.99/!@%!"+.<-$$

! <"-.-*0-E$%9$9(52=

! !"+.6$73$(

! 4(32%9$9(52=

! !"+.6$7!=4

!"#$$%&$'()*+,-.(/$$01(234-5(63*7,-5(8-,9:+5,;<(

=3*<+,.4;(>(?@;;4:A(B3C,;43(/$$0

/DE/D/$$0

#

! !"#$!%&'()*'+),-./'+0'1(2*'('3014*+-./'

4)0563+7''890:*'"0'+;*'%*+(9

! <-):+')*9*(:*'=>?$>@A'B01B*5

! C*30.5')*9*(:*'-.'D?$>@')*EF)-++*.

! G4*.'C06)3*

! H0+',*+'('I-(B9*':096+-0.

&*I-3*

%69+-EJ)03*::0) %69+-EJ)03*::0)

!

!"#$%&'()"*+ !"#$%&'()"*+

,(-.#(&'()"*+

/0 /0

/0 /0

/0 /0

! !

/0 /0

/0 /0

/0 /0

! !

%69+-EJ)03*::0)

!"#$%&'()"*+

/0 /0

/0 /0

/0 /0

! !

! ";*'K6.5(1*.+(9'6.-+'-:'+;*'!"#$%&'(#)*$!!)#

! !':3(9()L'1.23%(45*(#.1."2 K90(+-./'40-.+'!MN

! G.*'%69+-49,$!55'4*)'39032'3,39*

! 678 #OOOE@PQ'30149-(.+

! C+)*(1'4)03*::0):'()*'/)064*5'-.+0'&+,"-.

(#)*$!!)#!

! %69+-E4)03*::0):')6.'-.'/9',&%"#:1;(5

! !'.61B*)'0K'169+-E4)03*::0):'K0)1'('/$0-*$

%69+-EJ)03*::0)

<(3.1;(*1!"#$%&

'()"*+

!

/;*($)&0*"#(11"*

/=$*(>&'()"*+

?%"@$%&'()"*+

A"21;$2;&'()"*+

8(B;C*(&'()"*+

/;*($)&0*"#(11"*1&=$-(&$##(11&;"D

'()"*+&8+5( E##(11 /=$*.23

R*/-:+*): R*(5$S)-+* J)-I(+*

M03(9'%*10), R*(5$S)-+* J)-I(+*

C;()*5'%*10), R*(5$S)-+* %69+-EJ)03*::0)

T90B(9'%*10), R*(5$S)-+* &*I-3*

80.:+(.+'%*10), R*(5 &*I-3*

"*U+6)*'%*10), R*(5 &*I-3*

API

!"#$$%&$'()*+,-.(/$$01(234-5(63*7,-5(8-,9:+5,;<(

=3*<+,.4;(>(?@;;4:A(B3C,;43(/$$0

/D/#D/$$0

'

! !"#$%&#'(%#)%)(*%+*%*,(%)+-(%*#-(%+)%*,(%

./01*#20%#0321+*#204

!"#$"%!&'()*

! +,!$--. !"#$%&#'()*+,#-*#%."#&+*/#01*#2223444#

'&"%0(5"#("65%.0(5"#758*9.059:

! 5,(%12-6#7("%8(0("+*()%1+77)%*2%+77%$(3#1(%9:;%

*2%)(*/6%*,(%(<(1/*#20%(03#"20-(0*

! 9%)*"(+-%#)%+%)(=/(01(%2.%26("+*#20)%*,+*%

211/"%#0%2"$("%%>?8?

@? A26B%$+*+%."2-%,2)*%*2%$(3#1(

C? ><(1/*(%$(3#1(%./01*#20%

D? A26B%$+*+%."2-%$(3#1(%*2%,2)*

! 9%)*"(+-%#)%+%)(=/(01(%2.%26("+*#20)%*,+*%

211/"%#0%2"$("%%>?8?

@? A26B%$+*+%."2-%,2)*%*2%$(3#1(

C? ><(1/*(%$(3#1(%./01*#20%

D? A26B%$+*+%."2-%$(3#1(%*2%,2)*

! 9%)*"(+-%#)%+%)(=/(01(%2.%26("+*#20)%*,+*%

211/"%#0%2"$("

! E#..("(0*%)*"(+-)%1+0%F(%/)($%*2%-+0+8(%

1201/""(01B%%>?8?

G3("7+66#08%-(-2"B%126B%."2-%20(%)*"(+-%H#*,%*,(%./01*#20%(<(1/*#20%."2-%+02*,("

!"#$$%&$'()*+,-.(/$$01(234-5(63*7,-5(8-,9:+5,;<(

=3*<+,.4;(>(?@;;4:A(B3C,;43(/$$0

/DE/D/$$0

#

! !"#$!%&'()*'+),-./'+0'1(2*'('3014*+-./'

4)0563+7''890:*'"0'+;*'%*+(9

! <-):+')*9*(:*'=>?$>@A'B01B*5

! C*30.5')*9*(:*'-.'D?$>@')*EF)-++*.

! G4*.'C06)3*

! H0+',*+'('I-(B9*':096+-0.

&*I-3*

%69+-EJ)03*::0) %69+-EJ)03*::0)

!

!"#$%&'()"*+ !"#$%&'()"*+

,(-.#(&'()"*+

/0 /0

/0 /0

/0 /0

! !

/0 /0

/0 /0

/0 /0

! !

%69+-EJ)03*::0)

!"#$%&'()"*+

/0 /0

/0 /0

/0 /0

! !

! ";*'K6.5(1*.+(9'6.-+'-:'+;*'!"#$%&'(#)*$!!)#

! !':3(9()L'1.23%(45*(#.1."2 K90(+-./'40-.+'!MN

! G.*'%69+-49,$!55'4*)'39032'3,39*

! 678 #OOOE@PQ'30149-(.+

! C+)*(1'4)03*::0):'()*'/)064*5'-.+0'&+,"-.

(#)*$!!)#!

! %69+-E4)03*::0):')6.'-.'/9',&%"#:1;(5

! !'.61B*)'0K'169+-E4)03*::0):'K0)1'('/$0-*$

%69+-EJ)03*::0)

<(3.1;(*1!"#$%&

'()"*+

!

/;*($)&0*"#(11"*

/=$*(>&'()"*+

?%"@$%&'()"*+

A"21;$2;&'()"*+

8(B;C*(&'()"*+

/;*($)&0*"#(11"*1&=$-(&$##(11&;"D

'()"*+&8+5( E##(11 /=$*.23

R*/-:+*): R*(5$S)-+* J)-I(+*

M03(9'%*10), R*(5$S)-+* J)-I(+*

C;()*5'%*10), R*(5$S)-+* %69+-EJ)03*::0)

T90B(9'%*10), R*(5$S)-+* &*I-3*

80.:+(.+'%*10), R*(5 &*I-3*

"*U+6)*'%*10), R*(5 &*I-3*

API

“Device” Driver API(low-level)

!"#$$%&$'()*+,-.(/$$01(234-5(63*7,-5(8-,9:+5,;<(

=3*<+,.4;(>(?@;;4:A(B3C,;43(/$$0

/D/#D/$$0

&

! !"#$%&'($)*+,$(-.$/0*123#+$4567,2*6+$4*08

! '#127#$9:6:;#9#6,

! <#9*0=$9:6:;#9#6,

! >,0#:9$9:6:;#9#6,

! ?1#6,$9:6:;#9#6,

! !#@,50#$9:6:;9#6,

! A/#6BCD'20#7,E$26,#0*/#0:F2G2,=

! !"#$)*+,$(-.$2+$#@/*+#3$:+$,H*$3244#0#6,$

!"#$%&

! !"#$G*H$G#1#G$'#127#$(-.$I/0#42@8$75J

! !"#$"2;"$G#1#G$K56,29#$(-.$I/0#42@8$753:J

! >*9#$,"26;+$7:6$F#$3*6#$,"0*5;"$F*,"$(-.+L$*,"#0+$:0#$+/#72:G2M#3

! %:6$F#$92@#3$,*;#,"#0$IH2,"$7:0#J

! (GG$B-&$7*9/5,26;$2+$/#04*09#3$*6$:$3#127#! !*$:GG*7:,#$9#9*0=L$056$:$/0*;0:9L$#,7$*6$,"#$":03H:0#L$H#$6##3$:$!"#$%"&%'()"*)

! '#127#$7*6,#@,+$:0#$F*563$N8N$H2,"$"*+,$,"0#:3+$IO5+,$G2P#$A/#6BCQJ! >*L$#:7"$"*+,$,"0#:3$9:=$":1#$:,$9*+,$*6#$3#127#$7*6,#@,

! (63L$#:7"$3#127#$7*6,#@,$2+$:77#++2FG#$40*9$*6G=$*6#$"*+,$,"0#:3

! (GG$3#127#$(-.$7:GG+$0#,506$:6$#00*0D+577#++$

7*3#$*4$,=/#8$+,-"./0)

! (GG$056,29#$(-.$7:GG+$0#,506$:6$#00*0D+577#++$

7*3#$*4$,=/#$%/!12--'-3)

! (6$26,#;#0$1:G5#$H2,"$M#0*$R$6*$#00*0

! %/!14")51.)2--'-L$%/!14")2--'-6)-$(7

! K56,29#$(-.$7:GG+$:5,*9:,27:GG=$262,2:G2M#

! '#127#$(-.$7:GG+$95+,$7:GG$%/8($)

! !"#$420+,$I*/,2*6:GSJ$+,#/$2+$,*$#659#0:,#$,"#$

:1:2G:FG#$3#127#+

! %/9"#$%"4")+'/()

! %/9"#$%"4")

! %/9"#$%"4"):1;"

! %/9"#$%"4")<')10=";'->

! %/9"#$%"4")?))-$@/)"

! !

!"#$$%&$'()*+,-.(/$$01(234-5(63*7,-5(8-,9:+5,;<(

=3*<+,.4;(>(?@;;4:A(B3C,;43(/$$0

/DE/D/$$0

#

! !"#$!%&'()*'+),-./'+0'1(2*'('3014*+-./'

4)0563+7''890:*'"0'+;*'%*+(9

! <-):+')*9*(:*'=>?$>@A'B01B*5

! C*30.5')*9*(:*'-.'D?$>@')*EF)-++*.

! G4*.'C06)3*

! H0+',*+'('I-(B9*':096+-0.

&*I-3*

%69+-EJ)03*::0) %69+-EJ)03*::0)

!

!"#$%&'()"*+ !"#$%&'()"*+

,(-.#(&'()"*+

/0 /0

/0 /0

/0 /0

! !

/0 /0

/0 /0

/0 /0

! !

%69+-EJ)03*::0)

!"#$%&'()"*+

/0 /0

/0 /0

/0 /0

! !

! ";*'K6.5(1*.+(9'6.-+'-:'+;*'!"#$%&'(#)*$!!)#

! !':3(9()L'1.23%(45*(#.1."2 K90(+-./'40-.+'!MN

! G.*'%69+-49,$!55'4*)'39032'3,39*

! 678 #OOOE@PQ'30149-(.+

! C+)*(1'4)03*::0):'()*'/)064*5'-.+0'&+,"-.

(#)*$!!)#!

! %69+-E4)03*::0):')6.'-.'/9',&%"#:1;(5

! !'.61B*)'0K'169+-E4)03*::0):'K0)1'('/$0-*$

%69+-EJ)03*::0)

<(3.1;(*1!"#$%&

'()"*+

!

/;*($)&0*"#(11"*

/=$*(>&'()"*+

?%"@$%&'()"*+

A"21;$2;&'()"*+

8(B;C*(&'()"*+

/;*($)&0*"#(11"*1&=$-(&$##(11&;"D

'()"*+&8+5( E##(11 /=$*.23

R*/-:+*): R*(5$S)-+* J)-I(+*

M03(9'%*10), R*(5$S)-+* J)-I(+*

C;()*5'%*10), R*(5$S)-+* %69+-EJ)03*::0)

T90B(9'%*10), R*(5$S)-+* &*I-3*

80.:+(.+'%*10), R*(5 &*I-3*

"*U+6)*'%*10), R*(5 &*I-3*

API

!"#$$%&$'()*+,-.(/$$01(234-5(63*7,-5(8-,9:+5,;<(

=3*<+,.4;(>(?@;;4:A(B3C,;43(/$$0

/D/#D/$$0

E

! !"#$%&$%#'(()$%*%+$,-#$%&-.'%!"#$%&!$'$(

&$%/$.%*%+$,-#$%'*"+0$%(1%.23$%)*+$%&!$

! 4*"%"(&%#5$*.$%*%#(".$6.%&-.'%!")(,)-$.($

! 78".-9$%:;<%35(,-+$)%*%)-930-1-$+%-".$51*#$%

1(5%#5$*.-"/%*%#(".$6.=

! !"+.'$(#$%&!$)/"0(

! !"+.1$(#$%&!$

! :"+%.'$%8)$180=

! !"+.)2//3$#$%&!$

! !"#$%&$%'*,$%*%#(".$6.%>)*!/0($,(?%#*"%

*00(#*.$%9$9(52@%#*00%*%A;B%18"#.-("%$.#C%%

! 4(".$6.%-)%-930-#-.02%*))(#-*.$+%&-.'%#5$*.-"/%

.'5$*+

! D(%)2"#'5("-E$%*00%.'5$*+)%>4;B%'().%&-.'%

A;B%.'5$*+)?%#*00%!")(,140!2-/0&5$

! F*-.)%1(5%*00%A;B%.*)G)%.(%1-"-)'%

! :00(#*.$HI5$$%9$9(52=

! !"6$7899/!:;!"6$7<-$$

! <"-.-*0-E$%9$9(52=

! !"6$73$(

! 4(32%9$9(52=

! !"6$7!=4>(/#:;!"6$7!=4#(/>:;

!"6$7!=4#(/#

! F'$"%*00(#*.-"/%9$9(52%1(5%.'$%2/3(@%#*"%

8)$%!"##$% H%&'( H%!!")

! !5%8)$%!"6$7899/!>/3(@%!"6$7<-$$>/3(

! D'$)$%18"#.-(")%*00(#*.$%'().%9$9(52%.'*.%-)%

)"*'+#$%,'-

! ;$51(59*"#$%-935(,$+%1(5%#(32%.(H15(9%

3*/$J0(#G$+%'().%9$9(52

! :00(#*.$HI5$$%9$9(52=

! !"+.6.99/!@%!"+.<-$$

! <"-.-*0-E$%9$9(52=

! !"+.6$73$(

! 4(32%9$9(52=

! !"+.6$7!=4

!"#$$%&$'()*+,-.(/$$01(234-5(63*7,-5(8-,9:+5,;<(

=3*<+,.4;(>(?@;;4:A(B3C,;43(/$$0

/DE/D/$$0

#

! !"#$!%&'()*'+),-./'+0'1(2*'('3014*+-./'

4)0563+7''890:*'"0'+;*'%*+(9

! <-):+')*9*(:*'=>?$>@A'B01B*5

! C*30.5')*9*(:*'-.'D?$>@')*EF)-++*.

! G4*.'C06)3*

! H0+',*+'('I-(B9*':096+-0.

&*I-3*

%69+-EJ)03*::0) %69+-EJ)03*::0)

!

!"#$%&'()"*+ !"#$%&'()"*+

,(-.#(&'()"*+

/0 /0

/0 /0

/0 /0

! !

/0 /0

/0 /0

/0 /0

! !

%69+-EJ)03*::0)

!"#$%&'()"*+

/0 /0

/0 /0

/0 /0

! !

! ";*'K6.5(1*.+(9'6.-+'-:'+;*'!"#$%&'(#)*$!!)#

! !':3(9()L'1.23%(45*(#.1."2 K90(+-./'40-.+'!MN

! G.*'%69+-49,$!55'4*)'39032'3,39*

! 678 #OOOE@PQ'30149-(.+

! C+)*(1'4)03*::0):'()*'/)064*5'-.+0'&+,"-.

(#)*$!!)#!

! %69+-E4)03*::0):')6.'-.'/9',&%"#:1;(5

! !'.61B*)'0K'169+-E4)03*::0):'K0)1'('/$0-*$

%69+-EJ)03*::0)

<(3.1;(*1!"#$%&

'()"*+

!

/;*($)&0*"#(11"*

/=$*(>&'()"*+

?%"@$%&'()"*+

A"21;$2;&'()"*+

8(B;C*(&'()"*+

/;*($)&0*"#(11"*1&=$-(&$##(11&;"D

'()"*+&8+5( E##(11 /=$*.23

R*/-:+*): R*(5$S)-+* J)-I(+*

M03(9'%*10), R*(5$S)-+* J)-I(+*

C;()*5'%*10), R*(5$S)-+* %69+-EJ)03*::0)

T90B(9'%*10), R*(5$S)-+* &*I-3*

80.:+(.+'%*10), R*(5 &*I-3*

"*U+6)*'%*10), R*(5 &*I-3*

API

!"#$$%&$'()*+,-.(/$$01(234-5(63*7,-5(8-,9:+5,;<(

=3*<+,.4;(>(?@;;4:A(B3C,;43(/$$0

/D/#D/$$0

#

! !"#$%&''(!!"#$%"&''(%&$#"!"#$%&)#)(*+! ,&-"&'.("&''(%&$#"%&&%')#)(*+"/012

! 3**&+."&*#"%*#&$#4"56$7"&".8#%696%"564$7"&-4"7#6:7$"&-4"#'#)#-$"$+8#

! ;#)(*+"'&+(<$"6."(8$6)6=#4"/#>:>"8&%?6-:2"@+"*<-$6)#

! !"&))*+,)$*-$! !"&))*+.$/-)(+! !"#$%!0+.-(&! !"#$%!0+1-(&!"#

! 3")(4<'#"6."&"@'(@"(9"ABC"%(4#D4&$&"&'(-:"

56$7".()#"$+8#"6-9(*)&$6(-

! >%<@6- 96'#.

! 3")(4<'#"6."%*#&$#4"@+"'(&46-:"&"%<@6- 56$7"

!"#(2"'$,)$*-$ (*"!"#(2"'$3(*2.*-*

! ;(4<'#"%&-"@#"<-'(&4#4"56$7"

!"#(2"'$45'(*2

! E(&46-:"&")(4<'#"&'.("%(86#."6$"$("$7#"4#F6%#

! ,&-"$7#-":#$"$7#"&44*#.."(9"9<-%$6(-."&-4"

:'(@&'"F&*6&@'#.G

!"#(2"'$6$-7"5!-8(5

!"#(2"'$6$-6'(9*'

!"#(2"'$6$-:$;<$=

! H-%#"&")(4<'#"6."'(&4#4!"&-4"5#"7&F#"&"

9<-%$6(-"8(6-$#*!"5#"%&-"%&''"&"9<-%$6(-

! I#")<.$".#$<8"$7#"!"!#$%&'()!(*&+'(,!(%)

96*.$

! JK#%<$6(-"#-F6*(-)#-$"6-%'<4#.G

" L7*#&4"M'(%?"N6=#

" N7&*#4";#)(*+"N6=#

" O<-%$6(-"B&*&)#$#*.

" A*64"N6=#

! L7*#&4"M'(%?"N6=#G"

!"7"5!>$-?'(!@>A*0$

! N7&*#4";#)(*+"N6=#G

!"7"5!>$->A*)$2>8B$

! O<-%$6(-"B&*&)#$#*.G

!"C*)*%>$->8B$DE!"C*)*%>$-8DE

!"C*)*%>$-=DE!"C*)*%>$-F

!"#$$%&$'()*+,-.(/$$01(234-5(63*7,-5(8-,9:+5,;<(

=3*<+,.4;(>(?@;;4:A(B3C,;43(/$$0

/DE/D/$$0

#

! !"#$!%&'()*'+),-./'+0'1(2*'('3014*+-./'

4)0563+7''890:*'"0'+;*'%*+(9

! <-):+')*9*(:*'=>?$>@A'B01B*5

! C*30.5')*9*(:*'-.'D?$>@')*EF)-++*.

! G4*.'C06)3*

! H0+',*+'('I-(B9*':096+-0.

&*I-3*

%69+-EJ)03*::0) %69+-EJ)03*::0)

!

!"#$%&'()"*+ !"#$%&'()"*+

,(-.#(&'()"*+

/0 /0

/0 /0

/0 /0

! !

/0 /0

/0 /0

/0 /0

! !

%69+-EJ)03*::0)

!"#$%&'()"*+

/0 /0

/0 /0

/0 /0

! !

! ";*'K6.5(1*.+(9'6.-+'-:'+;*'!"#$%&'(#)*$!!)#

! !':3(9()L'1.23%(45*(#.1."2 K90(+-./'40-.+'!MN

! G.*'%69+-49,$!55'4*)'39032'3,39*

! 678 #OOOE@PQ'30149-(.+

! C+)*(1'4)03*::0):'()*'/)064*5'-.+0'&+,"-.

(#)*$!!)#!

! %69+-E4)03*::0):')6.'-.'/9',&%"#:1;(5

! !'.61B*)'0K'169+-E4)03*::0):'K0)1'('/$0-*$

%69+-EJ)03*::0)

<(3.1;(*1!"#$%&

'()"*+

!

/;*($)&0*"#(11"*

/=$*(>&'()"*+

?%"@$%&'()"*+

A"21;$2;&'()"*+

8(B;C*(&'()"*+

/;*($)&0*"#(11"*1&=$-(&$##(11&;"D

'()"*+&8+5( E##(11 /=$*.23

R*/-:+*): R*(5$S)-+* J)-I(+*

M03(9'%*10), R*(5$S)-+* J)-I(+*

C;()*5'%*10), R*(5$S)-+* %69+-EJ)03*::0)

T90B(9'%*10), R*(5$S)-+* &*I-3*

80.:+(.+'%*10), R*(5 &*I-3*

"*U+6)*'%*10), R*(5 &*I-3*

API

!"#$$%&$'()*+,-.(/$$01(234-5(63*7,-5(8-,9:+5,;<(

=3*<+,.4;(>(?@;;4:A(B3C,;43(/$$0

/D/#D/$$0

#

! !"#$%&''(!!"#$%"&''(%&$#"!"#$%&)#)(*+! ,&-"&'.("&''(%&$#"%&&%')#)(*+"/012

! 3**&+."&*#"%*#&$#4"56$7"&".8#%696%"564$7"&-4"7#6:7$"&-4"#'#)#-$"$+8#

! ;#)(*+"'&+(<$"6."(8$6)6=#4"/#>:>"8&%?6-:2"@+"*<-$6)#

! !"&))*+,)$*-$! !"&))*+.$/-)(+! !"#$%!0+.-(&! !"#$%!0+1-(&!"#

! 3")(4<'#"6."&"@'(@"(9"ABC"%(4#D4&$&"&'(-:"

56$7".()#"$+8#"6-9(*)&$6(-

! >%<@6- 96'#.

! 3")(4<'#"6."%*#&$#4"@+"'(&46-:"&"%<@6- 56$7"

!"#(2"'$,)$*-$ (*"!"#(2"'$3(*2.*-*

! ;(4<'#"%&-"@#"<-'(&4#4"56$7"

!"#(2"'$45'(*2

! E(&46-:"&")(4<'#"&'.("%(86#."6$"$("$7#"4#F6%#

! ,&-"$7#-":#$"$7#"&44*#.."(9"9<-%$6(-."&-4"

:'(@&'"F&*6&@'#.G

!"#(2"'$6$-7"5!-8(5

!"#(2"'$6$-6'(9*'

!"#(2"'$6$-:$;<$=

! H-%#"&")(4<'#"6."'(&4#4!"&-4"5#"7&F#"&"

9<-%$6(-"8(6-$#*!"5#"%&-"%&''"&"9<-%$6(-

! I#")<.$".#$<8"$7#"!"!#$%&'()!(*&+'(,!(%)

96*.$

! JK#%<$6(-"#-F6*(-)#-$"6-%'<4#.G

" L7*#&4"M'(%?"N6=#

" N7&*#4";#)(*+"N6=#

" O<-%$6(-"B&*&)#$#*.

" A*64"N6=#

! L7*#&4"M'(%?"N6=#G"

!"7"5!>$-?'(!@>A*0$

! N7&*#4";#)(*+"N6=#G

!"7"5!>$->A*)$2>8B$

! O<-%$6(-"B&*&)#$#*.G

!"C*)*%>$->8B$DE!"C*)*%>$-8DE

!"C*)*%>$-=DE!"C*)*%>$-F!"#$$%&$'()*+,-.(/$$01(234-5(63*7,-5(8-,9:+5,;<(

=3*<+,.4;(>(?@;;4:A(B3C,;43(/$$0

/DE/D/$$0

#

! !"#$!%&'()*'+),-./'+0'1(2*'('3014*+-./'

4)0563+7''890:*'"0'+;*'%*+(9

! <-):+')*9*(:*'=>?$>@A'B01B*5

! C*30.5')*9*(:*'-.'D?$>@')*EF)-++*.

! G4*.'C06)3*

! H0+',*+'('I-(B9*':096+-0.

&*I-3*

%69+-EJ)03*::0) %69+-EJ)03*::0)

!

!"#$%&'()"*+ !"#$%&'()"*+

,(-.#(&'()"*+

/0 /0

/0 /0

/0 /0

! !

/0 /0

/0 /0

/0 /0

! !

%69+-EJ)03*::0)

!"#$%&'()"*+

/0 /0

/0 /0

/0 /0

! !

! ";*'K6.5(1*.+(9'6.-+'-:'+;*'!"#$%&'(#)*$!!)#

! !':3(9()L'1.23%(45*(#.1."2 K90(+-./'40-.+'!MN

! G.*'%69+-49,$!55'4*)'39032'3,39*

! 678 #OOOE@PQ'30149-(.+

! C+)*(1'4)03*::0):'()*'/)064*5'-.+0'&+,"-.

(#)*$!!)#!

! %69+-E4)03*::0):')6.'-.'/9',&%"#:1;(5

! !'.61B*)'0K'169+-E4)03*::0):'K0)1'('/$0-*$

%69+-EJ)03*::0)

<(3.1;(*1!"#$%&

'()"*+

!

/;*($)&0*"#(11"*

/=$*(>&'()"*+

?%"@$%&'()"*+

A"21;$2;&'()"*+

8(B;C*(&'()"*+

/;*($)&0*"#(11"*1&=$-(&$##(11&;"D

'()"*+&8+5( E##(11 /=$*.23

R*/-:+*): R*(5$S)-+* J)-I(+*

M03(9'%*10), R*(5$S)-+* J)-I(+*

C;()*5'%*10), R*(5$S)-+* %69+-EJ)03*::0)

T90B(9'%*10), R*(5$S)-+* &*I-3*

80.:+(.+'%*10), R*(5 &*I-3*

"*U+6)*'%*10), R*(5 &*I-3*

API

!"#$$%&$'()*+,-.(/$$01(234-5(63*7,-5(8-,9:+5,;<(

=3*<+,.4;(>(?@;;4:A(B3C,;43(/$$0

/D/#D/$$0

'

! !"#$%&#'(%#)%)(*%+*%*,(%)+-(%*#-(%+)%*,(%

./01*#20%#0321+*#204

!"#$"%!&'()*

! +,!$--. !"#$%&#'()*+,#-*#%."#&+*/#01*#2223444#

'&"%0(5"#("65%.0(5"#758*9.059:

! 5,(%12-6#7("%8(0("+*()%1+77)%*2%+77%$(3#1(%9:;%

*2%)(*/6%*,(%(<(1/*#20%(03#"20-(0*

! 9%)*"(+-%#)%+%)(=/(01(%2.%26("+*#20)%*,+*%

211/"%#0%2"$("%%>?8?

@? A26B%$+*+%."2-%,2)*%*2%$(3#1(

C? ><(1/*(%$(3#1(%./01*#20%

D? A26B%$+*+%."2-%$(3#1(%*2%,2)*

! 9%)*"(+-%#)%+%)(=/(01(%2.%26("+*#20)%*,+*%

211/"%#0%2"$("%%>?8?

@? A26B%$+*+%."2-%,2)*%*2%$(3#1(

C? ><(1/*(%$(3#1(%./01*#20%

D? A26B%$+*+%."2-%$(3#1(%*2%,2)*

! 9%)*"(+-%#)%+%)(=/(01(%2.%26("+*#20)%*,+*%

211/"%#0%2"$("

! E#..("(0*%)*"(+-)%1+0%F(%/)($%*2%-+0+8(%

1201/""(01B%%>?8?

G3("7+66#08%-(-2"B%126B%."2-%20(%)*"(+-%H#*,%*,(%./01*#20%(<(1/*#20%."2-%+02*,("

!"#$$%&$'()*+,-.(/$$01(234-5(63*7,-5(8-,9:+5,;<(

=3*<+,.4;(>(?@;;4:A(B3C,;43(/$$0

/DE/D/$$0

#

! !"#$!%&'()*'+),-./'+0'1(2*'('3014*+-./'

4)0563+7''890:*'"0'+;*'%*+(9

! <-):+')*9*(:*'=>?$>@A'B01B*5

! C*30.5')*9*(:*'-.'D?$>@')*EF)-++*.

! G4*.'C06)3*

! H0+',*+'('I-(B9*':096+-0.

&*I-3*

%69+-EJ)03*::0) %69+-EJ)03*::0)

!

!"#$%&'()"*+ !"#$%&'()"*+

,(-.#(&'()"*+

/0 /0

/0 /0

/0 /0

! !

/0 /0

/0 /0

/0 /0

! !

%69+-EJ)03*::0)

!"#$%&'()"*+

/0 /0

/0 /0

/0 /0

! !

! ";*'K6.5(1*.+(9'6.-+'-:'+;*'!"#$%&'(#)*$!!)#

! !':3(9()L'1.23%(45*(#.1."2 K90(+-./'40-.+'!MN

! G.*'%69+-49,$!55'4*)'39032'3,39*

! 678 #OOOE@PQ'30149-(.+

! C+)*(1'4)03*::0):'()*'/)064*5'-.+0'&+,"-.

(#)*$!!)#!

! %69+-E4)03*::0):')6.'-.'/9',&%"#:1;(5

! !'.61B*)'0K'169+-E4)03*::0):'K0)1'('/$0-*$

%69+-EJ)03*::0)

<(3.1;(*1!"#$%&

'()"*+

!

/;*($)&0*"#(11"*

/=$*(>&'()"*+

?%"@$%&'()"*+

A"21;$2;&'()"*+

8(B;C*(&'()"*+

/;*($)&0*"#(11"*1&=$-(&$##(11&;"D

'()"*+&8+5( E##(11 /=$*.23

R*/-:+*): R*(5$S)-+* J)-I(+*

M03(9'%*10), R*(5$S)-+* J)-I(+*

C;()*5'%*10), R*(5$S)-+* %69+-EJ)03*::0)

T90B(9'%*10), R*(5$S)-+* &*I-3*

80.:+(.+'%*10), R*(5 &*I-3*

"*U+6)*'%*10), R*(5 &*I-3*

API

Outline




• Tools

• Libraries

© 2008 NVIDIA Corporation.

Execution Model

Software Hardware

Threads are executed by thread processors

Thread

Thread Processor

Thread Block Multiprocessor

Thread blocks are executed on multiprocessors

Thread blocks do not migrate

Several concurrent thread blocks can reside on one multiprocessor - limited by multiprocessor resources (shared memory and register file)

...

Grid Device

A kernel is launched as a grid of thread blocks

Only one kernel can execute on a device at one time

Threading Hierarchy


Thread Batching

Kernel launches a grid of thread blocksThreads within a block cooperate via shared memory

Threads within a block can synchronize

Threads in different blocks cannot cooperate

Allows programs to transparently scale to different GPUs

Grid

Thread Block 0

Shared Memory

Thread Block 1

Shared Memory

Thread Block N-1

Shared Memory

…


Transparent Scalability

Kernel grid

Block 2 Block 3

Block 4 Block 5

Block 6 Block 7

Device Device

Block 0 Block 1 Block 2 Block 3

Block 4 Block 5 Block 6 Block 7

Block 0 Block 1

Block 2 Block 3

Block 4 Block 5

Block 6 Block 7

Block 0 Block 1

Hardware is free to schedule thread blocks on any processor

A kernel scales across parallel multiprocessors

Thread Arithmetic

Indexing Arrays: Example

In this example, the red entry would have an index of 21:

int index = threadIdx.x + blockIdx.x * M;

= 5 + 2 * 8;

= 21;

blockIdx.x = 2

M = 8 threads/block

0 178 16 18 19 20 2121 3 4 5 6 7 109 11 12 13 14 15

Indexing Arrays: Example

In this example, the red entry would have an index of 21:

int index = threadIdx.x + blockIdx.x * M;

= 5 + 2 * 8;

= 21;

blockIdx.x = 2

M = 8 threads/block

0 178 16 18 19 20 2121 3 4 5 6 7 109 11 12 13 14 15

Addition with Threads and Blocks

The blockDim.x is a built-in variable for threads per block:

int index= threadIdx.x + blockIdx.x * blockDim.x;

A combined version of our vector addition kernel to use blocks and threads:

__global__ void add( int *a, int *b, int *c ) {

int index = threadIdx.x + blockIdx.x * blockDim.x;

c[index] = a[index] + b[index];

}

So what changes in main() when we use both blocks and threads?

Control Flow

Control Flow Divergence

What happens if you have the following code?

!"#"$$#%&'()*+*,-,../*$01#.2

3(45(/*$06#.2

3


Branch

Path A

Path B

Branch

Path A

Path B


Nested branches are handled as well

!"#"$$#%&'()*+*,-,../!"#0)'#%&'()*+*,-,..*$12#.3

(45(*$16#.3

7(45(*$18#.3


BranchBranch

Path A

Path C

Branch

Path B


for correctness (*)You might have to think about it for performance

Depends on your branch conditions


Performance drops off with the degree of divergence

!"#$%&'$&()*+,+-.- /0123%*!) 45...

%*!) 65...

7

Divergence

0

5

10

15

20

25

30

35

0 2 4 6 8 10 12 14 16 18

Performance

Divergence

Occupancy


!""#$%&"'

()*+%,-.&/0*#"0.1&/-%*+-+2+"#0+,-/+3#+&0.%44'5-/1-+2+"#0.&6-10)+*-7%*$/-./-0)+-1&4'-7%'-01-).,+-4%0+&".+/-%&,-8++$-0)+-)%*,7%*+-9#/'

!""#$%&"' :-;#<9+*-1=-7%*$/-*#&&.&6-"1&"#**+&04'-1&-%-<#40.$*1"+//1*-,.>.,+,-9'-<%2.<#<-&#<9+*-1=-7%*$/-0)%0-"%&-*#&-"1&"#**+&04'

?.<.0+,-9'-*+/1#*"+-#/%6+@A+6./0+*/B)%*+,-<+<1*'


!"#$%&'()*'+*,-'.)/*,&0,$&

1'#2'3"#$%&'4'1'#2'5/"0,(*#$)&&#*&6#'7""'5/"0,(*#$)&&#*&'879)'70'")7&0'#:)'3"#$%'0#');)$/0)

1'#2'3"#$%&'<'1'#2'5/"0,(*#$)&&#*&'4'=>/"0,(")'3"#$%&'$7:'*/:'$#:$/**):0"?',:'7'5/"0,(*#$)&&#*

!!"#$%&'()*+",-.%))('08)'87*-@7*)'3/&?6/3A)$0'0#'*)&#/*$)'797,"73,",0?' *)B,&0)*&C'&87*)-'5)5#*?

1'#2'3"#$%&'4'DEE'0#'&$7")'0#'2/0/*)'-)9,$)&!"#$%&');)$/0)-',:'(,()",:)'27&8,#:DEEE'3"#$%&'()*'B*,-'@,""'&$7")'7$*#&&'5/"0,(")'B):)*70,#:&

Outline




• Tools

• Libraries


Kernel Memory Access

Per-thread

Per-block

Per-device

ThreadRegisters

Local Memory

SharedMemory

Block

...Kernel 0

...Kernel 1

GlobalMemory

Time

On-chip

Off-chip, uncached

• On-chip, small

• Fast

• Off-chip, large

• Uncached

• Persistent across kernel launches

• Kernel I/O


!"#$$%&$'()*+,-.(/$$01(234-5(63*7,-5(8-,9:+5,;<(

=3*<+,.4;(>(?@;;4:A(B3C,;43(/$$0

/DE/D/$$0

'

!"#$%&"'%

! !"#$%&'()*%#&+,&#%$-./%#.&0%#&./#%")&

0#+1%..+#&23456&-'&.+)%&7"#89"#%:

! ;%#+<1=+1>&1?1=%&"11%..

! @/+#%&%-/7%#&A5*-/&-'/%$%#&+#&A5*-/&,=+"/

()*+,-."/)'0

! B&.)"==&0+#/-+'&+,&$=+*"=&)%)+#?&/7"/&-.&

0#-C"/%&/+&"&./#%")&0#+1%..+#

! D,/%'&(.%8&".&+C%#,=+9&,#+)&#%$-./%#.

! @=+9&/+&"11%..&2.")%&".&$=+*"=&)%)+#?:

12+'"3-."/)'0

! B&*=+1>&+,&)%)+#?&/7"/&-.&.7"#%8&*?&"==&

./#%")&0#+1%..+#.&-'&"&)(=/-<0#+1%..+#

! 3EFG&0%#&*=+1>H&./+#%8&-'&3EI3FG&*"'>.

! J%#?&,"./&/+&"11%..&2-K%K&".&,"./&".&#%$-./%#.L:&&9-/7+(/&!"#$%&'#()*&+,

4,)5+,-."/)'0

! M7%&="#$%&*=+1>&+,&)%)+#?&.7"#%8&*?&"==&

)(=/-<0#+1%..+#.&+'&/7%&1+)0(/%&8%C-1%

! @-N%&8%0%'8.&+'&8%C-1%&! 5OEPG&/+&3KOQG

! R-$7&*"'89-8/7&S&344QGT.

! @=+9&/+&"11%..&! .%C%#"=&7('8#%8&1=+1>&1?1=%&

="/%'1?K&

! 678-1"17%8

9):%&+:&-."/)'0

! B&*=+1>&+,&#%"8<+'=?&)%)+#?&.7"#%8&*?&"==&

)(=/-<0#+1%..+#.&2E6FG:

! U"17%8&C-"&VFG&1"17%&0%#&)(=/-<0#+1%..+#

! @=+9&/+&"11%..&! .%C%#"=&7('8#%8&1=+1>&1?1=%&="/%'1?&+'&1"17%&)-..

8";&<'"-."/)'0! B&="#$%&*=+1>&+,&#%"8<+'=?&)%)+#?&.7"#%8&*?&"==&)(=/-<0#+1%..+#.

! W%"8.&,#+)&/%I/(#%&)%)+#?&1"'&*%&"#$%&'()*#+,!X%"#%./&+#&=-'%"#&-'/%#0+="/-+'&,+#&,#%%L

! U"17%8&C-"&VFG&1"17%&0%#&)(=/-<0#+1%..+#! @=+9&/+&"11%..&! .%C%#"=&7('8#%8&1=+1>&1?1=%&="/%'1?&+'&1"17%&)-..

!"#$$%&$'()*+,-.(/$$01(234-5(63*7,-5(8-,9:+5,;<(

=3*<+,.4;(>(?@;;4:A(B3C,;43(/$$0

/DE/D/$$0

#

! !"#$!%&'()*'+),-./'+0'1(2*'('3014*+-./'

4)0563+7''890:*'"0'+;*'%*+(9

! <-):+')*9*(:*'=>?$>@A'B01B*5

! C*30.5')*9*(:*'-.'D?$>@')*EF)-++*.

! G4*.'C06)3*

! H0+',*+'('I-(B9*':096+-0.

&*I-3*

%69+-EJ)03*::0) %69+-EJ)03*::0)

!

!"#$%&'()"*+ !"#$%&'()"*+

,(-.#(&'()"*+

/0 /0

/0 /0

/0 /0

! !

/0 /0

/0 /0

/0 /0

! !

%69+-EJ)03*::0)

!"#$%&'()"*+

/0 /0

/0 /0

/0 /0

! !

! ";*'K6.5(1*.+(9'6.-+'-:'+;*'!"#$%&'(#)*$!!)#

! !':3(9()L'1.23%(45*(#.1."2 K90(+-./'40-.+'!MN

! G.*'%69+-49,$!55'4*)'39032'3,39*

! 678 #OOOE@PQ'30149-(.+

! C+)*(1'4)03*::0):'()*'/)064*5'-.+0'&+,"-.

(#)*$!!)#!

! %69+-E4)03*::0):')6.'-.'/9',&%"#:1;(5

! !'.61B*)'0K'169+-E4)03*::0):'K0)1'('/$0-*$

%69+-EJ)03*::0)

<(3.1;(*1!"#$%&

'()"*+

!

/;*($)&0*"#(11"*

/=$*(>&'()"*+

?%"@$%&'()"*+

A"21;$2;&'()"*+

8(B;C*(&'()"*+

/;*($)&0*"#(11"*1&=$-(&$##(11&;"D

'()"*+&8+5( E##(11 /=$*.23

R*/-:+*): R*(5$S)-+* J)-I(+*

M03(9'%*10), R*(5$S)-+* J)-I(+*

C;()*5'%*10), R*(5$S)-+* %69+-EJ)03*::0)

T90B(9'%*10), R*(5$S)-+* &*I-3*

80.:+(.+'%*10), R*(5 &*I-3*

"*U+6)*'%*10), R*(5 &*I-3*



Per-thread

Per-block

Per-device

ThreadRegisters

Local Memory

SharedMemory

Block

...Kernel 0

...Kernel 1

GlobalMemory

Time

On-chip

Off-chip, uncached

• On-chip, small

• Fast

• Off-chip, large

• Uncached


• Kernel I/O

Global Memory

Per-device

...Kernel 0

...Kernel 1

GlobalMemory

Time

• Off-chip, large

• Uncached


• Kernel I/O

• Different types of “global memory”

• Linear Memory

• Texture Memory

• Constant Memory

!"#$$%&$'()*+,-.(/$$01(234-5(63*7,-5(8-,9:+5,;<(

=3*<+,.4;(>(?@;;4:A(B3C,;43(/$$0

/DE/D/$$0

0

! !"#$%#&'#%(")'*+%,*"-'*+%."/0#'/#%'/1%2$3#45$%

6$6"57%'**%)"6$%85"6%#&$%0'6$%9&70:)'*%

6$6"57%9""*

! ;40#%1:88$5%:/%'))$00%9'##$5/0+%)')&:/<+%$#)=

! >%.?@>%1$A:)$%:0%'%&:<&*7%9'5'**$*%95")$00"5

! B$%'0046$%:#%)'/%$3$)4#$%6'/7%&4/15$10%"8%

#&5$'10%:/%9'5'**$*

! 2&5$'10%C%D#5$'6%E5")$00"50%F%G

! B&$/%H5:#:/<%.?@>%0"8#H'5$+%#&:/I%:/%#$560%"8%#&5$'10+%/"#%95")$00"50

! >%!"#$"% :0%$3$)4#$1%'0%'%!"#$

! >%&#'( :0%'%)"**$)#:"/%"8%%&"'($)*+,-./

! >%)*#"+(,-%./!,:0%'%)"**$)#:"/%"8%%&"'($/

! 2&5$'1%-*")I0%'/1%#&5$'10%'5$%<:A$/%4/:J4$%

:1$/#:8:$50%

! K1$/#:8:$50%-$%G@+%L@%"5%M@

! ?0$1%#"%&$*9%:1$/#:87%H&:)&%9'5#%"8%'%95"-*$6%

'%#&5$'1N-*")I%0&"4*1%"9$5'#$%"/

@$A:)$

,5:1

0)

12324

0

12354

0)

15324

0

15354

0)

16324

0)

16354

!

! 2&5$'1%O*")I%PG+GQ

7)

12324

7)

12354!

7)

15324

7)

15354

7)

16324

7)

16354

!

!

!"#$$%&$'()*+,-.(/$$01(234-5(63*7,-5(8-,9:+5,;<(

=3*<+,.4;(>(?@;;4:A(B3C,;43(/$$0

/DE/D/$$0

#

! !"#$!%&'()*'+),-./'+0'1(2*'('3014*+-./'

4)0563+7''890:*'"0'+;*'%*+(9

! <-):+')*9*(:*'=>?$>@A'B01B*5

! C*30.5')*9*(:*'-.'D?$>@')*EF)-++*.

! G4*.'C06)3*

! H0+',*+'('I-(B9*':096+-0.

&*I-3*

%69+-EJ)03*::0) %69+-EJ)03*::0)

!

!"#$%&'()"*+ !"#$%&'()"*+

,(-.#(&'()"*+

/0 /0

/0 /0

/0 /0

! !

/0 /0

/0 /0

/0 /0

! !

%69+-EJ)03*::0)

!"#$%&'()"*+

/0 /0

/0 /0

/0 /0

! !

! ";*'K6.5(1*.+(9'6.-+'-:'+;*'!"#$%&'(#)*$!!)#

! !':3(9()L'1.23%(45*(#.1."2 K90(+-./'40-.+'!MN

! G.*'%69+-49,$!55'4*)'39032'3,39*

! 678 #OOOE@PQ'30149-(.+

! C+)*(1'4)03*::0):'()*'/)064*5'-.+0'&+,"-.

(#)*$!!)#!

! %69+-E4)03*::0):')6.'-.'/9',&%"#:1;(5

! !'.61B*)'0K'169+-E4)03*::0):'K0)1'('/$0-*$

%69+-EJ)03*::0)

<(3.1;(*1!"#$%&

'()"*+

!

/;*($)&0*"#(11"*

/=$*(>&'()"*+

?%"@$%&'()"*+

A"21;$2;&'()"*+

8(B;C*(&'()"*+

/;*($)&0*"#(11"*1&=$-(&$##(11&;"D

'()"*+&8+5( E##(11 /=$*.23

R*/-:+*): R*(5$S)-+* J)-I(+*

M03(9'%*10), R*(5$S)-+* J)-I(+*

C;()*5'%*10), R*(5$S)-+* %69+-EJ)03*::0)

T90B(9'%*10), R*(5$S)-+* &*I-3*

80.:+(.+'%*10), R*(5 &*I-3*

"*U+6)*'%*10), R*(5 &*I-3*

!"#$$%&$'()*+,-.(/$$01(234-5(63*7,-5(8-,9:+5,;<(

=3*<+,.4;(>(?@;;4:A(B3C,;43(/$$0

/DE/D/$$0

'

!"#$%&"'%

! !"#$%&'()*%#&+,&#%$-./%#.&0%#&./#%")&

0#+1%..+#&23456&-'&.+)%&7"#89"#%:

! ;%#+<1=+1>&1?1=%&"11%..

! @/+#%&%-/7%#&A5*-/&-'/%$%#&+#&A5*-/&,=+"/

()*+,-."/)'0

! B&.)"==&0+#/-+'&+,&$=+*"=&)%)+#?&/7"/&-.&

0#-C"/%&/+&"&./#%")&0#+1%..+#

! D,/%'&(.%8&".&+C%#,=+9&,#+)&#%$-./%#.

! @=+9&/+&"11%..&2.")%&".&$=+*"=&)%)+#?:

12+'"3-."/)'0

! B&*=+1>&+,&)%)+#?&/7"/&-.&.7"#%8&*?&"==&

./#%")&0#+1%..+#.&-'&"&)(=/-<0#+1%..+#

! 3EFG&0%#&*=+1>H&./+#%8&-'&3EI3FG&*"'>.

! J%#?&,"./&/+&"11%..&2-K%K&".&,"./&".&#%$-./%#.L:&&9-/7+(/&!"#$%&'#()*&+,

4,)5+,-."/)'0

! M7%&="#$%&*=+1>&+,&)%)+#?&.7"#%8&*?&"==&

)(=/-<0#+1%..+#.&+'&/7%&1+)0(/%&8%C-1%

! @-N%&8%0%'8.&+'&8%C-1%&! 5OEPG&/+&3KOQG

! R-$7&*"'89-8/7&S&344QGT.

! @=+9&/+&"11%..&! .%C%#"=&7('8#%8&1=+1>&1?1=%&

="/%'1?K&

! 678-1"17%8

9):%&+:&-."/)'0

! B&*=+1>&+,&#%"8<+'=?&)%)+#?&.7"#%8&*?&"==&

)(=/-<0#+1%..+#.&2E6FG:

! U"17%8&C-"&VFG&1"17%&0%#&)(=/-<0#+1%..+#

! @=+9&/+&"11%..&! .%C%#"=&7('8#%8&1=+1>&1?1=%&="/%'1?&+'&1"17%&)-..

8";&<'"-."/)'0! B&="#$%&*=+1>&+,&#%"8<+'=?&)%)+#?&.7"#%8&*?&"==&)(=/-<0#+1%..+#.

! W%"8.&,#+)&/%I/(#%&)%)+#?&1"'&*%&"#$%&'()*#+,!X%"#%./&+#&=-'%"#&-'/%#0+="/-+'&,+#&,#%%L

! U"17%8&C-"&VFG&1"17%&0%#&)(=/-<0#+1%..+#! @=+9&/+&"11%..&! .%C%#"=&7('8#%8&1=+1>&1?1=%&="/%'1?&+'&1"17%&)-..

!"#$$%&$'()*+,-.(/$$01(234-5(63*7,-5(8-,9:+5,;<(

=3*<+,.4;(>(?@;;4:A(B3C,;43(/$$0

/DE/D/$$0

#

! !"#$!%&'()*'+),-./'+0'1(2*'('3014*+-./'

4)0563+7''890:*'"0'+;*'%*+(9

! <-):+')*9*(:*'=>?$>@A'B01B*5

! C*30.5')*9*(:*'-.'D?$>@')*EF)-++*.

! G4*.'C06)3*

! H0+',*+'('I-(B9*':096+-0.

&*I-3*

%69+-EJ)03*::0) %69+-EJ)03*::0)

!

!"#$%&'()"*+ !"#$%&'()"*+

,(-.#(&'()"*+

/0 /0

/0 /0

/0 /0

! !

/0 /0

/0 /0

/0 /0

! !

%69+-EJ)03*::0)

!"#$%&'()"*+

/0 /0

/0 /0

/0 /0

! !

! ";*'K6.5(1*.+(9'6.-+'-:'+;*'!"#$%&'(#)*$!!)#

! !':3(9()L'1.23%(45*(#.1."2 K90(+-./'40-.+'!MN

! G.*'%69+-49,$!55'4*)'39032'3,39*

! 678 #OOOE@PQ'30149-(.+

! C+)*(1'4)03*::0):'()*'/)064*5'-.+0'&+,"-.

(#)*$!!)#!

! %69+-E4)03*::0):')6.'-.'/9',&%"#:1;(5

! !'.61B*)'0K'169+-E4)03*::0):'K0)1'('/$0-*$

%69+-EJ)03*::0)

<(3.1;(*1!"#$%&

'()"*+

!

/;*($)&0*"#(11"*

/=$*(>&'()"*+

?%"@$%&'()"*+

A"21;$2;&'()"*+

8(B;C*(&'()"*+

/;*($)&0*"#(11"*1&=$-(&$##(11&;"D

'()"*+&8+5( E##(11 /=$*.23

R*/-:+*): R*(5$S)-+* J)-I(+*

M03(9'%*10), R*(5$S)-+* J)-I(+*

C;()*5'%*10), R*(5$S)-+* %69+-EJ)03*::0)

T90B(9'%*10), R*(5$S)-+* &*I-3*

80.:+(.+'%*10), R*(5 &*I-3*

"*U+6)*'%*10), R*(5 &*I-3*

Constant Memory

Constants set by CPU, read by GPU

Each SM has 8kiB cache for constants

Optimized for broadcast

Accessing different elements forces serialisation

Can speed some calculations

Can relieve register pressure

Declared at file scope__constant__ float dc_myConst;

Set via cudaMemcpyToSymbol API callcudaMemcpyToSymbol( “dc_myConst”, 3.14f, sizeof(float) )

Accessed by name in kernel__global__ MyKernel( ... ) { .... float myVal = dc_myConst+1; ....}

Constant Memory

Textures

Textures are essentially look up tables

Can only be written by the host

Cached on each multiprocessor (8kiB)

Optimised for 2D spatial locality

Hardware interpolation possible

Limited precision

Can clamp or wrap at boundaries

Textures

Declaration and setup rather involved

See programming guide

Accessed in kernels via texture fetches:tex1D, tex2D, tex3D, etc.

Co-ordinates at texel centres

Have to take care when accessing elements

Textures

Can improve load coalescing from global memory

If whole texture fits in 8kiB cache, has grid lifetime

Clamping/wrapping can aid edge case handling

Have to test to determine benefits

General Principles

Memory access patterns are crucial

Even CPUs are typically memory bound

GPUs have 100x FP

Only 10x memory bandwidth

Have to keep the GPU busy

!"#$$%&$'()*+,-.(/$$01(234-5(63*7,-5(8-,9:+5,;<(

=3*<+,.4;(>(?@;;4:A(B3C,;43(/$$0

/DE/D/$$0

F

!"#$%&'()*+,-

! !"#$%&'(")*+,-".'/"$'0&"12"#+"345"#$%&'(6

! !**"#$%&'(6"78"'"#$%&'(")*+,-"'%&"%18"+8"#$&"

6'.&".1*#792%+,&66+%

! :$16",'8",+..187,'#&"07'"6$'%&(".&.+%/

! !8("6/8,$%+87;&"

! :$%&'(6"+<"'")*+,-"'%&".1*#72*&=&("+8#+"'"

.1*#792%+,&66+%"'6"!"#$%

>?@

A+%#$)%7(B&

CD!E

F+1#$)%7(B&

F!:! G#$&%8&#

H%'2$7,6">'%("I"

>@C!

J%+8#"F7(&"K16

E&.+%/"K16 ?>L"K16

?>L9G=2

%&66"K16

!

! ./012 +%"./0$! D&2*',&("!H?

! ?5?M"J1**"C12*&="F&%7'*M"F/..&#%7,"K16! 53NEKI6")'8(O7(#$"78"&',$"(7%&,#7+8

! "#$$#%&'()#$*+%,-(+%.#($/&.+0&,(1&2%,3(,+8<7B1%'#7+86P""GPBQ! ?>L9G"4R="S"4R"*'8&6

! 4R"#7.&6"#$&")'8(O7(#$"TUHKI6V

! :$&">@C!"62&,7<7,'#7+8"$'6")&&8"12('#&(! W&%67+8"4PN"4 L87#7'*"%&*&'6&M"N4INX

! W&%67+8"4P4"4@2('#&"O7#$"8&O&%"$'%(O'%&M"NUINX

! K',-O'%(6",+.2'#7)*&

! G=2&,#&("12('#&6"78"8&'%"<1#1%&Q! W&%67+8"4P5"I"5PN

! RY9)7#"<*+'#78B"2+78#"6122+%#"T7P&P"(+1)*&V

! W&%67+8"4P4"'((&("6+.&"7.2+%#'8#"16&<1*"

<&'#1%&6Q

3*456%#$

! !6/8,$%+8+16".&.+%/",+27&6

! !6/8,$%+8+16"H?@"2%+B%'."*'18,$

7%#&6%#$

! !#+.7,".&.+%/"786#%1,#7+86

3+ Gb/s

8 GB/s

25+ GB/s

160+ GB/sto

VRAM

PC Architecture

modified from Matthew Bolitho

PCIe Transfers(first thing to optimize?)

!"#$%&'()##*+&%,

!"#$%&'()*+,

& -.()*+

&. /.'()*+

-./

0./

0./123+*"

-./123+*"

-+#"(4&5&(6*+3(-./(123+*"(5+0./(123+*"

789(-/4):2*92';<=

-+#"(4&5&(6*+30./(323+*"(5+-./(123+*"

-,$#<25

*Averaged observed bandwidth


Processing Flow

1. Copy input data from CPU memory to GPU memory

PCI Bus


Processing Flow


2. Load GPU program and execute,caching data on chip for performance

PCI Bus


Processing Flow


2. Load GPU program and execute,caching data on chip for performance

3. Copy results from GPU memory to CPU memory

PCI Bus

PCIe Transfers

PCIe 2.0 x16 bus has

Latency of 10 µs (observed)

Bandwidth of 8GB/s (theory), 5 GB/s (observed)

A lot of calculations can happen in these times

PCIe Transfers

PCIe transfers occur via DMA

GPU reads pages direct from CPU memory

Very bad if page gets moved mid-transfer

CUDA maintains internal pinned memory buffers

Used for cudaMemcpy calls

Data staged through these

!"#$%&'#'()*+(#$,-'#)

!"#$%#&%'()*+'(',-$."/0$1'#&2'!"#$%&'#'()32&$24'4#-$.521'#&26

7-$"/82'+9:6'+1;$.5&0$0-1*&/<2&'+9:6'!"#$"%!&'()*!" 0&'!"#$"%!&'()*+,-%!!"

!;$.5&0$0-1',-$."/0$1'=40.>'0$'#$;'?&/0&'#1;$.5&0$0-1'>2&$24'4#-$.521

!"#$%&'!"#$%&'!()')*&+,&-!"#$%&'!()

!"#$%&'()**"+),#"-.

/,)#'(01%(')*&+,&- #1(21*3-"#"4(

$&#)-(213.()'(21*3-"#"5

!"#$%&'(!&)&*+,$-./010$&2#)1&('"#'(6"7,8)1%5(9%,+"100(6"#:""&(;<=(2.2-"'(,&+(%"'31&'"('3""+!"#$%&'(!&)!2&#",&)3(4!"#$%&'(!&)!2&#",&5(&,#!"#$%&'(!&6,7!8(4-)94!

()*+'),-./,0#1,'23*+#&'45,6745'"5,6)'#5*74,8&#91

!"#$%&'($()"*!+,"

!""#$%&'()#'*%(+,-'./#0+.#+"/'#1%#+/).02('.'3/4#+.5#(%,3(.#-&&%5-+,%")

$%&'()#&3/,#1%#+""'0+,%5#+/#

!"#$%&'()*++'!,-!"./&'()*++'!,-6"5%(#7%(/-'.#'8#,2%/%#83.0,-'./#!"#$%*++'!&'(),-0!"./#/++'!&'(),-+"/'#9'(:4#

+,--./ &%&'()#+""'9/#5-(%0,#;$!#,(+./8%(/#1)#,2%#<=>#,'#

"'0:%5#,'#+#*2)/-0+"#+55(%//

!"#$%&'($()"*!+,"*-.($/01

!"#$%&'(')%*+%,&'-*%'./%/%0'/#'$+'12%'345

6+7',-/+82'"9%*2%-0'$&'"9%*,-##%0

:7+82*"+"/&'8-,,&'&2"/,0';%'0"+%'"/1&$0%'8*$1$8-,'&%81$"+&' &"<%'"='12%&%'-*%'%>#%+&$9%?

@+$1$-,$A-1$"+

B%<"*7'-,,"8-1$"+&

:1*%-<'C'D9%+1'8*%-1$"+

@+1%*"# *%&"/*8%''234"/'5/4($

!"#$%&'!"#$%&'!()*+),!!"#$%&"'"&()*"

#$%&"'"(+,*(%-./)",$01)*.102",0&34.2,567%1&"8%1&*,0&39)+.32/)"()+.32:"""""""

!"#$%&'!()*+),!')-&.,&/!"#$%&'!()*+),!

!"#$01.&$#2),!1.3,45&!:;

!"#$%&'($&)*'+$(),--$*'+'

.&+'$&/$()+'01((&&/2$-&+$/&3$0((,1'()+'01$4$5

6'),+/($711'%70)'89

6'),+/($711'%70)'89

6'),+/($711'%70)'89

:07)($-&+$';'+9)*7/<$&/$)*'$="#$)&$-7/7(*2$)*'/$+'),+/(

!"#$%&'()$'*#'+&#,'%-'.-$/%-0'(-123#%/-$!"#$%&'()*)+,+-.'&'()+

!"#$%#&'() !"#$%&'()*+'$)'/0+,+!"%&'()*+'$)'/0

!"#$%&'()*)+1)23#2('4!"#$%&'()-5'$)'/61)23#2('7804!"#$,'-!./01/(!/#'9)792"5!'7:;)'97!"#$<'=!>;129)?23'&@!'7804!"#$%&'()2'!3+#/1)23#2('04-/4'+('5!"#$,'-!./01/(!/#'9)792"5!'7:;)'97!"#$<'=!>;3'&@!'?2129)7804

!"#$%&'()6/(!7+3(89'/1)23#2('04

!"#$:7+'$#6/(!7+3(89'/04

*$+%,'-.,%'/0"'#1#")%2+34'(#/0"#!"%&'()A'!25#/1)23#2('0%0'50678#%#9'%2#3'"#%."3,

*$+%,'/0"'#1#")%2+34'03'%2#':;<'%0'/+3+,29'%2#3'"#%."3,

!;<'5$3'&0',%.//'2#"#

=2#'/+",%'6#60")'507)'+,'&03#9',0'%2#'6#60")'$%',0."5#'50.8&'(#'.,#&'$4$+3'()'%2#'!;<

!"#$"%&'()*+%#,$-)./01232"245)62"7)8#$539%#!""#$%&'()(*+,,'-(./0(1233'456(*+,78*#,7(1'&9'',(&:';

.#,$

244',&

!"#$%&'(&)$*$+,-.!/$0$1234%&'567,<<<((!""#$%&'(1233'4589!:;$<=/.";>?

!"#$%&'!()*+),!<1@34%&'A1234%&'5<%&'(&)BC>D67AE!F;AG&/HI;)G1JK.E#L.M;-!G;A+>,

')-&.,&/0')-&.,&/1

2.$3%)4.$'&<1234%&'5%&'(&)7>,

!"#$56.&$#7),!6.8,9:&<>,%&'(&)BB,$%&'(&)$D*6,

N

!"#$%&'()*+,!"#$#%&'()*&+,-.&/0123-4&

/5256,7,-8&9:&;<;&.5=4&5

>4>,?5-4>&$@%&4AB,A4

$@%&-C5A*D4C*&0=4C&(/#4&?5A&64

?0A?3CC4A-&+,-.&/)$%&E4CA47&4F4?3-,0AG

&'()*+, 5770+*&,A>424A>4A-&?0A?3CC4A-&,AH0C>4C&I3434*&0D&4F4?3-,0A

!"#$%&'($)*&+,-./&'($)!"#$%&'($)-'($&(01+,!"%&'($)-'($&(01

@37-,274&*-C451*&4F,*-&+,-.,A&5&*,AB74&?0A-4F-J&-.48&

*.5C4&1410C8&5A>&0-.4C&C4*03C?4*

-)+.(/ !.0'(.11)(

'()&@410C8

#-$%20340)

!.+56')20340)

78#%!.54),%.01/9%%!"#$%&'!()*+,-).! :*00.'%.;)(1*5<

GPU

!"#$%&'(%#%&$"$#

!""#!$%&' ()&'*+,&#-./+0*+0$#1.-0#.)"#$%& 2./.30*0/

4)&*+30#50/&0"#6.)&'1!!!"#$%&'%()*"+,-./'%()*'010 '%()*"'2$)34555

7/+-0/#!89.67368.9#$%&:;<8.=>68.2%-8*"?%&29*"9)%@92*"

;2$)34A

:,2+0$#;#50/&0"#".)&'10$#<+*1#*10#)%&$ $*/0.3#2./.30*0/#0=0')*0#*+,-#.$#

!"#$"%&'(

!"#$"%&')

!"#$"%&'*

!"#$"%&+(

!"#$"%&'(

!"#$"%&')

!"#$"%&+(

!"#$"%&'*

,-./&'(

,-./&+(

,-./&+)

,-./&')

,-./&+*

,-./&+0

,-./&'(

,-./&')

,-./&+(

,-./&+0

,-./&+)

,-./&+*

,12'&3456789

:'3!&' :'3!&+

,;<=">?@>6

,;8<A46">?@>6

Independent Tasks

Scheduling on GPU

!"#$%&'()$*+$,*-$#./

!"#$"%&'(%(%)&*+%&,$--%.&$"&/0%&1+.%+&21.%&$)&%3%2(/%.

01234&0!5/

671378&!9

671378&!:

671378&!;

671378&<9

=2>5&!9

=2>5&!:

=2>5&<9

=2>5&<?

=2>5&<:

=2>5&<;

'@17!A&!

'@17!A&<

671378&!9

=2>5&!:

=#BC&7.D$.(

=#EBF-(7.D$.(

=2>5&!9

671378&!:

671378&!;

=2>5&<9

=2>5&<:

671378&<9

=2>5&<;

=2>5&<?

!"#$%&'()*$'+#*$#,-./-0'12

(+34'12

,-./-0'15

,-./-0'16

(+34'72

(+34'75

,-./-0'72(+34'15

(+34'76

(+34'78

(+..-(9':14;

()<='->?@>$

()&<A"$->?@>$

,-./-0'12

,-./-0'15

,-./-0'16

,-./-0'72

(+34'12

(+34'15

(+34'72

(+34'78

(+34'75

(+34'76

!9.-1B'1

!9.-1B'7

!"#$#%&'(%)*+,'-+./#0%.01+%'2!(-3'45!6'7%+*8.*9,'%:#)#";0%6'&;0;'0+;"6$%+';:0*<%0/%+=

!"#$%&'(")*%&+,,-./0%+121+% 3')"45",*>/%'%;6,'?;,'0*';./#%@%'.*9,A.*)910%'*@%+:;9=61 7.+89'%!"#$%&+,,-./B

C1"0#)%D'!"#$%&'(&)*!&+,$-.23'?#0/'!"#$(&)*!&/$012.' $:;<

E+#@%+D'!"3'435&$'&23'?#0/'3673897/:;71<%8

:1 ;99"<+$'%,-..'=%5>?%('(")*C1"0#)%D'!"#$12.':,,2!=>F'16%'!"#$12.':,,2!/$00&# $:;<

E+#@%+D'!"/&?12.':,,2!=> 16%'36(:7/@/1<%8:AA<37(@BC3@/:;

@1 A'$%+%5?B;%='C-<'%,"-.$')%$"%$D-#%('(")*C1"0#)%D'!"#$12.'D&'(&)*!&;2*E'&5=>E+#@%+D'!"/&?12.'D&'(&)*!&;2*E'&5=>

E1 FG#$%G#'%$D+$%,"-.$')%-.%*"G)%2').'9#H

BG/%.H'0/%'!"#$"%&'()$*+',- A'./01234.20566748/620.590$5:0&;<60$2$;7=&%@#.%'9+*9%+0,'$:;<'0*'6%%'#$'I%+*8G*9,'#6';@;#:;J:%K

!"#$%&$'()*+,-".,/"0

!"#"$%&$#'"(&)*''*+$,-*'$#.*$/01*$23&$"3#,4"#%5"6678$

9&*$:.*($+"#"$%&$,(67$'*"+;:'%##*($,(5*9&*$),'$-*'7$&4"66$"4,3(#&$,)$+"#"$<(*:$-"'%"26*&8$0/9;=/9$5,443(%5"#%,(>9&*$:.*($5,4?3#*;4*4,'7$'"#%,$%&$-*'7$.%@.$"(+$,553?"(57$%&$.%@.8$&,$6"#*(57$,-*'$/01*$%&$.%++*(0,"6*&5%(@$%&$!"#$#!%&&'(%4?,'#"(#A

PCIe Transfers Optimization

PCIe bus is slow

Try to minimize transfers

Use pinned memory on host whenever possible

Try to perform copies asynchronously

Outline




• Tools

• Libraries


CUDA-GDB

Extended version of GDB with support for C for CUDA

Supported on Linux 32bit/64bit systems

Seamlessly debug both the host|CPU and device|GPU codeSet breakpoints on any source line or symbol nameSingle step executes only one warp except on sync threads Access and print all CUDA memory allocations, local, global, constant and shared vars.

Walkthrough example with sourcecode : CUDA-GDB manual


Linux GDB Integration with EMACS


Linux GDB Integration with DDD


CUDA-MemCheck

Detects/tracks memory errorsOut of bounds accessesMisaligned accesses (types must be aligned on their size)

Integrated into CUDA-GDBLinux and WinXPWin7 and Vista support coming

11©NVIDIA 2010


CUDA Driver Low-level Profiling support1. Set up environment variables

!"#$%&'()*+,-./012345

!"#$%&'()*+,-./0123,(6745

!"#$%&'()*+,-./0123,(/80194:$;<=>?&"&'

!"#$%&'()*+,-./0123,2/94#%$<=@!?:AB

2. Set up configuration fileFILE "config.txt":

>#CA&D%&&=E!A&DE#

=;A&%C:&=$;A

3. Run applicationED&%="FC@

4. View profiler output

FILE "profile.csv":G'()*+,-./0123,2/9,73.61/8'5?H

G'()*+,*371(3'I'9!0$%:!'JJII'9K

G'()*+,-./0123,(67'5

G'K1F36K+F-0+(K/.'<DLMLNN5!DL:5L:

>#CA&D%&&=E!A&DE#OE!&P$QO>#C&=E!O:#C&=E!O$::C#D;:RO=;A&%C:&=$;A

55H<S!DD5I!TNLLIOE!E:#RU&$*OV?TLJO5L?III

55H<S!DD5I!HQD:IOE!E:#RU&$*OH?WWSOS?III

55H<S!DD5I!MH:!IOE!E:#RU&$*OV?TLJOW?III

55H<S!DD5I<L!DWIO,X5IQED&%="EC@-<==6,==6,O5M?LMWOSI?IIIOI?TTTOST

HL

55H<S!DD5I<SSTDIOE!E:#R*&$UOV?VVWOTW?III


CUDA Visual Profiler - OverviewPerformance analysis tool to fine tune CUDA applications

Supported on Linux/Windows/Mac platforms

Functionality:

Execute a CUDA application and collect profiling data

Multiple application runs to collect data for all hardware performance counters

Profiling data for all kernels and memory transfers

Analyze profiling data


CUDA Visual Profiler data for kernels


CUDA Visual Profiler computed data for kernels

Instruction throughput: Ratio of achieved instruction rate to peak single issue instruction rate

Global memory read throughput (Gigabytes/second)

Global memory write throughput (Gigabytes/second)

Overall global memory access throughput (Gigabytes/second)

Global memory load efficiency

Global memory store efficiency


CUDA Visual Profiler data for memory transfers

Memory transfer type and direction(D=Device, H=Host, A=cuArray)

e.g. H to D: Host to Device

Synchronous / Asynchronous

Memory transfer size, in bytes

Stream ID


CUDA Visual Profiler data analysis viewsViews:

Summary table Kernel tableMemcopy table Summary plotGPU Time Height plotGPU Time Width plotProfiler counter plotProfiler table column plotMulti-device plotMulti-stream plot

Analyze profiler counters

Analyze kernel occupancy


CUDA Visual Profiler Misc.Multiple sessions

Compare views for different sessions

Comparison Summary plot

Profiler projects save & load

Import/Export profiler data (.CSV format)

Outline




• Tools

• Libraries

CUBLAS


CUBLASCUDA accelerated BLAS (Basic Linear Algebra Subprograms)

Create matrix and vector objects in GPU memory spaceFill objects with dataCall sequence of CUBLAS functionsRetrieve data from GPU (optionally)

!"#$%&'#((')'*+,-#.%/'00'1%$.+2%!'3'4.56-.5$'78

9:;$+4<=%*>?$5+.'+$6"+'@''1%$.+2%!'A'9:;$+4<15.&BC1-1CDC1-ECD7F'''''''''9:;$+4<+,6E&BC'+$6"+C1-1CDC1-,CD7F'

AA'%>%/E'GH'#.%/+.#524C'/%4.+/.'/%4#1:+$#?'&#'I'GH'@@'H7'8

9:;$+4<=%*>9:;$+4<956E&BC'1-;C'DC'1-/C'D7F9:;$+4<+,6E&BC'JDKHC'1-EC'DC'1-/C'D7F

L%$4%'

9:;$+4<+,6E&BCJ+$6"+C1-ECDC1-/CD7F'KKK'


CUBLAS FeaturesSingle precision data:

Level 1 (vector-vector O(N) )Level 2 (matrix-vector O(N2) )Level 3 (matrix-matrix O(N3) )

Complex single precision data:Level 1CGEMM

Double precision data:Level 1: DASUM, DAXPY, DCOPY, DDOT, DNRM2, DROT, DROTM, DSCAL, DSWAP, ISAMAX, IDAMIN Level 2: DGEMV, DGER, DSYR, DTRSVLevel 3: ZGEMM, DGEMM, DTRSM, DTRMM, DSYMM, DSYRK, DSYR2K


CUBLAS Performance: CPU vs GPU

CUBLAS: CUDA 2.3, Tesla C1060 MKL 10.0.3: Intel Core2 Extreme, 3.00GHz

!"#$%&'()*+,*-./0)

!"

#"

$"

%"

&"

'!"

'#"

'!#$ #!$& (!)# $!*% +'#! %'$$ )'%&

&1))

231'456'78$

7.9*:;'2:-)/5:,/5'<=;=>

,-.

/(0'

/(0#

Up to 2x average speedup over CUBLAS 3.1

Less variation in performancefor different dimensions vs. 3.1

!"##$%&'(%)%&'*%+,%-./0/1%$2345%!(676%89":;<%*6'('&'6(=%+,%>?5@A!+B2%/,C24%!+B2%DE%F-2G542HI

Average speedup of {S/D/C/Z}GEMM x {NN,NT,TN,TT}

MATLAB Interface

! 15+ functions! Up to 10x speedup

! Dense linear algebra! C/C++ & FORTRAN! 150+ Routines

GPU Accelerated Linear Algebra

Supercomputer Speeds

Performance 7x of

Partnership

Developed in partnership with NVIDIA

CULA (LAPACK for heterogeneous systems)

CULA - PerformanceSupercomputing Speeds

This graph shows the relative speed of many CULA functions when compared to

(Fermi) and an Intel Core i7 860. More at www.culatools.com

CUSPARSE

Sparse Matrix Performance: CPU vs. GPU

0x

5x

10x

15x

20x

25x

30x

35xMultiplication of a sparse matrix by multiple vectors

"Non-transposed""Transposed"MKL 10.2

Average speedup across S,D,C,Z

!"#$%&#'()*+(,-(./010%(23456(!+787(9$":;<(=7*+*)*7+>(,-(?@6AB!,C3(0-D35(!,C3(EF(G.3H653IJ


CUFFTCUFFT is the CUDA FFT libraryComputes parallel FFT on an NVIDIA GPU

Plan contains information about optimal configuration for a given transform.Plans can be persisted to prevent recalculation.Good fit for CUFFT because different kinds of FFTs require different thread/block/grid configurations.


CUFFT Features

1D, 2D and 3D transforms of complex and real-valued dataBatched execution for doing multiple 1D transforms in parallel1D transform size up to 8M elements2D and 3D transform sizes in the range [2,16384]In-place and out-of-place transforms for real and complex data.


CUFFT Example

!"#$%&#'()'*+,!"#$%&#'(-'.*/

01$$234&"5# 654&701$$289:65#; <%"424='<9"424701"4>45590??@9%"<<AB%"424='C%D#9$?01$$289:65#;A<()<(-A701"4>45590??@9%"<<AB9"424='C%D#9$?01$$289:65#;A<()<(-A7

E<'8F#42#'4'*G'HHI'654&J'<E01$$2K54&*"?B654&='()=(-='8LHHIM8*8A7

E<'LC#'2N#'8LHHI'654&'29'2F4&C$9F:'2N#'C%O&45'912'9$'6540#J'<E01$$2P;#08*8?654&='%"424='9"424='8LHHIMHQRSTRGA7

E<'U&@#FC#'2F4&C$9F:'2N#'C%O&45'%&'6540#J'<E01$$2P;#08*8?654&='9"424='9"424='8LHHIMU(VPRWPA7

E<'G#C2F9X'2N#'8LHHI'654&J'<E01$$2G#C2F9X?654&A7

01"4HF##?%"424A701"4HF##?9"424A7

Complex 2D transform


CUFFT Performance: CPU vs GPU

!"##$%&'()%*+,-,.%$/012%34565%789:;<%45'4=4)%>"2?@3A=/%,BC/1%3A=/%DE%F*/G21/HI%('&7JK

CUFFT 3.2: Improved Radix-3, -5, -7

!

"!

#!!

#"!

$!!

$"!

# $ % & " ' ( ) * #! ## #$ #% #& #"

!"#$%&

'()*+,-./0

123-45*6+&%768996(::06

+$!(!,-%.$

+$!(!,-%.#

/01

!

#!

$!

%!

&!

"!

'!

(!

# $ % & " ' ( ) * #! ## #$ #% #& #"

!"#$%&

'()*+,-./0

123-45*6+;%768996(::60

+$!(!,-%.$

+$!(!,-%.#

/01

9<""=6*>?6@6*>A6(B6CDE;EF6=/,'269?GHG6!%<IJ#6AG>?>*>G?K6(B6LM2359(N/6EBO/'69(N/6-H6+C/P2'/Q0

Radix-5, -7 and mixed radix improvements not shown

!"#$$

!"#$%&'&(")*"+$,+-./&*)&0'12/".'&'##/#".&$0$3$4/5"*)&"!678

9:";'&&$5"<=>?7?8@A"B:"CD/15"<6!7@A"E:"E/1,F.3'"<6!7@A"8:"7'4$G5)1"<6!7@A"E:"HI/1,"<6!7@A"J:"K+'1,"<6!7@

8#,)&$3+05

2FG..E2'1A"2FG..E/,0/13/GE2'1A"2FG..L/GF2/

2FG..E)&3A"2FG..L'1GA"2FG..E.'&5/9'3&$M>/23)&9F#3$.#(

8GG$3$)1'#"'#,)&$3+05"$1".&),&/55

N&'.+5A"0)&/"5)&3$1,A"3&//5A"+'5+$1,A"'F3)3F1$1,

!"#$$%&'()*+,

!"#$$!%&'()*+,-(%& .%&'() /010!"#$$23!456000

!"#$$24##60!"#$$27894:60!"#$$29$:;95279<=4<#0>?

!"#$$@,&ABC DB,&?0

!"#$$<CE*B- +CE*B-0/0.*ADD$B,&FGDB,&60

.%&'()60&*HIBCHC&-E60

J60KL?0

.*ADD3.,&FDB,&60A2%A,-,60A2(A,-,60&*HIBCHC&-EL?

Thrust

© 2008 NVIDIA Corporation

!   Programmer productivity !   Rapidly develop complex applications !   Leverage parallel primitives

!   Encourage generic programming !   Don’t reinvent the wheel !   E.g. one reduction to rule them all

!   High performance !   With minimal programmer effort

!   Interoperability !   Integrates with CUDA C/C++ code

Objectives

3

!"#$%&

!""#$%&'()$%#(*+,),-#./,#!0123*&*45#6$)78),8#9%&'()$%#:*+,),-#;69:<#

!/7$)*7%,5!"#$%!&&"'%!()*+!'#,-.!"#$%!&&/*)0+*()*+!'#,-.

2(=/,*$>&5!"#$%!&&%'#!12!"#$%!&&#*/$+*12!"#$%!&&03+4$%0)*(%+5312?$4@ 63

!!"#$%$&'($")*+"&'%,-."%/.0$&1"-%"(2$"2-1((2&/1(332-1(45$6(-&78%(9"245$6:)"77";<=>"(2&/1(33#$%$&'($:245$6?0$#8%:=@"245$6?$%,:=@"&'%,=>

!!"(&'%1A$&",'('"(-"(2$",$586$(2&/1(33,$586$45$6(-&78%(9",45$6 B"245$6>

!!"1-&(",'('"-%"(2$",$586$(2&/1(331-&(:,45$6?0$#8%:=@",45$6?$%,:==>

!!"(&'%1A$&",'('"0'6C"(-"2-1((2&/1(336-DE:,45$6?0$#8%:=@",45$6?$%,:=@"245$6?0$#8%:==>

!"#$%&'()*+,-.

!"#$%&'()'*((+,-'.(/-

!"#$%&'()*(&+"#,-

./)012-3

45$"0-6()(#56

7)#2#68&9#3(&:(;*"3(<"3-*3=

!"#$%$&'($)*+&',-

!"#$%&'($)*+#$&#$,-../+0%'#+#$&#&

12$%+34056$7$58'&&'($+9'6$%&$+:;2<6=$+(>?

;6#'($+64880%'#*>../+8$8@$5&+4%+60&2A0&$5&

PyCUDA


PyCUDA

3rd party open source, written by Andreas Klöckner Exposes all of CUDA via Python bindingsCompiles CUDA on the fly

presents CUDA as an interpreted languageIntegration with numpyHandles memory management, resource allocationCUDA programs are Python strings

Metaprogramming modify source code on-the-flyLike a really complex pre-processor

http://mathema.tician.de/software/pycuda


PyCUDA Example! "#$%&' $()*+,-+&"./&0,1 )*+,20 "#$%&' $()*+,-,*'%"3"'4 "#$%&' 3*#$(56 ,0703*#$(-&,3+%#-&,3+38595:-0,1'($/83*#$(-%,'42:; ,<=$*070)*+,-#/#<,>>%)8,-1"?/90,-+'($/-"'/#1"?/:@ )*+,-#/#)$(<A'%+8,<=$*90,:BC #%+070)*+,-D%*&)/E%+*>/8FFF!G <<=>%H,><<0.%"+0+%*H>"I(8I>%,'0J,:!! K!2 "3'0"+L070'A&/,+M+L-L0N0'A&/,+M+L-(J5O!4 ,P0"+L0Q0J702-GIO!5 R!6 FFF:!; I*3)070#%+-=/'<I*3)'"%38F+%*H>"I(F:!@ I*3)8,<=$*90H>%)S785959!::!B!C ,<+%*H>/+0703*#$(-/#$'(<>"S/8,:2G )*+,-#/#)$(<+'%A8,<+%*H>/+90,<=$*:2! $&"3' ,<+%*H>/+22 $&"3' ,

CURAND

RNG Performance: CPU vs. GPU

0x

5x

10x

15x

20x

25x

SP DP SP DP

Uniform Normal

Generating 100K Sobol' Samples

CURAND 3.2MKL 10.2

!"#$%&'()*'+,'%-.&.$'/0123'!*454'67"89:';4)*)()4*<'+,'=>3?@!+A0'.,B02'!+A0'CD'E%0F320GH

OpenVIDIA

Open source, supported by NVIDIA

Computer Vision Workbench (CVWB)

http://openvidia.sourceforge.net

GPU imaging & computer vision

Demonstrates most commonly used image processing primitives on CUDA

Demos, code & tutorials/information

OpenVIDIA

and many more...

References

• CUDA C Programming Guide

• CUDA C Best Practices Guide

• CUDA Reference Manual

• API Reference, PTX ISA 2.2

• CUDA-GDB User Manual

• Visual Profiler Manual

• User Guides: CUBLAS, CUFFT, CUSPARSE, CURAND

http://developer.nvidia.com/object/gpucomputing.html

http://developer.download.nvidia.com/compute/cuda/3_2_prod/toolkit/docs/CUDA_C_Programming_Guide.pdf

http://developer.download.nvidia.com/compute/cuda/3_2_prod/toolkit/docs/CUDA_C_Programming_Guide.pdf

http://developer.download.nvidia.com/compute/cuda/3_2_prod/toolkit/docs/CUDA_C_Best_Practices_Guide.pdf

http://developer.download.nvidia.com/compute/cuda/3_2_prod/toolkit/docs/CUDA_C_Best_Practices_Guide.pdf

http://developer.download.nvidia.com/compute/cuda/3_2_prod/toolkit/docs/CUDA_Toolkit_Reference_Manual.pdf

http://developer.download.nvidia.com/compute/cuda/3_2_prod/toolkit/docs/CUDA_Toolkit_Reference_Manual.pdf

http://developer.download.nvidia.com/compute/cuda/3_2_prod/toolkit/docs/online/index.html

http://developer.download.nvidia.com/compute/cuda/3_2_prod/toolkit/docs/online/index.html

http://developer.download.nvidia.com/compute/cuda/3_2_prod/toolkit/docs/ptx_isa_2.2.pdf

http://developer.download.nvidia.com/compute/cuda/3_2_prod/toolkit/docs/ptx_isa_2.2.pdf

http://developer.download.nvidia.com/compute/cuda/3_2_prod/toolkit/docs/cuda-gdb.pdf

http://developer.download.nvidia.com/compute/cuda/3_2_prod/toolkit/docs/cuda-gdb.pdf

http://developer.download.nvidia.com/compute/cuda/3_2_prod/toolkit/docs/VisualProfiler/Compute_Visual_Profiler_User_Guide.pdf

http://developer.download.nvidia.com/compute/cuda/3_2_prod/toolkit/docs/VisualProfiler/Compute_Visual_Profiler_User_Guide.pdf

http://developer.download.nvidia.com/compute/cuda/3_2_prod/toolkit/docs/CUBLAS_Library.pdf

http://developer.download.nvidia.com/compute/cuda/3_2_prod/toolkit/docs/CUBLAS_Library.pdf

http://developer.download.nvidia.com/compute/cuda/3_2_prod/toolkit/docs/CUFFT_Library.pdf

http://developer.download.nvidia.com/compute/cuda/3_2_prod/toolkit/docs/CUFFT_Library.pdf

http://developer.download.nvidia.com/compute/cuda/3_2_prod/toolkit/docs/CUSPARSE_Library.pdf

http://developer.download.nvidia.com/compute/cuda/3_2_prod/toolkit/docs/CUSPARSE_Library.pdf

http://developer.download.nvidia.com/compute/cuda/3_2_prod/toolkit/docs/CURAND_Library.pdf

http://developer.download.nvidia.com/compute/cuda/3_2_prod/toolkit/docs/CURAND_Library.pdf



iPhD one more thingor two...

Life/Code Hacking #2.xSpeed {listen,read,writ}ing

accelerated e-learning (c) / massively parallel {learn,programm}ing (c)

Life/Code Hacking #2.1Speed listening




• Step 1: Collect

• online videos, tutorials, podcasts, etc.

• audiobooks

• youtube-dl, get_flash_videos, jDownloader, ffmpeg, mplayer, etc.

• etc.



• Step 2: Accelerate (time-stretch)

• VLC (Playback > Faster)

• sox $f{,.1.8X.mp3} tempo 1.8 50

• iPod ? mp3splt -t 5.00 -o small-@n large.mp3



• Step 3: chill or do more ;-)

[Harvard CS264] 04 - Intermediate-level CUDA Programming

Education

Transcript of [Harvard CS264] 04 - Intermediate-level CUDA Programming