Using Language Oriented Programming to Execute Computations on the GPU
-
Upload
skills-matter -
Category
Technology
-
view
1.984 -
download
1
description
Transcript of Using Language Oriented Programming to Execute Computations on the GPU
Using Language Oriented
Programming to Execute
Computations on the GPUComputations on the GPU
Robert Pickering, ALTI
About the Presenter• Using F# for about 6 years
• Oldest F# user outside of Microsoft
• Written a book about F#
(now in its second edition)
• Spend at least 2 years as a professional
functional programmer
2
Contact me:
http://strangelights.com/blog
functional programmer
• I have 3 cats
ALTI
• Large French services company
• Consulting, engineering and
training
• Believe in helping clients • Believe in helping clients
discover upcoming niches
• Already offer F# training and
consulting
What is Language Oriented
Programming ?
• Creating programs that are an abstract
description of the a problem
• This description can then either be • This description can then either be
interpreted, compiled, or analyzed in another
way
LOP & Combinators
• F# has two main approaches to Language
Oriented Programming:
– Reinterpreting the F# syntax though the
quotations systemquotations system
– Using combinators create a new syntax
… both approaches are very similar
What is a Combinator?
A combinator is a higher-order function that
uses only function application and earlier
defined combinators to define a result from its defined combinators to define a result from its
arguments.
Source: Wikipedia, http://en.wikipedia.org/wiki/Combinatory_Logic
Combinatory Logic in Computing
In computer science, combinatory logic is used
as a simplified model of computation, used in
computability theory and proof theory. Despite
its simplicity, combinatory logic captures many its simplicity, combinatory logic captures many
essential features of computation.
Source: Wikipedia, http://en.wikipedia.org/wiki/Combinatory_Logic
Combinator Library
"A combinator library offers functions (the
combinators) that combine functions together to make
bigger functions"[1]. These kinds of libraries are
particularly useful for allowing domain-specific
programming languages to be easily embedded into a programming languages to be easily embedded into a
general purpose language by defining a few primitive
functions for the given domain.
Souce: Wikipedia http://en.wikipedia.org/wiki/Combinator_library
[1] “A History of Haskell” Hudak, Hughes, Peyton Jones, Wadler
History of Haskell: Combinator Libraries
What is a combinator library? The reader will
search in vain for a definition of this heavily
used term, but the key idea is this: a combinator
library offers functions (the combinators) that library offers functions (the combinators) that
combine functions together to make bigger
functions.
History of Haskell: Combinator Libraries
What is a combinator library? The reader will
search in vain for a definition of this heavily
used term, but the key idea is this: a combinator
library offers functions (the combinators) that library offers functions (the combinators) that
combine functions together to make bigger
functions.
History of Haskell: Combinator Libraries
Another productive way to think of a
combinator library is as a domain-specific
language (DSL) for describing values of a
particular type.particular type.
What is a Domain Specific Language?
A programming language tailored for a particular application
domain, which captures precisely the semantics of the
application domain -- no more, no less.
A DSL allows one to develop software for a particular A DSL allows one to develop software for a particular
application domain quickly, and effectively, yielding
programs that are easy to understand, reason about, and
maintain.
Hudak
Combinators vs DSLs
• Combinartor libraries are a special case of DSLs
– Sometimes called DSELs (Domain Specific Embed languages)
• DSELs have several advantages:
― Inherit non-domain-specific parts of the design.― Inherit non-domain-specific parts of the design.
― Inherit compilers and tools.
― Uniform “look and feel” across many DSLs
― DSLs integrated with full programming language, and with each other.
• DSELs one disadvantage:
― Constrained by host language syntax and type system
What Makes F# a Suitable for DSLs ?
• Union Types / Active Patterns
– type Option<'a> = Some x | None
• Lambda functions
– fun x -> x + 1– fun x -> x + 1
• Define and redefine operators
– let (++) x = x + 1
• Define custom numeric literals
– let x : Expression = 1.0N
Union Types – The Option Type
// The pre-defined option type
type Option<'a> =
| Some of 'a
| None
// constructing options// constructing options
let someValue = Some 1
let noValue = None
// pattern matching over optionslet convert value =
match value with| Some x -> Printf.sprintf "Value: %i" x| None -> "No value"
Union Types - Trees
// a binary tree definition
type BinaryTree<'a> =
| Node of BinaryTree<'a> * BinaryTree<'a>
| Leaf of 'a
// walk the tree collection valueslet rec collectValues acc tree =
match tree with| Node(ltree, rtree) ->
// recursively walk the left treelet acc = collectValues acc ltree// recursively walk the right treecollectValues acc rtree
| Leaf value -> value :: acc// add value to accumulator
Using the Tree
// define a tree
let tree =
Node(
Node(Leaf 1, Leaf 2), Node(Leaf 1, Leaf 2),
Node(Leaf 3, Leaf 4))
// recover all values from the leaves
let values = collectValues [] tree
Union Types
Union types play a key role in language oriented
programming, as they allow the user to easily
define a tree that will form the abstract syntax
tree of the languagetree of the language
Active Patterns
// definition of the active patternlet (|Bool|Int|Float|String|) input =
// attempt to parse a boollet sucess, res = Boolean.TryParse inputif sucess then Bool(res)else
// attempt to parse an int// attempt to parse an intlet sucess, res = Int32.TryParse inputif sucess then Int(res)else
// attempt to parse a float (Double)let sucess, res = Double.TryParse inputif sucess then Float(res)else String(input)
Active Patterns
// function to print the results by pattern// matching over the active patternlet printInputWithType input =
match input with| Bool b -> printfn "Boolean: %b" b| Int i -> printfn "Integer: %i" i| Int i -> printfn "Integer: %i" i| Float f -> printfn "Floating point: %f" f| String s -> printfn "String: %s" s
// print the results printInputWithType "true"printInputWithType "12"
Active Patterns
Active patterns play a supporting role in
language oriented programming in F#. They
allow the programmer to create abstractions of
complex operations on the abstract syntax treecomplex operations on the abstract syntax tree
Lambda Functions
fun () ->
lovin'()lovin'()
Lambda Functions
Lambda functions are important for language
oriented programming. They allow the users of
the language to embed actions within other
language elementslanguage elements
Custom Operators
let (++) x = x + 1
Custom Operators
Custom operators play a supporting role in
language oriented programming. They allow the
language designer to have a more flexible
syntax.syntax.
Custom Numeric Literals
type Expression =| Constant of int
module NumericLiteralN = let FromZero() = Constant 0let FromZero() = Constant 0let FromOne() = Constant 1let FromInt32 = Constant
let expr = 1N
Custom Numeric Literals
Numeric literals play a supporting role in
language oriented programming. They allow
numeric literals be treated in a more natural
within an embedded languagewithin an embedded language
The Anatomy of a DSLtype Expression =
| Add of Expression * Expression| Subtract of Expression * Expression| Multiply of Expression * Expression| Constant of int| Parameter of stringwith
static member (+) (x, y) = Add(x, y)
Syntax Tree
Combinators
static member (+) (x, y) = Add(x, y)static member (-) (x, y) = Subtract(x, y)static member (*) (x, y) = Multiply(x, y)
module NumericLiteralN = let FromZero() = Constant 0let FromOne() = Constant 1let FromInt32 = Constant
let param = Parameter
The Anatomy of a DSL
let expr = (1N + 2N) * (5N - 2N)
val expr : Expression =Multiply (Add (Constant 1,Constant 2),
Subtract (Constant 5,Constant 2))
The Anatomy of a DSL
• Expressions now have an abstract tree like representation:
― Multiply
― Add
― Constant 1
― Constant 2― Constant 2
― Subtract
― Constant 5
― Constant 2
• This can then be evaluated
• Or we can preform more advanced analysis
The Anatomy of a DSL
let evaluateExpression parameters =let rec innerEval tree =
match tree with| Multiply (x, y) -> innerEval x * innerEval y| Add (x, y) -> innerEval x + innerEval y| Subtract (x, y) -> innerEval x - innerEval y| Constant value -> value| Constant value -> value| Parameter key -> Map.find key parameters
innerEval
let expr = (1N + 2N) * (5N - 2N)
evaluateExpression Map.empty expr
The Anatomy of a DSL
The Anatomy of a DSL
let rec simplifyExpression exp =let simpIfPoss op exp1 exp2 =
let exp' = op (simplifyExpression exp1, simplifyExpression exp2)
if exp' = exp then exp' else simplifyExpression exp'match exp with| Multiply(Constant 0, Constant _) -> Constant 0| Multiply(Constant 0, Constant _) -> Constant 0| Multiply(Constant _, Constant 0) -> Constant 0| Multiply(Constant n1, Constant n2) -> Constant (n1 * n2)| Add(Constant n1, Constant n2) -> Constant (n1 + n2)| Subtract(Constant n1, Constant n2) -> Constant (n1 - n2)| Multiply(exp1, exp2) -> simpIfPoss Multiply exp1 exp2| Add(exp1, exp2) -> simpIfPoss Add exp1 exp2| Subtract(exp1, exp2) -> simpIfPoss Add exp1 exp2| Constant _ | Parameter _ -> exp
The Anatomy of a DSL
Quotations
let quotation = <@ (1 + 2) * (5 - 2) @>
val quotation : Quotations.Expr<int> =
Call (None, Int32 op_Multiply[Int32,Int32,Int32](Int32, Int32),
[Call (None, Int32 op_Addition[Int32,Int32,Int32](Int32, Int32),[Call (None, Int32 op_Addition[Int32,Int32,Int32](Int32, Int32),
[Value (1), Value (2)]),
Call (None, Int32 op_Subtraction[Int32,Int32,Int32](Int32, Int32),
[Value (5), Value (2)])])
Quotations
• Allow you to grab a tree structure that
represents a piece of F# code
• This tree structure can then be:• This tree structure can then be:
– Interpreted
– Compiled
– Converted to instructions understood by another
system or runtime
Microsoft Accelerator
• A project from MRS, now available on
Microsoft Connect [1]
• Automatically parallelize code for execution • Automatically parallelize code for execution
on the GPU or x64 multicore
• Implemented as an unmanaged library with
.NET wrapper
[1] http://connect.microsoft.com/acceleratorv2
Microsoft Accelerator
• Based on “Parallel Array”
• You define a set of operations to process the
content of the arraycontent of the array
• The Accelerator runtime will then process the
operations in parallel
Microsoft Accelerator
User Program – Define Operations
AcceleratorAccelerator
Accelerated Program
DX9Target X64Target
Input data Input data
Microsoft Accelerator – Code!
let nums = [| 6; 1; 5; 5; 3 |]let input = new FloatParallelArray(nums); let sum = ParallelArrays.Shift(input, 1) + input +
ParallelArrays.Shift(input, -1); let output = sum / 3.0f;
let target = new DX9Target(); let res = target.ToArray1D(output);
Game of Life
Game of Life
• Green - When an existing cell (green in the middle) has three or two neighbours it survives to the next round
• Red - When an existing cell has less than two neighbours it dies, because it is lonely (first red case), when it has more than three neighbours it dies of overcrowding (second red case)
• Blue - Finally, a new cell is born in an empty grid location if there are exactly three neighbours .
The implementation
1
1 1
1
etc. ...Sum of all
neighbours
initial rotation 1
Game of Life
GPUCPU
Game of Life
/// Evaluate next generation of the life game state
let nextGeneration (grid: Matrix<float32>) =
// Shift in each direction, to count the neighbourslet sum = shiftAndSum grid offsets
// Check to see if we're born or remain alive
GPUCPU
// Check to see if we're born or remain alive(sum =. threeAlive) ||. ((sum =. twoAlive) &&. grid)
Game of Life
/// Evaluate next generation of the life game state[<ReflectedDefinition>]let nextGeneration (grid: Matrix<float32>) =
// Shift in each direction, to count the neighbourslet sum = shiftAndSum grid offsets
// Check to see if we're born or remain alive
GPUCPU
// Check to see if we're born or remain alive(sum =. threeAlive) ||. ((sum =. twoAlive) &&. grid)
DEMO
Game of life in F# with Microsoft Accelerator
Thanks!
• A big thanks to Tomáš Petříček!
• More info on his blog:• More info on his blog:– http://tomasp.net/articles/accelerator-intro.aspx
Books
Shameless Plug!
• Robert Pickering’s Beginning F# Workshop:
– Thursday 9th Sept – 10th September 2010
http://skillsmatter.com/course/open-source-http://skillsmatter.com/course/open-source-
dot-net/robert-pickerings-beginning-f-workshop