U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science Fortress John Burgess and...

UUNIVERSITY OF NIVERSITY OF MMASSACHUSETTSASSACHUSETTS, A, AMHERST • MHERST • Department of Computer Science Department of Computer Science

Fortress

John Burgess and Richard Chang

CS691WUniversity of Massachusetts

Amherst

UUNIVERSITY OF NIVERSITY OF MMASSACHUSETTSASSACHUSETTS, A, AMHERST • MHERST • Department of Computer Science Department of Computer Science 2

Overview Sun’s DARPA HPCS project

High productivity a goal “Fortress” derived from intent to produce

“secure Fortran” Not backwards compatible with Fortran

Design decisions motivated by desire to make scientists/programmer’s job as easy as possible Mathematical syntax Implicit/explicit parallelism, handled by libraries Decisions still being made, no compiler currently

available


Design Goals High Performance

Operations for parallel execution and distribution of large data structures

Architecture-specific library implementations High Productivity

Type safety Type inference used to reconstruct types

Extensive Libraries Transactional memory Syntax emulates standard mathematical

notation Management of APIs/Components


Overview Fortress Syntax Objects and Traits Variables APIs and Components Parallelism in Fortress


“Advances” in Syntax

Syntax resembling mathematical notation Dot product (Ascii version):dotProd[\T extends Field\]

(left_vec:T[n],right_vec:T[n],foo_bar: T)=

Dot product (Unicode version):


Objects and Traits

Traits Like Java interfaces with code,

classes without fields Objects

Consist of fields and methods Fortress also has top-level

functions


Traits

Declare methods Abstract methods have only headers Concrete methods have definitions

May extends other Traits Allow new types to be defined


Objects

Consist of fields and methods Each object has a set of Traits

Multiple inheritance of code through traits

Inherits concrete methods of its Traits May override inherited methods Must include definition of all abstract

methods of its traits


List Traittrait List{\T\] extends Object

first:()->T

rest:()->List[\T\]

cons:T->List[\T\]

append:List[\T\])->List[\T\]

…(* Definitions of Object trait methods*)

end


The Object Trait Root of trait hierarchy

Every object in Fortress has trait Object Every object has implementations of

these methods (directly or inherits them from a trait)


Empty List Object object Empty traits {List}first() = throw Errorrest() = throw Errorcons(x) = Cons(x, self)append(xs) = xs

end Implements methods of List trait Defines an empty list object Cons is another object that implements

cons method for lists


Variables in Fortress

May be declared to be immutable or mutable

Variable declarations may not have explicit type declarations

Examples: myList:List[\Integer\] := Empty.cons(13)

(* mutable/explicit type declaration*) myList2 := Empty.cons(10)

(* mutable/inferred type*) maxValue = 99 (* immutable/inferred type *)


Components and APIs

Components Similar to modules in ML, scheme Export and import APIs

APIs Define types in traits, objects, and

functions implemented by components


Hello World component Hello

import print from io

export executable

run(args) = print “Hello world”

end

api io

print:String -> ()

end

api executable

run:(args:String…) -> ()

end


Parallelism in Fortress

Spawn expressions Transactional Memory Implicit Parallelism

Generators Distributions

Reductions Recursive Decomposition

Matrix unpasting syntax


Spawn Expressions Used to explicitly run sections of code in new thread

executing in parallel Syntax:

v = spawn do

expressions

end Synchronization methods

v.val() returns value computed by expressions Will block until v has completed

v.wait() Blocks until v completes but doesn’t return value

v.ready() Returns true if v has completed, returns false otherwise


Transactional Memory

Synchronization in Fortress done using atomic blocks

Code inside block appears to run atomically

Multiple threads accessing shared data will have access serialized

IO cannot be performed inside atomic block


Atomic arraySum In Fortress for loops are implicitly

parallel atomic modifier serializes updates

to sum


Implicit Parallelism Implicit threads created by

Fortress runtime for: Tuple expressions, for loops,

summations, etc Implicit barrier at the end of

implicitly parallel construct


Generators Used to express parallel iteration

For loops parallel by default:

Body of iterations run in separate implicit threads

Example generators: 0:9 a.indices() sequential(a.indices())


Regions Fortress programs execute within set of

regions Abstractly describe structure of a machine

Organized as a tree Every Fortress object resides in a region Library writers have control over how

objects/threads are distributed to regions Programmers perspective: shared global

address space Can specify how objects/threads are

distributed using distributions


Tree Abstraction


Distributions Attempt to separate task of data

distribution from ensuring program correctness

Architecture/platform specific library will provide default distributions for organizing data produced by generators and allocating and distributing arrays sequential(g) applies local distribution

to generator g


Distributions Cont. Built in distributions:

Default - machine/platform specific Sequential - Arrays allocated to one piece of

(local) memory Par - Blocked into chucks of size 1 Blocked - Blocked into roughly equal sized

chunks Plus others…

Library writers can provide other distributions Programmers can specify which distribution

array and generators should use: a = blocked.array(xSize, ySize)


Reductions Fortress provides reductions based on

those of OpenMP Reductions defined to have identity

value arraySum example:

Variable sum is reduced Sum assigned to identify of + at beginning of each

iteration At end of iterations, original value of sum is combined

with final values of sum from each iteration using the reduction (+) in arbitrary order


Recursive Subdivision

Recursively divide problems into smaller problems On-the-fly load balancing Computation independent of

machine size Generators provided by library do

this internally Can be used explicitly by

programmers


Recursive Sum Recursively subdivide generator

until it is small enough that runtime returns sequential generator


Matrix Unpasting Shorthand for breaking up matrix

into parts Useful for recursive subdivision Assume M is a matrix:

[topM

bottomM ] = M [leftM rightM ] = M

Concise, eliminates corner case errors


Matrix Multiplication


Conclusion Fortress philosophy:

Use abstractions to hide details of parallel execution as much as possible

Extensive libraries to handle data and thread distribution

Productivity improvements Mathematical syntax Component/APIs to organize large projects Type Checking

U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science Fortress John Burgess and...

Documents

Transcript of U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science Fortress John Burgess and...