A Cluster of Languages for Mathematical Computing 1 Stephen M. Watt Department of Computer Science...
-
Upload
oswald-elliott -
Category
Documents
-
view
213 -
download
1
Transcript of A Cluster of Languages for Mathematical Computing 1 Stephen M. Watt Department of Computer Science...
A Cluster of Languages for A Cluster of Languages for Mathematical ComputingMathematical Computing
1
Stephen M. Watt
Department of Computer ScienceWestern University London Ontario, Canada
DIKUUniversity of Copenhagen7 September 2012
Moving Windows AroundMoving Windows AroundAdd a borderAdd a scroll barRespond to a button
….
Derp, derp, …
We have harderproblems now.
2
Declaration of PrejudicesDeclaration of PrejudicesKey problem:
How to cascade efficient, effective abstractions.
…
3
Mathematics as a Mathematics as a Programming Language Programming Language CanaryCanary
4
Why?Why?Complex problems with many
partsComplex interactions among the
partsMany different levels of
abstractionPrecise definitionCan tell if an answer is right or
wrong
5
ExamplesExamplesGarbage collection
◦Lisp underground Java etcAlgebraic expressions
◦FortranBig integer
◦CryptoGenerics
◦ Java, C++, ***
6
Computer AlgebraComputer AlgebraSolve problems in terms of symbolic
parameters, rather than numerically.
Having the computer figure out the formulasrather than using formulas given by humans.
Algorithms – computational mathematics
Software – mathematical computation
7
Computer AlgebraComputer Algebra
Start with symbols and compute with symbols =>
Exact resultsHopefully, insightful results
Finding an AnswerFinding an AnswerOne day an individual went to the horse races. Instead of counting the number of humans and horses, she counted 74 heads and 196 legs. How many humans and horses were there?
humans + horses = 74humans × 2 + horses × 4 = 196
Finding an AnswerFinding an AnswerOne day an individual went to the horse races. Instead of counting the number of humans and horses, she counted 74 heads and 196 legs. How many humans and horses were there?
humans + horses = 74humans × 2 + horses × 4 = 196
horses = 24humans = 50
Finding an AnswerFinding an AnswerOne day an individual went to the horse races. Instead of counting the number of humans and horses, she counted H heads and L legs. How many humans and horses were there?
humans + horses = Hhumans × 2 + horses × 4 = L
horses = ‒H + L/2 humans = 2 H ‒ L/2
Computer AlgebraComputer AlgebraA couple of research problems of
personal interest◦Symbolic-numeric algorithms◦Symbolic exponents
12
Approximate PolynomialsApproximate Polynomials
Symbolic ExponentsSymbolic Exponents
ExamplesExamplesMapleAxiomAldorMathMLInkML
Warning: 3x too much stuff here.We will skip to what the audience wants.
15
Language 1: MapleLanguage 1: MapleWaterloo: 1980 onGeddes & Gonnet initiators.University, then company.
Collaboration.
Dynamically typed, interpreted language for scripting computer algebra programs.
16
An Example (small)An Example (small)
25
26
MapleMaple• Compiled kernel, interpreted library• What was compiled was hand-chosen• Support many students on shared 1980s hw
• Easy to lay down code, quick library growth• Language not very structured, so limitations
• Commercially viable project• Company focus education and CAE
27
Example 2: AxiomExample 2: Axiom• 1984 moved from Waterloo to IBM
Research
• Scratchpad in-house research project• Jenks and Trager initiators.• 1991 released as commercial product
by NAG
28
AxiomAxiom• Main idea: code re-use through
abstraction• Generic algorithms based on structures
of modern algebra (groups, rings, algebras, fields).
• The language is the thing• Compiled programming language for
writing libraries “in the large”• Syntactically similar, dynamically typed
interpreted language for scripting.
29
Type Inference in Type Inference in InterpreterInterpreter
30
More Complicated TypesMore Complicated Types
31
32
33
34
35
AxiomAxiom• Great concept for building well-structured
and flexible libraries.
• Not enough “dogfooding.”• Top-level tried to hide types from user,
but was not sufficiently successful at doing that.
• Powerful and flexible, but too complex for most users.
• Now open source.
36
Example 3: AldorExample 3: AldorRe-design of Axiom language 1984
on.Initiator Watt.
The language is the thing, writ largeEfficiency, elegance, take no prisonersNothing special about built-in types
Dependent types everywhereInteroperability with C and Lisp
37
Aldor and Its Type SystemAldor and Its Type SystemTypes and functions are values
◦ May be created dynamically◦ Provide representations of mathematical sets and functions
The type system has two levels◦ Each value belongs to a unique type, its domain, known
statically. ◦ This is an abstract data type that gives the representation.◦ The domains are values with domain Domain.
◦ Each value may belong to any number of subtypes of its domain.
◦ Subtypes of Domain are called categories.
Categories◦ specify what exports (operations, constants) a domain
provides.◦ fill the role of OO interfaces or abstract base classes.
Why Two Levels?Why Two Levels?OO inheritance pb with multi-argument
fns:
class SG { “*”: (SG, SG) -> SG; }DoubleFloat extends SG ...Permutation extends SG ...
x, y ∈ DoubleFloat ⊂ SGp, q ∈ Permutation ⊂ SG
x * y ✓p * q ✓p * y ✓ ??? Bad, Bad, Bad
Why Two Levels?Why Two Levels?OO inheritance pb with multi-argument
fns:
SG == ... { “*”: (%, %) -> %; }DoubleFloat: SG ...Permutation: SG ...
x, y ∈ DoubleFloat ∈ SGp, q ∈ Permutation ∈ SG
x * y ✓p * q ✓p * y ✗
Parametric PolymorphismParametric PolymorphismPP is via category- and domain-producing functions.
-- A function returning an integer.
factorial(n: Integer): Integer == { if n = 0 then 1 else n*factorial(n-1) }
-- Functions returning a category and a domain.
Module(R: Ring): Category == Ring with { *: (R, %) -> % }
Complex(R: Ring): Module(R) with {
complex: (%,%)->R; real: %->R; imag: %->R; conj: % -> %; ...
} == add {
Rep == Record(real: R, imag: R); 0: % == … 1: % == … (x: %) + (y: %): % == ...
}
Dependent TypesDependent TypesGive dynamic typing, e.g.
f: (n: Integer, R: Ring, m: IntegerMod(n)) -> SqMatrix(n, R)
Recover OO through dependent products:
prodl: List Record(S: Semigroup, s: S) == [[DoubleFloat, x],[Permutation, p],[DoubleFloat, y]
]
With categories, guarantee required operations available:
f(R: Ring)(a: R, b: R): R == a*b + b*a
Multi-sorted AlgebrasMulti-sorted AlgebrasCategory signature as a dependent product
type.
ArithmeticModel: Category == with {
Nat: IntegralDomain;
Rat: Field;
/: (Nat, Nat) -> Rat;}
Aldor and Its Type SystemAldor and Its Type SystemType producing expressions may be
conditional
UnivariatePolynomial(R: Ring): Module(R) with {
coeff: (%, Integer) -> R;
monomial: (R, Integer) -> %;
if R has Field then EuclideanDomain;
...
} == add {
...
}
Post facto extensions allow domains to belong to new categories after they have been initially defined.
Without Post Facto Extension Without Post Facto Extension forforStructuring LibrariesStructuring LibrariesDirectProduct(n: Integer, S: Set): Set with {
component: (Integer, %) -> S;
new: Tuple S -> %;
if S has Semigroup then Semigroup;
if S has Monoid then Monoid;
if S has Group then Group;
...
if S has Ring then Join(Ring, Module(S));
if S has Field then Join(Ring, VectorField(S));
...
if S has DifferentialRing then DifferentialRing;
if S has Ordered then Ordered;
...
} == add { ... }
Post Facto Extension forPost Facto Extension forStructuring LibrariesStructuring LibrariesDirectProduct(n: Integer, S: Set): Set with {
component: (Integer, %) -> S;
new: Tuple S -> %;
} == add { ... }
extend DirectProduct(n: Integer, S: Semigroup): Semigroup == ...extend DirectProduct(n: Integer, S: Monoid): Monoid == ...extend DirectProduct(n: Integer, S: Group): Group == ......extend DirectProduct(n: Integer, S: Ring): Join(Ring, Module(S)) == ...extend DirectProduct(n: Integer, S: Field): Join(Ring, VectorField(S)) == ......extend DirectProduct(n: Integer, S: Field): Join(Ring, VectorField(S)) == ...extend DirectProduct(n: Integer, S: DifferentialRing): DifferentialRing == ...extend DirectProduct(n: Integer, S: Ordered): Ordered == ......
Normally these extensions would all be in separate files.
Higher Order OperationsHigher Order OperationsE.g. Reorganizing constructionsPolynomial(x) Matrix(n) Complex R ≈ Complex Matrix(n) Polynomial(x) R
Slightly simpler example
List Array String R ≈ String Array List R
Higher Order OperationsHigher Order OperationsAg ==> (S: BasicType) -> LinearAggregate S;
swap(X:Ag, Y:Ag)(S:BasicType)(x:X Y S):Y X S == [[s for s in y] for y in x];
al: Array List Integer := array(list(i+j-1 for i in 1..3) for j in 1..3);
la: List Array Integer := swap(Array, List)(Integer)(al);
Phew!Phew!
Using GenericityUsing GenericityLinearOrdinaryDifferentialOperator(
A: DifferentialRing,
M: LeftModule(A) with differentiate: % -> %
) : MonogenicLinearOperator(A) with {
D: %;
apply: (%, M) -> M;
...
if A has Field then {
leftDivide: (%, %) -> (quotient: %, remainder: %);
rightDivide:(%, %) -> (quotient: %, remainder: %);
… // rgcd, lgcd
}
} == ...
Using GenericityUsing GenericityLinearOrdinaryDifferentialOperator(
A: DifferentialRing,
M: LeftModule(A) with differentiate: % -> %
) : ...
== SUP(A) add {
...
if A has Field then {
Op == OppositeOperator(%, A);
DOdiv == NonCommutativeOperatorDivision(%, A);
OPdiv == NonCommutativeOperatorDivision(Op,A);
leftDivide (a,b) == leftDivide(a, b)$DOdiv;
rightDivide(a,b) == leftDivide(a, b)$OPdiv;
}
...
}
Design Principles IDesign Principles INo compromises on flexibilityNo compromises on efficiencyUse optimization to bridge the gap.
Compilation. Separate compilation.Generated intermediate code is platform
independent, even though word-sizes, etc, vary.Libraries can be distributed, if desired, as binary
only.
Be a good citizen in a multi-language framework.◦ Call and be called by C/C++/Fortran/Lisp/Maple◦ Functional arguments◦ Cooperating memory management
Design Principles IIDesign Principles IILanguage-defined types should have no privilege
whatsoever over application-defined types.◦ Syntax, semantics (e.g. in type exprs),
optimization (e.g. constant folding)
Language semantics should be independent of type.◦ E.g. named constants overloaded, not functions
Combining libraries should be easy, O(n), not O(n2).◦ Should be able to extend existing things with new
concepts without touching old files or recompiling.
Safety through optimization removing run-time checks, not by leaving off the checks in the first place.
The Compiler as an The Compiler as an ArtefactArtefactWritten primarily in C (C++ too immature in 1990)1550 files, 295 K loc C + 65 K loc Aldor Intermediate code (FOAM):
◦ Primitive types: booleans, bytes, chars, numeric, arrays, closures
◦ Primitive operations: data access, control, data operationsRuntime system:
◦ Memory management◦ Big integers◦ Stack unwinding◦ Export lookup from domains◦ Dynamic linking◦ Written in C and Aldor
Example of OptimizationExample of OptimizationFrom the domain Segment(E: OrderedAbelianMonoid)generator(seg:Segment E):Generator E == generate {
(a, b) := (low seg, hi seg);
while a <= b repeat { yield a; a := a + 1 }
}
From the domain List(S: Set)generator(l: List S): Generator S == generate {
while not null? l repeat { yield first l; l := rest l }
}
Client codeclient() == {
ar := array(...); li := list(...);
s := 0;
for i in 1..#ar for e in l repeat { s := s + ar.i + e }
stdout << s
}
How Generators WorkHow Generators Workgenerator(seg:Segment Int):Generator Int== generate { a := lo seg; b := hi seg; while a <= b repeat { yield a; a := a + 1 }}
client() == { ar := array(...); s := 0; for i in 1..#ar repeat s := s + a.i; stdout << s}
Example of Optimization Example of Optimization (again)(again)From the domain Segment(E: OrderedAbelianMonoid)generator(seg:Segment E):Generator E == generate {
(a, b) := (low seg, hi seg);
while a <= b repeat { yield a; a := a + 1 }
}
From the domain List(S: Set)generator(l: List S): Generator S == generate {
while not null? l repeat { yield first l; l := rest l }
}
Client codeclient() == {
ar := array(...); li := list(...);
s := 0; -- NOTE PARALLEL TRAVERSAL.
for i in 1..#ar for e in l repeat { s := s + ar.i + e }
stdout << s
}
InlinedInlinedB0: ar := array(...); l := list(...); segment := 1..#ar; lab1 := B2; l2 := l; lab2 := B9; s := 0; goto B1;B1: goto @lab1;B2: a := segment.lo; b := segment.hi; goto B3;B3: if a > b then goto B6; else goto B4;B4: lab1 := B5; val1 := a; goto B7;B5: a := a + 1 goto B3;B6: lab1 := B7; goto B7;B7: if lab1 == B7 then goto B16; else goto B8B8: i := val1; goto @lab2;B9: goto B10B10: if null? l2 then goto B13; else goto B11B11: lab2 := B12 val2 := first l2; goto B14;B12: l2 := rest l2 goto B10B13: lab2 := B14 goto B14B14: if lab2 == B14 then goto B16; else goto B15B15: e := val2; s := s + ar.i + e goto B1;B16: stdout << s
Clone Blocks for 1Clone Blocks for 1stst Iterator Iterator
DataflowDataflow
[lab1 == B2, lab1 == B5, lab1 == B7]
Resolution of 1Resolution of 1stst Iterator Iterator
Clone Blocks for 2Clone Blocks for 2ndnd Iterator Iterator
Resolution of 2Resolution of 2ndnd Iterator Iterator
client() == { ar := array(...); l := list(...); l2 := l; s := 0; a := 1; b := #ar; if a > b then goto L2L1: if null? l2 then goto L2 e := first l2; s := s + ar.a + e a := a + 1 if a > b then goto L2 l2 := rest l2 goto L1L2: stdout << s}
Aldor vs C Aldor vs C (non-floating pt)(non-floating pt)
Aldor vs C Aldor vs C (floating point)(floating point)
Follow-on Research Follow-on Research ProjectsProjectsGeneric library inter-operabilityLocalized garbage collectionDynamic abstract data typesPerformance analysis of genericsEtc, etc
66
Lessons LearnedLessons Learned It is possible to be elegant, abstract and high-level
without sacrificing significant efficiency. Well-known optimization techniques can be
effectively adapted to the symbolic setting.Optimization of generated C code is not enough.
Procedural integration, dataflow analysis, subexpression elimination and constant folding are the primary wins.
Compile-time memory optimization, including data structure elimination, is important.◦ Removes boxing/unboxing, closure creation, dynamic
allocation of local objects, etc. Can move hot fields into registers.
Aldor LessonsAldor LessonsLanguage design 20+ years old.
◦ In the mean time, many of the ideas now mainstream.
◦ Many still are not.
Mathematics is a valuable canary in the coal mine of general purpose software.◦ The general world lags in recognizing needs.
It has to be free.◦ Free1 is the standard price. ◦ Free2 is required for engagement.
Example 4: MathMLExample 4: MathMLFirst XML application, ever.Language for exchange of
mathematical data.Initially
70
Example 4: MathMLExample 4: MathML
MathMLMathMLOpenMath effort initiated 1993 for data
exchange.Unfulfilled <math> element in HTML 3.2 Jan
1997.
Initial, unchartered Math WG defining microsyntax for <math>.
Internecine rivalry between syntax and semantics camps coming from TeX, Mathematica and SGML.
MathMLMathMLConvened “HTML-native” math group to
form unified proposal.
First ever XML application.XML proposed recommendation December
1997. MathML proposed recommendation February
1998.
Supported in major browsers, computer algebra systems, incorporated in HTML 5.
Example 5: InkMLExample 5: InkML Ink Messaging
Annotation
Archival
Pen-Based MathPen-Based Math
Input for CAS and document processing.
2D editing.Computer-mediated collaboration.
Pen-Based MathPen-Based MathDoes not require learning a special
language:\sum_{i=0}^r g_{r-i} X^i
sum(g[r-i]*X^i, i = 0..r);
Pen-Based MathPen-Based MathDifferent than natural language
recognition:◦ 2-D layout is a combination of writing and
drawing.◦ No fixed dictionary. ◦ Many similar few-stroke characters. ◦ Well segmented.◦ Highly ambiguous
Digital Ink FormatsDigital Ink Formats
• Collected by surface digitizer or camera
• Sequence of (x,y) points sampled at some known frequency
• Possibly other info (angles, pressure, etc)
• Grouping into traces, letters, words + labelling
InkML ConceptsInkML ConceptsTraces, trace groupsDevice information: sampling rate,
resolution, etc.Pre-defined and application defined
channelsTrace formats, coordinate
transformationsStreaming and archivalAnnotation text and XML
InkML EvolutionInkML EvolutionStarted as low-level language for traces
and hardware description. Explicitly disavowed semantics.
Wanted base language sufficiently rich to support full range of digital ink applications. Semantic grouping added, annotation, etc.
W3C StandardBuilt in to Microsoft Office 2010
81
Various Language ProjectsVarious Language ProjectsReflexAlmaJava/Aldor/C++ interopAbstract ObjectsLocal GCWWW GC
82
Research: Symbol RecognitionResearch: Symbol RecognitionMain idea:
Represent coordinate curves as truncated orthogonal series.
Advantages:◦ Compact – few coefficients needed◦ Geometric
– the truncation order is a property of the character set– gives a natural metric on the space of characters
◦ Algebraic – properties of curves can be computed algebraically (instead of numerically using heuristic parameters)
◦ Device independent – resolution of the device is not important
Choose a functional inner product, e.g.
⟨ f, g = ⟩ ∫ f(t) g(t) w(t) dt
This determines an orthonormal basis in the subspace of polynomials of degree d.Determine using GS on {1, t, t2, t3, ...}.
Can then approximate functions in subspaces
Inner Product and Basis Inner Product and Basis FunctionsFunctions
[a, b]
Like Symbols form CloudsLike Symbols form Clouds
ProblemsProblemsWant fast response –
how to work while trace is being captured.
Low RMS does not mean similar shape.
Pb 1. On-Line InkPb 1. On-Line InkThe main problem:
In handwriting recognition, the human and the computer take turns thinking and sitting idle.
We ask:Can the computer do useful work while the user is writing and thereby get the answer faster after the user stops writing?
We show:The answer is “Yes”!
On-Line Series On-Line Series CoefficientsCoefficientsIf we choose the right basis functions, then
the series coefficients can be computed on line.[Golubitsky+SMW CASCON 2008, ICFHR 2008]
The series coefficients are linear combinations of the moments, which can be computed by numerical integration as the points are received.
This is the Hausdorff moment problem (1921) , shown to be unstable by Talenti (1987).
It is just fine, however, for the orders we need.
Pb 2. Shape vs VariationPb 2. Shape vs VariationThe corners are not in the right places.
Work in a jet space to force coords & derivatives close.
Use a Legendre-Sobolev inner product
1st jet space => set μi = 0 for i > 1.Choose μ1 experimentally to maximize reco rate.Can be also done on-line.
[Golubitsky + SMW 2008, 2009]
Distance Between CurvesDistance Between CurvesApproximate the variation between curves
by some fn of distances between points.May be coordinate curves
or curves in a jet space.
Sequence alignmentInterpolation (“resampling”)
Why not just calculate the area?This is very fast in ortho series representation.
Distance Between CurvesDistance Between Curves
Comparison of Candidate to Comparison of Candidate to ModelsModelsUse Euclidean distance in the coefficient space.
Just as accurate as elastic matching.
Much less expensive.
Linear in d, the degree of the approximation.< 3 d machine instructions (30ns) vs several thousand!
Can trace through SVM-induced cells incrementally.
Normed space for characters gives other advantages.
Distance-Based Distance-Based ClassificationClassification
Distance-Based Distance-Based ClassificationClassification
GeometryGeometry
C = (1‒ t) A + t B
• Can compute distance of a sample to this line• Convex hull of a set of models• SVM separating planes
• Linear homotopies within a class
Distance-Based Distance-Based ClassificationClassification
Distance-Based Distance-Based ClassificationClassification
Error Rates as Fn of Error Rates as Fn of DistanceDistance
SVM Convex Hull
Error rate as fn of distance gives confidence measure for classifiers [MKM – Golubitsky + SMW]
Recognition SummaryRecognition SummaryDatabase of samples => set of LS
pointsCharacter to recognize =>
◦Integrate moments as being written◦Lin. trans. to obtain one point in LS space◦Classify by distance to convex hull of k-NN.
InkML allows natural representation of annotated database and real-time input.
Overall ConclusionsOverall ConclusionsMathematical problems provide
excellent challenges for language design.◦Rich, complex, hard◦Well-defined◦Performance matters – a lot!
Don’t be put off by the loud, confident proclamations of mass-market language designers.
100