Generic Programming and First-Class Libraries

Jaakko Jarvijarvi@cs.tamu.edu

September 15, 2004

Overview

I About software libraries

I Generic programming as a paradigm for constructing reusablelibraries

I Research agenda for generic programming and first-classlibraries.

I Theory of generic programming, language supportI Library-specific optimizationsI Library-specific analysesI Compiler support to enable the above

I Conclusion

Others behind this material...

I Andrew Lumsdaine, Douglas Gregor (IU)Dave R. Musser (RPI)Bjarne Stroustrup, Jaakko Jarvi (Texas A&M)Sibylle Schupp (Chalmers, Goteborg, Sweden)

Alex Stepanov (Adobe, no picture)

Software libraries

I Static collections of routines/functions/classes?

I Libraries (also) encapsulate domain specific knowledge

I Syntax appropriate for a domainI Optimizations specific to a domainI Safety checks specific to a domain

I Standardization and layering of libraries ⇒ reuseI Containers/algorithms in C++ standard library

Software libraries

I Static collections of routines/functions/classes?I Libraries (also) encapsulate domain specific knowledge

I Syntax appropriate for a domainI Optimizations specific to a domainI Safety checks specific to a domain

I Standardization and layering of libraries ⇒ reuseI Containers/algorithms in C++ standard library

Software libraries

I Static collections of routines/functions/classes?I Libraries (also) encapsulate domain specific knowledge

I Syntax appropriate for a domainI Optimizations specific to a domainI Safety checks specific to a domain

I Standardization and layering of libraries ⇒ reuseI Containers/algorithms in C++ standard library

Libraries do not get much attention in CS

Generic programming: discipline for library design

I Conceptual categorization of a computational domainI Description as interfaces between algorithms and types

I Decoupling of algorithms and problem domainI Reducing algorithms to their minimal conceptual requirements

I Including non-functional requirements in the abstractionprocess, design, and implementation

∼ methodology of generic programming

Generic programming: an emerging discipline

I Successful in C++I The Standard Template Library (STL) 1994

I Musser, Stepanov

I The Standard Template Adaptive Template Library (STAPL)1998

I Rauchwerger et al

I The Matrix Template Library (MTL) 1998, The Boost GraphLibrary (BGL) 2000

I Siek, Lumsdaine, Lee

I C++ Boost library collection (MultiArray, MPL, µBLAS, ...)

Tens of thousands of users.

I Growing interest both in industry and academia.

Generic programming: lifting process

Generic Programming: Algorithms

Lift

Minimal requirements: works with maximal

family of types

Concrete algorithm:

requires specific data type

Less specialized:

works with more

than one type

A0

A1

Lift

Lift

Am

Remove an unneeded

requirement on the type

Generic

algorithm

. . .

Start here

Maximal family of types?

Maximal with Respect to What?

Lift

Maintain usefulness of the algorithm – make efficiency part of the

requirements

Concrete algorithmA0

A1

Lift

Lift

Am

Generic

algorithm

. . .

When instantiated, the generic algorithm is as efficient asthe original concrete algorithm

I When instantiated, the generic algorithm is as efficient as theoriginal concrete algorithm.

Key ideas of generic programming

I More than minimal capabilities may enable a faster/betterimplementation

I Automatic dispatching to most efficient implementation

I Algorithms must be as efficient as if written for concrete typesI No abstraction penalty

I High-performance regardless of abstraction

I High-performance because of abstraction

I Serious approach to libraries

Generic programming: example

double sum(double∗ array, int n) {double s = 0;for (int i = 0; i < n; ++i)

s = s + array[i];return s;

}

I RequirementsI Element type must be of type doubleI Elements stored in an array

Generic programming: example

double sum(list node∗ first, list node∗ last) {double s = 0;while (first != last) {

s = s + first→ data; first = first→ next;}return s;

}

I RequirementsI Element type must be of type doubleI Elements stored in a linked list

Generic programming: example

double sum(double∗ array, int first, int last) {double s = 0;while (first != last)

s = s + array[first++];return s;

}

I RequirementsI Element type must be of type doubleI Elements stored in an array

Generic programming: example

template <class T>T sum(T∗ first, T∗ last) {

T s = 0;while (first != last)

s = s + ∗first++;return s;

}

I RequirementsI Element type must support +I Elements addressed by pointers

Generic programming: example

template <class T>T sum(T∗ first, T∗ last, T s) {

while (first != last)s = s + ∗first++;

return s;}

I RequirementsI Element type must support +I Elements addressed by pointers

Generic programming: example

template <class InputIterator>InputIterator::value typesum(InputIterator first, InputIterator last,

InputIterator::value type s) {while (first != last)

s = s + ∗first++;return s;

}

I RequirementsI Element type must support +I InputIterator must support !=, ++, ∗I Must be able to access the associated type value type

I ’minimal requirements’ 6= ’fewest requirements’

Example usage: programming with concepts

I sum can be used with any type that satisfies the concept ofInput Iterator

double x[10];vector<string> y(42);list<complex<double> > z(100);

double a = sum(x, x + 10, 0.0)string b = sum(y.begin(), y.end(), ””);complex<double> c = sum(z.begin(), z.end(), 0);

Concepts

I Group requirements on abstractions (usually types) intoreusable entities

I Valid expressionsI Semantic constraintsI Complexity guaranteesI Associated types

I Allow concise specification of constraints of parameters togeneric algorithms

I Types can model a concept

I Concepts can refine other concepts ⇒ concept taxonomies

I Design tool: manifestations in programming languages vary

Example: conjugate gradient

int cg(const LinearOperator& A, HilbertSpaceX& x,const HilbertSpaceB& b) {

typedef typename itl traits<HilbertSpaceX>::value type T;

T rho, rho previous, alpha, beta;HilbertSpaceX p(size(x)), q(size(x)), r(size(x)), z(size

(x));

itl::mult(A, itl::scaled(x, −1.0), b, z);itl::solve(L, z, r);

while (! iter.finished(r)) {rho = itl::dot conj(r, r);

if (!iter.first()) {beta = rho / rho previous;itl::add(z, itl::scaled(p, beta), p);

}

itl::mult(A, p, q);alpha = rho / itl::dot conj(p, q);itl::add(x, itl::scaled(p, alpha), x);

itl::add(r, itl::scaled(z, −alpha), r);rho previous = rho;

++iter;}return iter.error code();

}

I Example from ITL[Lumsdaine, Lee, Siek]

I Natural high-level (domainspecific) description of thealgorithm

I Can operate on any typesthat meet the conceptrequirements

I Abstract description of analgorithm is also itsimplementation

I No penalty for abstractions

Towards first class libraries

I The above is being used, and working (though not perfect)today.

I Concepts provide a mechanism for organizing libraries anddescribing their interfaces.

I Concepts provide an abstraction barrier.

I What is missing?

template <class InputIterator>InputIterator::value typesum(InputIterator first, InputIterator last,

InputIterator::value type s);

I Compiler is not aware of InputIterators

Languages vs. Libraries

I Libraries are domain specific embedded languages — yet not“real” languages

I Second-class citizens in compilation process

I Compilers cannot understand abstractions in libraries, andcannot thus optimize or analyze libraries

Towards first class libraries

I Understanding concepts

I Improved support of syntactic concepts in programminglanguages

I Compiler frameworks that can supportI Concept-based optimizations described in librariesI Static analyses/checks described in libraries

Concepts and improved support for syntactic concepts

I Several representations for conceptsI Many in C++: STL’s semi-formal concept descriptions,

Tecton’s descriptions as algebras, Caramel, built-in languagesupport in the works.

I Manifestations in other programming languages: Type classesin Haskell, ML module system, OO and F-boundedpolymorphism

I Unified view to type parameter constraints in differentprogramming languages

I Unified representation to capture all features of moderngeneric programming (and constraint systems)

I Basis for designing new language features to support genericprogramming

Compiler frameworks that can support ...

I previous talk

Simplicissimus: Extensible simplifier

I Traditional simplifierI Fixed set of rewrite rules applied to built-in types

I Extensible simplifierI Rewrite rules for user-defined typesI Extensible on the set of rewrite rulesI Extensible on the set of operations

I Descriptions of simplifications belong in the library

I Described using concepts

Concept-based rules

I Traditional rules are redundant:(+ x 0) → x(∗ x 1) → x(concat x ””) → x(and x true) → x

I Only one concept-based rule required to cover all cases:(op x id) → x

op an operator, id identity element

I Simplification enabled for a user-defined type by modeling theRight Identity concept

I Simplicissimus allows rewriting any C++ expressionI All rules in a library to be applied by the compiler

Library-specific static analyses

I STLlint: static checker for C++ code, that knows about STLI Checks iterator invalidation, past-the-end dereferencingI Checks proper use of sorting, searching, etc.I Extensible, all knowledge is in a library

I Enforces semantic constraints on the use of STL algorithms

I Decouples semantic checking from algorithm specificationI Generic algorithms involve high-level semantic properties

I introduce, remove, preserveI E.g. sort() introduces sorted propertyI Statically keep track of these properties

Library-specific static analysis

I STL: 10 container classes, 2 iterators per container class,∼ 70 algorithms

I Algorithms and container operations affect high-levelproperties

I New property → revisit every algorithm (and vice versa)I Redundant work hampers extensibility → algorithmic concepts

I Categorize behavior of algorithmsI No changes needed if new algorithm can be modeled using

existing concepts

STLlint example

vector<int> v;

// Sort v: after this, v should be ’sorted’ with predicate ’less<int>’sort(v.begin(), v.end(), less<int>());

// Find out where we should insert ’42’// This is an error: v was sorted with less<int>, not greater<int>!vector<int>::iterator i = lower bound(v.begin(), v.end(), 42, greater<

int>());

tryit.cpp", line 21, warning: sequence may have been sorted with a

different predicate than the one given

vector<int>::iterator i =

lower_bound(v.begin(), v.end(), 42, greater<int>());

in call to function lower_bound_pred at "tryit.cpp", line 21

Importance of standardization

I STL was a boon to C++ programmers

I Big reason for this: standardization!

I Never again must I write a string/array/vector class!

I But also, efforts to build tools (like STLlint) become easier tojustify, or layer libraries on top of STL.

Related approaches

I Different forms of constrained polymorphismI Type classes, module systems, predicate dispatch

I Data-type generic programming

I Generative programming

I Metaprogramming and staged languages

Future work

I Language support for concepts

I Theoretical foundations of concepts, libraries and genericprogramming

I Compiler support for first-class libraries

I Bigger, better, faster, optimized, checked libraries

Conclusion

I Software advances with libraries

I Libraries are more that collections of reusable static softwareelements

I Libraries must integrate more closely with compilers

I Generic programming and concepts provide a discipline forlibrary construction