5/10/2015 1 Domain-Specific Embedded Languages with Boost.Proto Or, "How I Learned to Stop Worrying...
-
Upload
douglas-hancock -
Category
Documents
-
view
216 -
download
0
Transcript of 5/10/2015 1 Domain-Specific Embedded Languages with Boost.Proto Or, "How I Learned to Stop Worrying...
04/18/23 1
Domain-Specific Embedded Languages
with Boost.ProtoOr, "How I Learned to Stop
Worrying and Love Expression Templates"
copyright 2007 Eric Niebler04/18/23 2
Talk OverviewGoal: Designing declarative domain-specific
libraries using Boost.Proto.1. Domain Specific Embedded Languages2. DSEL Design with Boost.Proto
1. Expression construction2. Expression evaluation3. Expression introspection4. Expression transformation
3. Extending Boost.Proto
copyright 2007 Eric Niebler04/18/23 3
Hello, Boost.Proto!// #includes ...using namespace boost::proto;terminal< std::ostream & >::type cout_ = { std::cout };
template< typename Expr >void evaluate( Expr const & expr ){ default_context ctx; eval(expr, ctx);}
int main(){ evaluate( cout_ << "hello" << ',' << " world" );}
copyright 2007 Eric Niebler04/18/23 6
DSEL Example: Boost.Lambda
DSEL for creating unnamed function objects
(and very long compiler errors)
Jaakko Järvi
std::for_each( a.begin(), a.end(), std::cout << _1 << ' ');
copyright 2007 Eric Niebler04/18/23 7
DSEL Example: Boost.Spirit
Parser Generatorsimilar in purpose to lex /
YACC DSEL for declaring
grammarsgrammars can be recursiveApproximates Backus-Naur
FormJoel de
Guzman
copyright 2007 Eric Niebler04/18/23 8
Infix Calculator Grammar In Extended Backus-Naur Form
group ::= '(' expr ')'fact ::= integer | groupterm ::= fact (('*' fact) | ('/' fact))*expr ::= term (('+' term) | ('-' term))*
copyright 2007 Eric Niebler04/18/23 9
template< class Derived > struct Op { ... };
template< class Tag, class L, class R >struct BinOp : Op< BinOp<Tag, L, R> > { L const & left; R const & right; ... };
template< class L, class R >BinOp< RShift, L, R > constoperator >> ( Op<L> const & l, Op<R> const & r ) { ... }
What is an Expression Template? Tree representation of an expression Built using templates and operator
overloading
Builds the tree representation of "expr1 >> expr2".
BinOp<> is a container for two sub-expressions.
copyright 2007 Eric Niebler04/18/23 10
group ::= '(' expr ')'fact ::= integer | groupterm ::= fact (('*' fact) | ('/' fact))*expr ::= term (('+' term) | ('-' term))*
Infix Calculator Grammar In Extended Backus-Naur Form In Boost.Spirit
spirit::rule group, fact, term, expr;
group = '(' >> expr >> ')';fact = spirit::int_p | group;term = fact >> *(('*' >> fact) | ('/' >> fact));expr = term >> *(('+' >> term) | ('-' >> term));
copyright 2007 Eric Niebler04/18/23 11
rule group = '(' >> expr >> ')';
What is an Expression Template? DSELs like Spirit interpret expression trees
in a domain-specific way.
>>
>> ')'
'(' expr
BinOp< RShift, BinOp< RShift, Term<char>, Term<rule> >, Term<char>>
copyright 2007 Eric Niebler04/18/23 12
rule group = '(' >> expr >> ')'
What is an Expression Template? DSELs like Spirit interpret expression trees
in a domain-specific way.
struct rule{ template< class Derived > rule & operator= ( Op< Derived > const & op ) { ... magic happens here ... return *this; } ...};
Somehow interpret op as a grammar rule.
This is hard!
copyright 2007 Eric Niebler04/18/23 13
DSELs in C++, the Sad Truth Very difficult to design Code is hard to read and maintain Terrible compiler error messages So why not a DSEL for defining
DSELs?!Expression constructionExpression evaluationExpression introspectionExpression transformation
copyright 2007 Eric Niebler04/18/23 15
A Calculator DSEL
template<class Int> struct placeholder : Int {};
terminal< placeholder< mpl::int_<1> > >::type _1;terminal< placeholder< mpl::int_<2> > >::type _2;
int main(){ (_1 + _2); return 0;}
Expression construction
expr< tag::plus, args< expr< tag::terminal, term< placeholder< mpl::int_<1> > >, 0 > &, expr<tag::terminal, term< placeholder< mpl::int_<2> > >, 0 > & >, 2>
copyright 2007 Eric Niebler04/18/23 16
Calculator, Reloaded
// To be defined ...struct calculator_context // ...
int main(){ // Placeholders have values 1. and 2. respectively: calculator_context ctx(1., 2.);
// Evaluate a calculator expression with the // calculator_context std::cout << eval(_1 + _2, ctx) << std::endl;}
Expression evaluation
copyright 2007 Eric Niebler04/18/23 17
Calculator, Reloaded
// This describes how to evaluate calculator expressionsstruct calculator_context : callable_context< calculator_context const >{ double d[2]; typedef double result_type; calculator_context(double d1, double d2) { d[0] = d1; d[1] = d2; }
template<typename Int> double operator()(tag::terminal, placeholder<Int> i) const { return d[i-1]; // handle argument placeholders }};
Expression evaluation, continued
copyright 2007 Eric Niebler04/18/23 18
The Context Concept All context types must look like this:struct Context{ template<typename Expr> struct eval { typedef unspecified result_type; result_type operator()(Expr &expr, Context &context) const; };};
Helpers for building contextsdefault_context / default_evalcallable_context / callable_eval
copyright 2007 Eric Niebler04/18/23 21
Expression Introspection Query the properties of
expression trees at compile time
Utilities for defining grammars proto::matches<> for
grammar-checking your expressions.
“Be grammatical!"
copyright 2007 Eric Niebler04/18/23 22
An Input/Output DSEL// Terminals for lazy input/output expressions:terminal< std::istream & >::type cin_ = { std::cin };terminal< std::ostream & >::type cout_ = { std::cout };
template<class IO>void do_io( IO const & io ){ eval( io, default_context() );}
int main(){ int i = 0; do_io( cin_ >> i ); // reads an int do_io( cout_ << i ); // writes an int}
What if do_io() needs to know whether IO
represents an input or an output operation?
copyright 2007 Eric Niebler04/18/23 23
Answer: Input/Output Patterns// A pattern to match Input expressionsstruct Input : shift_right< terminal< std::istream & >, _ >{};
// A pattern to match Output expressionsstruct Output : shift_left< terminal< std::ostream & >, _ >{};
template<class IO>void do_io(IO const &io){ if( matches<IO, Input>::value ) std::cout << "Input!\n"; if( matches<IO, Output>::value ) std::cout << "Output!\n"; // ...}
Evaluated at compile time!
Matches "cin_ >> i"
Matches "cout_ << i"
copyright 2007 Eric Niebler04/18/23 24
Whoops! A Problem ...// A pattern to match Output expressionsstruct Output : shift_left< terminal< std::ostream & >, _ >{};
Why doesn't this expression match the Output pattern?
cout_ << 1 << 2;
shift_left< shift_left< terminal< std::ostream & >::type , terminal< int const & >::type >::type , terminal< int const & >::type>::type
( cout_ << 1 ) << 2;
copyright 2007 Eric Niebler04/18/23 25
Solution: A Recursive Pattern// A pattern to match Output expressionsstruct Output : or_< shift_left< terminal< std::ostream & >, _ > , shift_left< Output, _ > >{};
or_<> alternates tried in order Short-circuit evaluation Notice recursion on Output pattern!
// A pattern to match Output expressionsstruct Output : or_< shift_left< terminal< std::ostream & >, _ > , shift_left< Output, _ > >{};
copyright 2007 Eric Niebler04/18/23 26
From Patterns to Grammars
struct Calculator;
struct Double : terminal< double > {};struct Placeholder : terminal< placeholder<_> > {};struct Plus : plus< Calculator, Calculator > {};struct Minus : minus< Calculator, Calculator > {};struct Multiplies : multiplies< Calculator, Calculator > {};struct Divides : divides< Calculator, Calculator > {};
struct Calculator : or_< Double, Placeholder, Plus, Minus, Multiplies, Divides >{};
A Proto Grammar for our Calculator:
copyright 2007 Eric Niebler04/18/23 27
Proto vs. EBNF Are Proto grammars as powerful as EBNF?
group ::= '(' expr ')'fact ::= double | groupterm ::= fact (('*' fact) | ('/' fact))*expr ::= term (('+' term) | ('-' term))*
struct Expr;struct Dbl : terminal<double> {};struct Plus : plus<Expr, Expr> {};struct Minus : minus<Expr, Expr> {};struct Mult : multiplies<Expr, Expr> {};struct Div : divides<Expr, Expr> {};
struct Expr : or_<Dbl, Plus, Minus, Mult, Div>{};
EBNF Proto
Why don’t Proto grammars need:• Sequencing?• Repetition?
• Precedence?• Associativity?
copyright 2007 Eric Niebler04/18/23 28
What are grammars good for?// A function that only accepts valid calculator expressions!template<class Expr>typename enable_if< matches< Expr, Calculator > >::typeevaluate( Expr const & expr ){ // ...}
enable_if<> tricks to prune overload sets mpl::if_<> for type selections Compile-time assertions Defining tree transformations Controlling Proto's operator overloading
"It's the Grammar, stupid."
copyright 2007 Eric Niebler04/18/23 29
Expression Transformation Turns an expression tree into some other object Transformations attached to grammar rules Transformations provided by Proto or user defined
Challenge!For any calculator expression, find the arity. Eg.:
_1 + 42 has arity 1 _2 * 42 has arity 2
copyright 2007 Eric Niebler04/18/23 30
Arity Calculation
Expression Arity
double 0
_1 1
_2 2
Left op Right max(arity(Left), arity(Right))
Calculator expression arity is calculated as follows:
copyright 2007 Eric Niebler04/18/23 31
From Grammars to Transforms// Ye olde calculator grammarstruct Arity;
struct Double : terminal< double > {};struct Placeholder : terminal< placeholder<_> > {};struct Plus : plus< Arity, Arity > {};struct Minus : minus< Arity, Arity > {};struct Multiplies : multiplies< Arity, Arity > {};struct Divides : divides< Arity, Arity > {};
struct Arity : or_< Double, Placeholder, Plus, Minus, Multiplies, Divides >{};
copyright 2007 Eric Niebler04/18/23 32
// Ye olde calculator grammarstruct Arity;
struct Double : terminal< double > {};struct Placeholder : terminal< placeholder<_> > {};struct Plus : plus< Arity, Arity > {};struct Minus : minus< Arity, Arity > {};struct Multiplies : multiplies< Arity, Arity > {};struct Divides : divides< Arity, Arity > {};
struct Arity : or_< Double, Placeholder, Plus, Minus, Multiplies, Divides >{};
Double Transform
struct Double : terminal< double > {};
struct Arity : or_< case_< Double, mpl::int_<0>() >, ... >{};
Anything that matches Double is transformed into mpl::int_<0>
copyright 2007 Eric Niebler04/18/23 33
Placeholder Transformtemplate<class Int> struct placeholder : Int {};terminal<placeholder<mpl::int_<1> > >::type _1;
struct Placeholder : terminal< placeholder<_> > {};
struct Arity : or_< ... , case_< Placeholder, _arg >, ... >{};
Proto’s _arg transform pulls the placeholder<> argument out of the terminal<>. And a placeholder<> IS-A compile-time integer representing its arity.
copyright 2007 Eric Niebler04/18/23 34
Binary Operator Transform
// A Lambda for evaluating the max arity of a binary// calculator expression.typedef mpl::max<Arity(_left), Arity(_right)> MaxArity;
struct Arity : or_< ... , case_< plus< Arity, Arity >, MaxArity() >, ... >{};
Anything that matches plus<...> is transformed according to the MaxArity() transform.
copyright 2007 Eric Niebler04/18/23 35
Fun with C++ Declaration Syntax What are all the things that this can
mean?
X( Y )Function call
Constructor call
Function type
Function-style cast
copyright 2007 Eric Niebler04/18/23 36
mpl::max<Arity(_left), Arity(_right)>()
mpl::max< , >()
Lambda Transforms Step 1: Recursively evaluate nested lambdas
mpl::max<Arity(_left), Arity(_right)>()
mpl::max<mpl::int_<X>, mpl::int_<Y> >()
Evaluate left and right sub-expressions according to the
Arity transform.
copyright 2007 Eric Niebler04/18/23 37
mpl::max<mpl::int_<X>, mpl::int<Y> >()
mpl::max<mpl::int_<X>, mp::int<Y> >::type()
mpl::int_<Z>()
Lambda Transforms Step 2: Evaluate meta-function
`
Look for a nested ::type typedef.
`
copyright 2007 Eric Niebler04/18/23 38
mpl::int_<Z>()
Lambda Transforms Step 3: Call object constructor
The function type is interpreted as a constructor invocation. Proto returns a
default constructed mpl::int_<Z>!
copyright 2007 Eric Niebler04/18/23 39
The Complete Transformstruct Arity;struct Double : terminal< double > {};struct Placeholder : terminal< placeholder<_> > {};struct Plus : plus< Arity, Arity > {};struct Minus : minus< Arity, Arity > {};struct Multiplies : multiplies< Arity, Arity > {};struct Divides : divides< Arity, Arity > {};
typedef mpl::max<Arity(_left), Arity(_right)> MaxArity;struct Arity : or_< case_< Double , mpl::int_<0>() > , case_< Placeholder , _arg > , case_< Plus , MaxArity() > , case_< Minus , MaxArity() > , case_< Multiplies , MaxArity() > , case_< Divides , MaxArity() > >{};
copyright 2007 Eric Niebler04/18/23 40
It works!// Defined as beforestruct Arity ... ;
// Display the arity of a calculator expressiontemplate<class Expr>void print_arity(Expr const & expr){ BOOST_MPL_ASSERT((matches<Expr, Arity>)); int i = 0; // dummy, not used std::cout << Arity::call(expr, i, i) << std::endl;}
int main(){ print_arity(_1 + 42); print_arity(_2 * 42);}
copyright 2007 Eric Niebler04/18/23 42
Calculator, Reloaded
// To be defined ...struct calculator_context // ...
int main(){ // Placeholders have values 1. and 2. respectively: calculator_context ctx(1., 2.);
// Evaluate a calculator expression with the calculator_context std::cout << eval(_1 + _2, ctx) << std::endl;}
Expression evaluation flashback
Q: Is eval() always needed to make proto expressions do something?
copyright 2007 Eric Niebler04/18/23 43
Calculator, Reloaded
int main(){ double data[] = { 1., 2., 3., 4. };
// Use the calculator DSEL to square each element std::transform( data, data + 4, data, _1 * _1 );}
A Calculator expression as an STL functor?
Whoops! "_1 * _1" doesn't have an operator() that takes and returns a double.
copyright 2007 Eric Niebler04/18/23 44
Calculator, Revolutions
struct calc_domain;
template< class Expr >struct calc : extends< Expr, calc< Expr >, calc_domain >{ calc( Expr const &expr = Expr() ) : extends< Expr, calc< Expr >, calc_domain >( expr ) {}
typedef double result_type; result_type operator()( double d1 = 0.0, double d2 = 0.0 ) const { calculator_context ctx( d1, d2 ); return eval( *this, ctx ); }};
Expression wrappers to add behaviors The "domain" of our
wrapped expressions
calc<Expr> extends Expr
Define operator() to evaluate expr with calculator_context
copyright 2007 Eric Niebler04/18/23 45
Calculator, Revolutions
// OK, _1 and _2 are calc<> expressions in the calc_domaincalc< terminal< placeholder< mpl::int_<1> > >::type > _1;calc< terminal< placeholder< mpl::int_<2> > >::type > _2;
Wrap our terminals ...
struct calc_domain : domain< generate< calc > >{};
Hook Proto's expression generator ...
In calc_domain, wrap all expression types in our calc<> wrapper.
copyright 2007 Eric Niebler04/18/23 47
Summing up ... Proto makes C++ DSEL design easy and fun! Good for expression template ...
... construction ... evaluation ... introspection ... transformation
Powerful extensibility mechanisms Used by xpressive, Spirit-2, Karma, etc ... Available in Boost Subversion and the File
Vault