LINQ in BreadthQuerying everything everywhere

Bart J.F. De SmetMicrosoft Corporationbartde@microsoft.comblogs.bartdesmet.net/bart

LINQ in your DNA

DNA is declarative

Infinite possibilities

Image: http://www.hawkdigital.com/images/InfiniteCre8tivity.jpg

LINQ to Anything?

toAlcohol

Image: http://www.hiphongkong.com/bars/linq.jpg

LINQ from Anywhere?

Image: http://www.linq.com.hk/findus.html

LINQ ~ ∞²

SQL X

ML

ObjectsSharePoint

Anything

C#

Visual Basic

F#

PowerShell

Anything

Fan

inFan out

LINQ

A different mindset?

Image: http://www.endlessgames.com/ns/Linq/Linq.gif

C# 3.0 – LINQ and Lambda

Image: http://vijay.screamingpens.com/archive/2008/05/04/linq-amp-lambda.aspx

Language Integrated Query

Lambdas

Extension

methods

Type inference

DemoTo refresh, hit F5

Syntactic sugar

Image: http://hummer.syntacticsugar.com:8080/kb_upload/Image/company_images/logo.png

from x in xs where x % 2 == 0 select x

xs.Where(x => x % 2 == 0).Select(x => x)

Operator

Operand

Expression

Query operators

IEnumerable<R> SelectMany<T,R>( this IEnumerable<T>, Func<T, IEnumerable<R>>)

Monads

Bind<T> :: M<T> (T M<R>) M<R>

Join<T> :: M<M<T>> M<T>

Return<T> :: T M<T>

IQueryable<R> SelectMany<T,R>( this IQueryable<T>, Expr<Func<T, IQueryable<R>>>)

Don't fear the monads

Source: http://channel9.msdn.com/shows/Going+Deep/Brian-Beckman-Dont-fear-the-Monads

from x in 123.AsMaybe()from y in Maybe<int>.Nothingselect x + y;

Nullables through

LINQ

Extension methods

Source: http://en.wikipedia.org/wiki/Extension_method

IEnumerable<R> SelectMany<T,R>( this IEnumerable<T>, Func<T, IEnumerable<R>>)

IQueryable<R> SelectMany<T,R>( this IQueryable<T>, Expression<Func<T, IQueryable<R>>>)

Extension by design

Declare Implement

interface IAmTooBig { Lorem Ipsum(Dolor sit); Amet Consectetur(); Adipiscing Elit(Ut eu); Felis At(Enim commodo); Consectetur Donec(); Tincidunt Laoreet(); Turpis Vestibulum(); …}

class MegaThingy : IAmTooBig { … public Amet Consectetur( { throw new NotImplementedException ("Sounds like Latin"); } …}

You gotta love lambdas…

Image: http://minimonimania.files.wordpress.com/2008/11/maimicode.png

…but don't overdo it

Image: http://stackoverflow.com/questions/515413/what-are-some-resources-for-learning-lambda-calculus

Expressiveness for freeParallel.For(0, height, j => { PixelData* outPixel = fastOut.GetInitialPixelForRow(j); PixelData* startPixel = fastStart.GetInitialPixelForRow(j); PixelData* endPixel = fastEnd.GetInitialPixelForRow(j);

for (int i = 0; i < width; i++) { outPixel->R = (byte)((startPixel->R * blend) + .5 + (endPixel->R * (1 - blend))); outPixel->G = (byte)((startPixel->G * blend) + .5 + (endPixel->G * (1 - blend))); outPixel->B = (byte)((startPixel->B * blend) + .5 + (endPixel->B * (1 - blend)));

outPixel++; startPixel++; endPixel++; }});

Operand

Operator

Closures

int max = 100;xs.Where(x => x < max)

Captured outer

<>__Closure c = new <>__Closure();c.max = 100;xs.Where(c.<>__Where);

class <>__Closure { public int max; public bool <>__Where(int x) { return x < max; }}

Know your side-effects!

Image: http://www.undoge.org/wp-content/uploads/2008/04/cartoon-sideeffect.gif

Func<string> read;using (var fs = File.OpenRead("bar.txt")) read = () => fs.ReadToEnd();

string result = read();IEnumerable<int> invert = null;try { invert = from x in xs select 1 / x;}catch (DivideByZeroException) { Console.WriteLine("Oops!");}

foreach (var x in invert) Console.WriteLine(x);

Lazy = good

• C# 2.0 iterators• yield return• On-demand fetch• E.g. LINQ to Objects

• C# 3.0 lambdas• () => compute• On-demand evaluation• E.g. TPL futures

• “Yielding control” to library/runtime

Image: http://onemansblog.com/2007/04/03/man-too-lazy-to-walk-dog/

Selectforeach

Select

Sele

ctforeac

hfo

reac

h

WhereWhere

Where

Query results

Data sourceIEnumerable<T>IEnumerable<R>

DemoLambda the ultimate

Query expressions cheat sheet

http://community.bartdesmet.net/blogs/bart/archive/2008/08/30/c-3-0-query-expression-translation-cheat-sheet.aspx

Freedom of choice

class Homer { public Marge Where(Func<Bart, bool> p) { … } …}class Marge { public Homer Select(Func<Maggie, Lisa> p) { … }}

var res = (from x in new Homer() where x.HasSkateboard select x.Sister).Father;

Are predicates Boolean-valued?Dim res = From product In ctx.Products _ Where (<And> <Gt> <FieldRef Name="UnitPrice"/> <Value Type="Integer">123</Value> </Gt> <BeginsWith> <FieldRef Name="ProductName"/> <Value Type="String">C</Value> </BeginsWith> </And>) _ Select New With {.Name = product.ProductName, _ .Price = product.UnitPrice }Func<T,

XElement>

DemoLINQ to Regular Expressions

The runtime in control

from x in xs where x > 0 where x < 100 select x

from x in xs orderby x where x < 100 select x

from x in xs where x > 0 && x < 100 select x

from x in xs where x < 100 orderby x select x

from x in xs where x < 0 && x > 0 select x

Always false

Where do expression trees come from?

• Visual Basic and C# lambdas• Expression<Func<…>>• Expression<Action<…>>

• Dynamic LINQ• Included in VS 2008 samples• ”Lambdas as strings”

But no statement trees yet!

Expression trees

Expression

Statement

Declaration

Dynamic Languag

e Runtime

You are here

Meta-programming

Func<int, int, int> add = (a, b) => a + b;var res = add(1, 2);Expression<Func<int, int, int>> add = (a, b) => a + b;var res = ((Func<int, int, int>)add.Compile())(1, 2); Code as

data

λ

+

a b

{ a, b }

ldarg.0ldarg.1addret

+ a b

Visitors

+

a b

<

0

string VisitAdd(BinaryExpression ex) { return VisitPrefix(ex, "+");}string VisitLess(BinaryExpression ex) { return VisitPrefix(ex, "<"); }string VisitConst(ConstantExpression ex) { return "" + ex.Value;}string VisitParam(ParameterExpression ex) { return ex.Name;}string VisitPrefix(BinaryExpression ex, string op) { var left = Visit(ex.Left); var right = Visit(ex.Right); return "(" + op + " " + left + " " + right + ")";}

a a b (+ a b) (+ a b) 0(< (+ a b) 0)

VisitLessVisitPrefix

VisitAddVisitPrefix

VisitParam

VisitConst

a b

+ 0

<

DemoFun with Expression Trees

LINQ through PowerShell

Life without expression trees?

Responsibility• Complex

visitors• Language

specifics• Rich source

domain

Expressive• Rich trees• Full

semantics• For free

Domain-specific trees

products.Where(p => p.Price > 100 && p.Name.StartsWith("C"))

NumberColumn TextColumn

Operator overloading

Image: http://www.railwaysleeper.com/KFdeliveryservice.htm

Intelligent operator overloading

p => p.Price > 100 && p.Name.StartsWith("C")

&

> StartsWith

“C”Name100Price

class NumberColumn { static Condition operator >( NumberColumn col, int value) { … }}

class TextColumn { Condition StartsWith( string value) { … }}

intNumberColumn string

TextColumn

Condition

Condition

Condition

Func<Product, Condition>

class Condition { static Condition operator &( Condition left, Condition right) { … }}

Fit-n-finish user experience

class Table<T> { public Filtered<T> Where(Func<T, Condition> filter) { … }} Determines next

operators in the query

Filtered<T>

Ordered<T>

Table<T>

OrderBy*

Where

ThenBy*

OrderBy*Where Projected<T>

Select

Select

Select

from p in products where p.Price > 100from p in products orderby p.Pricefrom p in products orderby p.Price, p.Namefrom p in products orderby p.Price where p.Price > 100from p in products where p.Price > 100 orderby p.Pricefrom p in products where p.Price > 100 select p.Namefrom p in products orderby p.Price select p.Namefrom p in products select p.Name

The result

• Expression trees• Everything works by default• Runtime exceptions to restrict• Possible clash of semantics

• Hand-crafted query patterns• Nothing works by default• Static compile-time checking• Own type universe

Image: http://atheistexile.com/libertyUSign5.jpg

DemoExceLinq and SignaLinq

Summary

• LINQ != a foreign concept• Design lessons learned

• Extension-by-design• Leverage language patterns• Get to know you monads!

• LINQify the world• Fan-in and fan-out• Data is everywhere

Image: http://da-crystal.net/GCMS/blog/arabic-book-start-linq/

Code = data!

Q & A