Kdahlby 200908 Stldodn Linqinternals 090903222505 Phpapp01
description
Transcript of Kdahlby 200908 Stldodn Linqinternals 090903222505 Phpapp01
LINQ Internals
Keith Dahlbyhttp://solutionizing.net/@dahlbyk
Who am I?
Iowa Native Iowa State University Cedar Rapids SharePoint Inetium Language Geek
Agenda
Terminology Enabling Technologies Query Expressions & Translation Standard Query Operators LINQ to Objects LINQ to IQueryable Beyond
Terminology
Language-INtegrated Query Query Expressions
o C# 3.0 & VB 9 Featureo Syntactic Sugar on Method Calls
LINQ Providero Standard Query Operatorso *LINQ / LINQ to *
Enabling Technologies
Anonymous Typeso Type Inference
Extension Methods Lambda Expressions
o Expression Trees
Anonymous Types
C# VB
var entry = new { Title = "Dear Diary", DateTime.Now};
Console.WriteLine("{0:d}: {1}", entry.Now, entry.Title);
Dim entry = New With { _ .Title = "Dear Diary", _ DateTime.Now _}
Console.WriteLine("{0:d}: {1}", _ entry.Now, entry.Title)
var and Dim types are inferred!
Extension Methods
Must be defined in static class (VB Module) Appear in IntelliSense if namespace in using/Imports
First argument serves as instance variableImports System.Runtime.CompilerServices
Module StringExtensions <Extension()> _ Public Sub Print(ByVal aString As String) Console.WriteLine(aString) End SubEnd Module
Dim hello = "Hello from StringExtensions"hello.Print()
Extension Methods
C# supports this keywordstatic void ForEach<T>(this IEnumerable<T> source, Action<T> action){ foreach (T o in source) action(o);} Generic Delegatesdelegate void Action();delegate void Action<T>(T arg);delegate void Action<T1, T2>(T1 arg1, T2 arg2);delegate TResult Func<TResult>();delegate TResult Func<T, TResult>(T arg);delegate TResult Func<T1, T2, TResult>(T1 arg1, T2 arg2);
Lambda Expressions
Evolution of C# Delegatesdelegate string MyDelegate(int i);
static string MyMethod(int i) { … }MyDelegate del1 = new MyDelegate(MyMethod); // C# 1.0MyDelegate del2 = delegate(int i) { … }; // C# 2.0MyDelegate del3 = MyMethod; // C# 2.0MyDelegate del4 = (int i) => { … }; // C# 3.0Func<int, string> del5 = i => { … }; // C# 3.0
New in VB 9.0Dim del4 As MyDelegate = Function(i As Integer) …
Expression Trees
System.Linq.Expressions Abstract Syntax Tree representing code Assignable from lambda expression:Expression<Func<int, int>> inc = (a => a + 1); Compiles into:ParameterExpression CS$a;Expression<Func<int, int>> inc = Expression.Lambda<Func<int, int>>( Expression.Add( CS$a = Expression.Parameter(typeof(int), "a"), Expression.Constant(1, typeof(int))), new ParameterExpression[] { CS$a });
Query Expressions
Provide query keywords in C# & VBvar names = from s in Students where s.Age % 2 == 0 orderby s.Name descending select s.Name;
Translated into method calls:IEnumerable<string> names = Students.Where(s => s.Age % 2 == 0) .OrderByDescending(s => s.Name) .Select(s => s.Name);
Query Translation
Part of language specificationo Section 7.15.2 of C# 3.0 Language Specificationo Section 11.21 of Visual Basic 9.0 Language Specification
C# translation is rule-based:o “A query expression is processed by repeatedly applying
the following translations until no further reductions are possible.”
VB translation is sequential:o “Query operator translation occurs in the order in which
the query operators occur in the expression.”
C# Query Translation
Select and GroupBy clauses with continuationso from … into x … from x in ( from … ) …
Explicit range variable typeso from T x in e from x in ( e ) . Cast < T > ( )o join T x in e on k1 equals k2 join x in ( e ) . Cast < T > ( ) on
k1 equals k2
Degenerate query expressionso from x in e select x ( e ) . Select ( x => x )
From, let, where, join and orderby clauseso from, let and join are complicatedo from x in e where f … from x in ( e ) . Where ( x => f ) …o from x in e orderby k1 , k2 , … , kn … from x in ( e ) . OrderBy ( x =>
k1 ) . ThenBy ( x => k2 ) . … . ThenBy ( x =>
kn ) …
Select clauseso from x in e select v ( e ) . Select ( x => v )
GroupBy clauseso from x in e group x by k ( e ) . GroupBy ( x => k )o from x in e group v by k ( e ) . GroupBy ( x => k , x =>
v )
Demo: Queries and Translation
Standard Query Operators: 51 Total
Aggregate Average Count LongCount
Max Min Sum Concat
AsEnumerable Cast ToArray ToDictionary
AsQueryable OfType ToList ToLookup
ElementAt First Last Single
ElementAtOrDefault
FirstOrDefault LastOrDefault SingleOrDefault
SequenceEqual GroupBy Join GroupJoin
DefaultIfEmpty Empty Repeat Range
Skip SkipWhile Take TakeWhile
Distinct Select SelectMany Where
Except Any All Contains
Intersect OrderBy ThenBy Reverse
Union OrderByDesc ThenByDesc
Standard Query Operators: 42 “Real” Operators
Aggregate Average Count LongCount
Max Min Sum Concat
AsEnumerable Cast ToArray ToDictionary
AsQueryable OfType ToList ToLookup
ElementAt First Last Single
ElementAtOrDefault
FirstOrDefault LastOrDefault SingleOrDefault
SequenceEqual GroupBy Join GroupJoin
DefaultIfEmpty Empty Repeat Range
Skip SkipWhile Take TakeWhile
Distinct Select SelectMany Where
Except Any All Contains
Intersect OrderBy ThenBy Reverse
Union OrderByDesc ThenByDesc
Standard Query Operators: 24 in Visual Basic 9
Aggregate Average Count LongCount
Max Min Sum Concat
AsEnumerable Cast ToArray ToDictionary
AsQueryable OfType ToList ToLookup
ElementAt First Last Single
ElementAtOrDefault
FirstOrDefault LastOrDefault SingleOrDefault
SequenceEqual GroupBy Join GroupJoin
DefaultIfEmpty Empty Repeat Range
Skip SkipWhile Take TakeWhile
Distinct Select SelectMany Where
Except Any All Contains
Intersect OrderBy ThenBy Reverse
Union OrderByDesc ThenByDesc
Standard Query Operators: 11 in C# 3
Aggregate Average Count LongCount
Max Min Sum Concat
AsEnumerable Cast ToArray ToDictionary
AsQueryable OfType ToList ToLookup
ElementAt First Last Single
ElementAtOrDefault
FirstOrDefault LastOrDefault SingleOrDefault
SequenceEqual GroupBy Join GroupJoin
DefaultIfEmpty Empty Repeat Range
Skip SkipWhile Take TakeWhile
Distinct Select SelectMany Where
Except Any All Contains
Intersect OrderBy ThenBy Reverse
Union OrderByDesc ThenByDesc
Standard Query Operators
Aggregation Operations Concatenation Operations Converting Data Types Element Operations Equality Operations Generation Operations Grouping Data Join Operations Partitioning Data Projection Operations Quantifier Operations Filtering Data Set Operations Sorting Data
Aggregations Operations
Aggregate Average Count LongCount
Max Min Sum Concat
AsEnumerable Cast ToArray ToDictionary
AsQueryable OfType ToList ToLookup
ElementAt First Last Single
ElementAtOrDefault
FirstOrDefault LastOrDefault SingleOrDefault
SequenceEqual GroupBy Join GroupJoin
DefaultIfEmpty Empty Repeat Range
Skip SkipWhile Take TakeWhile
Distinct Select SelectMany Where
Except Any All Contains
Intersect OrderBy ThenBy Reverse
Union OrderByDesc ThenByDesc
Concatenation Operations
Aggregate Average Count LongCount
Max Min Sum Concat
AsEnumerable Cast ToArray ToDictionary
AsQueryable OfType ToList ToLookup
ElementAt First Last Single
ElementAtOrDefault
FirstOrDefault LastOrDefault SingleOrDefault
SequenceEqual GroupBy Join GroupJoin
DefaultIfEmpty Empty Repeat Range
Skip SkipWhile Take TakeWhile
Distinct Select SelectMany Where
Except Any All Contains
Intersect OrderBy ThenBy Reverse
Union OrderByDesc ThenByDesc
Converting Data Types
Aggregate Average Count LongCount
Max Min Sum Concat
AsEnumerable Cast ToArray ToDictionary
AsQueryable OfType ToList ToLookup
ElementAt First Last Single
ElementAtOrDefault
FirstOrDefault LastOrDefault SingleOrDefault
SequenceEqual GroupBy Join GroupJoin
DefaultIfEmpty Empty Repeat Range
Skip SkipWhile Take TakeWhile
Distinct Select SelectMany Where
Except Any All Contains
Intersect OrderBy ThenBy Reverse
Union OrderByDesc ThenByDesc
Element Operations
Aggregate Average Count LongCount
Max Min Sum Concat
AsEnumerable Cast ToArray ToDictionary
AsQueryable OfType ToList ToLookup
ElementAt First Last Single
ElementAtOrDefault
FirstOrDefault LastOrDefault SingleOrDefault
SequenceEqual GroupBy Join GroupJoin
DefaultIfEmpty Empty Repeat Range
Skip SkipWhile Take TakeWhile
Distinct Select SelectMany Where
Except Any All Contains
Intersect OrderBy ThenBy Reverse
Union OrderByDesc ThenByDesc
Equality Operations
Aggregate Average Count LongCount
Max Min Sum Concat
AsEnumerable Cast ToArray ToDictionary
AsQueryable OfType ToList ToLookup
ElementAt First Last Single
ElementAtOrDefault
FirstOrDefault LastOrDefault SingleOrDefault
SequenceEqual GroupBy Join GroupJoin
DefaultIfEmpty Empty Repeat Range
Skip SkipWhile Take TakeWhile
Distinct Select SelectMany Where
Except Any All Contains
Intersect OrderBy ThenBy Reverse
Union OrderByDesc ThenByDesc
Generation Operations
Aggregate Average Count LongCount
Max Min Sum Concat
AsEnumerable Cast ToArray ToDictionary
AsQueryable OfType ToList ToLookup
ElementAt First Last Single
ElementAtOrDefault
FirstOrDefault LastOrDefault SingleOrDefault
SequenceEqual GroupBy Join GroupJoin
DefaultIfEmpty Empty Repeat Range
Skip SkipWhile Take TakeWhile
Distinct Select SelectMany Where
Except Any All Contains
Intersect OrderBy ThenBy Reverse
Union OrderByDesc ThenByDesc
Grouping Data
Aggregate Average Count LongCount
Max Min Sum Concat
AsEnumerable Cast ToArray ToDictionary
AsQueryable OfType ToList ToLookup
ElementAt First Last Single
ElementAtOrDefault
FirstOrDefault LastOrDefault SingleOrDefault
SequenceEqual GroupBy Join GroupJoin
DefaultIfEmpty Empty Repeat Range
Skip SkipWhile Take TakeWhile
Distinct Select SelectMany Where
Except Any All Contains
Intersect OrderBy ThenBy Reverse
Union OrderByDesc ThenByDesc
Join Operations
Aggregate Average Count LongCount
Max Min Sum Concat
AsEnumerable Cast ToArray ToDictionary
AsQueryable OfType ToList ToLookup
ElementAt First Last Single
ElementAtOrDefault
FirstOrDefault LastOrDefault SingleOrDefault
SequenceEqual GroupBy Join GroupJoin
DefaultIfEmpty Empty Repeat Range
Skip SkipWhile Take TakeWhile
Distinct Select SelectMany Where
Except Any All Contains
Intersect OrderBy ThenBy Reverse
Union OrderByDesc ThenByDesc
Partitioning Data
Aggregate Average Count LongCount
Max Min Sum Concat
AsEnumerable Cast ToArray ToDictionary
AsQueryable OfType ToList ToLookup
ElementAt First Last Single
ElementAtOrDefault
FirstOrDefault LastOrDefault SingleOrDefault
SequenceEqual GroupBy Join GroupJoin
DefaultIfEmpty Empty Repeat Range
Skip SkipWhile Take TakeWhile
Distinct Select SelectMany Where
Except Any All Contains
Intersect OrderBy ThenBy Reverse
Union OrderByDesc ThenByDesc
Projection Operations
Aggregate Average Count LongCount
Max Min Sum Concat
AsEnumerable Cast ToArray ToDictionary
AsQueryable OfType ToList ToLookup
ElementAt First Last Single
ElementAtOrDefault
FirstOrDefault LastOrDefault SingleOrDefault
SequenceEqual GroupBy Join GroupJoin
DefaultIfEmpty Empty Repeat Range
Skip SkipWhile Take TakeWhile
Distinct Select SelectMany Where
Except Any All Contains
Intersect OrderBy ThenBy Reverse
Union OrderByDesc ThenByDesc
Quantifier Operations
Aggregate Average Count LongCount
Max Min Sum Concat
AsEnumerable Cast ToArray ToDictionary
AsQueryable OfType ToList ToLookup
ElementAt First Last Single
ElementAtOrDefault
FirstOrDefault LastOrDefault SingleOrDefault
SequenceEqual GroupBy Join GroupJoin
DefaultIfEmpty Empty Repeat Range
Skip SkipWhile Take TakeWhile
Distinct Select SelectMany Where
Except Any All Contains
Intersect OrderBy ThenBy Reverse
Union OrderByDesc ThenByDesc
Filtering Data
Aggregate Average Count LongCount
Max Min Sum Concat
AsEnumerable Cast ToArray ToDictionary
AsQueryable OfType ToList ToLookup
ElementAt First Last Single
ElementAtOrDefault
FirstOrDefault LastOrDefault SingleOrDefault
SequenceEqual GroupBy Join GroupJoin
DefaultIfEmpty Empty Repeat Range
Skip SkipWhile Take TakeWhile
Distinct Select SelectMany Where
Except Any All Contains
Intersect OrderBy ThenBy Reverse
Union OrderByDesc ThenByDesc
Set Operations
Aggregate Average Count LongCount
Max Min Sum Concat
AsEnumerable Cast ToArray ToDictionary
AsQueryable OfType ToList ToLookup
ElementAt First Last Single
ElementAtOrDefault
FirstOrDefault LastOrDefault SingleOrDefault
SequenceEqual GroupBy Join GroupJoin
DefaultIfEmpty Empty Repeat Range
Skip SkipWhile Take TakeWhile
Distinct Select SelectMany Where
Except Any All Contains
Intersect OrderBy ThenBy Reverse
Union OrderByDesc ThenByDesc
Sorting Data
Aggregate Average Count LongCount
Max Min Sum Concat
AsEnumerable Cast ToArray ToDictionary
AsQueryable OfType ToList ToLookup
ElementAt First Last Single
ElementAtOrDefault
FirstOrDefault LastOrDefault SingleOrDefault
SequenceEqual GroupBy Join GroupJoin
DefaultIfEmpty Empty Repeat Range
Skip SkipWhile Take TakeWhile
Distinct Select SelectMany Where
Except Any All Contains
Intersect OrderBy ThenBy Reverse
Union OrderByDesc ThenByDesc
Closer Look: Join & GroupJoin
This query expression:from x1 in e1
join x2 in e2 on k1 equals k2 …
Translates into:from * in ( e1 ) . Join( e2 , x1 => k1 , x2 => k2 , (x1 , x2) => new { x1 , x2 })…
Join parameters:o this outer – The first sequence to join.o inner – The sequence to join to the first sequence.o outerKeySelector – Select key from first sequence. o innerKeySelector – Select key from second sequence.o resultSelector – Select result from two matching elements.
Closer Look: Join & GroupJoin
Use default equality comparero EqualityComparer<TKey>.Default
Composite keyso Type with good Equals() & GetHashcode()o Anonymous types good choice – required by some providers
Custom IEqualityComparer<TKey>o No query syntaxo Not alone
Operators With Comparer Overload
Aggregate Average Count LongCount
Max Min Sum Concat
AsEnumerable Cast ToArray ToDictionary
AsQueryable OfType ToList ToLookup
ElementAt First Last Single
ElementAtOrDefault
FirstOrDefault LastOrDefault SingleOrDefault
SequenceEqual GroupBy Join GroupJoin
DefaultIfEmpty Empty Repeat Range
Skip SkipWhile Take TakeWhile
Distinct Select SelectMany Where
Except Any All Contains
Intersect OrderBy ThenBy Reverse
Union OrderByDesc ThenByDesc
Closer Look: Join, Group Join and Left Outer Join
Join (no into)o “Correlates the elements of two sequences based on matching
keys.”o Func<TOuter, TInner, TResult> resultSelector
Group Join (with into)o “Correlates the elements of two sequences based on equality
of keys and groups the results.”o Hierarchical: One on left to many on righto Func<TOuter, IEnumerable<TInner>, TResult>
resultSelector Left Out Join
o Use DefaultIfEmpty() on Group Join result:from x1 in e1
join x2 in e2 on k1 equals k2 into jfrom xj in j . DefaultIfEmpty()…
Closer Look: SelectMany
Used to translate multiple from clauseso “Projects each element of a sequence to an IEnumerable<T>
and flattens the resulting sequences into one sequence.” This query expression:from x1 in e1
from x2 in e2
… Translates into:from * in ( e1 ) . SelectMany(
x1 => e2 , ( x1 , x2 ) => new { x1 , x2 } )… What is * ?
o Transparent Identifier = Intermediate Step
LINQ to Objects
Implementation of all Standard Query Operators System.Linq.Enumerable
o Extension methods on IEnumerable<T> Use C# Iterators Extensively
o yield return LINQBridge – .NET 2.0
o LINQ to Objectso Generic delegates (Action, Func)o ExtensionAttributeo http://code.google.com/p/linqbridge/
Manner of Execution
Immediateo Performed at the point where the query is declared.o All operators returning a non-enumerable result.
Deferredo Performed at the point where result is used.
• foreach• Immediate Operator
o Result depends on data source at point of use Deferred Streaming
o Consumes source data on demand Deferred Non-Streaming
o Caches all source data to perform operation
Immediate Execution
Aggregate Average Count LongCount
Max Min Sum Concat
AsEnumerable Cast ToArray ToDictionary
AsQueryable OfType ToList ToLookup
ElementAt First Last Single
ElementAtOrDefault
FirstOrDefault LastOrDefault SingleOrDefault
SequenceEqual GroupBy Join GroupJoin
DefaultIfEmpty Empty Repeat Range
Skip SkipWhile Take TakeWhile
Distinct Select SelectMany Where
Except Any All Contains
Intersect OrderBy ThenBy Reverse
Union OrderByDesc ThenByDesc
Deferred Streaming Execution
Aggregate Average Count LongCount
Max Min Sum Concat
AsEnumerable Cast ToArray ToDictionary
AsQueryable OfType ToList ToLookup
ElementAt First Last Single
ElementAtOrDefault
FirstOrDefault LastOrDefault SingleOrDefault
SequenceEqual GroupBy Join GroupJoin
DefaultIfEmpty Empty Repeat Range
Skip SkipWhile Take TakeWhile
Distinct Select SelectMany Where
Except Any All Contains
Intersect OrderBy ThenBy Reverse
Union OrderByDesc ThenByDesc
Deferred Non-Streaming Execution
Aggregate Average Count LongCount
Max Min Sum Concat
AsEnumerable Cast ToArray ToDictionary
AsQueryable OfType ToList ToLookup
ElementAt First Last Single
ElementAtOrDefault
FirstOrDefault LastOrDefault SingleOrDefault
SequenceEqual GroupBy Join GroupJoin
DefaultIfEmpty Empty Repeat Range
Skip SkipWhile Take TakeWhile
Distinct Select SelectMany Where
Except Any All Contains
Intersect OrderBy ThenBy Reverse
Union OrderByDesc ThenByDesc
LINQ to * via Objects
LINQ to XMLo System.Xml.Linqo All selectors return IEnumerable<>
LINQ to DataSeto System.Data.DataSetExtensionso EnumerableRowCollection<DataRow>
AsEnumerable(this DataTable source) o EnumerableRowCollectionExtensions provides some
operators LINQ to Legacy
o Take advantage of Cast<T>() and OfType<T>()o Query expression support, just like foreach:
from MyType obj in MyArrayList…
Tour of LINQ to XML and DataSet Internals
LINQ to IQueryable
Expression Trees from Standard Query Operators System.Linq.Queryable
o Extension methods on IQueryable<T>
public interface IQueryable : IEnumerable{ Type ElementType { get; } Expression Expression { get; } IQueryProvider Provider { get; }}
public interface IQueryable<T> : IEnumerable<T>, IQueryable, IEnumerable{ }
IQueryProvider
“Only” responsibility is processing expression trees
public interface IQueryProvider{ IQueryable CreateQuery(Expression expression); IQueryable<TElement> CreateQuery<TElement>(Expression expression);
object Execute(Expression expression); TResult Execute<TResult>(Expression expression);}
Queryable Operators
public static TSource First<TSource>( this IQueryable<TSource> source){ return source.Provider.Execute<TSource>( Expression.Call(null, ((MethodInfo) MethodBase.GetCurrentMethod()) .MakeGenericMethod(new Type[]
{ typeof(TSource) }), new Expression[] { source.Expression } ) );}
Tour of LINQ to SQL Internals
Beyond?
IEnumerable and IQueryable are not special Methods can come from anywhere Parameter types are flexible
o As long as the compiler can infer everything…o Who ever said LINQ predicates need to be Boolean-value
d?
New provider modelso LINQ to Eventso Lazy LINQ (coming soon)
Resources
C# in Depth by Jon Skeet (Manning) Bart De Smet – Microsoft Language Geek
o http://community.bartdesmet.net/blogs/bart/
Meo http://solutionizing.net/o keith@ ---^o @dahlbyk