Post on 11-Sep-2019
Cloud-scale Event Processing
using Reactive Extensions (Rx)
Johan Laanstra & Tom Verhoeff
Events are not first-class objects
[RunOnClient]public event EventHandler<MouseEventArgs> MouseMoved;
// Runs in cloudpublic void CloudCanvas(){
MouseMoved += (o, e) => { /* do stuff */ };}
An electric eel…
delegates
Action a = new Action(Foo); // explicit creation of delegate instanceAction b = Foo; // method group conversionAction c = () => { … }; // creates anonymous method
void Foo() { … }
event Action Bar // metadata that refers to …{
add { … } // add accessorremove { … } // remove accessor
}
http://en.wikipedia.org/wiki/First-class_citizen
interface IObservable<out T>{
IDisposable Subscribe(IObserver<T> observer);}
interface IObserver<in T>{
void OnNext(T value);void OnError(Exception error);void OnCompleted();
}
Notification grammar
OnNext* (OnError | OnCompleted)?
“Gang of four” bookAddison-Wesley
Composition
Composition
interface IScheduler{
IDisposable Schedule(Action work);…
}
static class Observable{
static IObservable<T> Where<T>(this IObservable<T> source, Func<T, bool> f);static IObservable<R> Select<T, R>(this IObservable<T> source, Func<T, R> p);…
}
Function compositionwww.Wikipedia.org
Function composition
www.Wikipedia.org
public static class Observable{
public static IObservable<T> Merge<T>(this IObservable<T> xs, IObservable<T> ys){
return Create<T>(observer =>{
var gate = new object();
return new CompositeDisposable{
xs.Subscribe(x => { lock (gate) { observer.OnNext(x); } }, …),ys.Subscribe(y => { lock (gate) { observer.OnNext(y); } }, …),
};});
}}
First-class: can build
extension methods
Composition of
resource management
Await Stephen KleeneKleene star (closure)
www.Wikipedia.org
Func<T> f
var x = wait f();
IEnumerable<T> xs
foreach (var x in xs) {f(x);
}
Task<T> t
var x = await t;
IObservable<T> xs
xs.Subscribe(x => {f(x);
});
On
eM
an
y
Synchronous Asynchronous
Code-as-data
Alan KayHomoiconicity, 1969
www.Wikipedia.org
IEnumerable<T> xs
from x in xs where f(x) …
IQueryable<T> xs
from x in xs where f(x) …
IObservable<T> xs
from x in xs where f(x) …
IQbservable<T> xs
from x in xs where f(x) …
Co
de
Data
Pull Push
Events
scalable abstraction
Satya Nadella“Cloud-first, mobile-first”
www.businessinsider.com
Cortana
Cloud and device
from w in weatherwhere w.City == “Seattle”select w
flights.Where(f => f.Code == “BA49”).DelaySubscription(departure – 24 * H).TakeUntil(arrival + 24 * H).Select(f => f.Status)
userLocation.Sample(1 * M).DistinctUntilChanged().Select(here =>
traffic(here, meetingLocation).TakeUntil(meeting – minimumTime).Select(t => Timer(meeting – t.EstimatedTime)).StartWith(Timer(meeting – estimatedTime)).Switch())
.Switch()
.Select(_ => “Leave now for ” + subject)
.DelaySubscription(meeting – estimatedTime)
Temporal query
operators
Device-side
event stream
// Insert cloud-side observable sequence// of time-to-leave timer here
Cloud-side
event stream
Higher-order
query operators
Remember
tier-splitting?
IReactiveProcessing
Bing cloud
IRP
Partner1 cloudIRP
Partner2 cloudIRP
Tablet
IRP
Phone
IRP
Node
IRP
Node
IRP Node
IRP
distributed
recover
density
reliability
Scalable
checkpointing
Acknowledge / replay
Unit of pausing
Proxies
Definitions
Metadata
Hyper-cube
Sync Async
Intr
insi
cEx
trin
sic
Higher dimensionsCalabi-Yau manifold
www.Wikipedia.org
Proxies
Definitions
Metadata
Hyper-cube
Synchronous versus asynchronous
Extrinsic versus intrinsic identities
Code-as-data versus code
Reliable versus non-reliable
Etc.
Higher dimensionsCalabi-Yau manifold
www.Wikipedia.org
AsyncEx
trin
sic
E.g. this is the
client-side API
Need string theory
context
proxies
var conn = new ReactiveServiceConnection(endpoint);var ctx = new ClientContext(conn);
var traffic = ctx.GetObservable<TrafficInfo>(trafficUri);var http = ctx.GetObserver<HttpData>(httpUri);
Explicit identifiers
provided for artifacts
extrinsic identifiers
Async
var traffic = ctx.GetObservable<TrafficInfo>(trafficUri);var http = ctx.GetObserver<HttpData>(httpUri);
var subscription = await traffic.Where(t => t.Road == “I-90”).Select(t => new HttpData { Uri = myService,
Body = t.ToString() }).SubscribeAsync(http, mySubUri);
Explicit identifiers
provided for artifacts
SubscribeAsync expression tree
serialized
IRP
var traffic = ctx.GetObservable<TrafficInfo>(trafficUri);var http = ctx.GetObserver<HttpData>(httpUri);
var subscription = await traffic.Where(t => t.Road == “I-90”).Select(t => new HttpData { Uri = …, Body = … }).SubscribeAsync(http, mySubUri);
Invoke
rx://subscribe Invoke
rx://select Invoke
rx://where trafficUri
httpUri
λt => new { bing://http/uri = …,
bing://http/body = … }
λ t => t.Road == “I-90”
Unbound parameters
processed by binder
Structural typing to
reduce coupling
onion
GitHub
bartde@microsoft.com
Service
Host
Data model
Query engine
Operators
Your feedback is important!
Scan the QR Code and let us know via the TechDays App.
Laat ons weten wat u van de sessie vindt via de TechDays App!
Scan de QR Code.
Bent u al lid van de Microsoft Virtual Academy?! Op MVA kunt u altijd iets nieuws leren over de laatste technologie van Microsoft. Meld u vandaag aan op de MVA Stand. MVA biedt 7/24 gratis online training on-demand voor IT-Professionals en Ontwikkelaars.