Reactive Streams in Practice
-
Upload
joost-de-vries -
Category
Technology
-
view
30 -
download
1
Transcript of Reactive Streams in Practice
joost@de-vries․namejoost-de-vries@jouke
REACTIVESTREAMSINPRACTICE
ReactiveStreamsmeetupUtrecht29-03-2017
THREEACTS1. Alittletaste:somepracticalcode
2. whatisstreamprocessingandwhatcanwedowithit
3. reactivestreamsLet'stalkaboutourexperi
enceswithstreams
LET'SSTARTWITHSOMECODEWewanttopollsomemarketplaceevery15minutesforthe
pricesofourproducts.Andwedon'twanttobesuspended,soweonlydo4
requestsatatime.'
LET'SSTARTWITHSOMECODEvaljobParallellism:Int=4
valtickStream=Spout.tick((),jobInterval)
defgetProductIds():List[String]=???//getthecurrentproductids
defpollPrice(productId:String):Future[Long]=???//invokethemarketplaceapi
valprices=tickStream//tickevery15mins.flatMap(tick=>getProductIds())//productidsstartstreamingevery15mins.mapAsync(jobParallellism)(productId=>pollPrice(productId))//pollmax4productidsinparallel.foreach(price=>println(s"priceis$price"))//printthepricesastheycomein
ThishappenstobeScala
LIKEACROSSBETWEENITERATORANDCOMPLETABLEFUTURE
synchronous asynchronous
one AgetA() ⇒ CompletableFuture[A]getA()
many Iterator[A] ⇒ Observable[A]
UsingJavaandRxJava
LIKEACROSSBETWEENITERATORANDCOMPLETABLEFUTURE
synchronous asynchronous
one AgetA() ⇒ CompletableFuture[A]getA()
many Iterator[A] ⇒ Observable[A]
Wecallfordatavswegetcalledwithdata
pullvspush
pollingvsreactive
Sohowisastreamdifferentfromacollection?
HMM..
Astreamispotentiallyinfinite
Astreamisspreadoutintimeinsteadofmemory
Astreamcanbenonrepeatable
CREATEASTREAM
constsource=Rx.Observable.create((observer)=>{observer.onNext(3)observer.onNext(2)observer.onNext(1)//0ormorevaluesobserver.onCompleted()//potentiallyendorerror})
3,2,1.
(element)*(complete|error)
AFAILINGSTREAM
constsource=Rx.Observable.create((observer)=>{observer.onNext(3)observer.onNext(2)observer.onError(newError("boom"))})
3,2,1x
WHATCANWEDOWITHSTREAMS?
IoT:dealingwithsensormeasurements
Systemintegration:pumpingdatabetweenservers
Userinterfacecode
Serverpush
(Near)realtime
Fastdata
Immediateresultsastheinputsarecomingin
DETERMINEHIGHWATERMARKFORSENSORMEASUREMENTS
caseclassMeasurement(rotorSpeed:Int,windSpeed:Int,timestamp:Instant)
defmeasureTurbine():Measurement=Measurement(rotorSpeed=Random.nextInt(),windSpeed=tickStream.map(tick=>measureTurbine()).scan(0){case(currentMaxRotorspeed,measurement)=>if(measurement.rotorSpeed>currentMaxRotorspeed)measurement.rotorSpeedelsecurrentMaxRotorspeed}.deduplicate//oftencalled'distinct'.onElement(max=>println(s"maxrotorspeedis$max")).drainToBlackHole()
USERINTERFACELOGIC:NETFLIXfunctionplay(movieId,cancelButton,callback){varmovieTicket,playError,tryFinish=()=>{if(playError){callback(null,playError);}elseif(movieTicket&&player.initialized){callback(null,ticket);}};cancelButton.addEventListener(“click”,()=>{playError=“cancel”;});if(!player.initialized){player.init((error)=>{playError=error;tryFinish();}}authorizeMovie(movieId,(error,ticket)=>{playError=error;movieTicket=ticket;tryFinish();});});
SeeJafarHusainstalk
USERINTERFACELOGIC:NETFLIXvarauthorizations=player.init().map(()=>playAttempts.map(movieId=>player.authorize(movieId).catch(e=>Observable.empty).takeUntil(cancels)).concatAll())).concatAll();
authorizations.forEach(license=>player.play(license),error=>showDialog(“Sorry,can’tplayrightnow.”));
SeeJafarHusainstalk
CLUSTEREDSTREAMSdefrankLangsReduceByKey(langs:List[String],articles:DStream[WikipediaArticle]):List[(Stringarticles.flatMap{art=>langs.map{lang=>if(art.text.split("").contains(lang))(lang,1)else(lang,0)}}.reduceByKey{case(acc,i)=>acc+i}.collect().toList.sortBy{case(lang,i)=>i}.reverse}
CLUSTEREDSTREAMSdefrankLangsReduceByKey(langs:List[String],articles:DStream[WikipediaArticle]):List[(Stringarticles.flatMap{art=>langs.map{lang=>if(art.text.split("").contains(lang))(lang,1)else(lang,0)}}.reduceByKey{case(acc,i)=>acc+i}.collect().toList//RDDcode!collectisn'tavailableonDStream.sortBy{case(lang,i)=>i}.reverse}
STARTWORKINGWHENDATAAVAILABLE
Pushinsteadofpull
Leastlatency
Processstuffwithoutloadingitallinmemory
Reverseofquery/polling
NONBLOCKINGBOUNDEDUSEOFRESOURCES
limitedthreads
boundedmemoryusage(seebackpressure)
importantformicroservices/IoT-mobilelongrunningreqs
CLEANPROGRAMMINGMODEL
Concurrencymodel
(flat)map:collection,parallelcollection,clusteredcollection,stream
Functionalprogramming
UILOGICINTYPESCRIPTRXJSconstincassos:Observable<Pagination<IncassoListResult>>=this.queryObservable.filter((incassoQuery)=>{if(incassoQuery.search)returnincassoQuery.search.length>=3elsereturntrue}).switchMap((incassoQuery)=>{returnthis.incassoService.getIncassos(incassoQuery,PaginationComponent.calculatePaginationRange(incassoQuery.pageNr,incassoQuery.pageSize))}).share()
Moveassignmentsandsubjects
totheoutsideofyourcode
Observablein,Observableout
Usepuretransformations
REACTIVESTREAMSInteroperabilitystandard
Oracle
Lightbend
Pivotal
Netflix
Redhat
Supportsbackpressure
BACKPRESSURE
Doesn'tmakesenseforsensordata,mouseclicks,.
..
Butifyoucan'tloseevents...
Downstreamcommunicatesdemand
⑥RESOURCES
AsyncJavaScriptatNetflix,"JafarHusainhttps://www.infoq.com/presentations/netflix-rx-extensions
ErikMeijeronobservables
http://reactivex.io/
TalksbyStephaneMaldini
etc-