Reactive Streams 1.0.0 and Why You Should Care (webinar)
-
Upload
typesafeinc -
Category
Technology
-
view
4.281 -
download
0
Transcript of Reactive Streams 1.0.0 and Why You Should Care (webinar)
Reactive Streams 1.0.0 and why you should care
Dr. Roland Kuhn @rolandkuhn — Akka Tech Lead
Reactive Streams 1.0.0
Philosophical Background
3
“You cannot step twice into the same stream. For as you are stepping in, other waters are ever flowing on to you.”
— Heraclitus
What is a Stream?
• ephemeral flow of data
• possibly unbounded in size
• focused on describing transformation
• can be formed into processing networks
4
Reactive Traits
The Problem:
Getting Data across an Asynchronous Boundary
without running OutOfMemory
Enter Reactive Streams
7
“Reactive Streams is an initiative to provide a standard for asynchronous stream processing with non-blocking back pressure on the JVM.”
— reactive-streams.org
Participants
• Engineers from (among others): • Netflix
• Pivotal
• Red Hat
• Typesafe
• Individuals like Doug Lea and Todd Montgomery
8
Motivation
• all participants face the same basic problem
• all are building tools for their community
• a common solution benefits everybody
• interoperability to make best use of efforts
• proposal to include in JDK9 (j.u.c.Flow)
9
Recipe for Success
• minimal interfaces
• rigorous specification of semantics
• full TCK for verification of implementation
• complete freedom for many idiomatic APIs
10
Supply and Demand
• data items flow downstream
• demand flows upstream
• data items flow only when there is demand • recipient is in control of incoming data rate
• data in flight is bounded by signaled demand
11
Publisher Subscriber
data
demand
Dynamic Push–Pull
• “push” behavior when consumer is faster
• “pull” behavior when producer is faster
• switches automatically between these
• batching demand allows batching data
12
Publisher Subscriber
data
demand
Back-Pressure is Contagious
• C is slow
• B must slow down
• A must slow down
13
CA B
Back-Pressure can be Propagated
• TCP for example has it built-in
14
CA B
netw
ork
host
s
The Meat
15
trait Publisher[T] { def subscribe(sub: Subscriber[T]): Unit } trait Subscription { def request(n: Int): Unit def cancel(): Unit } trait Subscriber[T] { def onSubscribe(s: Subscription): Unit def onNext(e: T): Unit def onError(t: Throwable): Unit def onComplete(): Unit }
How does it Connect?
16
SubscriberPublisher
subscribe
onSubscribeSubscription
How does it Flow?
17
SubscriberPublisher
request
onNextElements
request
onNextElements
request
How does it Complete?
18
SubscriberPublisher
request
onNextElements
onComplete
How does it Fail?
19
SubscriberPublisher
request
onNextElements
onError☠
Reactive Streams
• asynchronous non-blocking data flow
• asynchronous non-blocking demand flow
• minimal coordination and contention
• message passing allows for distribution acrossapplications, nodes, CPUs, threads, actors, …
20
http://reactive-‐streams.org/
What is it good for?
Streams are Ubiquitous
• Streaming SIMD Extensions (SSE) — Intel, 1999
• ingesting, transforming and emitting data
• requests & responses flowing through a system
• streams are graphical and intuitive
22
Moving Bulk Data over the Network
• content streaming (e.g. video or audio)
• data storage / backup
• data replication between data centers
23
Mobile Devices
• streams of updates from the server
• streams of user commands from the client
• bad connection quality—need for back-pressure
24
Internet of Things
• ingesting large numbers of low-rate streams
• conflating or extrapolating data
• aggregating many streams
• streaming towards the fleet of devices
25
Realtime Data Analysis
• real-time business intelligence
• complex event processing
• temporal correlations (e.g. credit card fraud)
26
The Ecosystem
RxJava
• «An API for asynchronous programming with observable streams» — reactivex.io
• inspiration for Reactive Streams interfaces
• RxJavaReactiveStreams provides RS compliant wrapper for Observable
28
io.reactivex:rxjava-reactive-streams:1.0.0
Project Reactor
• «a foundational library for building reactive fast-data applications on the JVM» — projectreactor.io
• Stream directly implements Publisher and offers a rich set of transformations
29
io.projectreactor:reactor-stream:2.0.3.RELEASE
Akka Streams
• compose any kind of stream processing topology from immutable and reusable blueprints
• creates RS compliant Publisher/Subscriber when materializing a flow graph into Actors
• high-level flow graph DSL
30
com.typesafe.akka:akka-stream-experimental:1.0-RC3
ratpack
• «Ratpack is a set of Java libraries that facilitate fast, efficient, evolvable and well tested HTTP applications.» — ratpack.io
• accepts Publishers of streams to be sent to the client • chunked responses, server-sent events, websockets
31
io.ratpack:ratpack-core:0.9.17
vert.x 3.0
• «Vert.x is a lightweight, high performance application platform for the JVM that's designed for modern mobile, web, and enterprise applications.» — vertx.io
• Reactive Streams Integration Module(https://github.com/vert-x3/vertx-reactive-streams)
32
io.vertx:vertx-reactive-streams:3.0.0-milestone6
Slick 3.0
• «Functional Relational Mapping for Scala» — slick.typesafe.com
• database result sets can be streamed, returning a Publisher instead of a collection
• no streaming inserts yet
33
com.typesafe.slick:slick_2.11:3.0.0
Kafka
• «Apache Kafka is publish-subscribe messaging rethought as a distributed commit log.» — kafka.apache.org
• Reactive Kafka (by Softwaremill):consumer/producer are RS Subscriber/Publisher
34
com.softwaremill:reactive-kafka_2.11:0.6.0
AMQP
• «To become the standard protocol for interoperability between all messaging middleware» — amqp.org
• https://github.com/ScalaConsultants/reactive-rabbit
• fully asynchronous API
• consumer/publisher are RS Subscriber/Publisher
35
io.scalac:reactive-rabbit_2.11:1.0.0
MongoDB
• «The official MongoDB Reactive Streams Java Driver, providing asynchronous stream processing with non-blocking back pressure for MongoDB.» —http://mongodb.github.io/mongo-java-driver-reactivestreams/
• all queries return Publishers
• no streaming inserts yet
36
org.mongodb:mongodb-driver-reactivestreams:1.0.1
… and more are coming (TBC)
• Cassandra
• Spark
• Riak
37
Reactive Streams in the Typesafe Reactive Platform
39
O/S-level network stack
Java NIO (JDK)
Akka IO
Akka HTTP Core
Akka HTTP
application level
Akka Streams Database Driver
Slick 3.0
Play Framework ReactiveStreamsIntegration: preview available
Streaming Database Results to the Client
40
import DefaultJsonProtocol._implicit val denormOrderFormat = jsonFormat5(DenormalizedOrder.apply)
val db = Database.forConfig("reportingDB")
private def getFromDb(userId: Int): Publisher[DenormalizedOrder] = db.stream(denormalizedOrders.filter(_.userId === userId).result)
Http().bindAndHandle( (get & path("orders" / IntNumber)) { userId => val pub = Source(getFromDb(userId)) .transform(() => new ToJsonArray) complete(HttpEntity.Chunked.fromData(`application/json`, pub)) }, "localhost", 8080)
©Typesafe 2015 – All Rights Reserved