Akka persistence == event sourcing in 30 minutes
137
Akka persistence (message sourcing in 30 minutes) Konrad 'ktoso' Malawski Scalar 2014 @ Warsaw, PL
-
Upload
konrad-malawski -
Category
Technology
-
view
5.627 -
download
1
description
Akka 2.3 introduces akka-persistence, a wonderful way of implementing event-sourced applications. Let's give it a shot and see how DDD and Akka are a match made in heaven :-)
Transcript of Akka persistence == event sourcing in 30 minutes
Akka persistence (message sourcing in 30 minutes)
Konrad 'ktoso' Malawski Scalar 2014 @ Warsaw, PL
Konrad `@ktosopl` Malawski
typesafe.com geecon.org
Java.pl / KrakowScala.pl sckrk.com / meetup.com/Paper-Cup @ London
GDGKrakow.pl meetup.com/Lambda-Lounge-Krakow
Konrad `@ktosopl` Malawski
typesafe.com geecon.org
Java.pl / KrakowScala.pl sckrk.com / meetup.com/Paper-Cup @ London
GDGKrakow.pl meetup.com/Lambda-Lounge-Krakow
Konrad `@ktosopl` Malawski
typesafe.com geecon.org
Java.pl / KrakowScala.pl sckrk.com / meetup.com/Paper-Cup @ London
GDGKrakow.pl meetup.com/Lambda-Lounge-Krakow
Konrad `@ktosopl` Malawski
typesafe.com geecon.org
Java.pl / KrakowScala.pl sckrk.com / meetup.com/Paper-Cup @ London
GDGKrakow.pl meetup.com/Lambda-Lounge-Krakow
Konrad `@ktosopl` Malawski
typesafe.com geecon.org
Java.pl / KrakowScala.pl sckrk.com / meetup.com/Paper-Cup @ London
GDGKrakow.pl meetup.com/Lambda-Lounge-Krakow
Konrad `@ktosopl` Malawski
typesafe.com geecon.org
Java.pl / KrakowScala.pl sckrk.com / meetup.com/Paper-Cup @ London
GDGKrakow.pl meetup.com/Lambda-Lounge-Krakow
hAkker @
mainly by Martin Krasser !!(as contractor for Typesafe) !inspired by:
akka-persistence
https://github.com/krasserm
https://github.com/eligosource/eventsourced
dependencies
libraryDependencies ++= Seq(! "com.typesafe.akka" %% “akka-actor" % "2.3.0",! "com.typesafe.akka" %% "akka-persistence-experimental" % "2.3.0"!)
Show of hands!
Show of hands!
Show of hands!
Show of hands!
sourcing styles
Command Sourcing Event Sourcing
msg: DoThing
msg persisted before receive
imperative, “do the thing”
business logic change, can be reflected in reaction
Processor
sourcing styles
Command Sourcing Event Sourcing
msg: DoThing msg: ThingDone
msg persisted before receive commands converted to events, must be manually persisted
imperative, “do the thing” past tense, “happened”
business logic change, can be reflected in reaction
business logic change, won’t change previous events
Processor EventsourcedProcessor
Plain Actors
count: 0 !
!
Actor
count: 0 !
!
ActorAn Actor that keeps count of messages it processed
count: 0 !
!
ActorAn Actor that keeps count of messages it processed
count: 0 !
!
ActorAn Actor that keeps count of messages it processed
Let’s send 2 messages to it
count: 0 !
!
ActorAn Actor that keeps count of messages it processed
Let’s send 2 messages to it(it’s “commands”)
Actor
!!class Counter extends Actor {! var count = 0! def receive = {! case _ => count += 1! }!}
count: 0 !
!
Actor
count: 0 !
!
Actor
count: 1 !
!
Actor
count: 1 !
!
Actor
crash!
Actor
crash!
Actor
restart
count: 0 !
!
Actor
restart
count: 0 !
!
Actor
restarted
count: 1 !
!
Actor
restarted
count: 1 !
!
Actor
restarted
count: 1 !
!wrong!
expected count == 2!
Actor
restarted
Processor
var count = 0 !
def processorId = “a” !
Journal (DB)
!
!
!
Processor
Journal (DB)
!
!
!
Processor
var count = 0 !
def processorId = “a” !
Journal (DB)
!
!
!
Processor
var count = 0 !
def processorId = “a” !
Journal (DB)
!
!
!
Processor
var count = 0 !
def processorId = “a” !
Journal (DB)
!
!
!
Processor
var count = 1 !
def processorId = “a” !
Journal (DB)
!
!
!
Processor
var count = 1 !
def processorId = “a” !
Journal (DB)
!
!
!
Processor
var count = 1 !
def processorId = “a” !
crash!
Journal (DB)
!
!
!
Processor
Journal (DB)
!
!
!
Processor
restart
Journal (DB)
!
!
!
Processor
var count = 0 !
def processorId = “a” !
restart
Journal (DB)
!
!
!
Processor
var count = 0 !
def processorId = “a” !
replay!
restart
Journal (DB)
!
!
!
Processor
var count = 0 !
def processorId = “a” !
Journal (DB)
!
!
!
Processor
var count = 0 !
def processorId = “a” !
replay!
Journal (DB)
!
!
!
Processor
var count = 1 !
def processorId = “a” !
Journal (DB)
!
!
!
Processor
var count = 1 !
def processorId = “a” !
replay!
Journal (DB)
!
!
!
Processor
var count = 1 !
def processorId = “a” !
Journal (DB)
!
!
!
Processor
var count = 2 !
def processorId = “a” !
Journal (DB)
!
!
!
Processor
var count = 2 !
def processorId = “a” !
yay!
Processorimport akka.persistence._!!class CounterProcessor extends Processor {! var count = 0! override val processorId = "counter"!! def receive = {! case Persistent(payload, seqNr) =>! // payload already persisted! count += 1! }!}
Processorimport akka.persistence._!!class CounterProcessor extends Processor {! var count = 0! override val processorId = "counter"!! def receive = {! case Persistent(payload, seqNr) =>! // payload already persisted! count += 1! }!}
counter ! Persistent(payload)
Processorimport akka.persistence._!!class CounterProcessor extends Processor {! var count = 0! override val processorId = "counter"!! def receive = {! case Persistent(payload, seqNr) =>! // payload already persisted! count += 1! }!}
counter ! Persistent(payload)
Processorimport akka.persistence._!!class CounterProcessor extends Processor {! var count = 0! override val processorId = "counter"!! def receive = {! case Persistent(payload, seqNr) =>! // payload already persisted! count += 1! }!}
counter ! Persistent(payload)
Processorimport akka.persistence._!!class CounterProcessor extends Processor {! var count = 0! override val processorId = "counter"!! def receive = {! case Persistent(payload, seqNr) =>! // payload already persisted! count += 1! }!}
counter ! Persistent(payload)
is already persisted!
Processorimport akka.persistence._!!class CounterProcessor extends Processor {! var count = 0! override val processorId = "counter"!! def receive = {! case Persistent(payload, seqNr) =>! // payload already persisted! count += 1! }!}
counter ! Persistent(payload)
sequenceNr (generated by akka)
is already persisted!
Processorimport akka.persistence._!!class CounterProcessor extends Processor {! var count = 0! override val processorId = "counter"!! def receive = {! case notPersisted =>! // will not replay this msg!! count += 1! }!}
counter ! payload
Processorimport akka.persistence._!!class CounterProcessor extends Processor {! var count = 0! override val processorId = "counter"!! def receive = {! case notPersisted =>! // will not replay this msg!! count += 1! }!}
counter ! payload
won’t persist
Processorimport akka.persistence._!!class CounterProcessor extends Processor {! var count = 0! override val processorId = "counter"!! def receive = {! case notPersisted =>! // will not replay this msg!! count += 1! }!}
counter ! payload
won’t persist
won’t replay
Processorimport akka.persistence._!!class CounterProcessor extends Processor {! var count = 0! override val processorId = "counter"!! def receive = {! case Persistent(payload, seqNr) =>! // payload already persisted! count += 1!! case notPersistentMsg =>! // msg not persisted - like in normal Actor! count += 1! }!}
ProcessorUpsides
• Persistent Command Sourcing “out of the box”
• Pretty simple, persist handled for you
• Once you get the msg, it’s persisted
• Pluggable Journals (HBase, Cassandra, Mongo, …)
• Can replay to given seqNr (post-mortem etc!)
ProcessorDownsides
• Exposes Persistent() to Actors who talk to you
• No room for validation before persisting
• There’s one Model, we act on the incoming msg
• Lower throughput than plain Actor (limited by DB)
EventsourcedProcessor
EventsourcedProcessor (longer name == more flexibility);-)
super quick domain modeling!
super quick domain modeling!
sealed trait Command!case class ManyCommand(nums: List[Int]) extends Command
Commands - input from user, “send emails”, not persisted
super quick domain modeling!
sealed trait Command!case class ManyCommand(nums: List[Int]) extends Command
Commands - input from user, “send emails”, not persisted
sealed trait Event!case class AddOneEvent(num: Int) extends Event!
Events - business events emitted by the processor, persisted
super quick domain modeling!
sealed trait Command!case class ManyCommand(nums: List[Int]) extends Command
Commands - input from user, “send emails”, not persisted
case class State(count: Int) {! def updated(more: Int) = State(count + more)!}
State - internal actor state, kept in var
sealed trait Event!case class AddOneEvent(num: Int) extends Event!
Events - business events emitted by the processor, persisted
var count = 0 !
def processorId = “a” !
EventsourcedProcessor
!
!
Journal
C1
var count = 0 !
def processorId = “a” !
EventsourcedProcessor
Command
!
!
Journal
C1
var count = 0 !
def processorId = “a” !
EventsourcedProcessor
!
!
Journal
C1
var count = 0 !
def processorId = “a” !
EventsourcedProcessor
!
!
Journal
C1
Generate Events
C1
var count = 0 !
def processorId = “a” !
EventsourcedProcessor
!
!
Journal
E1
C1
var count = 0 !
def processorId = “a” !
EventsourcedProcessor
!
!
Journal
E1
Event
C1
var count = 0 !
def processorId = “a” !
EventsourcedProcessor
!
!
Journal
E1
C1
var count = 0 !
def processorId = “a” !
EventsourcedProcessor
!
!
Journal
E1
ACK “persisted”
C1
var count = 1 !
def processorId = “a” !
EventsourcedProcessor
!
!
Journal
E1
E1
C1
var count = 1 !
def processorId = “a” !
EventsourcedProcessor
!
!
Journal
E1
E1
“Apply” event
EventsourcedProcessor
class MultiCounter extends EventsourcedProcessor {!! var state = State(count = 0)!! def updateState(e: Event): State = {! case event: AddOneEvent => state.updated(event.num)! }! ! // API:!! def receiveCommand = ??? // TODO!! def receiveRecover = ??? // TODO!!}!
EventsourcedProcessor
def receiveCommand = {! case ManyCommand(many) =>! for (event <- many map AddOneEvent)! persist(event) { state = updateState(_) }!! case command: Command =>! persist(AddOneEvent(command)) { state = updateState(_) }!}
EventsourcedProcessor
def receiveCommand = {! case ManyCommand(many) =>! for (event <- many map AddOneEvent)! persist(event) { state = updateState(_) }!! case command: Command =>! persist(AddOneEvent(command)) { state = updateState(_) }!}
EventsourcedProcessor
def receiveCommand = {! case ManyCommand(many) =>! for (event <- many map AddOneEvent)! persist(event) { state = updateState(_) }!! case command: Command =>! persist(AddOneEvent(command)) { state = updateState(_) }!}
EventsourcedProcessor
def receiveCommand = {! case ManyCommand(many) =>! for (event <- many map AddOneEvent)! persist(event) { state = updateState(_) }!! case command: Command =>! persist(AddOneEvent(command)) { state = updateState(_) }!}
async persist that event
EventsourcedProcessor
def receiveCommand = {! case ManyCommand(many) =>! for (event <- many map AddOneEvent)! persist(event) { state = updateState(_) }!! case command: Command =>! persist(AddOneEvent(command)) { state = updateState(_) }!}
async persist that event
async called after successful persist
EventsourcedProcessor
def receiveCommand = {! case ManyCommand(many) =>! for (event <- many map AddOneEvent)! persist(event) { state = updateState(_) }!! case command: Command =>! persist(AddOneEvent(command)) { state = updateState(_) }!}
async persist that event
async called after successful persist
It is guaranteed that no new commands will be received by a processor between a call to `persist` and the execution of its `handler`.
EventsourcedProcessor
def receiveCommand = {! case ManyCommand(many) =>! for (event <- many map AddOneEvent)! persist(event) { state = updateState(_) }!! case command: Command =>! persist(AddOneEvent(command)) { state = updateState(_) }!}
EventsourcedProcessorcase ManyCommand(many) =>! for (event <- many map AddOneEvent)! persist(event) { state = updateState(_) }
EventsourcedProcessorcase ManyCommand(many) =>! for (event <- many map AddOneEvent)! persist(event) { state = updateState(_) }
“style fix”
EventsourcedProcessorcase ManyCommand(many) =>! for (event <- many map AddOneEvent)! persist(event) { state = updateState(_) }
case ManyCommand(many) =>! persist(many map AddOneEvent) { state = updateState(_) }
“style fix”
Eventsourced, recovery
/** MUST NOT SIDE-EFFECT! */!def receiveRecover = {! case replayedEvent: Event => ! updateState(replayedEvent)!}
Eventsourced, snapshots
Eventsourced, snapshots/** MUST NOT SIDE-EFFECT! */!def receiveRecover = {! case SnapshotOffer(meta, snapshot: State) => ! this.state = state!! case replayedEvent: Event => ! updateState(replayedEvent)!}
Eventsourced, snapshots/** MUST NOT SIDE-EFFECT! */!def receiveRecover = {! case SnapshotOffer(meta, snapshot: State) => ! this.state = state!! case replayedEvent: Event => ! updateState(replayedEvent)!}
Eventsourced, snapshots/** MUST NOT SIDE-EFFECT! */!def receiveRecover = {! case SnapshotOffer(meta, snapshot: State) => ! this.state = state!! case replayedEvent: Event => ! updateState(replayedEvent)!}
snapshot!? how?
Eventsourced, snapshots
def receiveCommand = {! case command: Command =>! saveSnapshot(state) // async!!}
/** MUST NOT SIDE-EFFECT! */!def receiveRecover = {! case SnapshotOffer(meta, snapshot: State) => ! this.state = state!! case replayedEvent: Event => ! updateState(replayedEvent)!}
snapshot!? how?
Snapshots
Snapshots (in SnapshotStore)
…sum of states…
Snapshots
!
!
Journal
E1 E2 E3 E4
E5 E6 E7 E8
state until [E8]
Snapshots
S8!
!
Journal
E1 E2 E3 E4
E5 E6 E7 E8
state until [E8]
Snapshots
S8
!
!
Snapshot Store
snapshot!
!
!
Journal
E1 E2 E3 E4
E5 E6 E7 E8
state until [E8]
Snapshots
S8
!
!
Snapshot Store
!
!
Journal
E1 E2 E3 E4
E5 E6 E7 E8
S8
state until [E8]
Snapshots
S8
!
!
Snapshot Store
!
!
Journal
E1 E2 E3 E4
E5 E6 E7 E8
S8
crash!
Snapshots
!
!
Snapshot Store
!
!
Journal
E1 E2 E3 E4
E5 E6 E7 E8
S8
crash!
“bring me up-to-date!”
Snapshots
!
!
Snapshot Store
!
!
Journal
E1 E2 E3 E4
E5 E6 E7 E8
S8
restart!
“bring me up-to-date!”
Snapshots
!
!
Snapshot Store
!
!
Journal
E1 E2 E3 E4
E5 E6 E7 E8
S8
restart!replay!
“bring me up-to-date!”
Snapshots
!
!
Snapshot Store
S8
restart!replay!
!
!
Journal
E1 E2 E3 E4
E5 E6 E7 E8
“bring me up-to-date!”
Snapshots
!
!
Snapshot Store
S8
restart!replay!
!
!
Journal
E1 E2 E3 E4
E5 E6 E7 E8
“bring me up-to-date!”
Snapshots
!
!
Snapshot Store
S8
restart!replay!
S8!
!
Journal
E1 E2 E3 E4
E5 E6 E7 E8
state until [E8]
Snapshots
!
!
Snapshot Store
S8
restart!replay!
S8!
!
Journal
E1 E2 E3 E4
E5 E6 E7 E8
state until [E8]
Snapshots
!
!
Snapshot Store
S8
S8!
!
Journal
E1 E2 E3 E4
E5 E6 E7 E8
state until [E8]
Snapshots
!
!
Snapshot Store
S8
S8!
!
Journal
E1 E2 E3 E4
E5 E6 E7 E8
We could delete these!
trait MySummer extends Processor {! var nums: List[Int]! var total: Int! ! def receive = {! case "snap" => saveSnapshot(total)! case SaveSnapshotSuccess(metadata) => // ...! case SaveSnapshotFailure(metadata, reason) => // ...! }!}!
Snapshots, save
trait MySummer extends Processor {! var nums: List[Int]! var total: Int! ! def receive = {! case "snap" => saveSnapshot(total)! case SaveSnapshotSuccess(metadata) => // ...! case SaveSnapshotFailure(metadata, reason) => // ...! }!}!
Snapshots, save
Async!
trait MySummer extends Processor {! var nums: List[Int]! var total: Int! ! def receive = {! case "snap" => saveSnapshot(total)! case SaveSnapshotSuccess(metadata) => // ...! case SaveSnapshotFailure(metadata, reason) => // ...! }!}!
Snapshots, success
trait MySummer extends Processor {! var nums: List[Int]! var total: Int! ! def receive = {! case "snap" => saveSnapshot(total)! case SaveSnapshotSuccess(metadata) => // ...! case SaveSnapshotFailure(metadata, reason) => // ...! }!}!
Snapshots, success
final case class SnapshotMetadata(! processorId: String, sequenceNr: Long, ! timestamp: Long)
trait MySummer extends Processor {! var nums: List[Int]! var total: Int! ! def receive = {! case "snap" => saveSnapshot(total)! case SaveSnapshotSuccess(metadata) => // ...! case SaveSnapshotFailure(metadata, reason) => // ...! }!}!
Snapshots, success
Snapshot Recovery
class Counter extends Processor {! var total = 0! ! def receive = {! case SnapshotOffer(metadata, snap: Int) => ! total = snap!! case Persistent(payload, sequenceNr) => // ...! }!}
SnapshotsUpsides
• Simple!
• Faster recovery (!)
• Allows to delete “older” events
• “known state at point in time”
SnapshotsDownsides
• More logic to write
• Maybe not needed if events replay “fast enough”
• Possibly “yet another database” to pick
• snapshots are different than events, may be big!
Views
Journal (DB)
!
!
!
Views!
Processor !
def processorId = “a” !
Journal (DB)
!
!
!
Views!
Processor !
def processorId = “a” !
!View
!def processorId = “a”
!!!
Journal (DB)
!
!
!
Views!
Processor !
def processorId = “a” !
!View
!def processorId = “a”
!!!
pooling
Journal (DB)
!
!
!
Views!
Processor !
def processorId = “a” !
!View
!def processorId = “a”
!!!
pooling
!View
!def processorId = “a”
!!!
pooling
Journal (DB)
!
!
!
Views!
Processor !
def processorId = “a” !
!View
!def processorId = “a”
!!!
pooling
!View
!def processorId = “a”
!!!
pooling
different ActorPath, same processorId
View
class DoublingCounterProcessor extends View {! var state = 0! override val processorId = "counter"!! def receive = {! case Persistent(payload, seqNr) =>! // “state += 2 * payload” !! }!}
Akka Persistence Plugins
Akka Persistence PluginsPlugins are Community maintained!
(“not sure how production ready”)
http://akka.io/community/#journal_plugins
Akka Persistence PluginsPlugins are Community maintained!
(“not sure how production ready”)
http://akka.io/community/#journal_plugins
• Journals - Cassandra, HBase, Mongo …
Akka Persistence PluginsPlugins are Community maintained!
(“not sure how production ready”)
http://akka.io/community/#journal_plugins
• Journals - Cassandra, HBase, Mongo …
• SnapshotStores - Cassandra, HDFS, HBase, Mongo …
SnapshotStore Plugins!
http://akka.io/community/#journal_plugins
Links• Official docs: http://doc.akka.io/docs/akka/2.3.0/scala/persistence.html
• Patrik’s Slides & Webinar: http://www.slideshare.net/patriknw/akka-persistence-webinar
• Papers:
• Sagas http://www.cs.cornell.edu/andru/cs711/2002fa/reading/sagas.pdf
• Life beyond Distributed Transactions: http://www-db.cs.wisc.edu/cidr/cidr2007/papers/cidr07p15.pdf
• Pics:
• http://misaspuppy.deviantart.com/art/Messenger-s-Cutie-Mark-A-Flying-Envelope-291040459
Mailing List
groups.google.com/forum/#!forum/akka-user
LinksEric Evans - “the DDD book”
http://www.amazon.com/Domain-Driven-Design-Tackling-Complexity-Software/dp/0321125215 !!!!!!
Vaughn Vernon’s Book and Talk http://www.amazon.com/Implementing-Domain-Driven-Design-Vaughn-Vernon/dp/0321834577
video: https://skillsmatter.com/skillscasts/4698-reactive-ddd-with-scala-and-akka !!!
ktoso @ typesafe.com twitter: ktosopl github: ktoso blog: project13.pl team blog: letitcrash.com Scalar 2014 @ Warsaw, PL
!
Dzięki! Thanks! ありがとう！ !
!
ping me:
