Akka Persistence Scalar Conf 2014 140404160618 Phpapp02

Akka persistence

(message sourcing in 30 minutes)

Konrad 'ktoso' Malawski

Scalar 2014 @ Warsaw, PL

Konrad `@ktosopl` Malawski

typesafe.com

geecon.org

Java.pl / KrakowScala.pl

sckrk.com / meetup.com/Paper-Cup @ London

GDGKrakow.pl

meetup.com/Lambda-Lounge-Krakow

Konrad `@ktosopl` Malawski

typesafe.com

geecon.org

Java.pl / KrakowScala.pl

sckrk.com / meetup.com/Paper-Cup @ London

GDGKrakow.pl

meetup.com/Lambda-Lounge-Krakow

hAkker @

mainly by

Martin Krasser !!(as contractor for Typesafe)

!inspired by:

akka-persistence

https://github.com/krasserm

https://github.com/eligosource/eventsourced

dependencies

libraryDependencies ++= Seq(! "com.typesafe.akka" %% akka-actor" % "2.3.0",! "com.typesafe.akka" %% "akka-persistence-experimental" % "2.3.0"!)

Show of hands!

sourcing styles

Command Sourcing Event Sourcing

msg: DoThing

msg persisted before receive

imperative, do the thing

business logic change, can be reflected in reaction

Processor

sourcing styles

Command Sourcing Event Sourcing

msg: DoThing msg: ThingDone

msg persisted before receive commands converted to events, must be manually persistedimperative, do the thing past tense, happened

business logic change, can be reflected in reaction

business logic change, wont change previous events

Processor EventsourcedProcessor

Plain Actors

Actor

!!class Counter extends Actor {! var count = 0! def receive = {! case _ => count += 1! }!}

Actor

crash!

Actor

restart

Processor

var count = 0 !

def processorId = a !

Journal

(DB)

!!!

Processor

Journal

(DB)

!!!

Processor

var count = 0 !


Journal

(DB)

!!!

Processor

var count = 1 !


Journal

(DB)

!!!

Processor

var count = 1 !


crash!

Journal

(DB)

!!!

Processor

Journal

(DB)

!!!

Processor

restart

Journal

(DB)

!!!

Processor

var count = 0 !


restart

Journal

(DB)

!!!

Processor

var count = 0 !


replay!

restart

Journal

(DB)

!!!

Processor

var count = 0 !


Journal

(DB)

!!!

Processor

var count = 0 !


replay!

Journal

(DB)

!!!

Processor

var count = 1 !


Journal

(DB)

!!!

Processor

var count = 1 !


replay!

Journal

(DB)

!!!

Processor

var count = 1 !


Journal

(DB)

!!!

Processor

var count = 2 !


Journal

(DB)

!!!

Processor

var count = 2 !


yay!

Processorimport akka.persistence._!!class CounterProcessor extends Processor {! var count = 0! override val processorId = "counter"!! def receive = {! case Persistent(payload, seqNr) =>! // payload already persisted! count += 1! }!}


counter ! Persistent(payload)



is already persisted!



sequenceNr (generated by akka)

is already persisted!

Processorimport akka.persistence._!!class CounterProcessor extends Processor {! var count = 0! override val processorId = "counter"!! def receive = {! case notPersisted =>! // will not replay this msg!! count += 1! }!}

counter ! payload


counter ! payload

wont persist


counter ! payload

wont persist

wont replay

Processorimport akka.persistence._!!class CounterProcessor extends Processor {! var count = 0! override val processorId = "counter"!! def receive = {! case Persistent(payload, seqNr) =>! // payload already persisted! count += 1!! case notPersistentMsg =>! // msg not persisted - like in normal Actor! count += 1! }!}

ProcessorUpsides

Persistent Command Sourcing out of the box

Pretty simple, persist handled for you

Once you get the msg, its persisted

Pluggable Journals (HBase, Cassandra, Mongo, )

Can replay to given seqNr (post-mortem etc!)

ProcessorDownsides

Exposes Persistent() to Actors who talk to you

No room for validation before persisting

Theres one Model, we act on the incoming msg

Lower throughput than plain Actor (limited by DB)

EventsourcedProcessor


(longer name == more flexibility);-)

super quick domain modeling!


sealed trait Command!case class ManyCommand(nums: List[Int]) extends Command

Commands - input from user, send emails, not persisted




sealed trait Event!case class AddOneEvent(num: Int) extends Event!

Events - business events emitted by the processor, persisted




case class State(count: Int) {! def updated(more: Int) = State(count + more)!}

State - internal actor state, kept in var

sealed trait Event!case class AddOneEvent(num: Int) extends Event!

Events - business events emitted by the processor, persisted

var count = 0 !



!!

Journal

C1

var count = 0 !



Command

!!

Journal

C1

var count = 0 !



!!

Journal

C1

var count = 0 !



!!

Journal

C1

Generate Events

C1

var count = 0 !



!!

Journal

E1

C1

var count = 0 !



!!

Journal

E1

Event

C1

var count = 0 !



!!

Journal

E1

C1

var count = 0 !



!!

Journal

E1

ACK persisted

C1

var count = 1 !



!!

Journal

E1

E1

C1

var count = 1 !



!!

Journal

E1

E1

Apply event


class MultiCounter extends EventsourcedProcessor {!! var state = State(count = 0)!! def updateState(e: Event): State = {! case event: AddOneEvent => state.updated(event.num)! }! ! // API:!! def receiveCommand = ??? // TODO!! def receiveRecover = ??? // TODO!!}!


def receiveCommand = {! case ManyCommand(many) =>! for (event ! persist(AddOneEvent(command)) { state = updateState(_) }!}



async persist that event




async called after successful persist




async called after successful persist

It is guaranteed that no new commands will be received by a processor between a call to `persist` and the execution of its `handler`.

EventsourcedProcessorcase ManyCommand(many) =>! for (event

EventsourcedProcessorcase ManyCommand(many) =>! for (event ! persist(many map AddOneEvent) { state = updateState(_) }

style fix

Eventsourced, recovery

/** MUST NOT SIDE-EFFECT! */!def receiveRecover = {! case replayedEvent: Event => ! updateState(replayedEvent)!}

Eventsourced, snapshots

Eventsourced, snapshots/** MUST NOT SIDE-EFFECT! */!def receiveRecover = {! case SnapshotOffer(meta, snapshot: State) => ! this.state = state!! case replayedEvent: Event => ! updateState(replayedEvent)!}

Eventsourced, snapshots/** MUST NOT SIDE-EFFECT! */!def receiveRecover = {! case SnapshotOffer(meta, snapshot: State) => ! this.state = state!! case replayedEvent: Event => ! updateState(replayedEvent)!}

snapshot!? how?

Eventsourced, snapshots

def receiveCommand = {! case command: Command =>! saveSnapshot(state) // async!!}

/** MUST NOT SIDE-EFFECT! */!def receiveRecover = {! case SnapshotOffer(meta, snapshot: State) => ! this.state = state!! case replayedEvent: Event => ! updateState(replayedEvent)!}

snapshot!? how?

Snapshots

Snapshots

(in SnapshotStore)

sum of states

Snapshots

!!

Journal

E1 E2 E3 E4

E5 E6 E7 E8

state until [E8]

Snapshots

S8!!

Journal

E1 E2 E3 E4

E5 E6 E7 E8

state until [E8]

Snapshots

S8

!!

Snapshot Store

snapshot!

!!

Journal

E1 E2 E3 E4

E5 E6 E7 E8

state until [E8]

Snapshots

S8

!!

Snapshot Store

!!

Journal

E1 E2 E3 E4

E5 E6 E7 E8

S8

state until [E8]

Snapshots

S8

!!

Snapshot Store

!!

Journal

E1 E2 E3 E4

E5 E6 E7 E8

S8

crash!

Snapshots

!!

Snapshot Store

!!

Journal

E1 E2 E3 E4

E5 E6 E7 E8

S8

crash!

bring me up-to-date!

Snapshots

!!

Snapshot Store

!!

Journal

E1 E2 E3 E4

E5 E6 E7 E8

S8

restart!


Snapshots

!!

Snapshot Store

!!

Journal

E1 E2 E3 E4

E5 E6 E7 E8

S8

restart!replay!


Snapshots

!!

Snapshot Store

S8

restart!replay!

!!

Journal

E1 E2 E3 E4

E5 E6 E7 E8


Snapshots

!!

Snapshot Store

S8

restart!replay!

S8!!

Journal

E1 E2 E3 E4

E5 E6 E7 E8

state until [E8]

Snapshots

!!

Snapshot Store

S8

restart!replay!

S8!!

Journal

E1 E2 E3 E4

E5 E6 E7 E8

state until [E8]

Snapshots

!!

Snapshot Store

S8

S8!!

Journal

E1 E2 E3 E4

E5 E6 E7 E8

state until [E8]

Snapshots

!!

Snapshot Store

S8

S8!!

Journal

E1 E2 E3 E4

E5 E6 E7 E8

We could delete these!

trait MySummer extends Processor {! var nums: List[Int]! var total: Int! ! def receive = {! case "snap" => saveSnapshot(total)! case SaveSnapshotSuccess(metadata) => // ...! case SaveSnapshotFailure(metadata, reason) => // ...! }!}!

Snapshots, save


Snapshots, save

Async!


Snapshots, success


Snapshots, success

final case class SnapshotMetadata(! processorId: String, sequenceNr: Long, ! timestamp: Long)


Snapshots, success

Snapshot Recovery

class Counter extends Processor {! var total = 0! ! def receive = {! case SnapshotOffer(metadata, snap: Int) => ! total = snap!! case Persistent(payload, sequenceNr) => // ...! }!}

SnapshotsUpsides

Simple!

Faster recovery (!)

Allows to delete older events

known state at point in time

SnapshotsDownsides

More logic to write

Maybe not needed if events replay fast enough

Possibly yet another database to pick

snapshots are different than events, may be big!

Journal

(DB)

!!!

Views!

Processor !


Journal

(DB)

!!!

Views!

Processor !


!View

!def processorId = a

!!!

Journal

(DB)

!!!

Views!

Processor !


!View


!!!

pooling

Journal

(DB)

!!!

Views!

Processor !


!View


!!!

pooling

!View


!!!

pooling

Journal

(DB)

!!!

Views!

Processor !


!View


!!!

pooling

!View


!!!

pooling

different ActorPath, same processorId

View

class DoublingCounterProcessor extends View {! var state = 0! override val processorId = "counter"!! def receive = {! case Persistent(payload, seqNr) =>! // state += 2 * payload !! }!}

Akka Persistence Plugins

Akka Persistence PluginsPlugins are Community maintained!

(not sure how production ready)

http://akka.io/community/#journal_plugins




Journals - Cassandra, HBase, Mongo




Journals - Cassandra, HBase, Mongo

SnapshotStores - Cassandra, HDFS, HBase, Mongo

SnapshotStore Plugins!


Links Official docs: http://doc.akka.io/docs/akka/2.3.0/scala/persistence.html

Patriks Slides & Webinar: http://www.slideshare.net/patriknw/akka-

persistence-webinar

Papers:

Sagas http://www.cs.cornell.edu/andru/cs711/2002fa/reading/

sagas.pdf

Life beyond Distributed Transactions: http://www-db.cs.wisc.edu/cidr/cidr2007/papers/cidr07p15.pdf

Pics:

http://misaspuppy.deviantart.com/art/Messenger-s-Cutie-Mark-A-

Flying-Envelope-291040459

Mailing List

groups.google.com/forum/#!forum/akka-user

LinksEric Evans - the DDD book

http://www.amazon.com/Domain-Driven-Design-Tackling-Complexity-Software/dp/0321125215

!!!!!!

Vaughn Vernons Book and Talk

http://www.amazon.com/Implementing-Domain-Driven-Design-Vaughn-Vernon/dp/0321834577

video: https://skillsmatter.com/skillscasts/4698-reactive-ddd-with-scala-and-akka

!!!

ktoso @ typesafe.com twitter: ktosopl

github: ktoso

blog: project13.pl

team blog: letitcrash.com Scalar 2014 @ Warsaw, PL

!

Dziki!

Thanks!

!

!

ping me:

Akka Persistence Scalar Conf 2014 140404160618 Phpapp02

Documents

Transcript of Akka Persistence Scalar Conf 2014 140404160618 Phpapp02