Akka persistence == event sourcing in 30 minutes

Akka persistence (message sourcing in 30 minutes)

Konrad 'ktoso' Malawski Scalar 2014 @ Warsaw, PL

Konrad `@ktosopl` Malawski

typesafe.com geecon.org

Java.pl / KrakowScala.pl sckrk.com / meetup.com/Paper-Cup @ London

GDGKrakow.pl meetup.com/Lambda-Lounge-Krakow

hAkker @

mainly by Martin Krasser !!(as contractor for Typesafe) !inspired by:

akka-persistence

https://github.com/krasserm

https://github.com/eligosource/eventsourced

dependencies

libraryDependencies ++= Seq(! "com.typesafe.akka" %% “akka-actor" % "2.3.0",! "com.typesafe.akka" %% "akka-persistence-experimental" % "2.3.0"!)

Show of hands!

sourcing styles

Command Sourcing Event Sourcing

msg: DoThing

msg persisted before receive

imperative, “do the thing”

business logic change, can be reflected in reaction

Processor

sourcing styles

Command Sourcing Event Sourcing

msg: DoThing msg: ThingDone

msg persisted before receive commands converted to events, must be manually persisted

imperative, “do the thing” past tense, “happened”

business logic change, can be reflected in reaction

business logic change, won’t change previous events

Processor EventsourcedProcessor

Plain Actors

ActorAn Actor that keeps count of messages it processed

Let’s send 2 messages to it

Let’s send 2 messages to it(it’s “commands”)

!!class Counter extends Actor {! var count = 0! def receive = {! case _ => count += 1! }!}

crash!

restart

restart

restarted

restarted

restarted

!wrong!

expected count == 2!

restarted

Processor

var count = 0 !

def processorId = “a” !

Journal (DB)

Processor

Journal (DB)

Processor

var count = 0 !

Journal (DB)

Processor

var count = 0 !

Journal (DB)

Processor

var count = 0 !

Journal (DB)

Processor

var count = 1 !

Journal (DB)

Processor

var count = 1 !

Journal (DB)

Processor

var count = 1 !

crash!

Journal (DB)

Processor

Journal (DB)

Processor

restart

Journal (DB)

Processor

var count = 0 !

restart

Journal (DB)

Processor

var count = 0 !

replay!

restart

Journal (DB)

Processor

var count = 0 !

Journal (DB)

Processor

var count = 0 !

replay!

Journal (DB)

Processor

var count = 1 !

Journal (DB)

Processor

var count = 1 !

replay!

Journal (DB)

Processor

var count = 1 !

Journal (DB)

Processor

var count = 2 !

Journal (DB)

Processor

var count = 2 !

Processorimport akka.persistence._!!class CounterProcessor extends Processor {! var count = 0! override val processorId = "counter"!! def receive = {! case Persistent(payload, seqNr) =>! // payload already persisted! count += 1! }!}

counter ! Persistent(payload)

is already persisted!

sequenceNr (generated by akka)

is already persisted!

Processorimport akka.persistence._!!class CounterProcessor extends Processor {! var count = 0! override val processorId = "counter"!! def receive = {! case notPersisted =>! // will not replay this msg!! count += 1! }!}

counter ! payload

won’t persist

counter ! payload

won’t persist

won’t replay

Processorimport akka.persistence._!!class CounterProcessor extends Processor {! var count = 0! override val processorId = "counter"!! def receive = {! case Persistent(payload, seqNr) =>! // payload already persisted! count += 1!! case notPersistentMsg =>! // msg not persisted - like in normal Actor! count += 1! }!}

ProcessorUpsides

• Persistent Command Sourcing “out of the box”

• Pretty simple, persist handled for you

• Once you get the msg, it’s persisted

• Pluggable Journals (HBase, Cassandra, Mongo, …)

• Can replay to given seqNr (post-mortem etc!)

ProcessorDownsides

• Exposes Persistent() to Actors who talk to you

• No room for validation before persisting

• There’s one Model, we act on the incoming msg

• Lower throughput than plain Actor (limited by DB)

EventsourcedProcessor

EventsourcedProcessor (longer name == more flexibility);-)

super quick domain modeling!

sealed trait Command!case class ManyCommand(nums: List[Int]) extends Command

Commands - input from user, “send emails”, not persisted

sealed trait Event!case class AddOneEvent(num: Int) extends Event!

Events - business events emitted by the processor, persisted

case class State(count: Int) {! def updated(more: Int) = State(count + more)!}

State - internal actor state, kept in var

sealed trait Event!case class AddOneEvent(num: Int) extends Event!

Events - business events emitted by the processor, persisted

var count = 0 !

Journal

var count = 0 !

Command

Journal

var count = 0 !

Journal

var count = 0 !

Journal

Generate Events

var count = 0 !

Journal

var count = 0 !

Journal

var count = 0 !

Journal

var count = 0 !

Journal

ACK “persisted”

var count = 1 !

Journal

var count = 1 !

Journal

“Apply” event

class MultiCounter extends EventsourcedProcessor {!! var state = State(count = 0)!! def updateState(e: Event): State = {! case event: AddOneEvent => state.updated(event.num)! }! ! // API:!! def receiveCommand = ??? // TODO!! def receiveRecover = ??? // TODO!!}!

def receiveCommand = {! case ManyCommand(many) =>! for (event <- many map AddOneEvent)! persist(event) { state = updateState(_) }!! case command: Command =>! persist(AddOneEvent(command)) { state = updateState(_) }!}

async persist that event

async called after successful persist

It is guaranteed that no new commands will be received by a processor between a call to `persist` and the execution of its `handler`.

EventsourcedProcessorcase ManyCommand(many) =>! for (event <- many map AddOneEvent)! persist(event) { state = updateState(_) }

“style fix”

EventsourcedProcessorcase ManyCommand(many) =>! for (event <- many map AddOneEvent)! persist(event) { state = updateState(_) }

case ManyCommand(many) =>! persist(many map AddOneEvent) { state = updateState(_) }

“style fix”

Eventsourced, recovery

/** MUST NOT SIDE-EFFECT! */!def receiveRecover = {! case replayedEvent: Event => ! updateState(replayedEvent)!}

Eventsourced, snapshots

Eventsourced, snapshots/** MUST NOT SIDE-EFFECT! */!def receiveRecover = {! case SnapshotOffer(meta, snapshot: State) => ! this.state = state!! case replayedEvent: Event => ! updateState(replayedEvent)!}

snapshot!? how?

Eventsourced, snapshots

def receiveCommand = {! case command: Command =>! saveSnapshot(state) // async!!}

/** MUST NOT SIDE-EFFECT! */!def receiveRecover = {! case SnapshotOffer(meta, snapshot: State) => ! this.state = state!! case replayedEvent: Event => ! updateState(replayedEvent)!}

snapshot!? how?

Snapshots

Snapshots (in SnapshotStore)

…sum of states…

Snapshots

Journal

E1 E2 E3 E4

E5 E6 E7 E8

state until [E8]

Snapshots

Journal

E1 E2 E3 E4

E5 E6 E7 E8

state until [E8]

Snapshots

Snapshot Store

snapshot!

Journal

E1 E2 E3 E4

E5 E6 E7 E8

state until [E8]

Snapshots

Snapshot Store

Journal

E1 E2 E3 E4

E5 E6 E7 E8

state until [E8]

Snapshots

Snapshot Store

Journal

E1 E2 E3 E4

E5 E6 E7 E8

crash!

Snapshots

Snapshot Store

Journal

E1 E2 E3 E4

E5 E6 E7 E8

crash!

“bring me up-to-date!”

Snapshots

Snapshot Store

Journal

E1 E2 E3 E4

E5 E6 E7 E8

restart!

Snapshots

Snapshot Store

Journal

E1 E2 E3 E4

E5 E6 E7 E8

restart!replay!

Snapshots

Snapshot Store

restart!replay!

Journal

E1 E2 E3 E4

E5 E6 E7 E8

Snapshots

Snapshot Store

restart!replay!

Journal

E1 E2 E3 E4

E5 E6 E7 E8

Snapshots

Snapshot Store

restart!replay!

Journal

E1 E2 E3 E4

E5 E6 E7 E8

state until [E8]

Snapshots

Snapshot Store

restart!replay!

Journal

E1 E2 E3 E4

E5 E6 E7 E8

state until [E8]

Snapshots

Snapshot Store

Journal

E1 E2 E3 E4

E5 E6 E7 E8

state until [E8]

Snapshots

Snapshot Store

Journal

E1 E2 E3 E4

E5 E6 E7 E8

We could delete these!

trait MySummer extends Processor {! var nums: List[Int]! var total: Int! ! def receive = {! case "snap" => saveSnapshot(total)! case SaveSnapshotSuccess(metadata) => // ...! case SaveSnapshotFailure(metadata, reason) => // ...! }!}!

Snapshots, save

Async!

Snapshots, success

final case class SnapshotMetadata(! processorId: String, sequenceNr: Long, ! timestamp: Long)

Snapshots, success

Snapshot Recovery

class Counter extends Processor {! var total = 0! ! def receive = {! case SnapshotOffer(metadata, snap: Int) => ! total = snap!! case Persistent(payload, sequenceNr) => // ...! }!}

SnapshotsUpsides

• Simple!

• Faster recovery (!)

• Allows to delete “older” events

• “known state at point in time”

SnapshotsDownsides

• More logic to write

• Maybe not needed if events replay “fast enough”

• Possibly “yet another database” to pick

• snapshots are different than events, may be big!

Journal (DB)

Views!

Processor !

Journal (DB)

Views!

Processor !

!def processorId = “a”

Journal (DB)

Views!

Processor !

pooling

Journal (DB)

Views!

Processor !

pooling

Journal (DB)

Views!

Processor !

pooling

different ActorPath, same processorId

class DoublingCounterProcessor extends View {! var state = 0! override val processorId = "counter"!! def receive = {! case Persistent(payload, seqNr) =>! // “state += 2 * payload” !! }!}

Akka Persistence Plugins

Akka Persistence PluginsPlugins are Community maintained!

(“not sure how production ready”)

http://akka.io/community/#journal_plugins

• Journals - Cassandra, HBase, Mongo …

• SnapshotStores - Cassandra, HDFS, HBase, Mongo …

SnapshotStore Plugins!

Links• Official docs: http://doc.akka.io/docs/akka/2.3.0/scala/persistence.html

• Patrik’s Slides & Webinar: http://www.slideshare.net/patriknw/akka-persistence-webinar

• Papers:

• Sagas http://www.cs.cornell.edu/andru/cs711/2002fa/reading/sagas.pdf

• Life beyond Distributed Transactions: http://www-db.cs.wisc.edu/cidr/cidr2007/papers/cidr07p15.pdf

• Pics:

• http://misaspuppy.deviantart.com/art/Messenger-s-Cutie-Mark-A-Flying-Envelope-291040459

Mailing List

groups.google.com/forum/#!forum/akka-user

LinksEric Evans - “the DDD book”

http://www.amazon.com/Domain-Driven-Design-Tackling-Complexity-Software/dp/0321125215 !!!!!!

Vaughn Vernon’s Book and Talk http://www.amazon.com/Implementing-Domain-Driven-Design-Vaughn-Vernon/dp/0321834577

video: https://skillsmatter.com/skillscasts/4698-reactive-ddd-with-scala-and-akka !!!

ktoso @ typesafe.com twitter: ktosopl github: ktoso blog: project13.pl team blog: letitcrash.com Scalar 2014 @ Warsaw, PL

Dzięki! Thanks! ありがとう！ !

ping me:

Akka persistence == event sourcing in 30 minutes

Technology

Transcript of Akka persistence == event sourcing in 30 minutes

Event Sourcing using Akka on AWS

Akka Documentation · 2019-03-28 · Akka Documentation, Release 1.1.3 1.2Why Akka? 1.2.1What features can the Akka platform offer, over the competition? Akka is an uniﬁed runtime

A reactive platform for real-time asset trading case study ...reactive+platform...scala, akka persistence, akka cluster, akka streams, cassandra (dse), apache kafka, gatling, prometheus,

Group Akka

Avoiding the Pit of Despair - Event Sourcing with Akka and Cassandra

Akka patterns

Akka typed-channels

Akka 2.0 Reloaded

Akka Persistence Scaladays 2014 140621085129 Phpapp02

DDDing Tools = Akka Persistence

2010 AKKA NEWSLETTER Volume 3, Issue 1torakenryu.com/images/2010AKKAnewsletter.pdf · AKKA in Hungary AKKA in Europe June: AKKA Camp Volume 3, Issue 1 T he past year was a difficult

Akka JUGL 2012

Persistence that tells the truth: event sourcing

Akka Survey 2013 - downloads.typesafe.comdownloads.typesafe.com/website/general/Akka-Survey-2013.pdfQ What were the two biggest difficulties while learning Akka? Other Immutability

TOLL FREE 1.866.328.5538 | bulletproofacademy · Using Akka sd_akka_a01_it_enus Akka Essentials Advanced Features of Akka sd_akka_a02_it_enus Java Multi-Tenant Cloud Programming Fundamentals

Akka 2.2.0

· CONTENTS 1 Introduction 1 1.1 What is Akka?. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1 1.2 Why Akka

Effective akka scalaio

Lessons&Learned&in&Deploying&Akka& Persistencedownloads.typesafe.com/website/presentations/lessons-learned-akk… · Lessons&Learned&in&Deploying&Akka& Persistence Duncan&K.&DeVore

HBase RowKey design for Akka Persistence