Reactive DevelopmentCommands, Actors and Events – Oh My!

About Me• 6-Time .NET (Visual C#) MVP (April 2011)• Sr. Solutions Architect at Confluence• One of the organizers for Pittsburgh TechFest (

http://pghtechfest.com)• Organizer of Pittsburgh Reactive Dev Group (

http://meetup.com/reactive)• Past President of Pittsburgh .NET Users Group• Twitter - @DavidHoerster• Blog – http://blog.agileways.com

Agenda• What are we building?• Messages• What is CQRS?• …and isn’t it dead?

• Build a simple handler• Using Actors Instead of Procedural Domains• Communicating Back to the Client• Resilient Processing

ScheduleStart End Topic

8:30 AM 8:45 AM Intro / What are we building

8:45 AM 10:00 AM Part 1: CQRS / Messages / Lab 1 (Building a Handler)

10:00 AM 10:15 AM Break

10:15 AM 11:45 AM Part 2: Actors / Mini Labs

11:45 AM 12:30 PM Lunch!!

12:30 PM 2:00 PM Part 3: CQRS + Actors / Lab 2 (Actors in Handler; Projections)

2:00 PM 2:15 PM Break

2:15 PM 3:00 PM Lab 3 – End to End Communication (Real-Time Updates)

3:00 PM 3:45 PM Lab 4 – Events; Lab 5 – Querying Events (Restoring Actors from Events; Querying EventStore)

3:45 PM 4:00 PM Wrap-Up

What Are We Building?• An application that processes multiple files• Each file contains all of the plays from the 2015 baseball season for one

HOME team• The files are CSV

• Each row of the file will be sent as a message• A back end process will subscribe to the message and process the data

• Game data, batter data, pitcher data• Results will be published to a topic• Another service will subscribe to that message and push updates to a

simple HTML page via SignalR

Baseball Play Processor System

Producer

Producer

Producer

Topic

Consumer

Topic

API Gateway

ConsumerProjections

(Mongo)Events

(EventStore)Ad Hoc Query

Retro Files

Retro Files• RetroSheet.org files listing every play for every home game in 2015

(per team)• CSV files• 97 fields – we only use a handful• http://www.retrosheet.org/datause.txt

• Feel free to use more fields!!!

Producer and Consumer• Separate processes• Producer reads in Retro CSV files• Pushes each row into a message “topic”• Consumer subscribes to the new messages on the topic• Handles the row accordingly• Consumer will eventually use actors to process data• Run multiple producers, consume additional fields, create new actors,

create new projections!!!

Message “Topics”• Using a RethinkDB table per topic• Somewhat of a workaround for local development• EXE is a beta preview for Windows, so not ready for prime time

• RethinkDB allows clients to subscribe to changes• Acts like a topic• Multiple subscribers

• C# client is a port of the official Java client

MongoDB• Document database• Holds projections for batters and pitchers• And any other projections you’d like to create

• Using 3.X version with WiredTiger• RoboMongo 0.9.X supports WiredTiger (finally!)

• MongoDB C# client is official client

API Gateway• OWIN self-hosted Web API service• Subscribes to changes for batters• Exposes simple API for retrieving batter projection from MongoDB• Pushes changes from topic to client via SignalR• Only pushes to subscribed “group” – not broadcasting out all changes

• Create more pages and APIs!!!

Web Client• Stupid simple web site• One static HTML page• Uses jQuery for simple interaction• Uses SignalR JS library to subscribe to hub for batter changes• As new batter is queried, that is the “group” that is subscribed to• Feel free to create new pages, enhance look n’ feel, add more data!!!

EventStore• Used to hold events for players• Each player id is a new stream• Stream consists of 0 to many events• Can be of different types

• Events are used to restore state of player actors (batters for now) and also used for ad hoc querying later• Feel free to add new types of events (pitcher events???) and add

more data!!!

Ad Hoc Querying• Queries the EventStore for basic batter information• Average and Homers hit for each count (balls/strikes)• Some overall stats

• Uses EventStore C# client (official)• Create new queries, make it a web page, try to slice and dice data

differently!!!

Perspectives on Procedural

Real World Scenario

File File File

Join

Op

Join/End

Real World Scenario• Actually looks like this:

• Multiple sources

• Several transforms

• Several joins

• Configurable

Messages as Part of Application Design

CQRS• Command Query Responsibility Segregation• Coined by Greg Young

• Evolution of CQS (Command Query Separation)• Coined by Bertrand Meyer

• Both revolve around the separation of writes (command) and reads (query)• CQS is more at the unit level• CQRS is at the bounded context level, or so

Messages• The core of CQRS Messages• Communication from the service layer to the domain via commands• Command handlers could also update the underlying repository

• As actions happen, events are raised• Event handlers could issue other commands• Event handlers could update the underlying repository

Messages• Regardless of how it’s implemented, communication between

application parts is via messages• Not only helps communicate intent of the action• “PublishWidget”, “ExpireWorkItem”, “UpdateDefinition”, “DefinitionUpdated”

• But allows for remote handling of messages• Queue, REST payload, etc.

Messages are key to being reactive

So?• So how does this affect my system design?

• Let’s consider some procedural pitfalls

Word Counter Example• Simple program to count the occurrences of words in a document

• Print out the top 25 words and their counts at the end

• Can be accomplished easily via procedural code

Word Counter Example

Word Counter Example• What if you wanted to spread out the counting load?

• What if you wanted to scale out to other nodes/processes/machines?

• Fault tolerance, concurrent actions, etc.?

Building Concurrent Apps• In word counter, may need to spawn some threads

• Lock some critical sections, etc.

• The system becomes responsible for knowing everything going on• Very Proactive (not Reactive!) system

Side note: reactive vs. proactive• Isn’t proactive a good thing?

• A system taking upon itself to check state is proactive• It’s doing too much

• A reactive system reacts to events that occur• It’s doing just the right amount

Side note: reactive vs. proactive• Isn’t proactive a good thing?

• A system taking upon itself to check state is proactive• It’s doing too much

• A reactive system reacts to events that occur• It’s doing just the right amount

System

EvtEvt

System

EvtEvt

Proactive

Reactive

What is CQRS?

CQRS Background• Command Query Responsibility Segregation

• Coined by Greg Young

• Heavily influenced by Domain Driven Design (DDD)• Evolution of Meyer’s CQS (Command Query Separation)• Separation of writes (command) and reads (query)

• CQS is more at the unit level• CQRS is at a higher level bounded context / service

What is CQRS?• Simply: separate paths for reads and writes

• Communicate via messages

• Commands and Events• “CreateOrder” and “OrderCreated”

• Reads are against a thin read model (preferably optimized)

CQRS Visualized: Basic

Client

Backend

Commands Queries

CQRS Visualized

Controller

Handler

Domain

Persistence / Read Model

Read LayerCommandSide

QuerySide

CQRS Extended• Can move from direct references to queued

• Decouples handlers

• Increased isolation of areas

• Leads to single message causing several changes• Raising of events and subscriptions to events

CQRSVisualized

Controller

Handler

Domain

Persistence

Read Layer

Handler

Com

man

d / E

vent

Topi

cs Read Model

CommandSide

QuerySide

CQRS Handler

CQRS Handler

Benefits• Message Based

• Segregation of Responsibilities – Separate Paths

• Smaller, simpler classes• Albeit more classes

• Focused Approach to Domain• Move away from anemic domain models

CQRS Principles• Separate Paths for Command and Queries

• Message-Based

• Async Friendly

• Thin read layer

• Don’t fear lots of small classes

Is CQRS Dead?

<rant>

"CQRS is Hard"• Misunderstandings

• Has to be async• Has to have queueing• Need a separate read model• Need to have Event Sourcing

• Perceived as overly prescriptive, but little prescription

• Confusing• Command Handler vs. Event Handler vs. Domain• Command vs. Event

Has CQRS “Failed”?

• Focus on CQRS as architecture over principles• Search for prescriptive guidance

• Infrastructure/Implementation debates• Should domains have getters??

• “…there are times a getter can be pragmatic. The trick is learning where to use them.” – Greg Young (DDD/CQRS Group)

• “This list has its heads up its own [butts] about infrastructure. What business problems are you working on?” -- Greg Young (DDD/CQRS Group)

</rant>

Popular Architecture Topics• Microservices

• Event driven

• Distributed computing

• NoSQL

• Async

But Isn’t CQRS…?

Message Driven

Responsive Read Model

Isolated HandlersAsync Friendly

Group Handlers into Clusterable Groups

NoSQL Friendly (Flattened Read Models)

CQRS Can Help Drive Reactive Apps

Lab: Building a Simple Handler

Lab Info• Scaffold: Lab1-Handler

• Create a Producer EXE (service) that adds Plays to a RethinkDB table

• Create a Consumer EXE (service) that subscribes to the “plays” RethinkDB table for changes

• Use the Shared project for common entities and messages

• In RethinkDB, make sure you:• Create a database named “baseball” and a table named “plays”• To clear the table, go to Data Explorer and issue this command:

r.db("baseball").table("plays").delete();

Lab: Using RethinkDB• What is and Why

RethinkDB?• Document database• Can join tables/collections

• Big feature is subscribing to changes• Inserting into a table is

just adding a POCO

Lab: Using RethinkDB for Subscriptions• RunChanges is

blocking• Use Reactive

Extensions to subscribe to changes in the feed• “include_initial” is

an OptArg to indicate you want all rows to start with along with changes.

CQRS Evolved – CQRS + Actors

CQRS & the Reactive Manifesto

• Message Based• Core of CQRS

• Responsive• Commands are ack/nack• Queries are (can be) against an optimized read model

• Resillient• Not core to CQRS Implementation

• Elastic• Not core to CQRS Implementation

CQRS + Reactive• Resillient and Elastic core to

Reactive

• CQRS’ core is Message-Based and Responsive

• Combining the two is powerful

Actor ModelCQRS lacks prescriptive guidance

• Actor Model provides a message-based pattern• Provides prescriptive guidance• Makes for more generalized implementation

• Actor Model is driving more reactive-based development

Actor Model• Actor Model is a system made of small units of concurrent

computation• Formulated in the early 70’s

• Each unit is an actor

• Communicates to each other via messages

• Actors can have children, which they can supervise

What is an Actor?• Lightweight class that encapsulates state and behavior

• State can be persisted (Akka.Persistence)• Behavior can be switched (Become/Unbecome)

• Has a mailbox to receive messages• Ordered delivery by default• Queue

• Can create child actors• Has a supervisory strategy for child actors

• How to handle misbehaving children

• Are always thread safe• Have a lifecycle

• Started, stopped, restarted

Actor Hierarchy

baseball

gameCoordinator

gameInfo-x gameInfo-yplayerSupervisor

batter-abatter-bbatter-c

c-01 c-11 c-32

Akka: Resiliency• Actors supervise their children

• When they misbehave, they are notified

• Can punish (fail) all children, or tell the misbehaving on to try again

• Protects rest of the system

Akka.NET• Akka.NET is an open source framework• Actor Model for .NET

• Port of Akka (for JVM/Scala)

• http://getakka.net

• NuGet – searching for “akka.net”, shows up low in the list

Starting with Akka.NET• Akka.NET is a hierarchical system

• Root of the system is ActorSystem

• Expensive to instantiate – create and hold!

• ActorSystem can then be used to create Actors

Creating an Actor• An Actor has a unique address• Can be accessed/communicated to like a URI

• Call ActorOf off ActorSystem, provide it a name• URI (actor path) is created hierarchically

• Actor Path = akka://myName/user/announcer

Creating Children• Best to call ActorOf to get an IActorRef

• Create a static Props creator

• Actor path is relative to parent’s actor path

• akka://myDemo/user/countChocula/{childName}

• When creating a child, good idea to name it• So you can reference it later!

Sending a Message• Messages are basis of actor communication• Should be IMMUTABLE!!!

• “Tell” an actor a message• Async call

Immutable!!

Receiving A Message• Create your Actor

• Derive from ReceiveActor

• In constructor, register your message subscriptions

• Looks similar to a CQRS • Handler

Receiving A Message• Each actor has a mailbox

• Messages are received in the actor’s mailbox (queue)

• Actor is notified that there’s a message• It receives the message• Takes it out of the mailbox• Next one is queued up

• Ordered delivery by default• Other mailboxes (like Priority) that are out-of-order

Actor

Mai

lbox

Msg

Async• Actors have messages delivered to them in order via a Mailbox• As message is received, handed to Actor to process• Messages in mailbox queue up• As a result, async processing of messages isn’t possible• Receive<> (async msg => { … }) doesn’t work

• If you need to perform async tasks within Receive• PipeTo(Self) usually does the trick• Re-queues message back on Mailbox• When delivered, actor resumes

Actor

Mai

lbox

Msg

PipeTo

Lab: Breaking Bad ActorsAnd other simple actor projects

Lab Info• Scaffold: A1-HelloAkka

• Simple Two Actor Project

• Try including another actor or two (and another message!)

Lab Info• Scaffold: A2-ActorTimeoutTest

• Show how an actor can commit “suicide”

• Uses a Scheduler

• Try to implement this another way• Instead of parent issuing the Poison Pill, have the Child do it• Override AroundReceive()

CQRS + Actors

Elements of CQRS in Actor Model• Message-based

• Potentially async

• Focused code

• Actor System is your Command Side

CQRS + Actors• Combining the principles of CQRS and the patterns of the Actor Model

• Message-based communication with concurrent processing

CQRS + Actors: Messages• Messages in an Actor System include both Commands and Events

• Still encapsulate intent

• Still should still contain information necessary to process/handle command or event

• Basically, no big change here

CQRS + Actors: Messages

Lab: Actor Word Counter

Lab Info• Scaffold: A3-ActorWordCounter

• Notice how actors can call each other (and themselves)

• Update messages to be more like Commands and Events

• CHALLENGE: Can you use the RoundRobinPool???

CQRS + Actors – Handlers and Services

CQRS + Actors: Handlers• Handlers begin to encapsulate your

Actor System

• Generally a single Actor System per handler

• Have several actors at top level of hierarchy to handle initial messages

• Domain can be injected into the Actor System via Create()

Handler

Client

QueuePersistenceOther Handler

Direct call or via queue

CQRS + Actors: Handlers

CQRS + Actors: Handlers

CQRS + Actors: Handlers• Handlers act as the entry point• Web API• Service subscribed to queue

• CQRS IHandle<T> for those messages “exposed”

• Actor System is called by the CQRS Handler

• Actor System has many other “internal” actors

CQRS + Actors: Services

Word Counter Handler

super

Count Write Stats

A B G H Y Z

Word Counter Handler

CQRS + Actors: Services

super

Count

A B

Word Writer Handler

super

Write

G H

Word Stats Handler

super

Stats

Y Z

Message Bus

CQRS + Actors + Services

Service A Service B Service C

Message Bus

DW / DL / Read Model

Cmds / Events Cmds / Events

API Gateway SvcCommands Queries

RM RMRM

Queries (?)

Service AService AService A

CQRS + Actors + Services + Cluster (Akka.Cluster)

Service A Service B Service C

Message Bus

DW / DL / Read Model

Cmds / Events Cmds / Events

API Gateway SvcCommands Queries

RM RMRM

Queries (?)

CQRS + Actors + Services• Actors act as the workers for your handlers

• Services encapsulate handler into a bounded context

• CQRS is the messaging and communication pattern within system

• Using these three helps build killer reactive applications

Lab: Distributed Reactive Apps

Lab Info• Scaffold: Lab2-GameActor

• The Domain\Actors and Messages folders have been excluded from the project• Include them if you want to run things• Study them if you want to write your own

Lab Info• RethinkDB Setup• Database named: “baseball”• Tables named: “plays” and “batterStat”• To clear these tables, issue the command from Data Explorer

r.db("baseball").table("batterStat").delete();r.db("baseball").table("plays").delete();

• MongoDB shouldn’t need setup – database and collections should be created• Database is “baseball2015” and collections are “batter” and “pitcher”

Lab Info• Add a new collection to MongoDB’s baseball2015 database via

mongoimport• Command line• Bin\mongoimport /d baseball2015 /c players /headerline /type csv “data\players.csv”

• This will allow you to view players’ real names instead of just IDs (e.g. Andrew McCutchen vs. mccua001)

Lab Info: Actor Model/user

gameEventCoord

gameSupervisor

gameActor 1 gameActor 2

teamSupervisor

teamActor 1 teamActor 2

batterSupervisor

pitcherSupervisor

pitcherActor 1 pitcherActor 2

batterActor 1 batterActor 2

Lab: Communicating Back

Lab Info• Scaffold: Lab3-Display

• This lab subscribes to a topic/table in RethinkDB for batter events• Events are pushed to a web JavaScript client via SignalR• Web API uses SignalR groups to only send subscribed players to clients

(instead of all updates)

• Domain\Actors and Messages are excluded. Use if you’d like• UI-Api\Hubs and UI-Api\Utils are excluded. These set up SignalR Hubs and

Group Subscriptions. Use if you’d like.

Lab: End to End

Lab Info• Scaffold: Lab4-Events

• Nothing excluded• Try to add additional information to events or add pitcher events to

EventStore• Add a pitcher.html page, similar to default.html, to query on pitcher

information

Lab Info• Scaffold: Lab5-EventStoreSample

• Use a populated EventStore to query• Slice events differently• Create different types of queries

• Also, try to incorporate this into a web page instead of a console app• Show Batter Count stats on a web page, even in real time via SignalR

updates!!!

Thank you!