Domain Event Driven Architecture - QCon London · Domain Event Driven Architecture Stefan Norberg,...

66
Domain Event Driven Architecture Stefan Norberg, Head of Architecture at Unibet.com twitter: @stnor email: [email protected] blog: http://stnor.wordpress.com

Transcript of Domain Event Driven Architecture - QCon London · Domain Event Driven Architecture Stefan Norberg,...

Page 1: Domain Event Driven Architecture - QCon London · Domain Event Driven Architecture Stefan Norberg, Head of Architecture at Unibet.com twitter: @stnor ... • Operations & infrastructure

DomainEvent Driven Architecture

Stefan Norberg,Head of Architecture at Unibet.com

twitter: @stnoremail: [email protected]: http://stnor.wordpress.com

Page 2: Domain Event Driven Architecture - QCon London · Domain Event Driven Architecture Stefan Norberg, Head of Architecture at Unibet.com twitter: @stnor ... • Operations & infrastructure

@stnor

• 17 years as an IT professional

• Has worked with most aspects of IT

• Operations & infrastructure• IT security• Systems development• Software architecture• Enterprise architecture

• Head of Architecture at Unibet

Page 3: Domain Event Driven Architecture - QCon London · Domain Event Driven Architecture Stefan Norberg, Head of Architecture at Unibet.com twitter: @stnor ... • Operations & infrastructure

Agenda

• Basics and EDA intro

• The SOA problem

• How Domain EDA completes SOA

• Implementation notes from Unibet

• What to tell your manager

Page 4: Domain Event Driven Architecture - QCon London · Domain Event Driven Architecture Stefan Norberg, Head of Architecture at Unibet.com twitter: @stnor ... • Operations & infrastructure

The three styles of interaction

Type of interaction Initiator Participants

Time-driven Time The specified systems

Request-driven Client Client and Server

Event-driven Event Open-ended

Page 5: Domain Event Driven Architecture - QCon London · Domain Event Driven Architecture Stefan Norberg, Head of Architecture at Unibet.com twitter: @stnor ... • Operations & infrastructure

Time driven

Fruitsystem

run inventory check every 60

mins

Page 6: Domain Event Driven Architecture - QCon London · Domain Event Driven Architecture Stefan Norberg, Head of Architecture at Unibet.com twitter: @stnor ... • Operations & infrastructure

Request driven

TarzanFruit

systemMe want three bananas!

Page 7: Domain Event Driven Architecture - QCon London · Domain Event Driven Architecture Stefan Norberg, Head of Architecture at Unibet.com twitter: @stnor ... • Operations & infrastructure

Event driven

“Tarzan took three bananas”

“Fruit system is low on bananas”

Fruitsystem

Page 8: Domain Event Driven Architecture - QCon London · Domain Event Driven Architecture Stefan Norberg, Head of Architecture at Unibet.com twitter: @stnor ... • Operations & infrastructure

5 Principles of EDA

• “Real-time” events as they happen at the producer

• Push notifications

• One-way “fire-and-forget”

• Immediate action at the consumers

• Informational (“someone logged in”), not commands (“audit this”)

Page 9: Domain Event Driven Architecture - QCon London · Domain Event Driven Architecture Stefan Norberg, Head of Architecture at Unibet.com twitter: @stnor ... • Operations & infrastructure

Typical EDA architecture

system system system

Event Bus

EventProducers

EventTransport

system system systemEvent

Consumers

Page 10: Domain Event Driven Architecture - QCon London · Domain Event Driven Architecture Stefan Norberg, Head of Architecture at Unibet.com twitter: @stnor ... • Operations & infrastructure

Main benefits of EDA

• Supports the business demands for better service (no batch, less waiting)

• No point-to-point integrations (fire & forget)

• Enables high performance, highly scalable systems

Page 11: Domain Event Driven Architecture - QCon London · Domain Event Driven Architecture Stefan Norberg, Head of Architecture at Unibet.com twitter: @stnor ... • Operations & infrastructure

The birth of a system

Customer

Shop

Inventory

Page 12: Domain Event Driven Architecture - QCon London · Domain Event Driven Architecture Stefan Norberg, Head of Architecture at Unibet.com twitter: @stnor ... • Operations & infrastructure

The monolith

PaymentCustomer

ReportingInventory

Shop

Newsletter

Page 13: Domain Event Driven Architecture - QCon London · Domain Event Driven Architecture Stefan Norberg, Head of Architecture at Unibet.com twitter: @stnor ... • Operations & infrastructure

SOA architecture #1

Shop Payment Newsletter

InventoryCustomer

DB Reporting

Divide the problem domains into separate systems

Page 14: Domain Event Driven Architecture - QCon London · Domain Event Driven Architecture Stefan Norberg, Head of Architecture at Unibet.com twitter: @stnor ... • Operations & infrastructure

SOA architecture #1

Shop Payment Newsletter

InventoryCustomer

DB Reporting

A lot of point to point integration...

Page 15: Domain Event Driven Architecture - QCon London · Domain Event Driven Architecture Stefan Norberg, Head of Architecture at Unibet.com twitter: @stnor ... • Operations & infrastructure

SOA architecture #2

Shop Payment Newsletter

Service Bus

InventoryCustomer

DB Reporting

Monolith?

Page 16: Domain Event Driven Architecture - QCon London · Domain Event Driven Architecture Stefan Norberg, Head of Architecture at Unibet.com twitter: @stnor ... • Operations & infrastructure

Let’s add fraud checks

Shop Payment Newsletter

Service Bus

InventoryCustomer

DB Reporting

fraud

fraud

Page 17: Domain Event Driven Architecture - QCon London · Domain Event Driven Architecture Stefan Norberg, Head of Architecture at Unibet.com twitter: @stnor ... • Operations & infrastructure

Let’s add a loyalty system

Shop Payment Newsletter

Service Bus

InventoryCustomer

DB ReportingLoyalty

fraud

fraud

Page 18: Domain Event Driven Architecture - QCon London · Domain Event Driven Architecture Stefan Norberg, Head of Architecture at Unibet.com twitter: @stnor ... • Operations & infrastructure

Integration ripple effect

Shop Payment Newsletter

Service Bus

InventoryCustomer

DB ReportingLoyalty

fraud

fraud

Page 19: Domain Event Driven Architecture - QCon London · Domain Event Driven Architecture Stefan Norberg, Head of Architecture at Unibet.com twitter: @stnor ... • Operations & infrastructure

Problem summary

• SOA is all about dividing domain logic into separate systems and exposing it as services

• Some domain logic will, by its very nature, be spread out over many systems

• The result is domain pollution and bloat in the SOA systems

Page 20: Domain Event Driven Architecture - QCon London · Domain Event Driven Architecture Stefan Norberg, Head of Architecture at Unibet.com twitter: @stnor ... • Operations & infrastructure

“It's really become clear to me in the last couple of years that we need a new building block and that is the Domain Events”

-- Eric Evans, QCon 2009

Page 21: Domain Event Driven Architecture - QCon London · Domain Event Driven Architecture Stefan Norberg, Head of Architecture at Unibet.com twitter: @stnor ... • Operations & infrastructure

Domain Events and SOA

• A Domain Event is something that happened that the domain expert cares about

• A SOA system’s primary focus should be on the domain and domain logic

Page 22: Domain Event Driven Architecture - QCon London · Domain Event Driven Architecture Stefan Norberg, Head of Architecture at Unibet.com twitter: @stnor ... • Operations & infrastructure

Domain EDA

• By exposing relevant Domain Events on a shared event bus we can isolate cross cutting functions to separate systems

Page 23: Domain Event Driven Architecture - QCon London · Domain Event Driven Architecture Stefan Norberg, Head of Architecture at Unibet.com twitter: @stnor ... • Operations & infrastructure

Domain EDA + SOA

Shop Payment Newsletter

Inventory Customer

Reporting

LoyaltyEvent Bus

Page 24: Domain Event Driven Architecture - QCon London · Domain Event Driven Architecture Stefan Norberg, Head of Architecture at Unibet.com twitter: @stnor ... • Operations & infrastructure

Domain EDA + SOA

Shop Payment Newsletter

Inventory Customer

Reporting

LoyaltyEvent Bus

BAM

Fraud

Page 25: Domain Event Driven Architecture - QCon London · Domain Event Driven Architecture Stefan Norberg, Head of Architecture at Unibet.com twitter: @stnor ... • Operations & infrastructure

“You complete me”

SOADomain EDA

Page 26: Domain Event Driven Architecture - QCon London · Domain Event Driven Architecture Stefan Norberg, Head of Architecture at Unibet.com twitter: @stnor ... • Operations & infrastructure

Example how Domain EDA decouples

Page 27: Domain Event Driven Architecture - QCon London · Domain Event Driven Architecture Stefan Norberg, Head of Architecture at Unibet.com twitter: @stnor ... • Operations & infrastructure

Trading system

model

Reporting system

model

Coupled integration

Page 28: Domain Event Driven Architecture - QCon London · Domain Event Driven Architecture Stefan Norberg, Head of Architecture at Unibet.com twitter: @stnor ... • Operations & infrastructure

Trading system

model v2

Reporting system

model

Coupled integration

Page 29: Domain Event Driven Architecture - QCon London · Domain Event Driven Architecture Stefan Norberg, Head of Architecture at Unibet.com twitter: @stnor ... • Operations & infrastructure

Trading system

model

Reporting system

model

domain events

I’m interested in buy, sell and market

status events

subscribe

Domain EDA integration

Page 30: Domain Event Driven Architecture - QCon London · Domain Event Driven Architecture Stefan Norberg, Head of Architecture at Unibet.com twitter: @stnor ... • Operations & infrastructure

Trading system

model v2

Reporting system

model

domain events

I’m interested in buy, sell and market

status events

subscribe

Domain EDA integration

Page 31: Domain Event Driven Architecture - QCon London · Domain Event Driven Architecture Stefan Norberg, Head of Architecture at Unibet.com twitter: @stnor ... • Operations & infrastructure

- “So what? I can do that with SOA...”

- “Yes, but can you do THIS?” (drumroll)

Page 32: Domain Event Driven Architecture - QCon London · Domain Event Driven Architecture Stefan Norberg, Head of Architecture at Unibet.com twitter: @stnor ... • Operations & infrastructure

Trading system

model v2

Reporting system

model

domain events

I’m interested in buy, sell and market

status events

subscribe

Trading system NG

model

domain events

Domain EDA integration

Page 33: Domain Event Driven Architecture - QCon London · Domain Event Driven Architecture Stefan Norberg, Head of Architecture at Unibet.com twitter: @stnor ... • Operations & infrastructure

Trading system

model v2

Reporting system

model

I’m interested in buy, sell and market

status events

subscribe

Trading system NG

model

domain events

Domain EDA integration

Page 34: Domain Event Driven Architecture - QCon London · Domain Event Driven Architecture Stefan Norberg, Head of Architecture at Unibet.com twitter: @stnor ... • Operations & infrastructure

Example how EDA scales

Page 35: Domain Event Driven Architecture - QCon London · Domain Event Driven Architecture Stefan Norberg, Head of Architecture at Unibet.com twitter: @stnor ... • Operations & infrastructure

Traditional SOA scalability issue

Order processing

200 tx/s

Page 36: Domain Event Driven Architecture - QCon London · Domain Event Driven Architecture Stefan Norberg, Head of Architecture at Unibet.com twitter: @stnor ... • Operations & infrastructure

Traditional SOA scalability issue

Order processing

Loyalty system

request/response

200 tx/s

Page 37: Domain Event Driven Architecture - QCon London · Domain Event Driven Architecture Stefan Norberg, Head of Architecture at Unibet.com twitter: @stnor ... • Operations & infrastructure

Traditional SOA scalability issue

Order processing

Loyalty system

200 tx/s

100 tx/s

request/response

Page 38: Domain Event Driven Architecture - QCon London · Domain Event Driven Architecture Stefan Norberg, Head of Architecture at Unibet.com twitter: @stnor ... • Operations & infrastructure

SOA: Weakest link dictates the peak throughput

of the system

Order processing

Loyalty system

100 tx/s

100 tx/s

request/response

Page 39: Domain Event Driven Architecture - QCon London · Domain Event Driven Architecture Stefan Norberg, Head of Architecture at Unibet.com twitter: @stnor ... • Operations & infrastructure

Order processing

Loyalty system

200 tx/s

100 tx/s

EDA: Weakest link dictates the sustained throughput

of the system

event

Page 41: Domain Event Driven Architecture - QCon London · Domain Event Driven Architecture Stefan Norberg, Head of Architecture at Unibet.com twitter: @stnor ... • Operations & infrastructure

MQ considerations

• Ordering

• Durability

• Performance

• Once and only once delivery

Page 42: Domain Event Driven Architecture - QCon London · Domain Event Driven Architecture Stefan Norberg, Head of Architecture at Unibet.com twitter: @stnor ... • Operations & infrastructure

Our requirements

• 1000 messages / second sustained

• Guaranteed delivery

• Easy to use client

Page 43: Domain Event Driven Architecture - QCon London · Domain Event Driven Architecture Stefan Norberg, Head of Architecture at Unibet.com twitter: @stnor ... • Operations & infrastructure

Brokers tested

• Lots of brokers tested

• ActiveMQ, OpenMQ, HornetQ, Websphere, Fiorano, RabbitMQ, QPID

• Second round

• ActiveMQ, OpenMQ and RabbitMQ

• Finally went for ActiveMQ - fitted our requirements/use case the best

Page 44: Domain Event Driven Architecture - QCon London · Domain Event Driven Architecture Stefan Norberg, Head of Architecture at Unibet.com twitter: @stnor ... • Operations & infrastructure

Testing brokers

• Killing brokers

• Flow control

• Slow consumers

• Durable / non-durable

• Small configuration change can have huge impact on throughput

Page 45: Domain Event Driven Architecture - QCon London · Domain Event Driven Architecture Stefan Norberg, Head of Architecture at Unibet.com twitter: @stnor ... • Operations & infrastructure

Payload considerations

Page 46: Domain Event Driven Architecture - QCon London · Domain Event Driven Architecture Stefan Norberg, Head of Architecture at Unibet.com twitter: @stnor ... • Operations & infrastructure

The smorgasbord

Name Standardized Schema support Binary Easy to use X-platform Std API

ASN.1

CSV

XML

JSON

Java serialization

Hessian

Protocol Buffers

BSON

Yes Yes Yes No Yes No

Partial No No No? Yes No

Yes Yes No Yes Yes DOM, SAX, XQUERY, XPATH

Yes No No Yes Yes No

No Partial Yes Yes No No

No Partial Yes Yes Yes No

No Yes Yes Yes Yes No

No No Yes Yes Yes No

Page 47: Domain Event Driven Architecture - QCon London · Domain Event Driven Architecture Stefan Norberg, Head of Architecture at Unibet.com twitter: @stnor ... • Operations & infrastructure

Domain events need schema support

Name Standardized Schema support Binary Easy to use X-platform Std API

ASN.1

CSV

XML

JSON

Java serialization

Hessian

Protocol Buffers

BSON

Yes Yes Yes No Yes No

Partial No No No? Yes No

Yes Yes No Yes Yes DOM, SAX, XQUERY, XPATH

Yes No No Yes Yes No

No Partial Yes Yes No No

No Partial Yes Yes Yes No

No Yes Yes Yes Yes No

No No Yes Yes Yes No

Page 48: Domain Event Driven Architecture - QCon London · Domain Event Driven Architecture Stefan Norberg, Head of Architecture at Unibet.com twitter: @stnor ... • Operations & infrastructure

+ Efficient, x-platform and easy to use

Name Standardized Schema support Binary Easy to use X-platform Std API

ASN.1

XML

Protocol Buffers

Yes Yes No No Yes No

Yes Yes No Yes Yes DOM, SAX, XQUERY, XPATH

No Yes Yes Yes Yes No

Page 49: Domain Event Driven Architecture - QCon London · Domain Event Driven Architecture Stefan Norberg, Head of Architecture at Unibet.com twitter: @stnor ... • Operations & infrastructure

Protocol Buffers

• Flexible, efficient, automated mechanism for serializing (binary)

• Schema support (.proto files)

• Language neutral

• Invented and used by Google Open Sourced in July 2008

Page 50: Domain Event Driven Architecture - QCon London · Domain Event Driven Architecture Stefan Norberg, Head of Architecture at Unibet.com twitter: @stnor ... • Operations & infrastructure

Examplenewuserevent.proto:

option java_package = "com.example.foo";message NewUserEvent { required string name = 1; required int32 id = 2; optional string email = 3;

enum PhoneType { MOBILE = 0; HOME = 1; WORK = 2; }

message PhoneNumber { required string number = 1; optional PhoneType type = 2 [default = HOME]; }

repeated PhoneNumber phone = 4;}

Page 51: Domain Event Driven Architecture - QCon London · Domain Event Driven Architecture Stefan Norberg, Head of Architecture at Unibet.com twitter: @stnor ... • Operations & infrastructure

Generating Java Source

$ protoc --proto_path=IMPORT_PATH \ --java_out=DST_DIR \ path/to/file.proto

IMPORT_PATH specifies a directory in which to look for .proto files when resolving import directives.

If the DST_DIR ends in .zip or .jar, the compiler will write the output to a single ZIP-format archive file with the given name.

Page 52: Domain Event Driven Architecture - QCon London · Domain Event Driven Architecture Stefan Norberg, Head of Architecture at Unibet.com twitter: @stnor ... • Operations & infrastructure

Builders vs messages

NewUserEvent stefan = NewUserEvent.newBuilder() .setId(1337) .setName("Stefan Norberg") .setEmail(“[email protected]") .addPhone( NewUserEvent.PhoneNumber.newBuilder() .setNumber("+46 70 99 66 547") .setType(NewUserEvent.PhoneType.WORK)) .build();

• Messages classes are immutable

• Use builders to construct

Page 53: Domain Event Driven Architecture - QCon London · Domain Event Driven Architecture Stefan Norberg, Head of Architecture at Unibet.com twitter: @stnor ... • Operations & infrastructure

Evolving messages

• You cannot remove required fields

• New fields that you add should be optional or repeated

• New fields should have sensible defaults

• Prefix deprecated non-required fields with OBSOLETE_ (convention)

Page 54: Domain Event Driven Architecture - QCon London · Domain Event Driven Architecture Stefan Norberg, Head of Architecture at Unibet.com twitter: @stnor ... • Operations & infrastructure

Sample EDA producer

@AutowiredDomainEventPublisher publisher;

NewUserEvent stefan = NewUserEvent.newBuilder() .setId(1337) .setName("Stefan Norberg") .setEmail(“[email protected]") .addPhone( NewUserEvent.PhoneNumber.newBuilder() .setNumber("+46 70 99 66 547") .setType(NewUserEvent.PhoneType.WORK)) .build();

publisher.publish(stefan);

Page 55: Domain Event Driven Architecture - QCon London · Domain Event Driven Architecture Stefan Norberg, Head of Architecture at Unibet.com twitter: @stnor ... • Operations & infrastructure

Sample EDA consumerpublic class YourDomainListenerPOJO {

public void receive(LoginEvent loginEvent) {// do something

}

@Durable(clientId=”com.example.foo.bar”)public void receive(NewUserEvent newUserEvent) {

// do something}

}

<import resource=”classpath:spring-domain-events.xml />

<bean id=”domainEventListener” class=”com.foo.YourDomainListenerPOJO”/>

<domain-events:subscribe id=”eventSubscriber” subscriber=”domainEventListener”/>

Page 56: Domain Event Driven Architecture - QCon London · Domain Event Driven Architecture Stefan Norberg, Head of Architecture at Unibet.com twitter: @stnor ... • Operations & infrastructure

What to tell your manager

Page 57: Domain Event Driven Architecture - QCon London · Domain Event Driven Architecture Stefan Norberg, Head of Architecture at Unibet.com twitter: @stnor ... • Operations & infrastructure

What to tell your manager

• SOA+EDA will reduce time-to-market for new functionality

• SOA+EDA will enable a layer of high-value services that have a visible impact on the bottom line of the business

Page 58: Domain Event Driven Architecture - QCon London · Domain Event Driven Architecture Stefan Norberg, Head of Architecture at Unibet.com twitter: @stnor ... • Operations & infrastructure

Game changingvalue-add

Page 59: Domain Event Driven Architecture - QCon London · Domain Event Driven Architecture Stefan Norberg, Head of Architecture at Unibet.com twitter: @stnor ... • Operations & infrastructure

Complex Event Processing (CEP)

• 100% Event-driven

• Receives domain events

• Performs event correlation using a QL

• Fires domain events (“findings”)

Page 60: Domain Event Driven Architecture - QCon London · Domain Event Driven Architecture Stefan Norberg, Head of Architecture at Unibet.com twitter: @stnor ... • Operations & infrastructure

CEP Examples

Correlation(CEP)

events events “findings”

if there are no Mastercard deposits from France in 5 minutes during business hours,send NoDespositsEvent

if there are >30 failed logins using >5 accounts from the same ip within 2 minutes,block the ip for 24 hours

if customer with loyalty >= gold and puts goods in shopping cart for more than €200, send VIPShoppingEvent

if customer loses €2000 in the casino within 30 minutes and customer != highrollersend PotentialHighrollerEvent AND grant €200 casino premium

request/response“action”

Page 61: Domain Event Driven Architecture - QCon London · Domain Event Driven Architecture Stefan Norberg, Head of Architecture at Unibet.com twitter: @stnor ... • Operations & infrastructure

Business Process Management (BPM)

• Business control over definition and automation of business processes

• SOA services “orchestration”

• Processes can/should be started by domain events

Page 62: Domain Event Driven Architecture - QCon London · Domain Event Driven Architecture Stefan Norberg, Head of Architecture at Unibet.com twitter: @stnor ... • Operations & infrastructure

BPM example

EvaluateHighroller

Start

Profit > €5000

Manual investigation /

decision

Set VIP Status

Stop

Get Customer Info

decision?

Manager approval

Profit >€1000

approved?

Stop Stop

PotentialHighrollerEvent

CustomerSystem

Page 63: Domain Event Driven Architecture - QCon London · Domain Event Driven Architecture Stefan Norberg, Head of Architecture at Unibet.com twitter: @stnor ... • Operations & infrastructure

Business Activity Monitoring (BAM)

• Aggregation, analysis, and presentation of real time information about activities inside organizations and involving customers and partners

• BAM attempts to do for business processing what network management tools do for network operations

Page 64: Domain Event Driven Architecture - QCon London · Domain Event Driven Architecture Stefan Norberg, Head of Architecture at Unibet.com twitter: @stnor ... • Operations & infrastructure
Page 65: Domain Event Driven Architecture - QCon London · Domain Event Driven Architecture Stefan Norberg, Head of Architecture at Unibet.com twitter: @stnor ... • Operations & infrastructure

The real-time business eco system

Service Oriented Architecture (SOA)

Domain Event Driven Architecture (D-EDA)

Correlation(CEP)

Processes(BPM)

Monitoring(BAM)

Business Value

IT enhancements

Great business

IT systems

Great architecture

events

events

events

events

eventsrequest/response

request/response

Page 66: Domain Event Driven Architecture - QCon London · Domain Event Driven Architecture Stefan Norberg, Head of Architecture at Unibet.com twitter: @stnor ... • Operations & infrastructure

Thank you!