Dennis Doomen | @ddoomen | Aviva Solutions | The Continuous Improver

Also known as the Microservices Premium

Hybrid of Front-end

Technologies

Multiple patterns for the same problems

Long compile time

Large source control

repositories

Long-running unit tests

Too much coupling

Limitedteam scalability

Devs need to know everything

No

Isola

tion

Productivity drops over time.

• The network is reliable

• Latency is zero

• Bandwidth is infinite

• The network is secure

• Topology doesn't change

• There is one administrator

• Transport cost is zero

• The network is homogeneous.

https://en.wikipedia.org/wiki/Fallacies_of_distributed_computing

The First Law of Distributed Computing

Very scalable, very reliable

Very difficult to debug

distributed problems

Unreliable network requires

complicated techniques

Requires message

bus/broker

End-to-end testing very unpractical

Very mature DevOps culture is a prerequisite

Advanced monitoring and

visualization is a must

Security can not be an afterthought

High operational maintenance.

Can be deployed/upgraded

independently.

Monolith

Microservice

Microservice

Microservice

HTTP API

HTTP API

HTTP API

OWIN component released through

MyGet/Nuget

No network I/O at all

Can have its own database instance, shared with monolith,

or shared storage service

Microservice ownsthe schema

Separate repos with distinct owners.

Runs in-process, thus easier debugging

New version requires

redeployment of the monolith.

Loose coupling through HTTP

Less scalability options

HTTP is not the fastest

serialization format

Relies on libraries, not frameworks.

Client

IIS

Console

WinSvc

Unit Test

HTTP request

HTTP response

Middlewareenvironment

handler

next

Middleware

IDictionary<string, object>

Func<IDictionary<string, object>, Task>

Task

Func<Func<IDictionary<string, object>, Task>, Func<IDictionary<string, object>, Task>

>

Monolith

Microservice

HTTP API

Service Registry

Contract documented through

Swagger

Webhooks

Microservice

Microservice

Provides OWIN-aware HttpClient to connect to service

Event payload exposed as Swagger

(e.g. SwaggerHub)

Subscribers register URLs or in-process

OWIN message handler

API/payload consumers use liberal JSON interpretation

Circuit Breaking to handle unreliable

consumers

At-least-once payload delivery

Subcribers can request replays.

Payload includes ID to help idempotency

Each request requires correlation ID that flows

through all APIs, webhooksand services

Aggregates

Command Handlers

Queries

Commands

Events

Upconverters

Value Objects

Query Handlers

SchemaMigrations

Projectors

DataImporters

Query API

Jobs

Projections

HTML

CSS

JS

API Controllers

Projectors

Projections

RegistrationsWebhook API

Command API

Event Storage

Payload Definitions

Outer layers depend on inner layers

Domain is persistency ignorant

Can be tested independently

For eventual consistent business

rulesMaster data

To support domain queries

For microservices that provide a (partial) UI or

management screen

Based on Domain Driven Design

Supported by LibLog, TinyIoc

Designed according to the 12 Factor

principles

UI / HTTP APICommand

Service

ChangeUserEmailHandler

UserUnit of Work

User Projector

DAL

Read DB

Write DB

ChangeUserEmailCommand

Execute query

Get<User>(identity)

Invoke method

Event Store

Load(events)

Apply

Get changes

Dispatcher Events

Handle(UserEmailChangedEvent)

Submit changes

History

Dennis Doomen | @ddoomen | The Continuous Improver

Monolith

Microservice

HTTP API

Event Store

Events

Payload

Projector

Subscription Request

Payload ‘event store’

Projectors use LiquidProjections

Have their own checkpoints

Projects fine-grained events into payload

projections

Projector

Payload schema is documented using

Swagger

ProjectorProjector

Webhook Projectors

Separate instance per subscription

Subscriber HTTP API

‘Project’ payload ‘event store’ into

HTTP calls

Projected Data

Uses the incoming payloads as an ‘event

store’

Payload

Use Circuit Breakers to handle

slow/unavailable subscribers

SQL DB Schema changes using

FluentMigratoror using NoSQL

DB

Domain

Event Store

Events

App

RDBMS

EventsProjection

EventsProjectionProjection

ProjectorOptimized for

specific queries

Separate projections

database NoSQL

Projector

Projection-specific storage

Projector

HTML

Raw SQL or Dapper

Run asynchronously

Great for sharding

Dennis Doomen | @ddoomen | The Continuous Improver

Events

Transaction 6

Transaction 5

Transaction 4

Transaction 3

Transaction 2

Transaction 1

Temporal Projector

Read Store

Time

Projected until this point

Immutable, thus auditable and SOX

compliance

Dennis Doomen | @ddoomen | The Continuous Improver

Domain

Event Store

Events

App

EventsProjection

EventsProjectionProjection

ProjectorApplication projections

RDBMS

Reporting Projector

Traditional reporting model

Asynchronous

OLAP

Dennis Doomen | @ddoomen | The Continuous Improver

StreamId: User-Dedo, Rev: 5

Persisted State Changes

StreamId: User-Dedo, Rev: 4

Grant Role “Editor”

Change Password 2nd

StreamId: User-Dedo, Rev: 4

StreamId: User-Dedo, Rev: 3

Create User

Grant Role “Owner”

Add Phone Number

Change Password 1st

StreamId: User-Dedo, Rev: 5

Revoke Role “Owner”

StreamId: User-Dedo, Rev: 6

Grant Role “Editor”

Time

StreamId: User-Dedo, Rev: 7

Changed Password 2nd

Dennis Doomen | @ddoomen | The Continuous Improver

Events

1

2

3

6297

6298

6298

Projections

Documents

Groups

Users

Objects

Folders

Microservice V2

Changes Queries

Events

1

2

3

6299

6300

6301

Projections

Documents

Groups

Users

Objects

Folders

Microservice V1

ChangesQueries

Migration Process

Dennis Doomen | @ddoomen | The Continuous Improver

Events

Aggregate Root

Entity

Entity

Value Object

Aggregate Root

Aggregate Root

Aggregate Root

Entity

Value Object

Value Object

Dennis Doomen | @ddoomen | The Continuous Improver

• Asynchronous projections

• Idempotency of projections

• Self-tracking

• Projection autonomy -> more duplication

• Projection tracking and prediction.

• Functional archiving

• Clear separation between critical and auxiliary data -> prioritization

• Partitioning

• Dynamic rebuilding (after bugs, schema changes, etc).

• Projections that crash– Column constraints

– Changes in data invariants

– Unexpected projection dependencies

– Lookups vs stream dependencies.

• (Unexpected) projection dependencies– Synchronous -> asynchronous and existing

bugs

– Aggregrate dependencies.

• Bugs – Causing large event streams.

Prerequisites

- Microservice owns ‘schema’

- Supports upgrading and downgrading

- NoSQL preferred, but RDBMS can work too.

Rebuild schema in-place

– Rebuilding using event store

– Requires tracking API

– Involves down-time.

Disallow breaking changes

– Only new ‘columns’ and ‘tables’

– Partial rebuilding using event store

– Requires forwards compatibility.

Rebuild side-by-side

– Rebuilding using event store

– Requires tracking API

– Requires compatibility with previous ‘schema’

– Previous ‘schema’ is removed in the next release.

Microservice

Bunch of classes that should be used directly

X

No inheritance

Uses composition over inheritance

internally

Convenience classes that don’t hide the

magic

Library

Abstract Package

Interfaces

Delegates

DTOs

Only depend on abstract packages…

Stable Package

…or depend on more stable packages

Auxiliary Package

Classes that are not used together do not

belong togetherOptional Dependency

DependencyPackage

Consumers should not be faced with

optional dependencies

• Delegates

• Source-only Packages

• Merge dependencies

Monolith

Microservice

Microservice

Microservice

HTTP API

HTTP API

HTTP API

Apply routing conventions

Versioning using Accept Headers,Response Headers or

Routes

Optimize body using AVRO or Protobuf

Cross-service tracing using OpenTracing

Replace OWIN with (upcoming) in-process gRPC

Common contracts for diagnostics,

monitoring

Add KPI dashboards, e.g. Sumologic

1 Build a (container-based) cloud capability.

2 Deploy monolith to the cloud.3 Automate your delivery pipeline.4 Full end-to-end responsibility5 Break-up delivery teams and code6 Evaluate status-quo on micro-services

(products, platforms, etc)7 Slide-and-dice microservices one-by-

one.

• Liquid Projections – The Examplehttps://github.com/liquidprojections

• Bliki: MonolithFirsthttps://dzone.com/articles/martin-fowler%E2%80%94bliki

• In-process gRPChttps://github.com/grpc/grpc-go/issues/906

• Protobuf in WebAPIhttp://www.infoworld.com/article/2982579/application-architecture/working-with-protocol-buffers-in-web-api.html

• Ingredients for well-design OWIN componentshttp://www.continuousimprover.com/search/label/OWIN%20Recipes

• The good, the bad and the ugly of Event Sourcinghttp://www.continuousimprover.com/search/label/event%20sourcing