Designing and building a micro-services architecture. Stairway to heaven or a highway to hell?

@aahoogendoorn

DESIGNING AND BUILDING A MICRO-SERVICES ARCHITECTURE. STAIRWAY TO HEAVEN OR A HIGHWAY TO HELL?Sander Hoogendoornditisagile.nlMentoring ▪ Consulting ▪ Training ▪ Agile ▪ Software architecture ▪ Code

MICROSERVICES? STAIRWAY TO HEAVEN OR HIGHWAY TO HELL?©2015 ditisagile.nl. All Rights Reserved 2@aahoogendoorn

www.ditisagile.nl

Sander Hoogendoorn

Me Dad Mentor, trainer, software architect, programmer Books, articles, conferences

Work Owner ditisagile.nl CTO Klaverblad Verzekeringen [PTO Capgemini] [Global design authority agile Capgemini]

Web www.sanderhoogendoorn.com www.smartusecase.com www.speedbird9.com @aahoogendoorn [email protected]


www.ditisagile.nl

@aahoogendoorn

MONOLITHS Hard to deliver, even harder to test and impossible to maintain


www.ditisagile.nl

Monoliths

Advantages A single (layered) architecture

A single technology stack

A single code base maintained by multiple teams

Disadvantages All parts are interconnected

Many other systems are connected to your system

Hard to change, hard to maintain

Long time between releases, thereby increasing risks

Slow innovation

Hard to move to newer technologies

Doesn’t scale very well

Product Account

Order Customer

Products Accounts

Orders Customers


www.ditisagile.nl

Dependencies will kill you

@aahoogendoorn

A BRIEF HISTORY OF COMPONENTS AND SERVICES


www.ditisagile.nl

Client server


www.ditisagile.nl

Component based development


www.ditisagile.nl

Service oriented architecture


www.ditisagile.nl


www.ditisagile.nl

Microservices

@aahoogendoorn

MICROSERVICESBeyond the hype?


www.ditisagile.nl

Microservices. Beyond the hype?


www.ditisagile.nl

Gartner hype cycle

@aahoogendoorn

MICROSERVICESThe clear benefits


www.ditisagile.nl

BUT FIRST… A DEFINITION


www.ditisagile.nl

In short, the microservice architectural style is an approach to developing a single application as a suite of small services, each running in its own process and communicating with lightweight mechanisms, often an HTTP resource API.

These services are built around business capabilities and independently deployable by fully automated deployment machinery. There is a bare minimum of centralized management of these services, which may be written in different programming languages

and use different data storage technologies.Martin Fowler


www.ditisagile.nl

Monoliths. Scalability

Product Account

Order Customer

Product Account

Order Customer

Product Account

Order Customer


www.ditisagile.nl

Microservices. Scalability

Product Account

OrderCustomer


www.ditisagile.nl


Product Account

OrderCustomer

Product

CustomerCustomer Customer

Account Account


www.ditisagile.nl


Product Account

Order

Product

Customer

Account Account

CustomerCustomer


www.ditisagile.nl

Monoliths. Persistence

Product Account

Order Customer

Products Accounts

Orders Customers


www.ditisagile.nl

Microservices. Polyglot persistence

Product Account OrderCustomer

MongoDB

Customers

MongoDB

Orders

Active Directory

Accounts

Oracle

Products


www.ditisagile.nl

Microservices. Promises

Products not projects

Scalable

Decentralized governance

Replaceable parts

High performance

Technology independent

Polyglot persistence

Easy to build

Easy to test

Easier deployment than monoliths

ProductAccount

Order

Customer


www.ditisagile.nl

Microservices. But…

What is a microservice exactly?

How small is a microservice?

Requirements in a microservice world

Components or services

Who owns a microservice?

What technologies do you use?

What protocols do you apply?

How to define messages

How to test microservices

How to coordinate when business services run across components?

How to build deployment pipelines?

ProductAccount

Order

Customer


www.ditisagile.nl

Opinions, opinions, opinions

@aahoogendoorn

ARE MICROSERVICESA STAIRWAY TO HEAVEN?

@aahoogendoorn

OR A HIGHWAY TO HELL?

@aahoogendoorn

TWO REAL WORLD CASES


www.ditisagile.nl

Case 1. A major insurance company


www.ditisagile.nl

Where do we come from?


www.ditisagile.nl



www.ditisagile.nl

Outsourcing didn’t work


www.ditisagile.nl

Where do we go?


www.ditisagile.nl

Case 2. A product development company


www.ditisagile.nl


#NoArchitecture


www.ditisagile.nl



www.ditisagile.nl

FOR THE THINGS WE HAVE TO LEARN BEFORE WE CAN DO THEM, WE LEARN BY DOING THEM

Aristotle

@aahoogendoorn

SO WHAT DID WE LEARN?


www.ditisagile.nl

MICROSERVICES REQUIRE AN EVOLUTIONARY ARCHITECTURE

@aahoogendoorn

START WITH SOME GUIDING PRINCIPLES


www.ditisagile.nl

Case 1. Our guiding principles

We decided to go from here Client thinks in business processes, so we implement

business processes We move away from the mainframe, to a new systems

landscape, consisting of micro-applications and micro-components

Requirements and documentation are modeled rather than written

Applications implement a single (elementary) business process

Components serve a single purpose and offer services Applications and components all have their own bounded

context – a domain model Applications and components will have an similar internal

software architecture to facilitate ease of maintenance and allow for harvesting re-use

Communication between applications and components will use a simple open protocol - REST

@aahoogendoorn

BUSINESS PROCESSES FIRST


www.ditisagile.nl

Smart use cases


www.ditisagile.nl

Presentation

Process

Domain

Services

Outside world

PagesGrids / Panels, Controls

Use casesFlow

Domain objects, Factories / RepositoriesEnums / Value objects / Tupels / Reference objects

Service gateways, Service clientsInfo objects / Search objects

ComponentsRelations Dossiers Intermediaries Accounts Rates


www.ditisagile.nl

Service interface

Process

Domain

Data / Services

Outside world

ResourcesRepresentations

Use casesFlow

Domain objects, Factories / RepositoriesEnums / Value objects / Tupels / Reference objects

Storage gateways, Storage clientsInfo objects / Search objects

StorageRelations Dossiers Intermediaries DB2 MongoDB


www.ditisagile.nl

Case 2. Architectural requirements

Requirements Give clear and straightforward guidance to developers and

testers

Clear separation of concerns

Unify architecture over different applications

Ready for future mobile (and web) applications

Decoupling applications and components

Single access point to business logic and bounded contexts

Scaleable architecture

Easier to test applications and components

Improve quality of code

Maintainability of code

Easier to extend existing functionality

Minimize replications of data

@aahoogendoorn

ARCHITECTURE FIRST


www.ditisagile.nl

Case 2. Current architectural layout

App 1 App 2 App 3 App 4 App 5 App 6 App 7 App 8 App 9

Database

Controllers

Data access


www.ditisagile.nl

Case 2. New architectural layout

App 1Mobile

App 2 App 3 App 4 App 5 App 6 App 7 App 8 App 9

DB

Component ComponentComponent Component ComponentComponent Component Component

DB DB DB DB DB DB DB

REST REST


www.ditisagile.nl

Case 2. Brownfield migration…

App 1 App 2 App 3 App 7 App 8 App 9

Database

Controllers

Data access

…

License Account

App New

@aahoogendoorn

MODULAR DESIGN / DESIGNING MICROSERVICES


www.ditisagile.nl

DOING BIG UP-FRONT DESIGN IS DUMBDOING NO DESIGN IS EVEN DUMBER

Dave Thomas


www.ditisagile.nl

BOUNDED CONTEXTS


www.ditisagile.nl

The single unified domain model

Product

Vendor

StockOrder

Client

Delivery

Payment


www.ditisagile.nl

Bounded contexts

Product

Vendor

StockOrder

Client

Delivery

Payment

Product


www.ditisagile.nl

65

Bounded context

@aahoogendoorn

MODELING RESOURCES


www.ditisagile.nl

69Resource model


www.ditisagile.nl

GOOD COMMUNICATION IS HARD

@aahoogendoorn

RESTFUL COMMUNICATION IS NOT THAT EASY


www.ditisagile.nl

HTTP RETURN CODES CHEAT SHEET1**. Hold on

2**. Here you go 3**. Go away

4**. You fucked up 5**. I fucked up


www.ditisagile.nl

BE CONSERVATIVE IN WHAT YOU SEND, BE LIBERAL IN WHAT YOU ACCEPT

Postel’s Law


www.ditisagile.nl

Case 2. Contractual guidelines … but no REST

Resource Resource

Controller Controller Controller

Service interface Service interface

Service gateway Service gatewayApplication

Contract

Component

@aahoogendoorn

TESTING MICROSERVICES


www.ditisagile.nl

A service development lifecycle

Code Developer Test Test Integration Test Acceptance Test LivePrepare & Design


www.ditisagile.nl

What to test


DevelopersUnit tests

DevelopersQ & A

TestersScenario’s & API’s


Product ownerProduct


www.ditisagile.nl

Even though you might have brilliant testers


www.ditisagile.nl

… please automate your tests


DevelopersUnit tests

DevelopersQ & A



Product ownerProduct


www.ditisagile.nl

WHAT ABOUT BEING INDEPENDENTLY DEPLOYABLE?

@aahoogendoorn

DEPLOYING MICROSERVICESContinuous integration, build pipelines and continuous delivery


www.ditisagile.nl

A typical build pipeline



www.ditisagile.nl

ProductionAcceptanceIntegrationTestDevelopment

A typical build pipeline



www.ditisagile.nl

Build pipelines in Jenkins


www.ditisagile.nl

Microservices. Building a deployment pipeline

Code Developer Test Test Acceptance Test Acceptance Live





www.ditisagile.nl

Microservices. Pipeline hell?

Code v.2 Developer Test v.2 Test v.2 Acceptance Test v.2 Acceptance v.2


Test v.2 Acceptance Test v.2 Acceptance v.2 Live v.2

Developer Test Test Acceptance Test Acceptance Live

Code v.3 Developer Test v.3

Live

Code v.2


www.ditisagile.nl

Maintenance

From projects to continuous delivery?

Project

MaintenanceMVP

MaintenanceContinuous delivery


www.ditisagile.nl

There is no one tool to rule them all


www.ditisagile.nl

Moving towards DevOps is not easy

@aahoogendoorn

SOME LAST THOUGHTS


www.ditisagile.nl

Microservices are not for everyone

What are we going to benefit from the most?

Polyglot persistence?

Independentlydeployable?

Independentlydeployable?

Enforcemodular design?


www.ditisagile.nl

Minimal viable product

@aahoogendoorn

ALLOW YOUR TEAMS TO LEARN CONTINUOUSLY…


www.ditisagile.nl

The hockey stick model

@aahoogendoorn

… AND HAVE FUN

@aahoogendoorn

www.sanderhoogendoorn.comwww.smartusecase.comwww.speedbird9.com

[email protected]

@aahoogendoorn

REFERENCES AND QUESTIONS

Designing and building a micro-services architecture. Stairway to heaven or a highway to hell?

Software

Transcript of Designing and building a micro-services architecture. Stairway to heaven or a highway to hell?