Marcel de VriesCTO Xpirit

Architecting systems for

Continuous delivery

About me: Marcel de Vriesmdevries@xpirit.com

@marcelv

http://fluentbytes.comXpirit

Also regional director

AGENDA• Continuous delivery maturity model

– How fast do you need to go?

• Architectural concepts to solve bottlenecks– Smaller shippable units– Decreasing hard coupling between (sub) systems– Enabling zero downtime deployments– Different thinking around releases– Different thinking around testing

• Better operations by build in monitoring• Call to action

Source: http://www.infoq.com/resource/articles/Continuous-Delivery-Maturity-Model

Where do you want to be?

Where does your organization need to be?

How fast?

• How often do you need to deploy?

– Look at it from your organization perspective!

Gartner differentiation of systems

Systems of Innovation

Systems of Differentiation

Systems of Records

Innovative systems meant to introduce new concepts or new

business

High release cadence (Daily)

System lifespan: 6 months to 3 year

Support primary business functions (a.k.a. back office)

Low release cadence (quarterly, half year release cycles)

System lifespan: 10 to 20 years

Deliver moderate competitive advantage

Average release cadence (monthly-weekly)

System lifespan : 3 to 5 year

Syste

ms o

f Inn

ovatio

n

Syste

ms o

f Differen

tiation

Syste

ms o

f Reco

rds

Source: http://www.infoq.com/resource/articles/Continuous-Delivery-Maturity-Model

Many of these items have Architectural implications

Organizational challenges

Valued business drivers

Better

• Resilient

• Technology choice

Cheaper

• Test effort

• Cost of maintaining

Faster

• Change

• Deployment

• Execution

Dev Test Acceptance Production

Every step between silos adds waste & lead time to the delivery processWhy are we doing this?

Forces on shipping

Developer

Project managers

Tester

OperationsThe Product

Direction of shipping the product

Business stakeholderBusiness stakeholder

How can we converge these forces to ship?

• Operations: ensure we don’t break stuff. This has multiple sides:– Ship smaller increments, since less changes, less change of breaking stuff– Automate delivery to production, since most problems are caused by

misconfiguration or wrong install– Build in monitoring so we can predict better how systems behave under

certain conditions

• Business stakeholder: Provide them faster flow of value, make them understand things can be delivered in increments and make them in control of what matters most for their business

• Testers: proof you deliver a quality product

Architectural challenges of CD

• Faster delivery by – Smaller shippable units

– Decreasing hard coupling between (sub) systems

– Enabling zero downtime deployments

– Different thinking around releases

– Different thinking around testing

• Better operations by build in monitoring

Smaller shippable units

?SOAMonolith

Smaller shippable units

• What is the architecture of your system?

Monolith• Dependencies

– All software is tightly coupled. Most often the system is build in the same software technology (e.g. .NET with C#)

– Often one database with strong schema dependency

• Deployment strategy– De-install, upgrade/migrate DB Schema, Install new version– System will be unavailable during deployment– Takes hours to complete

• Scaling strategy– Scale up and out

• Monitoring– Based on home grown monitoring tools

Monolith

How do we compose systems?

• We can create a monolith in various ways– Composed out of classes– Composed out of components– Composed out of services

• When are these systems no longer a monolith?– When we use components & Services?

• What defines the monolith is the way the system is composable and flexible– Can we add/remove parts without bringing the system down

SOA• Dependencies

– Software can be in multiple technologies, communication via a Service Bus

– Strong dependency on message schema’s– Strong dependency of services on Middleware

technology

• Deployment strategy– Per service

• De-install, upgrade/migrate DB Schema, Install new version• System will be unavailable during deployment

– Takes hours to complete & coordinated effort– Upgrades of ESB also requires coordinated effort

SOA

SOA improved• SOA suffered from vendors pushing their broker software and

making all services dependent on this piece of middleware• We see many examples where we have the ESB as the

centralized piece of middleware– This often forces us to use one stack, or we need to add adapters

to our services– Creates a single point of failure and often also a scalability

challenge

• What if we eliminate this broker and create a mesh of small services?

Enter the world of Microservices

MSA

The many definitions of microservices

- Small andfocused on doingone thing well

- Autonomous

“Loosely coupled service oriented architecture with bounded contexts”Adrian Cockcroft (Netflix)

“SOA done right”Anonymous

“… services are independently deployable and scalable, each service also provides a firm module boundary, even allowing for different services to be written in different programming languages.”Martin Fowler (Thoughtworks)

Services as a network of independent parts

Microservices architecture

Which Service owns this page?

None!

Composition is key!

Decreasing hard coupling between (sub) systems

Using events to decouple services

• Systems keep their own state

• Events are used to notify changes

• Systems use events to replicate data

• Each service will have it’s own data store

• Downsides:– Latency in data changes– No central control of business logic– Challenge to reconcile in case of disaster

Why are event driven architectures relevant?

Characteristics of EDA• Broadcast Model

– Anyone can receive the events

• Completely Asynchronous

– publishers do not wait on any subscribers

• Fine Grained Events

– Individual events

• Ontology

– Hierarchy of events

– Publishers can subscribe to these selected events

Publisher

SubscriberSubscriber

Subscriber

SubscriberSubscriber

Subscriber

EventEvent

Event

Cross-service coupling

• Smart endpoints, dumb pipes

• No intelligent middleware

• Technologies to look at:– Queues (Azure, MSMQ, WebSphereMQ , ...)

– Service Bus (NServiceBus, Azure ServiceBus, ...)

– Language neutral data serialization(Google Protocol buffers, Apache Thrift)

Enabling zero downtime deployments

Minimize downtime

• Decompose the system into more granular parts that can be deployed independently

– Better management of system dependencies

• Dependencies on other Systems

• Dependencies on own components/services

• Dependencies on OS and infrastructure

Minimize downtime• Rethink how your application works• Traditional

– When deploying we need to upgrade the Database schema• Downtime

• Potential Solutions: – Use other database technologies that don’t require a strict

schema– Make your system more flexible towards the DB schema, e.g.

support multiple version of the schema– Upgrade the schema when the data is touched (just in time

migration)

Minimize downtime• In SOA systems become less dependent on

strict message schema– E.g. XML + CSD v.s JSON

– Get rid of the ESB dependency

• Cater for failure in downstream systems an provide fallback scenarios– Well known pattern here is e.g. Circuit breaker

– Let systems fall back gracefully

Circuit Breaker Pattern

• Handle faults that may take a variable amount of time to rectify when connecting remote service or resource.

• to a – This pattern can

improve the stability and resiliency of an application.

Retry Pattern • Enable an application to handle temporary failures

when connecting to a service or network resource by transparently retrying the operation in the expectation that the failure is transient. This pattern can improve the stability of the application.

Different thinking around releases

Use of feature flags/toggles• When you want to ship often, how do you cope

with non finished features?• Traditionally you use feature branches to isolate

– But what if you ship daily or weekly?

• Break up the changes in independent parts that can be released separately– Use feature toggles/flags to enable or disable the

feature– Give business control on who gets which feachures

Don’t build your own, many out there

• If-Else

• Nfeature (LGPL-3.0!, last update june 2012)

• Feature Toggle (Apache 2.0, last update dec.2014)

• Feature Switcher (Apache 2.0, Last update Jan 2014)

• FlipIt (Apache 2.0, Last update feb 2012)

http://www.philjhale.com/2012/07/a-brief-look-at-some-feature-toggle.html

Different thinking about (Automated) testing

Classical approach

• Unit testing

• Integration testing

• Manual testing

• CD tells us: Automate everything….

– Move to more automation

– Manual testing -> Use of UI Automation tools

“There is no place like Production”

Source: Brian Harry, Microsoft Technical Fellow

Testing in production

SU1

Feature

SU2 SU3

Feature

Feature

Feature

Feature

Feature

Feature

Feature

Feature

Feature

Feature

Feature

Feature

Feature

Feature

Feature

Feature

Feature

Feature

Feature

Feature

Feature

Feature

Feature

Feature Feature Feature

Customer segment A<1000

Customer segment B<10000

Customer segment CAll other

Choosing the right scale unit is crucial to enable this concept

Test in production• A/B Testing is a simple example• Azure supports this out of the box

– Put your new code in production– Throttle a small % of users to

the new code– Watch behavior

• Telemetry!

– No issues?• Slowly move all users over

• MS uses it extensively– VSO, Windows 10 insiders program, etc.

Better operations by isolation and build in monitoring

Hosting your services

Host

Application container

MicroService

MicroService

Hosting in application container

Host

MicroService

MicroService

MicroService

MicroService

High density hosting

Host

MicroService

Host

MicroService

Host

MicroService

Host

MicroService

One service per host

Services up and running

OSVirtual

machinesPaaS

Container runtimes

Monitoring build in

Performance UsageAvailability Diagnostics

Application Insights

Entering (Biz)DevOps

• Multidisciplinary teams

• “You build it, you run it”

– No more ops needs to fix it, you do it your self!

• Teams own a set of services

• Coordination cross teams where required

Recap• Establish a reference point for your system where you need to go in CD

– How fast do you need to go, no free lunch!

• Think of the organizational implication, it’s not only technology• Decompose your system in more independent deployable units

– Make services smaller– Rethink your dependencies between services– E.g. Adapt event driven style of communication– Make services failure resilient– Isolate them by minimizing environment dependencies– Ensure services can be monitored

• Rethink the way systems are tested and implications on your systems architecture

Further reading

• Microservice architecture, Sam Newman• Domain Driven Design, Eric Evans• Microservices, Martin Fowler• Continuous Delivery. Jez Humble• Xpirit magazine

– Azure service fabric, Micro services articles

Your feedback is important!

Scan the QR Code and let us know via the TechDays App.

Laat ons weten wat u van de sessie vindt!

Scan the QR Code via de TechDays App.

Bent u al lid van de Microsot Virtual Academy?! Op MVA kunt u altijd iets nieuws leren over de laatste technologie van Microsoft. Meld u vandaag aan op de MVA Stand. MVA biedt 7/24 gratis online training on-demand voor IT-Professionals en Ontwikkelaars.