vJUG24 2017 "Continuous Delivery Patterns for Contemporary Architecture"

Continuous Delivery Patterns for Modern Architectures and Java

Daniel Bryant

@danielbryantuk

Architecture: Expectations versus reality

25/10/2017 @danielbryantuk

“DevOps”

tl;dr

• We are moving from complicated systems to complex systems

• Architecture is becoming more about technical leadership

• We must encode all requirements into a continuous delivery pipeline


@danielbryantuk

• Independent Technical Consultant, CTO at SpectoLabs

• Architecture, DevOps, Java, microservices, cloud, containers

• Continuous Delivery (CI/CD) advocate

• Leading change through technology and teams


bit.ly/2jWDSF7

http://bit.ly/2jWDSF7

Setting the Scene


Continuous Delivery

• Produce valuable and robust software in short cycles

• Optimising for feedback and learning

• Not (necessarily) Continuous Deployment


Velocity (with stability) is key to business success

“Continuous delivery is achieved when stability and speed can satisfy business demand.

Discontinuous delivery occurs when stability and speed are insufficient.”

- Steve Smith (@SteveSmithCD)



Feedback: - Was our hypothesis proven?- How can we improve biz,

architecture and ops?

Let’s bring in some containers (or additional packaging)


What are our core challenges with modern Java app architectures?



Gated application deployment

1.

2.

3.


Independent service deployment

Architecture


From monolith to…


UI / Biz / Repo

Monolith

Domains

Modules, components,frameworks, libraries

http://scs-architecture.org/

http://scs-architecture.org/


Self Contained Systems (SCS)

Microservices

Function-as-a-Service“Serverless”


Monoliths SOA Microservices / SCS FaaS / Serverless

Scope Project Enterprise Product Feature (or glue?)

Focus Swiss Army Knife Reuse, governance, control

Domain modelling, SRP, evolution

Function (in/out),rapid evolution

Organisation Implemented and maintained (typically) by single team

Implemented by different org units. Maintenance done elsewhere

Services implemented and owned by product teams

Implemented by pioneers (hipsters?)

Deployment Monolithic deployment Monolithic orchestration of multiple services

Services deployed individually

Functions deployed individually

Management None Centralised Distributed Chaotic / Orchestrated

Inter-process communication

None RPC or messaging, typically via middleware (ESB/MQ)

RPC via dumb pipesand smart endpoints, messaging/events

Events

Pioneers / stewards

Organisations, community or individuals

Enterprises and Vendors

Community and highperf organisations

Vendors/community

Core ArchitecturalConstraints

Language and framework Canonical domain model, standards

Interoperability Cost (Gbs/ms)

Architecture fundamentals

• Coupling• ”Components have little or no knowledge of other components”

• Interfaces

• Schema

• Cohesion • “Degree to which the elements within a component belong together”

• Single reason to change

• Separation of concerns


Coupling, Cohesion and Continuous Delivery

• Design• Cohesion makes reasoning easy

• Loose coupling reduces impact

• Build, unit and integration• Cohesion limits dependencies

• Loose coupling allows simulation

• End-to-end tests• Cohesion/coupling orchestration

• Deployment• Cohesion minimises number of

components in play

• Coupling leads to “monoliths”

• Observability• Cohesive is easier to understand

• High coupling typically leads to large blast radius and “murder mystery” style debugging


A journey of a thousand miles…


Working locally


https://cloud.google.com/functions/docs/emulatorhttps://azure.microsoft.com/en-gb/blog/announcing-the-preview-of-java-support-for-azure-functions/https://github.com/awslabs/aws-sam-local

https://cloud.google.com/functions/docs/emulator

https://azure.microsoft.com/en-gb/blog/announcing-the-preview-of-java-support-for-azure-functions/

https://github.com/awslabs/aws-sam-local

Developing/Integrating locally


https://news.ycombinator.com/item?id=13960107https://opencredo.com/working-locally-with-microservices/https://www.datawire.io/telepresence/ | https://hoverfly.io/

https://news.ycombinator.com/item?id=13960107

https://opencredo.com/working-locally-with-microservices/

https://www.datawire.io/telepresence/

https://hoverfly.io/

Testing


Testing: Core concepts


/lisacrispin.com/2011/11/08/using-the-agile-testing-quadrants/martinfowler.com/bliki/TestPyramid.html

http://lisacrispin.com/2011/11/08/using-the-agile-testing-quadrants/

https://martinfowler.com/bliki/TestPyramid.html

Testing: Core concepts


martinfowler.com/articles/microservice-testing/#testing-progress-3

https://martinfowler.com/articles/microservice-testing/#testing-progress-3

Functional testing: End-to-end


Functional Testing: Outside-in


https://specto.io/blog/2016/8/16/recipe-for-designing-building-testing-microservices/http://www.thucydides.info/docs/serenity/

https://specto.io/blog/2016/8/16/recipe-for-designing-building-testing-microservices/

http://www.thucydides.info/docs/serenity/

Functional


Talking of contracts…


Talking of (service) contracts…

• APIs are service contracts

• Consumer-driven Contracts

• martinfowler.com/articles/consumerDrivenContracts.html


https://martinfowler.com/articles/consumerDrivenContracts.html

Talking of (service) contracts…

25/10/2017 @danielbryantukdocs.pact.io

cloud.spring.io/spring-cloud-contractgithub.com/spring-cloud-samples/spring-cloud-contract-samples

https://docs.pact.io/

https://cloud.spring.io/spring-cloud-contract/

https://github.com/spring-cloud-samples/spring-cloud-contract-samples

Talking of (messaging) contracts…

• What about messaging?

• Message schema are an API

• www.infoq.com/presentations/contracts-streaming-microservices


https://www.infoq.com/presentations/contracts-streaming-microservices

Talking of (messaging) contracts…


www.infoq.com/presentations/contracts-streaming-microservices

docs.confluent.io/current/schema-registry/docs/maven-plugin.html

https://www.infoq.com/presentations/contracts-streaming-microservices

https://docs.confluent.io/current/schema-registry/docs/maven-plugin.html

Contract verification

• Is it worth the cost?

• Tradeoff: Trust and comms• Startups / SMEs

• Enterprises…

• My (anecdotal) experience

• Choreography vs orchestration



https://netflix.github.io/conductor/http://rockscript.io/https://nifi.apache.org/docs.htmlhttps://aws.amazon.com/documentation/step-functions/http://camel.apache.org/

http://rockscript.io/


https://nifi.apache.org/docs.html



Contract verification

• Is it worth the cost?

• Tradeoff: Trust and comms• Startups / SMEs

• Enterprises…

• My (anecdotal) experience

• Choreography vs orchestration

• Choreography• Asserting behaviour (interactions)

• Contracts are part of this!

• Orchestration:• Asserting state (output)

• Verify orchestration DSL


Testing NFRs in the build pipeline

• Architecture quality• SonarQube / Code Climate

• Performance and Load testing • Gatling / Locust / Bees with m’guns

• Security testing • OWASP Dependency check / Bdd-security

• Docker Bench for Security / CoreOS Clair



www.owasp.org/index.php/OWASP_Dependency_Check

https://www.owasp.org/index.php/OWASP_Dependency_Check


github.com/arminc/clair-scanner

https://github.com/arminc/clair-scanner

Delaying NFRs to the ‘Last Responsible Moment’

Newsflash!

Sometimes the

last responsible moment

is up-front

Modern architectures don’t necessarily make this easier


Wrapping Up




Scope Project Enterprise Product Feature (or glue?)

Focus Swiss Army Knife Reuse, governance, control

Domain modelling, SRP, evolution

Function (in/out),rapid evolution

Organisation Implemented and maintained (typically) by single team

Implemented by different org units. Maintenance done elsewhere

Services implemented and owned by product teams

Implemented by pioneers (hipsters?)

Deployment Monolithic deployment Monolithic orchestration of multiple services

Services deployed individually

Functions deployed individually

Management None Centralised Distributed Chaotic / Orchestrated

Inter-process communication

None RPC or messaging, typically via middleware (ESB/MQ)

RPC via dumb pipes/smart endpoints, messaging/events

Events

Pioneers / stewards

Organisations, community or individuals

Enterprises and Vendors

Community and highperf organisations

Vendors/community

Core ArchitecturalConstraints

Language and framework Canonical domain model, standards

Interoperability Cost



Cohesion and coupling enforced at modules

CD focuses on end-to-end functionality

Provide goals and “best practice” examples

Cohesion and coupling enforced at

service level

CD focuses on service integrity

Provide objectives and standards


service API level

CD focuses on service interaction

Provide principles and guidelines


function API level

CD focuses on service output/state

Provide principles and guidelines

In conclusion…

• We are moving from complicated systems to complex systems• Design and test with coupling and cohesion in mind

• Architecture is becoming more about technical leadership• Recognise your situation and influence accordingly

• We must encode all requirements into a continuous delivery pipeline• Both functional and nonfunctional requirements


Bedtime reading…


blog.christianposta.com/microservices/low-risk-monolith-to-microservice-evolution-part-iii/

http://blog.christianposta.com/microservices/low-risk-monolith-to-microservice-evolution-part-iii/

Thanks for listening…

Twitter: @danielbryantuk

Email: [email protected]

Writing: https://www.infoq.com/profile/Daniel-Bryant

Talks: https://www.youtube.com/playlist?list=PLoVYf_0qOYNeBmrpjuBOOAqJnQb3QAEtM


bit.ly/2jWDSF7

Coming soon!

mailto:https://twitter.com/danielbryantuk

mailto:[email protected]

https://www.infoq.com/profile/Daniel-Bryant

https://www.youtube.com/playlist?list=PLoVYf_0qOYNeBmrpjuBOOAqJnQb3QAEtM

http://bit.ly/2jWDSF7

vJUG24 2017 "Continuous Delivery Patterns for Contemporary Architecture"

Technology

Transcript of vJUG24 2017 "Continuous Delivery Patterns for Contemporary Architecture"