Post on 21-Jan-2018
Continuous Delivery Patterns for Modern Architectures and Java
Daniel Bryant
@danielbryantuk
Architecture: Expectations versus reality
25/10/2017 @danielbryantuk
“DevOps”
tl;dr
• We are moving from complicated systems to complex systems
• Architecture is becoming more about technical leadership
• We must encode all requirements into a continuous delivery pipeline
25/10/2017 @danielbryantuk
@danielbryantuk
• Independent Technical Consultant, CTO at SpectoLabs
• Architecture, DevOps, Java, microservices, cloud, containers
• Continuous Delivery (CI/CD) advocate
• Leading change through technology and teams
25/10/2017 @danielbryantuk
bit.ly/2jWDSF7
Setting the Scene
25/10/2017 @danielbryantuk
Continuous Delivery
• Produce valuable and robust software in short cycles
• Optimising for feedback and learning
• Not (necessarily) Continuous Deployment
25/10/2017 @danielbryantuk
Velocity (with stability) is key to business success
“Continuous delivery is achieved when stability and speed can satisfy business demand.
Discontinuous delivery occurs when stability and speed are insufficient.”
- Steve Smith (@SteveSmithCD)
25/10/2017 @danielbryantuk
25/10/2017 @danielbryantuk
Feedback: - Was our hypothesis proven?- How can we improve biz,
architecture and ops?
Let’s bring in some containers (or additional packaging)
25/10/2017 @danielbryantuk
25/10/2017 @danielbryantuk
What are our core challenges with modern Java app architectures?
25/10/2017 @danielbryantuk
25/10/2017 @danielbryantuk
Gated application deployment
1.
2.
3.
25/10/2017 @danielbryantuk
Independent service deployment
Architecture
25/10/2017 @danielbryantuk
From monolith to…
25/10/2017 @danielbryantuk
UI / Biz / Repo
Monolith
Domains
Modules, components,frameworks, libraries
http://scs-architecture.org/
25/10/2017 @danielbryantuk
Self Contained Systems (SCS)
Microservices
Function-as-a-Service“Serverless”
25/10/2017 @danielbryantuk
Monoliths SOA Microservices / SCS FaaS / Serverless
Scope Project Enterprise Product Feature (or glue?)
Focus Swiss Army Knife Reuse, governance, control
Domain modelling, SRP, evolution
Function (in/out),rapid evolution
Organisation Implemented and maintained (typically) by single team
Implemented by different org units. Maintenance done elsewhere
Services implemented and owned by product teams
Implemented by pioneers (hipsters?)
Deployment Monolithic deployment Monolithic orchestration of multiple services
Services deployed individually
Functions deployed individually
Management None Centralised Distributed Chaotic / Orchestrated
Inter-process communication
None RPC or messaging, typically via middleware (ESB/MQ)
RPC via dumb pipesand smart endpoints, messaging/events
Events
Pioneers / stewards
Organisations, community or individuals
Enterprises and Vendors
Community and highperf organisations
Vendors/community
Core ArchitecturalConstraints
Language and framework Canonical domain model, standards
Interoperability Cost (Gbs/ms)
25/10/2017 @danielbryantuk
Monoliths SOA Microservices / SCS FaaS / Serverless
Scope Project Enterprise Product Feature (or glue?)
Focus Swiss Army Knife Reuse, governance, control
Domain modelling, SRP, evolution
Function (in/out),rapid evolution
Organisation Implemented and maintained (typically) by single team
Implemented by different org units. Maintenance done elsewhere
Services implemented and owned by product teams
Implemented by pioneers (hipsters?)
Deployment Monolithic deployment Monolithic orchestration of multiple services
Services deployed individually
Functions deployed individually
Management None Centralised Distributed Chaotic / Orchestrated
Inter-process communication
None RPC or messaging, typically via middleware (ESB/MQ)
RPC via dumb pipesand smart endpoints, messaging/events
Events
Pioneers / stewards
Organisations, community or individuals
Enterprises and Vendors
Community and highperf organisations
Vendors/community
Core ArchitecturalConstraints
Language and framework Canonical domain model, standards
Interoperability Cost (Gbs/ms)
Architecture fundamentals
• Coupling• ”Components have little or no knowledge of other components”
• Interfaces
• Schema
• Cohesion • “Degree to which the elements within a component belong together”
• Single reason to change
• Separation of concerns
25/10/2017 @danielbryantuk
Coupling, Cohesion and Continuous Delivery
• Design• Cohesion makes reasoning easy
• Loose coupling reduces impact
• Build, unit and integration• Cohesion limits dependencies
• Loose coupling allows simulation
• End-to-end tests• Cohesion/coupling orchestration
• Deployment• Cohesion minimises number of
components in play
• Coupling leads to “monoliths”
• Observability• Cohesive is easier to understand
• High coupling typically leads to large blast radius and “murder mystery” style debugging
25/10/2017 @danielbryantuk
A journey of a thousand miles…
25/10/2017 @danielbryantuk
Working locally
25/10/2017 @danielbryantuk
https://cloud.google.com/functions/docs/emulatorhttps://azure.microsoft.com/en-gb/blog/announcing-the-preview-of-java-support-for-azure-functions/https://github.com/awslabs/aws-sam-local
Developing/Integrating locally
25/10/2017 @danielbryantuk
https://news.ycombinator.com/item?id=13960107https://opencredo.com/working-locally-with-microservices/https://www.datawire.io/telepresence/ | https://hoverfly.io/
Testing
25/10/2017 @danielbryantuk
Testing: Core concepts
25/10/2017 @danielbryantuk
/lisacrispin.com/2011/11/08/using-the-agile-testing-quadrants/martinfowler.com/bliki/TestPyramid.html
Testing: Core concepts
25/10/2017 @danielbryantuk
martinfowler.com/articles/microservice-testing/#testing-progress-3
Functional testing: End-to-end
25/10/2017 @danielbryantuk
Functional Testing: Outside-in
25/10/2017 @danielbryantuk
https://specto.io/blog/2016/8/16/recipe-for-designing-building-testing-microservices/http://www.thucydides.info/docs/serenity/
Functional
25/10/2017 @danielbryantuk
Talking of contracts…
25/10/2017 @danielbryantuk
Talking of (service) contracts…
• APIs are service contracts
• Consumer-driven Contracts
• martinfowler.com/articles/consumerDrivenContracts.html
25/10/2017 @danielbryantuk
Talking of (service) contracts…
25/10/2017 @danielbryantukdocs.pact.io
cloud.spring.io/spring-cloud-contractgithub.com/spring-cloud-samples/spring-cloud-contract-samples
Talking of (messaging) contracts…
• What about messaging?
• Message schema are an API
• www.infoq.com/presentations/contracts-streaming-microservices
25/10/2017 @danielbryantuk
Talking of (messaging) contracts…
25/10/2017 @danielbryantuk
www.infoq.com/presentations/contracts-streaming-microservices
docs.confluent.io/current/schema-registry/docs/maven-plugin.html
Contract verification
• Is it worth the cost?
• Tradeoff: Trust and comms• Startups / SMEs
• Enterprises…
• My (anecdotal) experience
• Choreography vs orchestration
25/10/2017 @danielbryantuk
25/10/2017 @danielbryantuk
https://netflix.github.io/conductor/http://rockscript.io/https://nifi.apache.org/docs.htmlhttps://aws.amazon.com/documentation/step-functions/http://camel.apache.org/
Contract verification
• Is it worth the cost?
• Tradeoff: Trust and comms• Startups / SMEs
• Enterprises…
• My (anecdotal) experience
• Choreography vs orchestration
• Choreography• Asserting behaviour (interactions)
• Contracts are part of this!
• Orchestration:• Asserting state (output)
• Verify orchestration DSL
25/10/2017 @danielbryantuk
Testing NFRs in the build pipeline
• Architecture quality• SonarQube / Code Climate
• Performance and Load testing • Gatling / Locust / Bees with m’guns
• Security testing • OWASP Dependency check / Bdd-security
• Docker Bench for Security / CoreOS Clair
25/10/2017 @danielbryantuk
25/10/2017 @danielbryantuk
www.owasp.org/index.php/OWASP_Dependency_Check
Delaying NFRs to the ‘Last Responsible Moment’
Newsflash!
Sometimes the
last responsible moment
is up-front
Modern architectures don’t necessarily make this easier
25/10/2017 @danielbryantuk
Wrapping Up
25/10/2017 @danielbryantuk
25/10/2017 @danielbryantuk
Monoliths SOA Microservices / SCS FaaS / Serverless
Scope Project Enterprise Product Feature (or glue?)
Focus Swiss Army Knife Reuse, governance, control
Domain modelling, SRP, evolution
Function (in/out),rapid evolution
Organisation Implemented and maintained (typically) by single team
Implemented by different org units. Maintenance done elsewhere
Services implemented and owned by product teams
Implemented by pioneers (hipsters?)
Deployment Monolithic deployment Monolithic orchestration of multiple services
Services deployed individually
Functions deployed individually
Management None Centralised Distributed Chaotic / Orchestrated
Inter-process communication
None RPC or messaging, typically via middleware (ESB/MQ)
RPC via dumb pipes/smart endpoints, messaging/events
Events
Pioneers / stewards
Organisations, community or individuals
Enterprises and Vendors
Community and highperf organisations
Vendors/community
Core ArchitecturalConstraints
Language and framework Canonical domain model, standards
Interoperability Cost
25/10/2017 @danielbryantuk
Monoliths SOA Microservices / SCS FaaS / Serverless
Cohesion and coupling enforced at modules
CD focuses on end-to-end functionality
Provide goals and “best practice” examples
Cohesion and coupling enforced at
service level
CD focuses on service integrity
Provide objectives and standards
Cohesion and coupling enforced at
service API level
CD focuses on service interaction
Provide principles and guidelines
Cohesion and coupling enforced at
function API level
CD focuses on service output/state
Provide principles and guidelines
In conclusion…
• We are moving from complicated systems to complex systems• Design and test with coupling and cohesion in mind
• Architecture is becoming more about technical leadership• Recognise your situation and influence accordingly
• We must encode all requirements into a continuous delivery pipeline• Both functional and nonfunctional requirements
25/10/2017 @danielbryantuk
Bedtime reading…
25/10/2017 @danielbryantuk
blog.christianposta.com/microservices/low-risk-monolith-to-microservice-evolution-part-iii/
Thanks for listening…
Twitter: @danielbryantuk
Email: daniel.bryant@tai-dev.co.uk
Writing: https://www.infoq.com/profile/Daniel-Bryant
Talks: https://www.youtube.com/playlist?list=PLoVYf_0qOYNeBmrpjuBOOAqJnQb3QAEtM
25/10/2017 @danielbryantuk
bit.ly/2jWDSF7
Coming soon!