Microservices: Where do they fit within a rapidly evolving ...A microservice is a granular decoupled...

Microservices: Where do they fit within a rapidly evolving integration architecture?

Brian Petrini ([email protected])

Kim Clark ([email protected])

Evolving exposure of business function

Service Exposure (enterprise)

Low Level APIs (platform/package)

Application Integration (application)

Service/API Exposure (external known

consumers)

External API Exposure (public)

Future?

Differentiating between web APIs, SOA, and integration http://www.slideshare.net/kimjclark/ino-2739-impact2014integrationsoaapiv15 Related article on developerWorks http://www.ibm.com/developerworks/websphere/library/techarticles/1503_clark/1305_clark.html

Digital Transformation Creates Multi-Modal IT

Digital •  Timeframe: Days/Weeks •  Scope: Strategic •  Sponsor: CMO •  Budget: $ •  Complexity: Low •  Buying behavior: Self service

Enterprise •  Timeframe: Months/Years •  Scope: Strategic •  Sponsor: CIO •  Budget: $$$ •  Complexity: High •  Buying behavior: Human engagement

Enterprise

IT

Digital Teams

Explore, adopt, adapt with rapid, iterative prototypes

Always on, always available Security, control, and governance

Agenda

Defining microservices architecture Is this different from Service Oriented Architecture (SOA)? How is integration architecture changing? Where do microservices fit in across the future landscape? How to start an argument about microservices

3

What is microservices architecture

Monolithic application

Microservices application

Silo

Microservice (component)



A microservice is a granular decoupled component within a broader application

Agility Scalability Resilience

Simplistically, microservices architecture is about breaking down large silo applications into more manageable fully decoupled pieces

Microservice component



Encapsulation is key. Related logic and data should remain together, and which means drawing strong boundaries between microservices.



Silo logic

Silo data

Example operating system boundaries

Heterogeneous on the inside, homogeneous on the outside

Freedom to choose runtimes, languages, datastores etc. •  Wise to encourage preferred

technologies. •  Convergence often happens

naturally. Commonality is in the framework in terms of: •  Interconnectivity •  Scalability •  Resilience

6


Microservice (Node.js)

Microservice (Java +

MongoDB)

Microservice (Go +

CloudantDB)

Why Microservices?

Small scoped, independent, scalable components Scaling

Elastic scalability Workload orchestration

Agility Faster iteration cycles Bounded context (code and data)

Resilience Reduced dependencies Fail fast

7

Microservices: Why now? (technical standpoint)

•  Internet/intranet/network maturity •  Lightweight runtimes (node.js, WAS Liberty etc.) •  Methods & tools (Agile, DevOps, TDD, CI, XP, Puppet, Chef…) •  Lightweight protocols (RESTful APIs, lightweight messaging) •  Simplified infrastructure

•  OS virtualisation (hypervisors), containerisation (e.g. Docker), infrastructure as a service (IaaS), workload virtualisation (Kubernetes, Mesos, Spark…)

•  Platform as a service •  Auto-scaling, workload management, SLA management, messaging,

caching, build management. •  Alternative data persistance models (NoSQL, MapReduce, BASE,

CQRS) •  Standardised code management (Github, …)

8




WAS Liberty Node.js

What that might look like in IBM technology? (simplistic representation – NOT a reference architecture!)



WAS ND

DB2

MongoDb

MongoDb

Cloudant

Node.js

IBM Bluemix (PaaS)

Message Hub

(Kafka)

JSON/ HTTP

Microservice

Microservice Microservice

Microservice

Microservices inter-communication

Aim is decoupling for robustness Messaging where possible •  Lightweight messaging

(e.g. AMQP, Kafka) •  Publish/subscribe •  Eventual consistency Direct calls where necessary Lightweight protocols (e.g. JSON/HTTP) •  Load balancing/scaling via service

discovery •  Circuit breaker •  Caching


Microservice

Subscribe

JSON /HTTP

Microservice

Microservice

Message Hub

API

Microservice Publish

API

JSON /HTTP Publish

Service Discovery

Application frameworks at the network level

Microservices simplify the creation and maintenance of applications by inheriting key non-functional characteristics from their environment. Sounds familiar? Similar in principle to application container frameworks from the past (e.g. JEE), but: Different because •  Framework is agnostic to the

runtime/language used by each microservice

•  Microservices, and framework components sit on the network, rather than within an application server Microservices application



Microservice component Microservice

component Microservice component Microservice

component

Framework provides •  Scaling •  Availability •  Caching •  Messaging •  Connectivity •  …and more

Note that from this perspective there is greater parity between microservices architecture and application servers than there is with service oriented architecture 11

Can everything become a microservice? What about the things you can’t change?

•  You can’t refactor all systems to microservices •  Most applications have an “if it ain’t broke, don’t fix it” policy •  Old systems may be unrealistic to re-engineer

•  What if you can’t change the datastore? •  Are you doing microservices if your data remains in a silo? •  Can you manage without transactionality to the database? •  Will techniques like BASE, CQRS work?

•  How do existing systems fit into a microservices architecture? •  How do you isolate yourself from their availability issues? •  What if they don’t scale as well as your microservices do?

12

Challenges with microservices •  Maintenance

•  Given the aim of freedom of language and runtime, will you have the breadth of skillsets to maintain the microservices in the future.

•  Serialisation – data has to get over the wire •  Serialisation has advanced massively in recent years.

•  Latency •  A request/response chained down a set of microservices must incur some extra

latency from network hops and serialisation. •  Data sharing

•  Not all data can be split into a grid, some things are shared. •  Real-time dependencies and their combined availability

•  Microservices calling other microservices synchronously need careful consideration.

•  Tends to creep, as one service built on top of another. •  Managability

•  How do you manage and monitor a vast network of microservices •  How does persistence work?

•  Pessimistic vs. Optimistic •  How handle shared objects •  Relational/NoSQL •  ACID/BASE/CQRS/Event Sourcing?

13

Handling distributed objects

•  First law of distributed objects: Don’t distribute your objects •  http://martinfowler.com/bliki/FirstLaw.html

•  Microservices and the First Law of Distributed Objects •  http://martinfowler.com/articles/distributed-objects-

microservices.html

•  Don’t go too granular, as harder to gather up than to break out.

14

Are we really doing microservices?

Martin Fowler/James Lewis 1.  Componentization 2.  Organized around Business

Capabilities 3.  Products not Projects 4.  Smart endpoints and dumb pipes 5.  Decentralized Governance 6.  Decentralized Data Management 7.  Infrastructure Automation 8.  Design for failure 9.  Evolutionary Design

http://martinfowler.com/articles/microservices.html

15

12 factor apps I.  Codebase II.  Dependencies III.  Config IV.  Backing Services V.  Build, release, run VI.  Processes VII.  Port binding VIII.  Concurrency IX.  Disposability X.  Dev/prod parity XI.  Logs XII.  Admin processes

http://12factor.net

Are we really doing microservices or just aligning with some of the microservices principles?

Consider the adoption paths of SOA, XP, agile, devops etc. These often come with an “all or nothing” message, but can you take on the whole package?

Agenda


16


Common misconception resulting from the term “microservice”

Monolithic application Microservices application

Exposed services/APIs



Exposed services/APIs

Silo component

Microservices are just more fine grained web services

APIs are microservices

“micro” refers to the granularity of the components, not the granularity of the exposed interfaces

x 1 x 3

x 4 x 4

Is “microservices architecture” is really “micro-component architecture”?

Application

SOA relates to enterprise service exposure *

Application Application Application

Service oriented architecture (SOA) and microservices architecture relate to different scopes

Microservice application

µService

µService µService

µService

Microservices relate to application architecture

* this simple distinction can be contentious depending on your definition of SOA

“Microservices, SOA, and APIs: Friends or enemies?” http://www.ibm.com/developerworks/websphere/library/techarticles/1601_clark-trs/1601_clark.html

What was SOA really about? Integration or Components? Opinion 1: “SOA is about how to achieve integration often to aging complex back end systems in order to expose services” In this case SOA is primarily a connectivity p rob lem w i th l i t t l e re la t i onsh ip to microservices architecture and certainly at a different scope.

Opinion 2: “SOA is about re-factoring your IT landscape into components that better align with the business needs and expose the services that it requires” Here the connectivity problem is pushed down to the applications and the focus of SOA is on realignment of to the business needs. The service components look more like applications, and we might consider microservices as “more granular SOA”, or even “SOA done right”.

System of Record

Integration Hub

Adapter Adapter

Exposure Gateway

Exposed Services/APIs

System of Record

System of Record

Adapter

Exposed Services/APIs

System of Record

Adapter

Service Component

Service Component

Service Component

Exposure Gateway

However, SOA, despite it’s broader intent, resulted mostly in interface related technology (e.g. WS-*, ESBs). Microservices architecture is more specific on how components should be implemented, and benefits from more real examples of frameworks, and platforms in this area than SOA did at an equivalent time in it’s history. 19

What does a large scale integration landscape look like

Systems of engagement •  Modern languages/runtimes •  Agile •  Simple modern connectivity

Systems of record •  Older technology •  Harder to change •  Harder to integrate with

Integration Hub

Integration Hub Adapter Adapter

“Systems of Engagement” Applications

Exposure Gateway (internal)

“Systems of Record”

Applications

Mature large enterprise (simplified)

Exposure Gateway (external)

Adapter Adapter

“Hub and Spoke” integration

“Enterprise Service Bus?”

“Web API Gateway”

Why such split opinions on microservices vs SOA?

21

Integration Hub


Enga

gem

ent

App

licat

ions

SaaS

A

pplic

atio

n

Bus

ines

s Pa

rtne

r

Sy

stem

s of

R

ecor

d

Bus

ines

s Pa

rtne

r


Mature large enterprise Microservices are just one style of application

Exposing services is an integration and data challenge

Green field online start-up Much of landscape could be microservice based

The landscape is as (micro)service oriented architecture


Microservice application

µService

µService µService

µService

Exposure Gateway (internal) µService

µService

µService

µService

µService

µService µService

µService

µService

µService

µService

Ente

rpris

e B

ound

ary

Ente

rpris

e B

ound

ary

Adapter Adapter

Where do we typically see microservices in large organisations today?

22

Syst

ems

of

Rec

ord

Integration Hub


Enga

gem

ent

App

licat

ions

Mic

rose

rvic

e ap

plic

atio

ns

SaaS

App

licat

ions

(e

xter

nal)

Adapter

Externally Exposed Services/APIs



Bus

ines

s Pa

rtne

rs

Mapping to example IBM products

23

Syst

ems

of R

ecor

d (In

tern

al c

ore

appl

icat

ions

)

Integration Hub


Enga

gem

ent

App

licat

ions

Mic

rose

rvic

e ap

plic

atio

ns

SaaS

App

licat

ions

(e

xter

nal)

Adapter




Bus

ines

s Pa

rtne

rs

WebSphere Application

Server (WAS)

IBM API Connect or IBM Integration Bus

IBM Integration Bus

IBM Integration Bus

WAS Liberty, Node.js, MongoDb,

Cloudant etc. on IBM Bluemix

IBM API Connect + IBM DataPower

Gateway

IBM API Connect

Where might we see microservices going forward?

24

Syst

ems

of

Rec

ord

Integration Hub


Enga

gem

ent

App

licat

ions

Mic

rose

rvic

e ap

plic

atio

ns

SaaS

App

licat

ions

(e

xter

nal)

Adapter




Bus

ines

s Pa

rtne

rs

Integration Domain

Enterprise Ownership Boundary

The Integration Domain of the Hybrid Enterprise

Dedicated Cloud Public Cloud SaaS

Customers Business Partners

Appl. Appl. Appl. Appl. Appl. Appl.

Appl. Appl. Appl. Appl.

On-Premise

Appl. Appl. Appl. Appl. Appl.

Local Cloud

Appl. Appl.

Agenda


26

Application Application

Decentralising synchronous integration

Enga

gem

ent

App

licat

ions

System of Record

Integration

Public API

Enterprise API

API Gateway

System of Record System

of Record

Integration Ent

erpr

ise

Bou

ndar

y


SaaS Application Business

Partner

In a perfect world…

•  Application teams self-administer exposure of their own APIs

•  Application teams handle their own integration needs

•  Access to external APIs is governed using the same mechanisms used to govern access to internal APIs.

•  Application logic is firmly seated with the application teams

•  API monitoring/diagnostics would be gathered consistently across the organisation

•  Security models would be implemented more consistently

Application Application

So where is the ESB now? Where might we find microservices here? What about event driven integration?

IBM Confidential 28

IBM Hybrid Integration

Connect Seamlessly!Hundreds of end points to apps and data in the cloud and on premise

Develop Rapidly! Intuitive and robust tooling to transform data to meet business needs

Scale Efficiently ! Performance and scalability to meet the SLAs of your business applications

HYBRID INTEGRATION

SaaS PaaS On-Premise

CONNECT XFORM DELIVER COMPOSE EXPOSE

API MANAGEMENT

SECURE GATEWAY

INTEGRATION ENGINE

DESIGN - OPERATE - MANAGE - MONITOR - GOVERN

Data APIs Apps TH GS IN

MESSAGE &EVENT HUB

One cohesive offering serving all integration needs

What is an application boundary in an microservices world?

Similar to application boundaries today because •  Group related functionality •  Enable more coarse grained

ownership and responsibility (in addition to full ownership at the microservice level)

Different because •  Application sub-components

(microservices) are truly independent.

•  Application boundaries can be easily changed – it’s primarily about the right ownership

•  Application boundaries can spread across network boundaries (e.g. between clouds)…but should they!

29

µService µService

µService

µService

µService

µService µService

µService

µService

µService

µService


Inter-microservice vs. inter-application communication

Mic

rose

rvic

es

appl

icat

ion



Inter-microservice communication •  Lightweight protocols: HTTP,

application messaging •  Runtime component registry •  Client-side load balancing and circuit

breaker patterns

Mic

rose

rvic

es

appl

icat

ion

Exposure Gateway

Inter-application communication •  Enterprise protocols: Managed API

gateways, enterprise messaging •  Design time developer portals •  Gateway load balancing and throttling

JSON/HTTP RESTful communication styles may be present in both types of communication, but their implementation may be radically different.

JSON/HTTP

31

What are the fundamental questions that drive a design

Who owns it? Who creates and prioritises the requirements?

How long does it last? What is the runtime lifespan of its instances?

How often does it change?

How frequently do its requirements change?

Agenda


32

Caution with comparisons (or “how to start an argument”!)

Comparisons that make some level of sense

Microservice architecture vs. Service oriented architecture Trying to achieve many of the same things (enable the creation of re-usable business functions as isolated components), but at a very different scale. SOA evolved to become focused on the enterprise scale (though some would say that wasn’t the original intent). Microservices, is primarily at the application scale.

Microservice component vs. Service component Both describe the actual implementation of a business function. They differ primarily in granularity, but also arguably in maintainability, agility etc.

Confusing comparisons

Microservices vs. SOA Microservices are components. SOA is an architecture. What you’re probably trying to compare is Microservices Architecture with SOA (see above).

Microservices vs. APIs APIs are a interface. A mechanism for exposing business function. Microservices are the component architecture used to implement those business functions. It makes no sense to directly compare them.

Microservice vs. Service “Service” means different things to different people. Needs to be qualified. Service Implementation? Service exposure? Service definition? The same could be said for microservice. Need to be specific.

33

Questions?

34

http://www.slideshare.net/kimjclark/presentations

Notices and Disclaimers

35

Copyright © 2016 by International Business Machines Corporation (IBM). No part of this document may be reproduced or transmitted in any form without written permission from IBM.

U.S. Government Users Restricted Rights - Use, duplication or disclosure restricted by GSA ADP Schedule Contract with IBM.

Information in these presentations (including information relating to products that have not yet been announced by IBM) has been reviewed for accuracy as of the date of initial publication and could include unintentional technical or typographical errors. IBM shall have no responsibility to update this information. THIS DOCUMENT IS DISTRIBUTED "AS IS" WITHOUT ANY WARRANTY, EITHER EXPRESS OR IMPLIED. IN NO EVENT SHALL IBM BE LIABLE FOR ANY DAMAGE ARISING FROM THE USE OF THIS INFORMATION, INCLUDING BUT NOT LIMITED TO, LOSS OF DATA, BUSINESS INTERRUPTION, LOSS OF PROFIT OR LOSS OF OPPORTUNITY. IBM products and services are warranted according to the terms and conditions of the agreements under which they are provided.

Any statements regarding IBM's future direction, intent or product plans are subject to change or withdrawal without notice.

Performance data contained herein was generally obtained in a controlled, isolated environments. Customer examples are presented as illustrations of how those customers have used IBM products and the results they may have achieved. Actual performance, cost, savings or other results in other operating environments may vary.

References in this document to IBM products, programs, or services does not imply that IBM intends to make such products, programs or services available in all countries in which IBM operates or does business.

Workshops, sessions and associated materials may have been prepared by independent session speakers, and do not necessarily reflect the views of IBM. All materials and discussions are provided for informational purposes only, and are neither intended to, nor shall constitute legal or other guidance or advice to any individual participant or their specific situation.

It is the customer’s responsibility to insure its own compliance with legal requirements and to obtain advice of competent legal counsel as to the identification and interpretation of any relevant laws and regulatory requirements that may affect the customer’s business and any actions the customer may need to take to comply with such laws. IBM does not provide legal advice or represent or warrant that its services or products will ensure that the customer is in compliance with any law

Notices and Disclaimers Con’t.

36

Information concerning non-IBM products was obtained from the suppliers of those products, their published announcements or other publicly available sources. IBM has not tested those products in connection with this publication and cannot confirm the accuracy of performance, compatibility or any other claims related to non-IBM products. Questions on the capabilities of non-IBM products should be addressed to the suppliers of those products. IBM does not warrant the quality of any third-party products, or the ability of any such third-party products to interoperate with IBM’s products. IBM EXPRESSLY DISCLAIMS ALL WARRANTIES, EXPRESSED OR IMPLIED, INCLUDING BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE.

The provision of the information contained h erein is not intended to, and does not, grant any right or license under any IBM patents, copyrights, trademarks or other intellectual property right.

IBM, the IBM logo, ibm.com, Aspera®, Bluemix, Blueworks Live, CICS, Clearcase, Cognos®, DOORS®, Emptoris®, Enterprise Document Management System™, FASP®, FileNet®, Global Business Services ®, Global Technology Services ®, IBM ExperienceOne™, IBM SmartCloud®, IBM Social Business®, Information on Demand, ILOG, Maximo®, MQIntegrator®, MQSeries®, Netcool®, OMEGAMON, OpenPower, PureAnalytics™, PureApplication®, pureCluster™, PureCoverage®, PureData®, PureExperience®, PureFlex®, pureQuery®, pureScale®, PureSystems®, QRadar®, Rational®, Rhapsody®, Smarter Commerce®, SoDA, SPSS, Sterling Commerce®, StoredIQ, Tealeaf®, Tivoli®, Trusteer®, Unica®, urban{code}®, Watson, WebSphere®, Worklight®, X-Force® and System z® Z/OS, are trademarks of International Business Machines Corporation, registered in many jurisdictions worldwide. Other product and service names might be trademarks of IBM or other companies. A current list of IBM trademarks is available on the Web at "Copyright and trademark information" at: www.ibm.com/legal/copytrade.shtml.

Thank You Your Feedback is Important!

Access the InterConnect 2016 Conference Attendee Portal to complete your session surveys from your

smartphone, laptop or conference kiosk.

Microservices: Where do they fit within a rapidly evolving ...A microservice is a granular decoupled...

Documents

Transcript of Microservices: Where do they fit within a rapidly evolving ...A microservice is a granular decoupled...