Event-driven Microservices - The Conference for Java ...€¦ · §Domain Specific Languages (DSL)...

Event-drivenMicroservices

JeremyDeane

http://jeremydeane.net

Agenda

v Event-drivenArchitecture(EDA)

vMicoservicesArchitecturalStyle

vMessage-orientedMiddleware:ApacheActiveMQ

v EnterpriseIntegrationPatterns– ApacheCamel

v LightweightApplications– SpringBoot

v Event-drivenMicroservices

“All those… moments… will be lost in time, like tears… in… rain” - Roy Batty

Blade Runner

Event-drivenArchitecture(EDA)

Events

§ Eventsrepresent asnapshot intimeofan

occurrencewithinasystem

§ Information (noise)toActionable (signal)

EnterpriseIntegrationPatterns(EIP)

§ EventMessage, CommandEvent,EventSourcing

MiddlewareSolutions

§ ActiveMQ,RabbitMQ (JMS-AMQP)

§ Kafka,ZeroMQ

§ Akka,Storm,Flink

EDAPrinciples

EventsareemittedbyaProducerandreceivedasynchronously,andoptionally

actedupon,byastatelessConsumer

Raw

Event

<Event Producer>System

<Event Consumer>System

EDAPrinciples



StreamsaresetsofrelatedEvents

Related

Event



Related

EventRelated

Event

EDAPrinciples



IntermediateProcessors canenrichtherawEvent.

StreamsaresetsofrelatedEvents

<Event Processor>System

Raw

Event



Derived

Event

EDAPrinciples

Ideally,ProducerandConsumer shoulddecoupledsotheycanevolve

independently overtime.Inaddition,Producer shouldbemagnanimouswriter and

Consumer shouldbetolerantreader.

Messaging

Backbone


RawEvent



DerivedEvent

Queue A Queue B

EDAPrinciples

Consumers canlistentoEventQueuesorsubscribetoEventTopics

Ideally,ProducerandConsumer shoulddecoupledsotheycanevolve

independently overtime.Inaddition,Producer shouldbemagnanimouswriter and

Consumer shouldbetolerantreader.

Messaging

Backbone


RawEvent



DerivedEvent

Queue A Queue BTopic X


EDAExamples

²Fraud Prevention

²MedicalAlerting(ERCheck-in)

²FinancialPortfolioManagement

²SupplyChainManagement

“A new life awaits you in the Off-world colonies! A chance to begin again in a golden

land of opportunity and adventure!” – Advertisement Blade Runner

MicroservicesArchitecturalStyle

http://martinfowler.com/articles/microservices.htm

l

…anapproachtodeveloping asingleapplicationasasuite ofsmallservices,eachrunning

initsownprocessandcommunicating withlightweightmechanisms,oftenanHTTP

resourceAPI.These services arebuiltaroundbusinesscapabilitiesandindependently

deployablebyfullyautomateddeploymentmachinery.There isabareminimumof

centralizedmanagement oftheseservices,whichmaybewrittenindifferentprogramming

languages andusedifferentdatastoragetechnologies.

…youmustbethistall touse.

- NealFord

MicroservicesExample– RESTWebServices

Policy System<Execution Environment>

Quote Resource<WS Endpoint>

Renewal Resource<WS Endpoint>

RMV<SAAS>

Driver Resource<WS Endpoint>

Carfax<SAAS>

Car Resource<WS Endpoint>


Policy System<Execution Environment>

Quote Resource<WS Endpoint>

Renewal Resource<WS Endpoint>

RMV<SAAS>

Driver Resource<WS Endpoint>

Carfax<SAAS>

Car Resource<WS Endpoint>

ESB<Service Gateway>

Driver Facade<WS Endpoint>

Care Facade<WS Endpoint>


Quote Microservice

<Execution Environment>

Quote Resource

<WS Endpoint>

RMV

<SAAS>

Driver Resource

<WS Endpoint>

Carfax

<SAAS>

Car Resource

<WS Endpoint>

Driver Microservice


Driver Facade

<WS Endpoint>

Renewal Microservice


Renewal Resource

<WS Endpoint>Car Microservice


Car Facade

<WS Endpoint>

BreakingDowntheMonolithChallenges

§ LegacyApplications

§ InconsistentDomainModels

§ Pandora’sFunctionalityBox

§ ContinuousIntegrationandContinuousDelivery(CI/CD)Maturity

§ OrganizationalStructure(Conway’sLaw)

Event-drivenMicroservices

Event Endpoint

Monitoring API

Management API


Configurations

Logs

Process Monitoring

Process Management

Message

Memories! You're talking about memories! - Rick Deckard Blade Runner

ApacheActiveMQ

IntegrationOptions

JavaMessage Service(JMS)

AdvancedMessage QueuingProtocol(AMQP)

Deployment Flexibility

Stand-alone

Embedded

AdvancedTopologies

Master-SlaveHighAvailability(HA)

FederatedNetwork

Support

ActiveOpenSourceCommunity

Commercial 24X7Options

Languages- Transport

• Java-Scala– TCP/NIO

• Ruby,Perl,Python– Stomp

• C#(NMS)–TCP/NIO

ActiveMQ HighAvailability(HA)MessagingBackbone

failover:(tcp://PrimaryBroker:61616,tcp://SecondaryBroker:61616)?randomize=false

Linux VM

Queue


ActiveMQ (Master)

Topic

Linux VM

Queue


ActiveMQ (Slave)

Topic

<NFS Mount>

Kaha DB

ActiveMQ FederatedTopologies

Linux VM

Queue


ActiveMQ (Master)

Topic

Linux VM

Queue


ActiveMQ (Slave)

Topic

<NFS Mount>

Kaha DB

Linux VM

Queue


ActiveMQ (Master)

Topic

Linux VM

Queue


ActiveMQ (Slave)

Topic

<NFS Mount>

Kaha DB

Linux VM

Queue


ActiveMQ (Master)

Topic

Linux VM

Queue


ActiveMQ (Slave)

Topic

<NFS Mount>

Kaha DB

WebSecurityNetworkSegment

ServiceSecurityNetworkSegment

DataSecurityNetworkSegment

§ Network

§ Business

§ Other

ConsoleDemo

1. StartActiveMQ

2. Execute TestHarness

3. ViewActiveMQ Console

4. StartHawtio

5. OpeninBrowser

6. ConnectàLocal

7. Connect toActiveMQ &clickAgentURL

cd $ACTIVEMQ_HOME/bin

./activemq start OR ./activemq.bat

Open http://localhost:8161/admin/ {admin/admin}

Open http://localhost:8161/admin/queues.jsp

cd event-test-harness

mvn clean install

java –jar ./target/event-client-1.0.2.jar

java -jar hawtio-app.jar --port 8090

http://localhost:8090/hawtio

“Bryant: I need ya, Deck. This is a bad one, the worst yet. I need the old

blade runner, I need your magic.” Harry Bryant Blade Runner

ApacheCamel– EnterpriseIntegrationLibrary

CamelInAction byClausIbsenandJonathanAnstey

§ DomainSpecificLanguages(DSL)– Java,Scala,SpringDSL

§ RouteBuilders– createEndpoints(i.e.from&to)using

Components(e.g.protocol)andProcessors(e.g.Mediation,

Enrichment

§ RouteEngine– LoadsandexecutesRoutes

SpringBoot– ApacheCamelRouteProcessing

/**

* Splitter - xpath expression

*/

from("activemq:emagic.orders").

split(splitXPath).parallelProcessing().

wireTap("direct:ministry").

to("activemq:emagic.order");

/** Spring Boot – Camel Route Application */

@Configuration

@ImportResource("classpath:camel-route-spring.xml")

public class MagicRouterApplication {

public static void main(String[] args) {

SpringApplication.run(MagicRouterApplication.class, args);

}

}

Camel Route Engine

Camel Route

<JMS Component>Endpoint (From)

<EIP Processor>Some Processing

<DB Component>Endpoint (To)

CamelMessage

Configurations

ApacheCamel– DeploymentOptions

Linux VM

Camel

Route


Spring-Boot Camel

Camel

Route

Properties Log

Linux VM

Camel

Route


Spring-Boot Camel

Camel

Route

Properties Log

Port: 2100

Camel

Route


Spring-Boot Camel

Camel

Route

Properties Log

Port: 2101

§ SingleEvent-drivenMicroservice perVM

§MultipleEvent-DrivenMicroservicesperVM

§ SingleEvent-drivenMicroservices Container

§ Docker,Rocket,Capsule

§ CloudFoundry,OpenShift

ApacheCamel– ActiveMQ Integration

Linux VM

Queue


ActiveMQ (Master)

Topic

Linux VM

Queue


ActiveMQ (Slave)

Topic

Linux VM

Camel Route


Spring-Boot Camel

Camel Route

Properties Log

Linux VM

Camel Route


Spring-Boot Camel

Camel Route

Properties Log

Linux VM

Camel Route


Spring-Boot Camel

Camel Route

Properties Log

“This announcement is brought to you by the Shimato Dominguez Corporation -

helping America into the New World.– Announcement Blade Runner

Event-drivenMicroservices:Demo

<Event Driven Microservice>Care Management

HL7 FHIREncounter

<Event Driven Microservice>Event Auditing

Audit Repository

Messaging Backbone

<Event Driven Microservice>Encounter Ingestion

HL7 FHIREncounter

EnrichedEncounter

HL7 FHIREncounter

<Queue>cep

<Queue>tasks

<Queue> ingestion

<Queue>audit

<Event Driven Microservice>Complex Event Processing

<Queue>inpatient

<Topic>fraud

FraudulentEncounter

Event-drivenMicroservice - Instructions

RequiresActiveMQ listening ontcp://localhost:61616

1)Buildallthesub-projects- mvn clean install

- event-common

- event-test-harness

- event-ingestion

- event-auditing

- event-care-management

- event-cep

2)StartallMicroservices - mvn spring-boot:run -Drun.arguments="-Xmx256m,-Xms128m"

- event-ingestion

- event-auditing

- event-care-management

- event-cep

Event-drivenMicroservice - Instructions

RequiresActiveMQ listening ontcp://localhost:61616

3)RunTest Harness

cd event-test-harness/target

java -jar event-client-1.0.2.jar Leon

java -jar event-client-1.0.2.jar Roy

java -jar event-client-1.0.2.jar Eldon

4)ViewResults inActiveMQ orHawtio Console

defaultuid/pw=admin/admin

5)ViewAuditFileResults

cd event-auditing/target/events

Event-drivenMicroservice Throttle

HL7 FHIREncounter

Messaging Backbone

<Event Driven Microservice>

Encounter Ingestion

<Queue>

ingestion<Queue>

audit<Queue>

inpatient

Throttle

from("activemq:event.ingestion").

to("seda:audit");

from ("seda:audit").

throttle(100).asyncDelayed().

process(new TrackingIdProcessor()).

choice().

when().jsonpath("$[?(@.class==inpatient)]").

to("activemq:event.inpatient").

otherwise().

to("activemq:event.outpatient").

end().

to("activemq:event.audit");

Event-drivenMicroservice CircuitBreaker

<SAAS>

Complex Event Procesing

HL7 FHIREncounter

Messaging Backbone

<Event Driven Microservice>

Encounter Ingestion

<Queue>

ingestion<Queue>

audit<Queue>

inpatient

Breaker

from("activemq:event.ingestion").

to("direct:saas");

from("direct:saas").

log("Sending SAAS HTTP Request").

setHeader(Exchange.HTTP_METHOD,

constant(org.apache.camel.component.http4.HttpMethods.GET)).

loadBalance().

circuitBreaker(2, 5000L, CircuitBreakerOpenException.class).

to("http4://localhost:8080/event");

“Questions... Morphology? Longevity? Incept dates?” - Roy Batty Blade Runner

Event-drivenMicroservices:QualityAssurance

@Test

publicvoidtestSplitLogic()throwsException {

//Setexpectations

MockEndpoint order=getMockEndpoint("mock:order");

order.expectedMessageCount(3);

//readinasmallbatch

Stringxml=IOUtils.toString(this.getClass().

getResourceAsStream("batch.xml"),"UTF-8");

//executethetest

template.sendBody("direct:split", xml);

//shouldbethreeordersinthebatch

order.assertIsSatisfied();

}

§ SourceControlManagement- Git

§ FunctionalTests- CamelTestSupport

§ ContinuousIntegration(CI)– TeamCity

Event-DrivenMicroservices:Operations

§ Logging(e.g.Splunk,Logstash)

§ StandardFormat

§ Alerts

§ Processes(e.g.Zabbix,Nagios,Hyperic)

§ JMX

§ RESTWS

§ CommandEvents(e.g.PoisonPill)

§ SyntheticTransactions

§ TrackingIdentifier

§ Correlateactivitiesacrosssystems

§ Rapidlyidentifyproblemsandperformancebottlenecks

§ CommandConsoles(e.g.ActiveMQ,JavaConsole,Zabbix,Splunk)

Event-drivenMicroservices:ReleaseEngineering

§ ConventionoverConfiguration

§ DeploymentandConfigurationDirectoryStructure

§ Linuxinit.d SymbolicLink

§ Git Hooks

§ TriggerReleaseProcessandFunctionalRegression

§ ChefRecipesorAWSCloudFormation (IaaS)

§ InstantiateLinuxVM

§ ProvisionLinuxVM(JavaRuntimeEnvironment)

§ DeployExecutableJARandConfigurations

§ PlatformasaService(PaaS)

§ PushApplications

“He was wrong. Tyrell had told me Rachael was special. No termination date. I

didn't know how long we had together... Who does?” – Rick Deckard Blade Runner

Questions&FeedbackQuestions&Feedback

MyContact

information:

JeremyDeane

[email protected]

http://jeremydeane.net

https://github.com/jtdeane/event-driven-microservices

https://github.com/jtdeane/camel-standalone-router

Event-driven Microservices - The Conference for Java ...€¦ · §Domain Specific Languages (DSL)...

Documents

Transcript of Event-driven Microservices - The Conference for Java ...€¦ · §Domain Specific Languages (DSL)...