Wso2 integration platform deep dive eu con 2016
-
Upload
chanaka-fernando -
Category
Technology
-
view
70 -
download
1
Transcript of Wso2 integration platform deep dive eu con 2016
WSO2 Integration Platform Deep Dive
Chanaka Fernando - Senior Technical Lead
Ramith Jayasinghe - Senior Technical Lead
Agenda
• Concepts in Integration• Demonstration• Product roadmaps/features• Q&A
Evolution of Integration
Point to Point Integration● Custom code, data formats for
majority of integration ( data extraction, business rule processing, data loading)
● Costly over time (maintenance, complexity)
● Tight Coupling
● Scalability is an issue
● Less reusability
Evolution of Integration
Use of Enterprise Service Bus ● Consistency: Clear set of guidelines (EIPs) results in repeatable and successful integration efforts/projects.
● Reusability: Tested and preconfigured components provides repeatable approach to integration.(transports/connectors/mediators etc)
● Effectiveness: A common integration framework (with related abstract concepts) improves integration developer's productivity.
Connected Business
An internally and externally connected business.
Image courtesy hRp://jdamico.net/wp-content/uploads/2010/04/collaborate-with-B2B-channel-partners.jpg
ESB were used to connect internal systems in the past.
Current need is to connect internal/external parties quickly and flexible manner.
E.g. Employees, Internal/External systems, Partners, Customers.
Connected Business
Integration is a key factor for having frictionless interactions, inside and outside the organization.
Integration and APIs - The Close Cousins
• APIs cannot replace Integration
– Integration of internal services, systems, data and cloud apis
• Cannot mangle SOA for API Management needs
• Using SOA and API in combination is the key to success of
Connected Business.
Integration and APIs - The Close Cousins
WSO2 Integration + API Management Platform
In practise API management and integration co-exists providing flexible, faster ways to connect required parties such as users, internal/external systems, employees etc.
WSO2 Integration Platform
Fundamentals of WSO2 ESB
Message Processing units- Sequences, mediators
Message entry points- Proxy Services, APIs, Inbound
endpoints.Message exit points
- Endpoints
Mediators
Example MediatorsName Description
Log Mediator Logs full or part of the message, at various severity levels ( Trace, Debug, etc)
Sequence Mediator Invokes existing sequence - Sequence name can be static or dynamic
Send Mediator Sends a message out, using static information or endpoint definition.
Datamapper Mediator Transform data from one form to another
Switch Mediator Evaluates messages contents against regular expression and invokes the corresponding mediator (switch-case-default)
Validate Mediator Validates message or parts of message against XML schema (schema can be local or in registry)
PayloadFactory Mediator Create a message payload from an existing message or from scratch
Fault Mediator Transforms current message into custom Fault message
Mediators
• Transformation Mediators• Filters, Switches, Iterate, Aggregate, Clone• Property Mediator• Script Mediators• Custom Mediators
Connectors
• A connector is a ready made and convenient
tool to reach publicly available web API’s.
• ‘Cloud to Cloud’ and ‘Cloud to Enterprise’
Integration
• ESB Connector Store : https://store.wso2.com/
Connectors
• Every connector is self-contained and independent from ESB code
• Dynamically plug in to ESB• Dynamic Tooling support with WSO2 Developer
Studio• You can write, ‘your own connector’ and just plug it
in
Sequences
Fundamentals of WSO2 ESB
Message Processing units- Sequences, mediators
Message entry points- Proxy Services, APIs, Inbound
endpoints.Message exit points
- Endpoints
Proxy Service
• Acts like a virtual service.
• Receives messages and mediates
them before sending them to the
endpoint (usually an actual
service)
• Mainly to expose as SOAP Service
API/HTTP Service
• APIs can accept REST messages which allow clients
to provide additional information on how to
mediate the message
• Defined under a URL Context (e.g: /customer)
• Can define multiple resources under a URL context
• Resources
– Component of API accessed through HTTP call
– Similar to proxy service (in, out, and fault
sequences)
– Can restrict resource’s scope using URL
patterns and URI templates
API/HTTP Service
• URL mapping
– Path mappings (eg: /test/*, /foo/bar/*)
– Extension mappings (eg: *.jsp, *.do)
– Exact mappings (eg: /test, /test/foo)
• URI template
– /order/{orderId} would process /order/A0001
– /dictionary/{char}/{word} would process /dictionary/c/cat
– Use get-property to retrieve exact values
<log level="custom">
<property name="Character" expression="get-property('uri.var.char')"/>
<property name="Word" expression="get-property('uri.var.word')"/>
</log>
Inbound Endpoint
• Creating inbound messaging channels
dynamically.
– Listening Inbound – HTTP, HL7, TCP
– Polling Inbound – JMS, File/VFS
• Injects the inbound messages into an ESB message flow.
Polling Inbound endpoint in a Cluster
One Consumer Multiple Consumers
Fundamentals of WSO2 ESB
Message Processing units- Sequences, mediators
Message entry points- Proxy Services, APIs, Inbound
endpoints.Message exit points
- Endpoints
Endpoints
• Logical representation of one or more real backend
service, jms queue etc.
• Address, HTTP Endpoints – A logical endpoint for an
existing BE service
• Load balancing / Failover – Group of endpoints for
existing BE services
Message Store and Message Processor
• Asynchronous/One-way Messaging, JMS
• Store and forward
• Guaranteed Delivery and Rate matching
• JMS Message broker as the persistence store – WSO2 MB, Apache
Active MQ etc.
Message Processor in a Cluster
• One consumer per cluster – Guaranteed in-order delivery
• Multiple consumers – Guaranteed delivery
One Consumer Multiple Consumers
Tasks
• Allow configuration of scheduled jobs to execute internal/external
commands
• Inject a message into a proxy service or a named sequence
Service Chaining
• Orchestrating/Combining multiple Services
Load Balancing
Enterprise Integration Patterns
https://docs.wso2.com/display/IntegrationPatterns/Enterprise+Integration+Patterns+with+WSO2+ESB
Tooling Support for WSO2 ESB
WSO2 ESB 5.0 – Upcoming release
• Beta version is available (Released on May 2016)
• Improved usability of the ESB through debugging, data mapping capabilities through developer studio tooling
• The product is released as a tuple of runtime, analytics and tooling
• Dedicated analytics distribution(DAS) for ESB specific monitoring with pre built dashboards
WSO2 ESB 5.0 – Statistics/Tracing
• Coarse grained statistics about proxy/sequence/api/endpoint
WSO2 ESB 5.0 – Statistics/Tracing
• Fine grained statistics/tracing on mediators (message level)
WSO2 ESB 5.0 – Statistics/Tracing
• Fine grained tracing on messages
WSO2 ESB 5.0 – Visual Data Mapper
• Visual Data mapper– Different input/output options (XML, JSON, Files, Schemas)
– Apply functions while mapping the data (Split, Aggregate, Uppercase, Lowercase)
WSO2 ESB 5.0 – Mediation debugger
• Debug mediation flows from the Developer studio
WSO2 ESB – What’s new in 5.0
Other new features
• JMS 2.0 support
• Websocket support
• JMS distributed(XA) transaction support
• Rabbitmq/MQTT performance improvements
WSO2 Integration Cloud (Upcoming)
WSO2 Integration Cloud (Upcoming)
Integration Platform as a Service (iPaaS)
• Running integrations on the cloud
• Analytics integration
• Develop integrations using WSO2 Developer studio
WSO2 Integration Platform
Business Processes
WSO2 Business Process Server
• BPMN 2.0• WS-BPEL 2.0• WS-HumanTask 1.1• BPEL4People
● Define and execute business processes
● Define workflows interacting with People
● Graphical process modeling/administration/monitoring
Human Task Support
WSO2 Business Rules Server
● Integrates JSR94 compliant rule engines.
● Embed drools engine.
WSO2 Integration Platform
Fundamentals of WSO2 Message Broker
Messaging Models● RPC Style
- Request/Response (there is always a response)- Synchronous (client waits for response)- Non-persistent (message is lost if something failed)
● Point to Point (Queues)- A message is delivered only once to a single Broadcast a
message to all the subscribers● Publish/Subscribe (Topics)
- Broadcast a message to all the subscribers
Fundamentals of WSO2 Message Broker
Messaging Models
● Publish/Subscribe (Topics)- Broadcast a message to all the subscribers
● Durable Subscriptions- Same as topic subscriptions- Subscriber offline - Messages get queued- Subscriber back online - Messages get delivered- Unsubscription needed to remove queuing
Fundamentals of WSO2 Message Broker
● Transactional- Send/Receive multiple messages in a transactions.- ( send/consume -> rollback, reject, commit)
● Guaranteed Delivery/Acknowledgments , QOS ( MQTT, JMS).● Persistence
- Messages kept in memory or in reliable storage.
Fundamentals of WSO2 Message Broker● Dead letter channel
- Broker keeps invalid/rejected messages in this channel.- Users can inspect/reroute/delete these messages.
● Hierarchical topics
- Map relationships between data being published onto different topics.
- Can Subscribe to part of the hierarchy.
Games.* - receives messages published to ‘Games.FootBall’
and ‘Games.Cricket’
Games.Cricket.# - Receives messages published to
‘Games.Cricket.SriLanka’ and ‘Games.Cricket.UK’
Fundamentals of WSO2 Message Broker
Distributed Queues
● Provides strict or best effort support for inorder delivery
● There are no guarantee about the global order seen across subscribers
Fundamentals of WSO2 Message Broker
Distributed Queues with MB
● Provides strict or best effort support for inorder delivery
● There are no guarantee about the global order seen across subscribers
● Supported Protocols AMQP/JMS, MQTT
MB Roadmap
• Support for AMQP 1.0 and JMS 2.0
• Improvements to Storage Scheme, Performance improvements
• Analytics for MB
• C5 migration
• Improved GUI
• MQTT + Websockets
WSO2 Integration Platform
Introducing WSO2 Data Services Server
Painpoints: ● Application Silos / Different data
stores● Cumbersome data access/
transformation logic● Business logic vs Data access logic● Repetitive Code
DSS:● Well defined interfaces (standard based)● Encapsulated data logic● Configuration driven● Loose Coupling between App & Data stores● Scale as a separate architectural layer.
WSO2 Data Services Server : Features
• Multiple Data Sources
• Nested Queries
• Batch Processing
• Distributed Transactions
• OData (www.odata.org)– Quickly expose data as REST APIs
– Standardized [URL conventions, Request/Response Headers,
Status Codes ,Payload formats]
Demonstration - Use Case
• As part of an effort to improve traffic monitoring process, City transport authorities are making buses to send useful information while it's in operation such as speed, location data and direction.
• To ensure safety of passengers, officials would like to detect bus drivers who violate speed limits. Supervisors will investigate these incidents case-by-case basis.
• If a supervisor determines a particular incident is undue, bus driver will be placed under further investigation/review.
Demonstration - Solution Architecture
Demonstration - Concepts (OData)
Retrieve duty roster: GET http://localhost:9765/odata/AllocationData/DriverAllocations/duty_roster(1){ "@odata.context": "$metadata#duty_roster/$entity", "@odata.etag": "8d32bf38-8d8f-30cb-8e87-36ebeaedc2a9", "vehicleId": 1, "driverId": 1}
Retrieve employee information:GET http://localhost:9765/odata/HRDataAPI/records/employee(1){ "@odata.context": "$metadata#employee/$entity", "@odata.etag": "58bff0d2-09a4-3315-b2a0-667626dbf5c7", "id": 1, "supervisorId": 2, "username": "driver", "status": "UnderReview", "name": "Driver"}
Update Employee status:PATCH http://localhost:9765/odata/HRDataAPI/records/employee(1){ "status" :"UnderReview"}
Demonstration - Concepts (Inbound Endpoints)<inboundEndpoint name="speed-alert-receiver" onError="fault" protocol="jms" sequence="BusSpeedViolationsSeq" suspend="false" xmlns="http://ws.apache.org/ns/synapse"> <parameters> <parameter name="interval">1</parameter> <parameter name="sequential">true</parameter> <parameter name="coordination">true</parameter> <parameter name="transport.jms.Destination">SpeedViolations</parameter> <parameter name="transport.jms.CacheLevel">3</parameter> <parameter name="transport.jms.ConnectionFactoryJNDIName">QueueConnectionFactory</parameter> <parameter name="java.naming.factory.initial">org.wso2.andes.jndi.PropertiesFileInitialContextFactory</parameter> <parameter name="java.naming.provider.url">repository/conf/jndi.properties</parameter> <parameter name="transport.jms.SessionAcknowledgement">SESSION_TRANSACTED</parameter> <parameter name="transport.jms.SessionTransacted">true</parameter> <parameter name="transport.jms.SubscriptionDurable">false</parameter> <parameter name="transport.jms.ConnectionFactoryType">queue</parameter> <parameter name="transport.jms.ReceiveTimeout">100</parameter> <parameter name="transport.jms.ContentType">application/json</parameter> </parameters></inboundEndpoint>
<inboundEndpoint name="hr-request-receiver" onError="fault" protocol="http" sequence="HRSeq" suspend="false" xmlns="http://ws.apache.org/ns/synapse"> <parameters> <parameter name="inbound.http.port">8081</parameter> </parameters></inboundEndpoint>
Demonstration - Concepts (Endpoints)
<endpoint name="DutyRosterEP" xmlns="http://ws.apache.org/ns/synapse"> <http method="get" trace="disable" uri-template="http://localhost:9765/odata/AllocationData/DriverAllocations/duty_roster({uri.var.busId})"/></endpoint>
<endpoint name="GetEmployeeRecordsEP" xmlns="http://ws.apache.org/ns/synapse"> <http method="get" trace="disable" uri-template="http://localhost:9765/odata/HRDataAPI/records/employee({uri.var.employeeId})"/></endpoint>
<endpoint name="UpdateEmployStatusEP" xmlns="http://ws.apache.org/ns/synapse"> <http method="patch" trace="disable" uri-template="http://localhost:9765/odata/HRDataAPI/records/employee({uri.var.employeeId})"/></endpoint>
Usage:
<property expression="json-eval($.driverId)" name="uri.var.employeeId" scope="default" type="STRING"/><call> <endpoint key="GetEmployeeRecordsEP"/></call>
Demonstration - Concepts (Data Mapping)
<datamapper config="gov:datamapper/MapSpeedViolationResultToHRSystem.dmc"
description="Map-Speed-Violation-OutCome-To-HR-System"inputSchema="gov:datamapper/MapSpeedViolationResult
ToHRSystem_inputSchema.json" inputType="JSON"
outputSchema="gov:datamapper/MapSpeedViolationResultToHRSystem_outputSchema.json" outputType="JSON" />
Demonstration - Concepts (Payload Messages and Service Chains)
<sequence name="BusSpeedViolationsSeq" onError="fault" statistics="enable" trace="disable"
xmlns="http://ws.apache.org/ns/synapse">
<property expression="json-eval($.event.payloadData.id)"
name="uri.var.busId" scope="default" type="STRING"/>
<property expression="json-eval($.event.payloadData.id)"
name="busId" scope="default" type="STRING"/>
<property expression="json-eval($.event.payloadData.timeStamp)"
name="timeStamp" scope="default" type="STRING"/>
<property name="uri.var.busId" scope="default"
type="STRING" value="1"/>
<call>
<endpoint key="DutyRosterEP"/>
</call>
<property expression="json-eval($.driverId)"
name="uri.var.employeeId" scope="default" type="STRING"/>
<property expression="json-eval($.driverId)"
name="driverId" scope="default" type="STRING"/>
<call>
<endpoint key="GetEmployeeRecordsEP"/>
</call>
Demonstration - Concepts (Payload Messages and Service Chains)
<property expression="json-eval($.username)"
name="driverUsername" scope="default" type="STRING"/>
<property expression="json-eval($.name)"
name="driverName" scope="default" type="STRING"/>
<property expression="json-eval($.supervisorId)"
name="supervisorId" scope="default" type="STRING"/>
<property expression="json-eval($.supervisorId)"
name="uri.var.employeeId" scope="default" type="STRING"/>
<call>
<endpoint key="GetEmployeeRecordsEP"/>
</call>
<property expression="json-eval($.username)"
name="supervisorUserName" scope="default" type="STRING"/>
Demonstration - Concepts (Payload Messages and Service Chains) <payloadFactory media-type="json">
<format> {"processDefinitionKey": "trafficViolations","businessKey": "trafficKey","tenantId": "-1234",
"variables": [
{"name": "vehicleId", "value": "$1"},
{ "name": "timeStamp", "value": $2 },
{ "name": "driverId", "value": "$3"},
{ "name": "driverUserName", "value": "$4"},
{ "name": "driverName", "value": "$5"},
{ "name": "supervisorUserName", "value": "$6" }]}
</format>
<args>
<arg evaluator="xml" expression="$ctx:busId" xmlns:ns="http://org.apache.synapse/xsd"/>
<arg evaluator="xml" expression="$ctx:timeStamp" xmlns:ns="http://org.apache.synapse/xsd"/>
<arg evaluator="xml" expression="$ctx:driverId" xmlns:ns="http://org.apache.synapse/xsd"/>
<arg evaluator="xml" expression="$ctx:driverUsername" xmlns:ns="http://org.apache.synapse/xsd"/>
<arg evaluator="xml" expression="$ctx:driverName" xmlns:ns="http://org.apache.synapse/xsd"/>
<arg evaluator="xml" expression="$ctx:supervisorUserName" xmlns:ns="http://org.apache.synapse/xsd"/>
</args>
</payloadFactory>
<property name="messageType" scope="axis2" type="STRING" value="application/json"/>
<property expression="fn:concat('Basic ', base64Encode('admin:admin'))" name="Authorization" scope="transport"
type="STRING" xmlns:ns="http://org.apache.synapse/xsd"/>
<call> <endpoint key="CreateBPMNWorkFlowEP"/> </call>
<drop/>
</sequence>
Demonstration - Concepts (Payload Messages and Service Chains)<sequence name="HRSeq" statistics="enable" trace="disable"
xmlns="http://ws.apache.org/ns/synapse">
<property expression="json-eval($.driverId)"
name="uri.var.employeeId" scope="default" type="STRING" />
<datamapper config="gov:datamapper/MapSpeedViolationResultToHRSystem.dmc"
description="Map-Speed-Violation-OutCome-To-HR-System"
inputSchema="gov:datamapper/MapSpeedViolationResultToHRSystem_inputSchema.json"
inputType="JSON"
outputSchema="gov:datamapper/MapSpeedViolationResultToHRSystem_outputSchema.json"
outputType="JSON" />
<property name="FORCE_SC_ACCEPTED" scope="axis2" type="STRING" value="true" />
<property name="OUT_ONLY" scope="default" type="STRING" value="true" />
<call description="">
<endpoint key="UpdateEmployStatusEP" />
</call>
</sequence>
Demonstration - Concepts (BPMN Workflows)
Next generation Integration Platform
• WSO2 Gateway Framework– An ultra high performance, lightweight and reusable message
Gateway framework that encapsulates the messaging between source and target systems
– HTTP/s message gateway leveraging Netty, LMAX Disruptor and WSO2 Pass-Thru messaging architecture.
WSO2 Integration product suite
• Series of products based on WSO2 GW Core framework.
WSO2 Process Center
Process owner / analyst
Process participant
Developer
Create / improve / standardize Search / browse
/ follow
View
Develop executable processes
Executable process is associated with Process Center
Thank You!
#WSO2ConEU
Share your feedback for this session
wso2con.com/app