M19

© IBM Corporation 2011

IBM WebSphereTechnical Conference 2011October 10-14, 2011 Berlin, Germany

M19 WebSphere ESB Best Practices and Performance Recommendations


Callum Jackson | WebSphere ESB Runtime Architect | IBM

IBM WebSphere Technical Conference 2011 - Berlin


Agenda

� Brief Product Recap

� Best Practices

– Planning, Design and Architecture

– Process

– Development

– Performance

� References and Further Information

3



WebSphere ESB Essentials: Family Values...

WebSphere Application

Server

WebSphere Application Server ND

WebSphere ESB

IBM Business Process ManagerAdvanced Edition

App Server

Clustering

Mediation

ChoreographyAnd Solution

Viewpoint

4



Service Component Architecture

SCAComponentIExportI

ImportI

SCAComponent

I

R

R

SCA Module

ExportI

Request / Reply

One-way

Request /

Reply

5



Module and Component Types

� WebSphere ESB supports only Mediation Modules, with these components:

– Mediation Flows

– Java

� IBM BPM Advanced Edition also supports (Integration) Modules, with these additional components:

– BPEL (Process)

– Human Task

– Rule Group (Business Rules)

– etc…

6



Connectivity Support

• Supported Connectivity Bindings and Adapters for import and export nodes

– Web Services (SOAP/HTTP, SOAP/JMS)

– Messaging (JMS, MQ, JMS MQ, generic JMS (3rd party providers))

– J2EE Applications (EJB binding)

– Native (SCA cross module)

– HTTP Access (JSON, XML)

– IBM CICS ECI Resource Adapter

– IBM IMS Connector for Java

– IBM WebSphere Adapter for Email

– IBM WebSphere Adapter for FTP

– IBM WebSphere Adapter for Flat Files

– IBM WebSphere Adapter for JDBC

– IBM WebSphere Adapter for Lotus Domino

– IBM WebSphere Adapter for Enterprise Content Management

– IBM WebSphere Adapter for JD Edwards EnterpriseOne® *

– IBM WebSphere Adapter for Oracle® E-Business Suite *

– IBM WebSphere Adapter for PeopleSoft *

– IBM WebSphere Adapter for SAP® Software *

– IBM WebSphere Adapter for Siebel® Business Applications *

6

* runtime license required

7



Best Practices: Planning, Design and Architecture

• Brief Product Recap

• Best Practices


– Process

– Development

– Performance

• Summary

• References and Further Information



Use the Right Type of Module (1/2)

• Design consideration –mediation logic vs. process logic

• Use Mediation Modules (WebSphere ESB & BPM Advanced) for Mediation Flows – integration / mediation logic:

– Short-running, minimal choreography (augment,aggregate)

– Supports header manipulation

• Use (Integration) Modules (BPM Advanced only) for BPEL – business / process logic:

– Can be long-running, provides powerful choreography

• More Information:

– http://www.ibm.com/developerworks/websphere/library/techarticles/0803_fasbinder2/0803_fasbinder2.html



Use the Right Type of Module (2/2)

• As of V6.2, mediation flow components can be used in (Integration) Modules

• Can be necessary in some cases to mix mediation and business logic in same module e.g. performance

• In some limited cases, can also make sense to treat BPEL as a ‘visual programming language’ for complex integration

• However, be careful to keep the line between mediation and process logic clear in your design

10



Exploit supplied Integration Patterns

• Where possible exploit the supplied integration patterns

– Accelerate mediation development

– Generated modules follow best practices

• Currently available integration patterns:

– Static simple service proxy

– Dynamic simple service proxy

– Static service gateway

– Dynamic service gateway

– Proxy service gateway

– Service translator (Feature Pack)

– Service selector (Feature Pack)



Understand Invocation Styles

• Default ‘invocation style’ typically dictated by bindings – often OK but be aware of it

• Async interactions (between components or modules) go via an SCA/JMS/MQ queue which implies:

– Break in transactional scope

– Runtime exception causes rollback which after retry limit results in message going to exception destination – then handled via Failed Event Manager

• Can be hard to predict when interactions will be async – subtleties in performance optimisation etc.

• Assume async if in doubt, and use ‘preferredInteractionStyle’ liberally

• More details:

– http://www.ibm.com/developerworks/websphere/library/techarticles/0811_chacko/0811_chacko.html

12



Understand Invocation Styles

• An example of unintended asynchrony

– Preferred Interaction Style = Any

Async exports like JMS, MQ, and SCA Async, set interaction

style to “async”

Mediation flow component propagates caller’s interaction style if callee’s PI style is Any

Unintended asynchronous calls

13



Design Transactions

• Important to design transactions– Affects what happens when errors occur

– Integration Designer defaults are only a starting point

– If you are interacting with JDBC or a messaging system – probably want transactionality

• Guiding principles– Every transaction commit adds overhead

– Minimize transaction commits to implement desired transactional behavior

– Pay particular attention to SCA transaction qualifiers

14



Design Transactions - SCA Qualifiers

• Ensure transactions propagate across components until the desired commit point

• To achieve this, follow the Golden Path

– suspendTransaction = False

– Transaction = Any or Global

– joinTransaction = True

joinTransaction = True

Transaction = Any Transaction = Any

suspendTransaction = False



Design Transactions - SCA Qualifiers

• Use the Transaction Highlighting and Qualifiers editor

• More information: http://soatipsntricks.wordpress.com/2008/07/31/transactionality-in-sca-part-2-refactoring-interfaces/



Select your ‘Default’ Binding Carefully

• Often the binding type is dictated by circumstances

• But if you have the scope to decide:

– Prefer SCA default/native for intra/inter-ESB

communications – fast, efficient, and simple

– Prefer Web Services (JAX-WS) for synchronous service

exposure – mature, integrates well into SDO model

– Prefer JMS for asynchronous service exposure – integrates

well with Application Server platform



Consider How you Split up Modules

• How much content inside each module?

– Large number of modules impacts memory consumption / deployment / failover

– Small number impacts ease of development – in general, best not to have more than one

developer per module

– Default is share by copy for libraries – may result in large memory usage

– Consider share by reference – implications on module versioning

http://www-01.ibm.com/support/docview.wss?rs=2307&uid=swg21322617

Module A Module B

Library C

Module A

Library C

Application A Application B

Module B

Library Cbuild



Spend Time on Interfaces and Business Objects

• Refactoring support doesn’t allow you to arbitrarily change a design, so get as right as possible upfront

• Especially important for versioned long-running processes and for new live process migration in v7

• Adopt a naming convention

• Add constraints?

• Add modelled faults?

• Use namespaces – define an enterprise policy

• Configure default namespace policy before you start

19



Use mediation policies to control points of variability

• Mediation policies

– Allow declarative control of mediation function by externalizing mediation points

of variability

• Benefits

– Reduce mediation construction complexity

– Much more flexible mediation dynamic control

• Supported by supplied capabilities

– Dynamic context

– Policies authored via Business Space stored and governed in WSRR

– Policy Resolution mediation primitive

20



Best Practices: Process


• Best Practices


– Process

– Development

– Performance




Consider your Logging Strategy

• You will want one!

• Why logging - Diagnostics? Auditing? Monitoring?

• Options include:

– Message Logger (Mediations) – logs to classes implementing java.util.logging (as

of v6.2)

– Trace Primitive (as of v7) – not intended for production use

– JDBC or Flat File Adapter (in separate module?)

– Basic visual snippets for logging

– Custom primitives

– Cross-component Trace Facility

• Use subflows or separate module for common logging logic

• Turn trace off when not needed

22



Logging Best Practices

• Think about granularity

– Logging on entry to services and calls to other services –

‘topping and tailing’ – typically a good starting point

• Include a unique identifier for each interaction

• Property promotion can be good to disable logging



Use Source Control

• Use source control – Integration Designer/Eclipse integrates with several

• Only one developer per module at once – lock appropriate files if possible

– Don’t try to merge module changes – complex and error-prone

• Don’t keep too much checked out – project sets can help here.

• Do not check in generated artifacts

– <xxx>App, <xxx>EJB, and <xxx>Web projects, and anything marked in Eclipse as ‘derived’ - more information here: http://www.ibm.com/developerworks/websphere/library/techarticles/0604_beers/0604_beers.html

• In some rare circumstances, you need to make changes to generated files

– Use ANT scripts to make those changes when needed, so they can be repeated



Do Repeatable Unit Testing

� As of version 6.1, WebSphere Integration Developer has support for unit testing (called component testing)

� IBM Integration Designer 7.5 adds the ability to generate a test case from an Integration Test client execution

� Use it before check-in.

� Can be run from command line as part of automation.

� Can be deployed to server and run from Web-based GUI.

� Make sure tests can be run by next developer – check them in.

� Article with more information:– http://www.ibm.com/developerworks/websphere/library/techarticles/0806_gregory/0806_greg

ory.html

25



Best Practices: Development


• Best Practices


– Process

– Development

– Performance




Handle Faults Appropriately

• Two types of fault:

– Modelled faults

• Appear on the interface

• a.k.a business, checked faults

• Can have business or technical purposes

– Unmodelled faults don’t appear on an interface

• a.k.a. system, runtime, or unchecked

• Should have only a technical purpose

• Make provisions to handle both types

– Handle unmodelled faults as close to the source as possible

– Handle modelled faults where appropriate

27



Don’t use (Technical) Headers for Business Data

• Includes SOAP Headers, MQ Headers, JMS Headers, etc.

• Intended for point-to-point interactions for communicating technical information (routing, QoS, etc.)

• Only supported by mediation flows – best on ‘boundaries’

• Can require extra custom work, e.g. MQ Header Binding

28



Keep Schemas and WSDLs Simple

• Avoid anonymous complex types

• Don't use empty/null namespaces

• Don’t use the same namespace in two libraries/projects

• Use the gamut of XSD types: don't make everything an xsd:string



Use the Range of Debugging Tools Available

• Integration Test Client

• Debugger

– Supports Breakpoints, Step Over,

Inspection of SDO/SMO, etc.

• Cross-component trace integrated with the Server Log view

• Logging

• Generic Service Client (V7.5)



Understand your Message Manipulation Choices in Mediation Flows

• Message Element Setter

– Simple and high performance

– Cannot alter message type

– Parts of element map are directly promotable

• XSLT

– Prefer when you want to use XSLT functions or work with XSLT directly

– Better performance when dealing with untouched XML data – generally when

working with Web Services (but test relative performance!)

• BO Map

– If you want to share BO maps with WPS, use relationship service, or need/want

ordering capabilities of the BO mapper:



Promote Properties where Relevant

• Link them with the same alias name, where relevant, so that they can be modified together



Promote Properties where Relevant

• From V6.2, promoted properties are dynamic – can be overridden by values set in SMO dynamic context

• Policy Resolution mediation primitive allows property values to be resolved from WSRR based on ‘gate conditions’ from message content

– Allows governance of property values

– Potentially reduces flow complexity

• Not to be confused with policy on Web Service endpoints, for example to setup WS-Security with JAX-WS

33



Replace Older Custom Code with New Functionality

• UDDI Endpoint Lookup Primitive (7.0)

• Trace (7.0) / Enhanced Message Logger (6.2) for logging & tracing

• Message Validator (7.0)

• Flow Order Primitive (7.0)

• Type Casting in XSLT Map (7.0)

• … and many others!

34



Best Practices: Performance


• Best Practices


– Process

– Development

– Performance


35



Design and Use System Topology Early

• Reasons to cluster

– High availability

– Scalability

– Better HW utilization

– Managability

– Performance is only one consideration

• We recommend the “Remote Messaging and Remote Support” topology pattern



Design and Use System Topology Early

• Make your development and test environments look like production (including security settings)

• Use the real database and other supporting systems as early as possible

• Need a load balancer / HTTP server / proxy server?

• What other systems are you connecting to and how will they ensure failover / scalability?

• Redbook that discusses production topologies:http://www.redbooks.ibm.com/redpieces/abstracts/sg247854.html

37



Large Objects – Handle with Care

• Guiding principles

– Large object capability much improved from v6.1 onward, due to IBM J9 JVM

– 64-bit support effectively removes heap size limits

– Still inefficient to flow large data that is never touched through the system

• Best practices

– When large payload is a batch, send as multiple smaller payloads

– When only a small portion of an input message is used by the workload, use a “Claim Check” pattern

• http://www.ibm.com/developerworks/websphere/library/techarticles/1006_kharlamov/1006_kharlamov.html

– Exploit 64-bit when necessary

– Stay up-to-date – performance increases from release to release

– From V7.0.0.3, consider using Business Object Lazy Parsing mode

38



Business Object Lazy Parsing Mode

• New business object runtime optimized for XML processing and transformation

• Uses a data representation more closely coupled to XML

• Offers significant performance advantages for XML-oriented scenarios

• Available from WebSphere ESB 7.0.0.3 with interim fix XC70031 and WebSphere Integration Developer 7.0.0.3 Ifix001

• Default for new projects in IBM Integration Designer

39



Significant performance gains for large messages and complex flows

• Relative performance of lazy parsing mode versus eager mode varies according to the usage scenario

• Performance benefit is greatest when:

– Processing large XML documents, particularly when only partially accessed

– Using mediation flows that combine transformation and other primitives, particularly those

that use XPath

• For example, the following example banking flow (based on IFX schemas) performs up to 428% faster in lazy parsing mode

• Heap occupancy is also reduced with lazy parsing resulting in improved large message capability

MessageElementSetter

FanOut

BO Mapper

ServiceInvoke

BO Mapper

BO Mapper

ServiceInvoke

BO Mapper

FanIn


MessageFilter


40



Configuring lazy parsing mode

Business Integration view – context menu

Wizard for selecting parsing mode

41



Exploit Deferred Parsing

• Even when using eager parsing mode, some WebSphere ESB primitives can exploit an optimization which lowers XML processing costs in certain cases

• The benefit varies by primitive and its application as follows:

– Category 1 (greatest benefit)

• Message Filter (/header as root)

• XSLT Primitive (/body as root)

• Endpoint Lookup without XPath user properties

• Event Emitter (CBE Header Only)

– Category 2 (medium benefit)

• Message Filter (/body as root)

– Category 3 (no benefit)

• All other primitives

42



Exploit Deferred Parsing

• Best Practices

– Flows with only Category 1 primitives will perform best

• Where possible use protocol headers for decision making

• Include key information in SOAP, JMS, HTTP headers

– The sequence of primitives in a flow is important

• Place Category 1 primitives upstream of Category 2

• Place Category 2 primitives upstream of Category 3

• A Category 3 primitive precludes optimization of any downstream primitives

– All primitives have some invocation overhead, so minimizing the number of primitives used will also be helpful

43



Message enrichment by service invocation

• Simplified development of scenario where service invocation is used to enrich message content

• Previously required three mediation primitives:

1) XSLT to save message body to context

2) Service invoke

3) XSLT to merge results with original message body

●Explicit message enrichment mode added to Service Invoke mediation primitive

●XPaths specified to define location of request parameter for service invocation and location to place response

7.57.5NEWNEW

7.57.5NEWNEW

44



Aggregation Considerations

• Only carry the required data within the Service Message Object, consider the following techniques were possible:

– Remove/Delete the body after the FanOut primitive

– Only map the minimal information within a map

– Don't map the shared context after the FanIn has generated the

output message

• Consider if the Service Invoke Message Enrichment mode can be used

• Try to use BO Mappers as extensively as possible

45



Mapping Technology Selection

If the functionality supported by the BOMapper mediation primitive is sufficient for the data transformations then runtime performance will be significantly improved by using the BOMapper mediation primitive in place of the XSL Transformation (XSLT) mediation primitive. The observed improvement in runtime performance is directly linked to the size of the message being processed. The XSLT mediation primitive does provide additional functionality not supported by the BOMapper mediation primitive, so there may be circumstances that this is not viable.



References

• InfoCenter:

– http://publib.boulder.ibm.com/infocenter/dmndhelp/v7r0mx/index.jsp

• Support Sites:

– http://www-01.ibm.com/software/integration/wsesb/support/

• developerWorks:

– http://www.ibm.com/developerworks/websphere/zones/businessintegration/wesb.html

• developerWorks Forum:

– http://www.ibm.com/developerworks/forums/forum.jspa?forumID=1672

• IBM Education Assistant:

– http://publib.boulder.ibm.com/infocenter/ieduasst/v1r1m0/index.jsp

• WebSphere ESB Development Guide:

– http://www-01.ibm.com/software/integration/wsesb/library/

• SOA Tips ‘n’ Tricks Blog:

– http://soatipsntricks.wordpress.com/



Summary

� Brief Product Recap

� Best Practices


– Process

– Development

– Performance

� References and Further Information

48



Questions?

49



Copyright Information

© Copyright IBM Corporation 2011. All Rights Reserved. IBM, the IBM logo, ibm.com, AppScan, CICS, Cloudburst, Cognos, CPLEX, DataPower, DB2, FileNet, ILOG, IMS, InfoSphere, Lotus, Lotus Notes, Maximo, Quickr, Rational, Rational Team Concert, Sametime, Tivoli, WebSphere, and z/OS are trademarks or registered trademarks of International Business Machines Corporation in the United States, other countries, or both. If these and other IBM trademarked terms are marked on their first occurrence in this information with a trademark symbol (® or ™), these symbols indicate U.S. registered or common law trademarks owned by IBM at the time this information was published. Such trademarks may also be registered or common law trademarks in other countries. A current list of IBM trademarks is available on the Web at “Copyright and trademark information” at ibm.com/legal/copytrade.shtml.

Coremetrics is a trademark or registered trademark of Coremetrics, Inc., an IBM Company.

SPSS is a trademark or registered trademark of SPSS, Inc. (or its affiliates), an IBM Company.

Unica is a trademark or registered trademark of Unica Corporation, an IBM Company.

Java and all Java-based trademarks and logos are trademarks of Oracle and/or its affiliates. Other company, product and service names may be trademarks or service marks of others. References in this publication to IBM products and services do not imply that IBM intends to make them available in all countries in which IBM operates.

M19

Documents

Transcript of M19