Positions on serviceoriented architecture

Insight |

Service Oriented Architecture has exceeded the hype peak and is now reaching the level of maturity of proven technology.

Nevertheless, there is still the need to describe positions and approaches for the implementation of the technology and – even more

important – the methodology. This document, based of a number of blog entries I wrote over the last years, describes a number of

these positions. Statements made here are based on real life experiences and my reflections on questions and discussions.

Contents

Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .4

Position 1. Maturity Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .5

Position 2. Processes First . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .9

Position 3. Changing or Adapting Infrastructure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .11

Position 4. Information Management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .14

Position 5. Canonical vs. Integration Data Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .15

Position 6. Building Re-Useable Components . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .17

General Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .18

About the author . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .19

About BearingPoint . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .19

4 Insight |

Introduction

One of the most challenging and exciting parts of working as an architect and consultant with different clients is theexposure to all kind of issues and challenges. And – of course – the discussion with colleagues in the situation aboutsolutions and aspects of the different issues encountered. This is the part where – as an architect – one learns twomain things: You need to listen to all the different views, and also (and most important) you have to develop positionsas a basis for the discussions. This document is a compilation of positions. As positions they do not represent thesolution to problems, but a starting point for communication.

In this text I describe my point of view on

- Maturity Models,- Service Design,- Changing or Adapting Existing Solutions,- Information and Data Management, and- Designing of Re-useable Components.

Some of the positions you may agree with, some of them you might want to discuss. Please feel cordially invited todo so. I look forward to hear from you…

As you read this document we are working to document additional positions, covering different topics, such asGovernance, Security of Services, and Importance of non-functional requirements. If you are interested in these topics,please contact us and we will provide you with the material as it is published.

As service oriented architecture grows in popu-larity and many CIOs/CTOs get worried abouttheir "old" EAI environments, many companiesare looking into an upgrade of their businessand process integrations. These concerns relateto different aspects for their existing environ-ment: Is there too much organizational and cul-tural change required to achieve the objectiveand will it be too much pain? How about invest-ments made in infrastructure and licenses?

Below, we will look on different aspects and provide astarting point.

Service orientation is not a completely new concept arri-ving out of the blue. It is in many areas a consequence oflessons learned during early integration approaches. Ifthese lessons are applied to existing environments it ispossible to harvest the benefits of service orientationwithout wasting already performed infrastructure work.Upgrading from an integrated environment to a serviceoriented environment can be an evolutionary process - itdoes not necessary have to be a revolution of the ITdepartment.

To define this evolutionary roadmap, maturity modelsprovide a progress statement and guidelines for the orga-nization. Most organizations aim for higher maturitylevels, to increase their proficiency and gain benefitsfrom the increased maturity of the organization. If thereare maturity models for both (integration and serviceorientation) – is it possible to show that in essence theydirect towards the same destination? In this section twodifferent maturity models are mapped to demonstratethat, even if the philosophy and target between enter-prise integration and service orientation are different,there is significant common ground that allows changeof direction without losing the benefits of the work donein the organization.

Let's talk about maturity levels. This approach was takenduring early roll outs of EAI to identify the level of inte-gration reached in the organization. The idea was todefine a roadmap for companies and to provide a planhow to gain most out of integration. These integrationmodels varied from each other based on the group thatrolled out the maturity level. Differences can be seen bet-ween software vendors, integrators, and managementconsultants. The first were most interested to show howtheir products could evolve at the client, the second hadan interest to promote their approach, and the lastgroup tried to put themselves on the bandwagon by get-ting their company involved to perform the necessarychanges in the organizational infrastructure of theirclients.

Nevertheless, the maturity models are a useful tool toprovide maturity levels for the integration achieved inthe organization. By merging the different views onmaturity in integrated environments, we establish thefollowing model.

Figure 1: Integration Maturity Model

Insight | 5

Position 1. Maturity Models

Level 4

Level 3

Level 2

Level 1

Level 0 Stand-alone application with few interfaces,stovepipe processes, manual re-entry of data

Point-to-point integration

Structural integration

Processintegration

Externalintegration

To demonstrate the merging of maturity models we arenow introducing a model for service oriented architec-tures.

BearingPoint’s model for SOA maturity will be used forthis purpose2 .

1 Note: External integration in this context does not mean an integra-

tion with an external business (B2B integration) entity, but managed

integration outside of the internal domain. This is also described with

clusters.2 The complete model can be retrieved on the BearingPoint home

page or from the local BearingPoint office.

6 Insight |

Level Name Description

0 Pre-integration Stand-alone systems with few interfaces,“Stovepipe” processes,Manual re-entry of data bet-ween applications

1 Point-to-point Custom application-to-appli-cation interfaces using APIs,Middleware tools

2 StructuralIntegration

Middleware tools,Middleware features,Enterprise applications inter-face models exist

3 ProcessIntegration

Information not just shared,but managed between sys-tems, Middleware includesprocess automation mode-ling tools, Middleware fea-tures workflow modeling,automated decisions,Enterprise business modelexists

4 ExternalIntegration1

Leverages EAI enabled appli-cations from previous levelsacross applications

Level Name Description

1 Initial Services Initial Services represent theinitial learning and initialproject phase of SOA adop-tion. Projects here are typi-cally done to simultaneouslymeet a specific need toimplement functionalitywhile trying out specifictechnologies and anapproach to SOA. This matu-rity level also includes initialR&D activities testing theSOA technologies in a labora-tory environment. Usually,the initial introduction ofSOA is driven by the applica-tion development organiza-tion - often as part of anapplication integration pro-ject. New development skillsare learned and initialattempts at quantification ofROI are created.

2 ArchitectedServices

It is at this level that stan-dards are set as to the tech-nical governance of SOAimplementation, typicallyunder leadership of thearchitecture organization.The key business benefit ofthis level is development anddeployment cost reductionsthrough the use of SOA stan-dard infrastructure and com-ponents as compared tousing older technologies orcosts accumulated throughmultiple unique one-timeprojects. These benefits aregreater in the heterogeneousenvironments typical of mostenterprises.

This SOA Maturity Model provides a framework for dis-cussion between IT and business users about the applica-bility and benefits of SOA in an organization across fivelevels of maturity.

Like the integration maturity model, the SOA modelfocuses on the alignment of business and integration.

Using the same representation as for the EAI maturitymodel, the SOA maturity model looks as follows:

The two models actually allow the description of themigration from an integrated environment to a serviceoriented environment. Just by moving the models toge-ther shows the similarities of the models.

It actually also demonstrates the necessary growthwithin an integrated environment to reach the cross-overpoint into the service oriented pyramid.

Insight | 7

Level Name Description

3 Process-BasedServices(PartnershipBusiness –Technology)

The focus of SOA Maturity Level3 is on the partnership betweentechnology and business organi-zations in order to assure thatthe use of SOA provides clearbusiness responsiveness. Core tothe value of SOA is the linkagebetween business process anddigital processes. SOA MaturityLevel 3 is defined with two com-plementary paths to attainingthese goals - one, BusinessServices, focused on the impro-vement of internal business pro-cesses, and two, CollaborativeServices, focused on the impro-vement of collaborative pro-cesses with external partners.

4 ControlledServices

While SOA Maturity Level 3focuses on the implementationof internal and/or external busi-ness processes, SOA MaturityLevel 4 focuses on measuringand presenting these processesat the business level so as toprovide continuous feedback onthe performance and businessimpact of the processes imple-mented at Level 3. This levelincludes Business ActivityMonitoring (BAM) to allow busi-ness users to transform the waythey respond to business events.

5 OptimizedProcesses

SOA Maturity Level 5, OptimizedBusiness Processes SOA, addsautomatic response to the mea-surements and displays of Level4. In this way, the SOA informa-tion systems becomes the "enter-prise nerve system" and takesaction automatically according toevents occurring at the businesslevel according to rules optimi-zing business goals.

Level 3

Level 4

Level 5

Level 2

Level 1

Level 0

Level -1 No integration

Standardized Application Communication(Point - to - Point)

Initial Services

Architected Services

Process Based Services

ControlledServices

OptimizedServices

Level 5

Level -1 No integration

Standardized Application Communication(Point - to - Point)

Initial Services

Architected Services

Process Based Services

ControlledServices

OptimizedServices

Level 4

Level 3

Level 2

Level 1

Level 0 Stand-alone application with few interfaces,stovepipe processes, manual re-entry of data

Point-to-point integration

Structural integration

Processintegration

Externalintegration

The main point of criticism with the two pyramids is thatit implies a clear distinction between enterprise architec-ture integration vs. a service oriented architecture. Inreality this is not the case. The border is more gradual,with most of the more mature integrations being in azone which could be seen as both. As a rule of thumb,the more the basic rules of service architecture are follo-wed, the more of the migration towards SOA is done.

Conclusion

The migration from an integrated environment to a ser-vice oriented environment will always require the inte-gration of the business processes. It is also very highlyvisible that the migration to a higher level of maturitywill require at one point the migration into service orien-ted architecture and methodology. Financial and intel-lectual investments made earlier, following the EAImaturity model, are however not a write off, and a lot ofthe work performed can be adjusted. The objective tochange to a service oriented maturity model is to providea cleaned and extended integration growth path.

8 Insight |

So far I have discussed the maturity of an organizationand the transfer from an integrated environment to aservice oriented environment. The migration path froman (partially) integrated to a service oriented environ-ment requires the review of the business processes.Service orientation is based on the discovery of servicesin the business process and the modeling of these ser-vices in an adequate form in the organization’s IT environ-ment. Changing just the IT side will not give the benefitsexpected – the agility of the environment is still limited byefforts to map business functions to IT services.

This move of paradigms is the main challenge. Previousapplication owners are becoming process owners, whohave to share applications with other process owners.This will feel like (and in many cases is) lost control whenlooking at the applications; the reduced complexity byonly looking at the processes is only valued after themigration is completed.

To illustrate the approach to introduce services in anenvironment I use the following graphic:

Let us have a look at the different steps in detail:

Business Process Definition

The core term so far was ‘process’. When using the term‘process’ in this context I speak about an end-to-endbusiness process, from triggering it by a defined event(e.g. a prospect becoming a customer, a sales event,etc.) to the final process conclusion (customer paysbill…). In the analysis of the process we see all participa-ting parties (persons and applications) and we see howthey fit into the process. Very often just this analysis pro-cess makes the complexity of the process apparent andallows the process owner to document and maybe re-think the process. This alone is already beneficial to thebusiness. It is very important that the process analysis isin the responsibility of the process owners, their accep-tance and input is essential for the successful definitionof the process afterwards. The process owner is suppor-ted by coaches and analysts, who help them to formu-late the process in a prior specified form.

Insight | 9

Position 2. Processes First

BusinessProcess

Definition

Business Actorand ServiceDiscovery

Business ActorService

Consolidation

Updating Business Service

Catalogue

ServiceGeneration

Business Service<-> Service Translation

ServiceRe-Use

Evaluation

ServicesCatalogues

Business Actor and Service Discovery

During the process analysis we will indentify the diffe-rent actors, which contribute to the flow. These actorsprovide input to the flow – they provide a service. Inorder not to confuse this with the technical implementedservice we should call this service a business service. Thebusiness services are the basis for the implementation,but they are not necessarily a one-to-one copy of the ser-vices implemented.

Business Service Consolidation

In a third step all identified services need to be collectedand similarities between the services need to be identi-fied. Out of the generated picture we can identify whatbusiness services the company needs and which actorprovides the services. At this stage it becomes imperativethat existing business services are reviewed and added tothe picture. This can be done by using a service cata-logue, which holds the existing services and makes themavailable to the process owner. It is always better to re-use an existing process (the one which is known to work)then to build one from scratch.

This step is the most important and difficult - mainlybecause the process owner needs to have an open mindabout the processes available in his environment. But ifthis task can be completed successfully the first big steptowards a clean service oriented environment is completed.

Updating Business Service Catalogue

In the last paragraph I used the words Service Cataloguefor the first time. The concept behind it is simple – thebusiness services in an organization need to be collectedand made available to all interested parties. This is doneby a service catalogue. The catalogue should at leastcontain the business services, but can be extended tocontain also implemented services on a technical level.

Once the services have been collected, cleansed and re-used business services have been identified the cata-logue needs to be updated, so that the newly identifiedbusiness services are available. Please note that this doesnot only impact automated services – also services provi-ded by a human actor (such as call center functionalityor manual credit check) can be added into the servicecatalogue.

Business Service <-> Service Translation

As the business services and their actors have been iden-tified it is possible to translate these business servicesinto either work processes or to implement them as ser-vices in the applications. This step can be very formal –the objective is to provide the information to the actualimplementer about the nature and features of the newservices. In the literature a number of different ways todo this can be found, most frequently used are “UseCases”.

Service Re-Use Evaluation

Once the services on technical level are defined it is nowon the designers and developers to review the existingservices and determine whether the new service can besatisfied by combining existing services with new func-tionality. This also determines how and where servicesare built – possibilities can be the orchestration layer (ascombined services), in a service façade or in the applica-tion itself. Most important for the later owner of the ser-vices is that at this stage the non-functional require-ments and standards are in place. This stage is the mostcritical for the technical implementation, comparableto the business service discovery on the functional side.

Service Generation

Finally – the implementation of the services. All neces-sary steps of the software development cycle are perfor-med, performance and integration are tested… Here itshows how well the process definition is done.

Conclusion

From my experience, the described steps in the ServiceLifecycle are the most critical in every project. The archi-tecture fails or stands with the correct modeling of thebusiness services. I called this section “Processes First”not only because the process definition is the first step inthe generation of the flow, which – in the end – shallsatisfy the user, but because in this step it is possible tobuild the communication between the user and thedeveloper, which is a core essential for the acceptance ofthe result.

10 Insight |

In the first section I focused mainly on maturity. This partis more about the idea that in order to establish serviceoriented architecture a certain tooling in the middlewarearea is required. The starting point is that in order tohave a "true" architecture a "proper" service bus needsto be installed, and - as part of this migration - the exis-ting integration infrastructure becomes obsolete andneeds to be removed (or retired).

The definitions of an Enterprise Service Bus (ESB) whichcan be found in literature usually are primarily a state-ment of the opinion of the author. However there are anumber of criteria which all definitions do have in com-mon. These criteria are technology agnostic.

The ESB acts as a transport medium between applica-tions. In literature the ESB is compared to a message bro-ker. I would not necessarily follow this idea, because thebroker is an active component transferring informationbetween applications. The concept of a bus is a more pas-sive component which the publisher feeds into and thesubscriber picks those components from the bus that arerelevant to him. If we would use the 7 layers ISO/OSI net-work model to describe the layering of software, the ESBwould be located between layers 5 and 6, taking oversome functions of the transport and some of the presen-tation. The implementation of the data on the ESB iscompleting the 6th level of the model, presenting thedata in a canonical model.

The characteristics an ESB should fulfill are well descri-bed in literature. The following table (quoted fromWikipedia) gives a list of criteria all ESBs should fulfill. Iunderstand that some of the points are discussed inten-sively. I have not yet seen a presentation in which themain categories are very different; the categories are atthe current stage relatively stable.

3 Some do not consider process choreography an ESB function. I

agree with this criticism in general, but as process management tools

are moving into the ESB product world this discussion become more

and more philosophical. My conclusion is that in a 'pure' ESB it is not

a mandatory component.4 While Process Choreography supports implementation of complex

business processes that require coordination of multiple business ser-

vices (usually using BPEL), Service Orchestration enables coordination

of multiple implementation services (most suitably exposed as an

aggregate service) to serve individual requests.

Insight | 11

Position 3. Changing or Adapting Infrastructure

Category Function

Invocation Support for synchronous and asyn-chronous transport protocols, servicemapping (location and binding)

Routing Addressability, static/deterministicrouting, content based routing,rules-based routing, policy based rou-ting

Mediation Adapters, Protocol Transformation,Service Mapping

Messaging Message Processing, MessageTransformation, MessageEnhancement

ProcessChoreography3

Implementation of Complex BusinessProcesses

ServiceOrchestration4

Coordination of multiple implemen-tation services exposed as a single,aggregate service

Complex EventProcessing

Event-interpretation, correlation, pat-tern-matching

Other Qualityof Service

Security (encryption and signing),reliable delivery, transaction mana-gement

Management Monitoring, audit, logging, metering,admin console, BAM

Other characteristics for ESBs are named in this defini-tion, such as the support of web services and XML. Thesecharacteristics are optional; they do not change the abs-tract concept of the behavior of an ESB.

An interesting set of opinions about ESB has been compiled in the blog of Miko Matsumura(http://www.infoq.com /articles/ESB-Roundup-Part1-Defining-ESB). It is a comparison of the opinions of DonRippert, CTO of Accenture, and Anne Thomas Manes ofBurton Group. While Don Rippert takes a pure tool drivenapproach to describe an ESB by defining it as a mean ofcommunication between WebServices, Anne is arguingthat the use of SOA principles is more important than theroll out of a bus, actually that the use of an ESB can be anobstacle in the implementation of service architecture.

After some time of reviewing the arguments I personallydecided to follow Anne's line of argumentation. It fol-lows my ideas of service architecture as an integrationmethodology more than Don Rippert's. But the main rea-son I mentioned this blog entry here is the followingquote of Anne in describing expected features of an ESB:

"...an ESB is especially good for bridging to legacy appli-cations, and therefore it is a useful component in a ser-vices infrastructure. Many ESBs also support reliablemessaging, asynchronous messaging, and pub/subexchange patterns. These capabilities can also be prettyuseful--but probably not in the initial stages of a SOA pro-ject. (Every organization has lots of projects to choosefrom that don't require these capabilities.) At a laterstage in a SOA project, you might also want an orchestra-tion engine, and most ESBs supply one--but that defini-tely isn't where an organization should start a SOA initia-tive. All of these capabilities are things that you don'tneed when first getting started. Therefore an ESB shouldbe a later-stage purchase." The last statement is contro-versial. It is of course possible to build the services in away that they connect directly (point-to-point), but a bet-ter approach is to use an existing integration platform,but this means that the advantages an ESB brings arenot available5.

Finally the definition of Gartner of an ESB (taken from apresentation given by Oracle):

"An ESB is an architecture that exploits Web services,messaging middleware, intelligent routing, and transfor-mation. It must support request/response communica-tion between loosely coupled SOA business componentsand one-way message delivery for sending notificationsto event-driven business components. It must also allowmore-complex message exchange patterns (MEPs)."

This definition is even less concrete in the definition ofparticular features of an ESB, allowing (almost every)middleware product to be classified as an ESB (or at leastto be called one).

Based on those criteria above it is possible to build a listwith the different criteria and to apply these criteria tothe products on the market. And, amazingly most com-mercial products in use right now will fulfill the criteriaof an ESB.

So - if you really have all the requirements of the ESBalready in your existing integration software - do youreally need to retire the entire existing infrastructure andrebuild everything from scratch?

Technically – no! I think that dismissing the existinginfrastructure is a tool to justify the change in the pro-cesses and the way the integration is done. It is mucheasier to define new working processes in a new environ-ment than to change a running software factory with allthe personal and process legacies.

5 Additional note to this part of the text: This discussion is about prio-

rities. It is, according to the statement, more important to roll a good

methodology out than to invest in a tool selection. If the discussion

would be made from a cost perspective and the issue is about a good

value initial solution the reader might want to look into this: The cur-

rently available open source ESB implementations have reached a

level of maturity. It is always worth looking into those for initial and

test installations. Some currently available open source solutions also

have professional support mechanisms in place which allow introdu-

cing them into the organization.

12 Insight |

This argument is completely different in situations wherethe replacement of the middleware is required anyway.Some integration, starting in the early years of EAI, arecoming to age - and are planned to be replaced anyway.This is a great opportunity to phase services and architec-ture reforms in without causing a revolution in the ITdepartment.

So - here the discussion (and my thought process) takesan interesting twist from technology to people andchange management. Why are we thinking of changingthe infrastructure - while the main change lies in thedevelopment and design process?

Conclusion

1. Replacing the existing middleware as part of the intro-duction of service oriented architecture is not necessarilyrequired. It is sometimes easier to introduce the newintegration concepts by buying new middleware - but weshould name the animal correctly: We introduce newmiddleware not because we need to replace the techno-logy, but because we cannot justify the change of metho-dology (and the internal work processes) without achange of the toolset. If this is the background for theproposal - we might need to define our business case dif-ferently before we invest in the technical change and donot achieve the results we expect, project and promise.

2. If we need to change the integration anyway, this is agreat opportunity to review the way we work right now.By just introducing new infrastructure (and keeping thedevelopment processes) we will not achieve all the posi-tive effects of the change. We might even end up with ascenario worse than before the change.

Insight | 13

When software development methodology evolved fromStructured Programming to Object Oriented Programmingwe suddenly realized that the models we used to describethe relationships between the different entities modeled inthe software became less useful. ERDs were built to displaythe relationship of the data, not of the objects and themethods. Now doing the step from development to integra-tion we were facing the same challenge: The amount ofinformation in the different applications was getting toocomplex to handle directly. By building point-to-pointconnections we had to calculate6 huge efforts in the main-tenance, and over time nobody really did understand theownership of even core data entities. Out of this situationthe concept of canonical data models was born.

The canonical data model introduced a very first abstraction layerand allowed the rationalization of interfaces and connections.

It was only when we installed the first CDMs we realized howmuch advantage this concept brings. Instead of just abstrac-ting the data types between the applications it is now possi-ble to model the business in an abstract layer inside the inte-gration world, providing a business centric view of the infor-mation. By managing this model we not only harvest thetechnical advantages, we also introduce an improved vehicleof communication with the involved parties in the company.

Keeping an overview...

If we look into the definition of service oriented architec-ture, one of the main differences to the previous definedintegration is the removal of redundant business functio-nality. Services with the same business function shouldnot be replicated in the service domain. This implies,that the services need to be marshaled, which is techni-cally a role of UDDI or a service repository. As the numberof services in the organization is limited and scope of aservice is exceeding the previous scope of an integrationfunction point, it is normally considered sufficient todocument the service as part of the application.

6 Mathematically, the maximal number of logical interfaces to be

maintained when using point-to-point connections can be calculated

as follows: As they are n systems and each system provides services

to each other, this would mean n(n-1) interfaces. By using a canoni-

cal model, the number of interfaces is n. As soon as more than 2 sys-

tems are connected the number of logical interfaces is lower by using

a canonical model.

Actually, a number of SOA drivers are arguing, that thestrength of a service architecture lies is in the limitationof services. This argument fails as the service architec-ture is reaching an enterprise level and the number ofbusiness processes is exceeding the level that can behandled by a small number of architects.

A good example of this is the generic architecture for thetelecommunications industry and the related frame-work, known as TMF Framework. Born as an objectmodel, it became a framework for application architec-ture. Those of you who have the pleasure to work withthe model know that it very much founded on best prac-tices in the industry. But you will also agree that thecomplexity of the complete model is exceeding the nor-mal level, as it describes most of the processes in theindustry. But for the down-to-earth implementationwork, it does deliver a very powerful tool - the SID, a stan-dard object model.

The SID is a great tool to build integrations. But adjustingthe details of the SID to the business model of the parti-cular telecommunication company, it delivers a nicemodel of classes and methods for the processes.The SID is a model completely built for the Telecomindustry (later I will show that it can be translated forother services based industries). If the reader is workingin an industry that does not deliver services to a custo-mer the adaptation might be too costly, but the readermight then be interested to build his own object model.

Conclusion

The introduction of integration and services opens thedoor to change the view on the information (and there-fore the business) differently. In many organizations,information modeling and integration is handled by dif-ferent people in different departments (business intelli-gence vs. integration development vs. application deve-lopment). This implies automatically that valuablesynergy is lost. In an optimal case, all groups related tothe information management in the organization are joi-ned to build a concise and complete information modelwhich then derives into the details relevant to thevarious users.

14 Insight |

Position 4. Information Management

In the previous section we talked about informationmanagement as a critical factor in the introduction ofservices. This section discusses the more technicalaspects of data models.

I am talking about Canonical and Integration DataModels since the early days of integration. The concept isquite simple, but I still see integrations which comple-tely ignore it. This section summarizes the differentapproaches and arguments for the use of Canonical DataModels and Integration Data Models and outlines diffe-rent ways to use them.

I am using two different terms here together - CanonicalData Model and Integration Data Model. The reason isthat the term Canonical Data Model is very often connec-ted with the Enterprise Data Model, which displays allthe different entities and their relationship in anEnterprise Architecture. Very often this canonical datamodel is complex so that architects tend to use a redu-ced version – an integration data model. This approachcan only be successful if the transformation from CDM tointegration data model is fixed and therefore the integra-tion data model can be verified against the canonicalmodel. In the following text I will use the termsCanonical Data Model as a synonym for an IntegrationData Model. This approach is shared in some parts of theintegration and pattern literature.

To explain the idea of a canonical data model it is neces-sary to look into the meaning of the word canonical. Itactually means "ruling", so we are talking about a datamodel which "rules" - and is therefore independent fromthe representations in connected applications. The idea ofthe Canonical Data Model is based on the approach thatthe data representation (and structure) in the integrationshould be on a more abstract level (and therefore be inde-pendent from the entity representation in the applicationsthat are connected with the integration. This way the inte-gration can be more flexible and adjustable.

How does a Canonical (Integration) Model work

A canonical model builds an abstraction layer betweenthe different applications. It is located in the integrationlayer of the environment. For the use of a canonicalmodel it is not important how the middleware is used,either as an EAI environment or a service bus.

The model is based on a generalized, i.e. applicationindependent, representation of the business entities. Asit is located between the applications, it requires thetransformation of the entities of the applications to themodel. Even this looks like additional work in the begin-ning of the application (in a point-to-point integrationthe amount of transformations is just the one betweenthe applications), as soon as the number of connectedsystems reaches a size of 3, the transformations to acanonical model is less work than direct interfaces.

Additionally the generation of an application abstractmodel reduces the changes in the overall integration ifthe data representation in one of the applicationchanges, as a limited amount of adapters (and not allapplication adapters connected to the changes applica-tion) need to be adjusted.

Why not use an integration model which trans-fers the data as one big string?

Nowadays it becomes much more popular to produce asingle canonical document for all integrations, consis-ting of a header record with audit and routing informa-tion and move the actual data into an XML documentthat is transferred to the destination without being tou-ched. This approach is not a canonical representation,but simply a hidden point-to-point connection. The mainadvantage, a reduction in transformations and an abs-tract representation of the entity in the integration layeris not at all achieved. The only difference to an oldfashioned point-to-point connection is the location of thetransformation - it is not in the adapter anymore, but isnow located in the application.

Stability

The introduction of a canonical model (especially if theintegration is using a BPM application for the coordina-tion and directing of the business process) will increasethe stability of all integrations. The main reason for thiseffect is the reduction of transformation and the use ofidentical business objects in the integration itself. In theintegration the entity is defined uniquely, independentfrom the source - so whether the customer entity comesfrom the web interface or the CRM system - for the inte-gration there is no difference as long as the entity fol-lows the common rules for the entity in the integration.

Insight | 15

Position 5. Canonical vs. Integration Data Model

Data Ownership and Canonical Models

Very often canonical models are developed as an integra-tion independent exercise as an enterprise data model.Those data models make a very good start for the integra-tor; nevertheless the generation of this kind of data modeldoes take a significant amount of time and effort (costs).If the EDM is not available - should the integration still usea canonical model? The answer is simple: yes!

There are different ways to build this canonical model. Iam going to talk about two of them which I found ratheruseful.

The first approach is using a standard object or datamodel, as e.g. the SID in the telecommunication industry.This object model is designed by the leading telecommu-nications industry body (the Tele Management Forum)and is designed as a high level, abstract object model - tobe extended for the particular use in an organization. Asmost vendors of applications in the telecommunicationsindustry are members in the forum, the conversation ofthe SID into an application integration model is signifi-cantly simplified. Related models exist for other indus-tries. In some cases industries are close enough to be ableto share a model or to have one industry assimilate amodel (e.g. Telecoms and Utilities).

The second approach uses the ownership of the entity.Best practice is that every business entity in integrationhas a unique owner. This unique owner is an applicationthat holds the core information of the business entity. Inany case, the owner of the entity is leading, i.e. allchanges to the entity, which are duplicated in connectedapplications, have to be replicated. This core structure ofthe entity can serve as a start to the canonical representa-tion of the entity. By enhancing the features (methods andattributes) of the entities it is possible to use this as a prag-matic start for a canonical integration model - even as astart for a potential wanted bigger model (e.g. for a moreenterprise wide approach).

Semi-Canonical Models (Canonical Applications)

Semi-Canonical Model is a new term, which has not beenused frequently. I use this term for a development whichoccurred in the last years. With the change of some orga-nizations to re-introduce a very strong and central applica-tion (like a leading general ledger or ERP system), a lot ofenvironments became highly application centric. Viewing

into the enterprise architectures from outside, it is possi-ble to name this central application "canonical", too.These applications cover a majority of the functionality inthe organization, and the connected applications arebecoming supplements for missing functionality.

With these architectures it might be a valid solution to usethe data model of the central application as the core of thecanonical model. This model, even if it fulfills most of therequirements, is not a pure canonical model. But from apragmatic point of view, the introduction of a data modelthat is independent from the core application might causemore issues than it provides any help.

Conclusion

The use of a canonical representation in the integrationlayer is essential to achieve all the flexibilities and costreductions that started the integration projects in thefirst place. Using integration without canonical model isjust moving point-to-point connections from the level ofphysical implementations into the integration software.A missing canonical model is likely to be the main reasonfor a technical failure of an integration project (as failuremeans a miss on cost reduction and stability targets).

16 Insight |

Some of the position sections here are driven by discussions Ihad with friends and colleagues. This one here is an exampleof such a start. It is based on the discussion of what makes aservice re-useable. In general, re-usability is one of the mainsales stories (and business benefits if done correctly) for ser-vice oriented architecture. But it is also a major cause whyimplementations do really fall short on expectations.

The selection of the interface technology does not make amaterial difference when it comes to the degree of re-usa-bility. Even if all services are built using the same techno-logy - it does not make the underlying functionality re-useable by itself. The secret of a re-useable service lies inthe design and the coverage of the business process.

In order to build services which can be used by more thanone application, they have to be defined on an abstractlevel, in a way that they make sense in the business pro-cess. Just to avoid discussions - I am not talking about tri-vial services here, such as: "get time", but about serviceswithin corporate business processes, e.g. for the requisi-tion process for customers. In order to provide re-useableservices the business must be able to define its businessservices in a way that similar steps in the process aremodeled identically.

The idea is to build a fixed number of basic services whichare optimized for performance and flexibility (if possible,sometimes we need to make a decision what is moreimportant - performance or flexibility). These services arethe basic building blocks for complex composite services.Good candidates for these re-useable basic services arethose that are based on data replication and modification,which are handling the synchronization of entities of thecanonical data model. Those services, commonly modeledfollowing the CRUDS (Create, Read, Update, Delete, andSearch) paradigm, are allowing the compilation of ser-vices into composite services using the ESB or other ser-vers. They can be seen as the building blocks of a morecomplex architecture.

A composite service is using the basic CRUDS service in acomplete transaction, there-by conserving the atomicnature and exposing the nature of a generic service.

I like to compare the system with the Danish buildingblocks for children. By providing the bricks for the differentfunctions I can nicely build a number of different functionsfor all the different consumers in the environment.

The second concept of re-useable service is more top-down.This approach is using re-useable business processes. As anexample, the process of purchasing an item in a shop is usingmany identical steps independent of the various sales channelsused. To define these identical transactions and design themfor re-use it is important that the designer knows all differentrelated business processes. The designer can then define theprocess steps in a way that re-use of the service is maximized.

If you look into the two approaches you will see the differentqualities of the re-usability. The first is a total technicalapproach, the business side comes in (and requires the exis-tence) of a well defined corporate data model. The second onedefines the re-usability on a business service level. The secondapproach does ask for a strong corporate data model - but thekind of implemented technical services are less restrictive.

What you did not find above is any mentioning about the"how" the service is implemented (whether a WebService isused, etc.). Yes, the service can be using JDBC adapters, specia-lized application connectors etc. If you follow my argumenta-tion it is actually irrelevant how the service is implemented,the technology used is more dictated by other requirements,like whether the service is exposed to a certain user group(externally), performance, security, or a certain service bus.

Conclusion

This section of the paper introduced an approach for re-useable services which is a bit different from the descrip-tions of the identification and definition of services I madeabove. This might be seen as a contradiction within thetext, but it is not, as this approach is a technical view onbuilding services which have been identified in the pro-cesses before. I found this approach very useful in situationswhere the application development team did not have thetime or the resources to build complex services and wasonly able to generate and expose the basic building blocksto the outside world. By providing those building blocks it ispossible to decouple some of the development work bet-ween application and integration developer. It has provento be a pragmatic approach to build services under tightdeadlines and with limited trained resources.

Insight | 17

Position 6. Building Re-Useable Components

Application Service Layer

Composite Services C1... C2

Base Services C - Create, R - Read, U - Update, D - Delete, S - SearchURC

c1

CD S

Service Location

Orchestration Layer

c1 c2 c3

This text discussed a number of different positions andpoint of views directly related to the topic of services andservice oriented architecture. They have developed as aresult of experiences made. I learned that it is not onlysufficient to make the client happy, but that I also needto discover the reasons why a project performed well.

Those positions are reflections on issues encountered – theywill help me to start discussions not from scratch but with asolid starting point. I used the word position on purpose; aposition is a point which brings us forward. Positions are rea-ched as part of a development made from experiences.

The most important lesson we at BearingPoint learned isthat being excellent technical architects is not enough tobe successful with complex projects. It is our ability tobridge the communication gap between business andtechnology. Service orientation needs this bridge; itneeds the cooperation between business oriented, infor-mation oriented, and technical oriented architects.

My colleagues and I would like to hear from you. Youread about our positions and experiences, we would liketo hear about yours. Contact us directly atBearingPoint… we look forward to talk to you.

How can BearingPoint help you?

BearingPoint is a leading provider for services and solu-tions for integration and service oriented architecture.Supporting clients Europe-wide, we bring a wide range ofexperiences which can help you to be successful in yourapproach. We can help you by:

• Helping you to review your current situation in yourenvironment.

• Supporting you in the introduction of an end-to-endservice development process, starting from the busi-ness side of your project to the final roll-out.

• Selecting the right integration environment for you.BearingPoint praises itself for being an impartial advi-sor on the market.

• With experienced design, development, and testingstaff BearingPoint is not only able to fill your needswith expertise, but also to help you to build your ser-vice expertise and development stream.

• BearingPoint has extensive knowledge in the modelingof enterprise data. The combination of business exper-tise and technical background helps you build the datamodel that reflects your business.

• BearingPoint has experienced architects with an ave-rage of 12+ years of business background. We can helpyou to work on the most challenging situations.

We welcome your thoughts on this white paper. For com-ments and questions, please contact Andreas Grimm (andreas.grimm@bearingpointconsulting.com, telephone +31 (20) 504 9000), the author of this white-paper. We will be pleased to make an appointment withyou to explore the potential benefits of service orientedarchitectures for your organization.

18 Insight |

General Conclusion

We are BearingPoint, a European management and technology consulting company known for our deep industry expe-rience, high customer satisfaction and highly skilled workforce. Being a Partner led organization, with 3.250employees in 14 countries, we partner with our clients to achieve sustainable results. We leverage the experience ofour consultants, which averages more than 12 years and provides us with a highly experienced consultant base. Weearn recognition for our client service from leading analysts and trade publications but are proudest of our high cus-tomer satisfaction scores.

Technology Consulting

In addition to Service Oriented Architecture, Data and Process Integration, our Technology Consulting practice sup-ports clients to achieve improved Return on IT Investments with:

• Business Alignment: working with clients to ensure that IT designs properly implement the business requirements,and can be unambiguously verified against these requirements, to ensure that the IT implementation meets thebusiness expectations;

• Project and Program management: partnering with clients to plan, manage and govern complex IT projects. We alsosupport clients with quality assurance of such projects, providing test management and test services to ensure thataccepted solutions meet the specified requirements and expectations;

• SAP advisory & implementation to achieve better results and enhanced benefits from your SAP implementation;

• Oracle CRM (Siebel) advisory & implementation to achieve better results and enhanced benefits from your OracleCRM implementation.

Our Clients

Our client references include leading Dutch and European companies in the Communications, Utilities and FinancialServices industries, as well as major Public Service organizations.

Insight | 19

About the authorAndreas Grimm is Technology Architect at BearingPoint and a leader in the integration practice in theNetherlands. With over 17 years of experience in the Telecommunication, Utilities, and Financial Industry(of which more than 10 years are in the integration area) his main focus is in the design and implemen-tation of integration projects. Mr. Grimm holds a German degree in Computer Science (Dipl.-Inform.) anda British B.Sc.(Hons).

About BearingPoint

www.bearingpointconsulting.comwww.bearingpoint.nl