Enterprise application integration

7
Abstract— Middleware is a computer software that connects software components or applications. The software consists of a set of services that allows multiple processes running on one or more machines to interact. This technology evolved to provide for interoperability in support of the move to coherent distributed architectures, which are used most often to support and simplify complex, distributed applications. Some of those includes SAP Netweaver Process Integration, Oracle Data Integrator Enterprise Solution, and Openlink Virtuoso that support application development and delivery. These middleware is especially integral to modern information technology based on XML, SOAP, Web services, and service- oriented architecture. I. INTRODUCTION Middleware offers some unique technological advantages for business and industry. For example, traditional database systems are usually deployed in closed environments where users access the system only via a restricted network or intranet. With the phenomenal growth of the World Wide Web, users can access virtually any database for which they have proper access rights from anywhere in the world. Middleware addresses the problem of varying levels of interoperability among different database structures. Middleware facilitates transparent access to legacy database management systems (DBMSs) or applications via a web server without regard to database-specific characteristics. Middleware can help software developers avoid having to write to application programming interfaces (API) for every control program, by serving as an independent programming interface for their applications. In short, middleware has become a critical element across a broad range of industries, for its ability to bring together resources across networks or platforms. II. SAP NETWEAVER PROCESS INTEGRATION SAP NetWeaver[1] Process Integration (SAP PI) is SAP's enterprise application integration (EAI) software, a component of the NetWeaver product group used to facilitate the exchange of information among a company's internal software and systems and those of external parties. Before the current version, SAP PI was called SAP Exchange Infrastructure (SAP XI)[2]. Like other NetWeaver[1] components, SAP PI is compatible with software products of other companies. Function and capability SAP Exchange Infrastructure (SAP XI) [2] enables you to implement cross- system processes. It enables you to connect systems from different vendors (non-SAP and SAP) in different versions and implemented in different programming languages (Java, ABAP, and so on) to each other. SAP Exchange Infrastructure MIDDLEWARE TECHNOLOGIES “SAP NETWEAVER PROCESS INTEGRATION, ORACLE DATA INTEGRATOR ENTERPRISE SOLUTION, OPENLINK VIRTUOSOYasril Sjaf, Wachid Yoga Afrida, and Robertus Kristianto Faculty of Computer Science, University of Indonesia Email: [email protected], [email protected], [email protected]

Transcript of Enterprise application integration

Page 1: Enterprise application integration

Abstract— Middleware is a computer software that connects software components or applications. The software consists of a set of services that allows multiple processes running on one or more machines to interact. This technology evolved to provide for interoperability in support of the move to coherent distributed architectures, which are used most often to support and simplify complex, distributed applications. Some of those includes SAP Netweaver Process Integration, Oracle Data Integrator Enterprise Solution, and Openlink Virtuoso that support application development and delivery. These middleware is especially integral to modern information technology based on XML, SOAP, Web services, and service-oriented architecture.

I. INTRODUCTION

Middleware offers some unique technological advantages for business and industry. For example, traditional database systems are usually deployed in closed environments where users access the system only via a restricted network or intranet. With the phenomenal growth of the World Wide Web, users can access virtually any database for which they have proper access rights from anywhere in the world. Middleware addresses the problem of varying levels of interoperability among different database structures. Middleware facilitates transparent access to legacy database management systems (DBMSs) or applications via a web server without regard to database-specific characteristics. Middleware can help software developers avoid having to write to application programming interfaces (API) for every control program, by serving as an independent programming interface for their applications. In short, middleware has become a critical element across a broad range of industries, for its ability to bring together resources across networks or platforms.

II.SAP NETWEAVER PROCESS INTEGRATION

SAP NetWeaver[1] Process Integration (SAP PI) is SAP's enterprise application integration (EAI) software, a component of the NetWeaver product group used to facilitate the exchange of information among a company's internal software and systems

and those of external parties. Before the current version, SAP PI was called SAP Exchange Infrastructure (SAP XI)[2]. Like other NetWeaver[1] components, SAP PI is compatible with software products of other companies.

Function and capability

SAP Exchange Infrastructure (SAP XI)[2] enables you to implement cross-system processes. It enables you to connect systems from different vendors (non-SAP and SAP) in different versions and implemented in different programming languages (Java, ABAP, and so on) to each other. SAP Exchange Infrastructure is based on an open architecture, uses open standards (in particular those from the XML and Java environments) and offers those services that are essential in a heterogeneous and complex system landscape:

Modeling and design of messages, transformations, and cross-component integration processes

Configuration options for managing collaborative processes and message flow

Runtime for message and process management

Adapter Engine for integrating heterogeneous system components

Central monitoring for monitoring message flow and processes

Supports internal company scenarios and cross-company scenarios.

SAP XI is based on general standards so as to enable external systems to be integrated. At the center of the infrastructure is an XML-based communication that uses HTTP (Hyper Text Transfer Protocol)[3]. The application-specific contents are transferred in messages in user-defined XML (eXtensible Markup Language)[4] schema from the sender to the receiver using the Integration Server.

Senders and receivers that exchange messages using the Integration Server are separated from one another. This separation makes it easier to connect systems that are technologically different. Every system that can exchange messages with the Integration Server can also exchange messages with all other systems that are connected to the Integration

MIDDLEWARE TECHNOLOGIES

“SAP NETWEAVER PROCESS INTEGRATION, ORACLE DATA INTEGRATOR ENTERPRISE SOLUTION, OPENLINK VIRTUOSO”

Yasril Sjaf, Wachid Yoga Afrida, and Robertus Kristianto Faculty of Computer Science, University of Indonesia

Email: [email protected], [email protected], [email protected]

Page 2: Enterprise application integration

Server. SAP XI supports the following methods of communication with the Integration Server:

●     Direct communication using proxies, which you generate in the application systems using a description in WSDL (Web Service Description Language)[5].

●     Communication using adapters. In this case, you create interfaces for message exchange in the application system, or use existing interfaces.

ArchitechtureThe following graphic is a simple overview showing how SAP XI is positioned within SAP Netweaver.The important components and concepts of SAP XI are shown on the right-hand side

Figure 1.SAP NetWeaver XI is part of the usage type Process Integration.[6]

Advantages and disadvantagesTechnical Overview SAP Exchange Infrastructure /Sap PI

OS Support : Windows, Linux, IBM, Solaris

Database Support: JDBC, DB2, Oracle, Microsoft SQl Server

Advantages Performance

XML usually has faster responsiveness than other integration format

FlexibilityXML is very flexible standard data format to exchange data among systems[4]

ReliabilitySAP’s product which has proved its quality for ERP software

Disadvantages Just on XML form and HTTP protocol Limited database support (only for

famous DB) Expensive in development cost

III. ORACLE DATA INTEGRATOR ENTERPRISE SOLUTION

Oracle Data Integrator Enterprise Edition is a comprehensive data integration platform that covers all data integration requirements—from high-volume, high-performance batches, to event-driven, trickle-feed integration processes, to SOA-enabled data services[7].

Function and capabilityE-LT Architecture for High PerformanceOracle Data Integrator Enterprise Edition’s Extract, Load, Transform (E-LT) architecture leverages disparate relational database management systems (RDBMS) engines to process and transform the data. This approach optimizes performance and scalability and lowers overall solution costs.[7]Real-Time Data Warehousing Oracle Data Integration Suite turns the promise of real-time data integration into reality by providing all key components required to enable real-time data warehousing and operational data hubs. Additional native hooks to Oracle SOA Suite provide comprehensive and hot-pluggable (working seamlessly with existing systems) data services integration[8].High-Productivity Integration Design Oracle Data Integrator Enterprise Edition shortens implementation times with its highly productive design-time tools. Declarative design simplifies common data integration design and deployment use cases, shortening implementation times. Data integration designers describe source and target data formats and data integration processes. The business user or the developer can focus on describing what to do, not how to do it. ODI EE then generates, deploys and manages the code required to implement those processes across the various source and target systems.Knowledge Modules Provide Flexibility and Extensibility Knowledge Modules are at the core of the Oracle Data Integrator Enterprise Edition architecture. They make all Oracle Data Integrator Enterprise Edition processes modular, flexible, and extensible.Hot Pluggable Oracle Data Integrator Enterprise Edition provides heterogeneous support for 3rd party platforms, data-sources, and data warehousing appliances. While Oracle Data Integrator Enterprise Edition leverages optimizations for Oracle Database to perform EL-T data movement, transformation, data quality and standardization operations, Oracle Data Integrator Enterprise Edition is compatible with mixed targets, sources and applications [7].

ArchitectureThe ODI-EE architecture is organized around a modular repository, which is accessed in client-server mode by components graphical modules and

Page 3: Enterprise application integration

execution agents that are written entirely in Java. The architecture also includes a Web application, Metadata Navigator, which enables users to access information through a Web interface.Graphical modulesThe four graphical modules are Designer, Operator, Topology Manager, and Security Manager. These modules can be installed on any graphical platform that supports Java Virtual Machine 1.5 (J2SE), including Windows, Linux, HP-UX, Solaris, AIX, and Mac OS, among others.[9]The graphical modules are as follows:

Designer defines declarative rules for data transformation and data integrity.

Operator manages and monitors production. It is designed for production operators and shows execution logs with error counts, the number of rows processed, execution statistics, the actual code that is executed, and so on.

Topology Manager defines the physical and logical architecture of the infrastructure.

Security Manager manages user profiles and their access privileges. Security Manager can also assign access privileges to objects and features.

Figure 2. Graphical modules connect the repository [9]

Runtime Components At runtime, the Scheduler Agent

coordinates the execution of the scenarios. The Scheduler Agent can be installed on any platform that supports a Java Virtual Machine (J2SE), including Windows, Linux, HP-UX, Solaris, and IBM AIX, iSeries/AS400, zSeries/OS/390. Execution can be launched from one of the graphical modules, or the built-in scheduler or a thirdparty scheduler can trigger it [9].

With the Extract-Load Transform (E-LT) architecture, the Scheduler Agent rarely performs any transformation [10]. It simply retrieves code from the execution repository and then requests database servers, operating systems, or scripting engines to execute that code. When the execution is completed, the Scheduler Agent updates the execution logs in the repository and then reports error messages and execution

statistics. Users can view the execution logs from the Operator module or the Metadata Navigator Web interface.

RepositoriesThe repository consists of a master repository and several work repositories. These repositories are databases stored in relational database management systems. All objects that the modules configure, develop, or use are stored in one of these repositories, and are accessed in client-server mode by the various components of the architecture.

Figure 3. Master repository and work repository [9]

Metadata NavigatorMetadata Navigator is a Java 2 Enterprise Edition (J2EE) application that provides Web access to repositories. It allows users to browse objects, including projects, models, and execution logs. Metadata Navigator can be installed on an application server such as Oracle Container for Java (OC4J) or Apache Tomcat [11]. Business users, developers, operators, and administrators can access Metadata Navigator via a Web browser. Through its comprehensive Web interface, users can see flow maps, trace the source of all data, and even drill down to the field level to understand the transformations used to build the data. They can also launch and monitor scenarios from a Web browser through Metadata Navigator [9].

Advantages and disadvantages

Advantages Performance

Heterogeneous E-LT architecture for high performance and faster responsiveness [10]

FlexibilityEvent-driven platform supports all data integration styles, supports various pre-built business rules and application knowledge modules

ProductivityDeclarative design improves developer productivity and lowers total cost of ownership

Open100% Java and SOA Native, standards based.

Page 4: Enterprise application integration

Hot-PluggableHeterogeneous support for 3rd party platforms and data-sources

Disadvantages focus on ETL solutions, rather than in an

open context of data management tools are used mostly for batch-oriented

work, transformation rather than real-time processes or federation data delivery

long-awaited bond ODI brought only promises - customers confused in the functionality area and the future is uncertain

IV. OPENLINK VIRTUOSO

Functionality and capacityOpenlink Virtuoso, an open source version of virtuoso universal server, is a middleware and database engine hybrid that combines the functionality of a traditional RDBMS, ORDBMS, virtual database, RDF, XML, free-text, web application server and file server functionality in a single system. Rather than have dedicated servers for each of the aforementioned functionality realms, Virtuoso is a "universal server"; it enables a single multithreaded server process that implements multiple protocols [12].Virtuoso provides an extended object-relational model, which combines the flexibility of relational access with inheritance, run time data typing, late binding, and identity based access. Virtuoso Universal Server database includes physical file and in memory storage and operating system processes that interact with the storage. There is one main process, which has listeners on a specified port for HTTP, SOAP, and other protocols.Virtuoso provides dynamic locking, starting with row level locks and escalating to page level locks when a cursor holds a large percentage of a page's rows or when it has a history of locking entire pages. Lock escalation only happens when no other transactions hold locks on the same page, hence it never deadlocks. Virtuoso SQL provides means for exclusive read and for setting transaction isolation.All four levels of isolation are supported: Dirty read, read committed, repeatable read and serializable. The level of isolation may be specified operation by operation within a single transaction. Virtuoso can also act as a resource manager and/or transaction coordinator under Microsoft's Distributed Transaction Coordinator ( MS DTC ) or the XA standard [13].

ArchitectureVirtuoso is designed to take advantage of operating system threading support and multiple CPUs. It consists of a single process with an adjustable pool of threads shared between clients. Multiple threads may

work on a single index tree with minimal interference with each other. One cache of database pages is shared among all threads and old dirty pages are written back to disk as a background process.The database has at all times a clean checkpoint state and a delta of committed or uncommitted changes to this check pointed state. This makes it possible to do a clean backup of the checkpoint state while transactions proceed on the commit state.A transaction log file records all transactions since the last checkpoint. Transaction log files may be preserved and archived for an indefinite time, providing a full, recoverable history of the database.A single set of files is used for storing all tables. A separate set of files is used for all temporary data. The maximum size of a file set is 32 terabytes, for 4G x 8K pages.[14]

Advantages and DisadvantagesAdvantages

It is possible to do a clean backup of the checkpoint state while transactions proceed on the commit state. The database has at all times a clean checkpoint state and a delta of committed or uncommitted changes to this checkpointed state.

A transaction log file records all transactions since the last checkpoint. Transaction log files may be preserved and archived for an indefinite time, providing a full, recoverable history of the database.

Disadvantages Virtuoso’s packaging size is very big

Virtuoso is too difficult to be learned

V. CONCLUSION

SAP PI mediates between entities with varying requirements in terms of connectivity, format, and protocols. According to SAP, PI providing a common repository for interfaces. The central component of SAP PI is the SAP Integration Server, which facilitates interaction between diverse operating systems and applications across internal and external networked computer systems. Oracle Data Integrator Enterprise Edition (ODI-EE) is a lightweight, legacy-free, state-of-the-art integration platform. All components can run independently on any Java-compliant system. Because of this legacy-free architecture, ODI-EE installs within minutes on any platform. The challenges posed by the Semantic Web vision compliment the essence of Virtuoso's prowess and OpenLink's engineering skill set. After all, an RDF Triple store is fundamentally a specialized

Page 5: Enterprise application integration

database challenge that draws on deep database engineering and query optimization

REFERENCES

[1] http://en.wikipedia.org/wiki/NetWeaver[2] http://en.wikipedia.org/wiki/

SAP_NetWeaver_Process_Integration[3]  Hypertext Transfer Protocol -- HTTP/1.1, R.

Fielding, J. Getty, J. Mogul, H. Frystyk, L. Masinter, P. Leach, T. Berners-Lee (June 1999)

[4] http://www.w3.org/XML/[5] http://www.w3.org/TR/wsdl[6] http://help.sap.com/saphelp_nw70ehp1/helpdata/en/

0f/80243b4a66ae0ce10000000a11402f/content.htm[7] Oracle Coorporation (2009, February) Oracle Data

Integrator Entreprise Edition, Redwood Shores : Oracle data sheet

[8] http://www.oracle.com/us/technologies/soa/index.htm[9] Oracle Coorporation (2009, February)The Oracle

Data Integrator Entreprise Edition Architectur, Redwood Shores : Oracle white paper

[10] http://it.toolbox.com/blogs/abinitio-underground/etl-versus-elt-26874

[11] http://en.wikipedia.org/wiki/Oracle_Application_Server

[12] http://en.wikipedia.org/wiki/Virtuoso_Universal_Server

[13] http://en.wikipedia.org/wiki/Distributed_Transaction_Coordinator

[14] http://virtuoso.openlinksw.com/blog/