HOL PWX Architecture

1

Dan SmithPrincipal Subject Matter Expert

May 2012

Informatica PowerExchange Architecture Overview

2

Informatica PowerExchange

Batch Architecture Overview

3

Operating Environment

PowerExchange

User Applications

Data Maps for Non-Relational Access

Informatica PowerExchangeBatch Architecture

DataRecordsSQL

Listener

Remote DataPackagedApplications

Mainframe andMid-Range

Relational andFlat Files

Standards andMessaging

Data Integration Platform PowerCenter

4

Informatica PowerExchangeHow to Create Metadata…whatever your source

• A Datamap defines metadata• For non-relational databases, need to define metadata• Contains all information regarding the underlying dataset

• Created from existing information• VSAM – copybooks• IMS – DBDs, then overlay segment copybooks• ADABAS – FDT• IDMS - schemas

• Stored on the same platform as the data

• Not needed for relational sources

• Provides the information to convert an SQL statement to actual data base requests

5


PowerExchange

User Applications


Informatica PowerExchangeDesign Time – Metadata Driven Approach

Listener

Remote DataPackagedApplications

Mainframe andMid-Range

Relational andFlat Files

Standards andMessaging

Data Integration Platform (Create Datamap) PowerExchange

PowerExchangeNavigator

MetadataRecords

MetadataRequest

Hand KeyDatabaseDefinitions

SchemasAdabasDefinitions

Copybooks

6

Informatica PowerExchange“Row Test” the Data Map – Preview the data

`

7

Informatica PowerExchange

Change Data Capture Architecture Overview

8


PowerExchangeData Maps for

Non-Relational Access

Informatica PowerExchange CDCDefining the Changes to be Extracted

Listener

Data Integration Platform (Create Datamap) PowerExchange

PowerExchangeNavigator

DataRecordsSQL

ExtractionDefinitions

CaptureRegistrations

Extract on definitions are based on capture registrations

9

PowerExchange

Informatica PowerExchange CDCCapture-Once Deliver-Many

DataRecordsSQL

Listener


Mainframe Mid-RangeADABASDATACOMDB2IDMSIMSVSAM

DB2/400OracleSQL ServerDB2 LUW

PowerExchange Listener

DataRecordsSQL

Physical VSAM Linear Datasets Database vendor - journal, archive, redo logs

Collector Collector

Published Interfaces

Change Stream/Logger


10

Informatica PowerExchange CDCArchitecture for DB2 z/OS CDC

z/OS Environment

PowerExchange

DB2 SubSystem

Data Maps for special use cases

DataRecordsSQL

Listener


Collector


AgentDB2 z/OS DB2

Logs

IFI 306(z/OS)

1. PowerExchangecollects changes

2. Changed data persisted in change stream

3. PowerCenter issues standard SQL request

4. PowerExchange sends changed data records to PowerCenter

11

Informatica PowerExchange CDCArchitecture for DB2 i5/OS CDC

I5/OS Environment

PowerExchange

DB2 SubSystem


DataRecordsSQL

Listener


Journal Reader

DB2 i5/OS Journals

Journaling(i5/OS)

12

Informatica PowerExchange CDCArchitecture for DB2 LUW CDC

Linux, UNIX, or Windows Environment

PowerExchange

DB2 Instance


DataRecordsSQL

Listener


Collector

VirtualChange StreamDB2 LUW Archive

LogsLogReadAPI

13

Informatica PowerExchange CDCArchitecture for SQL Server CDC

Windows Environment

PowerExchange

SQL Server


DataRecordsSQL

Listener


Collector

VirtualChange StreamSQL Server

DatabaseSQL ServerDistribution DB

SQL query

14

Informatica PowerExchange CDCArchitecture for PWX Oracle LogMiner CDC

Linux, UNIX, Windows, or z/Linux Environment

PowerExchange

Oracle Instance


DataRecordsSQL

Listener


Collector/PWXCCL

Change Stream

Oracle OracleLogs

LogMiner1. Oracle collects and persists changes in Redo Logs

2. PowerExchange issues LogMinerrequest




15

Informatica PowerExchange CDCArchitecture for PWX Express Oracle CDC

Linux, UNIX, Windows, or z/Linux Environment

PowerExchange

Oracle Instance


DataRecordsSQL

Listener


Collector/PWXCCL

Change Stream

Oracle OracleLogs

Express Log Reader

1. Oracle collects and persists changes in Redo Logs

2. PWX Express Log Reader captures changes




16

Informatica PowerExchange CDCArchitecture for Mainframe VSAM CDC (Batch or CICS)

Mainframe Environment

PowerExchange

Batch or CICS


DataRecordsSQL

Listener


Collector

Change StreamVSAM

1.PowerExchange collects changes




JRNAD Exit (Batch)File/User Exits (CICS)

Agent

17

Informatica PowerExchange CDCArchitecture for Mainframe IMS Synchronous CDC


PowerExchange

Batch or Online IMS


DataRecordsSQL

Listener


PWXLibraries

Change Stream

AgentDLI Job orDBCTL

COLLECTOR OPERATES IN Address Space of Batch DLI JobOr IMS Control Region

18

Informatica PowerExchange CDCArchitecture for Mainframe IMS Log-Based CDC


PowerExchange

Batch or Online IMS


DataRecordsSQL

Listener


Collector

Change Stream Agent

DLI Job orDBCTL

ArchivedLogs

Recons

19

Informatica PowerExchange CDCArchitecture for Mainframe ADABAS CDC


PowerExchange

ADABAS


DataRecordsSQL

Listener


Collector

Change Stream Agent

ADABASNucleus

ActivePLOGs

ADABASUser Exit 2

ADABASPLOGs

ActivePLOGs

20

Informatica PowerExchange CDCArchitecture for Mainframe Datacom Synchronous CDC


PowerExchange

Datacom

DataRecordsSQL

Listener


ChangeController

Change Stream

AgentDatacomMUF

ECSA

ChangeCollectorSubtask Data Maps for

Non-Relational Access

21

Informatica PowerExchange CDCArchitecture for Mainframe Datacom Table-Based CDC


PowerExchange

Datacom


DataRecordsSQL

Listener


Collector

Change Stream Agent

TSN Table

DatacomMUF

MNT Table

22

Informatica PowerExchange CDCArchitecture for Mainframe IDMS Log-Based CDC


PowerExchange

Batch or Online IDMS


DataRecordsSQL

Listener


Collector

Change Stream Agent

IDMSCV

IDMSJournalSwitch

Journal Write Process

IDMS Journal

23

Informatica PowerExchangeSupported Sources & Targets

Informatica Batch CDC

IMS R/W Y

VSAM R/W Y

DB2 z/OS R/W Y

ADABAS R/W Y

IDMS R Y

Datacom/DB R Y

DB2 i5/OS R/W Y

Oracle R/W Y

SQL Server R/W Y

DB2 LUW R/W Y

HOL PWX Architecture

Documents

Transcript of HOL PWX Architecture