8.x Architecture
-
Upload
saumyabrata-das -
Category
Documents
-
view
113 -
download
0
Transcript of 8.x Architecture
AgendaAgenda
ETL BasicsPowerCenter 7x ArchitecturePowerCenter 7x Components and ConnectivityPowerCenter 8x ArchitecturePowerCenter 8x Components and ConnectivityOverview of
o Service Oriented Architectureo Domains and Nodeso Services:
• Service Manager • Domain Operations• Application Services
Informatica 7x Server Grid Concept Vs Informatica 8x Server Grid Concept
ETL BasicsETL Basics
Extract, Transform, and LoadExtract, Transform, and Load
RDBMSRDBMS MainframeMainframe OtherOther
Operational Systems
DataDataWarehouseWarehouse
Decision Support
• Transaction level data• Optimized for Transaction
Response Time• Current• Normalized or De-
Normalized data
• Aggregated data
• Historical
ETLETLExtract Load
Transform
Cleanse DataApply Business Rules
Aggregate DataConsolidate Data
De-normalize
HR3-LMS DWHHR3-LMS DWH
OHRDB
PlateauFiles
ETL ProcessBy
InformaticaHR3 DWH HR3 LMS
Universe
ETL ToolsETL Tools
Some of the ETL Tools which we have currently in Market.
1.Informatica
2.Ab Initio
3.Data Transformation Services
4.IBM Web Sphere Datastage
5.Oracle Warehouse Builder
Etc…
ArchitectureArchitecture• PowerCenter 7x ArchitecturePowerCenter 7x Architecture• PowerCenter 8.5 ArchitecturePowerCenter 8.5 Architecture
PowerCenter 7x Architecture PowerCenter 7x Architecture
Informatica Architecture & Connectivity – 6.x and 7.xInformatica Architecture & Connectivity – 6.x and 7.x
TargetsRDBMS
native native
TCP/IP
HeterogeneousTargets
Repository
RepositoryServer
RepositoryAgent
TCP/IP
native
Server
HeterogeneousSources
Repository Designer Workflow Workflow Rep Server Manager Manager Monitor Administration Console
ODBC ODBC
Informatica Architecture & Connectivity – 6.x and 7.xInformatica Architecture & Connectivity – 6.x and 7.x
• The management of the repository is done through the ‘Repository Server’, since it is the only process which interacts with the Repository. The Repository Server can be managed and administered through the ‘Repository Server Administrator Console’ interface.
• One Repository Server can manage multiple repositories. For each repository there is one Repository Agent.
• Repository Agent is a multi-threaded process which inserts, updates and retrieves metadata from Informatica Repository
• All the client tools and Informatica Server communicate with the Repository through Repository Server
• Repository Server Administration Console is a client tool used to create/maintain repositories and configure Repository Servers. All the tasks like starting a repository, backup/restore and upgrade are performed using this tool
PowerCenter 7x ComponentsPowerCenter 7x Components
Informatica Power Center Components:
PowerCenter Repository PowerCenter Repository Server PowerCenter Client
» Repository Manager» Repository Server
Administration Console» Designer» Workflow Manager» Workflow Monitor
PowerCenter Server
PowerCenter 7x ArchitecturePowerCenter 7x Architecture
PowerCenter 7x Architecture (cont)PowerCenter 7x Architecture (cont)
•Sources •Targets •Repository •Repository Server •PowerCenter Client •PowerCenter Server
•Load Manager Process •Data Transformation Manager
(DTM) Process •Code Pages and Data
Movement Modes •PowerCenter Metadata
Reporter
Load Manager ProcessLoad Manager Process
•Load Manager Process
Manages workflow scheduling.
Locks and reads the workflow.
Reads the parameter file.
Creates the workflow log file.
Runs workflow tasks and evaluates the conditional links connecting tasks.
Starts the DTM, which runs the session.
Writes historical run information to the repository.
Sends post-session email in the event of DTM failure.
DTM ProcessDTM Process
•Data Transformation Manager (DTM) ProcessReads session information from
the repository. Expands the server, session,
and mapping variables and parameters.
Creates the session log file. Validates source and target
code pages. Verifies connection object
permissions.
Runs pre-session shell commands, stored procedures and SQL.
Creates and runs mapping, reader, writer, and transformation threads to extract, transform, and load data.
Runs post-session stored procedures, SQL, and shell commands.
Sends post-session email.
PowerCenter Server
Repository Serverdev001
PowerCenter Server
dev002
dev003
dev004
PowerCenter Server
Repository
Current EnvironmentCurrent Environment
PowerCenter 8.5 Architecture PowerCenter 8.5 Architecture
PowerCenter 8.x ComponentsPowerCenter 8.x Components
Standards, Messaging,
Web Services
Packaged Applications
Relational/Flat Files
Mainframe/ Midrange
Standards, Messaging, Web Services
Packaged Applications
Relational/Flat Files
Mainframe/ Midrange
Sources Targets
Integration Service(s)
PowerCenter Connects PowerExchange
Core Services
Application Services
Client Tools Designer WF Manager Monitor
Rep. Manager Administration Console
SAP BW Service
Web Services Hub
Repository Service(s)
Configuration Service
Authentication Service
Domain Service
Log Service
Gateway Service
Administration Service
Repository Database
Standards, Messaging,
Web Services Standards,
Messaging, Web Services
Packaged Applications
Relational/Flat Files Relational/Flat Files
Mainframe/ Midrange Mainframe/ Midrange
Standards, Messaging, Web Services
Packaged Applications
Relational/Flat Files
Mainframe/ Midrange
Sources Targets
Integration Service(s)
PowerCenter Connects PowerExchange
Core Services
Application Services
Client Tools Designer Designer WF Manager WF Manager Monitor Monitor
Rep. Manager Rep. Manager Administration Console Administration Console
SAP BW Service
Web Services Hub
Repository Service(s)
Configuration Service
Authentication Service
Domain Service
Configuration Service Configuration Service
Authentication Service Authentication Service
Domain Service Domain Service
Log Service
Gateway Service
Administration Service
Log Service Log Service
Gateway Service Gateway Service
Administration Service Administration Service
Repository Database
Repository Database
OverviewOverview
• Service Oriented Architecture
• Domains, Nodes, and Services
• Services: Service Manager Domain Operations Application Services
Service Oriented Architecture (SOA)Service Oriented Architecture (SOA)
SOA: An application architecture in which all functions, or services, invoke software interfaces that perform business processes.
Service: A task performed by a service provider to achieve desired end results for a service consumer. Both provider and consumer are roles played by software agents on behalf of their owners.
Service Consumer
Service Provider
ResponseRequest
SOA (cont.)SOA (cont.)
•For example, in PowerCenter 8.5:PowerCenter Server has become a service, the
Integration ServiceNo more Repository Server, but PowerCenter includes a
Repository Service
•Client applications are the same, but work on top of the new services framework
Domains, Nodes and ServicesDomains, Nodes and Services
Domains, Nodes, and ServicesDomains, Nodes, and Services
Application Services
Integration Service
Repository Service
SAP BW Service
Web Services Hub
Core Services
Service Manager
Node 1 (machine)
Application Services
Node 2 (machine)
Domain
Application Services
Node 3 (machine)
Integration Service
Repository Service
SAP BW Service
Web Services Hub
Core Services
Service Manager
Integration Service
Repository Service
SAP BW Service
Web Services Hub
Core Services
Service Manager
PowerCenter DomainPowerCenter Domain
•Domain can be a single PowerCenter installatione.g. single Integration Service, Repository Service
•Or it can be multiple PowerCenter installations
PowerCenter Domain (cont.)PowerCenter Domain (cont.)
MarketingHuman Resources
Sales
DomainsDomains
• A collection of nodes and services
• Primary Unit of administration
• Domains and components are managed in the Administration Console
• Centralizes configuration and administration
Configuring DomainsConfiguring Domains
• Domains and components are managed in the Administration Console
• Centralizes configuration and administration
NodesNodes
• The logical representation of a machine in a domain •Each node runs a Service Manager
Lightweight service
•Types of Nodes: Gateway Node Worker Node
Nodes (cont)Nodes (cont)
• Gateway Nodes Entry Point to the domain from the PowerCenter Client Routes service requests from PowerCenter Client to available
nodes Only one node in domain serves as a gateway for domain at a
given point of time Master Gateway Backup Gateway
If gateway node is unavailable, domain cannot accept service requests
ServicesServices
•All PowerCenter processes run as services on a node•All services installed by default.
Define which services run by configuring the node and services.
•Two types:ApplicationCore
Core ServicesCore Services
•Built in to the domain•Support the domain and application services•Service Manager manages services
Core ServicesCore Services
•Gateway Service•Log Service•Configuration Service•Domain Service•Service Process Controller •Licensing Service•Authentication Service•Admin Service
Licensing ServiceLicensing Service
•Manages PowerCenter licensesMaster Gateway Node
•Assigns licenses and validates application services when they start
•Licenses keys areEncrypted in a binary fileOriginal keysIncremental keys (upgrade)
License TasksLicense Tasks
•Create and remove•Assign and unassign•Update to add license keys•View license details•Configure user permissions on license
Log ServiceLog Service
•Collects all PowerCenter log events•Processes log events for core and application services,
workflows, and sessions•Runs on gateway node by default•When Log Service becomes unavailable:
Core and application services write log events to temporary files•When Log Service becomes available again:
Service Manager processes the temporary files and sends them to the Log Service
Application ServicesApplication Services
•Represent server-based functionality in PowerCenter•Following types:
Integration Service Repository Service Reporting Service Metadata Manager Service SAP BW Service Web Services Hub
•You designate a node on which to run the service process
Service ProcessesService Processes
•The runtime instance of a service running on a node•Service processes do all the work
36
Service Manager
• Controls PowerCenter services running on the node
• Primary point of control for PowerCenter services
• Service on Windows, daemon on UNIX
• If Service Manager is not running, node is unavailable
37
Services (cont.)
Application Services
Integration Service
Repository Service
SAP BW Service
Web Services Hub
Core Services
Service Manager
Node 1
Application Services
Integration Service
Repository Service
SAP BW Service
Web Services Hub
Core Services
Service Manager
Node 2
Service installed, but not configured to run
Service installed and configured to run
Domain
Domain Operations: Services (cont)Domain Operations: Services (cont)
Node 1 (Master Gateway)
Service ManagerService Manager
User Management
Logging
Licensing
Node Configuration
Domain Configuration
Authorization
Authentication
AlertsDomain Operations Node 2 (Backup Gateway)
Service ManagerService Manager
Logging
Node Configuration
Domain Operations
Node 3 (Worker Node)
Service ManagerService Manager
Logging
Node Configuration
Domain Operations
DomainDomain
Application Services: Services (cont)Application Services: Services (cont)
Node 1
Service ManagerService Manager
Web Services Hub
SAP BW Service
Metadata Manager Service
Reporting Service
Repository Service
Integration ServiceApplication Services
DomainDomainNode 2
Service ManagerService Manager
Web Services Hub
SAP BW Service
Metadata Manager Service
Reporting Service
Repository Service
Integration ServiceApplication Services
Same Clients, New FrameworkSame Clients, New Framework
Node 1 (Worker Node)
Repository Repository Service 1Service 1
Integration Integration Service 1Service 1
Integration Integration Service 2Service 2
Node 2 (Gateway Node)
RepositoryRepository
PowerCenter Client
TargetsSources
Administration Administration ConsoleConsole
Domain
Same Clients, New Framework (cont)Same Clients, New Framework (cont)
Node 1 (Worker Node)
Repository Repository Service 1Service 1
Integration Integration Service 1Service 1
Integration Integration Service 2Service 2
Node 2 (Gateway Node)
RepositoryRepository
PowerCenter Client
Sources/Targets
Administration Administration ConsoleConsole
Domain
Repository ServiceRepository Service
• In 7x, the repository server can connect to multiple repositories starting a separate repository agent, one for each repository.
• In 8.5 one repository service can connect to only one repository. But one node can host more than one repository services.
Node 1
Repository Repository Service 1Service 1
Repository Repository Service 2Service 2
Repository Server
Repository 1Repository 1
Repository 2Repository 2
Repository 1Repository 1
Repository 2Repository 2Repository Repository
Agent 2Agent 2
Repository Repository Agent 1Agent 1
8.58.5
7x7x
Integration ServiceIntegration Service
•The Integration Service starts one or more Integration Service processes.
• Integration Service Process(8.5) = Load Manager(7x)
Source/Targets
Node 1 (Worker Node)
Integration Service 1Integration Service 1
Service Service ManagerManager
Node 2 (Gateway Node)
RepositoryRepositoryPowerCenter
Client
Domain
Repository Service 1Repository Service 1
11
667755
443322
88
Server GridServer Grid• 7x Server Grid Concept7x Server Grid Concept• 8.5 Server Grid Concept8.5 Server Grid Concept
dev003
dev001
dev002
dev004
Master Server
Worker Server
Worker Server
Worker Server
7x Grid Architecture7x Grid Architecture
PowerCenter 8.5 Server GridPowerCenter 8.5 Server Grid
• Steps:Assign Resources to the nodesCreate the grid using the nodesConfigure the Integration
Service to run on the nodesRun the Workflow on the
Integration Service
• System Resources: CPU usage DTM buffer memory Cache memory Connection Information
Node 1Node 1
Node 2Node 2
Node 3Node 3
Node 4Node 4
ApplicatioApplication n
DatabaseDatabaseIntegration Integration
ServiceService
PowerCenter 8.5 Server Grid (cont)PowerCenter 8.5 Server Grid (cont)
• Shared Location:Session LogsWorkflow LogsCache FilesBad FilesSource Files (Flat Files, etc)Target Files (Flat Files, etc)
Node 1
Repository Repository Service 1Service 1
IS Service IS Service process 1process 1
Node 3
IS Service IS Service process 3process 3
Node 2
IS Service IS Service process 2process 2
RepositoryRepository
Application Application DatabaseDatabase
Shared Shared LocationLocation
ISIS
Server Grid ComparisonServer Grid Comparison
PowerCenter 7xPowerCenter 7x PowerCenter 8.5PowerCenter 8.5Essentially a server grid. A number of Informatica servers were added to form one Server grid.
Nodes are grouped to form a grid.
Only homogenous Informatica Servers can be a part of the grid. For forming a grid the requirements for the participating servers are :Same OSSame no. of CPU’sSame amount of memory spaceSame directory structureAny server not fulfilling the above prerequisites cannot be a part of the grid
Heterogeneous nodes can be grouped to form a grid. There is no restriction on the OS, no. of CPU’s or memory space for the participating nodes. The only restriction is that all the nodes have a common shared directory structure.
Server Grid Comparison (cont)Server Grid Comparison (cont)
PowerCenter 7xPowerCenter 7x PowerCenter 8.5PowerCenter 8.5One server in the grid acts as the Master Server to whom all the requests are be routed while other servers are be worker servers.The master server runs all the non-session tasks assigned to the grid.
Grid is a service just like the Integration Service. The Load Balancer is the component of the Integration Service that dispatches the different tasks to the nodes or the different threads to the DTM processes running on the nodes in the grid. The Load Balancer distributes tasks or threads based on node and resource availability
Essentially acts as workflow on grid. Individual sessions are assigned to an Informatica Server.
Acts as session on grid whereby the different threads of a session like the reader, writer and transformation threads can be assigned to different nodes.
SOA AdvantagesSOA Advantages
Replaced several servers that were used within the PowerCenter 7.x installer
Administration Tool is Web Based
SOA supports several integration service processes to run concurrently
Enables High Availability Option
Thank YouThank You