Release Control with PowerCenter Release Control with PowerCenter at UBS / Switzerlandat UBS / Switzerland

Friedrich LehnFriedrich LehnConfiguration & Release ManagerConfiguration & Release ManagerOctober 2000October 2000

AgendaAgenda

Project Overview PowerCenter Configuration Configuration Management Deployment Process Release Process Summary

Project OverviewProject Overview

Global, integrated investment services firm offering- the full range of wealth management, asset management, investment banking and securities services- to the full spectrum of clients

World’s largest private bank

Top ten financial services firm by market capitalization- $62 billion dollars

Total client assets of US$1 trillion

UBSUBS

Project OverviewProject Overview

System EnvironmentsSystem Environments

DevelopmentDevelopment ProductionProductionTestTest

IBM AIX SP2 cluster DB2 UDB EEE PowerCenter 1.6

Project OverviewProject Overview

Load ProcessLoad ProcessFile

Preprocessing

Conditioning

AutoLoader

PowerCenter

SQL Apply

AutoLoader

System of Records

Common Data Warehouse

Project OverviewProject Overview

Main organizational element

Three letter acronym as base for naming standards

Each delivery stream gets the same infrastructure:UNIX directories, meta data area, PowerCenter folders

Each delivery stream has business responsible, data modeler, database administrator, delivery stream manager

Team size typically between 1 and 5

Delivery streams release independently from each other

Delivery StreamDelivery Stream

Project OverviewProject Overview

Application StructureApplication Structure

Delivery streams:

RCL release control toolsMDR meta data repositoryDSF DWP sourcing frameworkFDS feed specific data processing<SDA> subject data area<MAR> data mart

<MAR> <MAR>......

RCL MDR FDS

System of Records

DSFDSF

DWP Sourcing Framework

<SDA> <SDA>...

Common Data Warehouse

Project OverviewProject Overview

Two areas:/dwp_root UNIX based files, version controlled/dwp_data dynamic data, archival on demand

Additional directory level for the environment in order to support more than one environment on one system

/dwp_root is organized by delivery streams

Directory StructureDirectory Structure

Project OverviewProject Overview

/dwp_data is organized by logical processing stepsand by delivery streams

Symbolic links from PowerCenter program areato /dwp_data

Tool support for generation of directories(delta processing, integration with release procedures)

Directory Structure (continued)Directory Structure (continued)

PowerCenter ConfigurationPowerCenter Configuration

Start with one single (local) repository

Support different, clearly separated environments with different responsibilities

All environments have identical structure (products, databases, PowerCenter server configurations)

All program changes (UNIX, PowerCenter) are done on the development system

All changes on test and production systems go through the release process and are clearly tracked

Design PrinciplesDesign Principles

PowerCenter ConfigurationPowerCenter Configuration

Repository StructureRepository Structure

D S V S A S P

Server Server Server

D developmentS stagingV versioningA integration / user acceptance testP production

PowerCenter ConfigurationPowerCenter Configuration

Folder StructureFolder Structure

Release area is identical in all environments.

user area

(avoid shortcuts)

team area

release area

PowerCenter ConfigurationPowerCenter Configuration

developer development, unit and integration testing

delivery stream manager manager, release planning

release manager deployment, tracking, configuration controlPowerCenter administration

session manager session management for testand production systems

Roles and ResponsibilitiesRoles and Responsibilities

Role Responsibility

Configuration ManagementConfiguration Management

External tool for software configuration management (Rational Corporation)

Powerful version control, work space andbuild management

Same directory structure as in development environment

Fully automated access layer

ClearCaseClearCase

Configuration ManagementConfiguration Management

Integrate all release specific data:sourcing framework, DDL, documentation, scripts & SQL

Avoid problems with current PowerCenter version mechanism (shortcuts, sessions)

Enable automation of release process

Different responsibilities for different environments

Enforce change tracking

Why external version control?Why external version control?

Configuration ManagementConfiguration Management

<delivery stream>_<major>.<minor>.<patch>

e. g. ACC_1.0.1

<major> major release number(high level “wave” planning)

<minor> minor release number(delivery stream development plan)

<patch> patch level (bug fixes, emergency releases)

Release NamingRelease Naming

Configuration ManagementConfiguration Management

Versioning Versioning ExampleExample

Labeling Schema:

ACC_x.y.z release labelRLS_A current version in

user acceptance testRLS_P current version in production

/main/1/main/1

/main/2/main/2

/main/4/main/4

/main/5/main/5

ACC_1.0.0

ACC_1.0.1

ACC_1.0.2

ACC_2.0.0

RLS_P

RLS_A

reset_acc.sql

/main/3/main/3

Deployment ProcessDeployment Process

Repository Copy:- via GUI / database backup & restore- trackable (result is a file)- problem: overwrites server configuration

Folder Copy:- only via GUI- and therefore not trackable

Mapping Copy:- only via GUI- used only within development repository

PowerCenter Deployment FacilitiesPowerCenter Deployment Facilities

Deployment ProcessDeployment Process

“Repository Copy” is the only trackable method

Delivery Streams require folder level deployment

ApproachApproach

Introduce staging repositories in all environments

Staging repository is used to separate folder from repository, using the “Folder Copy” method

Staging repository is then deployed using the“Repository Copy” method

Use combination of both methods:

Deployment ProcessDeployment Process

Freeze, Deliver and Receive ProcessesFreeze, Deliver and Receive Processes

D S V S A S P

manual folder copydatabase unload / loadautomated processdelivery package

ClearCase

Freeze

Deliver

Receive

Deployment ProcessDeployment Process

sub unload {system("db2 connect to $repository user $user using $password");

db2select(\@tables, ”SELECT tabname FROM syscat.tablesWHERE tabschema = 'PCSTAGE'and (tabname LIKE 'OPB_%' or tabname LIKE 'REP_%') and type = 'T'ORDER BY tabname");

foreach $table (@tables) {system("db2 export to $table of del modified by coldel0x02 chardel0x03

select \\* from $user.$table”);};

system("tar cvf $tarfile *");}

Repository UnloadRepository Unload

Deployment ProcessDeployment Process

sub load {system("db2 connect to $repository user $user using $password");system("tar xvf $tarfile");

foreach $file (("OPB_REPOSIT", "OPB_REPOSIT_INFO")) {system("sed -e 's/CDW_StagingD/$name/g' $file > tmp.out");system("sed -e 's/DDBPCR/$dbs/g' tmp.out > $file");

}foreach $table (<*>) {

system("db2 delete from $user.$table”);system("db2 import from $table

of del modified by coldel0x02 chardel0x03 delprioritycharinsert into $user.$table”);

};}

Repository LoadRepository Load

Release ProcessRelease Process

Responsible for high level planning and impact analysis

Defines release scope and release numbers on baseof delivery streams

Assigns responsibilities (delivery stream manager, data modeler, database administrator, business responsible)

Result is documented in “wave plan” document

Wave plan is base for all subsequent design, implementation and quality assurance tasks

Change Control BoardChange Control Board

Release ProcessRelease Process

1. Preparation of UNIX and PowerCenter release areas delivery stream manager

2. Issue release request delivery stream manager

3. Create new release (Freeze) release manager- Copy to staging repository, unload- Create / update ClearCase file versions- Create release label and attach to new release

4. Create release package (Deliver) release manager- delta / full processing- lock release in ClearCase

5. Install release (Receive) IT integration / IT operation - Load, copy from staging repository- Post install script

6. Create / update database objects in target environment database administrator

7. Restart PowerCenter server IT integration / IT operation

Release ProcedureRelease Procedure Responsible

Release ProcessRelease Process

Provide easy-to-use information on all releases

Today: ClearCase based

In future: Integration with meta data repository

Release DatabaseRelease Database

DSF_1.1.4 A 27-Jul-00.17:32:27 new config filesDSF_1.1.3 20-Jul-00.13:16:38 fix for single node load problemDSF_1.1.2 P 19-Jul-00.14:05:30 correct JDM, error handlingDSF_1.1.1 18-Jul-00.17:51:19 removed environment dependenciesDSF_1.1.0 17-Jul-00.17:11:47 see release notesDSF_1.0.2 11-Jun-00.14:07:01 changed Gen_EnvironmentDSF_1.0.1 09-Jun-00.13:20:03 see release notesDSF_1.0.0 24-May-00.17:48:42 initial version

Example:

SummarySummary

PowerCenter was successfully introduced at UBS and proved to work well within the sourcing framework

Configuration control with ClearCase is efficient and straight forward

Release process meets the requirements and is efficient(so far ~200 successful releases, effort usually below ½ hour)

Generally little administration overhead although some improvements in PowerCenter would make administrator´sand release manager´s life easier

ExperiencesExperiences

SummarySummary

Copy of sessions

Export / import folders via (UNIX) command linefor further automation of release process

Avoid restart after deployment

Logical backups / restores via command line with result on server side (more flexible than database backup, avoid ODBC overhead)

User administration via command line

Wish ListWish List

Contact InformationContact Information

Friedrich Lehn

UBS AG, Switzerland

phone: +49 171 360 60 84

email: friedrich.lehn@t-online.de