Release Control with PowerCenter

at UBS / Switzerland

Friedrich Lehn
Configuration & Release Manager
October 2000
Agenda

 Project Overview
 PowerCenter Configuration
 Configuration Management
 Deployment Process
 Release Process
 Summary
Project Overview
UBS
 Global, integrated investment services firm offering
- the full range of wealth management, asset management,
investment banking and securities services
- to the full spectrum of clients
 World’s largest private bank
 Top ten financial services firm by market capitalization
- $62 billion dollars
 Total client assets of US$1 trillion
Project Overview
System Environments

Development
Development Test
Test Production
Production

 IBM AIX SP2 cluster

 DB2 UDB EEE
 PowerCenter 1.6
 System of Records

Project Overview File

Preprocessing

Load Process
Conditioning

AutoLoader

PowerCenter

AutoLoader

SQL Apply

Common Data Warehouse

Project Overview
Delivery Stream
 Main organizational element
 Three letter acronym as base for naming standards
 Each delivery stream gets the same infrastructure:
UNIX directories, meta data area, PowerCenter folders
 Each delivery stream has business responsible, data modeler,
database administrator, delivery stream manager
 Team size typically between 1 and 5
 Delivery streams release independently from each other
Project Overview
System of Records
Application Structure
DSF
RCL MDR DSF FDS

Common Data
Warehouse

<SDA> ... <SDA>

Delivery streams:

RCL release control tools DWP Sourcing Framework

MDR meta data repository
DSF
FDS
DWP sourcing framework
feed specific data processing <MAR> ... <MAR>
<SDA> subject data area
<MAR> data mart
Project Overview
Directory Structure

 Two areas:
/dwp_root UNIX based files, version controlled
/dwp_data dynamic data, archival on demand
 Additional directory level for the environment in order to
support more than one environment on one system
 /dwp_root is organized by delivery streams
Project Overview
Directory Structure (continued)

 /dwp_data is organized by logical processing steps

and by delivery streams
 Symbolic links from PowerCenter program area
to /dwp_data
 Tool support for generation of directories
(delta processing, integration with release procedures)
PowerCenter Configuration
Design Principles
 Start with one single (local) repository
 Support different, clearly separated environments with
different responsibilities
 All environments have identical structure (products,
databases, PowerCenter server configurations)
 All program changes (UNIX, PowerCenter) are done on the
development system
 All changes on test and production systems go through the
release process and are clearly tracked
PowerCenter Configuration
Repository Structure

D S V S A S P

Server Server Server

D development
S staging
V versioning
A integration / user acceptance test
P production
PowerCenter Configuration
Folder Structure

user area

(avoid shortcuts)

team area

release area

Release area is identical in all environments.

PowerCenter Configuration
Roles and Responsibilities

Role Responsibility

developer development, unit and integration testing

delivery stream manager manager, release planning

release manager deployment, tracking, configuration control

PowerCenter administration

session manager session management for test

and production systems
Configuration Management
ClearCase

 External tool for software configuration management

(Rational Corporation)
 Powerful version control, work space and
build management
 Same directory structure as in development
environment
 Fully automated access layer
Configuration Management
Why external version control?

 Integrate all release specific data:

sourcing framework, DDL, documentation, scripts & SQL
 Avoid problems with current PowerCenter version
mechanism (shortcuts, sessions)
 Enable automation of release process
 Different responsibilities for different environments
 Enforce change tracking
Configuration Management
Release Naming

 <delivery stream>_<major>.<minor>.<patch>
 e. g. ACC_1.0.1
 <major> major release number
(high level “wave” planning)
 <minor> minor release number
(delivery stream development plan)
 <patch> patch level (bug fixes, emergency releases)
Configuration Management
Versioning reset_acc.sql

Example /main/1
/main/1
ACC_1.0.0

/main/2
/main/2
ACC_1.0.1
/main/3
/main/3 ACC_1.0.2
Labeling Schema: RLS_P
/main/4
/main/4
ACC_x.y.z release label
RLS_A current version in ACC_2.0.0
user acceptance test /main/5
/main/5
RLS_P current version in production RLS_A
Deployment Process
PowerCenter Deployment Facilities
 Repository Copy:
- via GUI / database backup & restore
- trackable (result is a file)
- problem: overwrites server configuration
 Folder Copy:
- only via GUI
- and therefore not trackable
 Mapping Copy:
- only via GUI
- used only within development repository
Deployment Process
Approach
 “Repository Copy” is the only trackable method
 Delivery Streams require folder level deployment

Use combination of both methods:

 Introduce staging repositories in all environments
 Staging repository is used to separate folder from repository,
using the “Folder Copy” method
 Staging repository is then deployed using the
“Repository Copy” method
Deployment Process
Freeze, Deliver and Receive Processes

D S V S A S P

Freeze Receive

manual folder copy

ClearCase database unload / load
Deliver automated process
delivery package
Deployment Process
Repository Unload
sub unload {
system("db2 connect to $repository user $user using $password");

db2select(\@tables, ”SELECT tabname FROM syscat.tables

WHERE tabschema = 'PCSTAGE'
and (tabname LIKE 'OPB_%' or tabname LIKE 'REP_%') and type = 'T'
ORDER BY tabname");

foreach $table (@tables) {

system("db2 export to $table of del modified by coldel0x02 chardel0x03
select \\* from $user.$table”);
};

system("tar cvf $tarfile *");

}
Deployment Process
Repository Load
sub load {
system("db2 connect to $repository user $user using $password");
system("tar xvf $tarfile");

foreach $file (("OPB_REPOSIT", "OPB_REPOSIT_INFO")) {

system("sed -e 's/CDW_StagingD/$name/g' $file > tmp.out");
system("sed -e 's/DDBPCR/$dbs/g' tmp.out > $file");
}
foreach $table (<*>) {
system("db2 delete from $user.$table”);
system("db2 import from $table
of del modified by coldel0x02 chardel0x03 delprioritychar
insert into $user.$table”);
};
}
Release Process
Change Control Board
 Responsible for high level planning and impact analysis
 Defines release scope and release numbers on base
of delivery streams
 Assigns responsibilities (delivery stream manager, data
modeler, database administrator, business responsible)
 Result is documented in “wave plan” document
 Wave plan is base for all subsequent design,
implementation and quality assurance tasks
Release Process
Release Procedure Responsible
1. Preparation of UNIX and PowerCenter release areas delivery stream manager
2. Issue release request delivery stream manager
3. Create new release (Freeze) release manager
- Copy to staging repository, unload
- Create / update ClearCase file versions
- Create release label and attach to new release
4. Create release package (Deliver) release manager
- delta / full processing
- lock release in ClearCase
5. Install release (Receive) IT integration / IT operation
- Load, copy from staging repository
- Post install script
6. Create / update database objects in target environment database administrator
7. Restart PowerCenter server IT integration / IT operation
Release Process
Release Database
 Provide easy-to-use information on all releases
 Today: ClearCase based
 In future: Integration with meta data repository

DSF_1.1.4 A 27-Jul-00.17:32:27 new config files

Example: DSF_1.1.3 20-Jul-00.13:16:38 fix for single node load problem
DSF_1.1.2 P 19-Jul-00.14:05:30 correct JDM, error handling
DSF_1.1.1 18-Jul-00.17:51:19 removed environment dependencies
DSF_1.1.0 17-Jul-00.17:11:47 see release notes
DSF_1.0.2 11-Jun-00.14:07:01 changed Gen_Environment
DSF_1.0.1 09-Jun-00.13:20:03 see release notes
DSF_1.0.0 24-May-00.17:48:42 initial version
Summary
Experiences
 PowerCenter was successfully introduced at UBS and proved
to work well within the sourcing framework
 Configuration control with ClearCase is efficient and straight
forward
 Release process meets the requirements and is efficient
(so far ~200 successful releases, effort usually below ½ hour)
 Generally little administration overhead although some
improvements in PowerCenter would make administrator´s
and release manager´s life easier
Summary
Wish List
 Copy of sessions
 Export / import folders via (UNIX) command line
for further automation of release process
 Avoid restart after deployment
 Logical backups / restores via command line with result on
server side (more flexible than database backup, avoid ODBC
overhead)
 User administration via command line
Contact Information

Friedrich Lehn
UBS AG, Switzerland

phone: +49 171 360 60 84

email: friedrich.lehn@t-online.de