Sie sind auf Seite 1von 16

BRAIN DUMP

Based

on the IBM/Rational approach for e2e software delivery the broader BI strategy their tools/processes employ

Understand Look

at some of the potential technical challenges

PROBLEM STATEMENT

deliver a reporting solution enabling business users to :

Understand the effectiveness and efciency of existing models/processes Standardised reports for Operations and Finance (reduce costs of managing the business) Drive up standards by providing visibility of activities Monitor and improve reuse within requirement, design and implementation phases Line of sight from user demands through development and test E2E traceability from cradle to grave

DASHBOARD SOURCES
Code Stats Requirements Data Build Stats Testing Status Data

System Data

Platform Data

Dashboard
Incident Management CIT Stats

SDK Information Design Information Finance Information

Project Data

PHASE 1
Requirements

are a key component in obtaining this picture (for the time being) systems Planning/Proposal and Portfolio management Data

Investment

Systems/Development

BI STRATEGY
Business Operations Financial Analyst Lines of Business Customer Experience Technical Delivery Systems/Platforms Release Planning

BI STRATEGY
Business

Operations/Financial Analyst Releases (Investment, Consumer Demand) benets

Business

business costs drill

down on individual cost components

BI STRATEGY
Platforms/Systems operational cycle

data, business planning

time, dwell time, trends metrics (decomposition, outcomes, etc)

volume Time

on target

BI STRATEGY
Mid term views
software tie

lifecycle performance metrics

in design collateral in with stories, test/QA, vendors, etc up (and measure) re-use of strategic capabilities pinch points in OM delivery cycles

drive

identify pull

data from BER, DesignIT, RSA, etc

ARCHITECTURE

Near real-time reporting requires a different approach... to processing the data...

ETL Process
D1
Landing (raw) Tables

ETL Process
D1
Landing/Staging Tables

SQL Modules Source/Producer


DIM_LOADER

SQL Modules Producer/Consumer


FACT_LOADER

Load dimension tables

Load fact and aggregate tables

Staging Area

Staging Area
Transform, cleanse Transform, conform

SQL Modules
D2

SQL Modules
Pre-Warehouse Tables

D2 Staging Tables

Iteration-2 (real-time) reporting requires that we take a different approach... to processing the data...
refactor to process incoming records on-demand

ETL Process
D1
Landing (raw) Tables

ETL Process
D1
Landing/Staging Tables

SQL Modules Source/Producer


DIM_LOADER

SQL Modules Producer/Consumer


FACT_LOADER

Load dimension tables

Load fact and aggregate tables

Staging Area

Staging Area
Transform, cleanse Transform, conform

SQL Modules
D2

SQL Modules
Pre-Warehouse Tables

D2 Staging Tables

Introduce CEP/ESP tools such as Esper along side shorter load processes (see next 2 slides for the latter)

ETL Process
D1
Pre-Warehouse Tables

Bulk Loader Index/Bulk-Load

Staging Area

BULK_LOADER

Data Warehouse

Drop indexes, Bulk load and rebuild


Load/Indexing

ETL Process

Tablespace/Logical Database

ETL/Loading Process
D1
Hot table partition

D1

Pre-Warehouse Tables

Control Packages SQL Modules Overflow Manager

Partition Load Manager


PARTITION_LOADER

Data Warehouse "hot" Partitions

OVERFLOW

Incremental INSERTs into "hot" partition


Load/Indexing

Transfer (Bulk Load) "hot" partition data once 'watermark' is reached


Transform, conform

Bulk Loader
D2
Main Warehouse Tables

ETL Process Staging Area


D1
Landing/Staging Tables

Physical Database
SQL Modules Producer/Consumer
FACT_LOADER

Load fact and aggregate tables

Data Warehouse
Transform, conform

SQL Modules
D2
Pre-Warehouse Tables

Exposed surface area ETL/Loading Process


D1
Hot table partition

Tablespace/Logical Database

Control Packages Overflow Manager


OVERFLOW

Transfer (Bulk Load) "hot" partition data once 'watermark' is reached


Transform, conform

Data Warehouse "hot" Partitions

Bulk Loader
D2
Main Warehouse Tables

Fail-over Handler

Physical Database

Data Warehouse Data Warehouse

BACKUP SLIDES

Some

of the challenges:

Near

real time BI requires a more complex architecture than nightly batch processing structure of complex process oriented data, particularly decomposition in hierarchies to adopt new paradigms such as CEP/ESP

The

Need

Das könnte Ihnen auch gefallen