Making Sense out of

Metrics
ISDA / PRMIA
17
th
August 2004
In 2002
A global survey of 76 banks on the
existence of formal KRI programs:
Acknowledgement Raft International Limited
What is a KRI?
The number of fails has increased by 2%

Is this a KRI?
Some definitions
Metric something observed or calculated that is used
to show the presence or state of a condition or trend;
an instrument or gauge that measures something and
registers the measurement; something such as a light,
sign, or pointer that gives information, for example
about which direction to follow
KRI Key Risk Indicator
KPI Key Performance Indicator
KCI Key Control Indicator
KMI Key Management Indicator

Overall, prefer the general term indicator
What is a Risk Indicator?
Risk indicators are usually monitored over
time
Late trade processing in Bank X in May 2004:
London 9%, New York 10%, Singapore 9%
In which branch
do we have a
problem?
0%
5%
10%
15%
Feb Mar Apr May
Late Trade Processing
London
New York
Singapore
What is a Risk Indicator?
So, Singapore is the problem!
Is it?
0%
5%
10%
15%
Feb Mar Apr May
Late Trade Processing
London
New York
Singapore
0
500
1000
1500
2000
2500
3000
3500
Feb Mar Apr May
absolute numbers
Data without context may not expose the entire problem!
The Choice from the Multitude
A typical operation can
identify hundreds of
indicators
Some are risk, others
performance indicators
Indicators relevance
and weight change over
time
Some indicators are
meaningless on their
own

Graphics adapted from Reason, J.: Managing the Risks of Organizational
Accidents", Aldershot: Ashgate, 1997
Other factors
If a ratings agency is to rate operational
exposure, how will they compare
different organisations?
How will regulators evaluate the
effectiveness of different organisations
operational risk capabilities?
How does a business unit provide
senior management quality information?
Can the organisation use operational
metrics to provide stakeholder
information?

But, KRIs have been disappointing
No means of consistently relate the
occurrence of loss events and the
location of problems/situations
No means of classifying types of KRIs
Plenty of data but no idea of its relevance
No way to determine relevance
No observable best practice
No means of comparison, either internally
or externally
The solution?
An organisation needs a common
language or framework which identifies
areas of exposure and then allows..
Metrics to be identified to measure, monitor
and manage those exposures
Data on losses, near misses and control
failures to be recorded
Ongoing assessment of the exposure
Performance measurement around the
exposure, including use of capital
KRI Study - Background
Recognising the need, RiskBusiness
developed a strawman framework for
identifying risk points within an
institution
RMA and RiskBusiness co-sponsored
Part I of the Study to ascertain the
feasibility of using the framework as a
KRI framework
Seven institutions tested the framework
in a risk mapping exercise
Initial Participants
7 banks participated in Part I:
Citigroup
Deutsche Bank
Dresdner Kleinwort Wasserstein
JP Morgan Chase
KeyCorp
Royal Bank of Canada
State Street
ANZ and Abbey then joined to form the Study
Steering Group for Part II
The Study
Part I Proof of Concept
Part II 3 primary activities :
Broaden participation, transfer experience,
build industry risk profile
Define KRI Library
Develop detailed specifications
Future :
Industry benchmarking
Extend participation further
The KRI Framework
Validation
Initial risk maps were evaluated to
establish most risky risk categories
and business functions
Compared these to a similar evaluation
of the QIS 3 data and to the complete
Fitch Risk First database
Broad correlation, taking into account
the nature of the two data sources
Participant risk map deviations
Based on risk category:
RMA/RiskBusiness Risk
Categories


Stage 1
Rankings
(additive
High
ratings)
Participant
A
Participant
B
Participant
C
Participant
D
Participant
E
Participant
F
Participant
G
Risk
Business
Data Management 1 2 5 3 2 1 2 2 3
Improper Practices 2 1 1 1 1 4 1 7 2
Infrastructure & Systems 3 5 7 2 3 2 3 5 1
Transaction Management 4 7 6 6 6 6 5 1 4
Internal Theft & Fraud 5 3 2 5 5 3 4 9 5

Participant risk map deviations
Based on business function:
RMA/RiskBusiness Business
Functions


Stage 1
Rankings
(additive
High
ratings)
Participant
A
Participant
B
Participant
C
Participant
D
Participant
E
Participant
F
Participant
G
Risk
Business
Payment/Settlement/Collection
(cash/securities)
1 2 3 1 1 1 3 3 2
Instruction or Order Management 2 1 1 3 2 1 1 - 1
Custody and Actions (including
assets)
3 6 2 4 3 3 8 - 3
Transaction/Fees Capture and
Record Update
4 21 23 2 5 5 5 2 11
Relationship Management 5 3 4 19 11 14 18 1 14
Confirm/Affirm/Matching and
Documentation
6 5 9 19 4 4 2 - 4
Transaction Maintenance and
Administration
7 21 7 13 13 10 5 5 15
Infrastructure, Networks &
Maintenance
8 3 36 21 6 7 7 - 8
Pricing and Quotations 9 13 13 28 9 7 3 - 8
IT Security 10 10 5 14 5 13 9 - 12

Participants in Part II
Abbey
ABN Amro
ABSA
Acleda Bank
Alliance and Leicester
ANZ
Bank Austria Creditanstaldt
Bank Julius Ber
Bank of America
Bank Rakvat Indonesia
Bank Vontobel
BNP Paribas
Byblos Bank
Capital One
Citigroup
Commerzbank
De Lage Landen (sub of Rabobank)
Deutsche Bank
Dresdner Kleinwort Wasserstein
Erste Bank
Euroclear
Federation de Caisses Desjardins du Quebec
GMAC
Halifax Bank of Scotland
HSBC
Huntington National Bank
Investec
JP Morgan Chase
KeyCorp
Kookmin Bank
Macquarie Bank
Mizuho International
National Australia Group
National Bank of Canada
Nomura International
Northern Trust
Peoples Bank
Royal Bank of Canada
Royal Bank of Scotland
RZB
San Paolo IMI
SE Banken
Southwest Bank of Texas
Standard Bank of South Africa
State Street
Sumitomo Mitsui
TD Financial Group
Woori Bank
Washington Mutual

= Lead Participant

Defining Indicators
Identify candidates for each risk point
Evaluate candidates against qualifying
criteria (effectiveness, comparability, ease of
use/collection)
Agree descriptions for each qualifier
Prioritise nominated indicators
Participant comment and review
Generate detailed specifications, stored
in KRI Library at www.KRIeX.org
Considering each risk point..
Research
Historical Events
Identify Problems
in Progress
Look for Early
Warning Flags
Near misses, loss = 0
0 < Losses < threshold
Potential loss events, include
events that could lead this or another
institution to loss
After the problem has been generated
Before the problem has been generated
Losses > threshold
Issues for consideration
What is a KRI...KPIKCIor MIS?
An indicator can perform multiple roles
depending on who is using it
What about scaling and aggregation?
Do we scale then aggregate, or vice versa?
How many indicators should a firm be
monitoring?
The quest for the Magic 10 the KRI
Library has 1,600 indicators
Issues for consideration
Top-down versus Bottom-Up
Operations develop metrics for ongoing
use, Management want information
Combinations and clusters of indicators
Experience has demonstrated that in many
cases, it is groups of indicators which will
provide the best management information
Staff Turnover and Transaction Volumes
and Error Rates and
Next Steps - Benchmarking
Selected indicators will be driven totally by
broad participant agreement
Consists of participants delivering KRI data to
centralised function :
Data will be anonymous
Data will be collated, analysed and benchmark
values calculated
Participants have access to benchmarks for
comparative purposes
First submission expected in Q2, 2005
Next Steps KRI Library
Next intake of participants into KRI
Library starts in October 2004
currently have 68 additional firms
wishing to join
Insurance KRI Study starts during Q4
2004
Ongoing maintenance and extension to
the Library
In Summary..
A well-developed and structured
indicator program can deliver quality
management information and could
possibly be used as an adjustor to
capitalor at least as a measure of
efficient and effective use
Common language and standardisation
is imperative
The indicator program must deliver
value at all levels
Contact details
RiskBusiness International Limited
URL : www.riskbusiness.com
Study URL : www.kriex.org
Mike Finlay, Managing Director
Europe, Asia, Australia and Africa
Telephone : +44 7721 969 224
E-mail : mike.finlay@riskbusiness.com