Key role of data quality in Internal Ratings Based (IRB) approaches

International Congress of Risks Sao Paulo 22-23 October 2012

10/22/2012

Isabelle Thomazeau Banque de France - ACP

Disclaimer

The views in this presentation are those of the author and do not necessarily reflect those of the Autorit de Contrle Prudentiel (ACP) or of the Banque de France.

10/22/2012

Preliminary remarks
Another presentation in this seminar describes the validation process to be applied for IRB models. The review of data is part of the validation process. This presentation focuses on the data to be used in IRB approaches: To estimate IRB parameters (PD and LGD) for assets which have not defaulted on corporate (except Specialized lending), sovereign, banks and retail portfolios. EAD/CCF will not be detailed nor other Basel portfolios To calculate these IRB parameters on the whole perimeter Implementation

10/22/2012

Summary
I.

Introduction
1. 2.

Context Presentation of IRB approaches Rating 1. Scoring models 2. Expert given approach PD estimation 1. Context 2. Default data available 3. Low Default Portfolios Context No data available Recovery data available

II.

PD
1.

2.

III.

LGD
1. 2. 3.

IV. V. VI.

Implementation, information systems Back-testing Conclusion

10/22/2012

I.1 Introduction - Context

The Basel Committee on Banking Supervision (BCBS) introduced in April 1995 the notion of internal model to estimate the capital adequacy of banks trading book. The risk indicator used is called Value-at-Risk (VaR). The use of internal models was then extended to credit risk and operational risk in June 2004. Main reference from the Basel Committee on Banking Supervision for IRB approaches:
International Convergence of Capital Measurement and Capital Standards : A Revised Framework, Comprehensive Version june 2006 which is a compilation of some documents among which the 1996 Amendments to the Capital Accord. http://www.bis.org/publ/bcbs128.htm. [1]

In Europe, Committee of European Banking Supervisors (CEBS)

Guidelines on implementation, validation and assessment of Advanced Measurement (AMA) and Internal Ratings Based (IRB) Approaches . 04/04/2006

http://www.eba.europa.eu/Publications/Guidelines...

10/22/2012

I.1 Introduction - Context

In France, institutions may use Internal Ratings Based (IRB) approaches for the calculation of their minimum capital requirements for Credit Risk, instead of the Standardised Approach. French Commission Bancaire (now the Prudential Control Authority ACP) authorized the first institutions to use their IRB approaches at the end of 2007. Since then, French majors banks have developed internal models for credit risk for part or all of their banking book portfolios. Some information on these models are given in their annual reports. The Risk Modeling Control Unit (CCRM) team from the ACP is particularly in charge of the models on-site reviews. For IRB internal models the review must be made a priori, in order to determine whether, or not, the institution can be allowed to use its internal models for prudential purposes.
10/22/2012 6

I.2 Introduction - Presentation of IRB approaches

Under the IRB approaches, banks must categorize banking book exposures into 5 broad classes of assets : Corporate
(+ 5 sub-classes of specialized lending): project finance, object finance, commodities finance, income-producing real estate, high-volatility commercial real estate

Sovereign Bank Retail (3 sub-classes)

exposures secured by residential properties, qualifying revolving retail exposures, other retail exposures

Equity

10/22/2012

I.2 Introduction - Presentation of IRB approaches

The IRB approaches require the banks to use their own internal estimates of some or all of the following 4 risk components for a given exposure:
Probability of Default (PD) : defined per rating and classes of assets, it represents the one year probability of default of the borrower Loss Given Default (LGD) : defined by homogenous risk classes in terms of recoveries (or losses), LGD measures the loss given default as a percentage of the EAD Exposure At Default (EAD)/Credit Conversion Factor (CCF): once again defined by homogenous risk classes, these quantities measure the exposure rate when default happens Effective Maturity (M) effective maturity of the exposure calculated by the bank (only for Corporate portfolios)

10/22/2012

I.2 Introduction - Presentation of IRB approaches

One approach for Retail

Banks provide their own estimates of PD, LGD and EAD, with minimum standards.

Two approaches for corporate (except specialised lending), banks and sovereign:
IRB Foundation (IRBF): Banks provide their own estimates of PD Supervisory estimates for other risk components IRB Advanced (IRBA): Banks provide their own estimates of PD, LGD and EAD and their own calculation of M, with minimum standards (e.g. PD: 0.03% except for sovereign).

For both IRBF and IRBA approaches, the risk-weighted assets (RWA), which depend on PD, LGD, EAD (CCF) are provided in the Framework [1].

10/22/2012

II PD
Main steps in the process: Segmentation of the population in homogeneous risk classes in terms of default risk (i.e. rating)
Construction of a notation tool at a borrower or exposure level. Choice between: Scores Expert given approach Mixed method

PD estimation in each class

At least 7 non defaulted grades have to be defined (corporate, sovereign, banks) A PD must be calibrated for each grade inside a given class.

Validation
Independent periodic review of the PD modelling (score and segmentation) Back-testing

10/22/2012

10

II.1 PD - Rating
1. Scoring models Statistical approaches may be applied when a significant number of defaults is observed:
Retail, Small and Medium size Entities (SME)

Steps:
Define the sample used to build the score Choose the criteria to be explained (Y) : Select a list of qualitative and quantitative data candidates to explain Y Segmentation of into several brackets Xi manually or with optimization algorithms Measure of the ability to explain Y as a function of Xi using statistical indicators (Chi2, T-Tschuprov, VCramer, ) Final selection of Xi and coefficients, computed using statistical tools so that : Y = i Xi

10/22/2012

11

II.1 PD - Rating
Three main technics to select the final variables of the score :
Backward : Many variables are initially included in the model. Step by step some of them are removed if it does not really impact the discrimination of the model. Forward : Variables are added one by one to the model. The process stops when adding one more variable does not impact the discrimination of the model. Stepwise : At each step variables can be added or removed.

At the end of this process, the selected Xi have low correlations between them. Score tranching in homogeneous classes
Enough exposures or borrowers in each pool No undue concentration Correct risk differentiation and risk level stability

10/22/2012

12

II.1 PD - Rating
Sample
Large sample to be split in two parts, one to build the score, and the second to test it. Otherwise, bootstrapping methods can be applied. Sample representative of the population It would be identical to the population regarding all its characteristics. In fact, as there are a lot of variables, the distribution of the sample is generally studied regarding some main criteria For example Corporate : Region, sector, size, , The procedure used to build the calibration sample must not generate any selection bias

10/22/2012

13

II.1 PD - Rating
Data for each observation
For each observation (borrower/transaction), at the beginning of the process, take a large range of qualitative and quantitative data among which the Xi will be selected:
General characteristics Solvability criteria: Retail : profession, revenues, Corporate: Capital, debts,

The quality of the data is essential:

Missing values: a process to deal with missing values must be determined: Elimination of these data above a threshold of missing values in the sample Replace them by the average Replace them by the 50% percentile, The process used may have an impact on the result of the modelisation Errors: a very limited number of erroneous values in the database may impact significantly the calibration of the model: erroneous data to be removed or replaced.
10/22/2012 14

II.1 PD - Rating
Example 1
Simulation of two data with very low correlation (-4%) Change of value for these two data for one observation in the sample (these new values are erroneous ones, in red in the following plot) The correlation increases from -4% to 56%. At least one of these data will be removed from the score because of its high correlation with the other, due to an erroneous value. The plot represents one data as a function of the other for a set of observations.
14 12 10 8 6 4 2 0 -4 -2 -2 -4 0 2 4 6 8 10 12 14

10/22/2012

15

II.1 PD - Rating
Example 2
Simulation of two data with very high correlation (-75%) Change of value for one of these two data for one observation (to obtain an erroneous value). The correlation decreases from 75% to 13%. Perhaps these two data will be taken into account into the score, due to an erroneous value of one of them in one observation of the sample: low correlation due to one erroneous value.
5 0 -3 -2 -1 -5 -10 -15 -20 -25 0 1 2 3

10/22/2012

16

II.1 PD - Rating
Definition of the criteria to be explained
Criteria : Usually default (or not default) for a given time horizon. Default may be defined at a counterparty or transaction level (sometimes for Retail) This time horizon is often (but not always) of one year (but in any case the PD is a 1 year PD) For each borrower/transaction in the sample, a photo is taken at a date T and the default is observed during the period [T,T+time horizon], i.e. generally [T,T+1 year] Data have to be available one year before the observation default/no default Sometimes, data at T (which are data in the past) have to be reconstructed, especially qualitative ones.

All these tasks on data are time consuming but are essential for the score to be of good quality (discriminant,)
10/22/2012 17

II.1 PD - Rating

2. Expert given approach Usually applied for Low Default Portfolios: Large Corporate, Banks, Sovereign
But sometimes scoring model by replication of (external) ratings

Notation to be objective:
Anyone could approximately reproduce it Notation grids make easier this internal rating process

Correspondence between internal and external ratings: mapping are usually defined (for the further step of estimation of the PD in each class) Differences between internal and external ratings have to be explained.

10/22/2012

18

II.2 PD - Estimation
1. Context PD estimated for each class as a Long Term average of annual default rates.
Requirement : calculation with 1 year default rates observed during at least a 5 years period.

PD Through The Cycle (TTC)

Define the length of a cycle If PDs are not calculated on an entire cycle or on several entire cycles, they can be under or overestimated depending on the position in the cycle

Finally, PD have to be estimated using an average of observed default rates on a long period, preferably multiplier of a cycle Prudential margins may have to be added (for data quality, models uncertainties, TTC effects, )
10/22/2012 19

II.2 PD - Estimation
Two important components in the default definition: Materiality threshold:
If the threshold is too low, the PD are likely to go up and the LGD to decrease. Then the RWA are often underestimated. Some European countries have already defined a threshold Future CRR/CRD4: The supervisors will have to fix a threshold (Regulatory Technical Standards from the European Banking Authority).

Contagion :
The default rates will be higher when the default contagion is applied to the whole subsidiaries in a group, than if not. Clear rules of contagion to be defined: For example for retail portfolios, if the default is defined at a transaction level, contagion to all the transactions of the borrower, to clients linked to this borrower,

For Corporate, banks and sovereigns, materiality thresholds and contagion are more judgmental
10/22/2012 20

II.2 PD - Estimation
Calculation of the1 year observed default rates times series for each class:
Used for PD estimations when sufficient default data are available for every risk pool Used for Back-testing purposes

1 year default rate = number of observed defaults during the 1 year period divided by population in the class at the beginning of the period. Rectification techniques sometimes used in order to obtain comparable default rates in the time series
10/22/2012 21

II.2 PD - Estimation
So, a one year default rates depends on the approach used to determine:
(a): The number of default during the one year period considered, (b): The total population in the class.

Rules have to be specified for the treatment of specific cases:

Multi defaults during the period (impact on (a)) If they are taken into account, increase of the observed default rate, but probably decrease of the LGD Occurrences not available on the whole period (impact on (b)) Usually removed except if a default occurred as they were in the sample Half of these contracts removed from the population Technical defaults (impact on (a))
10/22/2012 22

II.2 PD - Estimation
2. Default data available The estimation of the PDs must be based on observed internal default rates calculated on the whole population and not on samples. Many factors impact the determination of the observed one year default rates:
thresholds, contagion rules, multi defaults treatment,

All these factors also impact the PDs.

10/22/2012

23

II.2 PD - Estimation
Update frequency:
Yearly: For example time series of one year defaults rates lengthened every year: question of the maximum length Length is not an entire number of economic cycles If the back-testing is not acceptable.

In France, when the first IRB models have been reviewed by the ACP, most observed defaults rates have been corrected to take into account data quality problems and additional margins have been included in the PD estimation.

10/22/2012

24

II.2 PD - Estimation
3. Low Default Portfolios (LDP) Use of external rating agencies long term default rates to determine the PD. Pay attention to:
Adequacy between the internal and external default definitions Criteria taken into account for external and internal ratings definitions (for example Moodys ratings represent jugement on the Expected Loss (EL), ie PD and LGD) Representativity of the external agencies population

More or less complex calculation tool to calculate the PDs to apply to each internal rating :
Mappings between internal and external ratings Generally the number of classes is not the same for internal and external ratings Every bank applies its own methodology to calculate its PDs based on default rates published by external agencies.
10/22/2012 25

II.2 PD - Estimation
Use of shared databases is allowed for PD (but not used for French banks). Academic litterature gives techniques which could be applied to LDP
Observation of migrations between risk classes, Theoretical PD scale based on the rating hierarchy,

When applying these different techniques, no internal historical data is used to calculate the PD, prudential margin is required.

10/22/2012

26

III LGD
1. Context LGD are defined at a transaction level.
(but LGD senior unsecured rather estimated at the counterparty level, then the garantees are taken into account at the transaction level)

Same steps than for PD estimations

Segmentation in homogeneous classes in terms of recovery (or losses) LGD estimation for each class

LGD apply to exposures not in default LGD Through The Cycle,

Length of the cycle

Prudential margins to add to the LGD estimations

10/22/2012 27

III LGD
Two cases:
No available data (Low Default Portfolio): Expert Given segmentation Recovery data available: modelling (Retail, PME)

2. No data available (Corporate, banks, sovereign) Expert given approach for both segmentation and values of the LGD in each class Examples of axis for a Corporate LGD segmentation:
Guarantee (secured/unsecured), region, economic sector, sales, LGD values fixed by experts for each class Then, correction of the senior unsecured LGD to take into account the different guarantees to estimate secured LGD. This last step is possible if guarantees have been registered in the systems.
10/22/2012 28

III LGD
3. Recovery Data available Segmentation by statistical approach or expert given LGD are estimated using recoveries observed on defaulted transactions
Importance of the default definition (like for PD)

Requirements: Historical data of at least 5 years of annual recoveries for retail, 7 years for corporate, sovereign and banks LGD have to be estimated using long term weighted averages losses (or recoveries) observed for exposures on counterparties in default:
Given at a transaction level and based on annual recoveries 1, 2, , n years after the default event,. Recovery data have to be as exhaustive as possible: correctly registered in the information system of the bank.
10/22/2012 29

III LGD
The post default recovery process may be very long:
Problem: usually lack of data available on the long term.

The final recovery depends on the way the long term recoveries are estimated. Several possibilities:
For each transaction, use of the whole data available Extrapolation: from the last available point, from a fixed delay after default, Choice of the horizon above which recoveries are extrapolated Fix a maximum time after which no more amount is recovered: Relevance of the maximum recovery horizon chosen Modelling with defaults closed or with closed and not closed defaults (to increase the size of the database)

Take into account the specificities of restructured debts.

10/22/2012 30

III LGD
LGD also depends on:
Discount factors applied to the recoveries Use the nominal rate of the contract is not relevant Costs They have to be deducted from the recoveries If they are partially taken into account, the LGD is underestimated Problem of accuracy (global estimations) Post default recoveries They have to be added to the recoveries If they are not taken into account, the LGD will be conservative (which is not a problem).

As LGD depends on the recovery politics of the bank, it is difficult to compare the banks LGD levels. Downturn LGD:
Often not really taken into account except via additional margins, and difficult to calibrate.
10/22/2012 31

IV Implementation, information systems

In order to calculate the RWA and capital requirements:

For each exposure, determination of the PD (depending on the rating), LGD (depending on some characteristics of the exposure) and EAD For corporate the same rating and PD are assigned to each transaction, with some exceptions (for example : non transfert risk ; guarantees attached to an exposure, these one require an adjustment of the notation)

The quality of the information system is essential in the process. The following part lists some major components of the information system (not exhaustive)
10/22/2012 32

IV Implementation, information systems

Borrowers/transactions administration tools:
Links between transactions and clients Corporate structures, including subsidiaries and the nature of the links between entities in the same group In order to be able to apply default or ratings contagion Exhaustivity of the clients/credits, with their main characteristics. Default registrations: dates of entry/exit, Defaults have to be taken into account without delay An important delay between a default and its integration in the system is not acceptable. A regular update process has to be planned.

10/22/2012

33

IV Implementation, information systems

Notation tools
Availability in the rating tools of all the data required to rate the counterparties/transaction: Missing values for data used in the rating process should be rare Coherence between data has to be checked before input in the system (no client born in 1800, no client 18 years old having accounts in the bank for 20 years, ) Data have to be updated on a regular basis and in case of changes in the characteristics of the counterparty

10/22/2012

34

IV Implementation, information systems

Client relationship officers may be located anywhere in the bank: Notation tools have to be available anywhere Notation must be updated at least once a year Consistency of the rating Same rating tool for any counterparty of a given nature Unicity of the rating for a given counterparty, even if it is rated in several locations. Treatments to apply to the non rated borrowers/transactions have to be specified these cases must be rare. Reliability of the ratings, which gives the PD (because of the direct correspondence between rating and PD) depends on the quality of the data in input of the notation tool.
10/22/2012 35

IV Implementation, information systems

Perimeter:
Exhaustivity: the whole IRB perimeter has to be taken into account in terms of: Foreign implentations of the bank: subsidiaries, Borrowers/products. Every counterparty/transaction in the IRB perimeter has to be rated with one of the notation tools

When different systems are implemented in different locations, data may not be homogeneous Be careful

10/22/2012

36

IV Implementation, information systems

Risk database
Identification of the Basel portfolio and sub-portfolio for each transaction, to include every exposure in the appropriate portfolio Exhaustivity of data required to classify exposures in Basel portfolios, PD, LGD, EAD/CCF Historisation of data: Counterparties/transactions, exposures, guarantors, guarantees, At a transaction level: data useful for PD, LGD, EAD/CCF attributions Times series of default rates per rating, recoveries per class of LGD, For transactions in default: dates of default and of exit of default, exposure at default, drawn amounts posterior to default, recoveries post default, losses amounts,
10/22/2012 37

IV Implementation, information systems

Calculation tools for PD, LGD, EAD/CCF Calculation tools for RWA and capital requirements Accounting/risks reconciliation tool

10/22/2012

38

V. Back-testing
Supervisors require banks to perform at least annual backtesting on PD, LGD and EAD/CCF estimates. Back-testing :
Ex-post comparison of the 1 year default rates, the losses and the EAD/CCF observed against the PD, LGD and EAD/CCF calibrated. Very useful to assess the quality of PD, LGD (and EAD/CCF) calculations. More difficult to make for LGD because of a lack of observed recoveries on long term horizons. Same problem with EAD/CCF.

10/22/2012

39

V. Back-testing
Several axes (PD example):
Comparison between the average of the 1 year default rates observed on the historical period and the PD Statistical validity of the estimation: determination of a confidence interval (under normal distribution assumptions for example) Comparison between the annual default rate observed (i.e. Point In Time: PIT) and the PD.

The back-testing methodology must be adapted depending on :

The update frequency of the estimators The time series used to calculate these estimators.

10/22/2012

40

V Back-testing

Example for PD:

Assume that an estimator (PD) is computed annually with an increase of the time series (default rates) A comparison of this estimator with the LT averages of observed defaults rates seems not very relevant, especially if exactly the same data are used to compute the estimates and the LT averages. Then this test has to be completed by others analysis.

A back-testing is essential to check whether the values of the Basel parameters are relevant or if a new calibration is required. IRB approaches would not be validated if no back-testing is available.

10/22/2012

41

VI Conclusion
The quality of the data is very important at each step of the process:
At the modelling step, to segment the population in risk classes (in terms of PD, LGD, and also EAD/CCF, not developed in this presentation) To calibrate PD, LGD (and EAD/CCF) For the regular calculations of capital after the approval of the model.

The information system is generally complex because it has to be able to capture data on a very large perimeter in terms of subsidiaries of the bank, of portfolios and of products.

The backtesting is a key test to control if the Basel estimators are correct.
10/22/2012 42