Lawrence Mielnicki, Ph.D.Lawrence Mielnicki, Ph.D.FleetBoston FinancialFleetBoston Financial

Director, Retail Credit Risk AnalysisDirector, Retail Credit Risk Analysis

Challenges In Validation: Taking Challenges In Validation: Taking the Study Findings Forwardthe Study Findings Forward

A Retail Perspective A Retail Perspective

Risk Ratings May Be Less Problematic Risk Ratings May Be Less Problematic For Retail Lending Than For Commercial For Retail Lending Than For Commercial LendingLending

•23 of 26 respondents use statistical models in some aspect of their decisioning/account management process

•Portfolios are large with sufficient defaults for validation purposes

•Robust internal and external data is available

•Default definition not consistent but this is not a roadblock

Retail Model Validation Is Already a Best Retail Model Validation Is Already a Best Practice Activity -- At least in the U.S.Practice Activity -- At least in the U.S.

• OCC Bulletin 2000-16 sets standards for the development, use, and validation of statistical models

• External vendor models can be validated on bank data, tracked, and re-validated at appropriate intervals

• Internally developed models must pass certain tests that go beyond the use of statistical measures such as GINI and K-S

• Data integrity may be an issue especially for external vendor supplied models

From Scorecards to Risk RatingsFrom Scorecards to Risk Ratings

• The preponderance of statistical risk measurement tools used in retail lending are in the form of “scorecards”

– Scorecards are just a convenient way to implement statistical models

• Underlying most scorecards is a probability something bad will either happen or not

– The probability may be default but most likely something else like some state of delinquency

• This prediction needs to be converted to a probability of default

Do We Need A Common Risk Rating Do We Need A Common Risk Rating Scheme Across All Asset Classes?Scheme Across All Asset Classes?

• Matter of personal preference

• Pros

• Places all credit risk on an equal footing for reporting

• …..

• Cons

• There are many more opportunities for risk segmentation in retail portfolios

• …..

• Based on the QIS 3, this is not a requirement for Basle II AIRB

Challenges Going ForwardChallenges Going Forward•Banks need to cleanse their processes of “black box” models

— external vendors will need the message that “black box” models no longer sufficient for regulatory purposes

•Scorecard development in the future should recognize the multiple uses of the underlying models

— stoplight decisions (red, amber, green) are only one application of the tools

•Stress testing of inputs•Guidance from the supervisors regarding the level of regulatory validation for Basle AIRB purposes would be helpful

— is the OCC 2000-16 approach taken (validate the internal validation process) or

— will the supervisors look to validate each model independently?

Questions and AnswersQuestions and Answers

Challenges In Validation: Taking Challenges In Validation: Taking the Study Findings Forwardthe Study Findings Forward A Retail Perspective A Retail Perspective

Lawrence Mielnicki, Ph.D.Lawrence Mielnicki, Ph.D.FleetBoston FinancialFleetBoston Financial

Director, Retail Credit Risk AnalysisDirector, Retail Credit Risk Analysis