Financial Management Phase III IGTC Bangalore

Basel I II III

Report submitted by

Sachin Todur (B0924)

Financial Management Phase III IGTC Bangalore

Introduction

Basel I is the round of deliberations by central bankers from around the world, and in 1988, the Basel Committee (BCBS) in Basel, Switzerland, published a set of minimal capital requirements for banks. This is also known as the 1988 Basel Accord, and was enforced by law in the Group of Ten (G-10) countries in 1992 . Basel I is now widely viewed as outmoded. Indeed, the world has changed as financial conglomerates, financial innovation and risk management have developed. Therefore, a more comprehensive set of guidelines, known as Basel II are in the process of implementation by several countries and new updates in response to the financial crisis commonly described as Basel III.

Background

The Committee was formed in response to the messy liquidation of a Cologne-based bank (Herstatt) in 1974. On 26 June 1974, a number of banks had released Deutsche Mark (German Mark) to the Bank Herstatt in exchange for dollar payments deliverable in New York. On account of differences in the time zones, there was a lag in the dollar payment to the counter-party banks, and during this gap, and before the dollar payments could be effected in New York, the Bank Herstatt was liquidated by German regulators.

This incident prompted the G-10 nations to form towards the end of 1974, the Basel Committee on Banking Supervision, under the auspices of the Bank of International Settlements (BIS) located in Basel, Switzerland.

Basel I, that is, the 1988 Basel Accord, primarily focused on credit risk. Assets of banks were classified and grouped in five categories according to credit risk, carrying risk weights of zero (for example home country sovereign debt), ten, twenty, fifty, and up to one hundred percent (this category has, as an example, most corporate debt). Banks with international presence are required to hold capital equal to 8 % of the risk-weighted assets.

Basel I, that is, the 1988 Basel Accord, primarily focused on credit risk. Assets of banks were classified and grouped in five categories according to credit risk, carrying risk weights of zero (for example home country sovereign debt), ten, twenty, fifty, and up to one hundred percent (this category has, as an example, most corporate debt). Banks with international presence are required to hold capital equal to 8 % of the risk-weighted assets. However, large banks like JPMorgan Chase found Basel I's 8% requirement to be unreasonable, and implemented credit default swaps so that in reality they would have to hold capital equivalent to only 1.6% of assets.

Since 1988, this framework has been progressively introduced in member countries of G-10, currently comprising 13 countries, namely, Belgium, Canada, France, Germany, Italy, Japan, Luxembourg, Netherlands, Spain, Sweden, Switzerland, United Kingdom and the United States of America. Most other countries, currently numbering over 100, have also adopted, at least in name, the principles prescribed

Financial Management Phase III IGTC Bangalore

under Basel I. The efficiency with which they are enforced varies, even within nations of the Group of Ten.

Basel II is the second of the Basel Accords, which are recommendations on banking laws and regulations issued by the Basel Committee on Banking Supervision. The purpose of Basel II, which was initially published in June 2004, is to create an international standard that banking regulators can use when creating regulations about how much capital banks need to put aside to guard against the types of financial and operational risks banks face. Advocates of Basel II believe that such an international standard can help protect the international financial system from the types of problems that might arise should a major bank or a series of banks collapse. In practice, Basel II attempts to accomplish this by setting up rigorous risk and capital management requirements designed to ensure that a bank holds capital reserves appropriate to the risk the bank exposes itself to through its lending and investment practices. Generally speaking, these rules mean that the greater risk to which the bank is exposed, the greater the amount of capital the bank needs to hold to safeguard its solvency and overall economic stability.

Objective

The final version aims at:

Ensuring that capital allocation is more risk sensitive;

Separating operational risk from credit risk, and quantifying both;

Attempting to align economic and regulatory capital more closely to reduce the scope for regulatory arbitrage.

While the final accord has largely addressed the regulatory arbitrage issue, there are still areas where regulatory capital requirements will diverge from the economic.

Basel II has largely left unchanged the question of how to actually define bank capital, which diverges from accounting equity in important respects. The Basel I definition, as modified up to the present, remains in place.

Basel II uses a "three pillars" concept – (1) minimum capital requirements (addressing risk), (2) supervisory review and (3) market discipline – to promote greater stability in the financial system.

The Basel I accord dealt with only parts of each of these pillars. For example: with respect to the first Basel II pillar, only one risk, credit risk, was dealt with in a simple manner while market risk was an afterthought; operational risk was not dealt with at all.

Financial Management Phase III IGTC Bangalore

The first pillarThe first pillar deals with maintenance of regulatory capital calculated for three major components of risk that a bank faces: credit risk, operational risk, and market risk. Other risks are not considered fully quantifiable at this stage.

The credit risk component can be calculated in three different ways of varying degree of sophistication, namely standardized approach, Foundation IRB and Advanced IRB. IRB stands for "Internal Rating-Based Approach".

For operational risk, there are three different approaches - basic indicator approach or BIA, standardized approach or TSA, and the internal measurement approach (an advanced form of which is the advanced measurement approach or AMA).

For market risk the preferred approach is VaR (value at risk).

As the Basel 2 recommendations are phased in by the banking industry it will move from standardised requirements to more refined and specific requirements that have been developed for each risk category by each individual bank. The upside for banks that do develop their own bespoke risk measurement systems is that they will be rewarded with potentially lower risk capital requirements. In future there will be closer links between the concepts of economic profit and regulatory capital.

Credit Risk can be calculated by using one of three approaches:

1. Standardised Approach

2. Foundation IRB (Internal Ratings Based) Approach

3. Advanced IRB Approach

The standardised approach sets out specific risk weights for certain types of credit risk. The standard risk weight categories are used under Basel 1 and are 0% for short term government bonds, 20% for exposures to OECD Banks, 50% for residential mortgages and 100% weighting on unsecured commercial loans. A new 150% rating comes in for borrowers with poor credit ratings. The minimum capital requirement (the percentage of risk weighted assets to be held as capital) remains at 8%.

For those Banks that decide to adopt the standardised ratings approach they will be forced to rely on the ratings generated by external agencies. Certain Banks are developing the IRB approach as a result.

Financial Management Phase III IGTC Bangalore

The second pillarThe second pillar deals with the regulatory response to the first pillar, giving regulators much improved 'tools' over those available to them under Basel I. It also provides a framework for dealing with all the other risks a bank may face, such as systemic risk, pension risk, concentration risk, strategic risk, reputational risk, liquidity risk and legal risk, which the accord combines under the title of residual risk. It gives banks a power to review their risk management system.

The Third pillar

Buyers and sellers in a market are said to be constrained by market discipline in setting prices because they have strong incentives to generate revenues and avoid bankruptcy. This means, in order to meet economic necessity, buyers must avoid prices that will drive them into bankruptcy and sellers must find prices that will generate revenue (or suffer the same fate).

Market discipline is a topic of particular concern because of banking deposit insurance laws. Most governments offer deposit insurance for people making deposits with banks. Normally, bank managers have strong incentives to avoid risky loans and other investments. However, mandated deposit insurance eliminates much of the risk to bankers. This constitutes a loss of market discipline. In order to counteract this loss of market discipline, governments introduce regulations aimed at preventing bank managers from taking excessive risk. Today market discipline is introduced into the Basel II Capital Accord as a pillar of prudential banking regulation.

Base II and the regulators

One of the most difficult aspects of implementing an international agreement is the need to accommodate differing cultures, varying structural models, and the complexities of public policy and existing regulation. Banks’ senior management will determine corporate strategy, as well as the country in which to base a particular type of business, based in part on how Basel II is ultimately interpreted by various countries' legislatures and regulators.

To assist banks operating with multiple reporting requirements for different regulators according to geographic location, there are several software applications available. These include capital calculation engines and extend to automated reporting solutions which include the reports required under COREP/FINREP.

Financial Management Phase III IGTC Bangalore

BASEL III (sometimes "Basel 3") refers to a new update to the Basel Accords that is under development. The term appeared in the literature as early as 2005 and is now in common usage anticipating this next revision to the Basel Accords. The Bank for International Settlements (BIS) itself began referring to this new international regulatory framework for banks as "Basel III" in September 2010.

The draft Basel III regulations include: o "tighter definitions of Tier 1 capital; banks must hold 4.5% by January 2015, then

a further 2.5%, totalling 7%.

o the introduction of a leverage ratio,

o a framework for counter-cyclical capital buffers,

o measures to limit counterparty credit risk,

o and short and medium-term quantitative liquidity ratios."

Professionals and officers with Basel II knowledge and experience will be required to lead the new Basel III projects, and they have started studying the differences from the Basel II framework.

Development of the New Basel III Standard

Summary of Changes Proposed in Basel III

First, the quality, consistency, and transparency of the capital base will be raised. o Tier 1 capital: the predominant form of Tier 1 capital must be common shares

and retained earnings

o Tier 2 capital instruments will be harmonised

o Tier 3 capital will be eliminated.

Second, the risk coverage of the capital framework will be strengthened.

o Strengthen the capital requirements for counterparty credit exposures arising from banks’ derivatives, repo and securities financing transactions

o Raise the capital buffers backing these exposures

o Reduce procyclicality and

Financial Management Phase III IGTC Bangalore

o Provide additional incentives to move OTC derivative contracts to central counterparties (probably clearing houses)

o Provide incentives to strengthen the risk management of counterparty credit exposures

Third, the Committee will introduce a leverage ratio as a supplementary measure to the Basel II risk-based framework.

o The Committee therefore is introducing a leverage ratio requirement that is intended to achieve the following objectives:

Put a floor under the build-up of leverage in the banking sector

Introduce additional safeguards against model risk and measurement error by supplementing the risk based measure with a simpler measure that is based on gross exposures.

Fourth, the Committee is introducing a series of measures to promote the build up of capital buffers in good times that can be drawn upon in periods of stress ("Reducing procyclicality and promoting countercyclical buffers").

o The Committee is introducing a series of measures to address procyclicality:

Dampen any excess cyclicality of the minimum capital requirement;

Promote more forward looking provisions;

Conserve capital to build buffers at individual banks and the banking sector that can be used in stress; and

o Achieve the broader macroprudential goal of protecting the banking sector from periods of excess credit growth.

Requirement to use long term data horizons to estimate probabilities of default,

downturn loss-given-default estimates, recommended in Basel II, to become mandatory

Improved calibration of the risk functions, which convert loss estimates into regulatory capital requirements.

Banks must conduct stress tests that include widening credit spreads in recessionary scenarios.

o Promoting stronger provisioning practices (forward looking provisioning):

Financial Management Phase III IGTC Bangalore

Advocating a change in the accounting standards towards an expected loss (EL) approach (usually, EL amount := LGD*PD*EAD).

Fifth, the Committee is introducing a global minimum liquidity standard for internationally active banks that includes a 30-day liquidity coverage ratio requirement underpinned by a longer-term structural liquidity ratio.

The Committee also is reviewing the need for additional capital, liquidity or other supervisory measures to reduce the externalities created by systemically important institutions.

Vendor models for credit risk measurement and management

Vendor models for credit risk measurement and management Observations from a review of selected models

Motivation and scope of the review A number of vendors produce and sell products that are designed to support credit-risk measurement and management functions within financial institutions. Such products may comprise risk measurement models, data, or systems developed by external commercial entities. Interest in such vendor products has grown recently, as financial institutions seek to meet new requirements such as the internal ratings-based (IRB) approaches of the Basel II framework. Indeed, the

Financial Management Phase III IGTC Bangalore

potential use of vendor models is explicitly contemplated within Basel II.1 Banks may choose to use vendor-developed models and datasets within the context of IRB processes to assign exposures to certain rating grades or segments, or to estimate IRB risk parameters. In its Newsletter No. 8 (March 2006), the Basel Committee on Banking Supervision (Basel Committee) provides guidance for the use of such products within the foundation or advanced IRB approaches.2

Even though individual institutions have primary responsibility for assessing the suitability of vendor products as part of their validation processes, supervisors have a natural interest in understanding the details of these products as well. Supervisors need to assess whether the use of the model, as opposed to the model itself, is in accordance with regulatory expectations and requirements. A solid understanding of a vendor model allows a supervisor to assess the conditions under which the model can be used by a given institution as part of the IRB approach. These conditions may include various ways of adjusting and complementing the model’s output in order to eliminate any systematic biases, or to bring it in line with credit risk characteristics that are specific to the institution’s portfolio. The supervisor would need to confirm whether a product does what the institution or vendor claims it does, and which aspects of the model meet (or do not meet) relevant regulatory and supervisory requirements. Against this backdrop, the Research Task Force (RTF) of the Basel Committee initiated a review of selected vendor credit-risk products, focusing on models that could be used to estimate probability of default (PD), loss-given-default (LGD), or exposure at default (EAD), and models that could be used to assign ratings or produce credit scores, for wholesale or retail credit exposures. The output of the review project was designed to meet two general objectives. First, it systematically described features of vendor products in a consistent way to facilitate comparisons among them. Second, it identified aspects of vendor models that relate to key supervisory issues. For its review of vendor credit-risk models, the RTF consulted with members of the validation subgroup of the Basel Committee Standards Implementation Group to identify models that were in reasonably wide use within the banking industry, were available as commerical off-the-shelf products, and were used by banks in more than one country. The resulting selection of models was not comprehensive, but did seem to be broadly representative of vendor models in current use in banking, although not necessarily of those used in every country represented on the Basel Committee. Vendors supported the RTF work by providing access to confidential technical documentation as well as to the models themselves. Overall, cooperation by vendors was strong. Vendors seemed to acknowledge the likely benefits of providing supervisors with a more robust understanding of their products, and they consistently made themselves available for in-depth discussions of the models. The rest of this paper provides a high-level discussion of certain observations from the RTF review of vendor products for credit risk measurement and management.

Overview of vendor credit-risk models Overall, the market for vendor models of credit risk is still developing. Descriptions of it –including this one – must be considered specific to the point in time at which they are prepared. The pace of change and innovation may be especially high due to the rapid rate of change in the discipline of credit risk measurement and management, as well as to recent regulatory initiatives like the Basel II framework. Additional evolution is likely, as recent market turmoil may provide a challenging testing ground for existing risk models. Even over the relatively brief course of the RTF review, vendors changed methodology as new methods appeared to provide superior performance. In addition, vendors

Financial Management Phase III IGTC Bangalore

continued to expand coverage for some models, adding to existing coverage of geography, types of exposures or types of borrowers. Most of the models presently available focus on the probability of a “credit event” such as delinquency or default, in which a borrower or counterparty fails to perform as contractually expected. The predominance of such models among vendor offerings likely reflects the historical development of quantitative credit-risk modelling; in addition, the probability of delinquency or default by an obligor generally is believed to be affected little if at all by practices of the lender, so that a single vendor model may be used by many lenders with no more than minor modifications. A small number of the models reviewed focused on conditional credit losses, described in practice as “loss in the event of default” or, as in Basel II, “loss-given-default” or LGD. These models clearly constitute a smaller segment of the overall market for vendor credit-risk models. Part of the explanation may be that relatively little data on recoveries (or default-related losses) is available publicly to support research and model development, and data may be difficult or costly to acquire. It is also possible that conditional credit losses depend heavily on firm-specific workout or recovery practices, creating a significant challenge for commercial off-the-shelf models, which by definition are based on data that are not firm specific. No vendor models were identified that addressed estimation of EAD.

In some cases, vendors provide multiple models as part of a “suite” within a single product offering. The alternative models may reflect differing measures of the credit event (such as different measures of default or delinquency), or may be based on different methods or different sets of variables. In other cases, modelling alternatives are offered as distinct products from a given vendor, or are marketed and supported by different organisational units within a single vendor firm. Banks using vendor models thus must choose not only among competing vendors, but among the models offered by a particular vendor. Since the most appropriate model generally varies depending on the specific business application, having a choice of models is valuable; however, the larger the number of choices, and the smaller the differences between the various alternatives, the more complicated the decision becomes for potential users. The number of users varies widely between models. Some models had few current clients, although this was unusual and may have simply reflected an early stage of the product’s life cycle. A few models have been widely adopted across the financial industry, although in some cases vendors appear to count firms as clients even if the model was purchased only for testing purposes, or was being used in a limited way by staff in one part of a large firm. Most products provide clear and intuitive output to model users. The output of many vendor products includes not only the essential modelling results, but also other information to support interpretation and use, such as key factors that determine the final result, benchmarks for comparison or mapping between different risk scales (such as between ratings or scores and default probabilities). In a few cases, ease of use appears to be one of the main “value propositions” of the vendor model, particularly where the product reflects simple or well-known methods and publicly available data. One potential barrier to use of any vendor product is the need for a technological platform, one that meets relevant technical requirements. For most vendor models, technical infrastructure requirements on the user side are modest. This is understandable, since placing significant hardware or other technical requirements on users would tend to raise the cost for clients, limiting the potential market and placing the vendor at a competitive disadvantage. Vendors generally find low-burden ways to deliver and maintain products. Many models are web-based; this form of delivery typically requires little or no specialised IT infrastructure on the part of clients.

Financial Management Phase III IGTC Bangalore

Key critical observations Over the course of the RTF review, several tendencies or observations emerged that may be of interest to users of vendor models, bank supervisors and model vendors. Some suggest potential directions for improvement in common practice. Ultimately, improvements in practices related to vendor models will help promote better credit risk management.

Advanced methods used in some models may not be well documented Many vendor models reflect sophisticated techniques. Computational and analytical complexity typically is not evident from the user interface; vendors are able to make very complicated models easy for users to apply. However, whether the methods on which a model is based are simple or complex, it is important that users be able to acquire a solid understanding of those methods in order to evaluate the suitability of the model for specific business applications under consideration. In this regard, clear documentation and description of the methods are essential. However, the review of vendor models found that many vendor products include important elements that are undocumented. In several cases, crucial documentation of aspects of models was made available only to paying clients, not to prospective clients or others. Although this is perhaps understandable as an approach to protecting intellectual property and proprietary model elements, such practices can make it difficult for prospective clients to properly evaluate the model.

Financial Management Phase III IGTC Bangalore

It can be difficult to distinguish facts from marketing in vendor documentation Although users should be expected to perform appropriate testing of a model before deciding to use it, review of documentation provided by the vendor also typically forms part of the decision process. Vendor documentation varies considerably in the level of detail provided about key elements, such as the nature of the reference data used for model development and the results of validation. Reasonably comprehensive technical information was available for some models, although certain details that could be vital for a complete understanding sometimes were not disclosed. More generally, vendors are understandably reluctant to include in their documentation any information that might reflect unfavourably on the product; as a result, it is difficult to regard vendor documentation as unbiased. Vendor documentation serves a dual purpose, and while one of those purposes is to inform, another is to promote the product. Third party publications in professional journals or other sources may be valuable supplements to information available from the vendor. Vendors frequently cite outside publications to support decisions made in the modelling process, as evidence of model effectiveness, or for other reasons. However, in some cases the authors of such publications actually are employed by or affiliated with the vendor, and these connections are not always made clear either in vendor documentation when the publications are cited or in the publications themselves. Since authors connected with the vendor are less likely to be entirely neutral in their view of the vendor product, credibility would be enhanced if relationships between authors and vendors were clearly stated, to allow clients and others to evaluate the reliability of such evidence. Most models are based on data from a limited geography or a limited population; vendors could provide guidance to users on types of obligors or exposures to which models are (or are not) applicable Many models are developed from data with limited geographic coverage. Perhaps as a result of the location of the primary vendors or data availability, some models are developed on data sets that primarily or exclusively reflect US data. Developmental or reference samples also often are skewed toward larger obligors or publicly traded obligors, either by design or because those are the only data available to the vendor at reasonable cost. Certain types of obligors, such as financial firms, are routinely excluded from reference data sets by many vendors. However, credit-risk models are most reliable when applied to obligors or exposures that are similar to those used in their development. Banks and other users should proceed more cautiously when attempting to apply a model to obligors or exposures that differ substantially from those in the reference data set for the model. It is important that users of a model have a clear understanding of the nature of the reference data set and its limitations. Some vendors provide greater levels of detail regarding the composition of the sample along key dimensions. Vendors often have the most comprehensive understanding of the limitations of the data, and therefore are in the best position to assist clients in understanding the relative applicability of the model to different exposures, obligor or lines of business. Vendors may be able to provide guidance on whether there are some situations in which the model should not be applied, or should be applied only with extreme caution. Although definitions of default and loss vary widely, most generally are portrayed as “consistent with Basel II” The correspondence between Basel II and vendor definitions of default and loss was an explicit component of this review. Almost all vendor credit-risk models included in their 4 Vendor models for credit risk measurement and management

Financial Management Phase III IGTC Bangalore

documentation a discussion of alignment with Basel II, and almost all portrayed the model as “consistent” with the requirements of Basel II. At the same time, the specific definitions of default and loss were observed to vary to a considerable degree across the vendor products. Basel definitions do provide for a limited degree of interpretive latitude in practice, so it is possible that implementations in vendor models could all differ from one another and still be consistent with the Basel II framework. However, it is more likely that many of them are actually at least somewhat inconsistent with a strict reading of the Basel definitions. A degree of inconsistency perhaps should be expected, since models must be developed from data, and in many cases the available data were generated over a period of time during which the Basel definition had not yet been determined. Moreover, certain aspects of the Basel II definitions of default are difficult to capture in a vendor dataset. For example, if an obligor is unlikely to repay obligations in full, this obligor should be characterised as “in default” under Basel II; while lenders may have relevant information to make a determination of “unlikely to pay”, vendors may not. Where definitions of default or loss inherent in reference data are different from those in Basel II, models based on those data may still be useful, but effective use likely requires some degree and type of adjustment, either through direct modification of model results or through adjustment of other aspects of broader risk measurement processes (eg adjusting the way the model results are used as an input to other components of a risk measurement system). Determining the appropriate adjustments requires a thorough understanding of the definitional differences, the impact on the model or data, and the implications for model results. This would be easier for model users if vendors were forthright about the differences and their impact rather than minimising those differences, and if vendors provided more support for users to make appropriate adjustments. Comparison across similar models can be helpful, but is difficult to do Comparison of alternative models – models based on different methods or different reference data – can be a highly constructive and valuable component of model evaluation. Comparisons can improve understanding of strengths and weaknesses of modelling methods and specifications, and support better interpretation of model performance. However, existing models can be difficult to compare due to differences in datasets and validation methodologies, even when they are based on similar conceptual foundations and use similar modelling processes. Performance measures for particular models often are motivated by the underlying methodology, or are tailored to reflect a vendor’s perception of the primary interests of users (eg whether the focus is on rank ordering and relative risk, or on the absolute magnitude of risk measures). Some vendors do provide their own comparisons to other models, but these may be limited or may not encompass the most relevant alternatives, and are not direct comparisons to other vendor products. Moreover, vendors may not have the proper incentives to provide unbiased comparisons. Very few third-party model comparisons exist in the public domain, although a few published research papers provide comparisons between examples of models that, even if not identical to vendor models, are similar enough to make the comparisons instructive. Some vendors provide informative validation results, although often these are in-sample results only Validation of models is crucial for determining whether a model is suitable for use. Effective validation requires that appropriate data be available, and that staff performing the validation are sufficiently competent and have a sufficiently detailed understanding of the model so that they can identify and test all relevant aspects. Because of the proprietary nature of the data or methods in many vendor models, technically proficient staff employed by the vendors often are in the best position to do thorough and effective validation. Of course, there is some question of whether vendors’ incentives are sufficiently well aligned with those of model users. Market forces may be of some help in this regard, since a model that performs poorly is unlikely to be a commercial success in the long run. However, banks and supervisors may be reluctant to rely on possibly slow market forces to sort out good models from bad.

Financial Management Phase III IGTC Bangalore

A review of available vendor validation results concluded that such results are helpful as a way to enhance the understanding of any model. However, there was clear variation across vendors in the quality and thoroughness of validation, with some quite good, others less so. One concern was that much validation of models was “in-sample”, with the same data used both for development and for validation. Some vendors create and use holdout samples for validation, but there is a natural tension inherent in holding out data during development that might improve the power of the model, which is likely to be seen as a first-order concern, in order to improve validation, which to some may appear to be second-order. The frequency of validation varies; conducting regular, periodic revalidation would appear to be a sound practice that some vendors follow, while for others revalidation is irregular and infrequent. Vendors could do more to encourage and support end-user validation, consistent with supervisory expectations Principles developed by the Basel Committee emphasise the importance of end-user assessments of the appropriateness of any vendor model, and of the key role of validation in such assessments.3 Some vendors explicitly support end-user validation of this sort, emphasising in their documentation and their communication with clients the importance of validation, and providing instructions or training on how validation can and should be performed. However, other vendors do little to support or encourage end-user validation. In some cases, data or modelling details that would seem to be essential for effective validation are not revealed by the vendor. Although vendors naturally take steps to protect proprietary elements of products, the variation in practice across vendors suggests that some vendors could strike a better balance in this regard. Vendors should recognise that market acceptance of the product over the longer term depends on the ability of users to conduct validation as part of their own risk management and to meet supervisory expectations for model use. Vendors may have information that would help users determine if contemplated modifications to models are reasonable Many models allow at least some degree of customisation; some include certain parameters that can be set or modified by users. The ability to change features of vendor models raises two potential issues for users. One is simply the need to have a comprehensive understanding of what the various potential changes are. Vendors could help ensure that clients have a thorough understanding of the aspects of the model that can be modified so that appropriate controls can be developed and put in place by users. The second issue is that model users need a good understanding of the impact of any user modifications, and how that impact may depend, if at all, on the nature of the obligor or exposure being considered. Vendors are likely to have insight into such issues, and could

Financial Management Phase III IGTC Bangalore

help clients achieve a solid understanding. They may also be able to advise users on types of changes that might be considered. For example, where input values or parameters are subject to user choice, certain values may be more reasonable than others. Developers of a model likely have a better sense of this than users, and could share this knowledge with their clients. Vendor products vary in the ease with which they can be used for stress testing Supervisory guidance and risk management practices increasingly emphasise the importance of rigorous stress testing to identify and assess risks in financial institutions. Some vendor models explicitly allow for the creation and evaluation of stress scenarios by users. However, others make stress testing difficult. In some cases, difficulties arise because key inputs or relationships cannot be modified by the user. In other cases, the model requires certain relationships to be maintained between variables, and it is difficult or impossible for the user to assess whether these relationships are being maintained as assumptions associated with a stress scenario are applied to the model. Stress testing using vendor models also confronts a trade-off related to model complexity. Models that include more variables or more complex relationships may lead to a richer and more realistic description of how risk is related to underlying factors. However, such models may also make stress testing more difficult; a larger number of variables requires a more elaborate specification of the stress scenario, creating the challenge of ensuring that all key elements of the scenario are internally consistent. Again, because of the necessarily proprietary nature of vendor products, it is difficult for vendors to fully reveal all of the model details that users might find useful for stress testing. However, vendors might further consider how their models can be used in stress testing, and whether additional helpful disclosure might be possible without jeopardising the competitive position and value of the product.

Conclusions Certain aspects of the observations described in this report suggest potential points of future emphasis or improvement, particularly for vendors or for model users. In many cases, there is an element of tension between the natural desire of vendors to protect proprietary elements of models and the needs of model users to have enough information about models to comply with regulatory expectations and principles of sound model use. These tensions likely will always exist to some extent, and the commercial interests of vendors cannot and should not be ignored. However, there appear to be a number of areas in which a modest amount of additional disclosure by vendors could significantly enhance the ability of institutions to use credit risk models in ways that conform more closely to supervisory expectations and sound management practices.

Points of potential improvement include:

Incomplete documentation or undocumented model features sometimes make it difficult for model users to properly evaluate models, especially on a prospective basis.

Vendor documentation can provide valuable information, but often reflects an inherent bias, and publications by staff or affiliated researchers are sometimes cited as if they are written by disinterested third parties.

Financial Management Phase III IGTC Bangalore

Limitations of reference data sets create a need for caution when applying models to obligors or exposures that differ markedly or systematically from observations in development samples.

Models generally are suitable for use in some cases but not in others, and vendors may be able to use their insight into the development and operation of models to better indicate for clients any specific business uses for which the vendors’ products are not suitable.

Model users may need to adjust model results (or their business use) for definitions of default or loss that differ from those required under Basel II, and vendors could provide valuable information to support appropriate adjustments.

Comparisons between competing models could be valuable, but are difficult to do and in practice are rarely performed in an informative way.

Vendors often are in the best position to conduct certain aspects of model validation, but may lack the proper incentives to conduct credible, unbiased testing.

Out-of-sample validation is valuable, but is not a common practice among vendors.

Regular, periodic re-validation is a sound practice for model vendors, but actual approaches to the frequency of re-validation vary.

While some vendors provide explicit support for highly important end-user validation of models, others provide little or no support.

Vendors can help clients understand which aspects of models can be modified, and the implications of those modifications for model applicability and performance.

Certain elements of models or their documentation might be modified to better support stress testing.

As the market for vendor credit-risk models develops and matures, vendors and end users may encounter opportunities to consider the implications of some of the points discussed in this report, and in particular to balance the legitimate proprietary interests of model vendors against the benefits of greater transparency. As noted above, improvements in practices related to vendor models, whether driven by vendors or by their clients, will help promote better credit risk management.