Qualitative rating factors
-
Dear RiskBowl,
One of the motivations of a bank for redeveloping their IRB models is that they heavily use qualitative factors in most, if not all, their rating models currently.
Is it correct that most of new model builds don’t rely on qualitative factors anymore?
If so, do you have experience you can share on the model change process (how difficult was it to convince the regulator, did you propose this change as part of a complete re-build or did you also see a narrower model change only focused on the qualitative factors, …?)
Thanks
-
In my last discussions on supervisory side the tendency was a bit less of a ‘try to get rid of qualitative’, but more of a ‘make sure the quantitative have enough weight’. This then indeed can lead to substantial criticism on overly-qualitative approaches, though.
Of course mileage varies in retail vs. corporate and for complex vs. vanilla portfolios
-
To your first question, while regulators are preferring quantitative factors driving the outcome of the model, they minimally still expect banks to include qualitative candidate factors in their factor analysis
The factor selection should then be primarily driven by the statistical performance of the variables but overlaid with sufficient expert judgment to ensure that the final variables are intuitive and complete. E.g., if a qual factor like “Investor diversity” did not make the statistical cut but experts were of the opinion that it is a critical predictive factor, then it should still be overridden into the model (and vice versa)
-
I’m sure there are some differences in focus between US and European regulators on this, but for what it’s worth, we have seen US regulators question or give pointed feedback on qualitative factors and overrides in several places recently, generally with the following as key concerns:
- Heavy reliance on subjective judgement (whether through overrides or highly subjective qualitative factors) can lead to a couple of issues
- Ratings that are not easily reproducible by a third party (internal credit review teams + the examiners themselves). If this results in rating discrepancies, examiners tend directly infer that there are issues with the accuracy of the ratings themselves.
- Rating processes that are more cumbersome and time-consuming, leading to challenges re-rating credits in a timely manner under stress
- Lack of transparency or empirical support for the impact of qualitative factors. Some US banks incorporate their qualitative factors through notching logic, which may avoid classifying the qualitative module as a model and can be a simple intuitive way to deal with rare but important factors. But when you apply that kind of approach to a qualitative module with several common factors, notching for the outputs of such a qualitative score tends to be more complex and less transparent than taking a simple linear combination of scores. We’ve seen several banks get questioned over that, and if there were other issues with the rating system, criticized more heavily for it.
Ultimately neither line of criticism has tended to assert that qualitative factors should be removed entirely, but we have seen regulators recommend or require:
-
Rationalizing the set of qualitative factors used, generally looking to reduce their number and demonstrate clearly that any remaining factors contribute to improving accuracy of the ratings
-
Providing further justification for the methods by which qualitative factors are incorporated, including any notching or weighting
-
Revisiting qualitative factor definitions or associated guidelines to limit the degree of subjectivity required, and make them more reproducible across different analysts
-
Tightening the monitoring of override rates, so that if e.g. a portfolio is regularly seeing 20%+ override rates, modeling and credit risk management teams both recognize it as a problem, and collaborate to diagnose and bring them down
-
While I recognize a lot of these points, I do think that we should not let the tail wag the dog
If the non-financial factors add predictive power, I don’t think there is any reason on a first principles basis to categorically exclude them. But of course, I do appreciate that these kind of factors can be subjective and therefore of lower quality, so we should keep an eye on that and encourage the clients to improve data quality
Also, many banks lump treatment of these kind of factors with overrides, which is almost always where the supervisory feedback is coming from. It is commonly used as a fudge factor, and that is poor practice. One can develop a disciplined, (high-quality) data based use of this type of information to avoid that pitfall