Airplanes are ships of the sky from an LGD perspective, but not from a PD perspective. Railcars are also ships of the land by the same token

From a PD perspective, they are generally separate. The maritime companies and airlines have very different obligor dynamics

From an LGD perspective, the main drivers are downtime (i.e. how long the asset sits idle after a default) and shortfall (i.e. the decrease in the new lease after a default since usually periods of default coincides with pressure on asset prices and lease rates), but not the value on the plane. It is seldom that you’d actually lose the asset in any meaningful way, so that does not affect the LGD. There are international treaties that lessors would seek the jurisdiction of the lessee be a part of, before sending an expensive plane over with a long lifetime left on the asset. The other jurisdictions get the older planes where the lessor does not care all that much, whether they get it back or not

To be clear, you’d parameterize the LGD model differently for different assets as the downtime and shortfall dynamics are different. But the model structure is the same. So, it becomes a bit of an optical choice on whether you call that a single model or not

While I recognize a lot of these points, I do think that we should not let the tail wag the dog

If the non-financial factors add predictive power, I don’t think there is any reason on a first principles basis to categorically exclude them. But of course, I do appreciate that these kind of factors can be subjective and therefore of lower quality, so we should keep an eye on that and encourage the clients to improve data quality

Also, many banks lump treatment of these kind of factors with overrides, which is almost always where the supervisory feedback is coming from. It is commonly used as a fudge factor, and that is poor practice. One can develop a disciplined, (high-quality) data based use of this type of information to avoid that pitfall

The other C&I related criteria I have seen for segmentation (may or may not be captured by your size / sector view):

Rated / quoted vs. not (on the basis these firms have access to additional sources of funding plus an extra source of predictive info, although that can be reflected by other mechanisms) Specialised lending vs not (Basel has some rules re: when should be viewed as specialised – to some extent it comes down to legal form) Legal form e.g. limited liability vs. partnership vs sole trader (at bottom end) Leverage finance / recent transaction e.g. divestiture, M&A … (on the basis that they are more sensitive to changes and historical performance data may be less relevant

@OP

In which case,

No change in default level – new Soft UTP is not adding any defaults (there must be fully correlated to any existing trigger or triggers e.g. 90DPD, bankruptcy, etc.)

No timing difference of default event – therefore no PD or LGD impact even due to discounting

Client has only limited historic time series to show proof

Both 1 and 2 indicate there is full correlation to already existing triggers at the existing time period (but level of correlation can differ at time (t) for each trigger toward the new Soft UTP trigger).

Based on the other responses to this question, a bootstrapping would be the correct approach, but the bank states on recent periods the outcome should be NULL (or similar like that)

Therefore, an idea could be to simulate the uncertainty of the correlation in different macroeconomic environments

Correlation analysis of macro economic factor vs existing triggers over time (t) Given that new Soft UTP trigger is correlated toward multiple triggers at time (t) (based on existing time period the bank has), the correlation variation found of each underlying existing trigger can be used to proxy new Soft UTP trigger back in time

PLUS above should still be supplemented with a qualitative statement

I think most have been said already, but just for the record I have also seen a PD master scale being validated by MRM (US).

I’d say as mentioned, the guiding principle should be how you derive the scale. If it involves mathematical and statistical methods, assumptions etc. then it most likely falls under the “model” definition and should be validated. I would tie it to how you define a model internally and what the MRM policy says about it

Lots of good answers here.

One tough learning from implementing this at scale is that unlike traditional ML, automated tests can only capture a small fraction of what can go wrong with GenAI. While the automated validation is necessary, it is not sufficient.

We have typically needed to also develop large manual testing protocols for releases, where humans (either developers or a set of test users), attempts a mixed of predefined and new prompts, and judge the quality of the answers. Often we will uncover “issues” that are very subjective, such as the answers technically being correct but pulling from different files that we wished, or answers being less/more detailed than the average user prefers, or an entirely new file format having issues (hence not covered by tests yet), or a million other things!

For one of our recent clients, we ran “hackathons” along with releases where both new and power users would try various prompts and score the output. It was incredibly helpful to identify things the tests had failed to see

We’ve seen a variant of this issue in US/Canada, with the large Canadian Banks generally having IRB approval at the Group level (including for their main US loan books) and their US subsidiaries being on standardized

In this case, there is no incentive for the US entity to seek IRB approval - but US regulators do care about the risk rating systems from a bank supervision perspective, which has raised some of the same questions about whether group models are suitable. Those banks have generally taken a view aligned to a previous poster, i.e., using local models for middle market and below, and trying to align to group models for larger companies and FI's

On the last point, I'll say that we've seen US regulators challenge the support for those decisions heavily, but at least in some cases it seems to have stood up to that challenge.  In one case where the bank's prior analysis and documentation didn't provide great support, they are being pushed to redevelop those as well, though they are trying to do so in a way that they can ultimately extend back to group as well, for a C&I model that was getting a bit long in the tooth anyway

+1 for the Amazon model. Driving adoption is tough. Predicated on the issue that almost nobody reads slides before the meeting. Building in 5-7 minutes for content review while in the meeting can be effective even if it isn’t the rigid memo format at Amazon. For example, you can summarize key decision points up top, then a cascade of a few pages for all to review. Instead of driving to ideation this drives to outcomes

I think most use a blended approach, with the largest / riskiest LPs getting a deeper review and others being rated based on higher level characteristics. LPs who are too risky and/or cant be rated should get kicked out of the borrowing base. NB: I don’t think LP risk is the only criteria, and similar to a reply above, Fund and GP factors are also really important (if not more so). Given the lack of historical defaults, it’s important to have a good conceptual understanding of what could drive defaults… which varies based on the type of LP, the legal frameworks for the LP’s commitments, and the likelihood of things going sideways for the fund/GP.

Worth skimming the characteristics the ratings agencies use to assess sub lines for some additional color. (portfolios of sub lines get rated both for distribution to US Life Insurers, who rely on IG ratings, and for other distribution mechanisms). Summarized nicely by KBRA below

Subscription Lines: Why Are Ratings Needed?
https://www.kbra.com/publications/KKbkyMsj/subscription-lines-why-are-ratings-needed?format=file

Subscription Finance Rating Criteria [Fitch]
https://www.fitchratings.com/research/fund-asset-managers/subscription-finance-rating-criteria-08-06-2023

Methodology For Rating Subscription Lines Secured By Capital Commitments [S&P]
https://disclosure.spglobal.com/ratings/en/regulatory/article/-/view/sourceId/13199000

Subscription Credit Facilities: Proposed methodology [Moodys]
https://ratings.moodys.com/api/rmc-documents/410678

I would also like to learn more about this

Thanks a lot, this was very helpful!

If you have any questions or comments please reply to this thread!

For our Fed remediation plan for Sanctions, Audit was key part of the remediation plan. We submitted a full Audit “TOM” including resource model, training, risk assessment, audit testing program, and senior mgmt. reporting. There was also a layered sign off for smaller vs. golden milestones, where Audit got involved to provide design assurance vs. operational effectiveness.

A noteworthy oddity potentially affecting the choice of modelling approach: In the slotting approach, the risk mitigating effect of ECA coverage cannot be recognised. I.e. for transactions with ECA coverage, there is a significant over-estimation of risk in the slotting approach. On the other hand, FIRB and AIRB as well as the standardized approach do allow for consideration of ECA coverage.

I agree, but wonder if there is something more here.

I have also seen such “findings” and in the two cases I saw, they were politically/ career minded where once the MRM team was fighting a past battle (comments on how the bank uses resources, and ones they didn’t get) and once where the MRM team lead wanted the lead modeler’s job (and was trying to make a point about how they would use resources). In neither case were such “resource” use findings actually anchored in trying to do MRM work…

This may just be a rumour and so worth trying to Google to verify it, but I recall a long time ago that the MKMV model didn’t work well for banks, this was a known shortcoming of the model

It’s not at all clear what other type of structural model (other than something like KMV) one could even attempt for a bank…

Greg Wiltshire is a Director in Oliver Wyman's Risk Delivery Team. Greg brings over 15 years of risk modelling and analytics experience, specialising in risk and technology transformations.

Matias Coggiola is a Manager at Oliver Wyman and specialises in Credit Risk modelling methodology and regulatory compliance. He curates the monthly Risky Business series.

The ORX global reference taxonomy was developed based on 60+ risk taxonomies used by financial institutions around the globe is probably the best representation of peer practices and has been adopted, with tailoring for the specific organization, by many since the taxonomy was developed ~5 years ago.

Within the ORX global reference taxonomy, regulatory compliance is a separate L2 category within the NFR taxonomy, and is defined as “the failure to comply with any legal or regulatory obligations that are not captured through other Level 1 risks within the NFR taxonomy”, because the risk of non-compliance with specific legal or regulatory obligations is relevant to most Level 1 risks in the NFR taxonomy and therefore we wanted to avoid overlap with these risks

Might also be worth taking a look at the detail in sections 2.5.5.1 - 2.5.5.2 dealing with customer migrations and migration matrix stability respectively, in the ECB's Instructions for reporting the validation results of internal models, pgs. 23 - 24

I'll just flag that a Canadian bank who had decided to extend several of their existing "global" models to their US subsidiary based on similar logic, and while the logic makes sense broadly, they ran into significant issues with their US subsidiary’s regulator about the way they did it

I would primarily attribute the root causes of those issues to:

Operating model for how those decisions were taken and where they were reviewed and challenged – the Group development and validation teams led the substantive assessment of “fit-for-use”, and while there were some US stakeholders involved, the ones most involved were not very senior in stature, and often deferred to the expertise of Group.  But those US stakeholders then couldn’t / didn’t do a good job of credibly defending those decisions to their regulators, leading the regulators to question if US senior management had been sufficiently involved in determining that these models were appropriate for the US portfolio

The documentation they produced justifying and validating the use of these models in the US was a narrow “fit for use assessment”, which taken as a standalone artifact fell far short of the comprehensive model documentation and validation expectations of SR 11-7.  This led regulators to question the “effective challenge” provided by US model risk management and more broadly by US senior management