Airplanes are ships of the sky from an LGD perspective, but not from a PD perspective. Railcars are also ships of the land by the same token

From a PD perspective, they are generally separate. The maritime companies and airlines have very different obligor dynamics

From an LGD perspective, the main drivers are downtime (i.e. how long the asset sits idle after a default) and shortfall (i.e. the decrease in the new lease after a default since usually periods of default coincides with pressure on asset prices and lease rates), but not the value on the plane. It is seldom that you’d actually lose the asset in any meaningful way, so that does not affect the LGD. There are international treaties that lessors would seek the jurisdiction of the lessee be a part of, before sending an expensive plane over with a long lifetime left on the asset. The other jurisdictions get the older planes where the lessor does not care all that much, whether they get it back or not

To be clear, you’d parameterize the LGD model differently for different assets as the downtime and shortfall dynamics are different. But the model structure is the same. So, it becomes a bit of an optical choice on whether you call that a single model or not

The other C&I related criteria I have seen for segmentation (may or may not be captured by your size / sector view):

Rated / quoted vs. not (on the basis these firms have access to additional sources of funding plus an extra source of predictive info, although that can be reflected by other mechanisms) Specialised lending vs not (Basel has some rules re: when should be viewed as specialised – to some extent it comes down to legal form) Legal form e.g. limited liability vs. partnership vs sole trader (at bottom end) Leverage finance / recent transaction e.g. divestiture, M&A … (on the basis that they are more sensitive to changes and historical performance data may be less relevant

I think most have been said already, but just for the record I have also seen a PD master scale being validated by MRM (US).

I’d say as mentioned, the guiding principle should be how you derive the scale. If it involves mathematical and statistical methods, assumptions etc. then it most likely falls under the “model” definition and should be validated. I would tie it to how you define a model internally and what the MRM policy says about it

We’ve seen a variant of this issue in US/Canada, with the large Canadian Banks generally having IRB approval at the Group level (including for their main US loan books) and their US subsidiaries being on standardized

In this case, there is no incentive for the US entity to seek IRB approval - but US regulators do care about the risk rating systems from a bank supervision perspective, which has raised some of the same questions about whether group models are suitable. Those banks have generally taken a view aligned to a previous poster, i.e., using local models for middle market and below, and trying to align to group models for larger companies and FI's

On the last point, I'll say that we've seen US regulators challenge the support for those decisions heavily, but at least in some cases it seems to have stood up to that challenge.  In one case where the bank's prior analysis and documentation didn't provide great support, they are being pushed to redevelop those as well, though they are trying to do so in a way that they can ultimately extend back to group as well, for a C&I model that was getting a bit long in the tooth anyway

+1 for the Amazon model. Driving adoption is tough. Predicated on the issue that almost nobody reads slides before the meeting. Building in 5-7 minutes for content review while in the meeting can be effective even if it isn’t the rigid memo format at Amazon. For example, you can summarize key decision points up top, then a cascade of a few pages for all to review. Instead of driving to ideation this drives to outcomes

Thanks a lot, this was very helpful!

Might also be worth taking a look at the detail in sections 2.5.5.1 - 2.5.5.2 dealing with customer migrations and migration matrix stability respectively, in the ECB's Instructions for reporting the validation results of internal models, pgs. 23 - 24

I'll just flag that a Canadian bank who had decided to extend several of their existing "global" models to their US subsidiary based on similar logic, and while the logic makes sense broadly, they ran into significant issues with their US subsidiary’s regulator about the way they did it

I would primarily attribute the root causes of those issues to:

Operating model for how those decisions were taken and where they were reviewed and challenged – the Group development and validation teams led the substantive assessment of “fit-for-use”, and while there were some US stakeholders involved, the ones most involved were not very senior in stature, and often deferred to the expertise of Group.  But those US stakeholders then couldn’t / didn’t do a good job of credibly defending those decisions to their regulators, leading the regulators to question if US senior management had been sufficiently involved in determining that these models were appropriate for the US portfolio

The documentation they produced justifying and validating the use of these models in the US was a narrow “fit for use assessment”, which taken as a standalone artifact fell far short of the comprehensive model documentation and validation expectations of SR 11-7.  This led regulators to question the “effective challenge” provided by US model risk management and more broadly by US senior management

There are two steps:

(1) raw social data collection; and (2) analysis with NLP or GenAI

(1) is the difficult part. You would need to know where the discussions are happening for your client or its industry – is it in the news, Facebook, Instagram, a forum? Once you identified the data source, you would need to find a way to collect the data, and the legal and cost question comes in to play.

I recommend you to buy data from a third party (there are many of them). They can either provide you only the raw data feed (e.g. give me all the Facebook discussions about my brand), or they can provide you with a dashboard with standard analytics. If you only need one off data analysis for maybe a proposal, getting raw data and crunch it yourself is a cheaper, better way forward

I agree, but wonder if there is something more here.

I have also seen such “findings” and in the two cases I saw, they were politically/ career minded where once the MRM team was fighting a past battle (comments on how the bank uses resources, and ones they didn’t get) and once where the MRM team lead wanted the lead modeler’s job (and was trying to make a point about how they would use resources). In neither case were such “resource” use findings actually anchored in trying to do MRM work…

This may just be a rumour and so worth trying to Google to verify it, but I recall a long time ago that the MKMV model didn’t work well for banks, this was a known shortcoming of the model

It’s not at all clear what other type of structural model (other than something like KMV) one could even attempt for a bank…

A couple of thoughts on this subject, from one of our experts:

The discrepancy is caused by the adjustment implicitly assuming that a Bank would have had more defaults and lower scores (and so a worse average score) – while applying the theorem to a population which still has the same set of defaulted cases. This means the average scores are not worse, and hence you predicted PD will be lower.

There are at least two approaches to deal with this effect:

Adjust the constant term in the logistic until it hits the 2% target Run a “goal seek analysis" so that the average PD after mapping scores to the Bank grades, and applying the appropriate post-rating adjustments so the PD reaches 2%

Especially for European banks IRB models are actually required to be quite conservative unless Banks have "perfect" data, so the long-run average can become a moot point to a certain extent

On the topic of perfect data: if the Bank has enough data and the PD model is really powerful, it should find that there is no straight-line relationship between PD from logistic model vs. observed default rate. This is actually caused by the fact that whilst the errors are broadly normally distributed in logOdds space, when the distribution is converted to PD/default rate space the expectation will be closer to the mean than the original prediction.

For Canada specifically, please refer to the latest Draft Guideline E-23 on model risk management by OSFI (our banking and insurance regulator).

To capture the risk posed by AI, the Draft Guideline modernizes the definition of “model” in the original 2017 Guideline to explicitly include AI and machine learning methods. Basically the AI-based solutions are considered “models” and are subject to model risk management requirements. Detailed requirements can be found in the guideline

I suspect the answer will depend on what regulators require – for example the PRA expects all FIRB firms to use effective maturity (there’s currently a carve-out for SMEs, but they want to remove that too under Basel 3.1 – see below)

I have to admit, I cannot recollect what ECB/ EBA has had to say about this, but I think it would be pretty hard to justify using it in most places but not some – would very much feel like a bank would be open to challenge around whether it was cherry-picking

PRA CP16/22 proposal around Effective Maturity

4.305 The PRA currently specifies within IRB permissions that firms using the
FIRB approach must calculate effective maturity rather than apply fixed
parameters. This is because the PRA considers that calculation of effective
maturity is a more risk-sensitive approach, which better reflects the economic
substance of the exposures, and thus enhances the safety and soundness of firms.
Furthermore, using effective maturity facilitates effective competition because
firms using the AIRB approach are also required to apply the effective maturity
approach.

4.306 The PRA proposes to maintain the substance of its existing approach and
that firms using  the FIRB approach would continue to be required to apply the
effective maturity approach. The PRA proposes to include this provision in its
rules as it considers this would be more appropriate than applying the
requirement on a firm-by-firm basis as is currently the case.

4.307 The PRA considers that the proposed approach is in line with the Basel 3.1
standards as these include a discretion for national supervisors to require
firms using the FIRB approach to calculate effective maturity for all exposures.

4.308 Similarly, to improve risk-sensitivity, the PRA proposes to remove the
option currently setout in the CRR that allows firms that are otherwise
calculating maturity to instead apply fixed maturity values for exposures to
small UK corporates.

Hello RiskbOWl community,
Oliver Wyman recently conducted a Model Risk Management Survey with 10+ GSIBs participating (thank you to the onces who participated). One of the questions we asked in this survey was "what are your current priorities for MRM?". The top answers were:

100% said: Expand the scope of MRM (i.e. adding new model types such as AI but also more and more non-models entering MRM) 92% said: Increase resource productivity via simplification, streamlining, etc. 92% said: Increase usage of AI tools to support validators 66% said: Increase validation quality less frequent answers: reduce validation frequency, change to event-based validation, increase offshoring or outsourcing

What are your current strategic priorities? Where do you see current challenges?

A number of UK banks have their wholesale PD models produce both “TTC” and “PIT” ratings – with the latter typically being driven by some sort of adjustment based on Moody’s KMV EDF’s or equivalents – you also have a number of other banks that for IFRS 9 will apply macro-economic model outputs on top of a largely TTC rating model to produce a PIT PD for IFRS 9 provisioning/ allowances purposes. The dual rating approach at big UK banks was heavily driven by a couple of modellers by the names of Scott Aguais and Lawrence Forest, and they have published a few papers describing their approach.

On the question of splitting by “investment Grade” vs “non-investment grade”, I’ve never seen this, although a split between leveraged and non-leveraged is common historically (when we built the model for one of the large German banks, we were able to reintegrate them). But size r whether a customer is rated/ quoted/ listed is a common basis for segmentation in the commercial space.

For EU banks, qualitative questions still tend to be included, although people are working to make them objective where possible, but not sure many have entirely removed them (again Scott Aguais was wanting to do this for one of the big UK banks – not sure whether he succeeded) – the issue with qualitatives is whether you can find the sweet-spot – at bottom end SME, often I don’t think credit officers have any real insights into their customers given how many they cover; at top-end, question is whether the credit officer has real insight beyond what can be captured by tools such as Factiva Sentiment Signals

Historically, it has been difficult to remove size segmentation from the entire corporate customer base, in part because you tend to see some factors have different relationships for small vs large firms (e.g. you might want a fair bit of cash on balance sheet of an SME, but for a large firm, this would be inefficient and you’d perhaps worry if management wasn’t trying to make their cash work hard )

The short answer is yes they should definitely be included. The group risk report is the bare minimum that should be included for a GSIB in terms of internal reports and you can go from there

Will to share some pages on the latest interpretation of the scope of BCBS239 by the ECB. A bit more expansive that the traditional industry approach but that is really going to be the new bar. We also have a full BCBS239 benchmarking database that covers 20+ banks (including most European GSIBs) and all dimensions of BCBS239 so maybe your client wants to participate in it

I have seen the following:

Percentage of the Business/Institutional portfolio in high transition risk sectors Proportion of the mortgage portfolio exposed to high physical risks (by 2050 under a 4-degree warming scenario is one specific example). Believe this is based on property level assessment and then some % increase in PD. Some reputational ones around ESG scores

However, I don’t believe anyone would set the thresholds at a level that would likely be binding. So skeptically, I think this is just for reporting and transparency at the moment – which is probably right given the limitations of climate risk modeling

There is certainly precedent for this in loss forecasting, given various companies that need to follow both IFRS9 and CECL at different legal entity levels, and/or to follow different stress testing guidance for different regulators.   I can’t think of a case where I’ve seen it for the primary credit risk rating models however (at least not for literally the same exposures receiving two different ratings)

Very interesting point. Would be great to know OW's experience with feedback from other players.