Risky Business #2: What do IRB model implementation and a house party have in common?
-
Dear reader, welcome back to Risky Business. This time around we have a guest writer, Greg Wiltshire, Director, who poses a very interesting question, can your amazing next party turn out akin to an IRB model implementation? Or will the next IRB implementation you undertake like a true party?
Will the party be a success? Or the implementation a failure? Hopefully the other way around!I hope you enjoy the insights from the below as much as I did.
Matias Coggiola
-
Developing a CRDIV model is like hosting a dazzling party*. The development phase buzzes with excitement—teams collaborating, ideas flowing, and progress celebrated. Each milestone feels like a defining moment in the revelry, driving the project forward. But then comes the inevitable hangover and clean up - implementation. With the party over and the lights back on, reality sets in as the hard work begins; rolling out the model, building data pipelines, and fine-tuning monitoring and compliance. It’s less glamorous and more gruelling - but equally critical.
Banks face several challenges in implementing their CRDIV approaches. Not leaning into these can create operational drag, where tactical compromises stack on top of each other, disproportionally sucking up resources and eroding the space and potential to break free and redefine ways of working. A red flag we see across the industry is a lack of planning for the implementation phase, with some banks expecting to rely on their development data and code for the foreseeable future.
Some key post approval challenges include:
Productionising the model
Change management: Introducing new models requires process and workflow changes. Banks need to see implementation as inseparable from development, keeping pace and pushing through the required change.
Integration with legacy systems: Many banks operate on legacy systems that may not be compatible with new modelling techniques or technologies. If the model was developed in python, are you really going to implement in SAS even if that’s where the other prudential models are, or are you going to invest in a python-based production capability?
Resource constraints: Implementing new models often requires specialized skills and expertise. After regulatory approval it’s natural for large development teams to move onto new projects, but retaining expertise is critical to ensure successful implementation and avoid code rot.
Maintaining the code
Reusability: The development code is a hugely valuable asset, but lack of clear ownership can limit the ability to reuse and repurpose code. This can lead to duplicated effort and wasted resource, as teams reinvent the wheel instead of leveraging an existing codebase.
Inconsistency in code quality: Banks often lack clear coding standard. So as their codebase evolves, different teams may use various libraries or approaches, leading to inconsistencies in code quality and functionality, and giving rise to bugs and errors that are difficult to find and fix.
Monitoring
Planning for model drift: New models start out accurate. But over time, they become less accurate as new data is available and credit policies change. Detecting and addressing model drift not only requires ongoing monitoring and validation but also planning for possible responses.
Data quality and availability: Effective model monitoring relies on high-quality data. Banks often struggle to build monitoring on top of their production data, instead preferring to maintain offline copies. As a result, monitoring can be inconsistent, incomplete and out of date, all of which can hinder accurate model assessment.
The implication is clear, banks who have successfully been through CRDIV approval have overcome a significant hurdle, but it’s only a single step and not the time to let up. There’s an opportunity to use CRDIV to transform the wider risk management, especially for banks who invested in technology as part of their journey - modern infrastructure not only supports modelling but also integration, consistency and efficiency across the model lifecycle.
So, let’s get the black bin bags out from under the kitchen sink and have a bit of a tidy up.
*Please note that individual experience may vary
Reach out to greg.wiltshire@oliverwyman.com to hear more about how Oliver Wyman can support your IRB model implementation
-
Greg Wiltshire is a Director in Oliver Wyman's Risk Delivery Team. Greg brings over 15 years of risk modelling and analytics experience, specialising in risk and technology transformations.
Matias Coggiola is a Manager at Oliver Wyman and specialises in Credit Risk modelling methodology and regulatory compliance. He curates the monthly Risky Business series.