Modelling of the Future - How to build sustainable and integrated modelling capabilities

[User 894]

Over the last years, banks have increasingly found themselves allocating a significant portion of their risk budgets to Internal Ratings-Based (IRB) remediation programmes. Part of the problem for some banks has been time pressure to build the models that resulted in tactical responses that gets them to the regulatory submission as fast as possible. In the various workshops we had with clients, the issues that teams raised were worryingly similar across institutions -

• Models built from scratch, duplicate efforts across teams working on different portfolios (e.g., Mortgages and Credit cards)

• Hard-coded and brittle model architecture makes it difficult to adopt to changing business and regulatory requirements

• Reliance on legacy tools (Excel, SAS) hinders building modelling pipelines and implementation speed

• Manual processes in data preparation, model documentation and validation increase error risk, reduce quality

However, banks which demanded more from their analytical teams have invested more into getting the programme right to get more out of it. The question they asked was "How will investment help us to develop analytics in an efficient and effective way in the future?". We have worked with UK and EU banks to develop effective approaches to build sustainable and integrated modelling capabilities that span the entire lifecycle

Figure 1: Summary of efficiency levers across the model development cycle

Three key focus areas of banks have been the following

1. New data operating model for the regulatory models
In the past, modelling and analytical teams were usually responsible for sourcing and preparing the data alongside their full-time modelling job. Given the scale of the regulatory requirements for data for IRB modelling (i.e., need to go back to 1990s downturn), this took significant time and effort from modelling teams. Additionally, modelling teams do not always possess data engineering skillset, which can lead to sub-optimal quality of the datasets produced making them not reusable going forward.

With the new data operating model we observe Data Engineers leading the data sourcing process and the new Data Development team acting as an intermediary between the two teams. This approach allows not only for better quality data which results in more robust models but also creates strategic data assets to ensure replicability of the process in the future and makes implementation of the models easier.

Figure 2: Revised way of working between business and data engineers

2. Development of a centralized toolkit
When working with the UK banks of IRB remediation programmes, we have brought in the centralized modelling and engineering teams that have created reusable modularized codebases. We have also worked with the internal validations teams to use this as an opportunity to centralize the 2nd LoD effort (from a model risk perspective) – for example, if certain modules of the codebase (e.g., linear regression, SFA, MFA) are centrally reviewed and approved by the internal validation team, this reduces the time required for review and validation of each individual model. This is resulting in significant gains in end-to-end model development timelines and avoiding the risk of low quality / inconsistent analytics code. Additionally, this centralized codebase can be deployed to a broad range of modelling and analytical projects beyond IRB work (e.g., IFRS9, Credit decisioning).

There are still challenges in maintaining this codebase as a continuous operating model. Industry should be open to thinking of novel ways to get this challenge right, that could bring in cost savings, the ability to embed new technologies (like AI) with less friction and better analytics outcomes (e.g., analytics utilities, centralized services).

3. Implementation of the models
As a result of the challenges described above, we observe model implementation costs increasing and the speed of adoption decreasing over recent years. However, developing strategic data assets (as described in 1) and centralized modular model codebase (as described in 2) makes model implementation and deployment faster and more cost-efficient. Additionally, as proven by our work with the UK banks, using these approaches allows the reduction of ongoing maintenance costs (e.g., Cloud storage and data querying costs) significantly.