9th of October Oliver Wyman MRM/AI Roundtable Discussion Group
-
Hi all,
Welcome to the channel for the Model Risk & AI Roundtable. We will use this channel to gather anonymous questions and keep the conversation going after the event.
We will be collating agenda and follow-up on this community channel.
Please post your anonymous questions here.
-
Some very interesting questions being sent via email by some of our attendees:
What are the general approaches to AI adoption and subsequent model risk management taken by the peer banks? Have you had feedback from the regulators on your approach?
Have peers received thematic feedback from the PRA on their SS1/23 compliance approaches?
How do peers risk-tier their models, and what is the frequency/depth of monitoring/validation for each tier?
Are peers leveraging AI/LLM for their internal MRM activities? e.g. report writing
What metrics are peers using to gauge performance of AI models?
-
More questions trickling in from our attendees:
Questions on DQMs
- Progress and Identification
- How far along is your firm in identifying material and complex deterministic quantitative methods (DQMs), as outlined in PRA SS1/23, and onboarding them into your model inventory?
- Review and Validation Practices
- Are you currently conducting any form of review or light validation of these onboarded DQMs? If so, what level of depth are these reviews reaching, and how are you prioritising them?
- Resourcing and Execution
- Are these DQM reviews being handled by your internal model validation teams, or are you outsourcing this work? What factors influenced your approach?
- Capacity and Resource Allocation
- How are you managing internal resource capacity to accommodate DQM reviews alongside traditional model validations? Have you had to make trade-offs or adjustments?
- Compliance and Readiness
- On a scale from early-stage to fully compliant, how would you assess your firm’s current alignment with the DQM-related expectations under SS1/23? What challenges have you encountered in meeting these requirements?
Questions on AI / ML Validation
How are firms adapting their model validation frameworks to address the unique challenges of AI and ML models, particularly around explainability, stability, and governance?
Follow-up to that is : Are traditional validation techniques sufficient, or are you developing new tools and metrics specifically for AI/ML?Join us for a morning full of insights!
-
How are banks classifying the use of AI agents and are Foundational models with configuration (e.g. chunking, changing parameters like temperature) treated as models or tools?
-
Are banks considering model risk as a Level 1 risk or as a Level 2 risk within the risk taxonomy?