Should Model Risk /Validation care about cost?
-
Quick question: do we see any model risk / validation teams taking the costs of models / data into account? Or is the view that this is a call for the business and provided methodology is okay, and model performance is good enough, if business feels it is worth / not worth paying extra for a model, that’s fine?
Wondering in the specific context of LLMs where one could choose to eg deploy a distilled model vs full scale model and wondering whether anyone has seen Model Risk critique use of full scale model as unnecessarily expensive - how question applies to other models and could go the other way of “you should absolutely we used X even though it would cost more”
-
From a first principles perspective, I think the answer is no. MRM decides if the model is acceptable or not, and that is regardless of the cost. MRM should not focus on whether something else is better, which I have seen is what creates the problem you're describing.
In practice though the answer is implicitly yes. Where they draw the line for acceptability implicitly includes a view of the costs.
-
I agree, but wonder if there is something more here.
I have also seen such “findings” and in the two cases I saw, they were politically/ career minded where once the MRM team was fighting a past battle (comments on how the bank uses resources, and ones they didn’t get) and once where the MRM team lead wanted the lead modeler’s job (and was trying to make a point about how they would use resources). In neither case were such “resource” use findings actually anchored in trying to do MRM work…