GenAI Avatar Validation: A Banking Model Risk Approach – Risk.net
Commerzbank has deployed a generative AI-powered avatar, named Ava, for customer service, a move that necessitates a modern approach to model validation, according to research published in the Journal of Operational Risk. Traditional model validation frameworks are insufficient for assessing the risks associated with generative AI due to its inherent opacity and probabilistic outputs.
The study details a systematic framework of “guardrails” designed to ensure trustworthiness in Ava, focusing on human oversight, fairness, transparency, and reliability. This framework emphasizes rigorous testing, real-time monitoring, scenario assessment, and effective governance. The research highlights the increased residual risks associated with vendor-provided GenAI systems, stemming from their complexity and lack of transparency.
The development comes as financial institutions increasingly explore the potential of generative AI to enhance customer service and streamline operations. Barclays recently harnessed GenAI to deliver what it describes as “world-class customer service,” and HSBC has deepened its AI push through a partnership with Mistral AI, according to reports from Financial IT and Banking Exchange respectively. These initiatives, like Commerzbank’s Ava, aim to leverage AI’s capabilities while navigating the challenges of risk management and regulatory compliance.
The Journal of Operational Risk study emphasizes that model risk management is crucial for building trust in AI-based virtual assistants. The framework outlined in the research aligns theoretical principles with practical processes, including lifecycle management and risk scenario assessment. The validation approach detailed in the study provides insights into achieving trustworthy artificial intelligence in highly regulated sectors, balancing innovation with the need for compliance.
The findings underscore a growing industry awareness of the need for robust validation procedures as AI adoption accelerates. A recent report highlighted by Google News emphasizes the importance of earning trust in the age of generative AI, particularly within the FinTech sector. PwC research, similarly reported by Google News, details how AI is reshaping the banking industry, noting the need for careful consideration of risk and regulation.