AI governance challenge ahead, warns Davies
Davies has published a new white paper, ‘How AI Agents Are Amplifying Conduct Risk in Insurance’, part of the AI Impact Series, exploring how the rise of autonomous AI agents is reshaping regulatory expectations across the insurance sector.
According to the company the next wave of AI adoption is autonomous AI agents that can make decisions, trigger actions and influence customer outcomes at scale. With far less human intervention, a significant governance challenge is emerging.
The white paper says that insurers are rapidly deploying autonomous AI agents but may be exposing themselves to conduct and governance risks.
Davies commented: “Compliance controls rebuilt for traditional AI may be failing or inadequate for autonomous AI. In our white paper, we explore why the rise of agentic AI is forcing insurers to rethink governance, accountability and risk management from the ground up.
“The central finding is that while there is no regulatory gap for AI, there is a growing governance gap. Existing frameworks including the FCA’s Consumer Duty, GDPR, Senior Managers & Certification Regime (SM&CR) and the Consumer Protection Code 2025 already apply to AI-driven decisions, but autonomous AI systems introduce new challenges around oversight, explainability, auditability and ongoing control.”
The report argues that insurers can no longer rely on pre-deployment testing and one-off compliance approvals. Instead, AI agents must be governed as living systems, requiring continuous monitoring, real-time human controls and lifecycle-based compliance throughout design, deployment, operation, retraining and retirement.
It also examines several technical and operational issues becoming increasingly relevant for insurers, including:
• How autonomous decision loops can create unintended outcomes at scale.
• Why model drift, data drift and adaptive learning create ongoing compliance risk.
• The need for meaningful human oversight rather than passive approval processes.
• Governance challenges associated with third-party AI models, cloud providers and vendor ecosystems.
• Why the EU AI Act is increasingly acting as a benchmark for AI governance, even for UK insurers.
To read the white paper contact Davies.
Did you get value from this story? Sign up to our free daily newsletters and get stories like this sent straight to your inbox.
