Artificial intelligence changes not only how insurance is bought but “decision authority and governance”, former Hollard tech chief Jamie Smith warns.

In a webinar hosted by insurtech Kanopi last week, Mr Smith said AI “amplifies” failures discovered only at claim time because more decisions are made “upstream, faster and often without visibility into the reasoning behind them”.

Insurance product decisions made behind the scenes by AI agents “don’t disappear”.

“They reappear later, at claim time or when there’s a complaint or dispute, and that’s when accountability becomes real,” Mr Smith said.

“From a customer’s perspective, there’s not much distinction. If something is interpreting a situation and recommending a policy, it feels a lot like advice. The challenge is we’re not always treating it the same in terms of who is responsible.

“Ownership is often assumed inside insurance companies, but with AI it needs to be explicit.”

He warned against assuming faster is always better, saying when decisions are made quickly at scale there is no “theory of consequence unless you’ve thought about it up front”.

Decisions made quickly “lose some of the natural friction where judgment sits ... That friction was doing more work than we realised. We spent years trying to remove it, but sometimes friction is what gives you control.

“You need to deliberately choose where you want straight-through processing and where you need checkpoints or escalation.”

Mr Smith said introducing AI requires the right people, processes, technology and governance.

“It’s about decision architecture, how work flows through your systems. Who decides where it goes next, who is ultimately accountable, and importantly, where should you intentionally insert friction so a human can check something – or where should there be no AI at all, relying purely on human judgment?”

AI systems “don’t show the working out ... lots of organisations have frameworks, policies and controls that look great on paper, but they only really get proven under stress”.

Claims teams are “dealing with the consequences of decisions made much earlier in the journey – product design, distribution choices, AI recommendations”, he said.

Claims team work will shift toward “resolving gaps between what was sold and what was expected, often without a clear record of how the original decisions were made,” Mr Smith says. “That’s where trust gets reinforced or broken.”