AI-written complaints too long and often wrong, ombudsman says
An insurance dispute resolution scheme has warned consumers’ use of AI is producing overly long complaint submissions that contain misleading information.
Policyholders who use artificial intelligence tools to help them submit complaints need to check information against trusted sources, because some advice can be wrong, New Zealand’s Insurance & Financial Services Ombudsman Scheme says.
“We’ve seen complaints which are 300 pages long, but more words aren’t necessarily better,” ombudsman Karen Stevens said.
“Clear information about what has gone wrong for someone is much more useful than multiple pages referencing legislation and case law.”
IFSO told insuranceNEWS.com.au the growing use of tools such as ChatGPT is resulting in cases taking longer to review.
Case managers must dig through AI-generated verbiage to find the real issue, and consumers can be left frustrated when outcomes do not match what the AI told them to expect.
IFSO has noted an increase in complaint numbers, which is “possibly because [AI] is making it easier for people to complain, but it could also be because consumers are being told by AI they will succeed”.
In one case, a Google AI summary suggested insurance claim decisions are “frequently overturned” when consumers complain and up to 90% of cases “can result in success if people persist”.
Ms Stevens says that advice is misleading: complaints that are escalated are often complex and stressful, and are not always resolved in the consumer’s favour.
“Many people come to us hoping we can put things right but, unfortunately, if the law or the contract doesn’t support their position, we can’t change that even when the outcome feels unfair.”
IFSO says using AI to draft complaints can “lead to confusion about what the heart of the issue is for the consumer”.
Ms Stevens added: “It can sometimes oversimplify complex policy wording, miss key exclusions and rely on overseas information that doesn’t apply.
“We’ve also seen instances of AI hallucinating previous cases and using them as examples. This can result in it giving incorrect advice.”
Insurers are advised to educate consumers on being cautious when relying on AI, and to provide guidance on how to best use it.
From the latest Insurance News magazine: The industry's tech experts discuss their hopes and fears as the AI revolution gathers pace