By Ron Arnold, founder 11eight

Almost all the conversation about insurance and AI focuses on how insurers can use it to price, underwrite, manage claims, detect fraud and so on.

And there is no shortage of frameworks for making AI “safe” to use, not just in insurance but everywhere. But there is very little discussion on how to insure AI.

It seems to be assumed that insurers will cover it. But the signals suggest otherwise.

Many questions around making the use of AI “insurable” remain open: whether the frameworks can be implemented and are effective; whether clear lines of liability can be established; where liability sits when things go wrong; and how much information is needed to understand AI risk well enough to underwrite and price it.

Regulators are setting clear expectations. The Australian Prudential Regulation Authority and the Australian Securities and Investments Commission recently wrote to boards, putting them on notice that AI governance is theirs to own. APRA warned that “assurance practices are not keeping pace with the scale, speed and complexity of AI adoption”.

That pace APRA referred to is a real challenge for insurers: AI is moving fast. Standards, regulation, boards, controls and underwriting are not moving as quickly, and the gap between what AI systems can do and what we can govern and manage may be widening, not narrowing.

It should be no surprise, then, that insurers are reassessing their AI insurance exposures, and cover is being withdrawn.

Why AI strains the insurance model

AI failures are already here. Last year, Deloitte refunded part of a $439,000 fee for a government report after a generative AI system provided fabricated references and an invented quote attributed to a Federal Court judge. Australian courts have separately sanctioned lawyers for filing submissions built on AI-invented case citations. In each case, professionals relied on AI, and the liability was theirs.

The risk increases as AI moves beyond drafting material and begins acting as an “autonomous agent”. Non-deterministic autonomous agents can deliver varied outputs or actions. This can make consistent execution and assurance difficult. There is also a risk that an agent progressively departs from its intended task while still appearing to operate normally.

Human oversight remains important but is unlikely to safeguard against subtle deviations, as they can be difficult to identify, particularly where the volume of agent actions is large. An agent's erroneous financial, customer, compliance, regulatory or legal decisions can accumulate undetected, for example, underpayments, overpayments, misstated reporting, and the wrong outcomes for customers, claimants and suppliers. These failures can form the basis of a dispute, regulatory action or litigation.

Gallagher Re notes AI liabilities – from inaccurate or fabricated outputs to biased decisions, model drift and flawed training data – “are often not clearly covered under standard policies”. AI losses may not arrive looking like “AI claims”. They may look like professional indemnity, cyber, directors and officers, errors and omissions, contractual or product claims, with AI buried somewhere in the causal chain.

The liability vacuum

AI liability is a legal grey area. An Australian man asked his personal AI agent to book a gym class. The agent exploited a gap in the booking software and, unprompted, bumped another member off the waiting list to move him up the queue, in what the ABC called the first known Australian autonomous cyberattack.

A third party incurred a “loss”, but who is responsible?

“Software is not a legal person. Only a legal person can be liable at law,” technology lawyer Hayden Delaney says.

Responsibility might rest with the user, the developer of the agent software, the developer of the model, or the operator of the system it exploited. Mr Delaney told the ABC it is “the unknown area of liability in Australia that we’re facing right now”.

For insurers, this is critical, because clarity about who is responsible for an autonomous agent’s actions informs what type of policy applies, who needs what cover and what for, the terms and conditions of that policy, how to underwrite and price it, and whose policy should respond.

The scale of the problem

Even where responsibility is clear, the potential scale is severe. These systems lean on a handful of common foundational models: three providers account for more than four-fifths of AI deployments. So a defect in any one, as Gallagher Re points out, need not stay contained to one business, and faults in widely adopted systems “could trigger claims across multiple sectors simultaneously”.

The combination of uncertainty, concentration and potential contagion challenges a core mechanism of insurance: diversification. Insurance works when losses are sufficiently independent across policyholders, allowing premiums from the many to cover claims from the few. But a hidden defect accumulating within a widely used foundational model undermines the risk-spreading mechanism – creating losses across many businesses, sectors and policies at much the same time. A risk that is systemic, highly correlated and capable of producing simultaneous large-scale losses is not simply difficult to price – it can become effectively uninsurable.

The American warning

The US market is already moving to exclude, limit or separately price this risk. State regulators have approved more than 80% of insurers’ requests to strip AI from standard corporate coverage, with Berkshire Hathaway, Chubb and Travelers among those cleared to do so. Verisk’s ISO arm has released standard endorsements that allow carriers to exclude certain generative AI claims from general liability coverage. Carriers are no longer willing to carry AI liability unpriced inside legacy wordings.

Australia has not moved this far – at least not yet – but AI exclusions are starting to appear. Law firm Landers & Rogers recently reported that “we are now seeing AI exclusions starting to appear in certain types of policy wordings”, and it warns that insurers and regulators are signalling AI governance failures may be treated as foreseeable and uninsured, rather than accidental.

The message is clear: governments, businesses and consumers cannot assume insurance will cover the actions and impacts of AI.

Getting on the front foot

Australian insurers may, one by one, remove or constrain AI cover. As with property insurance in high-risk catastrophe areas, the industry may then be blamed for “pricing” or removing cover for a risk it did not create.

To avoid being cast as the villain, the industry needs to get on the front foot and work across multiple fronts, including:

• Start communicating now. Explain openly why AI is hard to insure, and why; signal that cover may narrow or disappear; and set out what has to be true for insurers to stay in the market.
• Shape the rules. Get into the policy debate and help write the rules, standards and regulations while they are still forming.
• Map liability and agree the insurance contract wording. Establish where each risk originates and who is accountable, and settle the key definitions, seeking regulatory approval where required by the market.
• Agree where shared data can help. Identify where common data, such as a pool of incident and loss data, would help pricing and modelling, and seek any regulatory approvals and industry agreement needed to build it.
• Make the safeguards clear, then price to them. An underwriter can only assess controls that are defined and open to review, so the safeguards that make a deployment insurable have to be spelt out first; then price on what a system actually has in place – there is no useful loss history. Here, too, standardising those safeguards and seeking regulatory approval where needed is worth considering.
• Be straight with customers. Tell them what their AI exposures may be, and what governance helps and keeps them covered.

Insurance has historically made “taking risk” possible by helping to understand threats, defining standards and controls, and providing underwriting and pricing signals. The opportunity here is not simply to decide whether AI is insurable – it is for the industry to help make it insurable.