The UK audit regulator finds the biggest firms don't measure whether their AI tools improve audits

June 2025 — Britain's audit regulator found that the six largest audit firms had no formal way to measure whether their automated audit tools, including new AI ones, improve audit quality. Here's what it found, what it costs, and what went wrong.

Cost and outcome (as of June 26, 2025)

Outcome: most firms had "well-established processes" to certify tools before use, but some were "less mature and not supported by documented policies." (FRC review)
Cost: "There was no formal monitoring performed by the firms to quantify the audit quality impact of using ATTs." (FRC review)
Penalty: none. The guidance "does not introduce new regulatory requirements." (FRC)
Status: firms were planning changes for AI's "unique risks"; the FRC urged them to act "as soon as possible." (FRC review)

What the regulator looked at

The Financial Reporting Council reviewed how the six firms plan, build or buy, certify and monitor the tools their audit teams use to assess risk and gather evidence. It published the review on June 26, 2025, alongside its first guidance on AI in audit.

"AI tools are now moving beyond experimentation to becoming a reality in certain audit scenarios," said Mark Babington, the FRC's executive director of regulatory standards.

The review is a snapshot. The FRC sent firms an information request in April 2024, held feedback meetings in autumn 2024, and based its findings on firms' responses in the second quarter of 2024 and "a small sample of ATTs." Processes "may therefore have evolved," it says.

Most tools come from the firms' global networks or are built in the UK, the review found. Where no tool exists, most firms preferred to build in-house rather than buy from third-party vendors.

Counting users, not results

"Monitoring performed by firms was generally restricted to a sub-set of more widely used ATTs, and primarily performed to understand how many audit teams were using the ATTs in their audits (typically for licensing purposes)," the review says.

Firms generally had no key performance indicators for tool use, though one reported usage against targets. And "in general, firms do not monitor that audit teams are only using authorised and current ATTs," relying instead on controls that restrict downloads and remove old versions.

The FRC encouraged firms to set policies or metrics to evaluate how the tools affect audit quality. "Further monitoring capabilities would allow firms to focus resources on ATTs having the greatest impact on audit quality and also identify those that are not having the desired impact," it says.

Checking tools after launch was also informal. Most firms recertify a tool only when it changes. Two run annual confirmations by tool owners, and one requires recertification at least every three years.

Where AI comes in

Firms are deploying tools that use machine learning to analyse full populations of journal entries or revenue transactions and flag unusual patterns, the FRC found. Generative AI tools were then limited to aids such as chatbots.

The FRC listed risks that certification must cover: the model's source and version, its training data, whether outputs can be explained, bias, and training for audit teams. Firms "acknowledged that the use of AI within an ATT presented additional risks that may not be addressed by existing certification processes," the review says.

What went wrong

No measure of benefit. Firms tracked how many teams used a tool, mostly for licensing, not whether it improved audits.
No usage check. Firms generally did not monitor that teams used only approved, current versions.
AI outpaced the rules. Firms were still adapting certification for AI tools already in use.
What we don't know yet: which firms had the weaker processes, and whether any now measure audit-quality impact.

Our follow-up question for the FRC

Has any of the six firms since begun formally measuring whether its automated or AI audit tools improve audit quality?

Sources

Financial Reporting Council, “Thematic Review: Certification of Automated Tools and Techniques,” June 2025. frc.org.uk
Financial Reporting Council, “FRC publishes landmark guidance providing clarity to audit profession on the uses of AI,” June 26, 2025. frc.org.uk

Disclosure: [not in copy] Editor placeholder