The Western Australian government has said it is "extremely disappointed" and is seeking an "expedited" cost recovery from Deloitte after errors were discovered in a report about a potential university merger, including AI-hallucinated references and hundreds of millions of dollars in cost discrepancies.
The government paid Deloitte $166,000 for a feasibility study on merging the University of Western Australia, Curtin University and Murdoch University.
Deloitte considered three merger options between the three universities, with "low", "mid" and "high" cost scenarios, and broke down the calculations of those costs in the report's appendix.
Under the "high" cost scenario of a merger between Curtin and UWA, the report calculates the "total costs to day 1" as $514m, and "post day 1" over five years at $145m. The total of those two payments is $659m. The report wrote $399m, a discrepancy of $260m.
Under a "low" cost merger scenario between Curtin, UWA and Murdoch, the total costs to day one are $396m and the total post day one over five years is $116m. That total would be $512m. The report calculates the two numbers to $823m, a discrepancy of $311m.
Deloitte is already under pressure after it was revealed in the media this month that the same report contained hallucinated AI references, leading the state opposition to label it "AI slop". The consultancy firm admits in its report that it used generative AI to "support the collation of research, synthesis of information and editorial proofing", and had earlier dismissed the errors as "minor".
A spokesperson for the WA government told Guardian Australia, which reported the discrepancies, that the Department of Energy and Economic Diversification had written to Deloitte to expedite the cost recovery process for the report.
"We are extremely disappointed these errors are being found now," the spokesperson said. "We are advised they do not affect the modelling in the report or its findings, but any errors will be corrected as part of a review by Deloitte which will re-issue the report."
The WA tertiary education minister, Tony Buti, described the errors as "typos", though a day later said he was angry and disappointed with Deloitte. Guardian Australia also found that one of the references to a World Health Organization report has a ChatGPT metadata tag on the URL, and that Curtin University is referred to repeatedly as the Curtin University of Technology, a name the institution has not used since 2010.
Prof Stephen Bartos, an expert in public sector governance at the University of Canberra, said the errors discredited the report. "In my view, they're so extensive that you probably need a new report, or at the very least someone to go through that material from the ground up and rework it manually without the use of AI," he said. Of the diverging totals he added: "Unless the numbers add up, a government might not want to do it."
Prof Christian Downie, from the Australian National University's school of regulation and global governance, said human errors happened and mistakes in reports could be expected, but that there was a lack of transparency about how the technology had been used. "We need much more detail about how [AI is] being used and for what parts of the report writing process," he said. Deloitte did not respond to questions from Guardian Australia.
There is a defence here that nobody has yet put, and it rests on taking the two discrepancies together rather than one at a time. The first printed total is $260m too low. The second is $311m too high. The pair comes to $1.222bn against a true $1.171bn: an aggregate overstatement of $51m, or 4.4 per cent, across the two merger scenarios.
This newspaper regards that as the strongest finding in the appendix. Errors that point the same way accumulate. Errors that point opposite ways cancel. A document containing one mistake is wrong; a document containing two mistakes of comparable size and opposite sign is, in aggregate, approximately right, and the fee of $166,000 has bought accuracy at the level the state actually needs, which is the level of the total.
It is also consistent with the government's advice that the errors "do not affect the modelling in the report or its findings". If the sum of the appendix is broadly correct, the appendix is broadly correct. The only part of it that is wrong is the parts.
This does, however, create a procedural hazard that the cost recovery does not address. The review will correct the figures one at a time. Between the first correction and the second, the report will be more wrong than it was on delivery: fix the $260m alone and the net error rises to $311m; fix the $311m alone and it rises to $260m.
The Department of Unnecessary Research has accordingly written to the Department of Energy and Economic Diversification recommending that both appendix totals be corrected simultaneously, by the same person, in the same sitting, and that no half measures be attempted. In the accounting sense, an error that cancels another error is not an error. It is a control.