Deloitte will provide a partial refund to the federal government over a $440,000 report that contained several errors, after admitting it used generative artificial intelligence to help produce it.
The Department of Employment and Workplace Relations (DEWR) confirmed Deloitte would repay the final instalment under its contract, which will be made public after the transaction is finalised. It comes as one Labor senator accused the consultancy firm of having a “human intelligence problem”.
Deloitte was commissioned by the department to review the targeted compliance framework and its IT system, used to automate penalties in the welfare system if mutual obligations weren’t met by jobseekers, in December 2024.
Sign up: AU Breaking News email
The subsequent report found widespread issues, including a lack of “traceability” between the rules of the framework and the legislation behind it, as well as “system defects”. It said an IT system was “driven by punitive assumptions of participant non-compliance”.
It was first published on 4 July. It was re-uploaded to the DEWR website on Friday, after the Australian Financial Review in August reported that multiple errors had been found, including nonexistent references and citations.
University of Sydney academic, Dr Christopher Rudge, who first highlighted the errors, said the report contained “hallucinations” where AI models may fill in gaps, misinterpret data, or try to guess answers.
“Instead of just substituting one hallucinated fake reference for a new ‘real’ reference, they’ve substituted the fake hallucinated references and in the new version, there’s like five, six or seven or eight in their place,” he said.
“So what that suggests is that the original claim made in the body of the report wasn’t based on any one particular evidentiary source.”
The updated review noted a “small number of corrections to references and footnotes”, but the department has said there have been no changes to the review’s recommendations.
“Deloitte conducted the independent assurance review and has confirmed some footnotes and references were incorrect,” a spokesperson for the department said.
“The substance of the independent review is retained, and there are no changes to the recommendations.”
In the updated version of the report, Deloitte added reference to the use of generative AI in its appendix. It states that a part of the report “included the use of a generative artificial intelligence (AI) large language model (Azure OpenAI GPT – 4o) based tool chain licensed by DEWR and hosted on DEWR’s Azure tenancy.”
Deloitte did not state that artificial intelligence was the reason behind the errors in its original report. It also stood by the original findings of the review.
“The updates made in no way impact or affect the substantive content, findings and recommendations in the report,” it stated in the amended version.
A spokesperson for Deloitte said “the matter has been…
Source link
Disclaimer
We strive to uphold the highest ethical standards in all of our reporting and coverage. We blogs.grocliq.com want to be transparent with our readers about any potential conflicts of interest that may arise in our work. It’s possible that some of the investors we feature may have connections to other businesses, including competitors or companies we write about. However, we want to assure our readers that this will not have any impact on the integrity or impartiality of our reporting. We are committed to delivering accurate, unbiased news and information to our audience, and we will continue to uphold our ethics and principles in all of our work. Thank you for your trust and support.
Website Upgradation is going on for any glitch kindly connect at [email protected]
