TL;DR
Several governments have faced embarrassment after AI-generated errors in official documents came to light. These incidents reveal the risks of relying on AI without proper human verification and have prompted calls for stricter oversight.
Multiple governments have publicly disclosed incidents where AI-generated content, including fake citations and erroneous references, caused embarrassment and raised accountability concerns. These cases underscore the risks of deploying AI in official capacities without adequate verification processes.
In April 2025, South Africa withdrew its Draft National Artificial Intelligence Policy after discovering that at least six of 67 sources cited were AI hallucinations—fictitious research fabricated by AI. Minister Solly Malatsi attributed this to unverified AI-generated citations, marking the first time a government has retracted a document over such errors.
Similarly, in May 2025, the Trump administration released a report on children’s health containing numerous incorrect citations, including nonexistent studies and muddled author attributions. White House spokesperson Karoline Leavitt dismissed these as “formatting issues,” which were later corrected.
In August 2025, Australian authorities flagged a Deloitte report for containing fake academic references and made-up quotes. Deloitte confirmed that AI tools caused inaccuracies, leading to a refund of $290,000 to the government after reissuing a corrected report in September. Canada’s Newfoundland and Labrador government also faced issues with a Deloitte healthcare report, which included fake citations, prompting a re-release after corrections.
Europe’s cybersecurity agency ENISA admitted that two threat reports published in 2025 contained AI hallucinations, with 26 out of 492 footnotes being incorrect. Experts criticized the lack of verification protocols, warning about the risks of trusting AI-generated sources in critical institutional publications.
Why It Matters
These incidents highlight the potential dangers of integrating AI into official government work without rigorous human oversight. They raise questions about accountability, the reliability of AI-generated information, and the need for strict verification processes to prevent misinformation and reputational damage.
AI Workflow Tools for Researchers & Analysts: Automating Literature Reviews, Summaries, and Hypothesis Generation with ChatGPT, Claude, and Perplexity
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Background
Over the past two years, several governments and institutions have experimented with AI for drafting reports, policy documents, and research citations. While AI offers efficiency, these cases reveal vulnerabilities where AI hallucinations—fabricated or inaccurate outputs—can lead to embarrassment and undermine trust in official communications.
South Africa’s withdrawal of its AI policy marked a historic moment, but other incidents, such as the US, Australia, Canada, and Europe, show a pattern of AI errors causing real-world consequences. Experts warn that without proper safeguards, AI hallucinations could become more frequent and damaging.
“There will be consequence management for those responsible for drafting and quality assurance.”
— Solly Malatsi, South Africa’s Minister of Communications and Digital Technologies
“ENISA let AI touch the one layer it should never touch unguarded: the truth layer.”
— Chiara Gallese, AI law and data ethics researcher
Write with AI: Conquer Writer’s Block, Unleash Your Creativity, and Write Your Book Using Artificial Intelligence
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
What Remains Unclear
It remains unclear how widespread AI hallucination issues are across all government agencies and what long-term measures will be implemented to prevent future incidents. The extent of the impact on public trust and policy-making is still being assessed.
C2PA Content Credentials: Engineering Provenance for AI‑Generated Media
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
What’s Next
Authorities are expected to establish stricter verification protocols for AI use in official documents. Future audits and oversight measures are likely to be introduced to mitigate risks, with ongoing assessments of AI’s role in government processes.
Proofreading and Editing Precision (with CD-ROM)
Used Book in Good Condition
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Key Questions
How common are AI hallucination incidents in government work?
While these five cases are notable, the true prevalence is unknown. Experts warn that as AI use grows, similar errors could become more frequent without proper safeguards.
What steps are governments taking to prevent AI hallucinations?
Some governments are implementing stricter verification protocols, requiring human oversight, and revising procurement policies to include AI risk assessments.
Could AI hallucinations undermine public trust in government?
Yes, repeated incidents could erode trust, especially if they lead to misinformation or policy errors. Transparency and accountability measures are being discussed to address this risk.
Are these incidents likely to happen again?
It is possible unless comprehensive safeguards and verification processes are put in place. Experts recommend mandatory human review for AI-generated content in official documents.
