AI FactScan
In September 2026, a murder appeal before the New Mexico Supreme Court named four witnesses who had never testified. It also attributed false statements to real witnesses and misrepresented two legal authorities.
The lawyer, Stephen Aarons, acknowledged using ChatGPT to prepare the brief. He also admitted that he had not checked its factual claims or legal authorities before filing.
Courts have already encountered AI fake citations. In State v. Sandoval, the AI hallucination also changed the brief's account of the trial.
ChatGPT had the case files
Reuters reported that Aarons gave ChatGPT a computer-generated trial transcript and other case materials. He expected a "bulletproof summary."
ChatGPT still added people and testimony that were absent from the record. The false details covered alleged threats and the shooter's clothing and appearance.
Supplying the files did not make the summary reliable. Each claim still needed to be traced back to the transcript.
The lawyer filed it without checking
The court's order says Aarons admitted that he did not verify the factual claims or legal authorities. He signed and filed the brief anyway.
At the August 21 hearing, Justice C. Shannon Bacon asked whether he followed the news. She described lawyers relying on AI hallucinations as "an above-the-fold story every single day."
The court struck the brief
In its September 9 order, the court found Aarons in direct contempt and ordered him to pay $5,000. It referred him to the Disciplinary Board and barred him from appearing before the court while that process is pending.
The court struck all briefing in the appeal and appointed the Law Office of the Public Defender to provide new counsel. The order dealt with Aarons's conduct, not the merits of the murder appeal. New counsel will file new briefing, and the appeal will continue.
Asking ChatGPT again does not check the answer
"Are you sure?" gives ChatGPT another prompt. The next answer may correct the error or repeat it with equal confidence.
"Do not hallucinate" has the same limitation. It may change the wording, but it does not check the answer against an independent source. We examine this in Does "Do Not Hallucinate" Work?
Verification had to happen outside the chat: find each witness and statement in the transcript, then open the cited cases and read the relevant passages.
Find the source, then read it
AI FactScan finds and grades citations in an AI answer. That first pass can surface missing, weak or questionable references before they reach a report, paper or legal document.
Then open the source and read it. A real, reputable citation can still fail to support the sentence beside it. The original document is where the claim must hold up.