The line between genuine recordings and AI-generated fabrications is becoming harder to distinguish, prompting technology experts to warn the US judiciary that verifying audio and video evidence could soon become a far more complicated task.
Representatives from OpenAI and Meta raised the concern before a federal judicial advisory panel in Boston, where officials and specialists examined how courts should handle evidence potentially manufactured using artificial intelligence. The committee has been wrestling with the issue for several years as generative AI tools become increasingly capable of producing convincing fake content.
Shira Anderson, policy counsel at OpenAI, cautioned that advances in AI-generated imagery could eventually outpace the ability of detection systems to identify manipulated material.
“AI images are only getting better, and I do not think that there will be a scientific way going forward to detect them,” she told the panel.
The challenge is not limited to the sophistication of the technology. Experts also pointed to the ease and low cost of creating convincing synthetic recordings, raising concerns about how courts can reliably establish whether digital evidence is authentic.
Devin Norder, Meta’s director of product management, said media produced by the latest generations of advanced AI models can be indistinguishable to human observers from genuine recordings. Producing such material, he added, requires relatively little time or expense.
Courts weigh the threat against limited evidence
Despite growing warnings from the technology sector, the scale of the problem within the American judicial system remains uncertain.
A Federal Judicial Center survey conducted among judges found that only 15 of the 931 respondents had encountered a case in which a party challenged evidence on the grounds that it was a deepfake.
That limited experience has complicated efforts to introduce specific safeguards. While experts warn that fabricated evidence could become increasingly difficult to detect, some judges remain reluctant to overhaul existing rules before the courts encounter the problem more frequently.
US District Judge Jesse Furman, who chairs the Advisory Committee on Evidence Rules, acknowledged the gap between predictions about the threat and the cases reported so far. He appointed a subcommittee to examine the issue further.
The panel has been considering how to ensure that AI-generated material presented in court meets appropriate standards of authenticity and reliability, particularly when its origins or methods of production are disputed.
Detection tools offer no guaranteed protection
Technology companies have begun developing tools and services intended to help establish whether digital content is genuine. Meanwhile, legislation in California and the European Union has introduced watermarking requirements aimed at making AI-generated material easier to identify.
However, such measures are far from foolproof. Different companies use different systems, leaving investigators and courts without a universal method to identify every watermark or reliably authenticate all visual material.
Anderson warned that detection capabilities would continue to evolve, but she expressed little confidence that any system would offer complete protection against increasingly sophisticated fabrications.
The problem creates a difficult position for judges, who must determine whether evidence can be trusted even as the technology used to produce it continues to change.
Existing rules may already cover fabricated evidence
Not everyone on the advisory committee believes a new AI-specific legal framework is immediately necessary.
US District Judge Valerie Caproni cautioned against rushing into new rules without stronger evidence that deepfakes have become a widespread problem in court proceedings.
She noted that existing legal safeguards already prohibit the submission of fraudulent evidence and give judges the authority to assess whether material has been falsified.
Caproni argued that deception in court is not a new phenomenon, even if artificial intelligence has made it easier to manufacture convincing evidence.
The debate therefore centres on whether existing rules can keep pace with a technology that makes fabrication faster, cheaper and potentially harder to expose.
For the judiciary, the immediate challenge is to prepare for a threat that experts believe will grow, without introducing sweeping changes before its practical impact becomes clearer. As AI-generated recordings become more realistic, establishing what actually happened may increasingly depend on verification methods that courts cannot yet rely on with complete confidence.
