Recently, friend of the Lab Riana Pfefferkorn joined the Lab’s flagship course at Stanford, EE292J: Designing for Authenticity, to update the Lab and our students on the current status of federal debates on how best to respond to Deepfakes in the Courtroom.

Riana Pfefferkorn is currently a Policy Fellow at the Stanford Institute for Human-Centered Artificial Intelligence (HAI). Her seminal article Deepfakes in the Courtroom, published in 2020, remains a highly referenced piece (it’s worth a read!) regarding the evidentiary challenges courts may face when addressing deepfakes. It also argues that courts have sufficient tools already to deal with deepfakes—a position that Riana still stands by.

With her permission, here is a summary of her talk as a part of our efforts to keep everyone informed about legal developments and dilemmas about how cheap and accessible gen AI affects evidence.

The Current State of the Deepfake Debate

For the last several years, academics and the United States federal judiciary have engaged in a long debate about whether action is needed to prepare our courts for the looming rise of deepfake evidence (see, e.g., Grossman & Grimm 2025, Delfino 2025.)

Twice-yearly since late 2023, the Federal Courts Advisory Committee on Evidence Rules, the body charged with proposing amendments to the Federal Rules of Evidence, has considered whether amendments need to be made to keep deepfakes out of the courtroom. And, for the last three years, they have continued to decline the adoption of any new rules regarding deepfakes.

To some this may seem puzzling, but as Riana laid out to the students, it actually makes a lot of sense to still avoid raising the bar for admissibility, for two main reasons:

1. The concern about deepfakes is real, but also nothing new. The challenge of keeping altered, forged, or false evidence from infecting court proceedings has been an issue for every legal system whenever new technologies have arisen, and the rules for authenticating evidence are flexible enough to deal with this one as they have prior threats.
2. Secondly, the threat of deepfake evidence—that is, evidence which is a partial or complete fabrication but held out as genuine (as opposed to ‘AI-enhanced’ evidence, another category that is being highly debated)—seems to still be unproven. Despite skyrocketing model complexity in generative capability, there has yet to be any large presence of deepfake evidence in federal litigation.

Why Deepfakes are not Particularly New

For as long as we have had courts, we have had litigants trying to pull a fast one. As Riana puts it:

“While seeing has always been believing, we’ve also as a society, known that appearances can be deceiving, and that technological trickery is always possible.”

Indeed, as soon as photography became ubiquitous, so too did the question of manipulations. As early as the 1860s, courts were being forced to interrogate how to respond to fake double-exposed images. So too, in our more recent past, courts have had to come to live with the possibility of photoshopped images.

In Riana’s words: “this isn’t our first rodeo,” and the rules of courts do a pretty good job at enabling us to challenge forgeries. Even within our currently quite open standards for authenticating evidence, there are still ways to use and adjust the application of our rules towards deepfakes without explicitly changing them. For instance, in light of concerns that witnesses could be used to inadvertently authenticate deepfake evidence under F.R.E 901(b)(1) ‘Testimony of a witness with knowledge,’ judges can simply move towards adopting currently more minority interpretations which refuse to accept witness authentication unless said witnesses were actually at the scene (instead of simply, say, identifying the defendant in a video). Litigators and judges can also simply challenge digital items that aren’t up to snuff, attacking their provenance, chain of custody, or digital integrity.

While there are of course nuances here, this leads Riana to conclude generally that we shouldn’t raise the bar for authentication, especially because it’s unclear what it would even be stopping at this time.

Where is the ‘Wave’ of Deepfake Evidence?

Simply put, deepfakes have yet to truly reach the federal courts at any real scale. The Federal Courts Advisory Committee commissioned a survey, sent out in January of this year, which found that 98% of the over 900 federal judges who responded had yet to encounter any deepfake challenges in their court. And of the 15 actual judges who did, only 5 felt that changes to the rules should be considered.

Riana told the class she wasn’t surprised by these results; she has yet to be made aware of a single case where an attorney has knowingly attempted to submit a deepfake into evidence. The reason, she thinks, could be structural: Attorneys have a duty of candor to the court, which critically includes not offering evidence one knows to be false. It’s even Supreme Court precedent: lawyers are prohibited from “taking steps or in any way assisting the client in presenting false evidence,” Nix v. Whiteside, 475 U.S. 157, 166 (1986). As such, presenting a deepfake as evidence is likely too much to risk one’s bar license or even just one’s credibility in front of the court. And while there are a few cases where pro se litigants (those who are unrepresented) have tried to submit deepfakes as evidence, judges have successfully been able to reject such evidence on grounds familiar to the court.

Thus, despite warnings of a coming tide of deepfakes in the court, very few have yet to materialize. But one prediction has in fact become true: Citron’s and Chesney’s “Liar’s Dividend” or ‘the deepfake defense.’ This occurs when, in light of damning authentic evidence, parties attempt to claim, with no supporting materials, that such evidence must be rejected on the grounds that it could possibly be a deepfake. Luckily, courts haven’t bought this either, finding such arguments dubious and ridiculous.

What Concerns Do Remain? C2PA & Worsening Disparities

So, courts seem to be able to address deepfakes under the current rules just fine, and deepfakes don’t seem to be that common… So what is still at stake?

For Riana, the new technologies we may rely upon for additional certainty of authenticity, even under the same rules—provenance technology like C2PA—risks creating access barriers:

“One potential ramification of a world where some people have access to (…) the $9,000 cameras that have C2PA in it, and [where others] just have regular, run-of-the-mill smartphones, is that it may exacerbate existing disparities in who gets believed when they testify or produce evidence, and who does not. (…) Similarly, I think that if you look at who already comes from a disadvantaged socioeconomic background and therefore is not going to have a $9,000 camera (…) they are also going to be less likely to be believed when they do have authentic evidence.”

This is particularly an issue if we consider how, once jurors become aware of the existence of C2PA or other systems of content provenance, attorneys may be able to prime juries to be extra skeptical of images that lack it (known as the “Reverse CSI Effect”)—enabling their pre-existing biases about people to take root over the evidence being shown or its credibility.

Conclusion

Deepfakes in the courtroom remain a theoretical threat as much today as when Riana first wrote her piece. But the debate today seems to understand that the chances of completely fabricated generated evidence overwhelming the courts and making current rules and practices unusable is unlikely. Nonetheless, as provenance technologies become more mainstream and as prosecutors and defense attorneys alike explore more ‘edge’ cases of AI (such as the aforementioned “AI-Enhanced” debacle), it remains important to keep on eye on what arguments are being made and which are successful. The Law is, as always a constant battle, and just if the rules stay the same doesn’t mean that the results are guaranteed to further access to justice.

Thank you so much to Riana Pfefferkorn for joining us at the Lab!

International

• EU AI Act — Article 50 (Transparency Obligations) (Upcoming — effective Aug. 2, 2026). Requires machine-readable marking of all AI-generated outputs and visible disclosure of deepfakes by GPAI providers and deployers, with fines up to €15M or 3% of global turnover.
• China — Measures for the Labelling of AI-Generated and Synthetic Content (In effect — Sept. 1, 2025). Mandates both explicit user-facing labels and implicit metadata labels on AI-generated text, images, audio, and video; paired with the 2023 Deep Synthesis Provisions and GB/T 45654-2025 data-provenance standard.

United States — Federal

• NO FAKES Act (H.R. 2794 / S. 1213, 119th Congress) (Upcoming / pending). Federal private right of action against unauthorized AI voice and visual-likeness replicas; live First Amendment fight (FIRE has flagged it).
• COPIED Act (Content Origin Protection and Integrity from Edited and Deepfaked Media Act, S. 1396) (Upcoming / pending). Directs NIST to set provenance, watermarking, and synthetic-content detection standards; bars training on content with provenance information without consent.
• Take It Down Act (In effect — signed May 2025). Federal takedown obligation for non-consensual intimate imagery, including AI-generated and AI-altered content; the only enacted federal piece in this cluster.
• NIST AI Safety Institute / NIST GenAI program guidance on synthetic content provenance (In effect — ongoing rolling guidance). Non-binding but increasingly referenced by state statutes.

United States — State

• California — AI Transparency Act (SB 942) + AB 853 amendments (Upcoming — operative date deferred to Aug. 2, 2026). Latent and manifest disclosures, AI-detection tool requirement, capture-device provenance option, and platform-side provenance surfacing for large online platforms.
• New York — Stop Deepfakes Act (S6954A / A6540A) (Upcoming / pending in legislature). Three-part provenance regime: synthetic-content providers attach metadata, social platforms preserve it, state agencies attach it where practicable; AG rulemaking authority.
• New York — Synthetic Performer Disclosure (GBL § 396-b) (Upcoming — effective June 9, 2026). Conspicuous disclosure of synthetic performers in advertising; $1,000 first-offense / $5,000 subsequent civil penalties.
• Tennessee — ELVIS Act + Digital Content Provenance Pilot Program (In effect — July 1, 2024). Civil cause of action for AI voice/likeness misuse, plus a state-run provenance pilot for emergency-management content and election-content provenance requirements.

OSINT is increasingly available to prosecutors thanks to the popularity of cell phone cameras and social media, but finding the photographer may prove difficult – if not impossible. Generative AI creates additional challenges to admissibility.

The fictional scenario we designed involved all the staples of open-source methodologies (geolocation, chronolocation, etc.) and was the first time C2PA metadata was being interrogated in a court-like setting for its reliability and probative weight.

The Case Itself

The fictional case derives from an alleged war crime in Yemen (in fact, an active investigation of Bellingcat). In our training scenario, the prosecution argues that a jet fighter pilot conducted a “double-tap” airstrike on June 12, 2023 at approximately 10:30 AM – first dropping one bomb, then returning to strike again and killing at least six civilians who were conducting rescue operations. Mr. Smith, the defendant, claims he conducted only a single, legitimate military strike at 9:00 AM targeting a military communications center when no civilians were present.

Ultimately, the case is about the second strike (which makes it a war crime). If its criminal nature can’t be established beyond a reasonable doubt, Smith won’t face consequences for an alleged strike against a possibly legitimate target.

The Evidence Presented

The mainstays of open-source investigations are relied upon to interrogate when the acts might have taken place:

– Firstly, chronolocation aims to reliably and accurately estimate the time of the incident.

– Secondly, contextual verification is put forward by the prosecution to corroborate a piece of open-source evidence with other open-source evidence. (The scenario deliberately doesn’t include direct testimony, to focus the trainees on open-source investigations only.)

– Finally, and most relevantly for us at Starling Lab, some material includes C2PA metadata, a technical standard for provenance which is presently untested in judicial settings. We presented the trainees with a video including two Manifests: one with the media’s authentication metadata produced straight from the C2PA-compatible Sony camera that captured it; the second added to the video after the fact by an investigative journalism collective, claiming to have verified the video, in a similar fashion to what BBC Verify have experimented with.

In addition to contributing to the scenario design and the case papers, I took part in the training as an expert witness interrogated and cross-examined by a group of six barristers (three on each side). I testified to the whole of the prosecution expert report as though the case was real and it were my own report. It was an intense four hours!

Impressions and takeaways

I found it fascinating to get an insight into what barristers – with no pre-existing familiarity with digital investigations methods – make of this bundle of evidence. It appeared quite corroborating, all-pointing-in-one-direction, but perhaps somewhat circular. In a trial, nothing is taken for granted.

As an expert witness the reliability of what you say is under assault – all answers contribute to your own credibility on the stand, and in your report.

The experience highlighted the paramount importance of intellectual honesty for an expert witness. Rather than serving the interests of the party who commissioned their report, the expert’s primary responsibility lies with the court. This means providing unbiased, accurate, and complete information, even if certain nuances or findings might not directly support the commissioning party’s case. The mock trial served as a practical exercise in upholding this principle, underscoring that the expert’s credibility and the integrity of the judicial process depend on their unwavering commitment to the truth, irrespective of who is paying their fees.

The purpose of relying on expert witness testimony is to inform the court, but it comes with the responsibility of having to effectively communicate findings and opinions. Too long and you’ll be cut out; too in the weeds and jargony and you’ll lose the room. Analogies and relatable explanations are the name of the game there.

From a substance perspective, I found myself unable to get to some of the nuances I wanted to touch on. Because the exercise was quite focused on how much time was allocated to the metadata questions, we got somehow stuck on how C2PA data can be removed from an image, and how difficult it might be to acquire an X.509 certificate.

I had laid a few hooks for the defense to pick up on, and left a question or two unanswered. For example, the prosecution expert did not include a detailed verification report of the second organisation’s cert (they really should have).

Prior to this mock trial, I took a week-long “Courtroom Testimony for Expert Witnesses” course from LEVA and Jonathan Hak KC. This gave me a hunch that I was trying to be too clever: as a training exercise it’s not optimal to leave stones unturned, since no-one in the court is more of an expert than we are. In other words: The training aims to bring the lawyers to having some familiarity with the substance, and to practice the techniques – it is not for the defense to pick up on minute, detailed, very precise concerns from the report themselves.

Indeed, we did not stray far from the expert reports themselves due to lack of time. I’ll be proposing a different, much more back-and-forth structure in a future training, so that we can derive must-dos and mainstays of _any_ C2PA verification.

I’m very grateful to the following people and organisations for giving me this opportunity to contribute to this exercise, and for inviting me to take part on the day. Walking the small alleyways of Temple, Blackfriars, and Fleet St is so humbling for this former law undergrad who spent a bit of time in various newsrooms, and in Hackney. So a huge thank you to:

– Professor Yvonne McDermott from Swansea University and the TRUE Project;

– Judge Korner KC, presently at the International Criminal Court (ICC)

– The Honourable Society of the Inner Temple

– The Institute for International Legal Advocacy Training

Recent policy changes in the United States, retreating from leading positions in international efforts and aid, led to new relevance of already important questions we had covered in Washington DC, in September 2024, at Georgetown University and with Ambassador Van Schaack.

At a high level, the question remains: When it comes to digital evidence, what is changed (if anything) by widespread cheap and accessible generative AI?

In addition to a health check of our information environments and of its pollution (or not) by deepfakes and AI slop, we must return to existing protocols and guidelines – and interrogate whether and how they should be updated.

Starling did just that on July 11, 2025, teaming up with Fénix Foundation, a non-profit that leverages technology to support international justice, peace, and accountability through legal AI tools and judicial training programs.

We have previously worked with the co-founders of Fénix, on the Hala Protocol for the Collection, Processing, and Transfer of Audio Data, Sabrina Rewald and Emma Irving (they are the main drafters, and I am an adviser). Emma is one of the authors of the Leiden Guidelines on Digitally-Derived Evidence, and Sabrina supported the running of our Washington event in 2024.

Joined by a few additional friends from the academic world, we gathered at Leiden University’s The Hague campus to draft a short-term response to the rapid advances in AI’s sophistication and accessibility – and resulting implications on legal evidence.  Our efforts led to a set of “Preliminary Guiding Principles” for immediate use by courts, investigators, lawyers, and fact-finders. .

In addition, we are seeking support for an AI and Digital Evidence Knowledge Hub – an open-source, go-to resource that would contain cases and relevant procedural rules from various jurisdictions around the world. Using a decentralized approach to knowledge collection, it will rely on a wide network of contributors to populate the hub with case law, legal provisions, and best practices.

Notably, the meeting moved beyond a US-centric conversation, incorporating diverse legal traditions and regional experiences to address the complexities of AI-generated content in atrocity crimes prosecutions worldwide.

As legal authorities in Europe and universal jurisdiction countries undertake their important work, we hope to assist them by identifying specific areas where digital evidence guidelines need updating, as well as  where professional training can be beneficial.

UPDATE: Since this was published, the UN Human Rights Council adopted a resolution on protecting human rights defenders in the digital age. It addresses several important issues that our Lab has been focused on. Scroll to the end of this page for more details.

The report notes how Starling Lab supported efforts in Brazil’s Pantanal wetlands with tools that work even in low-connectivity environments, which is needed where the “digital divide hits many human rights defenders very hard.”

In a speech this month presenting her findings, Lawlor observed how some of those most at-risk include “journalists covering human rights issues at the local level.” Starling’s methodology was used by journalists to document environmental destruction – and confront climate change denialism. Our submission to Lawlor’s office focused on data authentication, decentralized storage, and cryptographic verification. Together, these ensure documentation remains tamper-evident and accessible, even when governments seek to dismiss the truth.

Her report includes several valuable recommendations to governments and other international actors, two of which resonate with our work:

• “Expand access to the Internet and secure communication tools, including by increasing funding for such digital security resources as encrypted communication applications and secure reporting channels.”
• “Support efforts to enable human rights defenders to store and safeguard their information securely, without fear of unlawful surveillance or data breaches, including putting in place robust legal safeguards to prevent the misuse of digital tools to suppress dissent or target defenders and ensure that their digital rights are protected.”

We appreciate being included among so many other human rights defenders, and remain grateful to Pablo Albarenga for his photojournalism in Brazil, as well as to Inside Climate News for publishing this important coverage.

And we hope this underscores the vital role of trusted digital evidence in defending human rights and environmental justice.

🔗 Read the Special Rapporteur’s remarks;
📄 Read the full UN report;
📕 Don’t miss our own case study;
📰 And the Inside Climate News article

Starling Lab was previously referenced in a report to the UNHRC in 2023 by the Rapporteur on the right to education, who acknowledged similar methods used by the lab as emerging “good practices” for documenting war crimes against civilian objects like schools.

Update (April 16, 2025): Our submission is also echoed in a new resolution from the UN Human Rights Council, which addresses the protection of human rights defenders in the digital age (full text: A/HRC/58/L.27/Rev.1).

The resolution emphasizes “universal connectivity” and “meaningful connectivity” as essential for defenders’ work, calls for “technical solutions for strong encryption and anonymity,” and advocates for secure information storage “without fear of unlawful surveillance.” It specifically recognizes the “growing number of digital attacks” on defenders and acknowledges the “protection gap” caused by lack of accountability.

These address key points from our submission on data integrity and authentication technologies for remote areas.

We’re particularly encouraged by the HRC’s recognition that “new and emerging digital technologies can hold great potential for strengthening democratic institutions and the resilience of civil society, empowering civic engagement and enabling the work of human rights defenders, public participation and the open and free exchange of ideas, and for the exercise of all human rights.” This aligns perfectly with our mission at Starling Lab to harness technology to establish trust in digital records.

Our earlier submission outlined innovative approaches to produce documentary evidence and combat digital denialism. This includes cryptographic methods that authenticate evidence from the point of capture, enabling defenders in remote areas to establish credibility despite connectivity challenges. The submission also referenced ongoing work on telecommunication technologies, including 5G, to further enhance these capabilities.

On October 22, the Starling Lab and Stanford HAI convened a diverse group of over 100 technologists, journalists, legal experts, and archivists at the Cecil H. Green Library for our conference, Trusting Digital Content in the Age of AI.

We want to extend a sincere thank you to everyone who took part – from speakers like Brewster Kahle and Zach Seward to attendees from the AP, BBC, and the Internet Archive. Together, we moved the conversation beyond the “arms race” of deepfake detection and toward “upstream” solutions: cryptography, provenance, and interoperable ecosystems of trust. 

Thank you for helping us design a more authentic digital future. We look forward to continuing this vital work with you in 2025.

The event convened a diverse group of experts to address the erosion of trust in digital ecosystems. James Landay (Stanford HAI) and Tsachy Weissman (Starling Lab) opened the conference, framing the urgent need to design new systems for authenticity.

In the first general session, Jonathan Dotan moderated a discussion on building interoperable systems of trust. Zach Seward (New York Times) addressed how newsrooms are adapting to AI, while Riana Pfefferkorn (Stanford Cyber Policy Center) explored the legal challenges of deepfakes and the “liar’s dividend.” Brewster Kahle (Internet Archive) spoke on the critical mission of preserving digital history amidst technical threats.

A second panel, led by Vanessa Parli, reviewed the year in Generative AI. Michael Bernstein (Stanford) discussed the logic of social media platforms, Oren Etzioni (TrueMedia.org) presented on deepfake detection at scale, and Aimee Rinehart (Associated Press) shared insights on AI procurement and combating misinformation in journalism.

Later sessions focused on solutions, with Dan Boneh (Stanford) demonstrating cryptographic proofs for content authenticity, joined by Jeff Hancock and Margaret Hagan on the human and legal aspects of trust. Finally, Ann Grimes, Basile Simon, and Adam Rose led discipline-specific roundtables on tools for journalism, law, and archiving.