Narrative Watch
Narrative Watch
Does bodycam footage promote accountability for police use of force? We build an authenticated survey questioning videos and claims, and archive the original material for the long-term.
Reading Time: 5min
Contents
Background
Over the past decade, there’s been a significant push for police departments to add more body cameras in the hopes of creating accountability and being a tool for reform. This collaborative project confronts the difficulty of establishing facts and interrogative bodycam footage, as well as to cryptographically authenticate and archive material primary and supportive to protect vulnerable public records.This case study follows publication of news articles detailing our findings, and is accompanied by a survey of criminal justice experts investigating the relative absence of reforms many advocates expected.
Contents
Context
Since 2020, the Starling Lab for Data Integrity awards fellowships to journalists who integrate and collaborate in case studies exploring the integration of technologies used to capture, store, and publish authenticated media. This project, called Narrative Watch, is one such fellowship between the Lab and two reporting groups: The Grio and Big Local News.
Launched in 2016, The Grio is an American television network and website with news, opinion, entertainment and video content geared toward Black Americans. Big Local News, a Stanford University-based team led the research into problems around access to public records and data for journalists working in policing, public health, policing, and more. The group works to develop tools to help journalists access, analyze, publish, and archive data.
Narrative Watch culminated in December 2023 with the publication by The Grio of two articles about police body cam footage, leveraging technical authentication tools developed by Starling. Both articles addressed the difficulty in using and obtaining body cam footage from police departments, despite regulations put in place aimed at facilitating police accountability.
While this project was a collaboration between the three teams, the following individuals were most notably involved:
- Big Local News is led by Chery Phillips (a 2023 Starling Journalism Fellow), along with Senior Data Scientist Eric Sagara acting as technical lead on the project.
- The Grio’s SVP and Chief Content Officer Geraldine Moriba oversaw the project, with additional reporting and editing support provided by Natasha Alford, and Josiah Bates.
- Starling’s work was overseen by Journalism Fellowship Director Ann Grimes and Project Manager Lindsay Walker.
- Related work around this subject was done by the California Reporting Project. Additional research and reporting support was provided by Dana Amihere, Dilcia Mercedes, Lisa Seyton, Irene Casado Sanchez, Lisa Pickoff White, and Ananya Tiwari.
The Narrative Watch project made public records requests from police departments for Use of Force cases where individuals were seriously injured or killed. They gathered reports, photos, audio, and video, focusing on cases involving claims like “I feared for my life” or “I was attacked.” The final project involved three data sets that could be cross-referenced with police reports. A survey of criminal justice experts was also conducted to see if consensus could be reached on disputed details from body cam footage.
One key set of records involved the 2020 beating of Tyre Nichols in Memphis, which were archived by the Starling Lab for Data Integrity using advanced authentication technologies. These tools preserve vulnerable public records, ensuring their accuracy and availability, especially in combating misinformation. As civil rights attorney Benjamin Crump explained, many states delay video release, leaving families without access to critical evidence. By using cryptographic methods and decentralized systems, public records can now be safeguarded against manipulation, even as the rise of AI makes such concerns more pressing.
Contents
Framework
Our guiding principle at Starling is to establish provenance as the backbone of authenticity, and to cryptographically secure the integrity of digital content. To do so, the Lab applied our three-step framework in this implementation: Capture, Store, Verify.
- Capture: Most of the ‘Capturing’ involved researchers at Big Local News send out public records requests to police stations. Though we couldn’t add authenticity at the point of capture (say from a police camera, or when this was entered into evidence), we used a leading web archiving tool, Webrecorder, to capture a small selection of public-facing documents related to the case study, such as the edited versions of the video used as a part of the survey.
- Store: Police stations sent us hours of video, police reports, images, and more, but only edited versions of the police videos were included in the survey. We decided to cryptographically preserve public-facing files, and to store them on IPFS and Filecoin to create a public record that can be recovered and verified in the future.
Verify: As much of the content from this study is sensitive in nature, we didn’t release or archive most versions of this content. The archives of websites and four video files used as a part of the survey can be downloaded and inspected. Most notably, the integrity and provenance of the files can be verified by comparing respectively their cryptographic hash, as well as their digital signature.
The Challenge & Prototype
A central focus of this project was the archiving and preservation of crucial public records. Starling played a vital role in the preservation of all materials received through public records requests, ensuring their long-term accessibility. These materials, obtained from police departments nationwide, included body camera footage, audio recordings, photos, and police reports related to Use of Force incidents where individuals were seriously injured or killed. Once received, all materials were authenticated and preserved to maintain their integrity and ensure they could be used for cross-referencing with police reports.
A significant component of the project’s work was the preservation of materials related to the high-profile Tyre Nichols case. Records sourced from the City of Memphis website and Vimeo account were meticulously archived and authenticated by Starling. Through the use of emerging cryptographic methodologies and decentralized systems, these public records are now safeguarded against potential disappearance and remain accessible for public review. This ensures that the authenticity and source of these historically significant materials can always be verified, providing a critical layer of transparency and accountability.
Following preservation the material, the next challenge was to highlight discrepancies between police written reports and body camera footage. At present, police incident footage plays a limited role in law enforcement investigations or disciplinary actions. To test hypotheses as to why, the team selected cases based on claims by officers about their reasons for the use of force, such as statements like “I feared for my life” or “I was attacked.” These cases were from Memphis, TN, Richmond, CA, and Rochester, NY, and involved incidents that were heavily documented with audio and video recordings.
This project sought to investigate how authenticated text and video could be utilized to verify the factual accuracy of reports and clarify critical statements such as, “Did the subject receive a command?”, “Was the subject visibly armed with a firearm?”, “Did the officers use force?”, or “Did the subject engage in a physical struggle?” To address this, the team conducted a survey involving various experts to determine if a consensus could be reached on these points, where the facts were often unclear. Notably, the experts found it difficult to reach agreement on many of these factual elements.
Read our technical dispatch on this preservation by clicking here.
Contents
Technology
Capture
Starling Certificates
This project focused on archiving verified content. The team used Starling Signing Certificates to attest to the authenticity of captured data, then registered that content on decentralized networks, in order to create an audit trail (or “chain of custody”) that identifies the history (known as provenance) of a piece of digital content. This serves to reduce information uncertainty and bolster trust in digital records. In the process of creating audit trails, Starling Lab surfaces metadata, which is data about digital media.
Hashing and Signing
When Starling archived these public records, an immutable digital ‘fingerprint’ called a hash was created using a mathematical formula to establish a snapshot of the records captured. If any single byte of the data is changed, be it a pixel of an image or the timestamp of when it was collected, one can use the hash to verify if a copy is different, which indicates the copy may have been manipulated. A hash mismatch is an indicator of altered data. It functions like a tamper-evident seal. A hash protects an original version of the public record, it is preserved as a part of the record we store, as it can be used down the line if the source of this research, the videos and archives of webpages, is called into question.
Because a hash is nearly impossible to fake, we can use the hashed version of this data to sign with a cryptographic signature, to establish exactly what version of this record was created and when. Using an asymmetric cryptographic key makes it possible to sign these records with a known identity, like a witness can sign a document stating that they have seen and can attest to the authenticity of something. Hashes are also used to create the CIDs (Content Identifiers) that are used in IPFS and as general identifiers that we use to name the files with original videos, websites, and metadata related to the media.
Webrecorder Tools
Web archives created with the Webrecorder suite of tools enabled the team to capture the full context of everything that existed on the web page in a zipped archive called a WACZ file. The information collected includes all content on a webpage, such as articles, comments, likes, and other multimedia. A WACZ file is a copy of the code and media that makes up that webpage, including an index of what content was captured. When users later display (or “replay”) the page using certain tools, it remains fully interactive like it was at the time of capture.
Store
Distributed Storage
In addition to blockchain registration, the WACZ files are given content identifiers, or CIDs, and pinned in the peer-to-peer data sharing system called IPFS using web3.storage. This service also packaged and created archives of the WACZ files on the Filecoin network.
Filecoin involves long-term agreement with Filecoin providers that store this data. By experimenting with immutable ledgers to register digital content, Starling enables experts to audit, or verify, the provenance and integrity of that content. Users can inspect these deals and the identity of the nodes that are archiving this data.
Verify
Blockchain Registration
These records, or the hashes of the content, were also registered on various blockchains to establish exactly what content existed, and when it existed. Manifests, or metadata records for each of the files were created that include the hash of the digital media alongside the transaction IDs of the registrations on public blockchains so users can validate what content existed, and when we established the record of their existence.
For example, this registration on the Numbers blockchain of one of the videos posted of Memphis PD body cams contains a pointer to the metadata file on IPFS labeled as “assetTreeCid”. In the metadata file you will find both a CID for the original content, as well as a Content ID for the entire archive. Content also was stored and registered on Avalanche, a fast, decentralized, open-source blockchain that offers smart contract functionality and LikeCoin, a go-to chain for decentralized publishingAfter creating and registering the manifests they were then stored in a distributed, peer-to-peer data sharing system called IPFS (InterPlanetary File System) from which users can download and view copies of some of the archives (only public facing archives, the websites, are available to inspect).
Validate the videos and web pages you have a copy of are the same ones registered on the blockchains by looking at the metadata record called “assetTreeCid”. You can view this record by using an IPFS gateway with one of the assetTreeCids from a blockchain registration record, such as https://ipfs-pin.numbersprotocol.io/ipfs/<assetTreeCid>.
Example: https://ipfs-pin.numbersprotocol.io/ipfs/bafkreiauefa46ksrrtvws7g6wszvck7jaf5oorwb3wf7dv7dykyljo3kq4
Contents
Learnings
From Cheryl Phillips, Director, Big Local program Stanford University, Department of Communication:
“We learned a lot during this project about the potential of cryptographic tools to assist in preserving immutable records of evidence – especially public records at risk of disappearing or being unavailable through other means. Especially now, in this era of generative AI, these tools are key to combating mis- and disinformation. By using emerging cryptographic methodologies and decentralized systems, these public records can now be protected from disappearance and made available for the public to review, should the source or authenticity of the digital media be called into question. This is very important. A lot went into pulling together the data used in this report: Time-consuming, costly FOIA requests to municipalities that were slow to respond.”
Analyzing and Selecting from a Large Set of Data
“Once we collected police reports and body cam footage, it also took a lot of time to Identify a selection of cases we wanted to consider as candidates for our analysis. We had to Connect incident reports to body cam footage (we wanted to have both the officer’s report and their footage). We had to break down the narratives into individual sentences – essentially statements of fact. We then had to Compare those statements of fact to other reports filed by officers who also responded, keeping an eye on when those reports diverged from each other. We had to review where these gaps occur in the body cam footage.Then we had to isolate the interesting statements we want to fact check, drilling down to a small number of cases. Once we received records, a key technical hurdle included connecting video to incident reports, figuring out a method to filter key points, working with the tech team to understand and streamline the authentication workflow and figure out how to best present the video and statements.
Body Cam Footage Interpretation
“Finally, more challenges arose when viewing police body cam footage – which is hard to interpret – as our survey of experts showed: Once we were able to narrow down footage to three municipalities our panel of 10 experts could not agree on what they saw and heard in that footage.
As reported by The Grio: “They had the most difficulty determining whether or not subjects were armed or even holding anything in their hands. In most cases, they couldn’t agree on whether subjects complied with police commands or if the subjects tried to back away from officers. They often could not agree on what types of force were used by police or even whether the officers tried to de-escalate the situation beforehand. There were some areas where the experts agreed: Whether or not the police issued orders, whether there was a foot pursuit, and whether the subjects approached or attacked police. Interesting findings.
As our data analysis showed, there are a lot of shortcomings of body cam footage as a driver for accountability and reform. The inherent subjectivity of the footage and the importance of perception when trying to derive meaning from body cam videos points to much needed work in the technical area of computer vision to help hold law enforcement agencies accountable.”
Contents
Archive & Resources
As mentioned above, records pertinent to the Tyre Nichols case used in the project which were sourced from the City of Memphis website and Vimeo account, were archived and preserved by the Starling Lab for Data Integrity. By using emerging cryptographic methodologies and decentralized systems, these public records, which have historical importance, can now be protected from disappearance and made available for the public to review, should the source or authenticity of the digital media be called into question.
Users can validate these records against the version of the videos they have to see if that version has been altered, faked, or modified.
(NOTE: If you are viewing body cam footage it is highly recommended you take the time to review the Dart Center’s trauma training.)
- IPFS File Copies
- Original Body Cam Videos – Video 1 | Video 2 | Video 3 | Video 4
- Website Archives – Vimeo Webpages | Memphis PD Website
- Filecoin Archive Information:
- Records of Storage – Video 1 | Video 2 | Video 3 | Video 4 | Vimeo Webpages | Memphis PD Website
- Blockchain Registrations:
- Numbers – Video 1 | Video 2 | Video 3 | Video 4 | Vimeo Webpages | Memphis PD Website
- Avalanche – Video 1 | Video 2 | Video 3 | Video 4 | Vimeo Webpages | Memphis PD Website
- ISCN on Likecoin– Video 1 | Video 2 | Video 3 | Video 4 | Vimeo Webpages | Memphis PD Website
How to Download and Inspect Records
- Click on the links in ‘IPFS File Copies’ to download this from the distributed storage network
- Locate where you downloaded the file. It should be in your Downloads folder.
- For videos, change the name by adding a .mp4 the end of the filename for body cam videos, and for the web archives add a .wacz to the end of the filename
- You can open the .mp4 file on your computer using any video player.
You can open the web archive files by visiting the website https://replayweb.page/ and dragging and dropping the .wacz files into the web interface.
