Creating the First Cryptographic Archive for a War Crimes Investigation
The Advanced Tech Breaking Open A War Crimes Investigation
A first-of-its-kind cryptographic archive published by Rolling Stone helped reopen a 30-year-old cold case. Released at the dawn of consumer-available generative AI, it illuminates new paths to overcome a range of modern challenges including denialism, deepfakes, and link rot.
Reading Time: 10min
Contents
Context
How can you trust that a news photograph is real?
We looked into the future of investigative journalism — by looking into the past.
In 1992 as civilians were killed in the streets, Ron Haviv captured the defining image of the Balkan War. But for some, seeing was not believing.
It had been three decades without justice or accountability for the soldier in this frame. He appears callous, flicking a cigarette as his boot – suspended in time – hangs over innocent civilians who were later found dead.
(This case study crops out the more graphic portions of the images.)
The photo has long been subject to denialism. It went “viral” – or what would be the equivalent in 1992 – after it first ran in Time Magazine. The image was soon used in a TV news report to confront the general who oversaw the soldier’s paramilitary unit. He denied that it shows what it clearly did.
More recently, when Russia first invaded Crimea in 2014, the image reemerged online paired with false claims that it depicted a Ukrainian soldier committing war crimes against civilians. This deception through false captions represents a more common twist on the deepfake – the “cheapfake.”
“People have said it’s fake. People have said I set it up. People have said these people aren’t dying. Captions have been changed. So there must be a way to authenticate what photographers are seeing.” – Ron Haviv
In recent years, society has been introduced to groundbreaking new technologies. Some, like generative AI, made it even harder to trust our own eyes.
But other technologies can help us restore that trust.
In The DJ and the War Crimes, Rolling Stone published a first-of-its-kind cryptographic archive. It authenticated Haviv’s photos, along with hundreds of other records from the conflict. Classic investigative and documentary journalism was paired with an immersive microsite.
We invited audiences to become the investigator.
Contents
Scope
Starling Lab’s collaboration with Rolling Stone exemplifies our work. Along with the world-class investigation, we teamed up to publish a unique front-end journalism archive that lets readers explore evidence in a way never available to audiences before. Meanwhile, the equally innovative back-end ensured that this evidence would be resilient against efforts to undermine its credibility or availability.
This particular story began with the iconic photo taken by Haviv in Bejilina, Bosnia. Ultimately the journalistic piece is a classic who dunnit. The reporting team, led by Sophia Jones, set out to answer the lingering question: “Who is the soldier in this iconic photo depicting an alleged war crime?”
The result included all the classic elements of investigative journalism. Over 11 months, reporters collected a staggering amount of evidence from firsthand witnesses, archives, and on-the-ground reporting.
The scale of the investigation was matched by the team involved. Jones led fellow reporters Nidžara Ahmetašević and Milivoje Pantović, as well as other local freelancers and photographers who opted to remain anonymous for their safety. At least two dozen individuals played a role, including editorial staff at Rolling Stone, contributors and engineers from Starling Lab, and designers from Gladeye.
Haviv always hoped that his photojournalism would lead to accountability. Not only did the investigation focus on his famous image, it included several others from his rolls of film that day which had never been published before.
The group wanted to ensure that ephemeral media – whether aging film slides or social media posts – could be preserved and made available to others along with all the evidence in the investigation. Our accounting identified nearly 2,000 documents (PDFs), 40 key images, and 183 web archives. The latter are more than just a screenshot, but a more robust way of capturing information that’s on the internet – and very much at risk of deletion or link rot.
Contents
Framework
We think about three important stages of a piece of digital media’s life cycle – Capture, Store, Verify – and address each through the Starling Framework.
- The Capture phase is based around a cryptographic root of trust. How do we embed verifiable metadata into documents?
- Next, how do we Store this material? Digitization is not the same as preservation.
- Finally, audiences must be able to Verify what they see.
We asked: How might we use advanced cryptographic systems to verify and secure assets and their metadata (like the date and time)?In the end, we applied over a dozen technologies to capture, store and verify these digital assets: C2PA content credentials, a custom capture app paired with a Canon 1Dx Mark III, the ProofMode authenticated capture app, WebRecorder website archiving, PGP encryption, Authsign, “ZK” proofs, and preservation on blockchains (OTS/Bitcoin, Avalanche, ISCN/Likecoin, IPFS, Filecoin, Storj).
Technology
Capture – Film Slides
The physical slide seen below was developed from the actual film in the camera used by Haviv in Bijeljina. That’s his hand holding the paper frame. In order to bring these images into the 21st Century, we had to digitize several of them. Haviv is depicted inserting one into physical bellows, which hold it in place for a modern digital camera to capture in high fidelity.
The camera – a Canon 1Dx Mark III – was equipped with special firmware provided by the manufacturer so that we could tether it to a mobile phone.
The phone – an HTC Exodus 1 – came from the manufacturer with a cryptographically secure chip. While popularized as a “wallet” by cryptocurrency fans, it has far more utility for our use case by authenticating the data from digitized photos.
A custom app on the phone, developed by Numbers, allowed us to take the digital equivalent of a fingerprint, add our own digital signature, and then record all of it in a virtual fingerprint registry.
In technical terms, we used a secure enclave, hashed and signed each file, and registered them on blockchain. This allowed us to establish a cryptographic “root of trust.”
Along the way we wanted to add additional authenticity markers. Fortunately, we had access to the one person who could make an attestation about which version was his own original film slide – not the ones that had been manipulated or lied about on the internet in recent decades.
Haviv’s personal testimony on video is more relatable to audiences than the 1s and 0s under the hood. Even our most modern tech-forward approach can’t exclude humans from the process. To the contrary, we sought ways to include them, their corroborations, and acknowledgements of their role.
Capture – Payroll PDFs on UN Servers
Another crucial piece of evidence was a collection of payroll records from the paramilitary unit at the center of the alleged atrocity. These first surfaced in a war crimes tribunal, and copies were eventually stored on United Nations servers. Unfortunately, custodians told our reporters that these were “confidential” and declined to release them. Fortunately, those same reporters kept searching and found them sitting in the clear (i.e., unencrypted) on the UN’s public servers. This led to a pair of challenges that technology was able to overcome.
First, the team had to assume that after publication the record keepers might respond by taking the documents offline. Beyond mere linkrot, the history of denialism around this unit raised legitimate concerns that someone might dismiss our version of the records as “fake.” With AI making it even easier to generate assets, our team was concerned about the general public being conditioned to doubt anything is real – a phenomena psychologists label the “liar’s dividend.”
To ensure proper preservation we used Webrecorder, a free and open source tool that works as an extension on any Chrome-based browser. No mere screenshot or “print” feature, it carefully captures all data and metadata during a browsing session. This means you can relive the browsing experience as it was when journalists visited the site – even offline years later. As part of the replay experience, links on a saved webpage can be clicked, and the linked pages can be read (assuming the initial investigator made sure to capture each page). Webrecorder backs this with cryptographically secure proof that the files were from specific servers, which we could use to demonstrate the materials indeed came from the UN’s tribunal archives.
Verify – Redacting Names From Payroll Records
Second, as a matter of good journalistic practice, it was important to redact names from the payroll records if someone wasn’t a subject of the investigation. If a person was a cook or truck driver and had no involvement with the atrocities, it wouldn’t be appropriate to put them through the same level of public scrutiny.
In order to redact the records, we turned to Trisha Datta, working on her PhD at Stanford and advised by Professor Dan Boneh, one of Starling’s principle investigators. She implemented a zero knowledge proof, which uses cryptography to guarantee that the only changes made to published PDFs were the additions of block boxes over specific pixels. This means that someone with a sophisticated understanding (like an expert witness in court) can literally check our math. This confirms nothing else was manipulated from the moment data left UN servers to the time the redacted records are viewed on your screen.
Capture – Social Media (OSINT)
Webrecorder was also a powerful tool to capture social media, like this Instagram post. The one person we didn’t blur out happens to be one of the individuals named on the payroll records. Jones did a masterful job tracking down and identifying the connections that different people from the unit still had in modern day.
Of course, social media can be ephemeral for many reasons. We’ve seen how mercurial CEOs can cause content to disappear. Users may set their accounts to private or delete individual posts. When the story went live, it was a real risk that accounts might go dark and make it challenging for prosecutors to reconstruct who was connected to whom.
This subject in the photo with wolves is one of several individuals who appeared to be leading a life of impunity. He was spotted in photos on various sites associating with the Nightwolves, a notorious Russian motorcycle gang which supported Russia’s initial invasion of Crimea.
Verify – User Experience For The Archive
The story’s archive lets audiences explore the evidence for themselves. All the documents are spread out so that anyone can build their own associations between them – social media posts, news clippings, payroll records, photographs scanned from 30 years ago, and more recent images captured in the current day. Any of these assets might be vulnerable, but because they are all stored and secured using decentralized technologies (more on that later) it empowers us to ensure they survive and are not prone to future denialism.
Each one of those assets – whether seen in the narrative’s main story, featurettes, or full archive – has an “eye” icon in the lower-left corner that when clicked opens up an authentication certificate. It’s a similar idea to the Content Credentials being popularized by major industry leaders through the Coalition for Content Provenance and Authenticity (C2PA). This coalition includes Adobe, Google, Meta, Canon, Sony, BBC, and many more hardware manufacturers, software developers, and media companies.
C2PA is a technical standard that produces a “manifest” of authenticity metadata for a digital asset. We were also able to incorporate it into each item for this project.
In the authentication certificate you can see how we establish different chains of trust, inspect the asset, and examine additional details. Readers again see the Starling Framework – Capture, Store, Verify – as applied to an individual asset.
In the capture phase, we indicate where the original version was registered on a variety of blockchains – OpenTimestamps, Numbers Avalanche and ISCN. Each is linked so that you can confirm the underlying transaction with an on-chain explorer. (For the non-technical audience, this is simply where we put the fingerprints of each file into a registry – and importantly, the registry is cryptographically secure so that no one can edit it later.)
In the storage phase, copies were preserved using decentralized storage systems like IPFS, Filecoin, and Storj. These are designed to offer a resilient, tamper-evident, and censor-resistant alternative to common corporate cloud-based systems like Google Drive, Microsoft Azure, Amazon Web Services, or Dropbox.
Under the thumbnail of each asset are a pair of buttons for the “full manifest” (which includes code for tech-savvy verifiers) and “inspect.” The latter allows audiences to examine changes to the evidence through a verification tool on the Content Credentials site. It provides a visual and user-friendly way of exploring the C2PA metadata – and lets anyone audit the edit history of an asset.
Any photo published in the news is likely to be edited somehow. We’re all used to cameras on our phones processing an image, often thought of as a filter. And in a journalistic project, it’s normal to do cropping and color correcting. We can think of these as permissible edits – but in journalism they should be transparent.
Adobe Photoshop was used to make all the photo adjustments in this story. Anyone who uses that software can go into preferences to turn on Content Credentials before starting an edit, and that generates metadata about the changes certified by Adobe itself.
The Verify tool allows you to explore those verified edits, either side-by-side or overlaid with a slider. Looking closer at the first edit to this image using the side-by-side view, there’s a noticeable black border around the top version – an artifact from the digitization process. The paper frame of film slides, which you’ll recall Haviv held to insert them in the bellows, doesn’t allow light to penetrate. The existence of the border is a clear indicator to the newsroom that the complete slide was digitized – but it’s also something a page designer wants to chop to the virtual cutting room floor.
The second edit to the photo, seen with a slider, confirms that color correction was relatively minor. Notice the sky’s lighter exposure to the left of the slider compared to the right.
Verified with your own eyes, you can be assured the changes were helpful – not a deceptive wholesale change like adding a bayonet.
Store – Where Does It All Go?
As seen earlier in the authentication certificates, all these assets were stored using decentralized networks like IPFS, Filecoin, and Storj. These offer promising alternatives to entrusting assets with one of the few megacorporations that dominate the commercial storage market.
Traditional digital storage uses “location addressing,” meaning a file can be located only by knowing the specific path through a directory and subdirectories (think of folders nested inside of other folders on your computer). It’s the same concept as a website address structured as www.website.com/directory/subdirectory/subsubdirectory/file.pdf. If “subdirectory” is renamed “subdirectory2,” everything breaks. In contrast, these innovative decentralized systems use “content addressing.” This approach starts by taking a digital fingerprint from the file, resulting in a code that can be about 64 characters long. When you later look for the file by searching for its unique code, the system checks for a matching copy anywhere on the internet. Ultimately you don’t care where it finds that copy, only that it’s a 100% perfect match as proven by cryptography.
The implications are profound. If a file is destroyed at a data center due to a natural disaster or an authoritarian government raid, you can confirm a full and authentic recovery when anyone else – even an unknown stranger – hands you a copy matching the original file’s fingerprint.
Instead of requiring you to place your trust in a centralized (and potentially corruptible) authority, this is considered a “trustless” system.
This fundamental design shift is a potential long-term solution to linkrot and related problems that have plagued newsrooms, archives, and even Supreme Court opinions.
Capture – Live From Novi Sad
The authentication certificate for this final photo shows a colorful scene. It also demonstrates additional ways to authenticate journalistic work in the field.
The person at the center of this modern day image is the same person in the middle of Haviv’s iconic image of a war crime in progress. In between, he went on to a notable career as a DJ, playing in European festivals over the years with crowd sizes comparable to Coachella.
This particular image came from an event where he was spinning in Novi Sad, Serbia. It depicts his life of impunity, as there had been no accountability for the war crimes decades before. This was one of several images and videos captured by a freelancer we hired to cover the event. However, we didn’t have a go-to person in Serbia. With the show scheduled days after we caught word of it, how could we quickly develop trust in a total stranger? This became more sensitive when the most promising local freelancer asked to remain anonymous, not wanting his name to be on the radar of his neighborhood war criminal.
Technology once again provided a solution. We decided to use Proofmode, developed by Guardian Project, which is a free and open source app that can be installed on any iOS or Android phone. It uses software signatures to authenticate each image as it’s taken, including a full C2PA manifest. This solution worked especially well for the freelancer, as it cost nothing to set up and allowed them to operate discretely with a phone – looking like any other partygoer. But when he returned to his car in the parking lot, he was able to upload images with manifests allowing us to confirm we were seeing the same moments as they were witnessed by the lens of his phone.
Contents
Learnings
Data Can Disappear
Not long before his set in Serbia, the subject of this story had started a new Instagram account and was leaving more of a digital trail than ever. Hours after the story published, he set it to private.
Data Can Disappear
Fortunately, we have most of that account saved thanks to Webrecorder and thanks to our content addressed decentralized archives. All of that evidence is available for investigators even though it’s no longer available to the public on Instagram.
Those subjects and their relationships – on social media and in real life – are the heart of any investigation. Here we see the primary subject on the top left still enjoying his cigarettes. On the far right is a younger version of the man seen posing with wolves. Another one of the people in this network carried the casket of the unit’s general at his funeral. The are nodes in a social network – and poignantly now captured on nodes in our decentralized archive.
Where There Is Tech, There Is Hope
Days after this story was released, local prosecutors reopened this 30-year-old cold case. Hopefully we will eventually see justice for the victims.
Over the next few months, the story earned a number of journalism industry accolades. Awards juries recognized the fine investigative work, and they also recognized the need to authenticate and archive. Starling’s hope is that these approaches illuminate new paths for all newsrooms to pursue and inspire new implementations that will benefit society and restore trust in journalism.
Contents
Archive
Links to News Articles
- Rolling Stone story
- 9-min mini-doc
- Direct link to the archive
- Featurettes: The Photograph• The Document• The Network
- Gladeye case study
Credits
A large team within Rolling Stone was led by Executive Editor Sean Woods, Creative Director Joseph Hutchinson, and Digital Director Lisa Tozzi.
A world-renowned design firm, Gladeye, handled the custom site development, overseen by Tarver Graham and implemented by Nathan Walker.
Starling’s work was spearheaded by Jonathan Dotan, Adam Rose, Benedict Lau, Yurko Jaremko, and Josh Lee.