On a Technicality: Novel Video Hashing + Blockchain to Combat Deepfakes
Will we ever be able to trust video again? Pitted against the growth of malicious AI altering video evidence, we at Amber Video are trialing a unique document fingerprinting approach.
Fake video and deepfakes is fundamentally changing the way we perceive and interact with video. As a result, proving authenticity of video is ever more important. While some digital “fingerprinting” solutions do exist to authenticate files, these solutions aren’t currently being used for video files, as they aren’t well adapted to the way we use and handle video files. Can we find another approach to proving authenticity of video, and perhaps more importantly, in protecting the due process of law?
The rate at which we hear of new fake videos being disseminated on the internet should concern us all. Artificial intelligence-based video forgeries, or “deepfakes” are quickly becoming threats to justice and democratic society. The problem is not just limited to the growing capability of deepfakes — the use of much simpler video forgery techniques appears to be ever-more on the rise. Doctored videos are currently used in making decisions that are directly affecting society and democracy.
Video evidence is a potent force in due process as “seeing is believing.” The rise of fake video is placing a fast-growing need on protecting the integrity of video, especially those of an evidentiary nature used in multi-stakeholder situations. A video recording taken from a body-worn camera worn by a police officer may need to be analysed by an internal investigation team. And then it may also need to be reviewed by the legal team of a defendant and prosecution in an ongoing criminal case, by the court system, civic groups, the media, and the general public.
What technologies can be leveraged to solve this type of scenario?
Protecting the integrity of data by so called “fingerprinting” of files is a well-known technique. This technique works by applying algorithms on data files to generate “hashes”, or very short strings of data. If the author of a file generates a fingerprint, and the recipient of a transferred file uses the same hashing algorithm on the file they received, the fingerprints will match. If, on the other hand, a minor change occurred in the file during its distribution, the fingerprints will be entirely different. This type of fingerprinting can check for data corruption in data transfer or storage, and it can flag-up possible malicious third-party alterations of files.
The growth of blockchain platforms is renewing the interest in data file fingerprinting. With the public ledger provided by blockchain technology, fingerprints can be generated as soon as a file is created, and those fingerprints can be stored in a time-stamped blockchain record. The fingerprints stored in this way provide an immutable record of a financial transaction, legal agreement, or any type of data-rich document. By storing the fingerprints on a publicly accessible blockchain, the fingerprints are available for all to witness and to perform any independent verification they feel is necessary, without requiring either party to place trust in any provider, only requiring trust in the mathematics underpinning blockchain technologies.
The question then becomes: can a video be fingerprinted at the moment it’s created, and can that fingerprint be used as a proof that it hasn’t been altered? To answer this, we need to consider the fact that video footage used for evidentiary purposes is often recorded on systems such as body-worn cameras, video surveillance equipment, and drones. These systems are designed to record footage for hours at a time. When content from these platforms are viewed, shared and commented upon, it is often in the form of clips lasting 30 seconds to a few minutes, taken out of these hours-long recordings. These shorter instances of evidence are what is usually shared with multiple stakeholders, and in some cases the general public.
This presents any would-be blockchain-based fingerprinting solution with a challenge. How can we fingerprint a video recording file in a way that is compatible with how video is used in practice? How can you prove that a clip from a longer video is an accurate and unaltered portrayal of the original recording?
This is a problem we’ve been working on, here at Amber.
Video files are typically presented in well-defined container formats, such as .mp4, .mov or .mkv files. Each of which can support a range of different video formats, encoded with different “codecs” (such as the ever-popular H.264, VP9 or HEVC). These files work by holding blocks of data, arranged in “tracks”. If you were to examine the bytes in the file, you would find a few frames-worth of video data (track 1), followed by a few frames-worth of audio (track 2), perhaps some subtitles (track 3) and so on, as the cycle repeats.
Amber fingerprints video in a novel and different way. Instead of fingerprinting the video file as a whole, we fingerprint bits of data on each of these “tracks”. This would mean generating not one, but many short strings of data for a given file. Each of these strings of data would correspond to a single frame of video or audio within the original footage. The longer the video, the more strings of data generated.
Any new video file that contains a portion of an original recording could then have the same algorithm applied to it. Any matching frames, common in both the edited video’s and the original video would be found. So long as the frame hadn’t otherwise been tampered with.
We faced a problem however. A typical one-hour long video file would generate in excess of 100’000 hashes. This would be a lot of data to store on a public ledger, once applied at scale.
We then had the idea of arranging the hashes in tree-like-structures, by generating fingerprints of the fingerprints. This provided us with a fingerprint for a pre-determined window of time. Taking this approach, we would be able to obtain a fingerprint representing an entire second, or a fingerprint representing a minute worth of video, as required. By arranging these hashes in trees, only the higher layers of these trees would need to be stored on the blockchain, with the fingerprint-per-frame stored on commodity cloud storage. This not only reduced the cost of storing fingerprints on the blockchain, it also reduced the amount of hashing and comparison required at playback. Video footage fingerprints could now be compared on, say, a per-minute basis. At least until a mismatch was found, after which comparisons could be done at lower levels, iteratively.
While this approach greatly simplified the work, there was one challenge remaining. And that was the fact that any video editing operation would change the timings of these hashing windows. If you’re performing one hash per minute, and you cut the video half-way through a minute, and perform the hashing on the edited video, the hashes will mismatch, as the minute durations won’t be the same any longer.
To address this, we developed a way of structuring the hashes so that instead of each layer in the tree of hashes having a fixed boundary (for example one minute), the boundaries would be chosen in a deterministic way, based on the content of the video. This would mean the hashing windows on each layer of the tree would not always be the same size, but they would always correspond to the same content.
This approach solved the problem for us, and allowed us to build an app, available on the Apple App Store. With our app, you can record video, hash it, store the hashes on the blockchain and upload it to our video sharing platform. If you make your video public, someone who wishes to edit your video can do so, adding new content or cutting content out, and re-upload it to our platform. When anyone watches that new video, a colored border around the video will let them know which parts of it match your original recording, and which ones don’t.
But this is just the beginning for us at Amber Video. In an environment where just the existence of fakes will cast doubt on legitimate video evidence and it may even get tossed and the case dismissed, Amber tech can preserve due process.
