The PAN footage industry events and reviews editor, freelance content researcher/producer, Steve Bergson spent last week digesting a new book from Illinois Press called The Future of Memory: A History of Lossless Format Standards in the Moving Image Archive – a great read for all you footage archive techies! … here’s what he found:
The Future of Memory: A History of Lossless Format Standards in the Moving Image Archive (The History of Media and Communication)
by Jimi Jones and Marek Jancovi
July 2025 – hard copy and Kindle
By Steve Bergson
This book tackles an important subject often overlooked as the world has rushed headlong possibly over the digital cliff, relieved to be released from the copying restrictions and inevitable physical degradation of all those flawed analogue formats. As every audiovisual archivist knows all too well however, for every potential Eldorado there’s a heavy price to negotiate. This well titled work tries to assess a very familiar dilemma.
Any printed output would struggle to match the pace of change in such a volatile area but the authors of this book also have to grapple with defining its readership because it is such a technocratic subject. They duck the geekfest temptation where they can, though inevitably there is a lot of discussion about file formats and “authorised” developments. There can after all, in reality, be no all-encompassing standards in a world so dependent not just on international agreement but also rigorous technical standards compliance. As the book points out: “Format standards revolve not just around the technicalities of video itself but also serve as proxies for notions like freedom and openness.” Both are underlined bolded trigger words now!
If we’re talking shifts in power taking place within the media landscape, it’s still all too easy to ignore the fundamental axiom that the raw data of our visual history is key. Standards are always going to be very much about the vicissitudes of power between broadcasters and archives, the film industry, computer makers and creative workers, as well as the archival institutions that depend on but also try to influence them.
There is an alphabet stew of acronyms in culinary play here naturally. The reader for example, really has to know the background of organisations like CELLAR – the working group in the IETF (Internet Engineering Task Force) – to be able to navigate the deliberations over formats. Yet this is an important subject for anyone concerned with archival preservation. It is vital to understand the role of the 2010 AMIA/IASA conference as an inflection point between the MXF and JPEG 2000 formats on one side and FFV1 and MKV on the other. It was there that the Austrian Mediathek stood out to promote the maverick, open source cause of the Matroska MKV format. Something of an outsider in this arena, MKV is not so much a file format as an open, non-proprietary media container which can hold an unlimited number of audio, video, image, and subtitle tracks in a single file. It’s not very helpful – and possibly a bit creepy – that this is named after Russian nesting dolls called Matryoshka because it is definitely a very useful technical spec. Rather than a series of interlocking dolls, it’s more like a space module acting as a flexible “vehicle” for different media streams, instead of a video or audio codec itself. Matroska files are characterised by some quite advanced features, such as support for chapters, multiple subtitle and audio tracks, full metadata (tags), fast seeking and error resilience. In archival terms, these are important elements; it’s no mean lunar module.
As Kara Van Malden writes in her piece for the Institute of Museum and Library, a US Universities project, Oral History in the Digital Age:
Preserving digital video requires addressing the entire life cycle of the content, from pre-production, to capture, edit, archiving, and providing access. Decisions made at the point of creation have implications for the other stages down the road. It is important to understand these implications and make decisions through out the workflow that will enable efficient, cost-effective, and accessible archiving over the long-term.
The implications of this are twofold: interopability across computer platforms that are forever changing – and secondly, the metadata that should be attached to all time based content like video to enable access in databases and therefore findability for online search engines – and of course, increasingly, AI bots.
There is no way to dodge getting caught up in the weeds of technical detail with this subject. The wide range of metadata needed for a file to be rendered (e.g. image width) needs to be preserved for reliable future playback or conversion to future formats. Not only has there been the kind of disagreement on standards set out here, but also a rather untidy proliferation of wrapper formats like MKV, to go along with different delivery and encoding methods. In the audio world, the highest quality file has seemed to settle on WAV formats – an application of the Resource Interchange File Format (RIFF) bitstream format method for storing data in chunks, similar to the 8SVX in the Amiga world and the Audio Interchange File Format (AIFF) format Apple uses. In stills photography, the choice of image file type is clearer, with the TIFF format for archiving master files. Even here, the RAW or DNG format is available, but for lossless compression, all of this is now open to challenge by the open source JPEG 2000 format. There is a divide in video between the industry standards of JPEG 2000 and MXF in opposition to the Matryoshka and FFV1 which facilitate online distribution of often copyrighted content such as Blu-ray formatted movies. As ever, the practical aspects of what is technically possible here are overshadowed by the legal battles of our old friend copyright and the whole world of pain for legitimate users that this often entails in the name of piracy prevention.
Conclusion
The authors here, Jimi Jones, a lecturer of library and information sciences for the School of Information Sciences at the University of Illinois Urbana-Champaign and Marek Jancovic, an assistant professor of media studies at Vrije Universiteit Amsterdam, have vigorously – and without doubt, courageously – grappled with this powerful beast which can at times, threaten to overwhelm the whole business, but it is nevertheless still very hard to see the social wood for the technical trees. At the very least this book has charted some significant battles in this theatre of conflict. We are possibly – indeed probably – not yet in a position to see above the treetops to afford a view of how this will all pan out. You hope that Gen-AI won’t be required or worse, dragooned to fill in the gaps not just in the actuality coverage of historical events, but also the shortcomings of the digital preservation process set out here. The loss of collective memory would be too high price to pay if this were to prove how things are resolved. For this reason alone, it is worth keeping at least one weather eye on the current state of archival preservation methods.