The digital treadmill

Everything has changed. So much of what I knew or what I thought I knew about moving image archiving and the preservation of film and video has changed since 1990 when I got my first job in a film archive.

This is all related to a complete shift in how technology has entered our lives and transformed it through computers, email, the web, MP3 players, smart phones, etcetera. With those changes and the advent of YouTube and the countless other ways media is being delivered to us there is a shared belief that everything is available on the Internet. Of course, not only is this not true, the fact is that we’re not even close to getting it all online. There is so much analog material remaining to be preserved and digitized in my field that it is now in direct competition with all born-digital content that is now on the “preservation treadmill.”

What is the digital preservation treadmill? The trouble begins and ends with technology. The faster technology changes, the more quickly we must develop preservation strategies that anticipate and adapt to these changes. And everything is sped up even further because the preservation of digital data is on the fast track.

We need playback devices for all of our content. A Babylonian clay tablet from 600 B.C. can still be seen, read, and studied today, but a file created and saved in an obsolete digital format maybe just a few short years ago can be lost forever.

For example, you may have heard about how the rescue and retrieval of a twenty-two-year-old game source code originally saved to a floppy disc is news in the world of gaming (see Prince of Persia Source Code–Posted!) Think about it: a file that is only twenty-two years old being considered lost.

In the world of what I’ve known as archives, twenty-two years is nothing. We just received home movie footage from a hunting plantation in South Georgia that was shot on 28mm film in 1918, and we’ll be able to have it transferred. In other words, it is easier to view and preserve film that is ninety-four years old than it is to view and ensure the stability of digital information that is twenty-two years old.

Longtime preservation’s hard work

I realize this may seem like an apples-versus-oranges comparison, but it’s really not. The reason it’s not is because we’re going to migrate that film to a digital format that will also climb onto the digital treadmill. That original film will live in storage, a digital master file will be created from the original and stored over time, and a second digital proxy will be used for viewing. I will go back to that digital master file in three to five years where I will determine its viability by checking for digital data bit rot and ensuring that it is still accessible and uncorrupted. So that home movie on 28mm film from 1918 that has been sitting in a house for almost one hundred years will now need to be revisited as a digital file every three to five years to make sure we can still access it. And of course, as we convert more film and video-based content to digital formats, we increase the number of files that require this level of analysis to ensure their long-term preservation.

Film is over one hundred years old, and we’ve been able to preserve it in its original format. Videotape has been around since 1956 and we’ve been able to preserve it as well, by migrating its content to other video formats. Now we migrate everything to digital files. Seems kind of astounding that the more advanced we become technologically speaking the more fragile the record of our lives becomes. That is the reality we live in right now.

What about after the cloud?

What does this have to do with you? I’m a professional after all, I get paid to worry about these things. Digital preservation has everything to do with you because I’m willing to bet that much of what you create is digital. Images, videos, emails, texts, etcetera. The things that document our lives are now something that have to be monitored and stored „in the cloud.“ What about after the cloud? What will be the next storage solution for all those images and digital videos that are being created by so many of us all the time? I don’t know. Do you currently pay for storage of your digital creations? Is that any different from paying for storage of physical items? I don’t think so.

The other issue is metadata. How do we describe all this data so that we can find it again? How do we name the files we are creating so that we have unique names for everything we create? How do we locate information in a file that isn’t revealed in its file name? How do we make sense of all this? Right now we are in the very infancy of exploring all these issues, and are splitting our energy between caring for materials created with legacy machinery and formats, converting and storing that information digitally, and worrying about the next place to migrate the digital content that has been created. And we attempt to do all of this while keeping up with newer technologies.

I realize this column is posing more questions than it answers, but that is the nature of where we are now with digital technology. The access to information of all kinds is great, and the immediacy and convenience is unprecedented. Where is all the content stored? As of last year it was estimated that Google had 900,000 servers. It is probably up to a million servers by now. Hard to imagine, but that is the tip of the ice berg to where we are going.

Each archive with moving image and audio collections is facing its own challenge with these issues. If you are interested in reading more, the archive I head up was featured as a case study in „The Digital Dilemma 2″, a publication that includes a cross-section of heavy-hitters in moving image preservation efforts in the United States; it was produced by the Science and Technology Council of the Academy of Motion Picture Arts and Sciences who partnered with the National Digital Information Infrastructure & Preservation Program, a collaborative initiative of the Library of Congress. Go to this link for a free download of „The Digital Dilemma 2„. You might also be interested in a short column from „The Economist“ that discusses future problems with accessing digital content: „Digital data Bit rot The world is losing its ability to reconstruct history.„