Preservation and Handling of Media & Digital...

1

Besser-Northwestern Library, 7/14/09

Preservation and Handling of Media & Digital Works

Howard Besser, Director Moving Image Archiving & Preservation Program

NYU’s Tisch School of the Arts http://www.tisch.nyu.edu/preservation


Preservation and Handling of Media & Digital Works

•  The problems posed by Digital Works, and the need for Digital Curation and Stewardship

•  Obsolescence & Reformatting, and how Audiovisual collections have been doing this for a long time

•  NYU’s MIAP Program & our work with Libraries and other collections


The Short Life of Digital Info: Digital Longevity Problems-

•  Digital Preservation •  The Viewing Problem •  The Scrambling Problem

•  Digital Curation •  The Inter-relation Problem •  The Custodial Problem •  The Translation Problem

Digital Preservation



The Viewing Problem

•  Digital Info requires a whole infrastructure to view it

•  Each piece of that infrastructure is changing at an incredibly rapid rate

•  How can we ever hope to deal with all the permutations and combinations


Viewing Problem

•  Requires new file formats and new physical strata at regular intervals

•  Needs a serious Managed Environment •  Main InterPARES finding--the need for

complete lifecycle management –  archivist needs to be incolved when record is

created and throughout active life

2


The Scrambling Problem�

Dangers from:  Compression to ease storage & delivery  Encryption to enhance digital commerce

Digital Curation



The Inter-relation Problem

 -Info is increasingly inter-related to other info

 -How do we make our own Info persist when it points to and integrates with Info owned by others?

 -What is the boundary of a set of information (or even of a digital object)?


The Custodial Problem

 In the past, much of survival was due to redundancy

 How do we decide what to save?  Who should save it?  Mellon-funded E-Journal Archives

 How should they save it?-


The Custodial Problem:�How to save information?

 Methods for later access  Refreshing  Migration  Emulation

 Issues of authenticity and evidence


The Translation Problem

 Content translated into new delivery devices changes meaning –  -A photo vs. a painting –  -If Info is produced originally in digital form in

one encoded format, will it be the same when translated into another format?

– Behaviors

3


The Translation Problem�Thinking of the Future (1/2)

•  Screens will be different resolutions and different aspect ratios

•  CRTs won’t exist •  A decade or 2 from now, today’s user

interfaces will look like arrow-key navigation looks like today


The Translation Problem�Thinking of the Future (2/2)

•  Today’s streaming media are small windows, slow speeds

•  As bandwidth increases, viewers will expect higher quality streams

•  Creators may need to consider how they’ll be able to deliver higher-bandwidth streams –  Delivery Derivatives vs. Masters encoded w/standards –  May also want to re-edit the piece to take advantage of

changes in technology, viewer expectations, society-


Screen Formats


Responding to serious Longevity Problems

 Previous formats required little ongoing intervention (remote storage facilities, Iron Mtn); digital formats require intense ongoing management

 Need for:  Preservation Repositories  Preservation Metadata

Digital Stewardship definitions

•  “acquire, manage, organize, preserve, and provide access to massive amounts of data for use and re-use by a variety of interdisciplinary and heterogeneous communities over time”

•  “Preservation + Use”

Besser-Media Art Conservation, 2/7/09 2

Digital Stewardship:�

Harvard Library OIS definition •  the management of digital objects over the long term through careful digital

asset management practices. Collection managers and DRS staff must work together to manage stored digital objects throughout all phases of the objects' life cycle. This section describes the major areas of digital stewardship in the life cycle of a digital object:

•  Assessment and selection phase: Collection manager performs a curatorial assessment of materials intended for DRS storage.

•  Acquisition and creation phase: Collection manager (in consultation with HUL analysts) selects digital formats and defines technical specifications and workflow processes for creation of objects and related metadata.

•  Deposit phase: DRS ensures a successful deposit by validating each package of digital objects and related metadata that is submitted to DRS.

•  Archive and preservation phase: DRS staff perform periodic checks to ensure the usability of digital objects over time. This includes periodic reports to collection managers about their objects.


4

Digital Stewardship •  Includes much of what cultural institutions have always

done (select, acquire, preserve, provide access, maintain context)

•  But also adds some of the activities that audiovisual collections have always done (maintain knowledge of the original equipment and formats, manage process of reformatting when older hardware is difficult to maintain

•  Also includes important notions from the digital world about ongoing management of re-formatting information, technical metadata, contextual info, repository issues, …


Digital Curation


DCC Digital Curation Life-cycle�http://www.dcc.ac.uk/


Digital Curation

•  Planning & risk management •  Open flexible file formats •  Format registries •  OAIS repositories •  Extensive Metadata •  Individuals & organizations-


Organisation to Engage & Collaborate

Besser-Media Art Conservation, 2/7/09

Industry

research collaborators

standards bodies

testbeds�& tools

communities of practice: users

community support & outreach

research

development co-ordination

service definition & delivery

management & admin support

Associates Network

curation �organisations �eg DPC

a centre of expertise in data curation and preservation

OCLC October 2006 2 Besser-Media Art Conservation, 2/7/09

Digital Preservation Players

•  Collection staff (need to reach agreement on SIP/DIP and acceptable AIP transformations) –  preservation/conservation staff – metadata staff –  access staff

•  Repository staff •  Agreement negotiators

2

5

Besser-Media Art Conservation, 2/7/09

Preservation Repositories:�too difficult for small institutions

•  Too complex for small institutions to manage •  Will be done through partnering (small museum or dance

company with University) or through consortia (museum association, state-wide organization, …) or through service bureaus (OCLC)

•  Archive or museum will direct what is needed, but digital repository will carry out the actual work (as defined in SIP/DIP/AIP agreement)

2 Besser-Media Art Conservation, 2/7/09 2

Storage Media

•  Removable media (like CDs) is not a long-term answer

•  The long-term answer requires ongoing management, and involves regular migration or emulation. This solution is only viable with storage on spinning disks-


Storing on CDs becomes a big problem over time


Consumers replace their CDs with a hard disk (& so should you)


Plain DVDs are no longer the latest format


Obsolescence

6


Difficult Materials become obsolete relatively quickly

•  The physical carriers decay or become obsolete

•  The technology required to view the carriers changes frequently

•  The encoding formats needed to decode the content shift


Obsolete or deteriorated Physical Carriers


Obsolete Carriers & Info Techn


Obsolete Carriers


Obsolete Carrier viewing Technology?


Kodak stops making some films

7


List of old Audio Formats Format Description Years in UseWax Cylinder Records 2- or 4-minute formats, wax

or wax compound1888– 1929

Recordable Disc Records(Direct or Acetate Discs)

7”, 12”, or 16”, recorded at33 or 78 revolutions perminute (rpm). Generallyvinyl on a paper, glass ormetal base

1929– 1960s

Recording Wire Spooled wire, usually in 15-to 30- minute lengths, onedirection only

c. 1945– 1955

Open reel recording tape 1/4”– 2”, 3”– 10 1/2" reels,1 7/8– 30 inches per second(IPS) speeds

c. 1945– Present

Compact Cassette 1/8” tape in hard case, 1 7/8IPS format

1965– Present

Microcassette/Minicassette Very small 2-4 cm cassettetapes

1977– Present

Digital disk, MP3, and otherdigital recorders

Audio recorded directly indigital files to optical disksor internal hard drives

2000– Present


Old Video Formats (www.vidipax.com)


Keep Old Video Equipment--NARA


Keep Old Video Equipment--NARA


Keep Old Audio Equipment-NARA


Reformatting and Ethics

8


Re-formatting is not a new idea


What is Reformatting?

•  A form of copying •  Usually copied onto a medium having

different physical characteristics than the original physical strata

•  Examples – Document on acidic paper onto non-acidic

paper – Newspaper microfilming


History of Conservation & Preservation Reformatting

•  In ancient times, in the library of Pamphilus at Caesaria, badly damaged papyrus manuscript pages were replaced with parchment (which was stronger)

-Saint Jerome •  The Bible was hand-copied for millenia •  1964 - US Newberry Library (Paul Banks) began 1st US

institutional preservation program •  1987 - US NEH begins funding massive microfilming of

brittle paper (mainly newspapers)


Why do we Reformat?-


Brittle Newspapers (Australia Battye Library)


Film Decay (LC Dayton)

9


Why do we Reformat? •  Because we cannot sustain the original object (its physical

characteristics are deteriorating too fast) •  Because continued access and handling of the original

object will rapidly decay its physical characteristics (so we create a surrogate for users and store the original in very good conditions, away from users)

•  Because viewing the work requires some kind of technology, and we can’t keep that technology working very far into the future-


Edison


Metal sound recording Disks�Casa Rui Barbosa


Paper print (LC Dayton)


Record Turntables


Slide Projector

10


Limitations of Reformatting

•  Authenticity issues (more later) •  User behaviors (newspaper, book, video

game, …) •  Users mistaking the reformatted work for

the original


Critiques of Reformatting

Mainly User Behaviors

•  Can’t view outside the library

•  Only sequential access •  Viewing and studying

is awkward •  …


But unless we Reformat, we totally lose some kinds of works�

(particularly audiovisual works like film) •  50% of all titles produced before 1950 have vanished (approximate

number as of late 1970s) •  This reflects full-length features; survival rates are much lower for

other types (studio newsreels, shorts, docs, independent, …), and these “orphans” are particularly in peril

•  Fewer than 20% of features from 1920s survive in complete form; survival rates of 1910s is <10% (& none of these are negatives)

-Film Preservation 1993: A Study of the Current State of American Film Preservation, Vol 1: Report, June 1993, Report of the Librarian of

Congress (http://www.loc.gov/film/study.html)


And sometimes we have to reformat because of technology changes

•  We don’t have video players to play tapes made 25 years ago

•  We don’t have 8-inch floppy disk drives, syquest drives, zip drives

•  We don’t have Windows 3 operating systems

•  But this is something that conservators have always dealt with…


Authenticity Issues with reformatting-

•  Is the work what it purports to be? •  Commercial reformatting examples •  Media archivists and interpretation •  Beyond Film & Video


Commercial Reformatting Issues:�PR at 1967 re-issue of GWTW

•  “Spherical Blow-Up” •  “In the Splendor of 70mm. Wide screen and full

stereophonic sound!” •  “For the thousands who remember its unparalleled drama,

action and romance! For the new thousands to whom the wonders will be revealed for the first time! Breathtaking spectacle, inspired acting by the greatest cast ever assembled! The screen's most exciting love story! The most-talked about picture ever made!”

•  http://www.imdb.com/title/tt0031381/taglines

11


But most people didn’t know that 70mm widescreen is different shape than 35mm

normal


Change in Aspect Ratio forces cutting


Pan and Scan example

•  7 Brides for 7 Brothers, Stanley Donen, MGM, 1954 from http://en.wikipedia.org/wiki/Pan_and_scan


This meant eliminating part of the frame


Eliminating even in famous scenes


But the re-release wasn’t governed by artistic concerns

•  Intellectual Property is owned by MGM, not by Fleming or Selznick

•  MGM is motivated maximizing profit, not in maintaining artistic integrity

•  Bigger is always better

•  Not radically different than another blow to artistic integrity/originality in Atlanta 20 years later…

12


Atlanta was also home of 1980s Colorization movement


Sometimes even the Director wants to go back and change their original film


Star Wars 1977 vs 2004


Someone needs to maintain the integrity of artistic works

•  This means preserving original versions –  even when commercial interests want to replace

the older version with something new and fancy

–  even when the “artist” wants to use more recent technological developments to “improve” their work


Media Archivists

•  Preserve original works •  Provide access to older versions •  Maintain the integrity of the “original” in any

restoration process •  Champion works that do not have commercial

entities pushing for their preservation and distribution

•  Try to make sure that works are viewed within their original context-


Viewing Context (images like in Dayton-Hudson)

13


Be concerned about © •  For preservation you may need to re-format, but with

recent changes in copyright laws, you may not have the right to re-format

•  Intellectual property rights are very difficult, particularly considering that most films and videos have extensive underlying rights that you could never get prior permission for (stock footage, historical footage, music composition, music performance, …) [“Eyes on the Prize”]

•  And even if you have the right to re-format for preservation, you might not have the right to show what you have preserved


Possible endless need for reformatting implies

•  Possible loss with each generation •  Requires managed environment •  Can lead to © violations


Managed Environment

•  More than temperature & humidity control •  Periodic monitoring of the works •  Periodic monitoring of the technical

environment for viewing the works (software, systems, hardware)

•  Trusted repositories


Media Repositories-


Univ of GA�Walter J Brown Media Archives



14








Hogan Jazz Archive�Tulane Univ Library


Cinemateca Brasileira (video storage)


Hampton Collection (interviews)

15


Various Formats Intermixed (Hampton)


MIAP Projects (Paper Tiger)


MIAP Projects (Paper Tiger)


Orphan Works


Home Movies--Emanuel Goldberg


Amateur Film-- UGA�Walter J Brown Media Archive

•  Athens GA, 1947 •  Shows contrast btwn White and African

-American neighborhoods •  http://www.libs.uga.edu/media/collections/homemovies/housing.html

16


Wolfson Florida Moving Image Archive

•  http://youtube.com/watch?v=vhWa307owQ0


Palmour Street (1957)

•  Gainesville, GA •  http://www.youtube.com/watch?v=VpgWZceHX30 •  Educational film from Southern Educational

Film Production Service •  Prelinger Archives


Physical Deterioration


Hampton Collection (atmosphere cntrl)


Academy-Atmosphere


IPI Preservation Index�temperature/humidity, Years until noticeable deterioration

% RH

Temperature Cº

2º 7º 13º 18º 24º 29º 35º 20 1250 600 250 125 60 30 16

30 900 400 200 90 45 25 12

40 700 300 150 150

70 35 18 10

50 500 250 100 50 25 14 7

60 350 175 80 40 20 11 6

70 250 125 60 30 16 9 5

80 200 100 50 25 13 7 4

17


Temperature & Humidity for Tape Storage

•  Variance of less than 2ºC and 5% RH per 24 hours

•  Ideally 8ºC and 25% RH •  Other options

–  20°C (68°F) and 20-30% RH –  15°C (59°F) and 20-40% RH –  10°C (50°F) and 20-50% RH

•  Never store below 8ºC

amianet.org 2003, & ISO 18923

Oral/Visual Histories-



New distribution:�

iTunes-U UCB Oral Histories


iTunes-U UCB Campus Events


iTunes-U Duster Video


Do we trust iTunes to preserve these?

18


The Re-Mix generation


Dueling Videos: Scholar Creates Remix of Another Academic's YouTube Hit�

http://chronicle.com/wiredcampus/article/2707/dueling-videos-scholar-creates-spoof-of-another-academics-youtube-hit


Chronicle report on Center for Social Media Re-Mix project

•  http://chronicle.com/wiredcampus/article/2638/video-report-a-legal-defense-of-videos-riffing-on-popular-culture


Some Re-Mix Videos •  Baby Got Book

http://www.godtube.com/view_video.php?viewkey=97759aa27a0c99bff671 •  Bush vs Zombies http://www.youtube.com/watch?v=IoXgRtDysLY •  George Bush Don’t Like Black People

http://revver.com/video/71633/george-bush-dont-like-black-people/ •  Victory in Iraq http://revver.com/video/79501/victory-in-iraq/

•  Fox News Edits a Democrat to Make Him Look Worse http://www.youtube.com/watch?v=oGqPxn7njqM

•  xx


Born-digital images

•  Where is the “original”?


Born-digital works are both easier and harder to preserve than analog works

•  +With a born-digital work, we don’t have to worry so much about the “original artifact” (there really isn’t one)

•  -We know that digital works face a whole range of obsolescence problems, so they must be reformatted at least once per decade

19


What can you do now?�For both Film & Video

•  Label elements as well as you can •  Try to keep things at a low humidity and

temperature •  Limit the number of formats as much as

possible •  Save important production elements


Risk Management •  We can’t say definitively that we can make every digital work persist •  What we CAN say is that the more a digital work conforms to

standards and best practices, the greater the likelihood that we can assure persistance

•  Our preservation repositories can even accept deposits of non-conforming works, but the less they conform, the less likely that they’ll be salvageable

•  Persistance is most likely for works that share standards, metadata, and best practices


Reformatting Best Practices (still images) •  Think about users (and potential users),

uses, and type of material/collection •  Scan at the highest quality that does not

exceed the likely potential users/uses/material

•  Do not let today’s delivery limitations influence your scanning file sizes; understand the difference between digital masters and derivative files used for delivery

•  Many documents which appear to be bitonal actually are better represented with greyscale scans

•  Include color bar and ruler in the scan

•  Use objective measurements to determine scanner settings (do NOT attempt to make the image good on your particular monitor or use image processing to color correct)

•  Don’t use lossy compression •  Store in a common (standardized)

file format •  Capture as much metadata as is

reasonably possible (including metadata about the scanning process itself)


So, with electronic works, the focus should be less on stable temperature�

(Helsinki underground vaults)


And less on the construction of Vaults (Helsinki underground vaults)


Paradigms Shifts needed Old New

Physical preservation

atmospheric cntrl ongoing mgmt

What to save? artifact idea + ancillary material & documentation

Cataloging Individual work in hand

FRBR

Later access Artifact & documentation

Restaging, ancillary material & documentation

20

Preservation and Handling of�Media & Digital Works�

Howard Besser�NYU Moving Image Archiving & Preservation Program�

http://www.tisch.nyu.edu/preservation

•  http://besser.tsoa.nyu.edu/howard/Talks/ •  http://www.ptvdigitalarchive.org/

•  http://www.dcc.ac.uk/

•  http://www.digitalpreservation.gov/ •  http://www.amianet.org/ •  http://www.clir.org/pubs/reports/pub54/ •  http://sunsite.berkeley.edu/Longevity/ •  http://www.imagepermanenceinstitute.org/ •  http://www.screensound.gov.au/screensound/screenso.nsf/ Besser-Northwestern Library, 7/14/09

Preservation and Handling of Media & Digital...

Documents

Transcript of Preservation and Handling of Media & Digital...