Page 1 Content from the Library of Congress DPOE Baseline Modules version 2.0, Nov 2011 Jody L....

Content from the Library of CongressDPOE Baseline Modulesversion 2.0, Nov 2011

Jody L. DeRidderUniversity of Alabama Libraries [email protected]

July 16, 2012

An Introduction to Digital Preservation

DPOE Modules

Identify - what digital content do you have?

Select - what portion of that content will be preserved?

Store - what issues are there for long term storage?

Protect - what steps are needed to protect your digital content?

Manage - what provisions are needed for long-term management?

Provide - what considerations are there for long-term access?

DPOE Baseline Modules: Intro, version 2.0, Nov 2011

identify

select

storeprotectmanage

provide

Managing Content Over Time

DPOE Baseline Modules: Intro, version 2.0, Nov 2011

Why do we identify content?

SCOPE!

DPOE Baseline Modules: Identify, version 2.0, Nov 2011

Image from: http://en.wikipedia.org/wiki/Telescopic_sight

Preservation requires an explicit commitment of resources Effective planning is based on knowing the extent of what will be preserved Identifying content is a first step to planning for current and future preservation needs An explicit inventory is the best way to identify content

http://en.wikipedia.org/wiki/Telescopic_sight

Content Categories

Inventories should include all relevant material: • Institutional records • Special collections• Scholarly content – licensed and open• Research data• Web content• Digitized collections

DPOE Baseline Modules: Identify, version 2.0, Nov 2011

Example entry:• Category: Special Collections• Title/Description: Railroad Photographs, SE U.S.• Type: images, digitized• Format: TIFF• Extent: 242 GB; 2,250 images• Location: archival server in Room A, Central IT• Coverage Dates: early 1900’s• Creation date: January-June 2006• Inventoried: 12/15/2011, by Fred Jones

Selecting Content for Preservation: Why do it?

• Storage may be cheap, but management is not … especially over time

• Sustaining the quality of content

takes effort

• Continually changing discovery and dissemination services will be needed as hardware and software change

… think scale, scope, performance, sustainability

DPOE Baseline Modules: Select, version 2.0, Nov 2011

Selection Criteria: matching mission to content…

• Acquisition or collection development policy• Departmental criteria (priorities, precedents)• Research criteria (interests, significance)• Uniqueness (only source) • Value (historical, evidential, can’t reproduce)


Practical Considerations

Stop if or when the answer is ‘no’…1. Content

– does the content have value? – does it fit your scope?

2. Technical– is it feasible for you to preserve the content?

3. Access– is it possible to make the content available?


Selection starts at the beginning…

Contact content creators

(as needed)– Arrange a convenient time for them– Prepare brief statement of outcomes– Identify list of materials to review with them– Send a reminder before the meeting– Document the results and send them a copy


Prevent later headaches!

STORAGE involves…

• What you store– File Formats– Metadata

• How you store it– Number of copies – Storage media– Repository selection

What are storage needs?

Archival Storage manages content as

objects

Digital content (files + metadata = object):• May include any types

– e.g., images, text, sound, video, maps

• Requires some identification

and description– Captured as metadata

DPOE Baseline Modules: Store, version 2.0, Nov 2011

Selecting File Formats for Text

“…the agency must clearly define the purpose and the requirements for preservation…

The appropriate answer will depend on:• the mission of the agency • the kind of information to be preserved• the uses to which the objects may be put in the future• the expectations of current and future users, and • how far into the future the objects are intended to

remain useful.

CENDI Digital Preservation Task Group. “Formats for Digital Preservation: A Review of Alternatives and Issues”, Revised Mar. 1, 2007. p.22.http://www.cendi.gov/publications/CENDI_PresFormats_WhitePaper_03092007.pdf

For text: TIFF XML PDF / A

Selecting File Formats for Images

Sustainability factors:• Disclosure• Adoption• Transparency• Self-documentation• External dependencies• Impact of patents• Technical protection mechanisms

Bill LeFurgy, October 12, 2011. “Digital Preservation-Friendly File Formats for Scanned Images” http://blogs.loc.gov/digitalpreservation/2011/10/digital-preservation-file-formats-for-scanned-images/

For images: TIFF JPEG 2000 PDF / A

http://blogs.loc.gov/digitalpreservation/2011/10/digital-preservation-file-formats-for-scanned-images/

http://blogs.loc.gov/digitalpreservation/2011/10/digital-preservation-file-formats-for-scanned-images/

Which Formats Are Best?

Sustainability of Digital FormatsPlanning for Library of Congress Collections http://www.digitalpreservation.gov/formats/

http://www.digitalpreservation.gov/formats/

Importance of Metadata• How do you know what an object is?

− Metadata uniquely identifies digital objects

• How do you use content in the future?– Metadata makes digital objects understandable

• How do you know an object is authentic?– Metadata allows objects to be traced over time

Metadata enables long-term preservation

DPOE Baseline Modules: Identify, version 2.0, Nov 2011DPOE Baseline Modules: Store, version 2.0, Nov 2011

Preservation MetadataContent (what), Fixity (unchanged), Provenance (life story),

Reference (this thing), Context (relationships)

Administrative(manage)

Structural(understand, use)

Descriptive(find, use)

Object-level Metadata

Diagram courtesy DPM Workshops


Object Metadata Characteristics

Content: preserve the substanceFixity: demonstrate content is unchanged Reference: identify as this content and no other Provenance: trace to its origin (or to deposit)Context: preserve linkages with other objects Original source: Preserving Digital Information Report, 1996


Number of Copies

How many copies are enough for you?

Minimum: two (2) copies in two locationsOptimum: six (6) copies

Examples of storage factors:• Video files are too large to store 6 copies• Possible legal restrictions (e.g., storage locations)• Types of media used for storing the content


Storage Media Options• Content (objects) are kept on storage media • Options include: online, near-line, offline• Factors for choosing options include

– Cost (available resources for preservation)– Quantity (size and number of files)– Expertise (skills required to manage)– Partners (achieving geographic distribution) – Services (outsourcing)


• Multiple, geographically distributed copies

• Storage Partners or Hosted Services

Storage Considerations

Services and collaborations can make it easier for organizations to manage content over timeDPOE Baseline Modules: Identify, version 2.0, Nov 2011DPOE Baseline Modules: Store, version 2.0,

Nov 2011

Repository Selection• Range of types to consider:

– general (any content) to special (format-specific)

– open source to proprietary

– unified to distributed

– easy to advanced installation and

management

• Each option has pros and cons

• No system is fully compliant to standards

Select best option for your content – for now


What are we protecting content from?

• Change and loss – accidental and intentional• Obsolescence – as technology evolves• Inappropriate access – e.g., confidential data• Non-compliance – standards and requirements• Disasters – emergencies of all kinds

DPOE Baseline Modules: Identify, version 2.0, Nov 2011DPOE Baseline Modules: Protect, version 2.0, Nov 2011

Obsolescence Prevention Terminology

• Refresh: moving content to newer media

• Migrate: moving content to newer formats that can be accessed with current hardware and software

• Normalize: migrate to archival formats that meet your specifications

• Emulate: attempting to provide the original look and feel of the content with newer software

ReadinessProper planning should allow you to:• Prevent – undesirable outcomes

• Predict – most likely risks and threats• Detect – errors, problems, damage• Respond – with appropriate measures• Repair – damage or possible loss


Everyday Protection

• Know where your content is located– Onsite and offsite; online and offline

• Know who can have access to it– DP staff, IT staff, others?

• Manage authentication information – For staff, depositors, users

• Track and review usage then adjust practices– Web use, internal use and activities, maintenance


Emergency Protection

• Engage in ongoing disaster planning – Establish committee and share information– Develop and maintain documents

• Identify possible outcomes and prepare– e.g., server goes down, media is damaged

DPOE Baseline Modules: Protect, version 2.0, Nov 2011

Disaster Planning Resources

DPOE Baseline Modules: Protect, version 2.0, Nov 2011

Why do we emphasize management?

Preserving DigitalInformation (PDI),

1996

DPOE Baseline Modules: Manage, version 2.0, Nov 2011

• Rapid technological obsolescence• Media fragility• Legal and organizational environment in flux• Complex practical issues• Lack of clarity as to procedures and responsibilities• Multiplicity of types of content in growing number of formats• Massive amounts of content

Balanced Management

An effective approach will address:• Organizational requirements and objectives• Technological opportunities and change• Resources – funding, staff, equipment, etc.


Kenney and McGovern, 2003. “The Five Organizational Stages of Digital Preservation” http://www.dpworkshop.org/

• Preservation Planning (ongoing)• Self-assessment (internal process)• Audit (external review by peers)• Business Continuity • Disaster Planning


Organizational Requirements: Planning

Organizational Objectives: DP Standards

Standards emerging since 1996 report :• Trusted Digital Repositories, 2002• Open Archival Information Systems (OAIS)

Reference Model, 2003 and 2009 revision• Preservation Metadata Implementation Strategies,

2005 plus updates• Trustworthy Repositories Audit and Certification

(TRAC), 2011 Common practices are emerging and evolving


Trusted Digital Repository

A TDR should have these characteristics:• community standards (OAIS Compliance )• commitment (Administrative Responsibility) • management (Organizational Viability)• resources (Financial Sustainability)• infrastructure (Technological … Suitability)• protection and control (System Security)• documentation (Procedural Accountability)


Community Expectations: Ten Principles

Available on the CRL website


1) Demonstrates organizational fitness (including financial, staffing, and processes) to fulfill its commitment.

2) Acquires and maintains requisite contractual and legal rights and fulfills responsibilities.

3) Has an effective and efficient policy framework.

4) Acquires and ingests digital objects based upon stated criteria that correspond to its commitments and capabilities.



5) Maintains/ensures the integrity, authenticity and usability of digital objects it holds over time. 6) Creates and maintains requisite metadata about:

actions taken on digital objects during preservation and relevant contexts before preservation:

• production • access • usage




7) The repository commits to continuing maintenance of digital objects for identified community/communities.

8) Fulfills requisite dissemination requirements.9) Has a strategic program for preservation planning and

action.10) Has technical infrastructure adequate to continuing

maintenance and security of its digital objects.


Technological Opportunities: Investing in Technology

• Prioritize: weigh requirements to be met• Assess: define criteria to select appropriate• Sequence: identify steps to meet goals• Fund: decide when to own/join/share• Anticipate: look ahead, be prepared• Evaluate: measure outcomes and success


Technological Opportunities: Adopting Technologies

Characteristics of sound software:• written in a well-documented language• usable on a wide variety of platforms• sustained support by creators/developers• modular in design• supports batch processing and workflows• licenses support secondary use


Resources: Designated Funding

• Funds set aside for digital preservation• Measurable indication of intent to preserve• Challenging to do, but important• Over time, contributes to track record• May not be explicit (e.g., budget line item)

… but must be able to make a compelling case


Resources: Sustainable Access

Effective and sustainable DP programs address:• Value – understand and stress content value• Roles – identify stakeholders and involve them• Incentives – identify “carrots” for preserving

Identify and address costs across life cycle

See: Blue Ribbon Task Force Report on Sustainable Preservation and Access Report

DPOE Baseline Modules: Provide, version 2.0, Nov 2011

What is Long-term Access?

Preservation • relies upon proven technologies

to preserve digital objects across generations of technology

• accumulates metadata over the life cycle to trace preserve content


Access• relies on cutting edge

technologies to provide best and fastest access at a point in time

• selects metadata needed to use and understand content

Preservation makes long-term access possible…

Preservation • preservation systems create new

versions of digital objects for access to deliver as needs change over time

• purpose: ensure long-term access

• focus: future users

Access• access systems deliver objects

with user-oriented services to make the objects

• purpose: provide content to users

• focus: current users

Preservation makes long-term access possible…


What is Long-term Access?

Understand Users

• Who are your users? Track and respond to them.

User expectations will change over time, and must be monitored.

• Preservation provides pathway from one generation of technology to the next

Digital content will need to be packaged in new ways for delivery over time.


Access Policies: Issues

• Who is allowed to have access to content?• Are access policies equal for all content?• If not, how are categories managed?• How are exceptions/special requests handled?• How do users request/get access?• What options (if any) do users have?

Consider using FAQs as a step to develop policies


• Legal issues include copyright, but copyright is only a portion of legal issues in DP

• Legal questions emerge throughout lifecycle … and most of us are not lawyers

• Access raises legal issues, but manage from submission (or before) throughout lifecycle

• DP requires well-formed, valid documentation

− agreements, contracts, licenses, policies, etc.

• Good legal advice should enable well-formed evidential documentation and transparency

Managing Life Cycle Legal Issues


http://www.lib.ua.edu/wiki/digcoll/index.php/Digital_Services_Permission_Agreement

The Donor grants […] and its agents the right to: • Digitize all submitted content, and create derivative

representations for web access • Reproduce and distribute reprints or derivative representations

for noncommercial scholarly purposes • Augment or create metadata to enhance accessibility and

management of content • Electronically view, present and display the full digital content

to others, including providing open access via the web • Electronically store, archive, copy and/or convert the digitized

content for preservation and access purposes

Create and use Permissions Agreements

DPOE Baseline Principles (1-2)

1. Define the digital content within your scope of responsibility [Identify]

2. Specify the digital content you need/want to preserve [Select]

DPOE Baseline Modules: Wrap Up, version 2.0, Nov 2011


3. Establish requirements for storing files in preservation formats [Store]4. Determine (and review) your best option for storing your content [Store]5. Ensure that your content is secure during day-to-day activities [Protect]6. Work to ensure that your content is prepared for an emergency [Protect]


7. Develop (and review) plans for managing content over time [Manage]

8. Use policies to contain and develop your preservation program [Manage]

9. Remember that long-term access is the purpose of preservation [Provide]

10. Make sure the means to deliver content to users remains current [Provide]



©iStockphoto.com/CGinspiration

Resources•“Digital Preservation Management: Implementing Short-Term Strategies for Long-Term Problems” Online Tutorial: http://www.dpworkshop.org/dpm-eng/eng_index.html Survey of Institutional Readiness: http://www.dpworkshop.org/

• "Planning for Digital Preservation: 20 Questions for Providers of Digital Storage Services," Bernard Reilly, Center for Research Libraries http://www.nedcc.org/resources/digital/downloads/QuestionstoAskProvidersofDigitalStoragefinal.pdf

• "Digital Preservation Metadata Standards," Angela Dappert and Marcus Enders, Information Standards Quarterly, Spring 2010, Volume 22, Issue 2 http://www.loc.gov/standards/premis/FE_Dappert_Enders_MetadataStds_isqv22no2.pdf

http://www.dpworkshop.org/dpm-eng/eng_index.html

http://www.dpworkshop.org/sites/default/files/readiness.pdf

http://www.nedcc.org/resources/digital/downloads/QuestionstoAskProvidersofDigitalStoragefinal.pdf

http://www.nedcc.org/resources/digital/downloads/QuestionstoAskProvidersofDigitalStoragefinal.pdf

http://www.loc.gov/standards/premis/FE_Dappert_Enders_MetadataStds_isqv22no2.pdf

http://www.loc.gov/standards/premis/FE_Dappert_Enders_MetadataStds_isqv22no2.pdf

More Resources• ICPSR Digital Curation:

http://www.icpsr.umich.edu/icpsrweb/ICPSR/curation/

• Trustworthy Repositories Audit & Certification (TRAC): Criteria and Checklist (2007):

http://www.crl.edu/sites/default/files/attachments/pages/trac_0.pdf

[NOTE: ISO 16363 version of TRAC approved fall 2011]

• Center for Research Libraries Reports on Digital Archives and Repositories: http://www.crl.edu/archiving-preservation/digital-archives/digital-archive-reports

• “Digital Preservation Outreach and Education,” Library of Congress. http://www.digitalpreservation.gov/education/




http://www.crl.edu/sites/default/files/attachments/pages/trac_0.pdf

http://www.crl.edu/archiving-preservation/digital-archives/digital-archive-reports



http://www.digitalpreservation.gov/education/

http://www.digitalpreservation.gov/education/

Page 1 Content from the Library of Congress DPOE Baseline Modules version 2.0, Nov 2011 Jody L....

Documents

Transcript of Page 1 Content from the Library of Congress DPOE Baseline Modules version 2.0, Nov 2011 Jody L....